168 lines
7.3 KiB
Markdown
168 lines
7.3 KiB
Markdown
# Conformance Claim Levels Decision
|
|
|
|
Status: Superseded
|
|
Cycle: Conformance agenda closure pass
|
|
|
|
Superseded by:
|
|
|
|
- `docs/general/specs/13. Conformance Test Specification.md` (Quality-Gate Baseline)
|
|
- `docs/general/specs/17. Compatibility and Evolution Policy.md` (Quality-Gate Baseline)
|
|
|
|
Superseded on: 2026-03-04
|
|
|
|
## 1. Context
|
|
|
|
PBS v1 needed a conformance-claim model that stays honest while the conformance corpus matures across frontend, runtime, and artifact-facing work.
|
|
|
|
The open questions from the conformance agenda closure pass and Workshop 1 were:
|
|
|
|
- whether PBS should use one conformance label or staged claims,
|
|
- what a partial implementation is allowed to claim,
|
|
- how published conformance claims must describe missing coverage,
|
|
- and how claim levels constrain later compatibility policy.
|
|
|
|
The adopted direction needed to avoid two failure modes:
|
|
|
|
- pretending an unfinished backend is already full PBS v1 conformance,
|
|
- and allowing vague marketing language that hides excluded domains such as artifact emission or verifier behavior.
|
|
|
|
## 2. Decision
|
|
|
|
PBS v1 adopts staged conformance claims.
|
|
|
|
The canonical claim levels are:
|
|
|
|
1. `frontend conformance`
|
|
2. `source-runtime conformance`
|
|
3. `full toolchain conformance`
|
|
|
|
These claims mean:
|
|
|
|
1. `frontend conformance` covers deterministic source acceptance and rejection behavior through parsing, manifest/import resolution, linking/static semantics, and the stable diagnostics surface required for those rejection tests.
|
|
2. `source-runtime conformance` covers all frontend obligations plus source-level runtime, memory/identity, and host-boundary behavior already defined as language-observable in the active specs.
|
|
3. `full toolchain conformance` covers all source-runtime obligations plus the artifact-facing, loader-facing, verifier-facing, and compatibility obligations required by the full active PBS v1 spec set.
|
|
|
|
Only `full toolchain conformance` may be published without qualifier as `PBS v1 conforming`.
|
|
|
|
Any narrower claim must remain explicitly qualified in published wording, for example:
|
|
|
|
- `PBS v1 frontend conforming`
|
|
- `PBS v1 source-runtime conforming`
|
|
|
|
An implementation that does not satisfy one complete canonical level must not claim PBS conformance merely because it implements a subset such as parsing-only or parser-plus-binder behavior.
|
|
|
|
Any partial claim must explicitly state excluded domains when they matter to user expectations, including as applicable:
|
|
|
|
- artifact emission,
|
|
- loader interaction,
|
|
- verifier behavior,
|
|
- supported runtime line,
|
|
- selected stdlib environment,
|
|
- host-registry line,
|
|
- and capability assumptions.
|
|
|
|
The unqualified claim `PBS v1 conforming` is reserved until the artifact-facing conformance surface is fully closed in `13`, `12`, `15`, and `19`.
|
|
|
|
## 3. Claim Meanings
|
|
|
|
### 3.1 Frontend Conformance
|
|
|
|
`frontend conformance` is the minimum canonical PBS conformance claim.
|
|
|
|
It requires:
|
|
|
|
- deterministic parsing behavior,
|
|
- deterministic source rejection for in-scope syntax, linking, and static-semantics failures,
|
|
- deterministic manifest/import resolution outcomes,
|
|
- regression protection for already-published behavior inside that claim scope,
|
|
- and the stable diagnostics fields required by the diagnostics contract for those failures.
|
|
|
|
It does not imply:
|
|
|
|
- runtime execution support,
|
|
- host-backed call execution,
|
|
- artifact emission,
|
|
- loader integration,
|
|
- or verifier integration.
|
|
|
|
### 3.2 Source-Runtime Conformance
|
|
|
|
`source-runtime conformance` extends `frontend conformance`.
|
|
|
|
It additionally requires:
|
|
|
|
- source-level runtime oracle coverage,
|
|
- source-level memory and identity oracle coverage,
|
|
- host-boundary behavior already fixed by the Host ABI and capability specs at source-visible level,
|
|
- and regression protection for already-published behavior inside that claim scope.
|
|
|
|
It does not by itself imply:
|
|
|
|
- one normative PBX image shape,
|
|
- one artifact-level golden contract,
|
|
- loader patching behavior beyond source-visible requirements,
|
|
- or verifier-surface conformance.
|
|
|
|
### 3.3 Full Toolchain Conformance
|
|
|
|
`full toolchain conformance` is the top-level PBS v1 conformance claim.
|
|
|
|
It requires:
|
|
|
|
- all `source-runtime conformance` obligations,
|
|
- all artifact-level oracle requirements once closed,
|
|
- loader/verifier-facing acceptance and rejection obligations that belong to PBS-facing conformance,
|
|
- required fixture modeling for stdlib, registry, capability, and runtime lines,
|
|
- and maintained regression/compatibility evidence for every published supported combination inside the claim.
|
|
|
|
Until those artifact-facing obligations are normatively closed, implementations may publish only the narrower qualified claims above.
|
|
|
|
## 4. Naming and Publication Rules
|
|
|
|
- If the word `conforming` is used in a PBS v1 product claim, it must use one of the canonical claim levels.
|
|
- The bare wording `PBS v1 conforming` is reserved for the full toolchain claim only.
|
|
- `frontend` and `source-runtime` claims must keep the qualifier adjacent to the published claim and must not be shortened to imply full PBS v1 conformance.
|
|
- If an implementation is still below `frontend conformance`, it should describe itself as partial, experimental, or implementation-specific rather than conforming.
|
|
- Published conformance claims must state the tested scope when environment selection changes observable behavior.
|
|
|
|
## 5. Invariants
|
|
|
|
- PBS conformance is staged, not all-or-nothing, during the v1 maturation path.
|
|
- The meaning of the unqualified top-level claim must remain stronger than any partial claim.
|
|
- Partial claims must not hide excluded domains behind vague wording.
|
|
- Conformance claims and compatibility claims are related but not identical; compatibility policy may refine promise strength per claim level without redefining the claim levels themselves.
|
|
- Published claims must be backed by executable evidence in the conformance or regression corpus appropriate to that claim level.
|
|
|
|
## 6. Explicit Non-Decisions
|
|
|
|
This decision record does not yet close:
|
|
|
|
- the full fixture model for stdlib, host-registry, capability, and runtime-line scenarios,
|
|
- the final artifact-level oracle for PBX or lowering invariants,
|
|
- the compatibility-matrix dimensions and retirement policy,
|
|
- or the exact repository layout of the conformance corpus.
|
|
|
|
## 7. Spec Impact
|
|
|
|
This decision should feed at least:
|
|
|
|
- `docs/general/specs/13. Conformance Test Specification.md`
|
|
- `docs/general/specs/17. Compatibility and Evolution Policy.md`
|
|
|
|
It should also constrain later work in:
|
|
|
|
- `docs/pbs/specs/13. Lowering IRBackend Specification.md`
|
|
- `docs/general/specs/15. Bytecode and PBX Mapping Specification.md`
|
|
- `docs/general/specs/19. Verification and Safety Checks Specification.md`
|
|
- `docs/pbs/decisions/Artifact-Level Conformance and Fixture Model Decision.md`
|
|
- `docs/pbs/decisions/Compatibility Matrix and Published Claims Decision.md`
|
|
- `docs/pbs/agendas/17.1. Compatibility Workshop 1 - Compatibility Domains and Promise Levels.md`
|
|
|
|
## 8. Validation Notes
|
|
|
|
The intended operational reading is:
|
|
|
|
- a parser-plus-binder implementation may claim `frontend conformance` only if it actually satisfies the complete frontend obligations, including deterministic rejection behavior and the stable diagnostics surface in scope,
|
|
- an interpreter or direct-execution implementation may claim `source-runtime conformance` without yet claiming artifact or verifier conformance,
|
|
- and no implementation should imply full PBS v1 conformance until the artifact-facing contract is normatively testable.
|