134 lines
7.1 KiB
Markdown
134 lines
7.1 KiB
Markdown
# Artifact-Level Conformance and Fixture Model Decision
|
|
|
|
Status: Superseded
|
|
Cycle: Conformance agenda closure pass
|
|
|
|
Superseded by:
|
|
|
|
- `docs/general/specs/13. Conformance Test Specification.md` (Quality-Gate Baseline)
|
|
- `docs/general/specs/19. Verification and Safety Checks Specification.md` (Quality-Gate Baseline)
|
|
|
|
Superseded on: 2026-03-04
|
|
|
|
## 1. Context
|
|
|
|
PBS v1 needed a decision for the point where conformance stops being source-only and begins asserting artifact-facing behavior.
|
|
|
|
The open questions from the conformance agenda closure pass and Workshop 3 were:
|
|
|
|
- whether artifact-level conformance is mandatory for every claim or only for full toolchain claims,
|
|
- whether conformance should snapshot whole binaries or only stable invariants,
|
|
- what the smallest honest fixture model is for stdlib and host-backed scenarios,
|
|
- and how `13`, `12`, `15`, and `19` divide responsibility.
|
|
|
|
The adopted direction needed to keep full conformance meaningful without freezing one emitter layout, one repository structure, or one implementation-specific binary image.
|
|
|
|
## 2. Decision
|
|
|
|
PBS v1 adopts the following artifact-level conformance baseline:
|
|
|
|
1. Artifact-level conformance belongs only to `full toolchain conformance`.
|
|
2. `frontend conformance` and `source-runtime conformance` do not require artifact-level oracle checks.
|
|
3. Full toolchain conformance must assert stable artifact invariants rather than one opaque byte-for-byte binary snapshot.
|
|
4. The conformance fixture model must explicitly declare the resolved environment assumptions that affect observable admission or lowering behavior.
|
|
5. `12` defines what semantic and artifact-boundary facts lowering must preserve.
|
|
6. `15` defines how the conformance-facing artifact facts appear in PBX/bytecode-visible form.
|
|
7. `19` defines which later safety and validation obligations are checked by loader and verifier rather than by lowering itself.
|
|
|
|
## 3. Artifact-Level Oracle Baseline
|
|
|
|
The v1 artifact-level oracle is invariant-based.
|
|
|
|
Full toolchain conformance must be able to assert, at minimum:
|
|
|
|
1. required canonical host-binding declarations are emitted for admitted host-backed uses;
|
|
2. `SYSC` entries are deduplicated by canonical identity;
|
|
3. `SYSC` entry order follows first occurrence in the lowered program;
|
|
4. host-backed callsites are emitted in pre-load form as `HOSTCALL <sysc_index>` referencing the emitted canonical binding set;
|
|
5. VM-owned builtin projections, builtin constants, and intrinsic callsites do not leak into host-binding metadata such as `SYSC`;
|
|
6. stdlib interface modules remain compile-time-only and do not emit executable bytecode by themselves;
|
|
7. emitted artifacts preserve at least one source-attribution hook for backend failures that remain source-attributable and user-actionable under the diagnostics contract.
|
|
|
|
This baseline does not require:
|
|
|
|
- one full PBX binary snapshot,
|
|
- one exact section byte layout outside the stable invariants above,
|
|
- one exact source-map format,
|
|
- or one verifier-internal trace shape.
|
|
|
|
## 4. Fixture Model
|
|
|
|
Conformance fixtures must model the resolved environment honestly whenever the selected environment affects acceptance, lowering, or artifact meaning.
|
|
|
|
Every fixture that depends on environment selection must declare, as applicable:
|
|
|
|
- selected stdlib environment or reserved stdlib line,
|
|
- selected host registry line,
|
|
- granted capability set,
|
|
- and selected runtime line.
|
|
|
|
If an axis is irrelevant for a fixture, the fixture should declare that explicitly rather than leaving it implicit.
|
|
|
|
The minimum reusable fixture families for v1 are:
|
|
|
|
1. source-only fixtures whose expectations do not depend on host registry or runtime selection;
|
|
2. stdlib-resolved fixtures that name the selected stdlib environment when reserved modules affect acceptance or behavior;
|
|
3. host-admitted fixtures that name both host registry line and granted capabilities for positive host-backed cases;
|
|
4. host-rejected fixtures that name the missing or unauthorized capability or host-registry condition for deterministic negative cases;
|
|
5. full-toolchain artifact fixtures that name the runtime line whenever emitted intrinsic or host-binding form depends on the selected executable target line.
|
|
|
|
This model is intentionally small. It is sufficient to prevent fake portability claims while avoiding one mandatory repository layout.
|
|
|
|
## 5. Boundary Across `13`, `12`, `15`, and `19`
|
|
|
|
The adopted boundary is:
|
|
|
|
- `13. Conformance Test Specification.md` defines which invariant families are pass/fail conformance obligations and which fixture assumptions must be declared;
|
|
- `13. Lowering IRBackend Specification.md` defines the semantic and artifact-boundary facts that lowering must preserve before loader or verifier work begins;
|
|
- `15. Bytecode and PBX Mapping Specification.md` defines the PBX/bytecode-visible mapping of those facts into inspectable artifact surfaces;
|
|
- `19. Verification and Safety Checks Specification.md` defines the later validation and rejection obligations applied after lowering and, where relevant, after loader patching.
|
|
|
|
Conformance therefore does not require `12` to define one binary encoding and does not require `15` to redefine semantic ownership already fixed in `12`.
|
|
|
|
## 6. Invariants
|
|
|
|
- Artifact-level conformance is required only for the full toolchain claim.
|
|
- The v1 artifact oracle is invariant-based, not whole-image-based.
|
|
- Fixture assumptions that change observable behavior must be explicit.
|
|
- Host-binding metadata remains distinct from VM-owned builtin and intrinsic lowering.
|
|
- The boundary among lowering, encoding, and verification must remain reviewable and non-overlapping.
|
|
- Source-attributable backend failures must retain at least one recoverable source location, but richer debug surfaces remain optional.
|
|
|
|
## 7. Explicit Non-Decisions
|
|
|
|
This decision record does not yet close:
|
|
|
|
- which PBX sections beyond the current stable invariant baseline become part of the PBS-facing contract,
|
|
- whether intrinsic declarations require an additional dedicated PBX preload section in v1,
|
|
- the full verifier test surface for conformance,
|
|
- the compatibility-matrix policy for aging and retirement of published combinations,
|
|
- or one repository structure for the conformance corpus.
|
|
|
|
## 8. Spec Impact
|
|
|
|
This decision should feed at least:
|
|
|
|
- `docs/general/specs/13. Conformance Test Specification.md`
|
|
- `docs/pbs/specs/13. Lowering IRBackend Specification.md`
|
|
- `docs/general/specs/15. Bytecode and PBX Mapping Specification.md`
|
|
|
|
It should also constrain later work in:
|
|
|
|
- `docs/general/specs/19. Verification and Safety Checks Specification.md`
|
|
- `docs/general/specs/17. Compatibility and Evolution Policy.md`
|
|
- `docs/pbs/decisions/Compatibility Matrix and Published Claims Decision.md`
|
|
- `docs/pbs/agendas/13.4. IR and Lowering Workshop 4 - Builtins, Host Bindings, and Artifact Invariants.md`
|
|
|
|
## 9. Validation Notes
|
|
|
|
The intended operational reading is:
|
|
|
|
- a direct interpreter may still claim `source-runtime conformance` without artifact checks,
|
|
- a full compiler toolchain must expose enough artifact surface to prove the stable invariants above,
|
|
- and a host-backed test is not valid conformance evidence unless the selected registry, capability set, and runtime assumptions are declared where they affect the outcome.
|