prometeu-studio/docs/pbs/agendas/13.2. Conformance Workshop 2 - Source-Level Oracle and Diagnostic Oracle.md

PBS Conformance Workshop 2

Status: Active

Purpose

Run the second focused discussion for 13. Conformance Test Specification.md on the source-level oracle:

• positive and negative suites,
• runtime-oracle scope,
• and how diagnostics participate in conformance.

Why This Slice Second

This slice follows claim levels because it defines the actual assertions needed for the frontend and source-runtime portions of conformance.

Proposed Meeting Order

1. Reconfirm the claim taxonomy from Workshop 1.
2. Close the positive and negative source-level suite boundaries.
3. Close the source-level runtime oracle.
4. Close the diagnostic oracle granularity.
5. Record artifact and fixture dependencies for Workshop 3.

Decisions To Produce

1. The exact source-level oracle categories.
2. The minimum diagnostic assertions conformance may check.
3. The distinction between acceptance/rejection tests and runtime-oracle tests.
4. The boundary between source-level conformance and artifact-level conformance.

Candidate Decisions

1. Keep Source-Level Oracle As The Core Baseline

Candidate direction:

• conformance must at minimum assert parsing, rejection, binding, typing, dynamic semantics, and memory/identity behavior at source level.

Rationale:

• This matches the current maturity of the corpus.

2. Diagnostic Oracle Checks Stable External Fields Only

Candidate direction:

• conformance may assert: acceptance or rejection, diagnostic phase, source attribution, and stable rejection class if adopted,
• but not exact message wording.

Rationale:

• This aligns 13 to the likely direction of 11.

3. Runtime Oracle Remains Language-Observable

Candidate direction:

• conformance asserts only language-observable outcomes at this stage,
• not implementation timing or internal GC behavior.

Rationale:

• This preserves portability across implementations.

Questions To Resolve In The Room

1. Should diagnostic codes become part of the conformance oracle if 11 later adopts them?
2. Must source-level runtime tests be required for every claim level or only some?
3. How should trap behavior be asserted without over-specifying runtime internals?
4. Which negative tests belong to parsing versus static semantics versus linking?
5. How much source attribution detail is enough for conformance?

Expected Outputs

1. a decision record for source-level oracle scope,
2. a decision record for diagnostics in conformance,
3. and suite-structure targets for 13.

Explicit Deferrals For Workshop 3

• artifact goldens,
• stdlib and host fixtures,
• and runtime-line environment modeling.

Inputs

• docs/pbs/specs/11. Diagnostics Specification.md
• docs/pbs/specs/13. Conformance Test Specification.md
• docs/pbs/agendas/11.1. Diagnostics Workshop 1 - Diagnostic Identity and Attribution.md
• docs/pbs/agendas/13.1. Conformance Workshop 1 - Conformance Claim Levels.md