prometeu-studio/docs/pbs/agendas/13.4. Conformance Workshop 4 - Regression and Compatibility Matrices.md

PBS Conformance Workshop 4

Status: Active

Purpose

Run the fourth focused discussion for 13. Conformance Test Specification.md on long-lived behavior preservation:

• regression corpus policy,
• compatibility matrices,
• and how published claims are tested over time.

Why This Slice Last

This slice should come last because it depends on:

• claim taxonomy,
• source-level oracle,
• artifact-level scope,
• and compatibility policy direction.

Proposed Meeting Order

1. Reconfirm the settled claim levels and oracle shapes.
2. Decide what becomes part of the regression corpus.
3. Decide how compatibility matrices are represented in tests.
4. Decide how already-published behavior claims are preserved.
5. Record final spec-integration tasks for 13 and 17.

Decisions To Produce

1. Regression corpus policy.
2. Compatibility-matrix testing policy.
3. The relationship between published claims and test obligations.
4. How partial claims and full claims age over time.

Candidate Decisions

1. Published Behavior Claims Must Enter A Regression Corpus

Candidate direction:

• once behavior is claimed as conforming or supported, it should become regression-protected in the corpus appropriate to that claim level.

Rationale:

• This turns compatibility promises into executable checks.

2. Compatibility Matrices Should Be Testable, Not Only Documented

Candidate direction:

• the matrix across language line, stdlib line, runtime line, and conformance claim must have test-backed meaning where support is promised.

Rationale:

• This prevents compatibility tables from becoming marketing-only artifacts.

3. Retired Claims Must Be Explicitly Removed, Not Silently Forgotten

Candidate direction:

• when support is dropped, the corresponding claim and tests must be retired deliberately with migration or policy notes.

Rationale:

• This aligns conformance with compatibility governance.

Questions To Resolve In The Room

1. What exactly enters the regression corpus: only bugs, or every published example and decision?
2. How granular should compatibility matrices be?
3. Can staged claims have separate regression corpora?
4. How are unsupported combinations represented in tests?
5. Which parts of this belong in 13 versus 17?

Expected Outputs

1. a decision record for regression policy,
2. a decision record for compatibility-matrix testing,
3. and a final alignment checklist between 13 and 17.

Explicit Deferrals

• release tooling implementation,
• repository layout for the test corpus,
• and changelog formatting.

Inputs

• docs/pbs/specs/13. Conformance Test Specification.md
• docs/pbs/specs/17. Compatibility and Evolution Policy.md
• docs/pbs/agendas/13.1. Conformance Workshop 1 - Conformance Claim Levels.md
• docs/pbs/agendas/13.3. Conformance Workshop 3 - Artifact-Level Conformance and Fixtures.md
• docs/pbs/agendas/17.3. Compatibility Workshop 3 - Compatibility Matrices and Published Claims.md