Files
prometeu-studio/discussion/lessons/DSC-0112-compiler-verifier-source-diagnostics/LSN-0104-faithful-compiler-attribution-and-structured-failures.md
T
bquarkz 1471b427f1
Test / Tests and coverage occurred while executing withChecks step.
Intrepid/Prometeu/Studio/pipeline/pr-master There was a failure building this commit
feat(studio): deliver asset authoring and automation workflows
Add shared human/task asset batches with reviewed atomic publication, explicit
reference rebinds, ownership inspection and qualified glyph/scene budgets.
Expose the same authoring and capacity results through Assets and MCP.

Deliver isolated guest test targets, retained visual evidence and immutable
runtime lifecycle/control observations. Preserve provider qualification and
installed authoring examples alongside the completed discussion lessons.

Close AGD-106 coordination and the delivered discussion workflows through
housekeeping, updating lesson references and preserving qualification limits.
Sound remains outside the asset delivery wave.

Validation: 579 module tests and 31 additional desktop, process and installed
checks passed; native glyph/scene 16/17 boundaries qualified. Discussion
validation passed after housekeeping.
2026-11-11 01:21:25 +00:00

13 KiB
Raw Blame History

id, ticket, title, created, tags
id ticket title created tags
LSN-0104 compiler-verifier-source-diagnostics Faithful Compiler Attribution and Structured Failures 2026-11-09
compiler
backend
diagnostics
testing

Original Problem and Scope

Owner: compiler, common verification diagnostics and public pipeline failure. Discussion: DSC-0112, originating in AGD-0114 under umbrella DSC-0104/AGD-0106. The user accepted DEC-0089 and authorized PLN-0167 followed by implementation. This lesson preserves the completed local compiler result after workflow cleanup.

IRVM validation knew the function and instruction PC, while the emission plan already carried source spans. The validation exception dropped that association. Lowering could fail before assigning ctx.irvm, and an optimizer failure could refer to a program whose offsets differed from the input program. Looking up source later in the stage could therefore use no program or the wrong program.

Separately, analysis errors could still produce a final pipeline success log. Public compile/build rejection exposed only a message-only BuildException even when analysis had structured diagnostics. An AI consumer had to recover context from logs or infer a location from a function index and PC.

Consolidated Decision

Capture diagnostic attribution while the exact checked program is available and carry it through the existing models and facade. Preserve verifier rules, codes, PBX serialization, source language semantics and legacy constructors/getters. The normative contract is now in Compiler 15 §7.1 and Compiler 19 §§12–13.

The implementation separates four questions that consumers must not collapse:

  1. Where was the problem detected? Function-relative IRVM byte offset and the source span mapped to that instruction, when known.
  2. What produced the instruction? Known source, explicitly generated code or unknown provenance. A missing span does not establish generated provenance.
  3. What precision does the source provide? Instruction location, enclosing function context or no usable location. Generated anchors are contextual.
  4. What failed? Program rejection, an internal compiler invariant, a known verification budget, observed infrastructure failure or an unclassified failure.

Detection does not establish cause. A RET identifies where the return-stack mismatch was detected; a join identifies the destination where incoming heights conflicted. A current pass identifies the validation context, not proof that the pass introduced the defect.

Final Implementation

Shared diagnostic values

Compiler-core provides SourceAttribution, VerificationContext, DiagnosticContext, FailureKind and SourceReference. The models are immutable and independent of Studio/MCP/LSP DTOs. Structured parameters are copied recursively and contain scalar/list/map data, not Throwable objects.

BuildingIssue retains its legacy fields and adds shared context, a precise source Span, related spans and template/placeholders. Its conversions to/from Diagnostic preserve those fields. Diagnostic.getReportedPhase() preserves a technical phase even if the semantic DiagnosticPhase enum classifies it as unknown. A long source interval is retained separately from the legacy integer projection.

RelatedSpan retains its old constructor and now records its contextual role and known relationship. An enclosing function can be related context while the primary instruction source is explicitly absent.

Attribution follows the checked program

BytecodeEmitter operation/function plans retain attribution metadata alongside the existing source hooks. Lowering records original instruction spans, known synthetic function provenance and module/function context. It does not promote the legacy bytecode span fallback to a proven instruction location when the original span is missing. Jump patching and optimizer rewrites retain the same attribution.

IRVMValidator enriches its exception at the function boundary using exact instruction boundaries calculated by encodedSize(). Lowering attaches lowering-output before return; optimization attaches optimization-input or optimization-pass-output and the pass name. The stages receive the enriched failure without consulting an unassigned or stale pipeline program.

The verification coordinates are pcSpace=irvm-function, pcUnit=byte, measured from zero inside that checked function. These are distinct from PBX offsets and executed runtime PCs. A module/program without an emission plan retains its original error and coordinates with absenceReason=no-emission-plan.

RET errors expose expectedHeight/actualHeight; join errors expose existingHeight/incomingHeight. No number extraction from the message is required.

Public facade failures retain useful data

The facade still exposes analyze/compile/build and rejects compilation/build through BuildException. The exception now provides:

try {
    var compiled = BuilderPipelineService.INSTANCE.compile(context, logs);
} catch (BuildException failure) {
    if (failure.hasDiagnosticPayload()) {
        var issues = failure.getDiagnostics();
        var stage = failure.getFailedStage();
        var files = failure.getSourceReferences();
        // Resolve each compilation-local FileId using this failure's own map.
    }
}

Legacy exceptions report hasDiagnosticPayload()==false. They are not evidence of a valid empty analysis. A completed analysis may contain program errors; compile/build preserve those issues when rejecting. Fail-fast also preserves earlier warnings. Technical exceptions fail the operation rather than returning an empty analysis.

Failure source references copy the known project/path identities into an immutable map before leaving the context. Two compilations can both use FileId 0 without sharing source identity. Snapshotting does not read live source or invent a content revision from timestamps. Successful results keep their analysis snapshot/file table.

Frontend projection now retains the original diagnostic message, phase, related spans, templates and parameters. Frontend success is logged only after projection confirms no errors. Pipeline groups log success only when that group passed; progress from an earlier successful dependency stage does not imply final build approval.

Existing preload work/scratch guard refusals carry verification-limit and resource/actual/limit data, preserving their rejection code and budgets. Artifact-write IOException carries infrastructure-failure and outputPath. Neither classification depends on message parsing. Compiler-produced invalid IR is internal-failure even when it has a source anchor.

Examples and Evidence

A controlled RET in the retained observations has a source span at offset zero and explicit internal failure, rather than blaming the user's source merely because a location exists. A controlled optimizer pass moves RET from PC 6 to PC 0; the failure reports the output PC and surviving source anchor with optimization-pass-output and the actual pass name.

The same report preserves these origin distinctions:

Case Origin Source availability
Source RET source, instruction, detection Real FileId/span.
Generated anchor generated, instruction, context Real anchor without a causal claim.
Generated without anchor generated, none generated-without-anchor.
Unknown origin unknown, none no-source-span; function context remains related.
No emission plan unknown, none no-emission-plan, with known function/PC retained.

The original PBS F1-direct source still rejects with E_SEM_EXEC_LOWERING_UNRESOLVED_CALLEE, now accompanied by structured source identity and program-rejection. F1-local still compiles through the public facade. This qualifies the diagnostic path; it is not the PBS product correction.

The evidence runner and Java probe produce nine structured observations using checkout classes and identified distribution dependency JARs. Provenance records class/source/JAR hashes, classpath order, worktree state and the frozen corpus hash. The qualification report maps the full local acceptance coverage to tests and retains the required regression outcome.

Validation and Limits

The required Gradle regression passed across compiler-core, build-pipeline, PBS, LSP, app and installed knowledge: 704 tests recorded, 691 passed, 13 skipped, zero failures/errors. Existing skipped integration tests are not presented as runtime proof. The exact command and per-suite counts are retained with the evidence.

Focused tests cover known-source RET/join, generated and unknown origins, function context, no-plan validation, pre-return lowering failure, shifted pass output, projection fidelity, true analysis/fail-fast results, independent reused FileIds, a synthetic frontend without PBS sources, the original PBS fixtures, backend phase round-trip, verifier limits and I/O classification. Existing valid/determinism/ optimizer controls also passed. See the conformance matrix.

Consumer regression uncovered an existing workspace-symbol test that expected Bla in the shared sample project after the sample had stopped declaring that type. The test now writes its own temporary physical project and retains its original symbol assertions. No LSP production behavior was changed for that fixture repair.

The AGD-0113 corpus hash remains 0ad291942af21e990f5cae3348a5be368dee865aa2843ba0d0effd1a9136cb87. Its earlier evidence was not overwritten. Historical Zelda BAD_RET and Scene reports remain non-reproduced; controlled failures do not claim to fix them. No runtime repository writes/commits or live Studio/MCP/LSP qualification occurred in this work. The app's installed-knowledge test passed, but the probe is checkout qualification.

Pitfalls

  • Looking up source after lowering throws can find no assigned program. Capture it before control leaves the verifier.
  • Looking up an optimized failure in the input program can produce a plausible, incorrect line. Preserve instruction/operation metadata through each rewrite.
  • Span absence does not prove code generation. Explicit generation and contextual anchors are separate from location precision.
  • A related function span cannot replace an absent primary instruction span.
  • The semantic phase enum cannot represent every backend stage. Preserve the reported phase and technical context during conversion.
  • FileIds are local. Retain this compilation's identity map rather than resolving them in whichever file table the consumer currently holds.
  • Logs describe progress. An earlier dependency success is compatible with a later failed analysis; the final operation outcome comes from the structured result.
  • Tests of physical workspace symbols should own their fixture declarations rather than depend on a changing demonstration game.

Studio Handoff and Remaining Work

LSN-0105 owns the Studio-neutral projection, operation/document/build version correlation, LSP query failure/cancellation/cleanup, public automation/MCP schemas and FE projection. The Studio implementation and integrated proof were completed on 2026-11-10 and are consolidated there. The user accepted that integrated first-delivery evidence later that day, releasing the PBS implementation gate. It must preserve the common compiler data and use stable FailureKind IDs, retaining the distinction between program diagnostics and failed operations. Shipper stays internal; no direct public Shipper endpoint/workspace is required.

The local compiler result completes DSC-0112; integrated acceptance also required the Studio evidence. After the user accepted both deliveries on 2026-11-10, DEC-0088/PLN-0166 corrected all eight PBS failures. The completed result and immutable before/after evidence are consolidated in LSN-0106. The umbrella records closure of the combined result. This subsequent work does not alter the historical diagnostic evidence recorded here.

Code References