feat(compiler): port runtime catalogs and enforce target ABI (AGD77)

Import both runtime catalogs and provenance, port all intrinsic metadata and exact constants, and validate target signatures and carriers through the common pipeline. Package matching compiler/SDK resources and add active parity plus production runtime conformance checks.

Validation: 420 compiler/PBS tests passed, with 12 existing skips; provider export, runtime execution and installed catalog checks passed. The pre-existing Gfx.clear typed-metadata failure remains active and assigned to AGD78 under the explicitly accepted scope revision recorded in LSN-0080.
This commit is contained in:
2026-10-15 13:07:49 +01:00
parent bd5694dbdf
commit 603ed01227
32 changed files with 1356 additions and 457 deletions
@@ -0,0 +1,182 @@
---
id: LSN-0080
ticket: compiler-runtime-catalog-parity
title: Manual runtime catalog port and independent conformance
created: 2026-10-15
tags: [runtime-integration]
---
## Original problem and ownership
Owner: `compiler/general`; consumers: `compiler/pbs`, `vm-arch` and `studio`.
This lesson consolidates AGD-0077, accepted DEC-0064 and completed PLN-0143
(DSC-0075), including the user's explicit acceptance revision on 2026-10-15.
Their workflow files were removed after propagation and CLI housekeeping.
The runtime published 105 intrinsics with 18 fields and 15 builtin constants
occupying 16 slots. Studio had 42 intrinsic records with 12 fields, no constant
mirror and a disabled parity test. Copying the CSV could not update the compiler's
manual ABI representation, reject retired endpoints or preserve new effects.
Against the previous Studio catalog, the port added 82 identities, removed 19
and updated 23 retained identities.
## Consolidated decision
The runtime owns identities, layouts, constant values, effects and costs.
Studio imports both CSVs unchanged at an explicit target revision, inventories
the differences, then manually ports the compiler representation and consumers.
`IRVMIntrinsicDefinition` represents the runtime supported by this compiler
version. Normal compilation uses local versioned code/resources; it neither
queries the latest runtime nor depends on a sibling checkout.
Manual maintenance keeps compiler adaptations explicit and reviewable, at the
cost of maintaining a port. Active comparisons against independently exported
CSV data detect drift. Enum generation, a shared implementation library and
moving the backend were not required. An explicit provider check uses runtime
source/tests to establish behavior and export provenance. A different runtime
commit is not itself proof of incompatibility; revision and hashes identify
the evidence target without introducing version negotiation.
Read dispatch, verifier and execution tests as well as CSVs. CSV fields describe
the published shape but cannot explain all null handling, native-reference
validation, traps or charging behavior.
## Final implementation
- `IRVMIntrinsicDefinition` and `IRVMIntrinsicRegistry` cover all 105 entries and
18 published fields. Exact name/version resolution and ID lookup reject
retired identities; old IDs are not compatibility aliases.
- `IRVMBuiltinConstantDefinition` independently groups all 15 constants by
target/name/version, preserving ordered slots and exact values/bits. Its
materialization supplies common bytecode constant values.
- Lowering checks caller counts against the selected definition. IRVM validation,
emission and bytecode precheck reject unknown IDs and inconsistent signature
caches, including unknown IDs in unreachable code. Module-only IRVM emission
derives intrinsic signatures from the same target definitions.
- Explicit carrier mappings distinguish Int32, Int64, Float, Bool, String,
color and Random. Unsupported tokens fail. Static facts do not claim to prove
native handle identity or session validity; the runtime remains authoritative.
- Compiler and PBS SDK JARs package both CSVs and provenance from checked-in
mirrors at `runtime-catalog/compiler/` and `runtime-catalog/pbs-sdk/`.
Installed-resource tests validate actual JAR bytes/hashes outside the checkout.
- Active parity tests, synthetic common-IR behavior and the production runtime
harness provide separate evidence. No new PBS facade or numeric/random
algorithm was implemented in this common backend work.
## Examples and pitfalls
**A retained name can have a different ABI.** `input.touch.x/y` moved to
`0x2134/0x2135`, lost their receivers and return Int64. Resolving the name alone
is insufficient; stale caller counts must fail. Input metadata coverage does
not mean the existing PBS input syntax already lowers to these direct queries.
**Constants are not generic zero-argument intrinsics.** `mathf.pi` and
`mathf.tau` materialize Float bits `0x400921FB54442D18` and
`0x401921FB54442D18`. `vec2.zero` materializes two Float slots and has no
intrinsic ID. Some color names independently have both mechanisms, which does
not establish a rule for other constants. Pool deduplication may share the
zero value but must preserve both emitted vec2 components.
**Determinism is not purity.** All four Random intrinsics mutate guest state;
only `random.new` allocates. The tested common program aliases one generator,
discards a `next` result, then mixes `nextf`, `range` and `next`. Removing the
discarded call changes the sequence. Optimized and unoptimized executions
matched the independent seed-42 vector and each used 100 cycles/20 steps.
`random.range` preserves base cost 4, cost 8 per attempt and limit 64. The VM
performs charging; storing metadata does not create a second Java scheduler.
**Static and dispatch rules can differ.** Runtime `abi_mask` permits String
only for intrinsic String arguments, while `value_matches_abi_type` accepts
String or null in dispatch. The Studio precheck therefore rejects a proven
null at load time. The production loader confirmed that rejection with
`LOAD_INVALID_CODE/TypeMismatch`; a String `#AABBCCDD` executed to color
`0xAABBCCDD`. Random's static reference mask includes null, leaving native
handle validity to runtime checks.
Do not use a serializer of the Java enum as its own expected catalog, infer
carrier identity from slot width, disable stale fixtures to hide regressions,
or equate JAR resource parity with successful PBS or installed-host execution.
## Qualification evidence
Selected runtime: `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`, clean checkout.
Host: macOS 27.0.1 arm64; JDK 25 for Studio build and JDK 21 for the installed
launcher. Provider exporter used Cargo test profile/default features; the
production VM harness used dev profile/default features and default load
limits. New catalog fixtures used NONE capabilities. Runtime was not modified.
| Artifact | SHA-256 |
| --- | --- |
| INTRINSICS.csv | `0c1bb78a9b81d1e3480ef5713f3d70f0b06772a7dfab2fd00c1f0a6364f98daa` |
| BUILTIN_CONSTANTS.csv | `e407e4718127aa7ef7b1097b5b4b67d86dd979235e73a551a7c36b628cddc039` |
| Check | Result |
| --- | --- |
| Build-pipeline suite | 187 tests: 175 passed, 12 existing skips, zero failures |
| PBS suite | 245 passed, zero failures; no claim of executable PBS migration |
| Normal catalog parity | All fields, identities, constant bits/groups and packaged hashes passed |
| Explicit provider check | Committed-source/mirror/hash comparison and real runtime exporter test passed; missing provider failed as required |
| Catalog execution corpus | Random with alias/discarded mutation, exact constants, vec2.zero and String conversion passed on the actual VM |
| Admission/availability negatives | Removed unreachable ID and proven-null String rejected; known input loaded then trapped UNAVAILABLE without hardware |
| Installed catalog gate | Passed using actual compiler/SDK JAR resources and bundled launcher outside the checkout |
| Full installed example gate | Failed before and after the port: missing typed metadata for Gfx.clear; explicit consumer handoff and acceptance revision below |
| Workflow/patch integrity | `discussion validate` and `git diff --check` passed |
The complete existing PBX harness corpus also passed with its already documented
runtime schema exceptions: LifecycleRole and NonFiniteValue remain absent from
the published fault-schema enum. The harness reports these gaps and preserves
payloads; this port does not claim the upstream schema was corrected.
Reproduce the catalog and consumer checks from the repository root:
```sh
./gradlew :prometeu-compiler:prometeu-build-pipeline:test :prometeu-compiler:frontends:prometeu-frontend-pbs:test --console=plain
./gradlew :prometeu-app:installedKnowledgeTest --tests '*installedCompilerAndSdkCarryTheSameVerifiedCatalog' --console=plain
python3 tools/runtime-catalog/verify.py --runtime ../runtime
python3 tools/pbx-conformance/run.py --runtime ../runtime
```
## Accepted boundary and remaining consumer work
The full `:prometeu-app:installedKnowledgeTest` failed with
`LOWER_IRVM_INVALID_TYPED_REGISTRY: typed function metadata required: Gfx.clear`.
The same failure was reproduced on an independent archive of previous Studio
HEAD `bd5694dbdf5b214b4295fc8e01ddbe0ba5479599`; only diagnostic stdout was
enabled in its test probe to reveal the exact existing error.
On 2026-10-15 the user explicitly approved concluding catalog acceptance and
leaving that correction with AGD-0078. DEC-0064 and PLN-0143 recorded this precise
revision before housekeeping: the installed catalog probe is this delivery's
mandatory installed gate; the full example test remains active and failing.
All catalog, common behavior, affected module and provider-execution checks
remain required and passed. No general waiver for future failures was granted.
- AGD-0078 owns PBS typed metadata, existing input/types and vec2 source
materialization. Correct the Gfx.clear producer path and rerun the full
installed example gate; do not weaken the common typed Registry contract.
- AGD-0083 owns math/mathf facades, overloads and source constant exposure.
`PbsExecutableBodyLowerer.emitBuiltinConstFetch` still assumes an intrinsic
with one return; pi/tau require the independent constant materialization path.
- AGD-0084 owns PBS Random handles, storage, aliasing and source replay tests.
Common-IR execution provides a base, not proof of the future PBS facade.
- AGD-0080 owns actual installed-runtime Build/Play/headless qualification.
Catalog resources identify the compiler target without proving that flow.
These handoffs and the explicit acceptance revision are recorded in the active
agendas and [integration umbrella](../../workflow/agendas/AGD-0075-studio-runtime-readiness.md).
Future catalog updates must review source, imports, manual consumers, provenance
and behavior together. Roll back that set together if necessary.
## References
- [Normative catalog support](../../../docs/compiler/general/specs/runtime-catalog-support.md)
- [Typed common Registry/PBX contract](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md)
- [VM mirror and provider verification policy](../../../docs/vm-arch/README.md)
- [Production runtime harness](../../../tools/pbx-conformance/README.md)
- Runtime sources at the recorded revision: catalog.rs, builtins.rs,
input_intrinsics.rs, math_intrinsics.rs, random_intrinsics.rs, random.rs,
registry_verifier/{heights,types}.rs and tests/numeric_contract.rs.
Workflow tooling note: the installed CLI registers an existing lesson but has
no lesson-allocation command. LSN-0080 was selected by reading the next counter;
`discussion housekeep` performed all index mutations and counter advancement.