diff --git a/discussion/lessons/DSC-0074-compiler-pbx-registry-conformance/LSN-0079-typed-common-ir-to-production-pbx.md b/discussion/lessons/DSC-0074-compiler-pbx-registry-conformance/LSN-0079-typed-common-ir-to-production-pbx.md new file mode 100644 index 00000000..2a10ec33 --- /dev/null +++ b/discussion/lessons/DSC-0074-compiler-pbx-registry-conformance/LSN-0079-typed-common-ir-to-production-pbx.md @@ -0,0 +1,145 @@ +--- +id: LSN-0079 +ticket: compiler-pbx-registry-conformance +title: Typed common IR through production PBX admission +created: 2026-10-14 +tags: [runtime-integration] +--- + +## Original problem and ownership + +Owner: compiler/general; interfaces: compiler/pbs and studio. + +The old producer carried parameter/result slot counts without complete semantic +types and emitted a retired PBX function layout. A writer-only repair could not +recover signatures, distinguish nominal identities or describe closures and +trait implementations. The fix had to cross the entire common producer chain. + +This lesson consolidates AGD-0076, accepted DEC-0063 and completed PLN-0142 +(DSC-0074). Those workflow artifacts were removed by housekeeping after their +normative content and evidence were propagated. The active integration umbrella +is [AGD75](../../workflow/agendas/AGD-0075-studio-runtime-readiness.md). + +## Consolidated decisions + +Use stable declaration references before assigning deterministic dense image +IDs. Preserve Struct/Trait nominal identity; canonicalize Builtin imports, +Array elements and Fn signatures. Names are descriptive, not nominal keys. +Parameters, results, captures, implementations and lifecycle roles are explicit +common values, preserved through IRVM, optimization and emission. + +The common IR remains serializable by design, but a concrete codec was +explicitly deferred. Complete representation does not require selecting a +transport format. References avoid recursive Java object graphs and the public +contract audit rejects mutable or frontend-specific shapes. + +PBS is one frontend. Optional and Result already belong to PBS's normative +language contract and have parser/semantic implementations. Defects in their +executable lowering do not justify new backend categories or rejection rules. +Their implementation and carrier choices remain frontend-owned. The user +explicitly scoped consumer migration to subsequent tickets and a single AGD75 +integrated delivery. No PBS semantics were added to this backend change. + +## Final implementation + +- IRRegistry and FunctionMetadata represent all five type categories, ordered + signatures/captures, roles and implementations. Aggregation remaps callable + references while preserving stable type identity. +- LinkedRegistry canonicalizes imports/structural declarations deterministically + with an iterative dependency traversal. Nominal anchors permit recursion; + purely structural cycles reject. +- Lowering derives widths from semantic signatures, links typed operands and + remaps implementation targets. IRVM coherence and optimization preserve the + Registry and function metadata. +- The PBX writer emits the 32-byte header and all eight mandatory sections, + optional debug, strict UTF-8, exact function records and explicit empty capture + environments. Width caches are not serialized. Constant deduplication now + relocates PUSH_CONST indices rather than silently changing their meanings. +- Local verification decodes all function ranges, validates references and + operand categories, forbids retired/raw opcodes and analyzes reachable stack + and type facts to convergence. Conflicts become Unknown; exact semantic groups + retain provenance when available. Unknown remains subject to runtime checks. +- Production loading remains authoritative for native identity/capability/ABI, + runtime budgets and executable admission. +- Studio previously discarded error.details when a headless response rejected + boot. AutomationFailure now preserves the entire runtime error envelope, + including initialization, binding and resource details, through MCP output. + +## Examples and pitfalls + +A `vec2 -> vec2` function has logical arity one/one and derived slot widths two/two. +The function record writes the two TypeId lists, not the numbers two/two. A test +compares that record with independent fixed bytes from the runtime contract. + +A capturing body is not a direct-call target. MAKE_CLOSURE names the body and +expected Fn type, consumes its explicit capture groups and produces a callable. +CALL_CLOSURE consumes that callable after the explicit arguments. The synthetic +frontend test proves this path without PBS source, SDK or language ASTs. + +Jumps contain absolute offsets from the current function start. They are not +relative to the next PC. A taken jump to function end is a return boundary; +ordinary fallthrough is invalid. These details were checked against the actual +runtime verifier, not inferred from older emitter behavior. + +Do not derive type identity from slot width, spelling or display names. Do not +claim a self-round-trip proves external conformance. Do not treat parser tests +as execution evidence or a load-only argument-taking entry as a boot test. + +Initialization diagnostics have null outer executed PC. A verifier PC remains +nested in initError.load.pc; file offsets remain offsets. Preserve the original +JSON structure rather than flattening it into human-readable messages. + +## Qualification evidence + +Runtime: `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`. +Platform: macOS 27.0.1 arm64. Rust harness uses the runtime Cargo.lock, default +load limits and NONE capabilities except the positive native case (SYSTEM). +Resource-negative cases explicitly lower Types, TemporaryBytes or Work. +Runtime checkout remained unchanged. + +| Gate | Result | +| --- | --- | +| Build-pipeline suite | 178 tests: 163 passed, 15 skipped, no failures | +| Frontend-api suite | 28 passed | +| PBS suite | 245 passed; not a claim of complete executable migration | +| Headless process suite | 9 passed, including six observed runtime fault payloads | +| Architecture suite | 17 passed | +| Independent binary oracle | Exact 194-byte minimal image and fixed vec2 function bytes | +| Production loader positives | 12 admitted images, including native binding | +| Genuine execution | 11 programs: direct/common path, closures/captures, trait, Array/Dynamic, joins, roles, intrinsic and native; vec2 signature is load-only | +| Production negative corpus | 32 rejected malformed/binding/code/resource fixtures | +| Workflow and patch integrity | discussion validate and git diff --check passed | + +The 15 skipped tests are older conditional integration adapters. The dedicated +production harness above ran explicitly; their skip state is not substituted +for runtime evidence. The intrinsic/native catalogue goldens are serialization +fixtures and do not claim current PBS catalogue compatibility. + +Reproduce with the commands in +[the harness README](../../../tools/pbx-conformance/README.md). The harness +creates its Cargo build outside the runtime checkout and saves generated PBX and +observed JSON under the build-pipeline build directory. + +## Remaining integration work + +The runtime LoadReason enum has four values missing from the published headless +schema: LifecycleRole, LifecycleSignature, DuplicateLifecycleRole and +NonFiniteValue. The corpus observes the first and last. Both payloads fail the +original schema solely because of the missing enum value. The harness explicitly +reports this upstream gap, validates the original payload with that single enum +addition, and leaves the payload unchanged. It does not claim those two cases +conform to the unmodified schema. Runtime must correct the schema under AGD75. + +PBS typed producer migration, Optional/Result executable lowering, catalogue +alignment, installed runtime delivery, hook selection, memcard gating and +persistence remain with their consumer tickets. The common schema and synthetic +producer work without those features. There is no legacy PBX fallback and no +slot-to-type inference to conceal an incomplete handoff. + +## References + +- [Typed common Registry/PBX specification](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md) +- [Backend conformance matrix](../../../docs/specs/compiler/22.%20Backend%20Spec-to-Test%20Conformance%20Matrix.md) +- [Runtime qualification harness](../../../tools/pbx-conformance/README.md) +- Runtime sources: registry_v0.rs, program_image.rs, registry_verifier, and + devtools/headless-protocol/response.schema.json at the revision above. diff --git a/docs/compiler/general/specs/typed-registry-and-pbx-v0.md b/docs/compiler/general/specs/typed-registry-and-pbx-v0.md new file mode 100644 index 00000000..f7361b05 --- /dev/null +++ b/docs/compiler/general/specs/typed-registry-and-pbx-v0.md @@ -0,0 +1,182 @@ +# Typed common Registry and PBX v0 + +Status: normative. Owner: `compiler/general`. Runtime contract reference: +`runtime` revision `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`, +`docs/specs/runtime/02c-program-registry-and-pbx-v0.md` and +`docs/vm-arch/ISA_CORE.md` in that repository. + +## Ownership and handoff + +Every executable frontend MUST provide its semantic types through the common +`IRRegistry` model and typed `FunctionMetadata`. The backend MUST NOT inspect +language ASTs, PBS names, Optional/Result syntax, or frontend-specific classes. +Language carriers and their lowering remain owned by the frontend. A carrier +can use ordinary Struct, Array, Trait, Fn and Builtin declarations without +introducing a language category in the backend. + +The public model consists of immutable values and ordered `ReadOnlyList` +collections. References use explicit declaration keys and callable IDs, not +object identity or recursive object graphs. It is serializable by design; +no concrete IR codec, wire format or persistence protocol is selected. + +A function declares ordered named parameters, ordered logical results, ordered +captures, a lifecycle role and an optional name. Empty names differ from absent +names. Captures are an environment, never hidden parameters. Slot counts are +derived from types; existing count fields are checked caches, not semantic +input. Missing nonempty type metadata is an invalid handoff, not a request to +infer types from counts. Optimizers MUST preserve metadata and typed operands. + +## Identity and linking + +TypeId, FunctionId and ImplementationId are separate dense image-local index +domains. Index zero is valid. Trait references use TypeIds; field and method +indices are local to their owner. Source declaration keys MUST identify the +same declaration across repeated module imports and distinguish unrelated +nominal declarations, even when names and shapes coincide. + +Linking is deterministic. The selected physical entry is FunctionId zero; +remaining functions follow canonical module/callable/source ordering. Struct +and Trait retain nominal identity. Builtin imports deduplicate by BuiltinId, +Array by canonical element identity, and Fn by ordered parameter/result type +identities, excluding names. Forward references and recursion through nominal +anchors are valid. Pure Array/Fn cycles are invalid. Every reference must +resolve before emission. Frontends should supply only used imports. + +| BuiltinId | Type | Slots | +| --- | --- | --- | +| `0x0001` | color | 1 | +| `0x0002` | vec2 | 2 | +| `0x0003` | pixel | 3 | +| `0x0010` | Int32 | 1 | +| `0x0011` | Int64 | 1 | +| `0x0012` | Float | 1 | +| `0x0013` | Bool | 1 | +| `0x0014` | String | 1 | +| `0x0015` | Dynamic | 2 | +| `0x0016` | Random | 1 | + +The retired input IDs `0x0100`–`0x0102` have no aliases. Struct, Array and Fn +values occupy one reference slot; Trait values occupy two slots. Width does +not establish type identity or authorize conversion. + +## Binary contract + +All fields are little-endian. The header is 32 bytes: `PBX\0`, u16 version zero, +zero endian/reserved bytes, u32 section count and twenty zero reserved bytes. +Directory entries are u32 kind, file-relative offset and length. Ranges must +not overlap or overflow. The version remains zero; no legacy fallback exists. + +Mandatory sections are constants (0), functions (1), Code (2), exports (4), +syscalls (5), types (6), implementations (7) and captures (8). Debug (3) is +optional. Empty tables contain a u32 zero count. Code is raw bytes, not a +counted table. An empty table set is serializable but is not an executable +program: production loading requires a valid ordinary entry. + +Lists have u32 counts. Optional names use u8 presence 0/1, then u32 UTF-8 byte +length and bytes when present. UTF-8 is strict. Named types contain TypeId then +optional name; results and captures contain TypeIds without names. + +| Record | Ordered payload | +| --- | --- | +| Type | u8 tag, optional name, category payload | +| Builtin (tag 0) | u32 BuiltinId | +| Struct (tag 1) | named fields | +| Array (tag 2) | u32 element TypeId | +| Trait (tag 3) | methods: optional name, parameters, results | +| Fn (tag 4) | parameters, results | +| Function | u8 role, optional name, parameters, results, u32 Code offset, length, extra locals, max temporaries | +| Implementation | u32 owner TypeId, trait TypeId, method count, ordered FunctionIds | +| Captures | function count and one ordered TypeId list per function, including empty lists | + +Function Code offsets are Code-relative. Parameter/result slot counts MUST NOT +be serialized. Max temporaries zero means zero. Signature names are independent +of debug names. Constants retain tags Null 0, Int64 1, Float64 2, Boolean 3, +String 4 and Int32 5. Float payloads must be finite. Constant deduplication +requires relocating every PUSH_CONST index; changing only the table is invalid. + +Roles are None 0, BeforePause 1, BeforeResume 2 and MemcardBoot 3. Nonzero roles +are unique, cannot target entry zero and cannot capture. Pause/resume have empty +signatures; MemcardBoot has `(Int64) -> Int32`. This metadata does not implement +frontend hook selection, mem.bootstrap gating or persistence. + +Implementations are unique per Struct/Trait pair, complete and ordered. Each +method target has an exact signature with a concrete Struct receiver first, +followed by the trait method's explicit parameters. Targets cannot capture. + +## Instructions and verification + +Instructions use u16 opcodes. Jump operands are absolute u32 byte offsets from +the current function start, not displacements from the next instruction. +A taken jump to the current function end is a return boundary; ordinary +fallthrough past the end is invalid. + +| Opcode | Typed operands | +| --- | --- | +| MAKE_CLOSURE `0x52` | FunctionId, Fn TypeId | +| CALL_CLOSURE `0x53` | expected Fn TypeId | +| GET_CAPTURE `0x57` | capture group index | +| CALL_TRAIT `0x58` | Trait TypeId, method index | +| DYN_PACK/UNPACK/CAN_UNPACK `0x90`–`0x92` | TypeId | +| Reference operations `0x93`–`0x96` | source TypeId, target TypeId | +| STRUCT_NEW `0xa0` | Struct TypeId | +| STRUCT_GET/SET `0xa1`–`0xa2` | Struct TypeId, field index | +| Array operations `0xa3`–`0xa6` | Array TypeId | + +Direct CALL cannot target a capturing body. Closure signatures match exactly. +Retired SPAWN/YIELD/SLEEP (`0x54`–`0x56`) and raw producer SYSCALL are invalid, +even when unreachable. All function ranges must decode completely; references, +categories, operand boundaries, finite immediates and branch destinations are +checked even in unreachable instructions. + +Reachable analysis uses a bounded worklist. Parameters initialize local facts; +extra locals start Null and may change type. Joins require equal temporary +heights. Equal facts survive; conflicting facts widen to Unknown. Type errors +are diagnosed after convergence. Unknown alone neither proves compatibility +nor justifies rejection. Runtime group checks remain authoritative. RET and +jump-to-end require exact result width and compatible known facts. GET/SET_LOCAL +address only parameters plus extra locals. No implicit numeric, trait or Dynamic +conversion is introduced by linking, writing or verification. + +HOSTCALL references a declared canonical `(module, name, version)` identity +with exact argument/result slot ABI. Duplicate identities with conflicting ABI, +unused declarations and invalid indices are rejected. Only the runtime loader +binds capabilities and rewrites HOSTCALL to SYSCALL. Intrinsic signatures are +explicit compiler metadata; their IDs and definitive ABI remain runtime-owned. + +## Bounds and diagnostics + +Product ceilings: PBX 64 MiB, Code 32 MiB, 65,536 types and functions each, +262,144 implementations, 1,048,576 nested records, names 4,096 bytes individually +and 4 MiB aggregate, frame 16,384 slots and results 1,024 slots. Runtime admission +also owns retained Registry 64 MiB, scratch 128 MiB and work 268,435,456. A guest +cannot raise these limits. Java validation is not a substitute for runtime +allocation accounting or final `ValidatedProgramImage::load` admission. + +Compilation failures use common LOWER_IRVM/MARSHAL stage diagnostics. Load, +binding and resource failures remain distinct from executed traps. Studio +preserves the full runtime error envelope in `AutomationFailure.runtimeError`, +including null fields and native binding details. The outer initialization +fault PC remains null. A verifier PC stays in `initError.load.pc`; a file offset +MUST NOT become an executed PC. + +## Reproducible conformance + +Run the focused common tests, then the production harness: + +```sh +./gradlew :prometeu-compiler:prometeu-build-pipeline:test --tests '*LinkedRegistryTest' --tests '*PbxRegistryConformanceTest' --tests '*TypedCommonPipelineTest' +python3 tools/pbx-conformance/run.py --runtime ../runtime +``` + +The Java tests write `build/registry-conformance` inside the build-pipeline +module. The harness builds outside the runtime checkout using its Cargo.lock, +loads Java-produced bytes, executes admitted zero-argument fixtures, validates +structured failures against the published schema and records the runtime hash, +platform, capabilities and limits. Parsing, loading and execution are separate +claims. The vec2 signature fixture is load-only; it is not a zero-argument boot. + +The fixed 194-byte minimal image comes from the independent runtime test oracle. +Synthetic common IR additionally exercises linking, lowering, optimization and +emission without PBS. Frontend migration and integrated release readiness are +tracked separately in AGD75; passing common conformance does not claim complete +PBS lowering or complete Studio/runtime integration. diff --git a/docs/specs/compiler/15. Bytecode and PBX Mapping Specification.md b/docs/specs/compiler/15. Bytecode and PBX Mapping Specification.md index 3e81bec9..bb9744bc 100644 --- a/docs/specs/compiler/15. Bytecode and PBX Mapping Specification.md +++ b/docs/specs/compiler/15. Bytecode and PBX Mapping Specification.md @@ -1,5 +1,10 @@ # Bytecode and PBX Mapping Specification +The current typed Registry/PBX v0 contract is normative in +[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md). +Its complete typed function records, mandatory sections and verification rules +replace the earlier slot-only artifact baseline. + Status: Draft v1 (Backend Baseline) Applies to: mapping from lowered/optimized backend programs into PBX sections, bytecode-facing artifacts, and source-to-artifact invariants required by loader/verifier/runtime @@ -74,7 +79,8 @@ Emitter output must map to a `BytecodeModule` shape containing: 3. `code`, 4. `exports`, 5. `syscalls`, -6. optional `debug_info` that still satisfies v1 minimum debug obligations. +6. typed Registry declarations, implementations and per-function capture lists, +7. optional `debug_info` that still satisfies v1 minimum debug obligations. ### 6.2 Function ordering and IDs @@ -168,7 +174,6 @@ At minimum, artifact conformance checks should assert: The following remain deferred: - richer optional debug/source-map formats, -- additional PBX section-level contracts beyond current baseline, - and profile-specific binary compatibility policy details beyond current v1 baseline. ## 11. Non-Goals diff --git a/docs/specs/compiler/19. Verification and Safety Checks Specification.md b/docs/specs/compiler/19. Verification and Safety Checks Specification.md index 364a8c55..740893d7 100644 --- a/docs/specs/compiler/19. Verification and Safety Checks Specification.md +++ b/docs/specs/compiler/19. Verification and Safety Checks Specification.md @@ -1,5 +1,10 @@ # Compiler Verification and Safety Checks Specification +The current typed Registry/PBX v0 contract is normative in +[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md). +Its complete typed function records, mandatory sections and verification rules +replace the earlier slot-only artifact baseline. + Status: Draft v1 (Quality-Gate Baseline) Applies to: test-backed safety gates for common compiler stages, validation ownership across executable frontends, and integration regression safety checks against runtime diff --git a/docs/specs/compiler/20. IRBackend to IRVM Lowering Specification.md b/docs/specs/compiler/20. IRBackend to IRVM Lowering Specification.md index dbc652e8..996f6ced 100644 --- a/docs/specs/compiler/20. IRBackend to IRVM Lowering Specification.md +++ b/docs/specs/compiler/20. IRBackend to IRVM Lowering Specification.md @@ -1,5 +1,10 @@ # IRBackend to IRVM Lowering Specification +The current typed Registry/PBX v0 contract is normative in +[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md). +Its complete typed function records, mandatory sections and verification rules +replace the earlier slot-only artifact baseline. + Status: Draft v1 (Backend Baseline) Applies to: executable-backend lowering from the common `IRBackend` handoff into `IRVM` diff --git a/docs/specs/compiler/22. Backend Spec-to-Test Conformance Matrix.md b/docs/specs/compiler/22. Backend Spec-to-Test Conformance Matrix.md index 9dca081e..3260f8cd 100644 --- a/docs/specs/compiler/22. Backend Spec-to-Test Conformance Matrix.md +++ b/docs/specs/compiler/22. Backend Spec-to-Test Conformance Matrix.md @@ -1,5 +1,10 @@ # Backend Spec-to-Test Conformance Matrix +The current typed Registry/PBX v0 contract is normative in +[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md). +Its complete typed function records, mandatory sections and verification rules +replace the earlier slot-only artifact baseline. + Status: Draft v1 (Traceability Baseline) Applies to: compiler/backend conformance traceability for canonical stage order and entrypoint-specific contracts @@ -140,3 +145,19 @@ Matrix integrity is enforced by automated document lint: - if `PROMETEU_MATRIX_HARD_GATE=true` and `PROMETEU_CHANGED_FILES` contains a path under conformance-relevant FE/BE prefixes, - then this matrix file MUST be present in `PROMETEU_CHANGED_FILES`, otherwise the gate fails. - CI policy uses strict Gate I mode via `PROMETEU_GATE_I_STRICT=true` (or `CI=true`) and requires `PROMETEU_RUNTIME_CHECK_CMD` for runtime-backed evidence. + +## 7. Typed Registry/PBX v0 qualification (2026-10-14) + +Owner: compiler/general. These rows supersede slot-only PBX coverage claims; +frontend syntax and lowering remain consumer-owned. + +| Requirement | Evidence | Status | Scope | +| --- | --- | --- | --- | +| Stable declarations, nominal identity, structural deduplication and cycles | LinkedRegistryTest; IRBackendExecutableContractTest | pass | Common API and linking | +| Complete typed signatures, captures, roles and implementations | PbxRegistryConformanceTest; TypedCommonPipelineTest | pass | Common IR through emission | +| Exact PBX mandatory tables and multi-slot signatures | Independent 194-byte oracle; fixed vec2 function bytes | pass | Wire format, not a self round-trip | +| Runtime final admission and execution | tools/pbx-conformance/run.py | pass | Closure, trait, array, Dynamic, intrinsic, roles, joins, native binding; vec2 load-only | +| Invalid structures, references, code and resource refusal | tools/pbx-conformance/main.rs mutation corpus | pass | Production loader, default/lowered limits | +| Structured initialization and binding diagnostics | HeadlessProcessSessionTest; observed runtime JSON fixtures | pass | Preserves full payload through MCP, including null outer PC | +| Published schema covers current runtime reasons | Actual LifecycleRole/NonFiniteValue payloads against runtime schema | partial | Runtime enum/schema mismatch recorded in AGD75; no payload rewriting | +| PBS migration and integrated delivery | AGD75 and consumer tickets | partial | Existing PBS suite passing is not complete executable lowering evidence | diff --git a/prometeu-app/src/main/java/p/studio/app/runtime/HeadlessProcessSession.java b/prometeu-app/src/main/java/p/studio/app/runtime/HeadlessProcessSession.java index ffb592ed..ef606b3d 100644 --- a/prometeu-app/src/main/java/p/studio/app/runtime/HeadlessProcessSession.java +++ b/prometeu-app/src/main/java/p/studio/app/runtime/HeadlessProcessSession.java @@ -245,7 +245,9 @@ final class HeadlessProcessSession implements HeadlessGateway.Session { default -> AutomationFailure.Code.INFRASTRUCTURE_FAILURE; }; pending.remove(item.id); - item.result.completeExceptionally(new AutomationException(code, "Runtime " + error.path("code").asText() + ": " + error.path("message").asText())); + Map runtimeError = mapper.convertValue(error, new com.fasterxml.jackson.core.type.TypeReference<>() {}); + item.result.completeExceptionally(new AutomationException(new AutomationFailure(code, + "Runtime " + error.path("code").asText() + ": " + error.path("message").asText(), runtimeError))); } else { var result = message.get("result"); if (item.command.equals("cancel") && (!result.path("requested").isBoolean() || !result.path("requested").booleanValue() diff --git a/prometeu-app/src/test/java/p/studio/app/runtime/HeadlessFixture.java b/prometeu-app/src/test/java/p/studio/app/runtime/HeadlessFixture.java index d6e91e54..48c41b46 100644 --- a/prometeu-app/src/test/java/p/studio/app/runtime/HeadlessFixture.java +++ b/prometeu-app/src/test/java/p/studio/app/runtime/HeadlessFixture.java @@ -50,7 +50,15 @@ public final class HeadlessFixture { var request = MAPPER.readTree(line); String id = request.path("requestId").asText(), command = request.path("command").asText(); switch (command) { case "initialize" -> response(id, MAPPER.createObjectNode().setAll(Map.of("protocolVersion", hello.path("protocolVersion"), "capabilities", hello.path("capabilities")))); - case "boot", "reset" -> response(id, state()); + case "boot", "reset" -> { + if (mode.startsWith("load-fault-")) { + var resource = "/runtime/fault-" + mode.substring("load-fault-".length()) + ".json"; + var fault = MAPPER.readTree(HeadlessFixture.class.getResourceAsStream(resource)); + var error = MAPPER.createObjectNode().put("code", "boot_failed").put("message", "Runtime rejected cartridge"); + error.putObject("details").set("fault", fault); + emit(MAPPER.createObjectNode().put("type", "response").put("sessionId", "fixture").put("requestId", id).set("error", error)); + } else response(id, state()); + } case "get_state" -> { if (!mode.equals("hang-call")) response(id, state()); } case "get_fault" -> response(id, MAPPER.nullNode()); case "advance" -> { diff --git a/prometeu-app/src/test/java/p/studio/app/runtime/HeadlessProcessSessionTest.java b/prometeu-app/src/test/java/p/studio/app/runtime/HeadlessProcessSessionTest.java index c36454ad..30713e3d 100644 --- a/prometeu-app/src/test/java/p/studio/app/runtime/HeadlessProcessSessionTest.java +++ b/prometeu-app/src/test/java/p/studio/app/runtime/HeadlessProcessSessionTest.java @@ -18,6 +18,20 @@ class HeadlessProcessSessionTest { var session = new HeadlessProcessSession(process, new ObjectMapper(), HeadlessFixture.limits(mode), "build"); sessions.add(session); return session; } @AfterEach void cleanup() { sessions.forEach(session -> session.abort("test cleanup")); } + @Test void productionLoadDiagnosticsSurviveBootFailureAndMcpSerialization() throws Exception { + var mapper = new ObjectMapper(); + for (String fixture : List.of("resource", "invalid-role", "retired-opcode", "native", "native-abi", "unknown-native")) { + var session = start("load-fault-" + fixture); + var failure = assertThrows(AutomationException.class, () -> session.initializeAndBoot(root)).failure(); + var fault = mapper.readTree(getClass().getResourceAsStream("/runtime/fault-" + fixture + ".json")); + var error = mapper.valueToTree(failure.runtimeError()); + assertEquals(fault, error.path("details").path("fault")); + assertTrue(error.path("details").path("fault").path("pc").isNull()); + var wire = p.studio.app.mcp.McpToolResult.failure(mapper, failure).wire(); + assertEquals(fault, wire.path("structuredContent").path("error").path("runtimeError").path("details").path("fault")); + assertThrows(UnsupportedOperationException.class, () -> failure.runtimeError().put("code", "changed")); + } + } @Test void fragmentedProtocolPreservesUnsignedFrameIdentityLayoutAndSeparateStderr() throws Exception { var session = start("normal"); session.initializeAndBoot(root); var outcome = session.advance(3, ignored -> {}); diff --git a/prometeu-app/src/test/resources/runtime/fault-invalid-role.json b/prometeu-app/src/test/resources/runtime/fault-invalid-role.json new file mode 100644 index 00000000..90c86340 --- /dev/null +++ b/prometeu-app/src/test/resources/runtime/fault-invalid-role.json @@ -0,0 +1,24 @@ +{ + "initError": { + "code": "registry_load_failed", + "load": { + "code": "LOAD_INVALID_REGISTRY", + "function": 0, + "id_domain": null, + "limit": null, + "member": null, + "offset": 136, + "pc": null, + "reason": "LifecycleRole", + "record": 0, + "referenced_id": null, + "requested": null, + "resource": null, + "section": 1 + }, + "message": "RegistryLoadFailed(LoadFailure { code: InvalidRegistry, reason: LifecycleRole, section: Some(1), offset: Some(136), record: Some(0), member: None, id_domain: None, referenced_id: None, function: Some(0), pc: None, resource: None, requested: None, limit: None })" + }, + "kind": "vm_init", + "pc": null, + "summary": "Conformance fixture" +} diff --git a/prometeu-app/src/test/resources/runtime/fault-native-abi.json b/prometeu-app/src/test/resources/runtime/fault-native-abi.json new file mode 100644 index 00000000..06cfbf1a --- /dev/null +++ b/prometeu-app/src/test/resources/runtime/fault-native-abi.json @@ -0,0 +1,19 @@ +{ + "initError": { + "binding": { + "declared_arg_slots": 1, + "declared_ret_slots": 1, + "expected_arg_slots": 0, + "expected_ret_slots": 1, + "module": "system", + "name": "has_cart", + "reason": "abi_mismatch", + "version": 1 + }, + "code": "loader_patch_failed", + "message": "LoaderPatchFailed(ResolveFailed(AbiMismatch { module: \"system\", name: \"has_cart\", version: 1, declared_arg_slots: 1, declared_ret_slots: 1, expected_arg_slots: 0, expected_ret_slots: 1 }))" + }, + "kind": "vm_init", + "pc": null, + "summary": "Conformance fixture" +} diff --git a/prometeu-app/src/test/resources/runtime/fault-native.json b/prometeu-app/src/test/resources/runtime/fault-native.json new file mode 100644 index 00000000..7cbc52ce --- /dev/null +++ b/prometeu-app/src/test/resources/runtime/fault-native.json @@ -0,0 +1,17 @@ +{ + "initError": { + "binding": { + "module": "system", + "name": "has_cart", + "provided": 0, + "reason": "missing_capability", + "required": 1, + "version": 1 + }, + "code": "loader_patch_failed", + "message": "LoaderPatchFailed(ResolveFailed(MissingCapability { required: 1, provided: 0, module: \"system\", name: \"has_cart\", version: 1 }))" + }, + "kind": "vm_init", + "pc": null, + "summary": "Conformance fixture" +} diff --git a/prometeu-app/src/test/resources/runtime/fault-resource.json b/prometeu-app/src/test/resources/runtime/fault-resource.json new file mode 100644 index 00000000..0d3b360d --- /dev/null +++ b/prometeu-app/src/test/resources/runtime/fault-resource.json @@ -0,0 +1,24 @@ +{ + "initError": { + "code": "registry_load_failed", + "load": { + "code": "LOAD_RESOURCE_LIMIT", + "function": null, + "id_domain": null, + "limit": 0, + "member": null, + "offset": null, + "pc": null, + "reason": "Limit", + "record": null, + "referenced_id": null, + "requested": 32, + "resource": "Work", + "section": null + }, + "message": "RegistryLoadFailed(LoadFailure { code: ResourceLimit, reason: Limit, section: None, offset: None, record: None, member: None, id_domain: None, referenced_id: None, function: None, pc: None, resource: Some(Work), requested: Some(32), limit: Some(0) })" + }, + "kind": "vm_init", + "pc": null, + "summary": "Conformance fixture" +} diff --git a/prometeu-app/src/test/resources/runtime/fault-retired-opcode.json b/prometeu-app/src/test/resources/runtime/fault-retired-opcode.json new file mode 100644 index 00000000..12090285 --- /dev/null +++ b/prometeu-app/src/test/resources/runtime/fault-retired-opcode.json @@ -0,0 +1,24 @@ +{ + "initError": { + "code": "registry_load_failed", + "load": { + "code": "LOAD_INVALID_CODE", + "function": 0, + "id_domain": null, + "limit": null, + "member": null, + "offset": null, + "pc": 0, + "reason": "Truncated", + "record": null, + "referenced_id": null, + "requested": null, + "resource": null, + "section": 2 + }, + "message": "RegistryLoadFailed(LoadFailure { code: InvalidCode, reason: Truncated, section: Some(2), offset: None, record: None, member: None, id_domain: None, referenced_id: None, function: Some(0), pc: Some(0), resource: None, requested: None, limit: None })" + }, + "kind": "vm_init", + "pc": null, + "summary": "Conformance fixture" +} diff --git a/prometeu-app/src/test/resources/runtime/fault-unknown-native.json b/prometeu-app/src/test/resources/runtime/fault-unknown-native.json new file mode 100644 index 00000000..06cd2914 --- /dev/null +++ b/prometeu-app/src/test/resources/runtime/fault-unknown-native.json @@ -0,0 +1,15 @@ +{ + "initError": { + "binding": { + "module": "system", + "name": "missing", + "reason": "unknown_syscall", + "version": 1 + }, + "code": "loader_patch_failed", + "message": "LoaderPatchFailed(ResolveFailed(UnknownSyscall { module: \"system\", name: \"missing\", version: 1 }))" + }, + "kind": "vm_init", + "pc": null, + "summary": "Conformance fixture" +} diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeEmitter.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeEmitter.java index 96359c59..5b7e61e0 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeEmitter.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeEmitter.java @@ -6,6 +6,7 @@ import java.io.ByteArrayOutputStream; import java.nio.ByteBuffer; import java.nio.ByteOrder; import java.util.ArrayList; +import java.util.HashMap; import java.util.LinkedHashMap; import java.util.LinkedHashSet; import java.util.Objects; @@ -48,8 +49,11 @@ public class BytecodeEmitter { final EmissionPlan plan) { final var inputPlan = plan == null ? EmissionPlan.empty() : plan; final var canonicalConstPool = dedupeConstPool(inputPlan.constPool()); + final var constantIndices = new HashMap(); + for (int i=0;i(inputPlan.functions().size()); + final var intrinsicSignatures = new LinkedHashMap(); final var orderedSyscalls = new LinkedHashMap(); final var syscallIndexByIdentity = new LinkedHashMap(); @@ -59,6 +63,12 @@ public class BytecodeEmitter { for (final var op : function.operations()) { final var pc = code.size(); switch (op.kind()) { + case REGISTRY -> { + var instruction = Objects.requireNonNull(op.registryInstruction()); + writeOpU32(code, instruction.op().opcode(), instruction.immediate()); + if (instruction.secondImmediate() != null) code.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(instruction.secondImmediate()).array()); + spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span()))); + } case HALT -> { writeOpNoImm(code, OP_HALT); spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span()))); @@ -72,7 +82,7 @@ public class BytecodeEmitter { spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span()))); } case PUSH_CONST -> { - writeOpU32(code, OP_PUSH_CONST, op.immediate()); + writeOpU32(code, OP_PUSH_CONST, constantIndices.get(inputPlan.constPool().get(op.immediate()))); spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span()))); } case POP -> { @@ -176,6 +186,10 @@ public class BytecodeEmitter { spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span()))); } case INTRINSIC -> { + if(op.expectedArgSlots()==null||op.expectedRetSlots()==null) throw new IllegalArgumentException("intrinsic signature is required"); + var signature = new BytecodeModule.IntrinsicSignature(op.immediate(),op.expectedArgSlots(),op.expectedRetSlots()); + var previous = intrinsicSignatures.putIfAbsent(op.immediate(),signature); + if(previous!=null&&!previous.equals(signature)) throw new IllegalArgumentException("conflicting intrinsic ABI"); writeOpU32(code, OP_INTRINSIC, op.immediate()); spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span()))); } @@ -197,6 +211,7 @@ public class BytecodeEmitter { syscallIndexByIdentity.put(identity, syscallIndexByIdentity.size()); orderedSyscalls.put(identity, decl); } + if (!orderedSyscalls.get(identity).equals(decl)) throw new IllegalArgumentException("conflicting HOSTCALL ABI for " + identity); final var index = syscallIndexByIdentity.get(identity); writeOpU32(code, OP_HOSTCALL, index); spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span()))); @@ -213,7 +228,7 @@ public class BytecodeEmitter { function.localSlots(), function.returnSlots(), function.maxStackSlots(), - ReadOnlyList.wrap(spans))); + ReadOnlyList.wrap(spans), function.metadata())); } final var layout = BytecodeFunctionLayoutBuilder.build(ReadOnlyList.wrap(functionFragments)); @@ -224,7 +239,7 @@ public class BytecodeEmitter { layout.code(), layout.debugInfo(), canonicalExports, - ReadOnlyList.wrap(orderedSyscalls.values())); + ReadOnlyList.wrap(orderedSyscalls.values()), inputPlan.registry(), ReadOnlyList.wrap(intrinsicSignatures.values())); } private static void writeOpNoImm(final ByteArrayOutputStream out, final int opcode) { @@ -283,10 +298,19 @@ public class BytecodeEmitter { } public record EmissionPlan( + int version, + ReadOnlyList constPool, + ReadOnlyList exports, + ReadOnlyList functions, + p.studio.compiler.models.registry.IRRegistry registry) { + public EmissionPlan( int version, ReadOnlyList constPool, ReadOnlyList exports, ReadOnlyList functions) { + this(version, constPool, exports, functions, p.studio.compiler.models.registry.IRRegistry.empty()); + } + public EmissionPlan { constPool = constPool == null ? ReadOnlyList.empty() : constPool; exports = exports == null ? ReadOnlyList.empty() : exports; @@ -299,12 +323,23 @@ public class BytecodeEmitter { } public record FunctionPlan( + String name, + int paramSlots, + int localSlots, + int returnSlots, + int maxStackSlots, + ReadOnlyList operations, + p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) { + public FunctionPlan( String name, int paramSlots, int localSlots, int returnSlots, int maxStackSlots, ReadOnlyList operations) { + this(name, paramSlots, localSlots, returnSlots, maxStackSlots, operations, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null); + } + public FunctionPlan { Objects.requireNonNull(name, "name"); operations = operations == null ? ReadOnlyList.empty() : operations; @@ -312,6 +347,7 @@ public class BytecodeEmitter { } public enum OperationKind { + REGISTRY, HALT, RET, PUSH_CONST, @@ -347,12 +383,23 @@ public class BytecodeEmitter { } public record Operation( + OperationKind kind, + int immediate, + BytecodeModule.SyscallDecl syscallDecl, + Integer expectedArgSlots, + Integer expectedRetSlots, + BytecodeModule.SourceSpan span, + p.studio.compiler.backend.irvm.IRVMInstruction registryInstruction) { + public Operation( OperationKind kind, int immediate, BytecodeModule.SyscallDecl syscallDecl, Integer expectedArgSlots, Integer expectedRetSlots, BytecodeModule.SourceSpan span) { + this(kind, immediate, syscallDecl, expectedArgSlots, expectedRetSlots, span, null); + } + public Operation { Objects.requireNonNull(kind, "kind"); } diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeFunctionLayoutBuilder.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeFunctionLayoutBuilder.java index 55606434..1e94863e 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeFunctionLayoutBuilder.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeFunctionLayoutBuilder.java @@ -25,7 +25,7 @@ public final class BytecodeFunctionLayoutBuilder { fn.paramSlots(), fn.localSlots(), fn.returnSlots(), - fn.maxStackSlots())); + fn.maxStackSlots(), fn.metadata())); names.add(new BytecodeModule.FunctionName(i, fn.name())); for (final var span : fn.instructionSpans()) { @@ -58,6 +58,15 @@ public final class BytecodeFunctionLayoutBuilder { } public record FunctionFragment( + String name, + byte[] code, + int paramSlots, + int localSlots, + int returnSlots, + int maxStackSlots, + ReadOnlyList instructionSpans, + p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) { + public FunctionFragment( String name, byte[] code, int paramSlots, @@ -65,6 +74,9 @@ public final class BytecodeFunctionLayoutBuilder { int returnSlots, int maxStackSlots, ReadOnlyList instructionSpans) { + this(name, code, paramSlots, localSlots, returnSlots, maxStackSlots, instructionSpans, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null); + } + public FunctionFragment { Objects.requireNonNull(name, "name"); code = code == null ? new byte[0] : code.clone(); diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeLinkPrecheckService.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeLinkPrecheckService.java index 83d336c0..86ddac4d 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeLinkPrecheckService.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeLinkPrecheckService.java @@ -46,16 +46,18 @@ public class BytecodeLinkPrecheckService { } private void validateHostcallIndices(final BytecodeModule module) { - var pc = 0; - while (pc < module.code().length) { - if (pc + 2 > module.code().length) { + for (var function : module.functions()) { + int end = function.codeOffset() + function.codeLen(); + var pc = function.codeOffset(); + while (pc < end) { + if (pc + 2 > end) { throw new BytecodeMarshalingException( BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION, "truncated opcode at pc=" + pc); } final var opcode = readU16(module.code(), pc); final var size = sizeForOpcode(opcode, pc); - if (pc + size > module.code().length) { + if (pc + size > end) { throw new BytecodeMarshalingException( BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION, "truncated instruction at pc=" + pc); @@ -70,6 +72,7 @@ public class BytecodeLinkPrecheckService { } pc += size; } + } } private int sizeForOpcode( diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeMarshalingErrorCode.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeMarshalingErrorCode.java index 5b155dad..37cab986 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeMarshalingErrorCode.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeMarshalingErrorCode.java @@ -1,6 +1,8 @@ package p.studio.compiler.backend.bytecode; public enum BytecodeMarshalingErrorCode { + MARSHAL_FORMAT_REGISTRY_INVALID, + MARSHAL_VERIFY_REGISTRY_INVALID, MARSHAL_FORMAT_STRING_TOO_LONG, MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG, MARSHAL_FORMAT_SYSC_NAME_TOO_LONG, diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeModule.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeModule.java index 07cfc78b..e26e90c0 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeModule.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeModule.java @@ -2,11 +2,7 @@ package p.studio.compiler.backend.bytecode; import p.studio.utilities.structures.ReadOnlyList; -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; import java.nio.charset.StandardCharsets; -import java.util.ArrayList; import java.util.Objects; public record BytecodeModule( @@ -16,16 +12,38 @@ public record BytecodeModule( byte[] code, DebugInfo debugInfo, ReadOnlyList exports, - ReadOnlyList syscalls) { + ReadOnlyList syscalls, + p.studio.compiler.models.registry.IRRegistry registry, + ReadOnlyList intrinsicSignatures) { + public BytecodeModule( + int version, + ReadOnlyList constPool, + ReadOnlyList functions, + byte[] code, + DebugInfo debugInfo, + ReadOnlyList exports, + ReadOnlyList syscalls, + p.studio.compiler.models.registry.IRRegistry registry) { + this(version, constPool, functions, code, debugInfo, exports, syscalls, registry, ReadOnlyList.empty()); + } - private static final int SECTION_KIND_CONST_POOL = 0; - private static final int SECTION_KIND_FUNCTIONS = 1; - private static final int SECTION_KIND_CODE = 2; - private static final int SECTION_KIND_DEBUG = 3; - private static final int SECTION_KIND_EXPORTS = 4; - private static final int SECTION_KIND_SYSCALLS = 5; + public BytecodeModule( + int version, + ReadOnlyList constPool, + ReadOnlyList functions, + byte[] code, + DebugInfo debugInfo, + ReadOnlyList exports, + ReadOnlyList syscalls) { + this(version, constPool, functions, code, debugInfo, exports, syscalls, p.studio.compiler.models.registry.IRRegistry.empty()); + } + + + public record IntrinsicSignature(int id, int argumentSlots, int resultSlots) {} public BytecodeModule { + intrinsicSignatures = intrinsicSignatures == null ? ReadOnlyList.empty() : ReadOnlyList.wrap(intrinsicSignatures.asList()); + registry = registry == null ? p.studio.compiler.models.registry.IRRegistry.empty() : registry; constPool = constPool == null ? ReadOnlyList.empty() : constPool; functions = functions == null ? ReadOnlyList.empty() : functions; code = code == null ? new byte[0] : code.clone(); @@ -45,191 +63,7 @@ public record BytecodeModule( } public byte[] serialize() { - final var constPoolData = serializeConstPool(); - final var functionData = serializeFunctions(); - final var codeData = code(); - final var debugData = serializeDebug(); - final var exportData = serializeExports(); - final var syscallData = serializeSyscalls(); - - final var sections = new ArrayList(); - if (constPoolData.length > 0) { - sections.add(new SectionData(SECTION_KIND_CONST_POOL, constPoolData)); - } - if (functionData.length > 0) { - sections.add(new SectionData(SECTION_KIND_FUNCTIONS, functionData)); - } - if (codeData.length > 0) { - sections.add(new SectionData(SECTION_KIND_CODE, codeData)); - } - if (debugData.length > 0) { - sections.add(new SectionData(SECTION_KIND_DEBUG, debugData)); - } - if (exportData.length > 0) { - sections.add(new SectionData(SECTION_KIND_EXPORTS, exportData)); - } - // SYSC section is mandatory, even when empty. - sections.add(new SectionData(SECTION_KIND_SYSCALLS, syscallData)); - - final var out = new ByteArrayOutputStream(); - - out.writeBytes(new byte[] { 'P', 'B', 'X', 0 }); - writeU16(out, version & 0xFFFF); - out.write(0); - out.write(0); - writeU32(out, sections.size()); - out.writeBytes(new byte[20]); - - var currentOffset = 32 + (sections.size() * 12); - for (final var section : sections) { - writeU32(out, section.kind); - writeU32(out, currentOffset); - writeU32(out, section.data.length); - currentOffset += section.data.length; - } - - for (final var section : sections) { - out.writeBytes(section.data); - } - - return out.toByteArray(); - } - - private byte[] serializeConstPool() { - if (constPool.isEmpty()) { - return new byte[0]; - } - final var out = new ByteArrayOutputStream(); - writeU32(out, constPool.size()); - for (final var entry : constPool) { - switch (entry) { - case NullConstant ignored -> out.write(0); - case Int64Constant value -> { - out.write(1); - writeI64(out, value.value()); - } - case Float64Constant value -> { - out.write(2); - writeF64(out, value.value()); - } - case BooleanConstant value -> { - out.write(3); - out.write(value.value() ? 1 : 0); - } - case StringConstant value -> { - out.write(4); - writeU32(out, value.utf8().length); - out.writeBytes(value.utf8()); - } - case Int32Constant value -> { - out.write(5); - writeI32(out, value.value()); - } - } - } - return out.toByteArray(); - } - - private byte[] serializeFunctions() { - if (functions.isEmpty()) { - return new byte[0]; - } - final var out = new ByteArrayOutputStream(); - writeU32(out, functions.size()); - for (final var function : functions) { - writeU32(out, function.codeOffset()); - writeU32(out, function.codeLen()); - writeU16(out, function.paramSlots()); - writeU16(out, function.localSlots()); - writeU16(out, function.returnSlots()); - writeU16(out, function.maxStackSlots()); - } - return out.toByteArray(); - } - - private byte[] serializeDebug() { - if (debugInfo == null || (debugInfo.pcToSpan().isEmpty() && debugInfo.functionNames().isEmpty())) { - return new byte[0]; - } - final var out = new ByteArrayOutputStream(); - writeU32(out, debugInfo.pcToSpan().size()); - for (final var entry : debugInfo.pcToSpan()) { - writeU32(out, entry.pc()); - writeU32(out, entry.span().fileId()); - writeU32(out, entry.span().start()); - writeU32(out, entry.span().end()); - } - writeU32(out, debugInfo.functionNames().size()); - for (final var fn : debugInfo.functionNames()) { - writeU32(out, fn.funcIdx()); - writeU32(out, fn.utf8Name().length); - out.writeBytes(fn.utf8Name()); - } - return out.toByteArray(); - } - - private byte[] serializeExports() { - if (exports.isEmpty()) { - return new byte[0]; - } - final var out = new ByteArrayOutputStream(); - writeU32(out, exports.size()); - for (final var export : exports) { - writeU32(out, export.funcIdx()); - writeU32(out, export.utf8Symbol().length); - out.writeBytes(export.utf8Symbol()); - } - return out.toByteArray(); - } - - private byte[] serializeSyscalls() { - final var out = new ByteArrayOutputStream(); - writeU32(out, syscalls.size()); - for (final var syscall : syscalls) { - if (syscall.utf8Module().length > 0xFFFF) { - throw new BytecodeMarshalingException( - BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG, - "syscall module name exceeds u16: " + syscall.module()); - } - if (syscall.utf8Name().length > 0xFFFF) { - throw new BytecodeMarshalingException( - BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_NAME_TOO_LONG, - "syscall name exceeds u16: " + syscall.name()); - } - writeU16(out, syscall.utf8Module().length); - out.writeBytes(syscall.utf8Module()); - writeU16(out, syscall.utf8Name().length); - out.writeBytes(syscall.utf8Name()); - writeU16(out, syscall.version()); - writeU16(out, syscall.argSlots()); - writeU16(out, syscall.retSlots()); - } - return out.toByteArray(); - } - - private static void writeU16(final ByteArrayOutputStream out, final int value) { - out.writeBytes(ByteBuffer.allocate(2).order(ByteOrder.LITTLE_ENDIAN).putShort((short) (value & 0xFFFF)).array()); - } - - private static void writeU32(final ByteArrayOutputStream out, final int value) { - out.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(value).array()); - } - - private static void writeI32(final ByteArrayOutputStream out, final int value) { - out.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(value).array()); - } - - private static void writeI64(final ByteArrayOutputStream out, final long value) { - out.writeBytes(ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN).putLong(value).array()); - } - - private static void writeF64(final ByteArrayOutputStream out, final double value) { - out.writeBytes(ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN).putDouble(value).array()); - } - - private record SectionData( - int kind, - byte[] data) { + return PbxRegistryWriter.write(this); } public sealed interface ConstantPoolEntry permits NullConstant, Int64Constant, Float64Constant, BooleanConstant, StringConstant, Int32Constant { @@ -271,12 +105,23 @@ public record BytecodeModule( } public record FunctionMeta( + int codeOffset, + int codeLen, + int paramSlots, + int localSlots, + int returnSlots, + int maxStackSlots, + p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) { + public FunctionMeta( int codeOffset, int codeLen, int paramSlots, int localSlots, int returnSlots, int maxStackSlots) { + this(codeOffset, codeLen, paramSlots, localSlots, returnSlots, maxStackSlots, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null); + } + } public record SourceSpan( diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeOpcodeLayout.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeOpcodeLayout.java index b0cf78ae..55afb0a1 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeOpcodeLayout.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodeOpcodeLayout.java @@ -1,58 +1,17 @@ package p.studio.compiler.backend.bytecode; -import java.util.Map; - final class BytecodeOpcodeLayout { - private static final Map SIZE_BY_OPCODE = Map.ofEntries( - Map.entry(0x01, 2), - Map.entry(0x02, 6), - Map.entry(0x03, 6), - Map.entry(0x04, 6), - Map.entry(0x10, 6), - Map.entry(0x11, 2), - Map.entry(0x14, 10), - Map.entry(0x15, 10), - Map.entry(0x16, 3), - Map.entry(0x17, 6), - Map.entry(0x18, 6), - Map.entry(0x20, 2), - Map.entry(0x21, 2), - Map.entry(0x22, 2), - Map.entry(0x23, 2), - Map.entry(0x24, 2), - Map.entry(0x30, 2), - Map.entry(0x31, 2), - Map.entry(0x32, 2), - Map.entry(0x33, 2), - Map.entry(0x34, 2), - Map.entry(0x35, 2), - Map.entry(0x36, 2), - Map.entry(0x3C, 2), - Map.entry(0x3D, 2), - Map.entry(0x3E, 2), - Map.entry(0x40, 6), - Map.entry(0x41, 6), - Map.entry(0x42, 6), - Map.entry(0x43, 6), - Map.entry(0x50, 6), - Map.entry(0x51, 2), - Map.entry(0x52, 10), - Map.entry(0x54, 10), - Map.entry(0x56, 6), - Map.entry(0x70, 6), - Map.entry(0x71, 6), - Map.entry(0x72, 6)); - - private BytecodeOpcodeLayout() { - } - - static int sizeForOpcode(final int opcode) { - final var size = SIZE_BY_OPCODE.get(opcode); - if (size == null) { - throw new BytecodeMarshalingException( - BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE, - "unknown opcode 0x%04X".formatted(opcode)); - } - return size; + private BytecodeOpcodeLayout() {} + static int sizeForOpcode(int opcode) { + return switch(opcode) { + case 0x00,0x01,0x05,0x11,0x12,0x13,0x20,0x21,0x22,0x23,0x24, + 0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0x3a,0x3b,0x3c,0x3d,0x3e,0x51,0x80 -> 2; + case 0x16 -> 3; + case 0x02,0x03,0x04,0x10,0x17,0x18,0x40,0x41,0x42,0x43,0x50,0x53,0x57,0x70,0x71,0x72, + 0x90,0x91,0x92,0xa0,0xa3,0xa4,0xa5,0xa6 -> 6; + case 0x14,0x15,0x52,0x58,0x93,0x94,0x95,0x96,0xa1,0xa2 -> 10; + default -> throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE, + "unknown or retired opcode 0x%04X".formatted(opcode)); + }; } } diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodePreloadVerifierService.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodePreloadVerifierService.java index 97180080..56e26a5e 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodePreloadVerifierService.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/bytecode/BytecodePreloadVerifierService.java @@ -1,70 +1,236 @@ package p.studio.compiler.backend.bytecode; -import java.nio.ByteBuffer; -import java.nio.ByteOrder; -import java.util.Set; +import p.studio.compiler.backend.registry.LinkedRegistry; +import p.studio.compiler.models.registry.IRRegistry.*; +import java.nio.*; +import java.util.*; +/** Producer verification. Native binding and final admission remain loader responsibilities. */ public class BytecodePreloadVerifierService { - private static final Set TERMINATORS = Set.of(0x01, 0x02, 0x51); - - public void verify(final BytecodeModule module) { - final var input = module == null ? BytecodeModule.empty() : module; - for (final var function : input.functions()) { - verifyFunction(input.code(), function); + public void verify(BytecodeModule module) { + try { verifyRegistry(module); } + catch (IllegalArgumentException | ArithmeticException error) { + throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_REGISTRY_INVALID, error.getMessage()); } } - - private void verifyFunction(final byte[] code, final BytecodeModule.FunctionMeta function) { - final var start = function.codeOffset(); - final var end = start + function.codeLen(); - if (end <= start) { - return; - } - - var pc = start; - var lastOpcode = -1; - while (pc < end) { - if (pc + 2 > end) { - throw new BytecodeMarshalingException( - BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION, - "truncated opcode in function at pc=" + pc); + private void verifyRegistry(BytecodeModule module) { + Objects.requireNonNull(module); + // Validate the complete declarative schema without accepting an old slot-only image. + module.serialize(); + var registry=LinkedRegistry.link(module.registry()); + var used=new boolean[module.syscalls().size()]; + for(int i=0;i(); + for(int pc=start;pc=6?i32(code,pc+2):0,b=size==10?i32(code,pc+6):0; + var instruction=new Instruction(pc,op,a,b,pc+size);instructions.put(pc,instruction); + switch(op) { + case 0x10 -> check(a>=0&&a check(code[pc+2]==0||code[pc+2]==1,"Boolean immediate"); + case 0x15 -> check(Double.isFinite(Double.longBitsToDouble(ByteBuffer.wrap(code,pc+2,8).order(ByteOrder.LITTLE_ENDIAN).getLong())),"nonfinite PushF64"); + case 0x18,0x40,0x41 -> check(a>=0,"negative count/index"); + case 0x42,0x43 -> check(a>=0&&(long)a {check(a>=0&&a {check(a>=0&&a check(m.intrinsicSignatures().stream().anyMatch(n->n.id()==a&&n.argumentSlots()>=0&&n.resultSlots()>=0),"intrinsic signature required"); + case 0x52 -> {check(a>=0&&a fn(r,a); + case 0x57 -> check(a>=0&&a method(r,a,b); + case 0x90,0x91,0x92 -> type(r,a); + case 0x93,0x94,0x95,0x96 -> {var src=type(r,a).body();var dst=type(r,b).body();check(src instanceof Struct||src instanceof Array||src instanceof Trait,"reference source category");check(op==0x94||op==0x95?dst instanceof Trait:dst instanceof Struct||dst instanceof Array,"reference target category");} + case 0xa0 -> check(type(r,a).body() instanceof Struct,"Struct category"); + case 0xa1,0xa2 -> field(r,a,b); + case 0xa3,0xa4,0xa5,0xa6 -> array(r,a); + default -> { } } - final var opcode = readU16(code, pc); - if (opcode == 0x70) { - throw new BytecodeMarshalingException( - BytecodeMarshalingErrorCode.MARSHAL_LINKAGE_RAW_SYSCALL_IN_PRELOAD, - "raw syscall is forbidden in pre-load artifact"); + pc+=size; + } + for(var instruction:instructions.values())if(instruction.opcode()>=2&&instruction.opcode()<=4) { + // Jump operands are absolute offsets within the current function. + long target=(long)start+instruction.first(); + check(target>=start&&target<=end&&(target==end||instructions.containsKey((int)target)),"jump outside function or into immediate"); + } + analyze(m,r,index,instructions,end); + } + private record Fact(String carrier,Integer type,int component,int width,Long origin,Long integer) { + Fact(String carrier,Integer type,int component,int width) {this(carrier,type,component,width,null,null);} + static final Fact UNKNOWN=new Fact("?",null,0,1),NULL=new Fact("null",null,0,1); + } + private static final class State { + Fact[] locals;ArrayList stack; + State(Fact[] locals,ArrayList stack){this.locals=locals;this.stack=stack;} + State copy(){return new State(locals.clone(),new ArrayList<>(stack));} + } + private List facts(LinkedRegistry r,TypeRef ref) { + int id=r.id(ref).index(),width=r.width(ref);var body=r.type(ref).body();String carrier; + if(body instanceof Builtin b)carrier=switch(b.id()){case 1,3,0x11->"i64";case 2,0x12->"f64";case 0x10->"i32";case 0x13->"bool";case 0x14->"string";case 0x15->"?";default->"ref";}; + else carrier=body instanceof Trait?"?":"ref"; + var result=new ArrayList();for(int i=0;i parameters(LinkedRegistry r,Signature signature){var result=new ArrayList();for(var p:signature.parameters())result.addAll(facts(r,p.type()));return result;} + private List results(LinkedRegistry r,Signature signature){var result=new ArrayList();for(var t:signature.results())result.addAll(facts(r,t));return result;} + private void analyze(BytecodeModule m,LinkedRegistry r,int index,TreeMap code,int end) { + var f=m.functions().get(index);var locals=new Fact[f.paramSlots()+f.localSlots()];Arrays.fill(locals,Fact.NULL); + var parameters=parameters(r,f.metadata().signature());for(int i=0;i();var queue=new ArrayDeque();states.put(f.codeOffset(),new State(locals,new ArrayList<>()));queue.add(f.codeOffset()); + long work=0; + while(!queue.isEmpty()) { + int pc=queue.remove();var state=states.get(pc).copy();var instruction=code.get(pc); + check((work+=1L+state.locals.length+state.stack.size())<=268_435_456L,"verification work limit"); + transfer(m,r,index,instruction,state,false); + for(int successor:successors(instruction,f.codeOffset(),end)) { + if(successor==end) {check(state.stack.size()==f.returnSlots(),"function-end result height");continue;} + var old=states.get(successor);if(old==null){check((long)(states.size()+1)*(locals.length+f.maxStackSlots())*24<=128L*1024*1024,"verification scratch limit");states.put(successor,state.copy());queue.add(successor);} + else {check(old.stack.size()==state.stack.size(),"join height mismatch");boolean changed=false; + for(int i=0;i end) { - throw new BytecodeMarshalingException( - BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION, - "truncated instruction in function at pc=" + pc); + } + // Diagnose proven type errors after widening has converged, not in traversal order. + for(var entry:states.entrySet()) { + var state=entry.getValue().copy();var instruction=code.get(entry.getKey());transfer(m,r,index,instruction,state,true); + for(int successor:successors(instruction,f.codeOffset(),end))if(successor==end)compatible(state.stack,results(r,f.metadata().signature()),true); + } + } + private List successors(Instruction i,int start,int end) { + if(i.opcode()==0x01||i.opcode()==0x05||i.opcode()==0x51)return List.of(); + if(i.opcode()==0x02)return List.of(start+i.first()); + if(i.opcode()==0x03||i.opcode()==0x04) { + if(i.next()==end)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION,"conditional fallthrough at function end"); + return List.of(i.next(),start+i.first()); + } + if(i.next()>=end)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION,"reachable fallthrough at function end");return List.of(i.next()); + } + private void transfer(BytecodeModule m,LinkedRegistry r,int index,Instruction i,State s,boolean typed) { + int a=i.first(),b=i.second(),op=i.opcode();var f=m.functions().get(index); + switch(op) { + case 0x00,0x01,0x02,0x05,0x80 -> { } + case 0x03,0x04 -> consume(s,List.of(carrier("bool")),typed); + case 0x10 -> {var c=m.constPool().get(a);s.stack.add(switch(c){case BytecodeModule.NullConstant n->Fact.NULL;case BytecodeModule.Int32Constant n->carrier("i32");case BytecodeModule.Int64Constant n->integer(n.value());case BytecodeModule.Float64Constant n->carrier("f64");case BytecodeModule.BooleanConstant n->carrier("bool");case BytecodeModule.StringConstant n->carrier("string");});} + case 0x11 -> pop(s,1); + case 0x12 -> {var value=pop(s,1);s.stack.addAll(value);s.stack.addAll(value);} + case 0x13 -> {var values=pop(s,2);s.stack.add(values.get(1));s.stack.add(values.get(0));} + case 0x14 -> s.stack.add(integer(Integer.toUnsignedLong(a)|((long)b<<32)));case 0x15->s.stack.add(carrier("f64"));case 0x16->s.stack.add(carrier("bool"));case 0x17->s.stack.add(carrier("i32"));case 0x18->pop(s,a); + case 0x40 -> s.stack.add(Fact.UNKNOWN);case 0x41->pop(s,1); + case 0x42 -> s.stack.add(s.locals[a]);case 0x43 -> {var value=pop(s,1).getFirst();s.locals[a]=value;} + case 0x50 -> {var metadata=m.functions().get(a).metadata();consume(s,parameters(r,metadata.signature()),typed);produce(s,results(r,metadata.signature()),i.pc());} + case 0x51 -> {check(s.stack.size()==f.returnSlots(),"RET result height");compatible(s.stack,results(r,f.metadata().signature()),typed);} + case 0x52 -> {var target=m.functions().get(a).metadata();var expected=new ArrayList();for(var t:target.captures())expected.addAll(facts(r,t));consume(s,expected,typed);produce(s,facts(r,type(r,b).identity()),i.pc());} + case 0x53 -> {consume(s,facts(r,type(r,a).identity()),typed);var signature=fn(r,a).signature();consume(s,parameters(r,signature),typed);produce(s,results(r,signature),i.pc());} + case 0x57 -> produce(s,facts(r,f.metadata().captures().get(a)),i.pc()); + case 0x58 -> {consume(s,facts(r,type(r,a).identity()),typed);var signature=method(r,a,b).signature();consume(s,parameters(r,signature),typed);produce(s,results(r,signature),i.pc());} + case 0x71 -> {var decl=m.syscalls().get(a);pop(s,decl.argSlots());for(int n=0;n { + var signature=m.intrinsicSignatures().stream().filter(n->n.id()==a).findFirst().orElseThrow(()->new IllegalArgumentException("intrinsic signature required")); + pop(s,signature.argumentSlots());for(int n=0;n { + consume(s,facts(r,type(r,a).identity()),typed); + s.stack.add(new Fact("i64|null",-1,0,2,(long)i.pc(),null)); + s.stack.add(new Fact("?",-1,1,2,(long)i.pc(),null)); + } + case 0x91 -> {dynamic(pop(s,2),typed);produce(s,facts(r,type(r,a).identity()),i.pc());} + case 0x92 -> {dynamic(pop(s,2),typed);s.stack.add(carrier("bool"));} + case 0x93,0x94,0x95,0x96 -> {consume(s,facts(r,type(r,a).identity()),typed);if(op==0x93||op==0x94)s.stack.add(carrier("bool"));else produce(s,facts(r,type(r,b).identity()),i.pc());} + case 0xa0 -> produce(s,facts(r,type(r,a).identity()),i.pc()); + case 0xa1 -> {consume(s,facts(r,type(r,a).identity()),typed);produce(s,facts(r,field(r,a,b).type()),i.pc());} + case 0xa2 -> {consume(s,facts(r,field(r,a,b).type()),typed);consume(s,facts(r,type(r,a).identity()),typed);} + case 0xa3 -> {consume(s,List.of(carrier("i64")),typed);produce(s,facts(r,type(r,a).identity()),i.pc());} + case 0xa4 -> {consume(s,facts(r,type(r,a).identity()),typed);s.stack.add(carrier("i64"));} + case 0xa5 -> {consume(s,List.of(carrier("i64")),typed);consume(s,facts(r,type(r,a).identity()),typed);produce(s,facts(r,array(r,a).element()),i.pc());} + case 0xa6 -> {consume(s,facts(r,array(r,a).element()),typed);consume(s,List.of(carrier("i64")),typed);consume(s,facts(r,type(r,a).identity()),typed);} + case 0x34,0x35 -> {consume(s,List.of(carrier("bool"),carrier("bool")),typed);s.stack.add(carrier("bool"));} + case 0x36 -> {consume(s,List.of(carrier("bool")),typed);s.stack.add(carrier("bool"));} + case 0x3e -> { + var value=pop(s,1).getFirst(); + if(typed)check(value.carrier().equals("?")||numeric(value.carrier()),"numeric operand required"); + s.stack.add(carrier(value.carrier())); + } + default -> { + var operands=pop(s,2);String left=operands.get(0).carrier(),right=operands.get(1).carrier(); + boolean comparison=op>=0x30&&op<=0x33||op==0x3c||op==0x3d; + boolean unknown=left.equals("?")||right.equals("?")||left.contains("|")||right.contains("|"); + boolean stringAdd=op==0x20&&(left.equals("string")||right.equals("string")); + if(typed&&!unknown&&op!=0x30&&op!=0x31) { + if(comparison)check(numeric(left)&&numeric(right)||left.equals(right)&&(left.equals("string")||left.equals("bool")),"comparison carrier mismatch"); + else if(!stringAdd)check(numeric(left)&&numeric(right)&&(op<0x37||!left.equals("f64")&&!right.equals("f64")),"numeric operand mismatch"); + } + String result="?"; + if(stringAdd)result="string"; + else if(numeric(left)&&numeric(right))result=left.equals("f64")||right.equals("f64")?"f64":left.equals("i64")||right.equals("i64")?"i64":"i32"; + s.stack.add(carrier(comparison?"bool":result)); } - lastOpcode = opcode; - pc += size; } - - if (!TERMINATORS.contains(lastOpcode)) { - throw new BytecodeMarshalingException( - BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION, - "function ends without terminator opcode"); + check(s.stack.size()<=f.maxStackSlots(),"maximum temporary slots exceeded (zero is literal)"); + } + private static boolean numeric(String carrier){return carrier.equals("i32")||carrier.equals("i64")||carrier.equals("f64")||carrier.equals("color");} + private static Fact carrier(String name){return new Fact(name,null,0,1);} + private static List pop(State s,int n){check(n>=0&&n<=s.stack.size(),"temporary stack underflow");var result=new ArrayList<>(s.stack.subList(s.stack.size()-n,s.stack.size()));s.stack.subList(s.stack.size()-n,s.stack.size()).clear();return result;} + private static void consume(State s,List expected,boolean typed){compatible(pop(s,expected.size()),expected,typed);} + private static Fact stamp(Fact fact,long origin) { + return new Fact(fact.carrier(),fact.type(),fact.component(),fact.width(),origin,fact.integer()); + } + private static Fact integer(long value){return new Fact("i64",null,0,1,null,value);} + private static void produce(State state,List values,int pc) { + for(int n=0;n1;case "i64","color"->2;case "f64"->4;case "bool"->8;case "string"->16|64;case "ref"->32|64;case "null"->64;default->127; + }; + return result; + } + private static void dynamic(List values,boolean typed) { + if(!typed)return; + check((mask(values.get(0).carrier())&(2|64))!=0,"Dynamic tag carrier"); + if(values.get(0).equals(Fact.NULL))check((mask(values.get(1).carrier())&64)!=0,"invalid null Dynamic pair"); + } + private static void compatible(List actual,List expected,boolean typed) { + if(!typed)return; + check(actual.size()==expected.size(),"group height"); + for(int i=0;i=0&&a.integer()<=0xffffffffL,"color range"); + if(e.component()==0&&e.width()==2&&e.carrier().contains("|" )&&a.equals(Fact.NULL)) + check((mask(actual.get(i+1).carrier())&64)!=0,"invalid null group pair"); } } - - private int sizeForOpcode( - final int opcode, - final int pc) { - try { - return BytecodeOpcodeLayout.sizeForOpcode(opcode); - } catch (BytecodeMarshalingException ignored) { - throw new BytecodeMarshalingException( - BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE, - "unknown opcode 0x%04X at pc=%d".formatted(opcode, pc)); - } - } - - private int readU16(final byte[] code, final int offset) { - return ByteBuffer.wrap(code, offset, 2).order(ByteOrder.LITTLE_ENDIAN).getShort() & 0xFFFF; - } + private static TypeDeclaration type(LinkedRegistry r,int id){check(id>=0&&id=0&&member=0&&member roles = EnumSet.noneOf(Role.class); + private long nested, names; + private PbxRegistryWriter(BytecodeModule module) { + this.module=module; + this.registry=LinkedRegistry.link(module.registry()); + } + static byte[] write(BytecodeModule module) { + try { return new PbxRegistryWriter(module).write(); } + catch (IllegalArgumentException | ArithmeticException error) { + throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_REGISTRY_INVALID, error.getMessage()); + } + } + private byte[] write() { + require(module.version()==0,"PBX version must be zero"); + require(module.code().length<=32*1024*1024,"Code limit"); + require(module.functions().size()<=65_536,"function limit"); + require(module.registry().implementations().size()<=262_144,"implementation limit"); + var sections=new TreeMap(); + sections.put(0,constants()); sections.put(1,functions()); sections.put(2,module.code()); + if(module.debugInfo()!=null) sections.put(3,debug()); + sections.put(4,exports()); sections.put(5,syscalls()); sections.put(6,types()); + sections.put(7,implementations()); sections.put(8,captures()); + require(nested<=1_048_576,"nested record limit"); require(names<=4*1024*1024,"Registry name limit"); + long total=32+12L*sections.size(); for(var bytes:sections.values()) total+=bytes.length; + require(total<=64*1024*1024,"PBX file limit"); + var out=new Bytes();out.raw(new byte[]{'P','B','X',0});out.u16(0);out.u8(0);out.u8(0);out.u32(sections.size());out.raw(new byte[20]); + int offset=32+12*sections.size(); + for(var section:sections.entrySet()) {out.u32(section.getKey());out.u32(offset);out.u32(section.getValue().length);offset+=section.getValue().length;} + sections.values().forEach(out::raw);return out.bytes(); + } + private void optionalName(Bytes out,String name) { + names+=LinkedRegistry.nameBytes(name);out.u8(name==null?0:1);if(name!=null)out.text32(name); + } + private void parameters(Bytes out,ReadOnlyList values) { + nested+=values.size();out.u32(values.size());for(var p:values) {out.u32(registry.id(p.type()).index());optionalName(out,p.name());} + } + private void results(Bytes out,ReadOnlyList values) { + nested+=values.size();out.u32(values.size());for(var r:values)out.u32(registry.id(r).index()); + } + private void signature(Bytes out,Signature signature) { + require(registry.parameterSlots(signature)<=16_384,"signature parameter limit"); + require(registry.resultSlots(signature)<=1024,"signature result limit"); + parameters(out,signature.parameters());results(out,signature.results()); + } + private byte[] types() { + var out=new Bytes();out.u32(registry.types().size()); + for(var t:registry.types()) { + int tag=switch(t.body()){case Builtin b->0;case Struct s->1;case Array a->2;case Trait t1->3;case Fn f->4;}; + out.u8(tag);optionalName(out,t.name()); + switch(t.body()) { + case Builtin b -> out.u32(b.id()); + case Struct s -> parameters(out,s.fields()); + case Array a -> out.u32(registry.id(a.element()).index()); + case Fn f -> signature(out,f.signature()); + case Trait t1 -> {nested+=t1.methods().size();out.u32(t1.methods().size());for(var m:t1.methods()){optionalName(out,m.name());signature(out,m.signature());}} + } + } + return out.bytes(); + } + private byte[] functions() { + var out=new Bytes();out.u32(module.functions().size());long previousEnd=0; + for(int i=0;i=0&&f.maxStackSlots()>=0&&(long)p+f.localSlots()+f.maxStackSlots()<=16_384&&r<=1024,"frame/result limit"); + long end=(long)f.codeOffset()+f.codeLen(); + require(f.codeOffset()>=previousEnd&&f.codeLen()>0&&end<=module.code().length,"invalid function Code range");previousEnd=end; + if(i==0)require(m.captures().isEmpty(),"boot function must have no captures"); + if(m.role()!=Role.NONE) { + require(roles.add(m.role()),"duplicate lifecycle role");require(i!=0&&m.captures().isEmpty(),"invalid lifecycle target"); + if(m.role()==Role.MEMCARD_BOOT) require(m.signature().parameters().size()==1&&m.signature().results().size()==1 + &&isBuiltin(m.signature().parameters().getFirst().type(),0x11)&&isBuiltin(m.signature().results().getFirst(),0x10),"MemcardBoot signature"); + else require(p==0&&r==0,"pause/resume signature"); + } + out.u8(m.role().wire());optionalName(out,m.name());signature(out,m.signature()); + out.u32(f.codeOffset());out.u32(f.codeLen());out.u32(f.localSlots());out.u32(f.maxStackSlots()); + } + return out.bytes(); + } + private boolean isBuiltin(TypeRef type,int expected) {return registry.type(type).body() instanceof Builtin b&&b.id()==expected;} + private byte[] captures(){var out=new Bytes();out.u32(module.functions().size());for(var f:module.functions())results(out,f.metadata().captures());return out.bytes();} + private byte[] implementations() { + var out=new Bytes();var declarations=new TreeMap(); + for(var impl:module.registry().implementations()) { + var old=declarations.putIfAbsent(impl.declarationKey(),impl);require(old==null||old.equals(impl),"conflicting implementation declaration"); + } + out.u32(declarations.size());var pairs=new HashSet>(); + for(var impl:declarations.values()) { + require(registry.type(impl.owner()).body() instanceof Struct,"implementation owner must be Struct"); + require(registry.type(impl.trait()).body() instanceof Trait,"implementation trait category"); + var trait=(Trait)registry.type(impl.trait()).body();int owner=registry.id(impl.owner()).index(),traitId=registry.id(impl.trait()).index(); + require(pairs.add(List.of(owner,traitId)),"duplicate implementation pair"); + require(trait.methods().size()==impl.methods().size(),"implementation method count"); + out.u32(owner);out.u32(traitId);out.u32(impl.methods().size());nested+=impl.methods().size(); + for(int i=0;iout.u8(0); + case BytecodeModule.Int64Constant n->{out.u8(1);out.i64(n.value());} + case BytecodeModule.Float64Constant n->{require(Double.isFinite(n.value()),"nonfinite Float constant");out.u8(2);out.i64(Double.doubleToRawLongBits(n.value()));} + case BytecodeModule.BooleanConstant n->{out.u8(3);out.u8(n.value()?1:0);} + case BytecodeModule.StringConstant n->{var bytes=Bytes.utf8(n.value());require(bytes.length<=4*1024*1024,"individual string limit");strings+=bytes.length;require(strings<=64*1024*1024,"constant string limit");out.u8(4);out.u32(bytes.length);out.raw(bytes);} + case BytecodeModule.Int32Constant n->{out.u8(5);out.i32(n.value());} + }return out.bytes(); + } + private byte[] exports(){var out=new Bytes();out.u32(module.exports().size());for(var e:module.exports()){require(e.funcIdx()>=0&&e.funcIdx()=0&&p.pc()=0&&p.span().end()>=p.span().start(),"debug span range");out.u32(p.pc());out.i32(p.span().fileId());out.u32(p.span().start());out.u32(p.span().end());} + out.u32(d.functionNames().size());for(var n:d.functionNames()){require(n.funcIdx()>=0&&n.funcIdx()>();out.u32(module.syscalls().size()); + for(var s:module.syscalls()){require(identities.add(List.of(s.module(),s.name(),s.version())),"duplicate syscall declaration");if(Bytes.utf8(s.module()).length>65535)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG,"syscall module name exceeds u16"); + if(Bytes.utf8(s.name()).length>65535)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_NAME_TOO_LONG,"syscall name exceeds u16"); + out.text16(s.module());out.text16(s.name());out.u16(s.version());out.u16(s.argSlots());out.u16(s.retSlots());}return out.bytes();} + static void require(boolean condition,String message){if(!condition)throw new IllegalArgumentException(message);} + private static final class Bytes { + final ByteArrayOutputStream out=new ByteArrayOutputStream(); + void u8(int v){require(v>=0&&v<=255,"u8 range");out.write(v);} + void u16(int v){require(v>=0&&v<=65535,"u16 range");out.write(v);out.write(v>>>8);} + void u32(int v){require(v>=0,"u32 range/product limit");i32(v);} + void i32(int v){for(int i=0;i<4;i++)out.write(v>>>(8*i));} + void i64(long v){for(int i=0;i<8;i++)out.write((int)(v>>>(8*i)));} + void raw(byte[] v){out.writeBytes(v);} + void text32(String v){var b=utf8(v);u32(b.length);raw(b);} + void text16(String v){var b=utf8(v);u16(b.length);raw(b);} + byte[] bytes(){return out.toByteArray();} + static byte[] utf8(String text){try{var b=StandardCharsets.UTF_8.newEncoder().encode(CharBuffer.wrap(text));var bytes=new byte[b.remaining()];b.get(bytes);return bytes;}catch(CharacterCodingException e){throw new IllegalArgumentException("invalid UTF-8 source",e);}} + } +} diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMFunction.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMFunction.java index 3f0a7397..56f13d05 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMFunction.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMFunction.java @@ -5,12 +5,23 @@ import p.studio.utilities.structures.ReadOnlyList; import java.util.Objects; public record IRVMFunction( + String name, + int paramSlots, + int localSlots, + int returnSlots, + int maxStackSlots, + ReadOnlyList instructions, + p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) { + public IRVMFunction( String name, int paramSlots, int localSlots, int returnSlots, int maxStackSlots, ReadOnlyList instructions) { + this(name, paramSlots, localSlots, returnSlots, maxStackSlots, instructions, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null); + } + public IRVMFunction { Objects.requireNonNull(name, "name"); instructions = instructions == null ? ReadOnlyList.empty() : instructions; diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMInstruction.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMInstruction.java index 4d0a136e..8bab934e 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMInstruction.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMInstruction.java @@ -3,10 +3,23 @@ package p.studio.compiler.backend.irvm; import java.util.Objects; public record IRVMInstruction( + IRVMOp op, + Integer immediate, + Integer secondImmediate) { + public IRVMInstruction( IRVMOp op, Integer immediate) { + this(op, immediate, null); + } + public IRVMInstruction { Objects.requireNonNull(op, "op"); + if (op.name().startsWith("REGISTRY_") && ((op.immediateSize() == 8) != (secondImmediate != null))) { + throw new IllegalArgumentException("registry operand count differs from opcode layout"); + } + if (!op.name().startsWith("REGISTRY_") && secondImmediate != null) { + throw new IllegalArgumentException("unexpected second immediate"); + } if (op.immediateSize() > 0 && immediate == null) { throw new IllegalArgumentException("immediate is required for opcode: " + op.name()); } diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMModule.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMModule.java index 02b2da79..9e46f00d 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMModule.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMModule.java @@ -3,8 +3,15 @@ package p.studio.compiler.backend.irvm; import p.studio.utilities.structures.ReadOnlyList; public record IRVMModule( + String vmProfile, + ReadOnlyList functions, + p.studio.compiler.models.registry.IRRegistry registry) { + public IRVMModule( String vmProfile, ReadOnlyList functions) { + this(vmProfile, functions, p.studio.compiler.models.registry.IRRegistry.empty()); + } + public IRVMModule { vmProfile = vmProfile == null || vmProfile.isBlank() ? "core-v1" : vmProfile; functions = functions == null ? ReadOnlyList.empty() : functions; diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMProfileFeatureGate.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMProfileFeatureGate.java index e4c17499..15a67c6f 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMProfileFeatureGate.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMProfileFeatureGate.java @@ -91,6 +91,7 @@ public final class IRVMProfileFeatureGate { final String profile, final IRVMOp op) { final var allowed = allowedByProfile.get(normalizeProfile(profile)); - return allowed != null && allowed.contains(op); + return allowed != null && (allowed.contains(op) || + op.name().startsWith("REGISTRY_") && java.util.Arrays.stream(p.studio.compiler.models.registry.IRRegistryInstruction.Kind.values()).anyMatch(k -> k.opcode() == op.opcode())); } } diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMProgram.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMProgram.java index e0d59147..df68a9fe 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMProgram.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/IRVMProgram.java @@ -54,13 +54,17 @@ public record IRVMProgram( private void validateCoherence( final IRVMModule module, final BytecodeEmitter.EmissionPlan emissionPlan) { + if (!module.registry().equals(emissionPlan.registry())) { + throw new IllegalArgumentException("irvm/emission plan registry mismatch"); + } if (module.functions().size() != emissionPlan.functions().size()) { throw new IllegalArgumentException("irvm module/function-plan size mismatch"); } for (var fnIndex = 0; fnIndex < module.functions().size(); fnIndex++) { final var irvmFunction = module.functions().get(fnIndex); final var planFunction = emissionPlan.functions().get(fnIndex); - if (!irvmFunction.name().equals(planFunction.name()) + if (!java.util.Objects.equals(irvmFunction.metadata(), planFunction.metadata()) + || !irvmFunction.name().equals(planFunction.name()) || irvmFunction.paramSlots() != planFunction.paramSlots() || irvmFunction.localSlots() != planFunction.localSlots() || irvmFunction.returnSlots() != planFunction.returnSlots() @@ -84,6 +88,7 @@ public record IRVMProgram( private boolean coherentPair( final IRVMInstruction instruction, final BytecodeEmitter.Operation operation) { + if (operation.kind() == BytecodeEmitter.OperationKind.REGISTRY) return instruction.equals(operation.registryInstruction()); if (instruction.op().internal()) { return true; } @@ -137,13 +142,13 @@ public record IRVMProgram( function.localSlots(), function.returnSlots(), function.maxStackSlots(), - ReadOnlyList.wrap(operations))); + ReadOnlyList.wrap(operations), function.metadata())); } return new BytecodeEmitter.EmissionPlan( 0, ReadOnlyList.empty(), ReadOnlyList.empty(), - ReadOnlyList.wrap(functions)); + ReadOnlyList.wrap(functions), module.registry()); } private boolean containsRawSyscall(final BytecodeEmitter.EmissionPlan plan) { @@ -171,6 +176,7 @@ public record IRVMProgram( } private BytecodeEmitter.Operation deriveOperation(final IRVMInstruction instruction) { + if (instruction.op().name().startsWith("REGISTRY_")) return new BytecodeEmitter.Operation(BytecodeEmitter.OperationKind.REGISTRY, 0, null, null, null, null, instruction); final var immediate = instruction.immediate() == null ? 0 : instruction.immediate(); return switch (instruction.op().opcode()) { case 0x01 -> BytecodeEmitter.Operation.halt(); @@ -240,13 +246,14 @@ public record IRVMProgram( functionPlan.localSlots(), functionPlan.returnSlots(), functionPlan.maxStackSlots(), - ReadOnlyList.wrap(instructions))); + ReadOnlyList.wrap(instructions), functionPlan.metadata())); } - return new IRVMModule("core-v1", ReadOnlyList.wrap(functions)); + return new IRVMModule("core-v1", ReadOnlyList.wrap(functions), inputPlan.registry()); } private static IRVMInstruction operationToInstruction(final BytecodeEmitter.Operation operation) { return switch (operation.kind()) { + case REGISTRY -> operation.registryInstruction(); case HALT -> new IRVMInstruction(IRVMOp.HALT, null); case RET -> new IRVMInstruction(IRVMOp.RET, null); case PUSH_CONST -> new IRVMInstruction(IRVMOp.PUSH_CONST, operation.immediate()); diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/LowerToIRVMService.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/LowerToIRVMService.java index 73660ddf..67d65003 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/LowerToIRVMService.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/LowerToIRVMService.java @@ -41,10 +41,12 @@ public class LowerToIRVMService { } validateCommonLifecycleStructure(backend); + final var registry = p.studio.compiler.backend.registry.LinkedRegistry.link(backend.getRegistry()); final var ordered = orderFunctions(backend); final var funcIdByCallableId = new HashMap(); for (var i = 0; i < ordered.size(); i++) { final var fn = ordered.get(i); + if (fn.metadata() == null) throw new IllegalArgumentException("typed function metadata required: " + fn.callableName()); if (funcIdByCallableId.putIfAbsent(fn.callableId(), i) != null) { throw loweringError( fn, @@ -60,6 +62,9 @@ public class LowerToIRVMService { for (var i = 0; i < ordered.size(); i++) { final var fn = ordered.get(i); + if (fn.metadata() == null) throw new IllegalArgumentException("typed function metadata required: " + fn.callableName()); + final var parameterSlots = registry.parameterSlots(fn.metadata().signature()); + final var resultSlots = registry.resultSlots(fn.metadata().signature()); final var instructions = new ArrayList(fn.instructions().size()); final var operations = new ArrayList(fn.instructions().size()); final var labelToPc = new HashMap(); @@ -68,6 +73,12 @@ public class LowerToIRVMService { for (final var instr : fn.instructions()) { final var sourceSpan = toBytecodeSpan(fn.fileId().getId(), instr.span()); switch (instr.kind()) { + case REGISTRY -> { + var typed = p.studio.compiler.backend.registry.RegistryInstructionLowerer.lower(instr.registryInstruction(), registry, fn, ordered, funcIdByCallableId); + instructions.add(typed); + operations.add(new BytecodeEmitter.Operation(BytecodeEmitter.OperationKind.REGISTRY, 0, null, null, null, sourceSpan, typed)); + functionPc += typed.encodedSize(); + } case PUSH_I32 -> { final int value; try { @@ -271,7 +282,7 @@ public class LowerToIRVMService { } final var calleeFunction = ordered.get(calleeId); if (instr.expectedArgSlots() != null - && instr.expectedArgSlots() != calleeFunction.paramSlots()) { + && instr.expectedArgSlots() != registry.parameterSlots(calleeFunction.metadata().signature())) { throw loweringError( fn, instr, @@ -279,10 +290,10 @@ public class LowerToIRVMService { "call arg_slots mismatch for %s. expected=%d actual=%d".formatted( instr.calleeCallableName(), instr.expectedArgSlots(), - calleeFunction.paramSlots())); + registry.parameterSlots(calleeFunction.metadata().signature()))); } if (instr.expectedRetSlots() != null - && instr.expectedRetSlots() != calleeFunction.returnSlots()) { + && instr.expectedRetSlots() != registry.resultSlots(calleeFunction.metadata().signature())) { throw loweringError( fn, instr, @@ -290,7 +301,7 @@ public class LowerToIRVMService { "call ret_slots mismatch for %s. expected=%d actual=%d".formatted( instr.calleeCallableName(), instr.expectedRetSlots(), - calleeFunction.returnSlots())); + registry.resultSlots(calleeFunction.metadata().signature()))); } instructions.add(new IRVMInstruction(IRVMOp.CALL, calleeId)); operations.add(BytecodeEmitter.Operation.callFunc(calleeId, sourceSpan)); @@ -404,27 +415,38 @@ public class LowerToIRVMService { } loweredFunctions.add(new IRVMFunction( fn.callableName(), - fn.paramSlots(), + parameterSlots, fn.localSlots(), - fn.returnSlots(), + resultSlots, fn.maxStackSlots(), - ReadOnlyList.wrap(instructions))); + ReadOnlyList.wrap(instructions), fn.metadata())); emissionFunctions.add(new BytecodeEmitter.FunctionPlan( fn.callableName(), - fn.paramSlots(), + parameterSlots, fn.localSlots(), - fn.returnSlots(), + resultSlots, fn.maxStackSlots(), - ReadOnlyList.wrap(operations))); + ReadOnlyList.wrap(operations), fn.metadata())); } + final var imageImplementations = new ArrayList(); + for (var impl : backend.getRegistry().implementations()) { + var methods = new ArrayList(); + for (var method : impl.methods()) { + var target = funcIdByCallableId.get(method); + if (target == null) throw new IllegalArgumentException("undefined implementation function: " + method); + methods.add(new CallableId(target)); + } + imageImplementations.add(new p.studio.compiler.models.registry.IRRegistry.Implementation(impl.declarationKey(), impl.owner(), impl.trait(), ReadOnlyList.wrap(methods))); + } + var imageRegistry = new p.studio.compiler.models.registry.IRRegistry(backend.getRegistry().types(), ReadOnlyList.wrap(imageImplementations)); final var program = new IRVMProgram( - new IRVMModule(vmProfile, ReadOnlyList.wrap(loweredFunctions)), + new IRVMModule(vmProfile, ReadOnlyList.wrap(loweredFunctions), imageRegistry), new BytecodeEmitter.EmissionPlan( 0, ReadOnlyList.wrap(constPool), ReadOnlyList.empty(), - ReadOnlyList.wrap(emissionFunctions))); + ReadOnlyList.wrap(emissionFunctions), imageRegistry)); validator.validate(program, false); return program; } diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/OptimizeIRVMService.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/OptimizeIRVMService.java index 39247208..3ec8e245 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/OptimizeIRVMService.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/irvm/OptimizeIRVMService.java @@ -124,12 +124,12 @@ public class OptimizeIRVMService { } return new IRVMProgram( - new IRVMModule(input.module().vmProfile(), ReadOnlyList.wrap(rewrittenFunctions)), + new IRVMModule(input.module().vmProfile(), ReadOnlyList.wrap(rewrittenFunctions), input.module().registry()), new BytecodeEmitter.EmissionPlan( input.emissionPlan().version(), input.emissionPlan().constPool(), input.emissionPlan().exports(), - ReadOnlyList.wrap(rewrittenPlans))); + ReadOnlyList.wrap(rewrittenPlans), input.emissionPlan().registry())); } private static boolean isJump(final IRVMOp op) { @@ -218,14 +218,14 @@ public class OptimizeIRVMService { function.localSlots(), function.returnSlots(), function.maxStackSlots(), - ReadOnlyList.wrap(newInstructions)); + ReadOnlyList.wrap(newInstructions), function.metadata()); final var rewrittenPlan = new BytecodeEmitter.FunctionPlan( functionPlan.name(), functionPlan.paramSlots(), functionPlan.localSlots(), functionPlan.returnSlots(), functionPlan.maxStackSlots(), - ReadOnlyList.wrap(newOperations)); + ReadOnlyList.wrap(newOperations), functionPlan.metadata()); return new FunctionRewrite(rewrittenFunction, rewrittenPlan, changed); } @@ -383,14 +383,14 @@ public class OptimizeIRVMService { function.localSlots(), function.returnSlots(), function.maxStackSlots(), - ReadOnlyList.wrap(instructions)), + ReadOnlyList.wrap(instructions), function.metadata()), new BytecodeEmitter.FunctionPlan( functionPlan.name(), functionPlan.paramSlots(), functionPlan.localSlots(), functionPlan.returnSlots(), functionPlan.maxStackSlots(), - ReadOnlyList.wrap(operations)), + ReadOnlyList.wrap(operations), functionPlan.metadata()), true); } diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/registry/LinkedRegistry.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/registry/LinkedRegistry.java new file mode 100644 index 00000000..442852d3 --- /dev/null +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/registry/LinkedRegistry.java @@ -0,0 +1,130 @@ +package p.studio.compiler.backend.registry; + +import p.studio.compiler.models.registry.IRRegistry; +import p.studio.compiler.models.registry.IRRegistry.*; +import p.studio.utilities.structures.ReadOnlyList; +import java.util.*; + +/** Immutable image type table and reference resolution. No language-specific type inference. */ +public final class LinkedRegistry { + public record TypeId(int index) { public TypeId { if (index < 0) throw new IllegalArgumentException("negative TypeId"); } } + public record FunctionId(int index) { public FunctionId { if (index < 0) throw new IllegalArgumentException("negative FunctionId"); } } + public record ImplementationId(int index) { public ImplementationId { if (index < 0) throw new IllegalArgumentException("negative ImplementationId"); } } + private final ReadOnlyList types; + private final Map references; + + private LinkedRegistry(List types, Map references) { + this.types = ReadOnlyList.wrap(types); + this.references = Map.copyOf(references); + } + public ReadOnlyList types() { return types; } + public TypeId id(TypeRef reference) { + final var id = references.get(reference); + if (id == null) throw new IllegalArgumentException("undefined type declaration: " + reference); + return id; + } + public TypeDeclaration type(TypeRef reference) { return types.get(id(reference).index()); } + public int width(TypeRef reference) { + return switch (type(reference).body()) { + case Builtin b -> switch (b.id()) { case 2, 0x15 -> 2; case 3 -> 3; default -> 1; }; + case Trait ignored -> 2; + default -> 1; + }; + } + public int parameterSlots(Signature signature) { + var slots = 0; + for (var p : signature.parameters()) slots = Math.addExact(slots, width(p.type())); + return slots; + } + public int resultSlots(Signature signature) { + var slots = 0; + for (var r : signature.results()) slots = Math.addExact(slots, width(r)); + return slots; + } + public boolean sameSignature(Signature a, Signature b) { + if (a.parameters().size() != b.parameters().size() || a.results().size() != b.results().size()) return false; + for (int i=0; i(); + for (var declaration : source.types()) { + final var old = declarations.putIfAbsent(declaration.identity().declarationKey(), declaration); + if (old != null && !old.equals(declaration)) throw new IllegalArgumentException("conflicting type declaration: " + declaration.identity()); + } + if (declarations.size() > 65_536) throw new IllegalArgumentException("type record limit exceeded"); + final var output = new ArrayList(); + final var ids = new HashMap(); + final var structuralKeys = new HashMap(); + final var remaining = new TreeMap(); + long names = 0, nested = 0; + for (var d : declarations.values()) { + names += nameBytes(d.name()); + for (var ref : references(d.body())) { + if (!declarations.containsKey(ref.declarationKey())) throw new IllegalArgumentException("undefined type: " + ref); + } + switch (d.body()) { + case Builtin b -> assign(d, List.of("builtin", b.id()), structuralKeys, output, ids); + case Struct s -> { assign(d, d.identity(), structuralKeys, output, ids); nested += s.fields().size(); for(var f:s.fields()) names += nameBytes(f.name()); } + case Trait t -> { + assign(d, d.identity(), structuralKeys, output, ids); nested += t.methods().size(); + for(var m:t.methods()) { names += nameBytes(m.name()); nested += m.signature().parameters().size()+m.signature().results().size(); for(var p:m.signature().parameters()) names += nameBytes(p.name()); } + } + case Array ignored -> remaining.put(d.identity().declarationKey(), d); + case Fn f -> { remaining.put(d.identity().declarationKey(), d); nested += f.signature().parameters().size()+f.signature().results().size(); for(var p:f.signature().parameters()) names += nameBytes(p.name()); } + } + } + if (names > 4L*1024*1024 || nested > 1_048_576) throw new IllegalArgumentException("registry name/nested-record limit exceeded"); + // Kahn traversal avoids recursive expansion and accepts nominal anchors. + final var pending = new HashMap(); + final var dependents = new HashMap>(); + final var ready = new PriorityQueue(Comparator.comparing(TypeRef::declarationKey)); + for(var d:remaining.values()) { + int count=0; + for(var r:references(d.body())) if(!ids.containsKey(r)) { count++; dependents.computeIfAbsent(r, ignored -> new ArrayList<>()).add(d.identity()); } + pending.put(d.identity(),count); + if(count==0) ready.add(d.identity()); + } + while(!ready.isEmpty()) { + var reference=ready.remove(); var d=remaining.get(reference.declarationKey()); + Object key; + if(d.body() instanceof Array a) key=List.of("array",ids.get(a.element())); + else { + var signature=((Fn)d.body()).signature(); + key=List.of("fn",signature.parameters().stream().map(p->ids.get(p.type())).toList(),signature.results().stream().map(ids::get).toList()); + } + assign(d,key,structuralKeys,output,ids); + for(var dependent:dependents.getOrDefault(reference,List.of())) if(pending.compute(dependent,(k,v)->v-1)==0) ready.add(dependent); + } + if(ids.size()!=declarations.size()) throw new IllegalArgumentException("structural Array/Fn cycle"); + return new LinkedRegistry(output,ids); + } + private static void assign(TypeDeclaration d,Object key,Map keys,List output,Map ids) { + var id=keys.get(key); + if(id==null) { id=new TypeId(output.size()); output.add(d); keys.put(key,id); } + ids.put(d.identity(),id); + } + private static List references(TypeBody body) { + final var result=new ArrayList(); + switch(body) { + case Struct s -> s.fields().forEach(f->result.add(f.type())); + case Array a -> result.add(a.element()); + case Fn f -> signatureReferences(f.signature(),result); + case Trait t -> t.methods().forEach(m->signatureReferences(m.signature(),result)); + case Builtin ignored -> { } + } + return result; + } + private static void signatureReferences(Signature signature,List result) { + signature.parameters().forEach(p->result.add(p.type())); result.addAll(signature.results().asList()); + } + public static int nameBytes(String name) { + if(name==null) return 0; + try { + int size=java.nio.charset.StandardCharsets.UTF_8.newEncoder().encode(java.nio.CharBuffer.wrap(name)).remaining(); + if(size>4096) throw new IllegalArgumentException("name exceeds 4096 UTF-8 bytes"); + return size; + } catch(java.nio.charset.CharacterCodingException e) { throw new IllegalArgumentException("invalid UTF-8 source string",e); } + } +} diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/registry/RegistryInstructionLowerer.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/registry/RegistryInstructionLowerer.java new file mode 100644 index 00000000..7d6da6ea --- /dev/null +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/backend/registry/RegistryInstructionLowerer.java @@ -0,0 +1,62 @@ +package p.studio.compiler.backend.registry; + +import p.studio.compiler.models.IRBackendExecutableFunction; +import p.studio.compiler.models.registry.IRRegistry.*; +import p.studio.compiler.models.registry.IRRegistryInstruction; +import p.studio.compiler.backend.irvm.*; +import p.studio.compiler.source.identifiers.CallableId; +import p.studio.utilities.structures.ReadOnlyList; +import java.util.Map; + +public final class RegistryInstructionLowerer { + private RegistryInstructionLowerer() {} + public static IRVMInstruction lower(IRRegistryInstruction instruction, LinkedRegistry registry, + IRBackendExecutableFunction current, ReadOnlyList functions, Map functionIds) { + int first=instruction.type()==null?instruction.member():registry.id(instruction.type()).index(); + Integer second=null;int pops=0,pushes=0; + var body=instruction.type()==null?null:registry.type(instruction.type()).body(); + switch(instruction.kind()) { + case MAKE_CLOSURE -> { + if(!(body instanceof Fn type))throw new IllegalArgumentException("closure type must be Fn"); + var target=functionIds.get(instruction.function());if(target==null)throw new IllegalArgumentException("undefined closure function"); + var metadata=functions.get(target).metadata(); + if(metadata==null||!registry.sameSignature(type.signature(),metadata.signature()))throw new IllegalArgumentException("closure signature mismatch"); + for(var capture:metadata.captures())pops=Math.addExact(pops,registry.width(capture)); + second=first;first=target;pushes=1; + } + case CALL_CLOSURE -> { + if(!(body instanceof Fn type))throw new IllegalArgumentException("call type must be Fn"); + pops=registry.parameterSlots(type.signature())+1;pushes=registry.resultSlots(type.signature()); + } + case GET_CAPTURE -> {if(instruction.member()>=current.metadata().captures().size())throw new IllegalArgumentException("capture index");pushes=registry.width(current.metadata().captures().get(instruction.member()));} + case CALL_TRAIT -> { + if(!(body instanceof Trait trait)||instruction.member()>=trait.methods().size())throw new IllegalArgumentException("trait/method operand"); + var signature=trait.methods().get(instruction.member()).signature();second=instruction.member();pops=registry.parameterSlots(signature)+2;pushes=registry.resultSlots(signature); + } + case DYN_PACK -> {pops=registry.width(instruction.type());pushes=2;} + case DYN_UNPACK -> {pops=2;pushes=registry.width(instruction.type());} + case DYN_CAN_UNPACK -> {pops=2;pushes=1;} + case REF_IS_EXACT,REF_IMPLEMENTS,REF_TO_TRAIT,REF_CAST -> { + if(!(body instanceof Struct||body instanceof Array||body instanceof Trait))throw new IllegalArgumentException("reference source category"); + var target=registry.type(instruction.targetType()).body();boolean trait=instruction.kind()==IRRegistryInstruction.Kind.REF_IMPLEMENTS||instruction.kind()==IRRegistryInstruction.Kind.REF_TO_TRAIT; + if(trait?!(target instanceof Trait):!(target instanceof Struct||target instanceof Array))throw new IllegalArgumentException("reference target category"); + second=registry.id(instruction.targetType()).index();pops=registry.width(instruction.type());pushes=instruction.kind()==IRRegistryInstruction.Kind.REF_TO_TRAIT?2:1; + } + case STRUCT_NEW,STRUCT_GET,STRUCT_SET -> { + if(!(body instanceof Struct struct))throw new IllegalArgumentException("Struct category required"); + if(instruction.kind()==IRRegistryInstruction.Kind.STRUCT_NEW)pushes=1; + else { + if(instruction.member()>=struct.fields().size())throw new IllegalArgumentException("field index"); + second=instruction.member();int width=registry.width(struct.fields().get(instruction.member()).type()); + pops=instruction.kind()==IRRegistryInstruction.Kind.STRUCT_SET?1+width:1;pushes=instruction.kind()==IRRegistryInstruction.Kind.STRUCT_GET?width:0; + } + } + case ARRAY_NEW,ARRAY_LEN,ARRAY_GET,ARRAY_SET -> { + if(!(body instanceof Array array))throw new IllegalArgumentException("Array category required"); + int width=registry.width(array.element()); + switch(instruction.kind()) {case ARRAY_NEW,ARRAY_LEN->{pops=1;pushes=1;}case ARRAY_GET->{pops=2;pushes=width;}case ARRAY_SET->pops=2+width;default->throw new AssertionError();} + } + } + return new IRVMInstruction(new IRVMOp("REGISTRY_"+instruction.kind(),instruction.kind().opcode(),second==null?4:8,pops,pushes,false,false,false),first,second); + } +} diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/lifecycle/LifecycleAssemblerService.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/lifecycle/LifecycleAssemblerService.java index 0dfd7cb5..7d9fb77b 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/lifecycle/LifecycleAssemblerService.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/lifecycle/LifecycleAssemblerService.java @@ -201,6 +201,7 @@ public final class LifecycleAssemblerService { frameExecutable.span())); return IRBackend.builder() + .registry(backend.getRegistry()) .entryPointCallableName(entryPointCallableName) .entryPointModuleId(entryPointModuleId) .functions(backend.getFunctions()) diff --git a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/workspaces/stages/LowerToIRVMPipelineStage.java b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/workspaces/stages/LowerToIRVMPipelineStage.java index 5b79bace..afaa9642 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/workspaces/stages/LowerToIRVMPipelineStage.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/main/java/p/studio/compiler/workspaces/stages/LowerToIRVMPipelineStage.java @@ -54,6 +54,10 @@ public class LowerToIRVMPipelineStage implements PipelineStage { .pc(e.pc()) .message("[BUILD]: lower to irvm validation failed: " + e.getMessage()) .exception(e)); + } catch (IllegalArgumentException e) { + return BuildingIssueSink.empty().report(builder -> builder.error(true) + .phase("BACKEND_LOWER_TO_IRVM").code("LOWER_IRVM_INVALID_TYPED_REGISTRY") + .message("[BUILD]: invalid common typed IR: " + e.getMessage()).exception(e)); } return BuildingIssueSink.empty(); } diff --git a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/bytecode/BytecodeEmitterTest.java b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/bytecode/BytecodeEmitterTest.java index 0e74740d..87c6ae71 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/bytecode/BytecodeEmitterTest.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/bytecode/BytecodeEmitterTest.java @@ -106,7 +106,7 @@ class BytecodeEmitterTest { 0, 1, ReadOnlyList.from( - BytecodeEmitter.Operation.intrinsic(0x2000), + BytecodeEmitter.Operation.intrinsic(0x2000, 0, 0, null), BytecodeEmitter.Operation.halt()))))); assertTrue(module.syscalls().isEmpty()); @@ -131,7 +131,7 @@ class BytecodeEmitterTest { ReadOnlyList.from( BytecodeEmitter.Operation.halt(new BytecodeModule.SourceSpan(1, 0, 1)), BytecodeEmitter.Operation.callFunc(0, new BytecodeModule.SourceSpan(1, 2, 3)), - BytecodeEmitter.Operation.intrinsic(0x2000, new BytecodeModule.SourceSpan(1, 4, 5)), + BytecodeEmitter.Operation.intrinsic(0x2000, 0, 0, new BytecodeModule.SourceSpan(1, 4, 5)), BytecodeEmitter.Operation.ret(new BytecodeModule.SourceSpan(1, 6, 7))))))); assertEquals(4, module.debugInfo().pcToSpan().size()); diff --git a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/bytecode/BytecodeModuleTest.java b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/bytecode/BytecodeModuleTest.java index 82227e1e..e02aa64b 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/bytecode/BytecodeModuleTest.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/bytecode/BytecodeModuleTest.java @@ -22,13 +22,13 @@ class BytecodeModuleTest { assertEquals(0, bytes[3]); final var sectionCount = readU32(bytes, 8); - assertEquals(1, sectionCount); + assertEquals(8, sectionCount); - final var sectionKind = readU32(bytes, 32); - final var sectionOffset = readU32(bytes, 36); - final var sectionLength = readU32(bytes, 40); + final var sectionKind = readU32(bytes, 32 + 4 * 12); + final var sectionOffset = readU32(bytes, 36 + 4 * 12); + final var sectionLength = readU32(bytes, 40 + 4 * 12); assertEquals(5, sectionKind); - assertEquals(44, sectionOffset); + assertEquals(140, sectionOffset); assertEquals(4, sectionLength); final var syscCount = readU32(bytes, sectionOffset); diff --git a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/LinkedRegistryTest.java b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/LinkedRegistryTest.java new file mode 100644 index 00000000..39597c11 --- /dev/null +++ b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/LinkedRegistryTest.java @@ -0,0 +1,53 @@ +package p.studio.compiler.backend.registry; + +import org.junit.jupiter.api.Test; +import p.studio.compiler.models.registry.IRRegistry; +import p.studio.compiler.models.registry.IRRegistry.*; +import p.studio.utilities.structures.ReadOnlyList; +import static org.junit.jupiter.api.Assertions.*; + +class LinkedRegistryTest { + private static TypeRef ref(String key) { return new TypeRef(key); } + private static TypeDeclaration decl(String key, TypeBody body) { return new TypeDeclaration(ref(key), "same display name", body); } + private static IRRegistry source(TypeDeclaration... types) { return new IRRegistry(ReadOnlyList.from(types), null); } + + @Test void nominalIdentityIsNotNameOrShapeAndRepeatedDeclarationsConverge() { + var a=decl("module:a",new Struct(null)); var b=decl("module:b",new Struct(null)); + var linked=LinkedRegistry.link(source(a,b,a)); + assertEquals(2,linked.types().size()); + assertNotEquals(linked.id(a.identity()),linked.id(b.identity())); + assertEquals(linked.types(),LinkedRegistry.link(source(b,a)).types()); + } + @Test void importsArraysAndFnSignaturesCanonicalizeWithoutUsingParameterNames() { + var signature=new Signature(ReadOnlyList.from(new NamedType(ref("v1"),"x")),ReadOnlyList.from(ref("v2"))); + var renamed=new Signature(ReadOnlyList.from(new NamedType(ref("v2"),"renamed")),ReadOnlyList.from(ref("v1"))); + var linked=LinkedRegistry.link(source(decl("v1",new Builtin(2)),decl("v2",new Builtin(2)), + decl("a1",new Array(ref("v1"))),decl("a2",new Array(ref("v2"))), + decl("f1",new Fn(signature)),decl("f2",new Fn(renamed)))); + assertEquals(3,linked.types().size()); + assertEquals(linked.id(ref("v1")),linked.id(ref("v2"))); + assertEquals(linked.id(ref("a1")),linked.id(ref("a2"))); + assertEquals(linked.id(ref("f1")),linked.id(ref("f2"))); + assertEquals(2,linked.parameterSlots(signature)); + assertEquals(2,linked.resultSlots(signature)); + } + @Test void nominalAnchorsAllowForwardReferencesButStructuralCyclesReject() { + var signature=new Signature(ReadOnlyList.from(new NamedType(ref("array"),null)),null); + assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source( + decl("fn",new Fn(signature)),decl("array",new Array(ref("fn")))))); + assertDoesNotThrow(()->LinkedRegistry.link(source( + decl("fn",new Fn(signature)),decl("array",new Array(ref("node"))), + decl("node",new Struct(ReadOnlyList.from(new NamedType(ref("fn"),null))))))); + } + @Test void missingReferencesConflictingDeclarationsAndRemovedImportsReject() { + assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source(decl("a",new Array(ref("missing")))))); + assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source(decl("a",new Builtin(1)),decl("a",new Builtin(2))))); + assertThrows(IllegalArgumentException.class,()->new Builtin(0x100)); + assertEquals(1,LinkedRegistry.link(source(decl("random",new Builtin(0x16)))).width(ref("random"))); + } + @Test void namesAreStrictUtf8AndBounded() { + assertThrows(IllegalArgumentException.class,()->LinkedRegistry.nameBytes("\uD800")); + assertThrows(IllegalArgumentException.class,()->LinkedRegistry.nameBytes("é".repeat(2049))); + assertEquals(4096,LinkedRegistry.nameBytes("é".repeat(2048))); + } +} diff --git a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/PbxRegistryConformanceTest.java b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/PbxRegistryConformanceTest.java new file mode 100644 index 00000000..345562b8 --- /dev/null +++ b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/PbxRegistryConformanceTest.java @@ -0,0 +1,188 @@ +package p.studio.compiler.backend.registry; + +import org.junit.jupiter.api.Test; +import p.studio.compiler.backend.bytecode.*; +import p.studio.compiler.models.registry.IRRegistry; +import p.studio.compiler.models.registry.IRRegistry.*; +import p.studio.utilities.structures.ReadOnlyList; +import java.nio.*; +import java.nio.file.*; +import java.util.*; +import static org.junit.jupiter.api.Assertions.*; + +class PbxRegistryConformanceTest { + private static byte[] instructions(int[]... instructions) { + var out=new java.io.ByteArrayOutputStream(); + for(var instruction:instructions) { + out.write(instruction[0]);out.write(instruction[0]>>>8); + for(int n=1;n>>(8*b)); + } + return out.toByteArray(); + } + private static int[] op(int... values) { return values; } + private static final BytecodePreloadVerifierService VERIFIER = new BytecodePreloadVerifierService(); + private static IRRegistry registry(int builtin) { + return new IRRegistry(ReadOnlyList.from(new TypeDeclaration(TypeRef.builtin(builtin), null, new Builtin(builtin))), null); + } + private static byte[] hex(String value) { return HexFormat.of().parseHex(value.replaceAll("\\s", "")); } + private static BytecodeModule module(IRRegistry registry, byte[] code, int parameters, int results, int max, FunctionMetadata metadata) { + return new BytecodeModule(0, null, ReadOnlyList.from(new BytecodeModule.FunctionMeta(0, code.length, parameters, 0, results, max, metadata)), code, null, null, null, registry); + } + private static BytecodeModule emptyFunction(byte[] code, int max) { + return module(registry(0x13), code, 0, 0, max, FunctionMetadata.empty()); + } + private static void fixture(String name, BytecodeModule module) throws Exception { + Path directory=Path.of("build/registry-conformance"); Files.createDirectories(directory); + Files.write(directory.resolve(name+".pbx"), module.serialize()); + } + private static byte[] section(byte[] bytes, int kind) { + var b=ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN); + for(int i=0;imodule(registry(2),module.code(),1,1,2,metadata).serialize()); + } + @Test void absoluteJumpToFunctionEndAndUnreachableOperands() throws Exception { + var module=emptyFunction(hex("02 00 08 00 00 00 00 00"),0); + VERIFIER.verify(module);fixture("jump-end",module); + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("02 00 03 00 00 00 01 00"),0))); + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("01 00 50 00 ff ff ff ff"),0))); + for(int op:new int[]{0x54,0x55,0x56,0x70})assertThrows(RuntimeException.class,()->VERIFIER.verify(emptyFunction(new byte[]{1,0,(byte)op,0,0,0,0,0},0))); + } + @Test void literalZeroAndExactReturnHeight() { + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 11 00 51 00"),0))); + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 51 00"),1))); + assertDoesNotThrow(()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 11 00 51 00"),1))); + } + @Test void nonfiniteUnreachableImmediateAndConstantReject() { + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("01 00 15 00 00 00 00 00 00 00 f0 7f"),0))); + var m=emptyFunction(hex("01 00"),0); + var bad=new BytecodeModule(0,ReadOnlyList.from(new BytecodeModule.Float64Constant(Double.NaN)),m.functions(),m.code(),null,null,null,m.registry()); + assertThrows(BytecodeMarshalingException.class,bad::serialize); + } + @Test void signatureNamesAreIndependentFromDebugAndStrictUtf8() { + var metadata=new FunctionMetadata(Signature.empty(),null,Role.NONE,""); + var module=module(registry(0x13),hex("01 00"),0,0,0,metadata); + byte[] function=section(module.serialize(),1); + assertEquals(1,function[5]);assertEquals(34,function.length); + var bad=new FunctionMetadata(Signature.empty(),null,Role.NONE,"\uD800"); + assertThrows(BytecodeMarshalingException.class,()->module(registry(0x13),hex("01 00"),0,0,0,bad).serialize()); + } + @Test void unusedAndConflictingNativeDeclarationsRejectWithoutBindingFallback() { + var m=emptyFunction(hex("01 00"),0); + var declaration=new BytecodeModule.SyscallDecl("test","native",0,0,0); + var unused=new BytecodeModule(0,null,m.functions(),m.code(),null,null,ReadOnlyList.from(declaration),m.registry()); + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(unused)); + var duplicate=new BytecodeModule(0,null,m.functions(),m.code(),null,null,ReadOnlyList.from(declaration,declaration),m.registry()); + assertThrows(BytecodeMarshalingException.class,duplicate::serialize); + } + @Test void closuresCarryExplicitEnvironmentAndExactFnIdentity() throws Exception { + var integer=TypeRef.builtin(0x10);var fn=new TypeRef("fixture:fn"); + var signature=new Signature(null,ReadOnlyList.from(integer)); + var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(fn,null,new Fn(signature))),null); + int fnId=LinkedRegistry.link(registry).id(fn).index(); + byte[] boot=instructions(op(0x17,42),op(0x52,1,fnId),op(0x53,fnId),op(1)); + byte[] body=instructions(op(0x57,0),op(0x51)); + var bytes=new java.io.ByteArrayOutputStream();bytes.writeBytes(boot);bytes.writeBytes(body); + var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,boot.length,0,0,0,1,FunctionMetadata.empty()), + new BytecodeModule.FunctionMeta(boot.length,body.length,0,0,1,1,new FunctionMetadata(signature,ReadOnlyList.from(integer),Role.NONE))); + var m=new BytecodeModule(0,null,functions,bytes.toByteArray(),null,null,null,registry); + VERIFIER.verify(m);fixture("closure-capture",m); + byte[] direct=m.code().clone();direct[6]=0x50; + var invalid=new BytecodeModule(0,null,functions,direct,null,null,null,registry); + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(invalid)); + } + @Test void traitImplementationUsesConcreteReceiverAndOrderedMethods() throws Exception { + var integer=TypeRef.builtin(0x10);var owner=new TypeRef("fixture:owner");var trait=new TypeRef("fixture:trait"); + var result=new Signature(null,ReadOnlyList.from(integer)); + var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)), + new TypeDeclaration(owner,null,new Struct(null)),new TypeDeclaration(trait,null,new Trait(ReadOnlyList.from(new Method("answer",result))))), + ReadOnlyList.from(new Implementation("fixture:impl",owner,trait,ReadOnlyList.from(new p.studio.compiler.source.identifiers.CallableId(1))))); + var linked=LinkedRegistry.link(registry);int ownerId=linked.id(owner).index(),traitId=linked.id(trait).index(); + byte[] boot=instructions(op(0xa0,ownerId),op(0x95,ownerId,traitId),op(0x58,traitId,0),op(1)); + byte[] body=instructions(op(0x17,42),op(0x51)); + var bytes=new java.io.ByteArrayOutputStream();bytes.writeBytes(boot);bytes.writeBytes(body); + var implementation=FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(owner,"self")),ReadOnlyList.from(integer))); + var m=new BytecodeModule(0,null,ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,boot.length,0,0,0,2,FunctionMetadata.empty()), + new BytecodeModule.FunctionMeta(boot.length,body.length,1,0,1,1,implementation)),bytes.toByteArray(),null,null,null,registry); + VERIFIER.verify(m);fixture("trait-call",m); + } + @Test void arrayAndDynamicOperationsUseRegistryOperands() throws Exception { + var integer=TypeRef.builtin(0x10);var array=new TypeRef("fixture:array"); + var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(array,null,new Array(integer))),null); + var linked=LinkedRegistry.link(registry);int arrayId=linked.id(array).index(),intId=linked.id(integer).index(); + byte[] code=instructions(op(0x14,1,0),op(0xa3,arrayId),op(0x14,0,0),op(0xa5,arrayId),op(0x90,intId),op(0x91,intId),op(1)); + var m=module(registry,code,0,0,2,FunctionMetadata.empty()); + VERIFIER.verify(m);fixture("array-dynamic",m); + } + @Test void bindingFixturesKeepExactCanonicalIdentityAndAbi() throws Exception { + var m=emptyFunction(instructions(op(0x71,0),op(0x11),op(1)),1); + for(String name:List.of("native", "unknown-native", "native-abi", "unused-native")) { + var declaration=new BytecodeModule.SyscallDecl("system",name.equals("unknown-native")?"missing":"has_cart",1,name.equals("native-abi")?1:0,1); + byte[] code=name.equals("unused-native")?instructions(op(1)):m.code(); + var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,code.length,0,0,0,1,FunctionMetadata.empty())); + var module=new BytecodeModule(0,null,functions,code,null,null,ReadOnlyList.from(declaration),m.registry()); + fixture(name,module); + if(name.equals("native"))VERIFIER.verify(module); + } + } + @Test void rolesStayMetadataAndValidateSignatureAndUniqueness() throws Exception { + var pause=new FunctionMetadata(Signature.empty(),null,Role.BEFORE_PAUSE); + var resume=new FunctionMetadata(Signature.empty(),null,Role.BEFORE_RESUME); + var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,2,0,0,0,0,FunctionMetadata.empty()), + new BytecodeModule.FunctionMeta(2,2,0,0,0,0,pause),new BytecodeModule.FunctionMeta(4,2,0,0,0,0,resume)); + var m=new BytecodeModule(0,null,functions,instructions(op(1),op(0x51),op(0x51)),null,null,null,IRRegistry.empty()); + VERIFIER.verify(m);fixture("roles",m); + assertThrows(BytecodeMarshalingException.class,()->module(IRRegistry.empty(),instructions(op(1)),0,0,0,pause).serialize()); + var duplicated=ReadOnlyList.from(functions.get(0),functions.get(1),new BytecodeModule.FunctionMeta(4,2,0,0,0,0,pause)); + assertThrows(BytecodeMarshalingException.class,()->new BytecodeModule(0,null,duplicated,m.code(),null,null,null,m.registry()).serialize()); + } + @Test void joinsWidenFactsAndLoopsReachFixedPoint() throws Exception { + // bool; branch to alternate scalar; equal-height join becomes Unknown, then POP. + byte[] code=instructions(op(0x10,0),op(0x03,24),op(0x17,1),op(0x02,30),op(0x10,1),op(0x11),op(1)); + var base=emptyFunction(code,1); + var m=new BytecodeModule(0,ReadOnlyList.from(new BytecodeModule.BooleanConstant(false),new BytecodeModule.StringConstant("alternate")),base.functions(),code,null,null,null,base.registry()); + VERIFIER.verify(m);fixture("join",m); + byte[] loop=instructions(op(0x40,0),op(0x03,18),op(0x02,0),op(1)); + assertDoesNotThrow(()->VERIFIER.verify(emptyFunction(loop,1))); + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("16 00 01 16 00 00 20 00 01 00"),2))); + } + @Test void memcardRoleHasExactTypedSignatureWithoutLanguagePolicy() throws Exception { + var input=TypeRef.builtin(0x11);var output=TypeRef.builtin(0x10); + var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(input,null,new Builtin(0x11)),new TypeDeclaration(output,null,new Builtin(0x10))),null); + var metadata=new FunctionMetadata(new Signature(ReadOnlyList.from(new NamedType(input,"status")),ReadOnlyList.from(output)),null,Role.MEMCARD_BOOT); + var code=instructions(op(1),op(0x17,0),op(0x51)); + var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,2,0,0,0,0,FunctionMetadata.empty()),new BytecodeModule.FunctionMeta(2,8,1,0,1,1,metadata)); + var module=new BytecodeModule(0,null,functions,code,null,null,null,registry); + VERIFIER.verify(module);fixture("memcard-role",module); + } + @Test void knownSemanticGroupsCannotSilentlyChangeType() { + var color=TypeRef.builtin(1);var integer=TypeRef.builtin(0x11); + var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(color,null,new Builtin(1)),new TypeDeclaration(integer,null,new Builtin(0x11))),null); + var metadata=FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(color,null)),ReadOnlyList.from(integer))); + var module=module(registry,instructions(op(0x42,0),op(0x51)),1,1,1,metadata); + assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(module)); + } + @Test void intrinsicFixtureUsesExplicitAbiAndKeepsNativeBindingSeparate() throws Exception { + // Runtime vec2.dot@1 (0x1000): four Float carriers -> one Float. + byte[] code=instructions(op(0x15,0,0x3ff00000),op(0x15,0,0x40000000), + op(0x15,0,0x40080000),op(0x15,0,0x40100000),op(0x72,0x1000),op(1)); + var base=emptyFunction(code,4); + var module=new BytecodeModule(0,null,base.functions(),code,null,null,null,base.registry(),ReadOnlyList.from(new BytecodeModule.IntrinsicSignature(0x1000,4,1))); + VERIFIER.verify(module);fixture("intrinsic-dot",module); + } +} diff --git a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/TypedCommonPipelineTest.java b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/TypedCommonPipelineTest.java new file mode 100644 index 00000000..f0076e8c --- /dev/null +++ b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/TypedCommonPipelineTest.java @@ -0,0 +1,46 @@ +package p.studio.compiler.backend.registry; + +import org.junit.jupiter.api.Test; +import p.studio.compiler.backend.bytecode.*; +import p.studio.compiler.backend.irvm.*; +import p.studio.compiler.models.*; +import p.studio.compiler.models.registry.*; +import p.studio.compiler.models.registry.IRRegistry.*; +import p.studio.compiler.source.identifiers.*; +import p.studio.utilities.structures.ReadOnlyList; +import java.nio.file.*; +import static org.junit.jupiter.api.Assertions.*; + +class TypedCommonPipelineTest { + private static IRBackendExecutableFunction.Instruction plain(IRBackendExecutableFunction.InstructionKind kind, String literal) { + return new IRBackendExecutableFunction.Instruction(kind,null,null,null,null,null,literal,null,null,null,null); + } + private static IRBackendExecutableFunction.Instruction typed(IRRegistryInstruction operand) { + return new IRBackendExecutableFunction.Instruction(IRBackendExecutableFunction.InstructionKind.REGISTRY,null,null,null,null,null,null,null,null,null,null,operand); + } + @Test void syntheticFrontendClosureSurvivesLinkLowerOptimizeAndEmit() throws Exception { + var integer=TypeRef.builtin(0x10);var fn=new TypeRef("synthetic:fn"); + var signature=new Signature(null,ReadOnlyList.from(integer)); + var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(fn,null,new Fn(signature))),null); + var capture=new CallableId(71); + var main=new IRBackendExecutableFunction(new FileId(0),new ModuleId(0),"main",new CallableId(91),0,0,0,0,0,1, + ReadOnlyList.from(plain(IRBackendExecutableFunction.InstructionKind.PUSH_I32,"42"), + typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.MAKE_CLOSURE,fn,null,capture,0)), + typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.CALL_CLOSURE,fn,null,null,0)), + plain(IRBackendExecutableFunction.InstructionKind.POP,null),plain(IRBackendExecutableFunction.InstructionKind.RET,null)),null,FunctionMetadata.empty()); + var body=new IRBackendExecutableFunction(new FileId(0),new ModuleId(0),"captured",capture,0,0,0,0,1,1, + ReadOnlyList.from(typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.GET_CAPTURE,null,null,null,0)), + plain(IRBackendExecutableFunction.InstructionKind.RET,null)),null,new FunctionMetadata(signature,ReadOnlyList.from(integer),Role.NONE)); + var backend=IRBackend.builder().entryPointModuleId(new ModuleId(0)).registry(registry).executableFunctions(ReadOnlyList.from(body,main)).build(); + var lowered=new LowerToIRVMService().lower(backend); + var optimized=new OptimizeIRVMService().optimize(lowered); + assertEquals(registry,optimized.module().registry()); + assertEquals(body.metadata(),optimized.module().functions().get(1).metadata()); + var module=new BytecodeEmitter().emit(optimized.coherentEmissionPlan()); + new BytecodePreloadVerifierService().verify(module); + var repeated=new BytecodeEmitter().emit(new OptimizeIRVMService().optimize(new LowerToIRVMService().lower(backend)).coherentEmissionPlan()); + assertArrayEquals(module.serialize(),repeated.serialize()); + Files.createDirectories(Path.of("build/registry-conformance")); + Files.write(Path.of("build/registry-conformance/common-pipeline.pbx"),module.serialize()); + } +} diff --git a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/workspaces/stages/LowerToIRVMPipelineStageTest.java b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/workspaces/stages/LowerToIRVMPipelineStageTest.java index 42e68ac8..afcf903e 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/workspaces/stages/LowerToIRVMPipelineStageTest.java +++ b/prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/workspaces/stages/LowerToIRVMPipelineStageTest.java @@ -1,6 +1,8 @@ package p.studio.compiler.workspaces.stages; import org.junit.jupiter.api.Test; +import p.studio.compiler.models.registry.IRRegistry; +import p.studio.compiler.models.registry.IRRegistry.*; import p.studio.compiler.messages.BuilderPipelineConfig; import p.studio.compiler.models.BuilderPipelineContext; import p.studio.compiler.models.IRBackend; @@ -144,6 +146,7 @@ class LowerToIRVMPipelineStageTest { final var ctx = BuilderPipelineContext.fromConfig(new BuilderPipelineConfig(false, ".")); ctx.irBackend = IRBackend.builder() .entryPointModuleId(new p.studio.compiler.source.identifiers.ModuleId(0)) + .registry(new IRRegistry(ReadOnlyList.from(new TypeDeclaration(TypeRef.builtin(0x10),null,new Builtin(0x10))),null)) .executableFunctions(ReadOnlyList.from( new IRBackendExecutableFunction( new FileId(0), @@ -162,7 +165,7 @@ class LowerToIRVMPipelineStageTest { null, null, Span.none())), - Span.none()), + Span.none(), FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(TypeRef.builtin(0x10),null)),null))), new IRBackendExecutableFunction( new FileId(0), new p.studio.compiler.source.identifiers.ModuleId(0), diff --git a/prometeu-compiler/prometeu-build-pipeline/src/test/resources/golden/bytecode-module.hex b/prometeu-compiler/prometeu-build-pipeline/src/test/resources/golden/bytecode-module.hex index a71187fb..5c8c6246 100644 --- a/prometeu-compiler/prometeu-build-pipeline/src/test/resources/golden/bytecode-module.hex +++ b/prometeu-compiler/prometeu-build-pipeline/src/test/resources/golden/bytecode-module.hex @@ -1 +1 @@ -504258000000000004000000000000000000000000000000000000000000000001000000500000001400000002000000640000000e00000003000000720000004400000005000000b60000001d00000001000000000000000e000000000000000000020071000000000072000020000051000300000000000000010000000000000000000000060000000100000000000000000000000c0000000100000000000000000000000100000000000000040000006d61696e01000000050067667832640a00647261775f706978656c010000000000 +5042580000000000090000000000000000000000000000000000000000000000000000008c0000000400000001000000900000001e00000002000000ae0000000e00000003000000bc0000004400000004000000000100000400000005000000040100001d000000060000002101000004000000070000002501000004000000080000002901000008000000000000000100000000000000000000000000000000000e000000000000000200000071000000000072000020000051000300000000000000010000000000000000000000060000000100000000000000000000000c0000000100000000000000000000000100000000000000040000006d61696e0000000001000000050067667832640a00647261775f706978656c01000000000000000000000000000100000000000000 diff --git a/prometeu-compiler/prometeu-build-pipeline/src/test/resources/registry/minimal-runtime-v0.hex b/prometeu-compiler/prometeu-build-pipeline/src/test/resources/registry/minimal-runtime-v0.hex new file mode 100644 index 00000000..29d34c9b --- /dev/null +++ b/prometeu-compiler/prometeu-build-pipeline/src/test/resources/registry/minimal-runtime-v0.hex @@ -0,0 +1,20 @@ +# Independent runtime registry_v0.rs minimal_bytes oracle (194 bytes) +50 42 58 00 00 00 00 00 08 00 00 00 +00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 +00 00 00 00 80 00 00 00 04 00 00 00 +01 00 00 00 84 00 00 00 1e 00 00 00 +02 00 00 00 a2 00 00 00 02 00 00 00 +04 00 00 00 a4 00 00 00 04 00 00 00 +05 00 00 00 a8 00 00 00 04 00 00 00 +06 00 00 00 ac 00 00 00 0a 00 00 00 +07 00 00 00 b6 00 00 00 04 00 00 00 +08 00 00 00 ba 00 00 00 08 00 00 00 +00 00 00 00 +01 00 00 00 00 00 00 00 00 00 00 00 00 00 +00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 +01 00 +00 00 00 00 +00 00 00 00 +01 00 00 00 00 00 13 00 00 00 +00 00 00 00 +01 00 00 00 00 00 00 00 diff --git a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackend.java b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackend.java index 920233eb..438f06a2 100644 --- a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackend.java +++ b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackend.java @@ -1,6 +1,7 @@ package p.studio.compiler.models; import lombok.Builder; +import p.studio.compiler.models.registry.IRRegistry; import lombok.Getter; import p.studio.compiler.source.identifiers.CallableId; import p.studio.compiler.source.identifiers.IntrinsicId; @@ -34,6 +35,8 @@ public class IRBackend implements IRBackendReader { private final ReadOnlyList intrinsicPool = ReadOnlyList.empty(); @Builder.Default private final IRReservedMetadata reservedMetadata = IRReservedMetadata.empty(); + @Builder.Default + private final IRRegistry registry = IRRegistry.empty(); private final IRLifecycleDeclaration lifecycleDeclaration; public static IRBackendAggregator aggregator() { @@ -41,6 +44,8 @@ public class IRBackend implements IRBackendReader { } public static final class IRBackendAggregator { + private final ArrayList registryTypes = new ArrayList<>(); + private final ArrayList registryImplementations = new ArrayList<>(); private String entryPointCallableName; private ModuleId entryPointModuleId = ModuleId.none(); private final ArrayList functions = new ArrayList<>(); @@ -90,6 +95,11 @@ public class IRBackend implements IRBackendReader { } final var callableRemap = reindexCallables(backendFile.callableSignatures(), moduleRemap); final var intrinsicRemap = reindexIntrinsics(backendFile.intrinsicPool()); + registryTypes.addAll(backendFile.registry().types().asList()); + for (final var impl : backendFile.registry().implementations()) { + registryImplementations.add(new IRRegistry.Implementation(impl.declarationKey(), impl.owner(), impl.trait(), + ReadOnlyList.wrap(impl.methods().stream().map(id -> remapCallableId(id, callableRemap, "implementation")).toList()))); + } for (final var function : backendFile.executableFunctions()) { executableFunctions.add(remapExecutableFunction(function, moduleRemap, callableRemap, intrinsicRemap)); } @@ -200,7 +210,9 @@ public class IRBackend implements IRBackendReader { instruction.targetLabel(), instruction.expectedArgSlots(), instruction.expectedRetSlots(), - instruction.span())); + instruction.span(), instruction.registryInstruction() == null ? null : + new p.studio.compiler.models.registry.IRRegistryInstruction(instruction.registryInstruction().kind(), instruction.registryInstruction().type(), instruction.registryInstruction().targetType(), + instruction.registryInstruction().function() == null ? null : remapCallableId(instruction.registryInstruction().function(), callableRemap, "Registry instruction"), instruction.registryInstruction().member()))); } return new IRBackendExecutableFunction( function.fileId(), @@ -214,7 +226,7 @@ public class IRBackend implements IRBackendReader { function.returnSlots(), function.maxStackSlots(), ReadOnlyList.wrap(remappedInstructions), - function.span()); + function.span(), function.metadata()); } private IRLifecycleFileDeclaration remapLifecycleFileDeclaration( @@ -367,6 +379,7 @@ public class IRBackend implements IRBackendReader { public IRBackend emit() { return IRBackend .builder() + .registry(new IRRegistry(ReadOnlyList.wrap(registryTypes), ReadOnlyList.wrap(registryImplementations))) .entryPointCallableName(resolveEntryPointCallableName()) .entryPointModuleId(resolveEntryPointModuleId()) .functions(ReadOnlyList.wrap(functions)) diff --git a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendExecutableFunction.java b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendExecutableFunction.java index 30f7f410..e07dde62 100644 --- a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendExecutableFunction.java +++ b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendExecutableFunction.java @@ -10,6 +10,20 @@ import p.studio.utilities.structures.ReadOnlyList; import java.util.Objects; public record IRBackendExecutableFunction( + FileId fileId, + ModuleId moduleId, + String callableName, + CallableId callableId, + int sourceStart, + int sourceEnd, + int paramSlots, + int localSlots, + int returnSlots, + int maxStackSlots, + ReadOnlyList instructions, + Span span, + p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) { + public IRBackendExecutableFunction( FileId fileId, ModuleId moduleId, String callableName, @@ -22,6 +36,9 @@ public record IRBackendExecutableFunction( int maxStackSlots, ReadOnlyList instructions, Span span) { + this(fileId, moduleId, callableName, callableId, sourceStart, sourceEnd, paramSlots, localSlots, returnSlots, maxStackSlots, instructions, span, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null); + } + public IRBackendExecutableFunction( final FileId fileId, @@ -82,6 +99,19 @@ public record IRBackendExecutableFunction( } public record Instruction( + InstructionKind kind, + ModuleId calleeModuleId, + String calleeCallableName, + CallableId calleeCallableId, + HostCallMetadata hostCall, + IntrinsicCallMetadata intrinsicCall, + String label, + String targetLabel, + Integer expectedArgSlots, + Integer expectedRetSlots, + Span span, + p.studio.compiler.models.registry.IRRegistryInstruction registryInstruction) { + public Instruction( InstructionKind kind, ModuleId calleeModuleId, String calleeCallableName, @@ -93,6 +123,9 @@ public record IRBackendExecutableFunction( Integer expectedArgSlots, Integer expectedRetSlots, Span span) { + this(kind, calleeModuleId, calleeCallableName, calleeCallableId, hostCall, intrinsicCall, label, targetLabel, expectedArgSlots, expectedRetSlots, span, null); + } + public Instruction( final InstructionKind kind, final ModuleId calleeModuleId, @@ -217,7 +250,11 @@ public record IRBackendExecutableFunction( if (expectedRetSlots != null && expectedRetSlots < 0) { throw new IllegalArgumentException("expectedRetSlots must be non-negative"); } + if ((kind == InstructionKind.REGISTRY) != (registryInstruction != null)) throw new IllegalArgumentException("Registry operand mismatch"); switch (kind) { + case REGISTRY -> { + if (hostCall != null || intrinsicCall != null || calleeCallableId != null || !label.isEmpty() || !targetLabel.isEmpty()) throw new IllegalArgumentException("unexpected Registry instruction metadata"); + } case PUSH_I32 -> { if (label.isBlank()) { throw new IllegalArgumentException("PUSH_I32 requires immediate label payload"); @@ -387,6 +424,7 @@ public record IRBackendExecutableFunction( } public enum InstructionKind { + REGISTRY, PUSH_I32, PUSH_BOOL, PUSH_CONST, diff --git a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendFile.java b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendFile.java index 3f495a38..a47c573e 100644 --- a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendFile.java +++ b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendFile.java @@ -7,8 +7,22 @@ import p.studio.compiler.source.tables.ModuleReference; import p.studio.utilities.structures.ReadOnlyList; import java.util.Objects; +import p.studio.compiler.models.registry.IRRegistry; public record IRBackendFile( + FileId fileId, + ReadOnlyList functions, + ReadOnlyList syntheticFunctions, + ReadOnlyList globals, + ReadOnlyList executableFunctions, + IRReservedMetadata reservedMetadata, + ReadOnlyList modulePool, + ReadOnlyList callableSignatures, + ReadOnlyList intrinsicPool, + IRLifecycleFileDeclaration lifecycleDeclaration, + IRRegistry registry) { + + public IRBackendFile( FileId fileId, ReadOnlyList functions, ReadOnlyList syntheticFunctions, @@ -19,7 +33,11 @@ public record IRBackendFile( ReadOnlyList callableSignatures, ReadOnlyList intrinsicPool, IRLifecycleFileDeclaration lifecycleDeclaration) { + this(fileId, functions, syntheticFunctions, globals, executableFunctions, reservedMetadata, modulePool, callableSignatures, intrinsicPool, lifecycleDeclaration, IRRegistry.empty()); + } + public IRBackendFile { + registry = registry == null ? IRRegistry.empty() : registry; fileId = Objects.requireNonNull(fileId, "fileId"); functions = functions == null ? ReadOnlyList.empty() : functions; syntheticFunctions = syntheticFunctions == null ? ReadOnlyList.empty() : syntheticFunctions; diff --git a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendReader.java b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendReader.java index d3e8768f..6ddf8a4b 100644 --- a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendReader.java +++ b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/IRBackendReader.java @@ -7,6 +7,7 @@ import p.studio.compiler.source.tables.ModuleReference; import p.studio.utilities.structures.ReadOnlyList; public interface IRBackendReader { + p.studio.compiler.models.registry.IRRegistry getRegistry(); String getEntryPointCallableName(); ModuleId getEntryPointModuleId(); ReadOnlyList getFunctions(); diff --git a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/registry/IRRegistry.java b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/registry/IRRegistry.java new file mode 100644 index 00000000..e6b48f61 --- /dev/null +++ b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/registry/IRRegistry.java @@ -0,0 +1,82 @@ +package p.studio.compiler.models.registry; + +import p.studio.compiler.source.identifiers.CallableId; +import p.studio.utilities.structures.ReadOnlyList; +import java.util.Objects; + +/** Frontend-neutral declarations. References are stable keys, never Java object identity or image IDs. */ +public record IRRegistry(ReadOnlyList types, ReadOnlyList implementations) { + public IRRegistry { + types = copy(types); + implementations = copy(implementations); + } + public static IRRegistry empty() { return new IRRegistry(null, null); } + private static ReadOnlyList copy(ReadOnlyList values) { + return values == null ? ReadOnlyList.empty() : ReadOnlyList.wrap(values.asList()); + } + public record TypeRef(String declarationKey) { + public TypeRef { + Objects.requireNonNull(declarationKey, "declarationKey"); + if (declarationKey.isBlank()) throw new IllegalArgumentException("empty type declaration key"); + } + public static TypeRef builtin(int id) { return new TypeRef("builtin:" + id); } + } + public record TypeDeclaration(TypeRef identity, String name, TypeBody body) { + public TypeDeclaration { Objects.requireNonNull(identity); Objects.requireNonNull(body); } + } + public sealed interface TypeBody permits Builtin, Struct, Array, Trait, Fn {} + public record Builtin(int id) implements TypeBody { + public Builtin { + switch (id) { + case 1, 2, 3, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16 -> { } + default -> throw new IllegalArgumentException("unsupported BuiltinId: " + id); + } + } + } + public record Struct(ReadOnlyList fields) implements TypeBody { + public Struct { fields = copy(fields); } + } + public record Array(TypeRef element) implements TypeBody { + public Array { Objects.requireNonNull(element); } + } + public record Trait(ReadOnlyList methods) implements TypeBody { + public Trait { methods = copy(methods); } + } + public record Fn(Signature signature) implements TypeBody { + public Fn { Objects.requireNonNull(signature); } + } + public record NamedType(TypeRef type, String name) { + public NamedType { Objects.requireNonNull(type); } + } + public record Signature(ReadOnlyList parameters, ReadOnlyList results) { + public Signature { parameters = copy(parameters); results = copy(results); } + public static Signature empty() { return new Signature(null, null); } + } + public record Method(String name, Signature signature) { + public Method { Objects.requireNonNull(signature); } + } + public record Implementation(String declarationKey, TypeRef owner, TypeRef trait, ReadOnlyList methods) { + public Implementation { + Objects.requireNonNull(declarationKey); + if (declarationKey.isBlank()) throw new IllegalArgumentException("empty implementation key"); + Objects.requireNonNull(owner); Objects.requireNonNull(trait); + methods = copy(methods); + } + } + public enum Role { + NONE(0), BEFORE_PAUSE(1), BEFORE_RESUME(2), MEMCARD_BOOT(3); + private final int wire; + Role(int wire) { this.wire = wire; } + public int wire() { return wire; } + } + public record FunctionMetadata(Signature signature, ReadOnlyList captures, Role role, String name) { + public FunctionMetadata(Signature signature, ReadOnlyList captures, Role role) { + this(signature, captures, role, null); + } + public FunctionMetadata { + Objects.requireNonNull(signature); captures = copy(captures); Objects.requireNonNull(role); + } + public static FunctionMetadata ordinary(Signature signature) { return new FunctionMetadata(signature, null, Role.NONE); } + public static FunctionMetadata empty() { return ordinary(Signature.empty()); } + } +} diff --git a/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/registry/IRRegistryInstruction.java b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/registry/IRRegistryInstruction.java new file mode 100644 index 00000000..657be458 --- /dev/null +++ b/prometeu-compiler/prometeu-frontend-api/src/main/java/p/studio/compiler/models/registry/IRRegistryInstruction.java @@ -0,0 +1,30 @@ +package p.studio.compiler.models.registry; + +import p.studio.compiler.source.identifiers.CallableId; +import java.util.Objects; + +/** Semantic Registry operands before image linking. No frontend syntax is exposed. */ +public record IRRegistryInstruction(Kind kind, IRRegistry.TypeRef type, IRRegistry.TypeRef targetType, + CallableId function, int member) { + public enum Kind { + MAKE_CLOSURE(0x52), CALL_CLOSURE(0x53), GET_CAPTURE(0x57), CALL_TRAIT(0x58), + DYN_PACK(0x90), DYN_UNPACK(0x91), DYN_CAN_UNPACK(0x92), + REF_IS_EXACT(0x93), REF_IMPLEMENTS(0x94), REF_TO_TRAIT(0x95), REF_CAST(0x96), + STRUCT_NEW(0xa0), STRUCT_GET(0xa1), STRUCT_SET(0xa2), + ARRAY_NEW(0xa3), ARRAY_LEN(0xa4), ARRAY_GET(0xa5), ARRAY_SET(0xa6); + private final int opcode; + Kind(int opcode) { this.opcode=opcode; } + public int opcode(){return opcode;} + } + public IRRegistryInstruction { + Objects.requireNonNull(kind); + if(member<0)throw new IllegalArgumentException("negative member index"); + if(kind!=Kind.GET_CAPTURE)Objects.requireNonNull(type,"type"); + if(kind==Kind.MAKE_CLOSURE) {Objects.requireNonNull(function,"function");function.getIndex();} + else if(function!=null)throw new IllegalArgumentException("unexpected function operand"); + boolean pair=kind.opcode()>=0x93&&kind.opcode()<=0x96; + if(pair)Objects.requireNonNull(targetType,"targetType"); + else if(targetType!=null)throw new IllegalArgumentException("unexpected target type"); + if(kind==Kind.GET_CAPTURE&&type!=null)throw new IllegalArgumentException("capture index is not a TypeId"); + } +} diff --git a/prometeu-compiler/prometeu-frontend-api/src/test/java/p/studio/compiler/models/IRBackendExecutableContractTest.java b/prometeu-compiler/prometeu-frontend-api/src/test/java/p/studio/compiler/models/IRBackendExecutableContractTest.java index 6156c8c7..5009e2b3 100644 --- a/prometeu-compiler/prometeu-frontend-api/src/test/java/p/studio/compiler/models/IRBackendExecutableContractTest.java +++ b/prometeu-compiler/prometeu-frontend-api/src/test/java/p/studio/compiler/models/IRBackendExecutableContractTest.java @@ -1,6 +1,8 @@ package p.studio.compiler.models; import org.junit.jupiter.api.Test; +import p.studio.compiler.models.registry.IRRegistry; +import p.studio.compiler.models.registry.IRRegistryInstruction; import p.studio.compiler.source.Span; import p.studio.compiler.source.identifiers.CallableId; import p.studio.compiler.source.identifiers.FileId; @@ -25,6 +27,26 @@ import java.util.function.Function; import static org.junit.jupiter.api.Assertions.*; class IRBackendExecutableContractTest { + @Test + void registryAggregationRemapsImplementationCallablesWithoutChangingTypeIdentity() { + var owner = new IRRegistry.TypeRef("shared:owner"); + var trait = new IRRegistry.TypeRef("shared:trait"); + var types = ReadOnlyList.from(new IRRegistry.TypeDeclaration(owner, "Owner", new IRRegistry.Struct(null)), + new IRRegistry.TypeDeclaration(trait, "Trait", new IRRegistry.Trait(null))); + var aggregator = IRBackend.aggregator(); + for (int file = 0; file < 2; file++) { + var registry = new IRRegistry(types, ReadOnlyList.from(new IRRegistry.Implementation( + "implementation:" + file, owner, trait, ReadOnlyList.from(new CallableId(0))))); + aggregator.merge(new IRBackendFile(new FileId(file), null, null, null, null, null, null, + ReadOnlyList.from(new CallableSignatureRef("method" + file, 0, "() -> unit")), null, null, registry)); + } + var registry = aggregator.emit().getRegistry(); + assertEquals(4, registry.types().size()); // Linking, not aggregation, canonicalizes declarations. + assertEquals(registry.types().get(0), registry.types().get(2)); + assertEquals(new CallableId(0), registry.implementations().get(0).methods().getFirst()); + assertEquals(new CallableId(1), registry.implementations().get(1).methods().getFirst()); + assertEquals(owner, registry.implementations().get(1).owner()); + } private static final String FORBIDDEN_PBS_PACKAGE = "p.studio.compiler.pbs"; private static final Set> AUDITED_SUPPORT_TYPES = Set.of( FileId.class, @@ -41,6 +63,23 @@ class IRBackendExecutableContractTest { Long.class, Boolean.class); private static final List> PUBLIC_CONTRACT_TYPES = List.of( + IRRegistry.class, + IRRegistryInstruction.class, + IRRegistryInstruction.Kind.class, + IRRegistry.TypeRef.class, + IRRegistry.TypeDeclaration.class, + IRRegistry.TypeBody.class, + IRRegistry.Builtin.class, + IRRegistry.Struct.class, + IRRegistry.Array.class, + IRRegistry.Trait.class, + IRRegistry.Fn.class, + IRRegistry.NamedType.class, + IRRegistry.Signature.class, + IRRegistry.Method.class, + IRRegistry.Implementation.class, + IRRegistry.Role.class, + IRRegistry.FunctionMetadata.class, IRBackend.class, IRBackend.IRBackendAggregator.class, IRBackendReader.class, diff --git a/prometeu-studio-api/src/main/java/p/studio/api/automation/AutomationException.java b/prometeu-studio-api/src/main/java/p/studio/api/automation/AutomationException.java index 5d2cc787..94337889 100644 --- a/prometeu-studio-api/src/main/java/p/studio/api/automation/AutomationException.java +++ b/prometeu-studio-api/src/main/java/p/studio/api/automation/AutomationException.java @@ -7,8 +7,12 @@ public final class AutomationException extends IllegalStateException { private final AutomationFailure failure; public AutomationException(AutomationFailure.Code code, String message) { - super(message); - failure = new AutomationFailure(Objects.requireNonNull(code), message); + this(new AutomationFailure(Objects.requireNonNull(code), message)); + } + + public AutomationException(AutomationFailure failure) { + super(Objects.requireNonNull(failure).message()); + this.failure = failure; } public AutomationFailure failure() { return failure; } diff --git a/prometeu-studio-api/src/main/java/p/studio/api/automation/AutomationFailure.java b/prometeu-studio-api/src/main/java/p/studio/api/automation/AutomationFailure.java index 68388ab5..2a2739db 100644 --- a/prometeu-studio-api/src/main/java/p/studio/api/automation/AutomationFailure.java +++ b/prometeu-studio-api/src/main/java/p/studio/api/automation/AutomationFailure.java @@ -1,9 +1,14 @@ package p.studio.api.automation; import java.util.Objects; +import java.util.Map; +import java.util.List; +import java.util.LinkedHashMap; +import java.util.Collections; /** Stable domain failure codes, independent of HTTP status and JSON-RPC errors. */ -public record AutomationFailure(Code code, String message) { +public record AutomationFailure(Code code, String message, Map runtimeError) { + public AutomationFailure(Code code, String message) { this(code, message, Map.of()); } public enum Code { PROJECT_NOT_OPEN, NOT_GRANTED, INVALID_TASK, BUSY, CLOSED, CLEANUP_FAILED, UNSUPPORTED_LANGUAGE, UNSUPPORTED_CAPABILITY, REFERENCE_NOT_FOUND, DOCUMENTATION_UNAVAILABLE, @@ -11,5 +16,19 @@ public record AutomationFailure(Code code, String message) { RUNTIME_UNAVAILABLE, RUNTIME_INCOMPATIBLE, SESSION_NOT_FOUND, BUILD_NOT_FOUND, NO_FRAME_AVAILABLE, LIMIT_EXCEEDED, NOT_CANCELABLE, INFRASTRUCTURE_FAILURE } - public AutomationFailure { Objects.requireNonNull(code); Objects.requireNonNull(message); } + public AutomationFailure { + Objects.requireNonNull(code); Objects.requireNonNull(message); + var copy = new LinkedHashMap(); + if (runtimeError != null) runtimeError.forEach((key, value) -> copy.put(key, freeze(value))); + runtimeError = Collections.unmodifiableMap(copy); + } + private static Object freeze(Object value) { + if (value instanceof Map map) { + var copy = new LinkedHashMap(); + map.forEach((key, item) -> copy.put(key, freeze(item))); + return Collections.unmodifiableMap(copy); + } + if (value instanceof List list) return list.stream().map(AutomationFailure::freeze).toList(); + return value; + } } diff --git a/tools/pbx-conformance/README.md b/tools/pbx-conformance/README.md new file mode 100644 index 00000000..29e17715 --- /dev/null +++ b/tools/pbx-conformance/README.md @@ -0,0 +1,47 @@ +# Studio PBX producer conformance + +Owner: compiler/general. This harness uses the production runtime loader and VM; +it does not implement a Java approximation of runtime admission. + +From the Studio root: + +```sh +./gradlew :prometeu-compiler:prometeu-build-pipeline:test --tests '*LinkedRegistryTest' --tests '*PbxRegistryConformanceTest' --tests '*TypedCommonPipelineTest' +python3 tools/pbx-conformance/run.py --runtime ../runtime +``` + +The Java tests generate PBX in the build-pipeline module's +`build/registry-conformance`. `run.py` creates a temporary Cargo project, copies +the runtime lockfile and references its bytecode/hal/vm/system crates by path. +It never modifies the runtime checkout. Add `--online` for an uncached machine; +by default Cargo uses only cached dependencies. Java 25, Rust and the runtime +checkout's dependencies are required. + +The output identifies runtime revision, OS/architecture and load configuration. +Positive fixtures distinguish loading from execution. The vec2 signature fixture +is deliberately load-only because its entry requires arguments. Native binding +uses SYSTEM for the positive case and NONE for missing-capability rejection; +all other cases use NONE. Resource tests explicitly lower default limits. + +Negative fixtures mutate Java-produced bytes independently of the writer to +exercise malformed input that the writer correctly refuses to construct. +Initialization JSON comes from `prometeu_system::CrashReport`, and is checked +against the runtime's published headless schema. Saved JSON files remain under +the generated fixture directory. Studio process tests replay selected observed +payloads from `prometeu-app/src/test/resources/runtime` and assert unchanged MCP +projection, including null executed PC and nested verifier context. + +## Known upstream schema defect + +At runtime `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`, the LoadReason Rust enum +includes LifecycleRole, LifecycleSignature, DuplicateLifecycleRole and +NonFiniteValue, absent from the published JSON schema enum. This corpus +observes LifecycleRole and NonFiniteValue. The harness prints an explicit +`UPSTREAM SCHEMA GAP` for those two reasons and verifies that adding exactly +the missing enum value makes the original payload valid. It never alters the +payload or reports those cases as conforming to the unmodified schema. Any +other schema failure remains fatal. Track the runtime correction in AGD75; +remove the exception once the schema is corrected. + +This corpus qualifies common backend production, not PBS lowering, lifecycle +selection, memcard persistence or installed-runtime deployment. diff --git a/tools/pbx-conformance/main.rs b/tools/pbx-conformance/main.rs new file mode 100644 index 00000000..8fb3b9bc --- /dev/null +++ b/tools/pbx-conformance/main.rs @@ -0,0 +1,357 @@ +use prometeu_bytecode::Value; +use prometeu_bytecode::load_limits::{LoadBudget, LoadLimits}; +use prometeu_hal::syscalls::caps; +use prometeu_vm::{ + HostContext, HostReturn, NativeInterface, ValidatedProgramImage, VirtualMachine, +}; + +struct HasCart(usize); +impl NativeInterface for HasCart { + fn syscall( + &mut self, + id: u32, + args: &[Value], + ret: &mut HostReturn, + _: &mut HostContext, + ) -> Result<(), prometeu_hal::vm_fault::VmFault> { + assert_eq!(id, 1); + assert!(args.is_empty()); + self.0 += 1; + ret.push_bool(false); + Ok(()) + } +} +struct NoNative; +impl NativeInterface for NoNative { + fn syscall( + &mut self, + _: u32, + _: &[Value], + _: &mut HostReturn, + _: &mut HostContext, + ) -> Result<(), prometeu_hal::vm_fault::VmFault> { + panic!("fixture unexpectedly invoked native code") + } +} + +fn main() { + let root = std::path::PathBuf::from(std::env::args().nth(1).expect("fixture directory")); + for name in [ + "minimal", + "jump-end", + "vec2-signature", + "closure-capture", + "trait-call", + "array-dynamic", + "common-pipeline", + "roles", + "join", + "memcard-role", + "intrinsic-dot", + ] { + let bytes = std::fs::read(root.join(format!("{name}.pbx"))).unwrap(); + let image = ValidatedProgramImage::load( + &bytes, + &mut LoadBudget::new(LoadLimits::default()), + caps::NONE, + ) + .unwrap_or_else(|e| panic!("{name}: {e:?}")); + println!("LOAD PASS {name}"); + if name != "vec2-signature" { + let mut vm = VirtualMachine::from_image(image, Default::default()); + vm.prepare_boot_call().unwrap(); + let report = vm + .run_budget(1000, &mut NoNative, &mut HostContext::new(None)) + .unwrap(); + let reason = format!("{:?}", report.reason); + assert!( + reason == "Halted" || reason == "EndOfRom", + "{name}: {report:?}" + ); + if name == "closure-capture" || name == "trait-call" { + assert!(matches!(vm.operand_stack_top(1)[0], Value::Int32(42))); + } + if name == "intrinsic-dot" { + assert!( + matches!(vm.operand_stack_top(1)[0], Value::Float(value) if value.to_bits() == 11f64.to_bits()) + ); + } + if name == "array-dynamic" { + assert!(matches!(vm.operand_stack_top(1)[0], Value::Int32(0))); + } + println!( + "EXEC PASS {name}: {reason}, {} steps", + report.steps_executed + ); + } + } + for name in ["native", "unknown-native", "native-abi", "unused-native"] { + let bytes = std::fs::read(root.join(format!("{name}.pbx"))).unwrap(); + if name == "native" { + let image = ValidatedProgramImage::load( + &bytes, + &mut LoadBudget::new(LoadLimits::default()), + caps::SYSTEM, + ) + .unwrap(); + let mut vm = VirtualMachine::from_image(image, Default::default()); + vm.prepare_boot_call().unwrap(); + let mut native = HasCart(0); + let report = vm + .run_budget(1000, &mut native, &mut HostContext::new(None)) + .unwrap(); + assert_eq!(native.0, 1); + assert_eq!(format!("{:?}", report.reason), "Halted"); + println!("BIND + EXEC PASS native with SYSTEM"); + } + let capabilities = if name == "native" { + caps::NONE + } else { + caps::SYSTEM + }; + let error = ValidatedProgramImage::load( + &bytes, + &mut LoadBudget::new(LoadLimits::default()), + capabilities, + ) + .unwrap_err(); + record_fault(&root, name, error); + } + let baseline = std::fs::read(root.join("minimal.pbx")).unwrap(); + for (name, offset, value) in [ + ("version", 4, 1), + ("unsupported-section", 116, 9), + ("unknown-builtin", 178, 0), + ("invalid-role", 136, 4), + ("retired-opcode", 162, 0x54), + ] { + let mut bytes = baseline.clone(); + bytes[offset] = value; + reject(&root, name, &bytes, LoadLimits::default()); + } + for (name, code, max) in [ + ("raw-syscall", vec![0x70, 0, 1, 0, 0, 0, 1, 0], 0), + ("hostcall-index", vec![0x71, 0, 0, 0, 0, 0, 1, 0], 0), + ( + "retired-spawn", + vec![0x54, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0], + 0, + ), + ("retired-yield", vec![0x55, 0, 1, 0], 0), + ("retired-sleep", vec![0x56, 0, 0, 0, 0, 0, 1, 0], 0), + ( + "unreachable-reference", + vec![1, 0, 0x50, 0, 255, 255, 255, 255], + 0, + ), + ( + "nonfinite", + vec![1, 0, 0x15, 0, 0, 0, 0, 0, 0, 0, 0xf0, 0x7f], + 0, + ), + ("jump-immediate", vec![2, 0, 3, 0, 0, 0, 1, 0], 0), + ("zero-max", vec![0x17, 0, 0, 0, 0, 0, 1, 0], 0), + ("return-height", vec![0x17, 0, 0, 0, 0, 0, 0x51, 0], 1), + ("fallthrough", vec![0, 0], 0), + ] { + let mut functions = section(&baseline, 1).to_vec(); + functions[18..22].copy_from_slice(&(code.len() as u32).to_le_bytes()); + functions[26..30].copy_from_slice(&(max as u32).to_le_bytes()); + let bytes = replace(&replace(&baseline, 1, &functions), 2, &code); + reject(&root, name, &bytes, LoadLimits::default()); + } + // Two mutually recursive Array declarations: a structural cycle, with valid IDs. + reject( + &root, + "array-cycle", + &replace( + &baseline, + 6, + &[2, 0, 0, 0, 2, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0], + ), + LoadLimits::default(), + ); + reject( + &root, + "structural-cycle", + &replace( + &baseline, + 6, + &[ + 2, 0, 0, 0, 4, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, + ], + ), + LoadLimits::default(), + ); + reject( + &root, + "invalid-type-reference", + &replace(&baseline, 6, &[1, 0, 0, 0, 2, 0, 99, 0, 0, 0]), + LoadLimits::default(), + ); + reject( + &root, + "invalid-utf8", + &replace( + &baseline, + 6, + &[1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 255, 0x13, 0, 0, 0], + ), + LoadLimits::default(), + ); + reject( + &root, + "removed-input", + &replace(&baseline, 6, &[1, 0, 0, 0, 0, 0, 0, 1, 0, 0]), + LoadLimits::default(), + ); + let mut overlap = baseline.clone(); + overlap[36..40].copy_from_slice(&0u32.to_le_bytes()); + reject(&root, "overlap", &overlap, LoadLimits::default()); + reject( + &root, + "truncated", + &baseline[..baseline.len() - 1], + LoadLimits::default(), + ); + let mut duplicate = baseline.clone(); + duplicate[44..48].copy_from_slice(&0u32.to_le_bytes()); + reject( + &root, + "duplicate-section", + &duplicate, + LoadLimits::default(), + ); + reject( + &root, + "empty-entry", + &replace(&replace(&baseline, 1, &[0, 0, 0, 0]), 8, &[0, 0, 0, 0]), + LoadLimits::default(), + ); + reject( + &root, + "resource-types", + &baseline, + LoadLimits { + types: 0, + ..Default::default() + }, + ); + reject( + &root, + "resource-scratch", + &baseline, + LoadLimits { + temporary_bytes: 0, + ..Default::default() + }, + ); + reject( + &root, + "resource", + &baseline, + LoadLimits { + work: 0, + ..Default::default() + }, + ); +} + +fn reject(root: &std::path::Path, name: &str, bytes: &[u8], limits: LoadLimits) { + let error = + ValidatedProgramImage::load(bytes, &mut LoadBudget::new(limits), caps::NONE).unwrap_err(); + record_fault(root, name, error); +} +fn record_fault(root: &std::path::Path, name: &str, error: prometeu_vm::VmInitError) { + if let prometeu_vm::VmInitError::RegistryLoadFailed(ref failure) = error { + let expected = match name { + "array-cycle" => Some("ArrayCycle"), + "structural-cycle" => Some("StructuralTypeCycle"), + "invalid-type-reference" => Some("TypeId"), + "invalid-utf8" => Some("Utf8"), + "removed-input" => Some("Builtin"), + "retired-spawn" | "retired-yield" | "retired-sleep" => Some("RemovedOpcode"), + "nonfinite" => Some("NonFiniteValue"), + "unreachable-reference" => Some("FunctionId"), + "return-height" => Some("ReturnHeight"), + "fallthrough" => Some("UnterminatedPath"), + "zero-max" => Some("StackOverflow"), + "duplicate-section" => Some("DuplicateSection"), + _ => None, + }; + if let Some(reason) = expected { + assert_eq!(format!("{:?}", failure.reason), reason, "{name}"); + } + } + let report = prometeu_system::CrashReport::VmInit { error }; + assert_eq!(report.pc(), None); + let fault = serde_json::json!({"kind":"vm_init", "pc":null, "summary":"Conformance fixture", "initError":report.init_diagnostic().unwrap()}); + let full: serde_json::Value = serde_json::from_slice( + &std::fs::read(std::env::args().nth(2).expect("schema path")).unwrap(), + ) + .unwrap(); + let schema = jsonschema::validator_for( + &serde_json::json!({"$defs":full["$defs"],"$ref":"#/$defs/fault"}), + ) + .unwrap(); + if !schema.is_valid(&fault) { + let reason = fault["initError"]["load"]["reason"].as_str().unwrap(); + assert!( + ["LifecycleRole", "NonFiniteValue"].contains(&reason), + "unexpected schema failure: {fault}" + ); + // Validate the ORIGINAL payload with only the observed missing enum value added. + // This is an explicit upstream defect report, not a claim of published-schema conformance. + let mut corrected = full.clone(); + corrected["$defs"]["loadFailure"]["properties"]["reason"]["enum"] + .as_array_mut() + .unwrap() + .push(serde_json::json!(reason)); + jsonschema::validator_for( + &serde_json::json!({"$defs":corrected["$defs"],"$ref":"#/$defs/fault"}), + ) + .unwrap() + .validate(&fault) + .unwrap(); + println!( + "UPSTREAM SCHEMA GAP: {reason} missing from published enum; original payload preserved" + ); + } + std::fs::write( + root.join(format!("fault-{name}.json")), + serde_json::to_vec_pretty(&fault).unwrap(), + ) + .unwrap(); + println!("REJECT PASS {name}: {}", fault["initError"]); +} + +fn word(bytes: &[u8], at: usize) -> usize { + u32::from_le_bytes(bytes[at..at + 4].try_into().unwrap()) as usize +} +fn section(bytes: &[u8], kind: usize) -> &[u8] { + for i in 0..word(bytes, 8) { + let at = 32 + 12 * i; + if word(bytes, at) == kind { + let offset = word(bytes, at + 4); + return &bytes[offset..offset + word(bytes, at + 8)]; + } + } + panic!("missing section {kind}") +} +fn replace(bytes: &[u8], kind: usize, payload: &[u8]) -> Vec { + let count = word(bytes, 8); + let mut output = bytes[..32 + 12 * count].to_vec(); + for i in 0..count { + let at = 32 + 12 * i; + let body = if word(bytes, at) == kind { + payload + } else { + section(bytes, word(bytes, at)) + }; + let offset = output.len() as u32; + output[at + 4..at + 8].copy_from_slice(&offset.to_le_bytes()); + output[at + 8..at + 12].copy_from_slice(&(body.len() as u32).to_le_bytes()); + output.extend_from_slice(body); + } + output +} diff --git a/tools/pbx-conformance/run.py b/tools/pbx-conformance/run.py new file mode 100644 index 00000000..125518d4 --- /dev/null +++ b/tools/pbx-conformance/run.py @@ -0,0 +1,27 @@ +"""Build a production-loader harness without writing into the runtime checkout.""" +import argparse +import json +import pathlib +import platform +import subprocess +import tempfile + +parser = argparse.ArgumentParser() +parser.add_argument("--runtime", type=pathlib.Path, default=pathlib.Path("../runtime")) +parser.add_argument("--fixtures", type=pathlib.Path, default=pathlib.Path("prometeu-compiler/prometeu-build-pipeline/build/registry-conformance")) +parser.add_argument("--online", action="store_true", help="Allow Cargo to fetch uncached dependencies") +args = parser.parse_args() +runtime = args.runtime.resolve() +print("runtime", subprocess.check_output(["git", "-C", str(runtime), "rev-parse", "HEAD"], text=True).strip(), flush=True) +print("platform", platform.platform(), "limits=default capabilities=NONE", flush=True) +with tempfile.TemporaryDirectory(prefix="pbx-conformance-") as directory: + root = pathlib.Path(directory) + manifest = '[package]\nname="studio-pbx-conformance"\nversion="0.1.0"\nedition="2024"\n[dependencies]\n' + for crate in ["prometeu-bytecode", "prometeu-hal", "prometeu-vm", "prometeu-system"]: + manifest += f'{crate}={{path={json.dumps(str(runtime / "crates/console" / crate))}}}\n' + manifest += 'serde_json="1.0.149"\njsonschema={version="0.33",default-features=false}\n' + (root / "Cargo.toml").write_text(manifest) + (root / "Cargo.lock").write_text((runtime / "Cargo.lock").read_text()) + (root / "src").mkdir() + (root / "src/main.rs").write_text(pathlib.Path(__file__).with_name("main.rs").read_text()) + subprocess.run(["cargo", "run", *([] if args.online else ["--offline"]), "--manifest-path", str(root / "Cargo.toml"), "--", str(args.fixtures.resolve()), str(runtime / "devtools/headless-protocol/response.schema.json")], check=True)