feat(compiler): implement typed registry and production PBX conformance (AGD76)

This commit is contained in:
2026-10-14 21:04:04 +01:00
parent a9ede541af
commit bd5694dbdf
54 changed files with 2298 additions and 357 deletions
@@ -0,0 +1,145 @@
---
id: LSN-0079
ticket: compiler-pbx-registry-conformance
title: Typed common IR through production PBX admission
created: 2026-10-14
tags: [runtime-integration]
---
## Original problem and ownership
Owner: compiler/general; interfaces: compiler/pbs and studio.
The old producer carried parameter/result slot counts without complete semantic
types and emitted a retired PBX function layout. A writer-only repair could not
recover signatures, distinguish nominal identities or describe closures and
trait implementations. The fix had to cross the entire common producer chain.
This lesson consolidates AGD-0076, accepted DEC-0063 and completed PLN-0142
(DSC-0074). Those workflow artifacts were removed by housekeeping after their
normative content and evidence were propagated. The active integration umbrella
is [AGD75](../../workflow/agendas/AGD-0075-studio-runtime-readiness.md).
## Consolidated decisions
Use stable declaration references before assigning deterministic dense image
IDs. Preserve Struct/Trait nominal identity; canonicalize Builtin imports,
Array elements and Fn signatures. Names are descriptive, not nominal keys.
Parameters, results, captures, implementations and lifecycle roles are explicit
common values, preserved through IRVM, optimization and emission.
The common IR remains serializable by design, but a concrete codec was
explicitly deferred. Complete representation does not require selecting a
transport format. References avoid recursive Java object graphs and the public
contract audit rejects mutable or frontend-specific shapes.
PBS is one frontend. Optional and Result already belong to PBS's normative
language contract and have parser/semantic implementations. Defects in their
executable lowering do not justify new backend categories or rejection rules.
Their implementation and carrier choices remain frontend-owned. The user
explicitly scoped consumer migration to subsequent tickets and a single AGD75
integrated delivery. No PBS semantics were added to this backend change.
## Final implementation
- IRRegistry and FunctionMetadata represent all five type categories, ordered
signatures/captures, roles and implementations. Aggregation remaps callable
references while preserving stable type identity.
- LinkedRegistry canonicalizes imports/structural declarations deterministically
with an iterative dependency traversal. Nominal anchors permit recursion;
purely structural cycles reject.
- Lowering derives widths from semantic signatures, links typed operands and
remaps implementation targets. IRVM coherence and optimization preserve the
Registry and function metadata.
- The PBX writer emits the 32-byte header and all eight mandatory sections,
optional debug, strict UTF-8, exact function records and explicit empty capture
environments. Width caches are not serialized. Constant deduplication now
relocates PUSH_CONST indices rather than silently changing their meanings.
- Local verification decodes all function ranges, validates references and
operand categories, forbids retired/raw opcodes and analyzes reachable stack
and type facts to convergence. Conflicts become Unknown; exact semantic groups
retain provenance when available. Unknown remains subject to runtime checks.
- Production loading remains authoritative for native identity/capability/ABI,
runtime budgets and executable admission.
- Studio previously discarded error.details when a headless response rejected
boot. AutomationFailure now preserves the entire runtime error envelope,
including initialization, binding and resource details, through MCP output.
## Examples and pitfalls
A `vec2 -> vec2` function has logical arity one/one and derived slot widths two/two.
The function record writes the two TypeId lists, not the numbers two/two. A test
compares that record with independent fixed bytes from the runtime contract.
A capturing body is not a direct-call target. MAKE_CLOSURE names the body and
expected Fn type, consumes its explicit capture groups and produces a callable.
CALL_CLOSURE consumes that callable after the explicit arguments. The synthetic
frontend test proves this path without PBS source, SDK or language ASTs.
Jumps contain absolute offsets from the current function start. They are not
relative to the next PC. A taken jump to function end is a return boundary;
ordinary fallthrough is invalid. These details were checked against the actual
runtime verifier, not inferred from older emitter behavior.
Do not derive type identity from slot width, spelling or display names. Do not
claim a self-round-trip proves external conformance. Do not treat parser tests
as execution evidence or a load-only argument-taking entry as a boot test.
Initialization diagnostics have null outer executed PC. A verifier PC remains
nested in initError.load.pc; file offsets remain offsets. Preserve the original
JSON structure rather than flattening it into human-readable messages.
## Qualification evidence
Runtime: `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`.
Platform: macOS 27.0.1 arm64. Rust harness uses the runtime Cargo.lock, default
load limits and NONE capabilities except the positive native case (SYSTEM).
Resource-negative cases explicitly lower Types, TemporaryBytes or Work.
Runtime checkout remained unchanged.
| Gate | Result |
| --- | --- |
| Build-pipeline suite | 178 tests: 163 passed, 15 skipped, no failures |
| Frontend-api suite | 28 passed |
| PBS suite | 245 passed; not a claim of complete executable migration |
| Headless process suite | 9 passed, including six observed runtime fault payloads |
| Architecture suite | 17 passed |
| Independent binary oracle | Exact 194-byte minimal image and fixed vec2 function bytes |
| Production loader positives | 12 admitted images, including native binding |
| Genuine execution | 11 programs: direct/common path, closures/captures, trait, Array/Dynamic, joins, roles, intrinsic and native; vec2 signature is load-only |
| Production negative corpus | 32 rejected malformed/binding/code/resource fixtures |
| Workflow and patch integrity | discussion validate and git diff --check passed |
The 15 skipped tests are older conditional integration adapters. The dedicated
production harness above ran explicitly; their skip state is not substituted
for runtime evidence. The intrinsic/native catalogue goldens are serialization
fixtures and do not claim current PBS catalogue compatibility.
Reproduce with the commands in
[the harness README](../../../tools/pbx-conformance/README.md). The harness
creates its Cargo build outside the runtime checkout and saves generated PBX and
observed JSON under the build-pipeline build directory.
## Remaining integration work
The runtime LoadReason enum has four values missing from the published headless
schema: LifecycleRole, LifecycleSignature, DuplicateLifecycleRole and
NonFiniteValue. The corpus observes the first and last. Both payloads fail the
original schema solely because of the missing enum value. The harness explicitly
reports this upstream gap, validates the original payload with that single enum
addition, and leaves the payload unchanged. It does not claim those two cases
conform to the unmodified schema. Runtime must correct the schema under AGD75.
PBS typed producer migration, Optional/Result executable lowering, catalogue
alignment, installed runtime delivery, hook selection, memcard gating and
persistence remain with their consumer tickets. The common schema and synthetic
producer work without those features. There is no legacy PBX fallback and no
slot-to-type inference to conceal an incomplete handoff.
## References
- [Typed common Registry/PBX specification](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md)
- [Backend conformance matrix](../../../docs/specs/compiler/22.%20Backend%20Spec-to-Test%20Conformance%20Matrix.md)
- [Runtime qualification harness](../../../tools/pbx-conformance/README.md)
- Runtime sources: registry_v0.rs, program_image.rs, registry_verifier, and
devtools/headless-protocol/response.schema.json at the revision above.
@@ -0,0 +1,182 @@
# Typed common Registry and PBX v0
Status: normative. Owner: `compiler/general`. Runtime contract reference:
`runtime` revision `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`,
`docs/specs/runtime/02c-program-registry-and-pbx-v0.md` and
`docs/vm-arch/ISA_CORE.md` in that repository.
## Ownership and handoff
Every executable frontend MUST provide its semantic types through the common
`IRRegistry` model and typed `FunctionMetadata`. The backend MUST NOT inspect
language ASTs, PBS names, Optional/Result syntax, or frontend-specific classes.
Language carriers and their lowering remain owned by the frontend. A carrier
can use ordinary Struct, Array, Trait, Fn and Builtin declarations without
introducing a language category in the backend.
The public model consists of immutable values and ordered `ReadOnlyList`
collections. References use explicit declaration keys and callable IDs, not
object identity or recursive object graphs. It is serializable by design;
no concrete IR codec, wire format or persistence protocol is selected.
A function declares ordered named parameters, ordered logical results, ordered
captures, a lifecycle role and an optional name. Empty names differ from absent
names. Captures are an environment, never hidden parameters. Slot counts are
derived from types; existing count fields are checked caches, not semantic
input. Missing nonempty type metadata is an invalid handoff, not a request to
infer types from counts. Optimizers MUST preserve metadata and typed operands.
## Identity and linking
TypeId, FunctionId and ImplementationId are separate dense image-local index
domains. Index zero is valid. Trait references use TypeIds; field and method
indices are local to their owner. Source declaration keys MUST identify the
same declaration across repeated module imports and distinguish unrelated
nominal declarations, even when names and shapes coincide.
Linking is deterministic. The selected physical entry is FunctionId zero;
remaining functions follow canonical module/callable/source ordering. Struct
and Trait retain nominal identity. Builtin imports deduplicate by BuiltinId,
Array by canonical element identity, and Fn by ordered parameter/result type
identities, excluding names. Forward references and recursion through nominal
anchors are valid. Pure Array/Fn cycles are invalid. Every reference must
resolve before emission. Frontends should supply only used imports.
| BuiltinId | Type | Slots |
| --- | --- | --- |
| `0x0001` | color | 1 |
| `0x0002` | vec2 | 2 |
| `0x0003` | pixel | 3 |
| `0x0010` | Int32 | 1 |
| `0x0011` | Int64 | 1 |
| `0x0012` | Float | 1 |
| `0x0013` | Bool | 1 |
| `0x0014` | String | 1 |
| `0x0015` | Dynamic | 2 |
| `0x0016` | Random | 1 |
The retired input IDs `0x0100`–`0x0102` have no aliases. Struct, Array and Fn
values occupy one reference slot; Trait values occupy two slots. Width does
not establish type identity or authorize conversion.
## Binary contract
All fields are little-endian. The header is 32 bytes: `PBX\0`, u16 version zero,
zero endian/reserved bytes, u32 section count and twenty zero reserved bytes.
Directory entries are u32 kind, file-relative offset and length. Ranges must
not overlap or overflow. The version remains zero; no legacy fallback exists.
Mandatory sections are constants (0), functions (1), Code (2), exports (4),
syscalls (5), types (6), implementations (7) and captures (8). Debug (3) is
optional. Empty tables contain a u32 zero count. Code is raw bytes, not a
counted table. An empty table set is serializable but is not an executable
program: production loading requires a valid ordinary entry.
Lists have u32 counts. Optional names use u8 presence 0/1, then u32 UTF-8 byte
length and bytes when present. UTF-8 is strict. Named types contain TypeId then
optional name; results and captures contain TypeIds without names.
| Record | Ordered payload |
| --- | --- |
| Type | u8 tag, optional name, category payload |
| Builtin (tag 0) | u32 BuiltinId |
| Struct (tag 1) | named fields |
| Array (tag 2) | u32 element TypeId |
| Trait (tag 3) | methods: optional name, parameters, results |
| Fn (tag 4) | parameters, results |
| Function | u8 role, optional name, parameters, results, u32 Code offset, length, extra locals, max temporaries |
| Implementation | u32 owner TypeId, trait TypeId, method count, ordered FunctionIds |
| Captures | function count and one ordered TypeId list per function, including empty lists |
Function Code offsets are Code-relative. Parameter/result slot counts MUST NOT
be serialized. Max temporaries zero means zero. Signature names are independent
of debug names. Constants retain tags Null 0, Int64 1, Float64 2, Boolean 3,
String 4 and Int32 5. Float payloads must be finite. Constant deduplication
requires relocating every PUSH_CONST index; changing only the table is invalid.
Roles are None 0, BeforePause 1, BeforeResume 2 and MemcardBoot 3. Nonzero roles
are unique, cannot target entry zero and cannot capture. Pause/resume have empty
signatures; MemcardBoot has `(Int64) -> Int32`. This metadata does not implement
frontend hook selection, mem.bootstrap gating or persistence.
Implementations are unique per Struct/Trait pair, complete and ordered. Each
method target has an exact signature with a concrete Struct receiver first,
followed by the trait method's explicit parameters. Targets cannot capture.
## Instructions and verification
Instructions use u16 opcodes. Jump operands are absolute u32 byte offsets from
the current function start, not displacements from the next instruction.
A taken jump to the current function end is a return boundary; ordinary
fallthrough past the end is invalid.
| Opcode | Typed operands |
| --- | --- |
| MAKE_CLOSURE `0x52` | FunctionId, Fn TypeId |
| CALL_CLOSURE `0x53` | expected Fn TypeId |
| GET_CAPTURE `0x57` | capture group index |
| CALL_TRAIT `0x58` | Trait TypeId, method index |
| DYN_PACK/UNPACK/CAN_UNPACK `0x90`–`0x92` | TypeId |
| Reference operations `0x93`–`0x96` | source TypeId, target TypeId |
| STRUCT_NEW `0xa0` | Struct TypeId |
| STRUCT_GET/SET `0xa1`–`0xa2` | Struct TypeId, field index |
| Array operations `0xa3`–`0xa6` | Array TypeId |
Direct CALL cannot target a capturing body. Closure signatures match exactly.
Retired SPAWN/YIELD/SLEEP (`0x54`–`0x56`) and raw producer SYSCALL are invalid,
even when unreachable. All function ranges must decode completely; references,
categories, operand boundaries, finite immediates and branch destinations are
checked even in unreachable instructions.
Reachable analysis uses a bounded worklist. Parameters initialize local facts;
extra locals start Null and may change type. Joins require equal temporary
heights. Equal facts survive; conflicting facts widen to Unknown. Type errors
are diagnosed after convergence. Unknown alone neither proves compatibility
nor justifies rejection. Runtime group checks remain authoritative. RET and
jump-to-end require exact result width and compatible known facts. GET/SET_LOCAL
address only parameters plus extra locals. No implicit numeric, trait or Dynamic
conversion is introduced by linking, writing or verification.
HOSTCALL references a declared canonical `(module, name, version)` identity
with exact argument/result slot ABI. Duplicate identities with conflicting ABI,
unused declarations and invalid indices are rejected. Only the runtime loader
binds capabilities and rewrites HOSTCALL to SYSCALL. Intrinsic signatures are
explicit compiler metadata; their IDs and definitive ABI remain runtime-owned.
## Bounds and diagnostics
Product ceilings: PBX 64 MiB, Code 32 MiB, 65,536 types and functions each,
262,144 implementations, 1,048,576 nested records, names 4,096 bytes individually
and 4 MiB aggregate, frame 16,384 slots and results 1,024 slots. Runtime admission
also owns retained Registry 64 MiB, scratch 128 MiB and work 268,435,456. A guest
cannot raise these limits. Java validation is not a substitute for runtime
allocation accounting or final `ValidatedProgramImage::load` admission.
Compilation failures use common LOWER_IRVM/MARSHAL stage diagnostics. Load,
binding and resource failures remain distinct from executed traps. Studio
preserves the full runtime error envelope in `AutomationFailure.runtimeError`,
including null fields and native binding details. The outer initialization
fault PC remains null. A verifier PC stays in `initError.load.pc`; a file offset
MUST NOT become an executed PC.
## Reproducible conformance
Run the focused common tests, then the production harness:
```sh
./gradlew :prometeu-compiler:prometeu-build-pipeline:test --tests '*LinkedRegistryTest' --tests '*PbxRegistryConformanceTest' --tests '*TypedCommonPipelineTest'
python3 tools/pbx-conformance/run.py --runtime ../runtime
```
The Java tests write `build/registry-conformance` inside the build-pipeline
module. The harness builds outside the runtime checkout using its Cargo.lock,
loads Java-produced bytes, executes admitted zero-argument fixtures, validates
structured failures against the published schema and records the runtime hash,
platform, capabilities and limits. Parsing, loading and execution are separate
claims. The vec2 signature fixture is load-only; it is not a zero-argument boot.
The fixed 194-byte minimal image comes from the independent runtime test oracle.
Synthetic common IR additionally exercises linking, lowering, optimization and
emission without PBS. Frontend migration and integrated release readiness are
tracked separately in AGD75; passing common conformance does not claim complete
PBS lowering or complete Studio/runtime integration.
@@ -1,5 +1,10 @@
# Bytecode and PBX Mapping Specification
The current typed Registry/PBX v0 contract is normative in
[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md).
Its complete typed function records, mandatory sections and verification rules
replace the earlier slot-only artifact baseline.
Status: Draft v1 (Backend Baseline)
Applies to: mapping from lowered/optimized backend programs into PBX sections, bytecode-facing artifacts, and source-to-artifact invariants required by loader/verifier/runtime
@@ -74,7 +79,8 @@ Emitter output must map to a `BytecodeModule` shape containing:
3. `code`,
4. `exports`,
5. `syscalls`,
6. optional `debug_info` that still satisfies v1 minimum debug obligations.
6. typed Registry declarations, implementations and per-function capture lists,
7. optional `debug_info` that still satisfies v1 minimum debug obligations.
### 6.2 Function ordering and IDs
@@ -168,7 +174,6 @@ At minimum, artifact conformance checks should assert:
The following remain deferred:
- richer optional debug/source-map formats,
- additional PBX section-level contracts beyond current baseline,
- and profile-specific binary compatibility policy details beyond current v1 baseline.
## 11. Non-Goals
@@ -1,5 +1,10 @@
# Compiler Verification and Safety Checks Specification
The current typed Registry/PBX v0 contract is normative in
[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md).
Its complete typed function records, mandatory sections and verification rules
replace the earlier slot-only artifact baseline.
Status: Draft v1 (Quality-Gate Baseline)
Applies to: test-backed safety gates for common compiler stages, validation ownership across executable frontends, and integration regression safety checks against runtime
@@ -1,5 +1,10 @@
# IRBackend to IRVM Lowering Specification
The current typed Registry/PBX v0 contract is normative in
[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md).
Its complete typed function records, mandatory sections and verification rules
replace the earlier slot-only artifact baseline.
Status: Draft v1 (Backend Baseline)
Applies to: executable-backend lowering from the common `IRBackend` handoff into `IRVM`
@@ -1,5 +1,10 @@
# Backend Spec-to-Test Conformance Matrix
The current typed Registry/PBX v0 contract is normative in
[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md).
Its complete typed function records, mandatory sections and verification rules
replace the earlier slot-only artifact baseline.
Status: Draft v1 (Traceability Baseline)
Applies to: compiler/backend conformance traceability for canonical stage order and entrypoint-specific contracts
@@ -140,3 +145,19 @@ Matrix integrity is enforced by automated document lint:
- if `PROMETEU_MATRIX_HARD_GATE=true` and `PROMETEU_CHANGED_FILES` contains a path under conformance-relevant FE/BE prefixes,
- then this matrix file MUST be present in `PROMETEU_CHANGED_FILES`, otherwise the gate fails.
- CI policy uses strict Gate I mode via `PROMETEU_GATE_I_STRICT=true` (or `CI=true`) and requires `PROMETEU_RUNTIME_CHECK_CMD` for runtime-backed evidence.
## 7. Typed Registry/PBX v0 qualification (2026-10-14)
Owner: compiler/general. These rows supersede slot-only PBX coverage claims;
frontend syntax and lowering remain consumer-owned.
| Requirement | Evidence | Status | Scope |
| --- | --- | --- | --- |
| Stable declarations, nominal identity, structural deduplication and cycles | LinkedRegistryTest; IRBackendExecutableContractTest | pass | Common API and linking |
| Complete typed signatures, captures, roles and implementations | PbxRegistryConformanceTest; TypedCommonPipelineTest | pass | Common IR through emission |
| Exact PBX mandatory tables and multi-slot signatures | Independent 194-byte oracle; fixed vec2 function bytes | pass | Wire format, not a self round-trip |
| Runtime final admission and execution | tools/pbx-conformance/run.py | pass | Closure, trait, array, Dynamic, intrinsic, roles, joins, native binding; vec2 load-only |
| Invalid structures, references, code and resource refusal | tools/pbx-conformance/main.rs mutation corpus | pass | Production loader, default/lowered limits |
| Structured initialization and binding diagnostics | HeadlessProcessSessionTest; observed runtime JSON fixtures | pass | Preserves full payload through MCP, including null outer PC |
| Published schema covers current runtime reasons | Actual LifecycleRole/NonFiniteValue payloads against runtime schema | partial | Runtime enum/schema mismatch recorded in AGD75; no payload rewriting |
| PBS migration and integrated delivery | AGD75 and consumer tickets | partial | Existing PBS suite passing is not complete executable lowering evidence |
@@ -245,7 +245,9 @@ final class HeadlessProcessSession implements HeadlessGateway.Session {
default -> AutomationFailure.Code.INFRASTRUCTURE_FAILURE;
};
pending.remove(item.id);
item.result.completeExceptionally(new AutomationException(code, "Runtime " + error.path("code").asText() + ": " + error.path("message").asText()));
Map<String, Object> runtimeError = mapper.convertValue(error, new com.fasterxml.jackson.core.type.TypeReference<>() {});
item.result.completeExceptionally(new AutomationException(new AutomationFailure(code,
"Runtime " + error.path("code").asText() + ": " + error.path("message").asText(), runtimeError)));
} else {
var result = message.get("result");
if (item.command.equals("cancel") && (!result.path("requested").isBoolean() || !result.path("requested").booleanValue()
@@ -50,7 +50,15 @@ public final class HeadlessFixture {
var request = MAPPER.readTree(line); String id = request.path("requestId").asText(), command = request.path("command").asText();
switch (command) {
case "initialize" -> response(id, MAPPER.createObjectNode().setAll(Map.of("protocolVersion", hello.path("protocolVersion"), "capabilities", hello.path("capabilities"))));
case "boot", "reset" -> response(id, state());
case "boot", "reset" -> {
if (mode.startsWith("load-fault-")) {
var resource = "/runtime/fault-" + mode.substring("load-fault-".length()) + ".json";
var fault = MAPPER.readTree(HeadlessFixture.class.getResourceAsStream(resource));
var error = MAPPER.createObjectNode().put("code", "boot_failed").put("message", "Runtime rejected cartridge");
error.putObject("details").set("fault", fault);
emit(MAPPER.createObjectNode().put("type", "response").put("sessionId", "fixture").put("requestId", id).set("error", error));
} else response(id, state());
}
case "get_state" -> { if (!mode.equals("hang-call")) response(id, state()); }
case "get_fault" -> response(id, MAPPER.nullNode());
case "advance" -> {
@@ -18,6 +18,20 @@ class HeadlessProcessSessionTest {
var session = new HeadlessProcessSession(process, new ObjectMapper(), HeadlessFixture.limits(mode), "build"); sessions.add(session); return session;
}
@AfterEach void cleanup() { sessions.forEach(session -> session.abort("test cleanup")); }
@Test void productionLoadDiagnosticsSurviveBootFailureAndMcpSerialization() throws Exception {
var mapper = new ObjectMapper();
for (String fixture : List.of("resource", "invalid-role", "retired-opcode", "native", "native-abi", "unknown-native")) {
var session = start("load-fault-" + fixture);
var failure = assertThrows(AutomationException.class, () -> session.initializeAndBoot(root)).failure();
var fault = mapper.readTree(getClass().getResourceAsStream("/runtime/fault-" + fixture + ".json"));
var error = mapper.valueToTree(failure.runtimeError());
assertEquals(fault, error.path("details").path("fault"));
assertTrue(error.path("details").path("fault").path("pc").isNull());
var wire = p.studio.app.mcp.McpToolResult.failure(mapper, failure).wire();
assertEquals(fault, wire.path("structuredContent").path("error").path("runtimeError").path("details").path("fault"));
assertThrows(UnsupportedOperationException.class, () -> failure.runtimeError().put("code", "changed"));
}
}
@Test void fragmentedProtocolPreservesUnsignedFrameIdentityLayoutAndSeparateStderr() throws Exception {
var session = start("normal"); session.initializeAndBoot(root);
var outcome = session.advance(3, ignored -> {});
@@ -0,0 +1,24 @@
{
"initError": {
"code": "registry_load_failed",
"load": {
"code": "LOAD_INVALID_REGISTRY",
"function": 0,
"id_domain": null,
"limit": null,
"member": null,
"offset": 136,
"pc": null,
"reason": "LifecycleRole",
"record": 0,
"referenced_id": null,
"requested": null,
"resource": null,
"section": 1
},
"message": "RegistryLoadFailed(LoadFailure { code: InvalidRegistry, reason: LifecycleRole, section: Some(1), offset: Some(136), record: Some(0), member: None, id_domain: None, referenced_id: None, function: Some(0), pc: None, resource: None, requested: None, limit: None })"
},
"kind": "vm_init",
"pc": null,
"summary": "Conformance fixture"
}
@@ -0,0 +1,19 @@
{
"initError": {
"binding": {
"declared_arg_slots": 1,
"declared_ret_slots": 1,
"expected_arg_slots": 0,
"expected_ret_slots": 1,
"module": "system",
"name": "has_cart",
"reason": "abi_mismatch",
"version": 1
},
"code": "loader_patch_failed",
"message": "LoaderPatchFailed(ResolveFailed(AbiMismatch { module: \"system\", name: \"has_cart\", version: 1, declared_arg_slots: 1, declared_ret_slots: 1, expected_arg_slots: 0, expected_ret_slots: 1 }))"
},
"kind": "vm_init",
"pc": null,
"summary": "Conformance fixture"
}
@@ -0,0 +1,17 @@
{
"initError": {
"binding": {
"module": "system",
"name": "has_cart",
"provided": 0,
"reason": "missing_capability",
"required": 1,
"version": 1
},
"code": "loader_patch_failed",
"message": "LoaderPatchFailed(ResolveFailed(MissingCapability { required: 1, provided: 0, module: \"system\", name: \"has_cart\", version: 1 }))"
},
"kind": "vm_init",
"pc": null,
"summary": "Conformance fixture"
}
@@ -0,0 +1,24 @@
{
"initError": {
"code": "registry_load_failed",
"load": {
"code": "LOAD_RESOURCE_LIMIT",
"function": null,
"id_domain": null,
"limit": 0,
"member": null,
"offset": null,
"pc": null,
"reason": "Limit",
"record": null,
"referenced_id": null,
"requested": 32,
"resource": "Work",
"section": null
},
"message": "RegistryLoadFailed(LoadFailure { code: ResourceLimit, reason: Limit, section: None, offset: None, record: None, member: None, id_domain: None, referenced_id: None, function: None, pc: None, resource: Some(Work), requested: Some(32), limit: Some(0) })"
},
"kind": "vm_init",
"pc": null,
"summary": "Conformance fixture"
}
@@ -0,0 +1,24 @@
{
"initError": {
"code": "registry_load_failed",
"load": {
"code": "LOAD_INVALID_CODE",
"function": 0,
"id_domain": null,
"limit": null,
"member": null,
"offset": null,
"pc": 0,
"reason": "Truncated",
"record": null,
"referenced_id": null,
"requested": null,
"resource": null,
"section": 2
},
"message": "RegistryLoadFailed(LoadFailure { code: InvalidCode, reason: Truncated, section: Some(2), offset: None, record: None, member: None, id_domain: None, referenced_id: None, function: Some(0), pc: Some(0), resource: None, requested: None, limit: None })"
},
"kind": "vm_init",
"pc": null,
"summary": "Conformance fixture"
}
@@ -0,0 +1,15 @@
{
"initError": {
"binding": {
"module": "system",
"name": "missing",
"reason": "unknown_syscall",
"version": 1
},
"code": "loader_patch_failed",
"message": "LoaderPatchFailed(ResolveFailed(UnknownSyscall { module: \"system\", name: \"missing\", version: 1 }))"
},
"kind": "vm_init",
"pc": null,
"summary": "Conformance fixture"
}
@@ -6,6 +6,7 @@ import java.io.ByteArrayOutputStream;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.Objects;
@@ -48,8 +49,11 @@ public class BytecodeEmitter {
final EmissionPlan plan) {
final var inputPlan = plan == null ? EmissionPlan.empty() : plan;
final var canonicalConstPool = dedupeConstPool(inputPlan.constPool());
final var constantIndices = new HashMap<BytecodeModule.ConstantPoolEntry, Integer>();
for (int i=0;i<canonicalConstPool.size();i++) constantIndices.put(canonicalConstPool.get(i),i);
final var canonicalExports = dedupeExports(inputPlan.exports());
final var functionFragments = new ArrayList<BytecodeFunctionLayoutBuilder.FunctionFragment>(inputPlan.functions().size());
final var intrinsicSignatures = new LinkedHashMap<Integer, BytecodeModule.IntrinsicSignature>();
final var orderedSyscalls = new LinkedHashMap<SyscallIdentity, BytecodeModule.SyscallDecl>();
final var syscallIndexByIdentity = new LinkedHashMap<SyscallIdentity, Integer>();
@@ -59,6 +63,12 @@ public class BytecodeEmitter {
for (final var op : function.operations()) {
final var pc = code.size();
switch (op.kind()) {
case REGISTRY -> {
var instruction = Objects.requireNonNull(op.registryInstruction());
writeOpU32(code, instruction.op().opcode(), instruction.immediate());
if (instruction.secondImmediate() != null) code.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(instruction.secondImmediate()).array());
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
}
case HALT -> {
writeOpNoImm(code, OP_HALT);
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
@@ -72,7 +82,7 @@ public class BytecodeEmitter {
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
}
case PUSH_CONST -> {
writeOpU32(code, OP_PUSH_CONST, op.immediate());
writeOpU32(code, OP_PUSH_CONST, constantIndices.get(inputPlan.constPool().get(op.immediate())));
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
}
case POP -> {
@@ -176,6 +186,10 @@ public class BytecodeEmitter {
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
}
case INTRINSIC -> {
if(op.expectedArgSlots()==null||op.expectedRetSlots()==null) throw new IllegalArgumentException("intrinsic signature is required");
var signature = new BytecodeModule.IntrinsicSignature(op.immediate(),op.expectedArgSlots(),op.expectedRetSlots());
var previous = intrinsicSignatures.putIfAbsent(op.immediate(),signature);
if(previous!=null&&!previous.equals(signature)) throw new IllegalArgumentException("conflicting intrinsic ABI");
writeOpU32(code, OP_INTRINSIC, op.immediate());
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
}
@@ -197,6 +211,7 @@ public class BytecodeEmitter {
syscallIndexByIdentity.put(identity, syscallIndexByIdentity.size());
orderedSyscalls.put(identity, decl);
}
if (!orderedSyscalls.get(identity).equals(decl)) throw new IllegalArgumentException("conflicting HOSTCALL ABI for " + identity);
final var index = syscallIndexByIdentity.get(identity);
writeOpU32(code, OP_HOSTCALL, index);
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
@@ -213,7 +228,7 @@ public class BytecodeEmitter {
function.localSlots(),
function.returnSlots(),
function.maxStackSlots(),
ReadOnlyList.wrap(spans)));
ReadOnlyList.wrap(spans), function.metadata()));
}
final var layout = BytecodeFunctionLayoutBuilder.build(ReadOnlyList.wrap(functionFragments));
@@ -224,7 +239,7 @@ public class BytecodeEmitter {
layout.code(),
layout.debugInfo(),
canonicalExports,
ReadOnlyList.wrap(orderedSyscalls.values()));
ReadOnlyList.wrap(orderedSyscalls.values()), inputPlan.registry(), ReadOnlyList.wrap(intrinsicSignatures.values()));
}
private static void writeOpNoImm(final ByteArrayOutputStream out, final int opcode) {
@@ -283,10 +298,19 @@ public class BytecodeEmitter {
}
public record EmissionPlan(
int version,
ReadOnlyList<BytecodeModule.ConstantPoolEntry> constPool,
ReadOnlyList<BytecodeModule.Export> exports,
ReadOnlyList<FunctionPlan> functions,
p.studio.compiler.models.registry.IRRegistry registry) {
public EmissionPlan(
int version,
ReadOnlyList<BytecodeModule.ConstantPoolEntry> constPool,
ReadOnlyList<BytecodeModule.Export> exports,
ReadOnlyList<FunctionPlan> functions) {
this(version, constPool, exports, functions, p.studio.compiler.models.registry.IRRegistry.empty());
}
public EmissionPlan {
constPool = constPool == null ? ReadOnlyList.empty() : constPool;
exports = exports == null ? ReadOnlyList.empty() : exports;
@@ -299,12 +323,23 @@ public class BytecodeEmitter {
}
public record FunctionPlan(
String name,
int paramSlots,
int localSlots,
int returnSlots,
int maxStackSlots,
ReadOnlyList<Operation> operations,
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
public FunctionPlan(
String name,
int paramSlots,
int localSlots,
int returnSlots,
int maxStackSlots,
ReadOnlyList<Operation> operations) {
this(name, paramSlots, localSlots, returnSlots, maxStackSlots, operations, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
}
public FunctionPlan {
Objects.requireNonNull(name, "name");
operations = operations == null ? ReadOnlyList.empty() : operations;
@@ -312,6 +347,7 @@ public class BytecodeEmitter {
}
public enum OperationKind {
REGISTRY,
HALT,
RET,
PUSH_CONST,
@@ -347,12 +383,23 @@ public class BytecodeEmitter {
}
public record Operation(
OperationKind kind,
int immediate,
BytecodeModule.SyscallDecl syscallDecl,
Integer expectedArgSlots,
Integer expectedRetSlots,
BytecodeModule.SourceSpan span,
p.studio.compiler.backend.irvm.IRVMInstruction registryInstruction) {
public Operation(
OperationKind kind,
int immediate,
BytecodeModule.SyscallDecl syscallDecl,
Integer expectedArgSlots,
Integer expectedRetSlots,
BytecodeModule.SourceSpan span) {
this(kind, immediate, syscallDecl, expectedArgSlots, expectedRetSlots, span, null);
}
public Operation {
Objects.requireNonNull(kind, "kind");
}
@@ -25,7 +25,7 @@ public final class BytecodeFunctionLayoutBuilder {
fn.paramSlots(),
fn.localSlots(),
fn.returnSlots(),
fn.maxStackSlots()));
fn.maxStackSlots(), fn.metadata()));
names.add(new BytecodeModule.FunctionName(i, fn.name()));
for (final var span : fn.instructionSpans()) {
@@ -58,6 +58,15 @@ public final class BytecodeFunctionLayoutBuilder {
}
public record FunctionFragment(
String name,
byte[] code,
int paramSlots,
int localSlots,
int returnSlots,
int maxStackSlots,
ReadOnlyList<InstructionSpan> instructionSpans,
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
public FunctionFragment(
String name,
byte[] code,
int paramSlots,
@@ -65,6 +74,9 @@ public final class BytecodeFunctionLayoutBuilder {
int returnSlots,
int maxStackSlots,
ReadOnlyList<InstructionSpan> instructionSpans) {
this(name, code, paramSlots, localSlots, returnSlots, maxStackSlots, instructionSpans, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
}
public FunctionFragment {
Objects.requireNonNull(name, "name");
code = code == null ? new byte[0] : code.clone();
@@ -46,16 +46,18 @@ public class BytecodeLinkPrecheckService {
}
private void validateHostcallIndices(final BytecodeModule module) {
var pc = 0;
while (pc < module.code().length) {
if (pc + 2 > module.code().length) {
for (var function : module.functions()) {
int end = function.codeOffset() + function.codeLen();
var pc = function.codeOffset();
while (pc < end) {
if (pc + 2 > end) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION,
"truncated opcode at pc=" + pc);
}
final var opcode = readU16(module.code(), pc);
final var size = sizeForOpcode(opcode, pc);
if (pc + size > module.code().length) {
if (pc + size > end) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION,
"truncated instruction at pc=" + pc);
@@ -70,6 +72,7 @@ public class BytecodeLinkPrecheckService {
}
pc += size;
}
}
}
private int sizeForOpcode(
@@ -1,6 +1,8 @@
package p.studio.compiler.backend.bytecode;
public enum BytecodeMarshalingErrorCode {
MARSHAL_FORMAT_REGISTRY_INVALID,
MARSHAL_VERIFY_REGISTRY_INVALID,
MARSHAL_FORMAT_STRING_TOO_LONG,
MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG,
MARSHAL_FORMAT_SYSC_NAME_TOO_LONG,
@@ -2,11 +2,7 @@ package p.studio.compiler.backend.bytecode;
import p.studio.utilities.structures.ReadOnlyList;
import java.io.ByteArrayOutputStream;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Objects;
public record BytecodeModule(
@@ -16,16 +12,38 @@ public record BytecodeModule(
byte[] code,
DebugInfo debugInfo,
ReadOnlyList<Export> exports,
ReadOnlyList<SyscallDecl> syscalls) {
ReadOnlyList<SyscallDecl> syscalls,
p.studio.compiler.models.registry.IRRegistry registry,
ReadOnlyList<IntrinsicSignature> intrinsicSignatures) {
public BytecodeModule(
int version,
ReadOnlyList<ConstantPoolEntry> constPool,
ReadOnlyList<FunctionMeta> functions,
byte[] code,
DebugInfo debugInfo,
ReadOnlyList<Export> exports,
ReadOnlyList<SyscallDecl> syscalls,
p.studio.compiler.models.registry.IRRegistry registry) {
this(version, constPool, functions, code, debugInfo, exports, syscalls, registry, ReadOnlyList.empty());
}
private static final int SECTION_KIND_CONST_POOL = 0;
private static final int SECTION_KIND_FUNCTIONS = 1;
private static final int SECTION_KIND_CODE = 2;
private static final int SECTION_KIND_DEBUG = 3;
private static final int SECTION_KIND_EXPORTS = 4;
private static final int SECTION_KIND_SYSCALLS = 5;
public BytecodeModule(
int version,
ReadOnlyList<ConstantPoolEntry> constPool,
ReadOnlyList<FunctionMeta> functions,
byte[] code,
DebugInfo debugInfo,
ReadOnlyList<Export> exports,
ReadOnlyList<SyscallDecl> syscalls) {
this(version, constPool, functions, code, debugInfo, exports, syscalls, p.studio.compiler.models.registry.IRRegistry.empty());
}
public record IntrinsicSignature(int id, int argumentSlots, int resultSlots) {}
public BytecodeModule {
intrinsicSignatures = intrinsicSignatures == null ? ReadOnlyList.empty() : ReadOnlyList.wrap(intrinsicSignatures.asList());
registry = registry == null ? p.studio.compiler.models.registry.IRRegistry.empty() : registry;
constPool = constPool == null ? ReadOnlyList.empty() : constPool;
functions = functions == null ? ReadOnlyList.empty() : functions;
code = code == null ? new byte[0] : code.clone();
@@ -45,191 +63,7 @@ public record BytecodeModule(
}
public byte[] serialize() {
final var constPoolData = serializeConstPool();
final var functionData = serializeFunctions();
final var codeData = code();
final var debugData = serializeDebug();
final var exportData = serializeExports();
final var syscallData = serializeSyscalls();
final var sections = new ArrayList<SectionData>();
if (constPoolData.length > 0) {
sections.add(new SectionData(SECTION_KIND_CONST_POOL, constPoolData));
}
if (functionData.length > 0) {
sections.add(new SectionData(SECTION_KIND_FUNCTIONS, functionData));
}
if (codeData.length > 0) {
sections.add(new SectionData(SECTION_KIND_CODE, codeData));
}
if (debugData.length > 0) {
sections.add(new SectionData(SECTION_KIND_DEBUG, debugData));
}
if (exportData.length > 0) {
sections.add(new SectionData(SECTION_KIND_EXPORTS, exportData));
}
// SYSC section is mandatory, even when empty.
sections.add(new SectionData(SECTION_KIND_SYSCALLS, syscallData));
final var out = new ByteArrayOutputStream();
out.writeBytes(new byte[] { 'P', 'B', 'X', 0 });
writeU16(out, version & 0xFFFF);
out.write(0);
out.write(0);
writeU32(out, sections.size());
out.writeBytes(new byte[20]);
var currentOffset = 32 + (sections.size() * 12);
for (final var section : sections) {
writeU32(out, section.kind);
writeU32(out, currentOffset);
writeU32(out, section.data.length);
currentOffset += section.data.length;
}
for (final var section : sections) {
out.writeBytes(section.data);
}
return out.toByteArray();
}
private byte[] serializeConstPool() {
if (constPool.isEmpty()) {
return new byte[0];
}
final var out = new ByteArrayOutputStream();
writeU32(out, constPool.size());
for (final var entry : constPool) {
switch (entry) {
case NullConstant ignored -> out.write(0);
case Int64Constant value -> {
out.write(1);
writeI64(out, value.value());
}
case Float64Constant value -> {
out.write(2);
writeF64(out, value.value());
}
case BooleanConstant value -> {
out.write(3);
out.write(value.value() ? 1 : 0);
}
case StringConstant value -> {
out.write(4);
writeU32(out, value.utf8().length);
out.writeBytes(value.utf8());
}
case Int32Constant value -> {
out.write(5);
writeI32(out, value.value());
}
}
}
return out.toByteArray();
}
private byte[] serializeFunctions() {
if (functions.isEmpty()) {
return new byte[0];
}
final var out = new ByteArrayOutputStream();
writeU32(out, functions.size());
for (final var function : functions) {
writeU32(out, function.codeOffset());
writeU32(out, function.codeLen());
writeU16(out, function.paramSlots());
writeU16(out, function.localSlots());
writeU16(out, function.returnSlots());
writeU16(out, function.maxStackSlots());
}
return out.toByteArray();
}
private byte[] serializeDebug() {
if (debugInfo == null || (debugInfo.pcToSpan().isEmpty() && debugInfo.functionNames().isEmpty())) {
return new byte[0];
}
final var out = new ByteArrayOutputStream();
writeU32(out, debugInfo.pcToSpan().size());
for (final var entry : debugInfo.pcToSpan()) {
writeU32(out, entry.pc());
writeU32(out, entry.span().fileId());
writeU32(out, entry.span().start());
writeU32(out, entry.span().end());
}
writeU32(out, debugInfo.functionNames().size());
for (final var fn : debugInfo.functionNames()) {
writeU32(out, fn.funcIdx());
writeU32(out, fn.utf8Name().length);
out.writeBytes(fn.utf8Name());
}
return out.toByteArray();
}
private byte[] serializeExports() {
if (exports.isEmpty()) {
return new byte[0];
}
final var out = new ByteArrayOutputStream();
writeU32(out, exports.size());
for (final var export : exports) {
writeU32(out, export.funcIdx());
writeU32(out, export.utf8Symbol().length);
out.writeBytes(export.utf8Symbol());
}
return out.toByteArray();
}
private byte[] serializeSyscalls() {
final var out = new ByteArrayOutputStream();
writeU32(out, syscalls.size());
for (final var syscall : syscalls) {
if (syscall.utf8Module().length > 0xFFFF) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG,
"syscall module name exceeds u16: " + syscall.module());
}
if (syscall.utf8Name().length > 0xFFFF) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_NAME_TOO_LONG,
"syscall name exceeds u16: " + syscall.name());
}
writeU16(out, syscall.utf8Module().length);
out.writeBytes(syscall.utf8Module());
writeU16(out, syscall.utf8Name().length);
out.writeBytes(syscall.utf8Name());
writeU16(out, syscall.version());
writeU16(out, syscall.argSlots());
writeU16(out, syscall.retSlots());
}
return out.toByteArray();
}
private static void writeU16(final ByteArrayOutputStream out, final int value) {
out.writeBytes(ByteBuffer.allocate(2).order(ByteOrder.LITTLE_ENDIAN).putShort((short) (value & 0xFFFF)).array());
}
private static void writeU32(final ByteArrayOutputStream out, final int value) {
out.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(value).array());
}
private static void writeI32(final ByteArrayOutputStream out, final int value) {
out.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(value).array());
}
private static void writeI64(final ByteArrayOutputStream out, final long value) {
out.writeBytes(ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN).putLong(value).array());
}
private static void writeF64(final ByteArrayOutputStream out, final double value) {
out.writeBytes(ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN).putDouble(value).array());
}
private record SectionData(
int kind,
byte[] data) {
return PbxRegistryWriter.write(this);
}
public sealed interface ConstantPoolEntry permits NullConstant, Int64Constant, Float64Constant, BooleanConstant, StringConstant, Int32Constant {
@@ -271,12 +105,23 @@ public record BytecodeModule(
}
public record FunctionMeta(
int codeOffset,
int codeLen,
int paramSlots,
int localSlots,
int returnSlots,
int maxStackSlots,
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
public FunctionMeta(
int codeOffset,
int codeLen,
int paramSlots,
int localSlots,
int returnSlots,
int maxStackSlots) {
this(codeOffset, codeLen, paramSlots, localSlots, returnSlots, maxStackSlots, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
}
}
public record SourceSpan(
@@ -1,58 +1,17 @@
package p.studio.compiler.backend.bytecode;
import java.util.Map;
final class BytecodeOpcodeLayout {
private static final Map<Integer, Integer> SIZE_BY_OPCODE = Map.ofEntries(
Map.entry(0x01, 2),
Map.entry(0x02, 6),
Map.entry(0x03, 6),
Map.entry(0x04, 6),
Map.entry(0x10, 6),
Map.entry(0x11, 2),
Map.entry(0x14, 10),
Map.entry(0x15, 10),
Map.entry(0x16, 3),
Map.entry(0x17, 6),
Map.entry(0x18, 6),
Map.entry(0x20, 2),
Map.entry(0x21, 2),
Map.entry(0x22, 2),
Map.entry(0x23, 2),
Map.entry(0x24, 2),
Map.entry(0x30, 2),
Map.entry(0x31, 2),
Map.entry(0x32, 2),
Map.entry(0x33, 2),
Map.entry(0x34, 2),
Map.entry(0x35, 2),
Map.entry(0x36, 2),
Map.entry(0x3C, 2),
Map.entry(0x3D, 2),
Map.entry(0x3E, 2),
Map.entry(0x40, 6),
Map.entry(0x41, 6),
Map.entry(0x42, 6),
Map.entry(0x43, 6),
Map.entry(0x50, 6),
Map.entry(0x51, 2),
Map.entry(0x52, 10),
Map.entry(0x54, 10),
Map.entry(0x56, 6),
Map.entry(0x70, 6),
Map.entry(0x71, 6),
Map.entry(0x72, 6));
private BytecodeOpcodeLayout() {
}
static int sizeForOpcode(final int opcode) {
final var size = SIZE_BY_OPCODE.get(opcode);
if (size == null) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE,
"unknown opcode 0x%04X".formatted(opcode));
}
return size;
private BytecodeOpcodeLayout() {}
static int sizeForOpcode(int opcode) {
return switch(opcode) {
case 0x00,0x01,0x05,0x11,0x12,0x13,0x20,0x21,0x22,0x23,0x24,
0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0x3a,0x3b,0x3c,0x3d,0x3e,0x51,0x80 -> 2;
case 0x16 -> 3;
case 0x02,0x03,0x04,0x10,0x17,0x18,0x40,0x41,0x42,0x43,0x50,0x53,0x57,0x70,0x71,0x72,
0x90,0x91,0x92,0xa0,0xa3,0xa4,0xa5,0xa6 -> 6;
case 0x14,0x15,0x52,0x58,0x93,0x94,0x95,0x96,0xa1,0xa2 -> 10;
default -> throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE,
"unknown or retired opcode 0x%04X".formatted(opcode));
};
}
}
@@ -1,70 +1,236 @@
package p.studio.compiler.backend.bytecode;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.Set;
import p.studio.compiler.backend.registry.LinkedRegistry;
import p.studio.compiler.models.registry.IRRegistry.*;
import java.nio.*;
import java.util.*;
/** Producer verification. Native binding and final admission remain loader responsibilities. */
public class BytecodePreloadVerifierService {
private static final Set<Integer> TERMINATORS = Set.of(0x01, 0x02, 0x51);
public void verify(final BytecodeModule module) {
final var input = module == null ? BytecodeModule.empty() : module;
for (final var function : input.functions()) {
verifyFunction(input.code(), function);
public void verify(BytecodeModule module) {
try { verifyRegistry(module); }
catch (IllegalArgumentException | ArithmeticException error) {
throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_REGISTRY_INVALID, error.getMessage());
}
}
private void verifyFunction(final byte[] code, final BytecodeModule.FunctionMeta function) {
final var start = function.codeOffset();
final var end = start + function.codeLen();
if (end <= start) {
return;
}
var pc = start;
var lastOpcode = -1;
while (pc < end) {
if (pc + 2 > end) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION,
"truncated opcode in function at pc=" + pc);
private void verifyRegistry(BytecodeModule module) {
Objects.requireNonNull(module);
// Validate the complete declarative schema without accepting an old slot-only image.
module.serialize();
var registry=LinkedRegistry.link(module.registry());
var used=new boolean[module.syscalls().size()];
for(int i=0;i<module.functions().size();i++)verifyFunction(module,registry,i,used);
for(int i=0;i<used.length;i++)check(used[i],"unused syscall declaration "+i);
}
private record Instruction(int pc,int opcode,int first,int second,int next) {}
private void verifyFunction(BytecodeModule m,LinkedRegistry r,int index,boolean[] used) {
var f=m.functions().get(index);byte[] code=m.code();int start=f.codeOffset(),end=start+f.codeLen();
var instructions=new TreeMap<Integer,Instruction>();
for(int pc=start;pc<end;) {
check(pc+2<=end,"truncated opcode");int op=u16(code,pc);
if(op==0x70)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_LINKAGE_RAW_SYSCALL_IN_PRELOAD,"raw SYSCALL in producer image");int size=BytecodeOpcodeLayout.sizeForOpcode(op);check((long)pc+size<=end,"truncated instruction");
int a=size>=6?i32(code,pc+2):0,b=size==10?i32(code,pc+6):0;
var instruction=new Instruction(pc,op,a,b,pc+size);instructions.put(pc,instruction);
switch(op) {
case 0x10 -> check(a>=0&&a<m.constPool().size(),"constant index");
case 0x16 -> check(code[pc+2]==0||code[pc+2]==1,"Boolean immediate");
case 0x15 -> check(Double.isFinite(Double.longBitsToDouble(ByteBuffer.wrap(code,pc+2,8).order(ByteOrder.LITTLE_ENDIAN).getLong())),"nonfinite PushF64");
case 0x18,0x40,0x41 -> check(a>=0,"negative count/index");
case 0x42,0x43 -> check(a>=0&&(long)a<f.paramSlots()+f.localSlots(),"local index");
case 0x50 -> {check(a>=0&&a<m.functions().size(),"FunctionId");check(m.functions().get(a).metadata().captures().isEmpty(),"direct call into capturing function");}
case 0x71 -> {check(a>=0&&a<m.syscalls().size(),"HOSTCALL declaration index");used[a]=true;}
case 0x72 -> check(m.intrinsicSignatures().stream().anyMatch(n->n.id()==a&&n.argumentSlots()>=0&&n.resultSlots()>=0),"intrinsic signature required");
case 0x52 -> {check(a>=0&&a<m.functions().size(),"closure FunctionId");var target=m.functions().get(a).metadata();var fn=fn(r,b);check(r.sameSignature(fn.signature(),target.signature()),"closure signature");}
case 0x53 -> fn(r,a);
case 0x57 -> check(a>=0&&a<f.metadata().captures().size(),"capture index");
case 0x58 -> method(r,a,b);
case 0x90,0x91,0x92 -> type(r,a);
case 0x93,0x94,0x95,0x96 -> {var src=type(r,a).body();var dst=type(r,b).body();check(src instanceof Struct||src instanceof Array||src instanceof Trait,"reference source category");check(op==0x94||op==0x95?dst instanceof Trait:dst instanceof Struct||dst instanceof Array,"reference target category");}
case 0xa0 -> check(type(r,a).body() instanceof Struct,"Struct category");
case 0xa1,0xa2 -> field(r,a,b);
case 0xa3,0xa4,0xa5,0xa6 -> array(r,a);
default -> { }
}
final var opcode = readU16(code, pc);
if (opcode == 0x70) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_LINKAGE_RAW_SYSCALL_IN_PRELOAD,
"raw syscall is forbidden in pre-load artifact");
pc+=size;
}
for(var instruction:instructions.values())if(instruction.opcode()>=2&&instruction.opcode()<=4) {
// Jump operands are absolute offsets within the current function.
long target=(long)start+instruction.first();
check(target>=start&&target<=end&&(target==end||instructions.containsKey((int)target)),"jump outside function or into immediate");
}
analyze(m,r,index,instructions,end);
}
private record Fact(String carrier,Integer type,int component,int width,Long origin,Long integer) {
Fact(String carrier,Integer type,int component,int width) {this(carrier,type,component,width,null,null);}
static final Fact UNKNOWN=new Fact("?",null,0,1),NULL=new Fact("null",null,0,1);
}
private static final class State {
Fact[] locals;ArrayList<Fact> stack;
State(Fact[] locals,ArrayList<Fact> stack){this.locals=locals;this.stack=stack;}
State copy(){return new State(locals.clone(),new ArrayList<>(stack));}
}
private List<Fact> facts(LinkedRegistry r,TypeRef ref) {
int id=r.id(ref).index(),width=r.width(ref);var body=r.type(ref).body();String carrier;
if(body instanceof Builtin b)carrier=switch(b.id()){case 1,3,0x11->"i64";case 2,0x12->"f64";case 0x10->"i32";case 0x13->"bool";case 0x14->"string";case 0x15->"?";default->"ref";};
else carrier=body instanceof Trait?"?":"ref";
var result=new ArrayList<Fact>();for(int i=0;i<width;i++) {
String componentCarrier=carrier;
if(body instanceof Builtin b&&(b.id()==1||b.id()==3&&i==2))componentCarrier="color";
if(body instanceof Trait)componentCarrier=i==0?"ref|null":"i64|null";
if(body instanceof Builtin b&&b.id()==0x15)componentCarrier=i==0?"i64|null":"?";
result.add(new Fact(componentCarrier,id,i,width));
}return result;
}
private List<Fact> parameters(LinkedRegistry r,Signature signature){var result=new ArrayList<Fact>();for(var p:signature.parameters())result.addAll(facts(r,p.type()));return result;}
private List<Fact> results(LinkedRegistry r,Signature signature){var result=new ArrayList<Fact>();for(var t:signature.results())result.addAll(facts(r,t));return result;}
private void analyze(BytecodeModule m,LinkedRegistry r,int index,TreeMap<Integer,Instruction> code,int end) {
var f=m.functions().get(index);var locals=new Fact[f.paramSlots()+f.localSlots()];Arrays.fill(locals,Fact.NULL);
var parameters=parameters(r,f.metadata().signature());for(int i=0;i<parameters.size();i++)locals[i]=stamp(parameters.get(i),-1L-(i-parameters.get(i).component()));
var states=new HashMap<Integer,State>();var queue=new ArrayDeque<Integer>();states.put(f.codeOffset(),new State(locals,new ArrayList<>()));queue.add(f.codeOffset());
long work=0;
while(!queue.isEmpty()) {
int pc=queue.remove();var state=states.get(pc).copy();var instruction=code.get(pc);
check((work+=1L+state.locals.length+state.stack.size())<=268_435_456L,"verification work limit");
transfer(m,r,index,instruction,state,false);
for(int successor:successors(instruction,f.codeOffset(),end)) {
if(successor==end) {check(state.stack.size()==f.returnSlots(),"function-end result height");continue;}
var old=states.get(successor);if(old==null){check((long)(states.size()+1)*(locals.length+f.maxStackSlots())*24<=128L*1024*1024,"verification scratch limit");states.put(successor,state.copy());queue.add(successor);}
else {check(old.stack.size()==state.stack.size(),"join height mismatch");boolean changed=false;
for(int i=0;i<old.locals.length;i++)if(!old.locals[i].equals(state.locals[i])&&!old.locals[i].equals(Fact.UNKNOWN)){old.locals[i]=Fact.UNKNOWN;changed=true;}
for(int i=0;i<old.stack.size();i++)if(!old.stack.get(i).equals(state.stack.get(i))&&!old.stack.get(i).equals(Fact.UNKNOWN)){old.stack.set(i,Fact.UNKNOWN);changed=true;}
if(changed)queue.add(successor);
}
}
final var size = sizeForOpcode(opcode, pc);
if (pc + size > end) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION,
"truncated instruction in function at pc=" + pc);
}
// Diagnose proven type errors after widening has converged, not in traversal order.
for(var entry:states.entrySet()) {
var state=entry.getValue().copy();var instruction=code.get(entry.getKey());transfer(m,r,index,instruction,state,true);
for(int successor:successors(instruction,f.codeOffset(),end))if(successor==end)compatible(state.stack,results(r,f.metadata().signature()),true);
}
}
private List<Integer> successors(Instruction i,int start,int end) {
if(i.opcode()==0x01||i.opcode()==0x05||i.opcode()==0x51)return List.of();
if(i.opcode()==0x02)return List.of(start+i.first());
if(i.opcode()==0x03||i.opcode()==0x04) {
if(i.next()==end)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION,"conditional fallthrough at function end");
return List.of(i.next(),start+i.first());
}
if(i.next()>=end)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION,"reachable fallthrough at function end");return List.of(i.next());
}
private void transfer(BytecodeModule m,LinkedRegistry r,int index,Instruction i,State s,boolean typed) {
int a=i.first(),b=i.second(),op=i.opcode();var f=m.functions().get(index);
switch(op) {
case 0x00,0x01,0x02,0x05,0x80 -> { }
case 0x03,0x04 -> consume(s,List.of(carrier("bool")),typed);
case 0x10 -> {var c=m.constPool().get(a);s.stack.add(switch(c){case BytecodeModule.NullConstant n->Fact.NULL;case BytecodeModule.Int32Constant n->carrier("i32");case BytecodeModule.Int64Constant n->integer(n.value());case BytecodeModule.Float64Constant n->carrier("f64");case BytecodeModule.BooleanConstant n->carrier("bool");case BytecodeModule.StringConstant n->carrier("string");});}
case 0x11 -> pop(s,1);
case 0x12 -> {var value=pop(s,1);s.stack.addAll(value);s.stack.addAll(value);}
case 0x13 -> {var values=pop(s,2);s.stack.add(values.get(1));s.stack.add(values.get(0));}
case 0x14 -> s.stack.add(integer(Integer.toUnsignedLong(a)|((long)b<<32)));case 0x15->s.stack.add(carrier("f64"));case 0x16->s.stack.add(carrier("bool"));case 0x17->s.stack.add(carrier("i32"));case 0x18->pop(s,a);
case 0x40 -> s.stack.add(Fact.UNKNOWN);case 0x41->pop(s,1);
case 0x42 -> s.stack.add(s.locals[a]);case 0x43 -> {var value=pop(s,1).getFirst();s.locals[a]=value;}
case 0x50 -> {var metadata=m.functions().get(a).metadata();consume(s,parameters(r,metadata.signature()),typed);produce(s,results(r,metadata.signature()),i.pc());}
case 0x51 -> {check(s.stack.size()==f.returnSlots(),"RET result height");compatible(s.stack,results(r,f.metadata().signature()),typed);}
case 0x52 -> {var target=m.functions().get(a).metadata();var expected=new ArrayList<Fact>();for(var t:target.captures())expected.addAll(facts(r,t));consume(s,expected,typed);produce(s,facts(r,type(r,b).identity()),i.pc());}
case 0x53 -> {consume(s,facts(r,type(r,a).identity()),typed);var signature=fn(r,a).signature();consume(s,parameters(r,signature),typed);produce(s,results(r,signature),i.pc());}
case 0x57 -> produce(s,facts(r,f.metadata().captures().get(a)),i.pc());
case 0x58 -> {consume(s,facts(r,type(r,a).identity()),typed);var signature=method(r,a,b).signature();consume(s,parameters(r,signature),typed);produce(s,results(r,signature),i.pc());}
case 0x71 -> {var decl=m.syscalls().get(a);pop(s,decl.argSlots());for(int n=0;n<decl.retSlots();n++)s.stack.add(Fact.UNKNOWN);}
case 0x72 -> {
var signature=m.intrinsicSignatures().stream().filter(n->n.id()==a).findFirst().orElseThrow(()->new IllegalArgumentException("intrinsic signature required"));
pop(s,signature.argumentSlots());for(int n=0;n<signature.resultSlots();n++)s.stack.add(Fact.UNKNOWN);
}
case 0x90 -> {
consume(s,facts(r,type(r,a).identity()),typed);
s.stack.add(new Fact("i64|null",-1,0,2,(long)i.pc(),null));
s.stack.add(new Fact("?",-1,1,2,(long)i.pc(),null));
}
case 0x91 -> {dynamic(pop(s,2),typed);produce(s,facts(r,type(r,a).identity()),i.pc());}
case 0x92 -> {dynamic(pop(s,2),typed);s.stack.add(carrier("bool"));}
case 0x93,0x94,0x95,0x96 -> {consume(s,facts(r,type(r,a).identity()),typed);if(op==0x93||op==0x94)s.stack.add(carrier("bool"));else produce(s,facts(r,type(r,b).identity()),i.pc());}
case 0xa0 -> produce(s,facts(r,type(r,a).identity()),i.pc());
case 0xa1 -> {consume(s,facts(r,type(r,a).identity()),typed);produce(s,facts(r,field(r,a,b).type()),i.pc());}
case 0xa2 -> {consume(s,facts(r,field(r,a,b).type()),typed);consume(s,facts(r,type(r,a).identity()),typed);}
case 0xa3 -> {consume(s,List.of(carrier("i64")),typed);produce(s,facts(r,type(r,a).identity()),i.pc());}
case 0xa4 -> {consume(s,facts(r,type(r,a).identity()),typed);s.stack.add(carrier("i64"));}
case 0xa5 -> {consume(s,List.of(carrier("i64")),typed);consume(s,facts(r,type(r,a).identity()),typed);produce(s,facts(r,array(r,a).element()),i.pc());}
case 0xa6 -> {consume(s,facts(r,array(r,a).element()),typed);consume(s,List.of(carrier("i64")),typed);consume(s,facts(r,type(r,a).identity()),typed);}
case 0x34,0x35 -> {consume(s,List.of(carrier("bool"),carrier("bool")),typed);s.stack.add(carrier("bool"));}
case 0x36 -> {consume(s,List.of(carrier("bool")),typed);s.stack.add(carrier("bool"));}
case 0x3e -> {
var value=pop(s,1).getFirst();
if(typed)check(value.carrier().equals("?")||numeric(value.carrier()),"numeric operand required");
s.stack.add(carrier(value.carrier()));
}
default -> {
var operands=pop(s,2);String left=operands.get(0).carrier(),right=operands.get(1).carrier();
boolean comparison=op>=0x30&&op<=0x33||op==0x3c||op==0x3d;
boolean unknown=left.equals("?")||right.equals("?")||left.contains("|")||right.contains("|");
boolean stringAdd=op==0x20&&(left.equals("string")||right.equals("string"));
if(typed&&!unknown&&op!=0x30&&op!=0x31) {
if(comparison)check(numeric(left)&&numeric(right)||left.equals(right)&&(left.equals("string")||left.equals("bool")),"comparison carrier mismatch");
else if(!stringAdd)check(numeric(left)&&numeric(right)&&(op<0x37||!left.equals("f64")&&!right.equals("f64")),"numeric operand mismatch");
}
String result="?";
if(stringAdd)result="string";
else if(numeric(left)&&numeric(right))result=left.equals("f64")||right.equals("f64")?"f64":left.equals("i64")||right.equals("i64")?"i64":"i32";
s.stack.add(carrier(comparison?"bool":result));
}
lastOpcode = opcode;
pc += size;
}
if (!TERMINATORS.contains(lastOpcode)) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION,
"function ends without terminator opcode");
check(s.stack.size()<=f.maxStackSlots(),"maximum temporary slots exceeded (zero is literal)");
}
private static boolean numeric(String carrier){return carrier.equals("i32")||carrier.equals("i64")||carrier.equals("f64")||carrier.equals("color");}
private static Fact carrier(String name){return new Fact(name,null,0,1);}
private static List<Fact> pop(State s,int n){check(n>=0&&n<=s.stack.size(),"temporary stack underflow");var result=new ArrayList<>(s.stack.subList(s.stack.size()-n,s.stack.size()));s.stack.subList(s.stack.size()-n,s.stack.size()).clear();return result;}
private static void consume(State s,List<Fact> expected,boolean typed){compatible(pop(s,expected.size()),expected,typed);}
private static Fact stamp(Fact fact,long origin) {
return new Fact(fact.carrier(),fact.type(),fact.component(),fact.width(),origin,fact.integer());
}
private static Fact integer(long value){return new Fact("i64",null,0,1,null,value);}
private static void produce(State state,List<Fact> values,int pc) {
for(int n=0;n<values.size();n++)state.stack.add(stamp(values.get(n),((long)pc<<32)|(n-values.get(n).component())));
}
private static int mask(String carrier) {
int result=0;
for(String part:carrier.split("\\|"))result|=switch(part) {
case "i32"->1;case "i64","color"->2;case "f64"->4;case "bool"->8;case "string"->16|64;case "ref"->32|64;case "null"->64;default->127;
};
return result;
}
private static void dynamic(List<Fact> values,boolean typed) {
if(!typed)return;
check((mask(values.get(0).carrier())&(2|64))!=0,"Dynamic tag carrier");
if(values.get(0).equals(Fact.NULL))check((mask(values.get(1).carrier())&64)!=0,"invalid null Dynamic pair");
}
private static void compatible(List<Fact> actual,List<Fact> expected,boolean typed) {
if(!typed)return;
check(actual.size()==expected.size(),"group height");
for(int i=0;i<actual.size();i++) {
var a=actual.get(i);var e=expected.get(i);
if(e.component()==0&&e.type()!=null&&a.type()!=null&&a.origin()!=null&&a.component()==0&&a.width()==e.width()) {
boolean complete=true;
for(int n=0;n<e.width();n++) {
var member=actual.get(i+n);
complete&=Objects.equals(member.type(),a.type())&&Objects.equals(member.origin(),a.origin())&&member.component()==n;
}
// A full proven semantic group cannot silently become another type.
if(complete) {
if(a.type()==-1)check(e.carrier().equals("i64|null")&&expected.get(i+1).carrier().equals("?"),"Dynamic semantic group mismatch");
else check(a.type().equals(e.type()),"proven semantic group mismatch");
}
}
check((mask(a.carrier())&mask(e.carrier()))!=0,"proven carrier mismatch");
if(e.carrier().equals("color")&&a.integer()!=null)check(a.integer()>=0&&a.integer()<=0xffffffffL,"color range");
if(e.component()==0&&e.width()==2&&e.carrier().contains("|" )&&a.equals(Fact.NULL))
check((mask(actual.get(i+1).carrier())&64)!=0,"invalid null group pair");
}
}
private int sizeForOpcode(
final int opcode,
final int pc) {
try {
return BytecodeOpcodeLayout.sizeForOpcode(opcode);
} catch (BytecodeMarshalingException ignored) {
throw new BytecodeMarshalingException(
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE,
"unknown opcode 0x%04X at pc=%d".formatted(opcode, pc));
}
}
private int readU16(final byte[] code, final int offset) {
return ByteBuffer.wrap(code, offset, 2).order(ByteOrder.LITTLE_ENDIAN).getShort() & 0xFFFF;
}
private static TypeDeclaration type(LinkedRegistry r,int id){check(id>=0&&id<r.types().size(),"TypeId");return r.types().get(id);}
private static Fn fn(LinkedRegistry r,int id){check(type(r,id).body() instanceof Fn,"Fn category");return (Fn)type(r,id).body();}
private static Method method(LinkedRegistry r,int id,int member){check(type(r,id).body() instanceof Trait,"Trait category");var t=(Trait)type(r,id).body();check(member>=0&&member<t.methods().size(),"method index");return t.methods().get(member);}
private static NamedType field(LinkedRegistry r,int id,int member){check(type(r,id).body() instanceof Struct,"Struct category");var t=(Struct)type(r,id).body();check(member>=0&&member<t.fields().size(),"field index");return t.fields().get(member);}
private static Array array(LinkedRegistry r,int id){check(type(r,id).body() instanceof Array,"Array category");return (Array)type(r,id).body();}
private static int u16(byte[] code,int offset){return ByteBuffer.wrap(code,offset,2).order(ByteOrder.LITTLE_ENDIAN).getShort()&65535;}
private static int i32(byte[] code,int offset){return ByteBuffer.wrap(code,offset,4).order(ByteOrder.LITTLE_ENDIAN).getInt();}
private static void check(boolean ok,String message){if(!ok)throw new IllegalArgumentException(message);}
}
@@ -0,0 +1,154 @@
package p.studio.compiler.backend.bytecode;
import p.studio.compiler.backend.registry.LinkedRegistry;
import p.studio.compiler.models.registry.IRRegistry.*;
import p.studio.utilities.structures.ReadOnlyList;
import java.io.ByteArrayOutputStream;
import java.nio.*;
import java.nio.charset.*;
import java.util.*;
/** Current PBX v0 field encoding. Never serialize a native layout or infer a semantic type. */
final class PbxRegistryWriter {
private final BytecodeModule module;
private final LinkedRegistry registry;
private final Set<Role> roles = EnumSet.noneOf(Role.class);
private long nested, names;
private PbxRegistryWriter(BytecodeModule module) {
this.module=module;
this.registry=LinkedRegistry.link(module.registry());
}
static byte[] write(BytecodeModule module) {
try { return new PbxRegistryWriter(module).write(); }
catch (IllegalArgumentException | ArithmeticException error) {
throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_REGISTRY_INVALID, error.getMessage());
}
}
private byte[] write() {
require(module.version()==0,"PBX version must be zero");
require(module.code().length<=32*1024*1024,"Code limit");
require(module.functions().size()<=65_536,"function limit");
require(module.registry().implementations().size()<=262_144,"implementation limit");
var sections=new TreeMap<Integer,byte[]>();
sections.put(0,constants()); sections.put(1,functions()); sections.put(2,module.code());
if(module.debugInfo()!=null) sections.put(3,debug());
sections.put(4,exports()); sections.put(5,syscalls()); sections.put(6,types());
sections.put(7,implementations()); sections.put(8,captures());
require(nested<=1_048_576,"nested record limit"); require(names<=4*1024*1024,"Registry name limit");
long total=32+12L*sections.size(); for(var bytes:sections.values()) total+=bytes.length;
require(total<=64*1024*1024,"PBX file limit");
var out=new Bytes();out.raw(new byte[]{'P','B','X',0});out.u16(0);out.u8(0);out.u8(0);out.u32(sections.size());out.raw(new byte[20]);
int offset=32+12*sections.size();
for(var section:sections.entrySet()) {out.u32(section.getKey());out.u32(offset);out.u32(section.getValue().length);offset+=section.getValue().length;}
sections.values().forEach(out::raw);return out.bytes();
}
private void optionalName(Bytes out,String name) {
names+=LinkedRegistry.nameBytes(name);out.u8(name==null?0:1);if(name!=null)out.text32(name);
}
private void parameters(Bytes out,ReadOnlyList<NamedType> values) {
nested+=values.size();out.u32(values.size());for(var p:values) {out.u32(registry.id(p.type()).index());optionalName(out,p.name());}
}
private void results(Bytes out,ReadOnlyList<TypeRef> values) {
nested+=values.size();out.u32(values.size());for(var r:values)out.u32(registry.id(r).index());
}
private void signature(Bytes out,Signature signature) {
require(registry.parameterSlots(signature)<=16_384,"signature parameter limit");
require(registry.resultSlots(signature)<=1024,"signature result limit");
parameters(out,signature.parameters());results(out,signature.results());
}
private byte[] types() {
var out=new Bytes();out.u32(registry.types().size());
for(var t:registry.types()) {
int tag=switch(t.body()){case Builtin b->0;case Struct s->1;case Array a->2;case Trait t1->3;case Fn f->4;};
out.u8(tag);optionalName(out,t.name());
switch(t.body()) {
case Builtin b -> out.u32(b.id());
case Struct s -> parameters(out,s.fields());
case Array a -> out.u32(registry.id(a.element()).index());
case Fn f -> signature(out,f.signature());
case Trait t1 -> {nested+=t1.methods().size();out.u32(t1.methods().size());for(var m:t1.methods()){optionalName(out,m.name());signature(out,m.signature());}}
}
}
return out.bytes();
}
private byte[] functions() {
var out=new Bytes();out.u32(module.functions().size());long previousEnd=0;
for(int i=0;i<module.functions().size();i++) {
var f=module.functions().get(i);var m=f.metadata();require(m!=null,"typed function metadata required");
int p=registry.parameterSlots(m.signature()),r=registry.resultSlots(m.signature());
require(p==f.paramSlots()&&r==f.returnSlots(),"slot cache differs from typed signature");
require(f.localSlots()>=0&&f.maxStackSlots()>=0&&(long)p+f.localSlots()+f.maxStackSlots()<=16_384&&r<=1024,"frame/result limit");
long end=(long)f.codeOffset()+f.codeLen();
require(f.codeOffset()>=previousEnd&&f.codeLen()>0&&end<=module.code().length,"invalid function Code range");previousEnd=end;
if(i==0)require(m.captures().isEmpty(),"boot function must have no captures");
if(m.role()!=Role.NONE) {
require(roles.add(m.role()),"duplicate lifecycle role");require(i!=0&&m.captures().isEmpty(),"invalid lifecycle target");
if(m.role()==Role.MEMCARD_BOOT) require(m.signature().parameters().size()==1&&m.signature().results().size()==1
&&isBuiltin(m.signature().parameters().getFirst().type(),0x11)&&isBuiltin(m.signature().results().getFirst(),0x10),"MemcardBoot signature");
else require(p==0&&r==0,"pause/resume signature");
}
out.u8(m.role().wire());optionalName(out,m.name());signature(out,m.signature());
out.u32(f.codeOffset());out.u32(f.codeLen());out.u32(f.localSlots());out.u32(f.maxStackSlots());
}
return out.bytes();
}
private boolean isBuiltin(TypeRef type,int expected) {return registry.type(type).body() instanceof Builtin b&&b.id()==expected;}
private byte[] captures(){var out=new Bytes();out.u32(module.functions().size());for(var f:module.functions())results(out,f.metadata().captures());return out.bytes();}
private byte[] implementations() {
var out=new Bytes();var declarations=new TreeMap<String,Implementation>();
for(var impl:module.registry().implementations()) {
var old=declarations.putIfAbsent(impl.declarationKey(),impl);require(old==null||old.equals(impl),"conflicting implementation declaration");
}
out.u32(declarations.size());var pairs=new HashSet<List<Integer>>();
for(var impl:declarations.values()) {
require(registry.type(impl.owner()).body() instanceof Struct,"implementation owner must be Struct");
require(registry.type(impl.trait()).body() instanceof Trait,"implementation trait category");
var trait=(Trait)registry.type(impl.trait()).body();int owner=registry.id(impl.owner()).index(),traitId=registry.id(impl.trait()).index();
require(pairs.add(List.of(owner,traitId)),"duplicate implementation pair");
require(trait.methods().size()==impl.methods().size(),"implementation method count");
out.u32(owner);out.u32(traitId);out.u32(impl.methods().size());nested+=impl.methods().size();
for(int i=0;i<impl.methods().size();i++) {
int function=impl.methods().get(i).getIndex();require(function<module.functions().size(),"implementation FunctionId");
var metadata=module.functions().get(function).metadata();var actual=metadata.signature();var expected=trait.methods().get(i).signature();
require(metadata.captures().isEmpty()&&!actual.parameters().isEmpty()&&registry.id(actual.parameters().getFirst().type()).index()==owner,"implementation receiver/captures");
require(registry.sameSignature(new Signature(actual.parameters().immutableSubList(1,actual.parameters().size()),actual.results()),expected),"implementation signature mismatch");out.u32(function);
}
}
return out.bytes();
}
private byte[] constants(){
var out=new Bytes();require(module.constPool().size()<=1_048_576,"constant count");out.u32(module.constPool().size());long strings=0;
for(var c:module.constPool())switch(c){
case BytecodeModule.NullConstant n->out.u8(0);
case BytecodeModule.Int64Constant n->{out.u8(1);out.i64(n.value());}
case BytecodeModule.Float64Constant n->{require(Double.isFinite(n.value()),"nonfinite Float constant");out.u8(2);out.i64(Double.doubleToRawLongBits(n.value()));}
case BytecodeModule.BooleanConstant n->{out.u8(3);out.u8(n.value()?1:0);}
case BytecodeModule.StringConstant n->{var bytes=Bytes.utf8(n.value());require(bytes.length<=4*1024*1024,"individual string limit");strings+=bytes.length;require(strings<=64*1024*1024,"constant string limit");out.u8(4);out.u32(bytes.length);out.raw(bytes);}
case BytecodeModule.Int32Constant n->{out.u8(5);out.i32(n.value());}
}return out.bytes();
}
private byte[] exports(){var out=new Bytes();out.u32(module.exports().size());for(var e:module.exports()){require(e.funcIdx()>=0&&e.funcIdx()<module.functions().size(),"export FunctionId");out.u32(e.funcIdx());out.text32(e.symbol());}return out.bytes();}
private byte[] debug(){
var out=new Bytes();var d=module.debugInfo();out.u32(d.pcToSpan().size());
for(var p:d.pcToSpan()){require(p.pc()>=0&&p.pc()<module.code().length&&p.span().start()>=0&&p.span().end()>=p.span().start(),"debug span range");out.u32(p.pc());out.i32(p.span().fileId());out.u32(p.span().start());out.u32(p.span().end());}
out.u32(d.functionNames().size());for(var n:d.functionNames()){require(n.funcIdx()>=0&&n.funcIdx()<module.functions().size(),"debug FunctionId");out.u32(n.funcIdx());out.text32(n.name());}return out.bytes();
}
private byte[] syscalls(){var out=new Bytes();var identities=new HashSet<List<Object>>();out.u32(module.syscalls().size());
for(var s:module.syscalls()){require(identities.add(List.of(s.module(),s.name(),s.version())),"duplicate syscall declaration");if(Bytes.utf8(s.module()).length>65535)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG,"syscall module name exceeds u16");
if(Bytes.utf8(s.name()).length>65535)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_NAME_TOO_LONG,"syscall name exceeds u16");
out.text16(s.module());out.text16(s.name());out.u16(s.version());out.u16(s.argSlots());out.u16(s.retSlots());}return out.bytes();}
static void require(boolean condition,String message){if(!condition)throw new IllegalArgumentException(message);}
private static final class Bytes {
final ByteArrayOutputStream out=new ByteArrayOutputStream();
void u8(int v){require(v>=0&&v<=255,"u8 range");out.write(v);}
void u16(int v){require(v>=0&&v<=65535,"u16 range");out.write(v);out.write(v>>>8);}
void u32(int v){require(v>=0,"u32 range/product limit");i32(v);}
void i32(int v){for(int i=0;i<4;i++)out.write(v>>>(8*i));}
void i64(long v){for(int i=0;i<8;i++)out.write((int)(v>>>(8*i)));}
void raw(byte[] v){out.writeBytes(v);}
void text32(String v){var b=utf8(v);u32(b.length);raw(b);}
void text16(String v){var b=utf8(v);u16(b.length);raw(b);}
byte[] bytes(){return out.toByteArray();}
static byte[] utf8(String text){try{var b=StandardCharsets.UTF_8.newEncoder().encode(CharBuffer.wrap(text));var bytes=new byte[b.remaining()];b.get(bytes);return bytes;}catch(CharacterCodingException e){throw new IllegalArgumentException("invalid UTF-8 source",e);}}
}
}
@@ -5,12 +5,23 @@ import p.studio.utilities.structures.ReadOnlyList;
import java.util.Objects;
public record IRVMFunction(
String name,
int paramSlots,
int localSlots,
int returnSlots,
int maxStackSlots,
ReadOnlyList<IRVMInstruction> instructions,
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
public IRVMFunction(
String name,
int paramSlots,
int localSlots,
int returnSlots,
int maxStackSlots,
ReadOnlyList<IRVMInstruction> instructions) {
this(name, paramSlots, localSlots, returnSlots, maxStackSlots, instructions, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
}
public IRVMFunction {
Objects.requireNonNull(name, "name");
instructions = instructions == null ? ReadOnlyList.empty() : instructions;
@@ -3,10 +3,23 @@ package p.studio.compiler.backend.irvm;
import java.util.Objects;
public record IRVMInstruction(
IRVMOp op,
Integer immediate,
Integer secondImmediate) {
public IRVMInstruction(
IRVMOp op,
Integer immediate) {
this(op, immediate, null);
}
public IRVMInstruction {
Objects.requireNonNull(op, "op");
if (op.name().startsWith("REGISTRY_") && ((op.immediateSize() == 8) != (secondImmediate != null))) {
throw new IllegalArgumentException("registry operand count differs from opcode layout");
}
if (!op.name().startsWith("REGISTRY_") && secondImmediate != null) {
throw new IllegalArgumentException("unexpected second immediate");
}
if (op.immediateSize() > 0 && immediate == null) {
throw new IllegalArgumentException("immediate is required for opcode: " + op.name());
}
@@ -3,8 +3,15 @@ package p.studio.compiler.backend.irvm;
import p.studio.utilities.structures.ReadOnlyList;
public record IRVMModule(
String vmProfile,
ReadOnlyList<IRVMFunction> functions,
p.studio.compiler.models.registry.IRRegistry registry) {
public IRVMModule(
String vmProfile,
ReadOnlyList<IRVMFunction> functions) {
this(vmProfile, functions, p.studio.compiler.models.registry.IRRegistry.empty());
}
public IRVMModule {
vmProfile = vmProfile == null || vmProfile.isBlank() ? "core-v1" : vmProfile;
functions = functions == null ? ReadOnlyList.empty() : functions;
@@ -91,6 +91,7 @@ public final class IRVMProfileFeatureGate {
final String profile,
final IRVMOp op) {
final var allowed = allowedByProfile.get(normalizeProfile(profile));
return allowed != null && allowed.contains(op);
return allowed != null && (allowed.contains(op) ||
op.name().startsWith("REGISTRY_") && java.util.Arrays.stream(p.studio.compiler.models.registry.IRRegistryInstruction.Kind.values()).anyMatch(k -> k.opcode() == op.opcode()));
}
}
@@ -54,13 +54,17 @@ public record IRVMProgram(
private void validateCoherence(
final IRVMModule module,
final BytecodeEmitter.EmissionPlan emissionPlan) {
if (!module.registry().equals(emissionPlan.registry())) {
throw new IllegalArgumentException("irvm/emission plan registry mismatch");
}
if (module.functions().size() != emissionPlan.functions().size()) {
throw new IllegalArgumentException("irvm module/function-plan size mismatch");
}
for (var fnIndex = 0; fnIndex < module.functions().size(); fnIndex++) {
final var irvmFunction = module.functions().get(fnIndex);
final var planFunction = emissionPlan.functions().get(fnIndex);
if (!irvmFunction.name().equals(planFunction.name())
if (!java.util.Objects.equals(irvmFunction.metadata(), planFunction.metadata())
|| !irvmFunction.name().equals(planFunction.name())
|| irvmFunction.paramSlots() != planFunction.paramSlots()
|| irvmFunction.localSlots() != planFunction.localSlots()
|| irvmFunction.returnSlots() != planFunction.returnSlots()
@@ -84,6 +88,7 @@ public record IRVMProgram(
private boolean coherentPair(
final IRVMInstruction instruction,
final BytecodeEmitter.Operation operation) {
if (operation.kind() == BytecodeEmitter.OperationKind.REGISTRY) return instruction.equals(operation.registryInstruction());
if (instruction.op().internal()) {
return true;
}
@@ -137,13 +142,13 @@ public record IRVMProgram(
function.localSlots(),
function.returnSlots(),
function.maxStackSlots(),
ReadOnlyList.wrap(operations)));
ReadOnlyList.wrap(operations), function.metadata()));
}
return new BytecodeEmitter.EmissionPlan(
0,
ReadOnlyList.empty(),
ReadOnlyList.empty(),
ReadOnlyList.wrap(functions));
ReadOnlyList.wrap(functions), module.registry());
}
private boolean containsRawSyscall(final BytecodeEmitter.EmissionPlan plan) {
@@ -171,6 +176,7 @@ public record IRVMProgram(
}
private BytecodeEmitter.Operation deriveOperation(final IRVMInstruction instruction) {
if (instruction.op().name().startsWith("REGISTRY_")) return new BytecodeEmitter.Operation(BytecodeEmitter.OperationKind.REGISTRY, 0, null, null, null, null, instruction);
final var immediate = instruction.immediate() == null ? 0 : instruction.immediate();
return switch (instruction.op().opcode()) {
case 0x01 -> BytecodeEmitter.Operation.halt();
@@ -240,13 +246,14 @@ public record IRVMProgram(
functionPlan.localSlots(),
functionPlan.returnSlots(),
functionPlan.maxStackSlots(),
ReadOnlyList.wrap(instructions)));
ReadOnlyList.wrap(instructions), functionPlan.metadata()));
}
return new IRVMModule("core-v1", ReadOnlyList.wrap(functions));
return new IRVMModule("core-v1", ReadOnlyList.wrap(functions), inputPlan.registry());
}
private static IRVMInstruction operationToInstruction(final BytecodeEmitter.Operation operation) {
return switch (operation.kind()) {
case REGISTRY -> operation.registryInstruction();
case HALT -> new IRVMInstruction(IRVMOp.HALT, null);
case RET -> new IRVMInstruction(IRVMOp.RET, null);
case PUSH_CONST -> new IRVMInstruction(IRVMOp.PUSH_CONST, operation.immediate());
@@ -41,10 +41,12 @@ public class LowerToIRVMService {
}
validateCommonLifecycleStructure(backend);
final var registry = p.studio.compiler.backend.registry.LinkedRegistry.link(backend.getRegistry());
final var ordered = orderFunctions(backend);
final var funcIdByCallableId = new HashMap<CallableId, Integer>();
for (var i = 0; i < ordered.size(); i++) {
final var fn = ordered.get(i);
if (fn.metadata() == null) throw new IllegalArgumentException("typed function metadata required: " + fn.callableName());
if (funcIdByCallableId.putIfAbsent(fn.callableId(), i) != null) {
throw loweringError(
fn,
@@ -60,6 +62,9 @@ public class LowerToIRVMService {
for (var i = 0; i < ordered.size(); i++) {
final var fn = ordered.get(i);
if (fn.metadata() == null) throw new IllegalArgumentException("typed function metadata required: " + fn.callableName());
final var parameterSlots = registry.parameterSlots(fn.metadata().signature());
final var resultSlots = registry.resultSlots(fn.metadata().signature());
final var instructions = new ArrayList<IRVMInstruction>(fn.instructions().size());
final var operations = new ArrayList<BytecodeEmitter.Operation>(fn.instructions().size());
final var labelToPc = new HashMap<String, Integer>();
@@ -68,6 +73,12 @@ public class LowerToIRVMService {
for (final var instr : fn.instructions()) {
final var sourceSpan = toBytecodeSpan(fn.fileId().getId(), instr.span());
switch (instr.kind()) {
case REGISTRY -> {
var typed = p.studio.compiler.backend.registry.RegistryInstructionLowerer.lower(instr.registryInstruction(), registry, fn, ordered, funcIdByCallableId);
instructions.add(typed);
operations.add(new BytecodeEmitter.Operation(BytecodeEmitter.OperationKind.REGISTRY, 0, null, null, null, sourceSpan, typed));
functionPc += typed.encodedSize();
}
case PUSH_I32 -> {
final int value;
try {
@@ -271,7 +282,7 @@ public class LowerToIRVMService {
}
final var calleeFunction = ordered.get(calleeId);
if (instr.expectedArgSlots() != null
&& instr.expectedArgSlots() != calleeFunction.paramSlots()) {
&& instr.expectedArgSlots() != registry.parameterSlots(calleeFunction.metadata().signature())) {
throw loweringError(
fn,
instr,
@@ -279,10 +290,10 @@ public class LowerToIRVMService {
"call arg_slots mismatch for %s. expected=%d actual=%d".formatted(
instr.calleeCallableName(),
instr.expectedArgSlots(),
calleeFunction.paramSlots()));
registry.parameterSlots(calleeFunction.metadata().signature())));
}
if (instr.expectedRetSlots() != null
&& instr.expectedRetSlots() != calleeFunction.returnSlots()) {
&& instr.expectedRetSlots() != registry.resultSlots(calleeFunction.metadata().signature())) {
throw loweringError(
fn,
instr,
@@ -290,7 +301,7 @@ public class LowerToIRVMService {
"call ret_slots mismatch for %s. expected=%d actual=%d".formatted(
instr.calleeCallableName(),
instr.expectedRetSlots(),
calleeFunction.returnSlots()));
registry.resultSlots(calleeFunction.metadata().signature())));
}
instructions.add(new IRVMInstruction(IRVMOp.CALL, calleeId));
operations.add(BytecodeEmitter.Operation.callFunc(calleeId, sourceSpan));
@@ -404,27 +415,38 @@ public class LowerToIRVMService {
}
loweredFunctions.add(new IRVMFunction(
fn.callableName(),
fn.paramSlots(),
parameterSlots,
fn.localSlots(),
fn.returnSlots(),
resultSlots,
fn.maxStackSlots(),
ReadOnlyList.wrap(instructions)));
ReadOnlyList.wrap(instructions), fn.metadata()));
emissionFunctions.add(new BytecodeEmitter.FunctionPlan(
fn.callableName(),
fn.paramSlots(),
parameterSlots,
fn.localSlots(),
fn.returnSlots(),
resultSlots,
fn.maxStackSlots(),
ReadOnlyList.wrap(operations)));
ReadOnlyList.wrap(operations), fn.metadata()));
}
final var imageImplementations = new ArrayList<p.studio.compiler.models.registry.IRRegistry.Implementation>();
for (var impl : backend.getRegistry().implementations()) {
var methods = new ArrayList<CallableId>();
for (var method : impl.methods()) {
var target = funcIdByCallableId.get(method);
if (target == null) throw new IllegalArgumentException("undefined implementation function: " + method);
methods.add(new CallableId(target));
}
imageImplementations.add(new p.studio.compiler.models.registry.IRRegistry.Implementation(impl.declarationKey(), impl.owner(), impl.trait(), ReadOnlyList.wrap(methods)));
}
var imageRegistry = new p.studio.compiler.models.registry.IRRegistry(backend.getRegistry().types(), ReadOnlyList.wrap(imageImplementations));
final var program = new IRVMProgram(
new IRVMModule(vmProfile, ReadOnlyList.wrap(loweredFunctions)),
new IRVMModule(vmProfile, ReadOnlyList.wrap(loweredFunctions), imageRegistry),
new BytecodeEmitter.EmissionPlan(
0,
ReadOnlyList.wrap(constPool),
ReadOnlyList.empty(),
ReadOnlyList.wrap(emissionFunctions)));
ReadOnlyList.wrap(emissionFunctions), imageRegistry));
validator.validate(program, false);
return program;
}
@@ -124,12 +124,12 @@ public class OptimizeIRVMService {
}
return new IRVMProgram(
new IRVMModule(input.module().vmProfile(), ReadOnlyList.wrap(rewrittenFunctions)),
new IRVMModule(input.module().vmProfile(), ReadOnlyList.wrap(rewrittenFunctions), input.module().registry()),
new BytecodeEmitter.EmissionPlan(
input.emissionPlan().version(),
input.emissionPlan().constPool(),
input.emissionPlan().exports(),
ReadOnlyList.wrap(rewrittenPlans)));
ReadOnlyList.wrap(rewrittenPlans), input.emissionPlan().registry()));
}
private static boolean isJump(final IRVMOp op) {
@@ -218,14 +218,14 @@ public class OptimizeIRVMService {
function.localSlots(),
function.returnSlots(),
function.maxStackSlots(),
ReadOnlyList.wrap(newInstructions));
ReadOnlyList.wrap(newInstructions), function.metadata());
final var rewrittenPlan = new BytecodeEmitter.FunctionPlan(
functionPlan.name(),
functionPlan.paramSlots(),
functionPlan.localSlots(),
functionPlan.returnSlots(),
functionPlan.maxStackSlots(),
ReadOnlyList.wrap(newOperations));
ReadOnlyList.wrap(newOperations), functionPlan.metadata());
return new FunctionRewrite(rewrittenFunction, rewrittenPlan, changed);
}
@@ -383,14 +383,14 @@ public class OptimizeIRVMService {
function.localSlots(),
function.returnSlots(),
function.maxStackSlots(),
ReadOnlyList.wrap(instructions)),
ReadOnlyList.wrap(instructions), function.metadata()),
new BytecodeEmitter.FunctionPlan(
functionPlan.name(),
functionPlan.paramSlots(),
functionPlan.localSlots(),
functionPlan.returnSlots(),
functionPlan.maxStackSlots(),
ReadOnlyList.wrap(operations)),
ReadOnlyList.wrap(operations), functionPlan.metadata()),
true);
}
@@ -0,0 +1,130 @@
package p.studio.compiler.backend.registry;
import p.studio.compiler.models.registry.IRRegistry;
import p.studio.compiler.models.registry.IRRegistry.*;
import p.studio.utilities.structures.ReadOnlyList;
import java.util.*;
/** Immutable image type table and reference resolution. No language-specific type inference. */
public final class LinkedRegistry {
public record TypeId(int index) { public TypeId { if (index < 0) throw new IllegalArgumentException("negative TypeId"); } }
public record FunctionId(int index) { public FunctionId { if (index < 0) throw new IllegalArgumentException("negative FunctionId"); } }
public record ImplementationId(int index) { public ImplementationId { if (index < 0) throw new IllegalArgumentException("negative ImplementationId"); } }
private final ReadOnlyList<TypeDeclaration> types;
private final Map<TypeRef, TypeId> references;
private LinkedRegistry(List<TypeDeclaration> types, Map<TypeRef, TypeId> references) {
this.types = ReadOnlyList.wrap(types);
this.references = Map.copyOf(references);
}
public ReadOnlyList<TypeDeclaration> types() { return types; }
public TypeId id(TypeRef reference) {
final var id = references.get(reference);
if (id == null) throw new IllegalArgumentException("undefined type declaration: " + reference);
return id;
}
public TypeDeclaration type(TypeRef reference) { return types.get(id(reference).index()); }
public int width(TypeRef reference) {
return switch (type(reference).body()) {
case Builtin b -> switch (b.id()) { case 2, 0x15 -> 2; case 3 -> 3; default -> 1; };
case Trait ignored -> 2;
default -> 1;
};
}
public int parameterSlots(Signature signature) {
var slots = 0;
for (var p : signature.parameters()) slots = Math.addExact(slots, width(p.type()));
return slots;
}
public int resultSlots(Signature signature) {
var slots = 0;
for (var r : signature.results()) slots = Math.addExact(slots, width(r));
return slots;
}
public boolean sameSignature(Signature a, Signature b) {
if (a.parameters().size() != b.parameters().size() || a.results().size() != b.results().size()) return false;
for (int i=0; i<a.parameters().size(); i++) if (!id(a.parameters().get(i).type()).equals(id(b.parameters().get(i).type()))) return false;
for (int i=0; i<a.results().size(); i++) if (!id(a.results().get(i)).equals(id(b.results().get(i)))) return false;
return true;
}
public static LinkedRegistry link(IRRegistry source) {
final var declarations = new TreeMap<String, TypeDeclaration>();
for (var declaration : source.types()) {
final var old = declarations.putIfAbsent(declaration.identity().declarationKey(), declaration);
if (old != null && !old.equals(declaration)) throw new IllegalArgumentException("conflicting type declaration: " + declaration.identity());
}
if (declarations.size() > 65_536) throw new IllegalArgumentException("type record limit exceeded");
final var output = new ArrayList<TypeDeclaration>();
final var ids = new HashMap<TypeRef, TypeId>();
final var structuralKeys = new HashMap<Object, TypeId>();
final var remaining = new TreeMap<String, TypeDeclaration>();
long names = 0, nested = 0;
for (var d : declarations.values()) {
names += nameBytes(d.name());
for (var ref : references(d.body())) {
if (!declarations.containsKey(ref.declarationKey())) throw new IllegalArgumentException("undefined type: " + ref);
}
switch (d.body()) {
case Builtin b -> assign(d, List.of("builtin", b.id()), structuralKeys, output, ids);
case Struct s -> { assign(d, d.identity(), structuralKeys, output, ids); nested += s.fields().size(); for(var f:s.fields()) names += nameBytes(f.name()); }
case Trait t -> {
assign(d, d.identity(), structuralKeys, output, ids); nested += t.methods().size();
for(var m:t.methods()) { names += nameBytes(m.name()); nested += m.signature().parameters().size()+m.signature().results().size(); for(var p:m.signature().parameters()) names += nameBytes(p.name()); }
}
case Array ignored -> remaining.put(d.identity().declarationKey(), d);
case Fn f -> { remaining.put(d.identity().declarationKey(), d); nested += f.signature().parameters().size()+f.signature().results().size(); for(var p:f.signature().parameters()) names += nameBytes(p.name()); }
}
}
if (names > 4L*1024*1024 || nested > 1_048_576) throw new IllegalArgumentException("registry name/nested-record limit exceeded");
// Kahn traversal avoids recursive expansion and accepts nominal anchors.
final var pending = new HashMap<TypeRef, Integer>();
final var dependents = new HashMap<TypeRef, List<TypeRef>>();
final var ready = new PriorityQueue<TypeRef>(Comparator.comparing(TypeRef::declarationKey));
for(var d:remaining.values()) {
int count=0;
for(var r:references(d.body())) if(!ids.containsKey(r)) { count++; dependents.computeIfAbsent(r, ignored -> new ArrayList<>()).add(d.identity()); }
pending.put(d.identity(),count);
if(count==0) ready.add(d.identity());
}
while(!ready.isEmpty()) {
var reference=ready.remove(); var d=remaining.get(reference.declarationKey());
Object key;
if(d.body() instanceof Array a) key=List.of("array",ids.get(a.element()));
else {
var signature=((Fn)d.body()).signature();
key=List.of("fn",signature.parameters().stream().map(p->ids.get(p.type())).toList(),signature.results().stream().map(ids::get).toList());
}
assign(d,key,structuralKeys,output,ids);
for(var dependent:dependents.getOrDefault(reference,List.of())) if(pending.compute(dependent,(k,v)->v-1)==0) ready.add(dependent);
}
if(ids.size()!=declarations.size()) throw new IllegalArgumentException("structural Array/Fn cycle");
return new LinkedRegistry(output,ids);
}
private static void assign(TypeDeclaration d,Object key,Map<Object,TypeId> keys,List<TypeDeclaration> output,Map<TypeRef,TypeId> ids) {
var id=keys.get(key);
if(id==null) { id=new TypeId(output.size()); output.add(d); keys.put(key,id); }
ids.put(d.identity(),id);
}
private static List<TypeRef> references(TypeBody body) {
final var result=new ArrayList<TypeRef>();
switch(body) {
case Struct s -> s.fields().forEach(f->result.add(f.type()));
case Array a -> result.add(a.element());
case Fn f -> signatureReferences(f.signature(),result);
case Trait t -> t.methods().forEach(m->signatureReferences(m.signature(),result));
case Builtin ignored -> { }
}
return result;
}
private static void signatureReferences(Signature signature,List<TypeRef> result) {
signature.parameters().forEach(p->result.add(p.type())); result.addAll(signature.results().asList());
}
public static int nameBytes(String name) {
if(name==null) return 0;
try {
int size=java.nio.charset.StandardCharsets.UTF_8.newEncoder().encode(java.nio.CharBuffer.wrap(name)).remaining();
if(size>4096) throw new IllegalArgumentException("name exceeds 4096 UTF-8 bytes");
return size;
} catch(java.nio.charset.CharacterCodingException e) { throw new IllegalArgumentException("invalid UTF-8 source string",e); }
}
}
@@ -0,0 +1,62 @@
package p.studio.compiler.backend.registry;
import p.studio.compiler.models.IRBackendExecutableFunction;
import p.studio.compiler.models.registry.IRRegistry.*;
import p.studio.compiler.models.registry.IRRegistryInstruction;
import p.studio.compiler.backend.irvm.*;
import p.studio.compiler.source.identifiers.CallableId;
import p.studio.utilities.structures.ReadOnlyList;
import java.util.Map;
public final class RegistryInstructionLowerer {
private RegistryInstructionLowerer() {}
public static IRVMInstruction lower(IRRegistryInstruction instruction, LinkedRegistry registry,
IRBackendExecutableFunction current, ReadOnlyList<IRBackendExecutableFunction> functions, Map<CallableId,Integer> functionIds) {
int first=instruction.type()==null?instruction.member():registry.id(instruction.type()).index();
Integer second=null;int pops=0,pushes=0;
var body=instruction.type()==null?null:registry.type(instruction.type()).body();
switch(instruction.kind()) {
case MAKE_CLOSURE -> {
if(!(body instanceof Fn type))throw new IllegalArgumentException("closure type must be Fn");
var target=functionIds.get(instruction.function());if(target==null)throw new IllegalArgumentException("undefined closure function");
var metadata=functions.get(target).metadata();
if(metadata==null||!registry.sameSignature(type.signature(),metadata.signature()))throw new IllegalArgumentException("closure signature mismatch");
for(var capture:metadata.captures())pops=Math.addExact(pops,registry.width(capture));
second=first;first=target;pushes=1;
}
case CALL_CLOSURE -> {
if(!(body instanceof Fn type))throw new IllegalArgumentException("call type must be Fn");
pops=registry.parameterSlots(type.signature())+1;pushes=registry.resultSlots(type.signature());
}
case GET_CAPTURE -> {if(instruction.member()>=current.metadata().captures().size())throw new IllegalArgumentException("capture index");pushes=registry.width(current.metadata().captures().get(instruction.member()));}
case CALL_TRAIT -> {
if(!(body instanceof Trait trait)||instruction.member()>=trait.methods().size())throw new IllegalArgumentException("trait/method operand");
var signature=trait.methods().get(instruction.member()).signature();second=instruction.member();pops=registry.parameterSlots(signature)+2;pushes=registry.resultSlots(signature);
}
case DYN_PACK -> {pops=registry.width(instruction.type());pushes=2;}
case DYN_UNPACK -> {pops=2;pushes=registry.width(instruction.type());}
case DYN_CAN_UNPACK -> {pops=2;pushes=1;}
case REF_IS_EXACT,REF_IMPLEMENTS,REF_TO_TRAIT,REF_CAST -> {
if(!(body instanceof Struct||body instanceof Array||body instanceof Trait))throw new IllegalArgumentException("reference source category");
var target=registry.type(instruction.targetType()).body();boolean trait=instruction.kind()==IRRegistryInstruction.Kind.REF_IMPLEMENTS||instruction.kind()==IRRegistryInstruction.Kind.REF_TO_TRAIT;
if(trait?!(target instanceof Trait):!(target instanceof Struct||target instanceof Array))throw new IllegalArgumentException("reference target category");
second=registry.id(instruction.targetType()).index();pops=registry.width(instruction.type());pushes=instruction.kind()==IRRegistryInstruction.Kind.REF_TO_TRAIT?2:1;
}
case STRUCT_NEW,STRUCT_GET,STRUCT_SET -> {
if(!(body instanceof Struct struct))throw new IllegalArgumentException("Struct category required");
if(instruction.kind()==IRRegistryInstruction.Kind.STRUCT_NEW)pushes=1;
else {
if(instruction.member()>=struct.fields().size())throw new IllegalArgumentException("field index");
second=instruction.member();int width=registry.width(struct.fields().get(instruction.member()).type());
pops=instruction.kind()==IRRegistryInstruction.Kind.STRUCT_SET?1+width:1;pushes=instruction.kind()==IRRegistryInstruction.Kind.STRUCT_GET?width:0;
}
}
case ARRAY_NEW,ARRAY_LEN,ARRAY_GET,ARRAY_SET -> {
if(!(body instanceof Array array))throw new IllegalArgumentException("Array category required");
int width=registry.width(array.element());
switch(instruction.kind()) {case ARRAY_NEW,ARRAY_LEN->{pops=1;pushes=1;}case ARRAY_GET->{pops=2;pushes=width;}case ARRAY_SET->pops=2+width;default->throw new AssertionError();}
}
}
return new IRVMInstruction(new IRVMOp("REGISTRY_"+instruction.kind(),instruction.kind().opcode(),second==null?4:8,pops,pushes,false,false,false),first,second);
}
}
@@ -201,6 +201,7 @@ public final class LifecycleAssemblerService {
frameExecutable.span()));
return IRBackend.builder()
.registry(backend.getRegistry())
.entryPointCallableName(entryPointCallableName)
.entryPointModuleId(entryPointModuleId)
.functions(backend.getFunctions())
@@ -54,6 +54,10 @@ public class LowerToIRVMPipelineStage implements PipelineStage {
.pc(e.pc())
.message("[BUILD]: lower to irvm validation failed: " + e.getMessage())
.exception(e));
} catch (IllegalArgumentException e) {
return BuildingIssueSink.empty().report(builder -> builder.error(true)
.phase("BACKEND_LOWER_TO_IRVM").code("LOWER_IRVM_INVALID_TYPED_REGISTRY")
.message("[BUILD]: invalid common typed IR: " + e.getMessage()).exception(e));
}
return BuildingIssueSink.empty();
}
@@ -106,7 +106,7 @@ class BytecodeEmitterTest {
0,
1,
ReadOnlyList.from(
BytecodeEmitter.Operation.intrinsic(0x2000),
BytecodeEmitter.Operation.intrinsic(0x2000, 0, 0, null),
BytecodeEmitter.Operation.halt())))));
assertTrue(module.syscalls().isEmpty());
@@ -131,7 +131,7 @@ class BytecodeEmitterTest {
ReadOnlyList.from(
BytecodeEmitter.Operation.halt(new BytecodeModule.SourceSpan(1, 0, 1)),
BytecodeEmitter.Operation.callFunc(0, new BytecodeModule.SourceSpan(1, 2, 3)),
BytecodeEmitter.Operation.intrinsic(0x2000, new BytecodeModule.SourceSpan(1, 4, 5)),
BytecodeEmitter.Operation.intrinsic(0x2000, 0, 0, new BytecodeModule.SourceSpan(1, 4, 5)),
BytecodeEmitter.Operation.ret(new BytecodeModule.SourceSpan(1, 6, 7)))))));
assertEquals(4, module.debugInfo().pcToSpan().size());
@@ -22,13 +22,13 @@ class BytecodeModuleTest {
assertEquals(0, bytes[3]);
final var sectionCount = readU32(bytes, 8);
assertEquals(1, sectionCount);
assertEquals(8, sectionCount);
final var sectionKind = readU32(bytes, 32);
final var sectionOffset = readU32(bytes, 36);
final var sectionLength = readU32(bytes, 40);
final var sectionKind = readU32(bytes, 32 + 4 * 12);
final var sectionOffset = readU32(bytes, 36 + 4 * 12);
final var sectionLength = readU32(bytes, 40 + 4 * 12);
assertEquals(5, sectionKind);
assertEquals(44, sectionOffset);
assertEquals(140, sectionOffset);
assertEquals(4, sectionLength);
final var syscCount = readU32(bytes, sectionOffset);
@@ -0,0 +1,53 @@
package p.studio.compiler.backend.registry;
import org.junit.jupiter.api.Test;
import p.studio.compiler.models.registry.IRRegistry;
import p.studio.compiler.models.registry.IRRegistry.*;
import p.studio.utilities.structures.ReadOnlyList;
import static org.junit.jupiter.api.Assertions.*;
class LinkedRegistryTest {
private static TypeRef ref(String key) { return new TypeRef(key); }
private static TypeDeclaration decl(String key, TypeBody body) { return new TypeDeclaration(ref(key), "same display name", body); }
private static IRRegistry source(TypeDeclaration... types) { return new IRRegistry(ReadOnlyList.from(types), null); }
@Test void nominalIdentityIsNotNameOrShapeAndRepeatedDeclarationsConverge() {
var a=decl("module:a",new Struct(null)); var b=decl("module:b",new Struct(null));
var linked=LinkedRegistry.link(source(a,b,a));
assertEquals(2,linked.types().size());
assertNotEquals(linked.id(a.identity()),linked.id(b.identity()));
assertEquals(linked.types(),LinkedRegistry.link(source(b,a)).types());
}
@Test void importsArraysAndFnSignaturesCanonicalizeWithoutUsingParameterNames() {
var signature=new Signature(ReadOnlyList.from(new NamedType(ref("v1"),"x")),ReadOnlyList.from(ref("v2")));
var renamed=new Signature(ReadOnlyList.from(new NamedType(ref("v2"),"renamed")),ReadOnlyList.from(ref("v1")));
var linked=LinkedRegistry.link(source(decl("v1",new Builtin(2)),decl("v2",new Builtin(2)),
decl("a1",new Array(ref("v1"))),decl("a2",new Array(ref("v2"))),
decl("f1",new Fn(signature)),decl("f2",new Fn(renamed))));
assertEquals(3,linked.types().size());
assertEquals(linked.id(ref("v1")),linked.id(ref("v2")));
assertEquals(linked.id(ref("a1")),linked.id(ref("a2")));
assertEquals(linked.id(ref("f1")),linked.id(ref("f2")));
assertEquals(2,linked.parameterSlots(signature));
assertEquals(2,linked.resultSlots(signature));
}
@Test void nominalAnchorsAllowForwardReferencesButStructuralCyclesReject() {
var signature=new Signature(ReadOnlyList.from(new NamedType(ref("array"),null)),null);
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source(
decl("fn",new Fn(signature)),decl("array",new Array(ref("fn"))))));
assertDoesNotThrow(()->LinkedRegistry.link(source(
decl("fn",new Fn(signature)),decl("array",new Array(ref("node"))),
decl("node",new Struct(ReadOnlyList.from(new NamedType(ref("fn"),null)))))));
}
@Test void missingReferencesConflictingDeclarationsAndRemovedImportsReject() {
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source(decl("a",new Array(ref("missing"))))));
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source(decl("a",new Builtin(1)),decl("a",new Builtin(2)))));
assertThrows(IllegalArgumentException.class,()->new Builtin(0x100));
assertEquals(1,LinkedRegistry.link(source(decl("random",new Builtin(0x16)))).width(ref("random")));
}
@Test void namesAreStrictUtf8AndBounded() {
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.nameBytes("\uD800"));
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.nameBytes("é".repeat(2049)));
assertEquals(4096,LinkedRegistry.nameBytes("é".repeat(2048)));
}
}
@@ -0,0 +1,188 @@
package p.studio.compiler.backend.registry;
import org.junit.jupiter.api.Test;
import p.studio.compiler.backend.bytecode.*;
import p.studio.compiler.models.registry.IRRegistry;
import p.studio.compiler.models.registry.IRRegistry.*;
import p.studio.utilities.structures.ReadOnlyList;
import java.nio.*;
import java.nio.file.*;
import java.util.*;
import static org.junit.jupiter.api.Assertions.*;
class PbxRegistryConformanceTest {
private static byte[] instructions(int[]... instructions) {
var out=new java.io.ByteArrayOutputStream();
for(var instruction:instructions) {
out.write(instruction[0]);out.write(instruction[0]>>>8);
for(int n=1;n<instruction.length;n++)for(int b=0;b<4;b++)out.write(instruction[n]>>>(8*b));
}
return out.toByteArray();
}
private static int[] op(int... values) { return values; }
private static final BytecodePreloadVerifierService VERIFIER = new BytecodePreloadVerifierService();
private static IRRegistry registry(int builtin) {
return new IRRegistry(ReadOnlyList.from(new TypeDeclaration(TypeRef.builtin(builtin), null, new Builtin(builtin))), null);
}
private static byte[] hex(String value) { return HexFormat.of().parseHex(value.replaceAll("\\s", "")); }
private static BytecodeModule module(IRRegistry registry, byte[] code, int parameters, int results, int max, FunctionMetadata metadata) {
return new BytecodeModule(0, null, ReadOnlyList.from(new BytecodeModule.FunctionMeta(0, code.length, parameters, 0, results, max, metadata)), code, null, null, null, registry);
}
private static BytecodeModule emptyFunction(byte[] code, int max) {
return module(registry(0x13), code, 0, 0, max, FunctionMetadata.empty());
}
private static void fixture(String name, BytecodeModule module) throws Exception {
Path directory=Path.of("build/registry-conformance"); Files.createDirectories(directory);
Files.write(directory.resolve(name+".pbx"), module.serialize());
}
private static byte[] section(byte[] bytes, int kind) {
var b=ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN);
for(int i=0;i<b.getInt(8);i++) {int at=32+12*i; if(b.getInt(at)==kind)return Arrays.copyOfRange(bytes,b.getInt(at+4),b.getInt(at+4)+b.getInt(at+8));}
throw new AssertionError("missing section "+kind);
}
@Test void matchesIndependent194ByteOracle() throws Exception {
var module=emptyFunction(hex("01 00"),0);
String source=Files.readString(Path.of("src/test/resources/registry/minimal-runtime-v0.hex"));
byte[] expected=hex(source.substring(source.indexOf('\n')+1));
assertEquals(194, expected.length);
assertArrayEquals(expected,module.serialize());
VERIFIER.verify(module);fixture("minimal",module);
}
@Test void vec2SignatureHasLogicalArityAndNoWireWidthCaches() throws Exception {
var type=TypeRef.builtin(2);
var metadata=FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(type,null)),ReadOnlyList.from(type)));
var module=module(registry(2),hex("42 00 00 00 00 00 42 00 01 00 00 00 51 00"),2,2,2,metadata);
assertArrayEquals(hex("01 00 00 00 00 00 01 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 0e 00 00 00 00 00 00 00 02 00 00 00"),section(module.serialize(),1));
VERIFIER.verify(module);fixture("vec2-signature",module);
assertThrows(BytecodeMarshalingException.class,()->module(registry(2),module.code(),1,1,2,metadata).serialize());
}
@Test void absoluteJumpToFunctionEndAndUnreachableOperands() throws Exception {
var module=emptyFunction(hex("02 00 08 00 00 00 00 00"),0);
VERIFIER.verify(module);fixture("jump-end",module);
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("02 00 03 00 00 00 01 00"),0)));
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("01 00 50 00 ff ff ff ff"),0)));
for(int op:new int[]{0x54,0x55,0x56,0x70})assertThrows(RuntimeException.class,()->VERIFIER.verify(emptyFunction(new byte[]{1,0,(byte)op,0,0,0,0,0},0)));
}
@Test void literalZeroAndExactReturnHeight() {
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 11 00 51 00"),0)));
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 51 00"),1)));
assertDoesNotThrow(()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 11 00 51 00"),1)));
}
@Test void nonfiniteUnreachableImmediateAndConstantReject() {
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("01 00 15 00 00 00 00 00 00 00 f0 7f"),0)));
var m=emptyFunction(hex("01 00"),0);
var bad=new BytecodeModule(0,ReadOnlyList.from(new BytecodeModule.Float64Constant(Double.NaN)),m.functions(),m.code(),null,null,null,m.registry());
assertThrows(BytecodeMarshalingException.class,bad::serialize);
}
@Test void signatureNamesAreIndependentFromDebugAndStrictUtf8() {
var metadata=new FunctionMetadata(Signature.empty(),null,Role.NONE,"");
var module=module(registry(0x13),hex("01 00"),0,0,0,metadata);
byte[] function=section(module.serialize(),1);
assertEquals(1,function[5]);assertEquals(34,function.length);
var bad=new FunctionMetadata(Signature.empty(),null,Role.NONE,"\uD800");
assertThrows(BytecodeMarshalingException.class,()->module(registry(0x13),hex("01 00"),0,0,0,bad).serialize());
}
@Test void unusedAndConflictingNativeDeclarationsRejectWithoutBindingFallback() {
var m=emptyFunction(hex("01 00"),0);
var declaration=new BytecodeModule.SyscallDecl("test","native",0,0,0);
var unused=new BytecodeModule(0,null,m.functions(),m.code(),null,null,ReadOnlyList.from(declaration),m.registry());
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(unused));
var duplicate=new BytecodeModule(0,null,m.functions(),m.code(),null,null,ReadOnlyList.from(declaration,declaration),m.registry());
assertThrows(BytecodeMarshalingException.class,duplicate::serialize);
}
@Test void closuresCarryExplicitEnvironmentAndExactFnIdentity() throws Exception {
var integer=TypeRef.builtin(0x10);var fn=new TypeRef("fixture:fn");
var signature=new Signature(null,ReadOnlyList.from(integer));
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(fn,null,new Fn(signature))),null);
int fnId=LinkedRegistry.link(registry).id(fn).index();
byte[] boot=instructions(op(0x17,42),op(0x52,1,fnId),op(0x53,fnId),op(1));
byte[] body=instructions(op(0x57,0),op(0x51));
var bytes=new java.io.ByteArrayOutputStream();bytes.writeBytes(boot);bytes.writeBytes(body);
var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,boot.length,0,0,0,1,FunctionMetadata.empty()),
new BytecodeModule.FunctionMeta(boot.length,body.length,0,0,1,1,new FunctionMetadata(signature,ReadOnlyList.from(integer),Role.NONE)));
var m=new BytecodeModule(0,null,functions,bytes.toByteArray(),null,null,null,registry);
VERIFIER.verify(m);fixture("closure-capture",m);
byte[] direct=m.code().clone();direct[6]=0x50;
var invalid=new BytecodeModule(0,null,functions,direct,null,null,null,registry);
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(invalid));
}
@Test void traitImplementationUsesConcreteReceiverAndOrderedMethods() throws Exception {
var integer=TypeRef.builtin(0x10);var owner=new TypeRef("fixture:owner");var trait=new TypeRef("fixture:trait");
var result=new Signature(null,ReadOnlyList.from(integer));
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),
new TypeDeclaration(owner,null,new Struct(null)),new TypeDeclaration(trait,null,new Trait(ReadOnlyList.from(new Method("answer",result))))),
ReadOnlyList.from(new Implementation("fixture:impl",owner,trait,ReadOnlyList.from(new p.studio.compiler.source.identifiers.CallableId(1)))));
var linked=LinkedRegistry.link(registry);int ownerId=linked.id(owner).index(),traitId=linked.id(trait).index();
byte[] boot=instructions(op(0xa0,ownerId),op(0x95,ownerId,traitId),op(0x58,traitId,0),op(1));
byte[] body=instructions(op(0x17,42),op(0x51));
var bytes=new java.io.ByteArrayOutputStream();bytes.writeBytes(boot);bytes.writeBytes(body);
var implementation=FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(owner,"self")),ReadOnlyList.from(integer)));
var m=new BytecodeModule(0,null,ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,boot.length,0,0,0,2,FunctionMetadata.empty()),
new BytecodeModule.FunctionMeta(boot.length,body.length,1,0,1,1,implementation)),bytes.toByteArray(),null,null,null,registry);
VERIFIER.verify(m);fixture("trait-call",m);
}
@Test void arrayAndDynamicOperationsUseRegistryOperands() throws Exception {
var integer=TypeRef.builtin(0x10);var array=new TypeRef("fixture:array");
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(array,null,new Array(integer))),null);
var linked=LinkedRegistry.link(registry);int arrayId=linked.id(array).index(),intId=linked.id(integer).index();
byte[] code=instructions(op(0x14,1,0),op(0xa3,arrayId),op(0x14,0,0),op(0xa5,arrayId),op(0x90,intId),op(0x91,intId),op(1));
var m=module(registry,code,0,0,2,FunctionMetadata.empty());
VERIFIER.verify(m);fixture("array-dynamic",m);
}
@Test void bindingFixturesKeepExactCanonicalIdentityAndAbi() throws Exception {
var m=emptyFunction(instructions(op(0x71,0),op(0x11),op(1)),1);
for(String name:List.of("native", "unknown-native", "native-abi", "unused-native")) {
var declaration=new BytecodeModule.SyscallDecl("system",name.equals("unknown-native")?"missing":"has_cart",1,name.equals("native-abi")?1:0,1);
byte[] code=name.equals("unused-native")?instructions(op(1)):m.code();
var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,code.length,0,0,0,1,FunctionMetadata.empty()));
var module=new BytecodeModule(0,null,functions,code,null,null,ReadOnlyList.from(declaration),m.registry());
fixture(name,module);
if(name.equals("native"))VERIFIER.verify(module);
}
}
@Test void rolesStayMetadataAndValidateSignatureAndUniqueness() throws Exception {
var pause=new FunctionMetadata(Signature.empty(),null,Role.BEFORE_PAUSE);
var resume=new FunctionMetadata(Signature.empty(),null,Role.BEFORE_RESUME);
var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,2,0,0,0,0,FunctionMetadata.empty()),
new BytecodeModule.FunctionMeta(2,2,0,0,0,0,pause),new BytecodeModule.FunctionMeta(4,2,0,0,0,0,resume));
var m=new BytecodeModule(0,null,functions,instructions(op(1),op(0x51),op(0x51)),null,null,null,IRRegistry.empty());
VERIFIER.verify(m);fixture("roles",m);
assertThrows(BytecodeMarshalingException.class,()->module(IRRegistry.empty(),instructions(op(1)),0,0,0,pause).serialize());
var duplicated=ReadOnlyList.from(functions.get(0),functions.get(1),new BytecodeModule.FunctionMeta(4,2,0,0,0,0,pause));
assertThrows(BytecodeMarshalingException.class,()->new BytecodeModule(0,null,duplicated,m.code(),null,null,null,m.registry()).serialize());
}
@Test void joinsWidenFactsAndLoopsReachFixedPoint() throws Exception {
// bool; branch to alternate scalar; equal-height join becomes Unknown, then POP.
byte[] code=instructions(op(0x10,0),op(0x03,24),op(0x17,1),op(0x02,30),op(0x10,1),op(0x11),op(1));
var base=emptyFunction(code,1);
var m=new BytecodeModule(0,ReadOnlyList.from(new BytecodeModule.BooleanConstant(false),new BytecodeModule.StringConstant("alternate")),base.functions(),code,null,null,null,base.registry());
VERIFIER.verify(m);fixture("join",m);
byte[] loop=instructions(op(0x40,0),op(0x03,18),op(0x02,0),op(1));
assertDoesNotThrow(()->VERIFIER.verify(emptyFunction(loop,1)));
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("16 00 01 16 00 00 20 00 01 00"),2)));
}
@Test void memcardRoleHasExactTypedSignatureWithoutLanguagePolicy() throws Exception {
var input=TypeRef.builtin(0x11);var output=TypeRef.builtin(0x10);
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(input,null,new Builtin(0x11)),new TypeDeclaration(output,null,new Builtin(0x10))),null);
var metadata=new FunctionMetadata(new Signature(ReadOnlyList.from(new NamedType(input,"status")),ReadOnlyList.from(output)),null,Role.MEMCARD_BOOT);
var code=instructions(op(1),op(0x17,0),op(0x51));
var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,2,0,0,0,0,FunctionMetadata.empty()),new BytecodeModule.FunctionMeta(2,8,1,0,1,1,metadata));
var module=new BytecodeModule(0,null,functions,code,null,null,null,registry);
VERIFIER.verify(module);fixture("memcard-role",module);
}
@Test void knownSemanticGroupsCannotSilentlyChangeType() {
var color=TypeRef.builtin(1);var integer=TypeRef.builtin(0x11);
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(color,null,new Builtin(1)),new TypeDeclaration(integer,null,new Builtin(0x11))),null);
var metadata=FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(color,null)),ReadOnlyList.from(integer)));
var module=module(registry,instructions(op(0x42,0),op(0x51)),1,1,1,metadata);
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(module));
}
@Test void intrinsicFixtureUsesExplicitAbiAndKeepsNativeBindingSeparate() throws Exception {
// Runtime vec2.dot@1 (0x1000): four Float carriers -> one Float.
byte[] code=instructions(op(0x15,0,0x3ff00000),op(0x15,0,0x40000000),
op(0x15,0,0x40080000),op(0x15,0,0x40100000),op(0x72,0x1000),op(1));
var base=emptyFunction(code,4);
var module=new BytecodeModule(0,null,base.functions(),code,null,null,null,base.registry(),ReadOnlyList.from(new BytecodeModule.IntrinsicSignature(0x1000,4,1)));
VERIFIER.verify(module);fixture("intrinsic-dot",module);
}
}
@@ -0,0 +1,46 @@
package p.studio.compiler.backend.registry;
import org.junit.jupiter.api.Test;
import p.studio.compiler.backend.bytecode.*;
import p.studio.compiler.backend.irvm.*;
import p.studio.compiler.models.*;
import p.studio.compiler.models.registry.*;
import p.studio.compiler.models.registry.IRRegistry.*;
import p.studio.compiler.source.identifiers.*;
import p.studio.utilities.structures.ReadOnlyList;
import java.nio.file.*;
import static org.junit.jupiter.api.Assertions.*;
class TypedCommonPipelineTest {
private static IRBackendExecutableFunction.Instruction plain(IRBackendExecutableFunction.InstructionKind kind, String literal) {
return new IRBackendExecutableFunction.Instruction(kind,null,null,null,null,null,literal,null,null,null,null);
}
private static IRBackendExecutableFunction.Instruction typed(IRRegistryInstruction operand) {
return new IRBackendExecutableFunction.Instruction(IRBackendExecutableFunction.InstructionKind.REGISTRY,null,null,null,null,null,null,null,null,null,null,operand);
}
@Test void syntheticFrontendClosureSurvivesLinkLowerOptimizeAndEmit() throws Exception {
var integer=TypeRef.builtin(0x10);var fn=new TypeRef("synthetic:fn");
var signature=new Signature(null,ReadOnlyList.from(integer));
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(fn,null,new Fn(signature))),null);
var capture=new CallableId(71);
var main=new IRBackendExecutableFunction(new FileId(0),new ModuleId(0),"main",new CallableId(91),0,0,0,0,0,1,
ReadOnlyList.from(plain(IRBackendExecutableFunction.InstructionKind.PUSH_I32,"42"),
typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.MAKE_CLOSURE,fn,null,capture,0)),
typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.CALL_CLOSURE,fn,null,null,0)),
plain(IRBackendExecutableFunction.InstructionKind.POP,null),plain(IRBackendExecutableFunction.InstructionKind.RET,null)),null,FunctionMetadata.empty());
var body=new IRBackendExecutableFunction(new FileId(0),new ModuleId(0),"captured",capture,0,0,0,0,1,1,
ReadOnlyList.from(typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.GET_CAPTURE,null,null,null,0)),
plain(IRBackendExecutableFunction.InstructionKind.RET,null)),null,new FunctionMetadata(signature,ReadOnlyList.from(integer),Role.NONE));
var backend=IRBackend.builder().entryPointModuleId(new ModuleId(0)).registry(registry).executableFunctions(ReadOnlyList.from(body,main)).build();
var lowered=new LowerToIRVMService().lower(backend);
var optimized=new OptimizeIRVMService().optimize(lowered);
assertEquals(registry,optimized.module().registry());
assertEquals(body.metadata(),optimized.module().functions().get(1).metadata());
var module=new BytecodeEmitter().emit(optimized.coherentEmissionPlan());
new BytecodePreloadVerifierService().verify(module);
var repeated=new BytecodeEmitter().emit(new OptimizeIRVMService().optimize(new LowerToIRVMService().lower(backend)).coherentEmissionPlan());
assertArrayEquals(module.serialize(),repeated.serialize());
Files.createDirectories(Path.of("build/registry-conformance"));
Files.write(Path.of("build/registry-conformance/common-pipeline.pbx"),module.serialize());
}
}
@@ -1,6 +1,8 @@
package p.studio.compiler.workspaces.stages;
import org.junit.jupiter.api.Test;
import p.studio.compiler.models.registry.IRRegistry;
import p.studio.compiler.models.registry.IRRegistry.*;
import p.studio.compiler.messages.BuilderPipelineConfig;
import p.studio.compiler.models.BuilderPipelineContext;
import p.studio.compiler.models.IRBackend;
@@ -144,6 +146,7 @@ class LowerToIRVMPipelineStageTest {
final var ctx = BuilderPipelineContext.fromConfig(new BuilderPipelineConfig(false, "."));
ctx.irBackend = IRBackend.builder()
.entryPointModuleId(new p.studio.compiler.source.identifiers.ModuleId(0))
.registry(new IRRegistry(ReadOnlyList.from(new TypeDeclaration(TypeRef.builtin(0x10),null,new Builtin(0x10))),null))
.executableFunctions(ReadOnlyList.from(
new IRBackendExecutableFunction(
new FileId(0),
@@ -162,7 +165,7 @@ class LowerToIRVMPipelineStageTest {
null,
null,
Span.none())),
Span.none()),
Span.none(), FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(TypeRef.builtin(0x10),null)),null))),
new IRBackendExecutableFunction(
new FileId(0),
new p.studio.compiler.source.identifiers.ModuleId(0),
@@ -1 +1 @@
504258000000000004000000000000000000000000000000000000000000000001000000500000001400000002000000640000000e00000003000000720000004400000005000000b60000001d00000001000000000000000e000000000000000000020071000000000072000020000051000300000000000000010000000000000000000000060000000100000000000000000000000c0000000100000000000000000000000100000000000000040000006d61696e01000000050067667832640a00647261775f706978656c010000000000
5042580000000000090000000000000000000000000000000000000000000000000000008c0000000400000001000000900000001e00000002000000ae0000000e00000003000000bc0000004400000004000000000100000400000005000000040100001d000000060000002101000004000000070000002501000004000000080000002901000008000000000000000100000000000000000000000000000000000e000000000000000200000071000000000072000020000051000300000000000000010000000000000000000000060000000100000000000000000000000c0000000100000000000000000000000100000000000000040000006d61696e0000000001000000050067667832640a00647261775f706978656c01000000000000000000000000000100000000000000
@@ -0,0 +1,20 @@
# Independent runtime registry_v0.rs minimal_bytes oracle (194 bytes)
50 42 58 00 00 00 00 00 08 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 80 00 00 00 04 00 00 00
01 00 00 00 84 00 00 00 1e 00 00 00
02 00 00 00 a2 00 00 00 02 00 00 00
04 00 00 00 a4 00 00 00 04 00 00 00
05 00 00 00 a8 00 00 00 04 00 00 00
06 00 00 00 ac 00 00 00 0a 00 00 00
07 00 00 00 b6 00 00 00 04 00 00 00
08 00 00 00 ba 00 00 00 08 00 00 00
00 00 00 00
01 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00
01 00
00 00 00 00
00 00 00 00
01 00 00 00 00 00 13 00 00 00
00 00 00 00
01 00 00 00 00 00 00 00
@@ -1,6 +1,7 @@
package p.studio.compiler.models;
import lombok.Builder;
import p.studio.compiler.models.registry.IRRegistry;
import lombok.Getter;
import p.studio.compiler.source.identifiers.CallableId;
import p.studio.compiler.source.identifiers.IntrinsicId;
@@ -34,6 +35,8 @@ public class IRBackend implements IRBackendReader {
private final ReadOnlyList<IntrinsicReference> intrinsicPool = ReadOnlyList.empty();
@Builder.Default
private final IRReservedMetadata reservedMetadata = IRReservedMetadata.empty();
@Builder.Default
private final IRRegistry registry = IRRegistry.empty();
private final IRLifecycleDeclaration lifecycleDeclaration;
public static IRBackendAggregator aggregator() {
@@ -41,6 +44,8 @@ public class IRBackend implements IRBackendReader {
}
public static final class IRBackendAggregator {
private final ArrayList<IRRegistry.TypeDeclaration> registryTypes = new ArrayList<>();
private final ArrayList<IRRegistry.Implementation> registryImplementations = new ArrayList<>();
private String entryPointCallableName;
private ModuleId entryPointModuleId = ModuleId.none();
private final ArrayList<IRFunction> functions = new ArrayList<>();
@@ -90,6 +95,11 @@ public class IRBackend implements IRBackendReader {
}
final var callableRemap = reindexCallables(backendFile.callableSignatures(), moduleRemap);
final var intrinsicRemap = reindexIntrinsics(backendFile.intrinsicPool());
registryTypes.addAll(backendFile.registry().types().asList());
for (final var impl : backendFile.registry().implementations()) {
registryImplementations.add(new IRRegistry.Implementation(impl.declarationKey(), impl.owner(), impl.trait(),
ReadOnlyList.wrap(impl.methods().stream().map(id -> remapCallableId(id, callableRemap, "implementation")).toList())));
}
for (final var function : backendFile.executableFunctions()) {
executableFunctions.add(remapExecutableFunction(function, moduleRemap, callableRemap, intrinsicRemap));
}
@@ -200,7 +210,9 @@ public class IRBackend implements IRBackendReader {
instruction.targetLabel(),
instruction.expectedArgSlots(),
instruction.expectedRetSlots(),
instruction.span()));
instruction.span(), instruction.registryInstruction() == null ? null :
new p.studio.compiler.models.registry.IRRegistryInstruction(instruction.registryInstruction().kind(), instruction.registryInstruction().type(), instruction.registryInstruction().targetType(),
instruction.registryInstruction().function() == null ? null : remapCallableId(instruction.registryInstruction().function(), callableRemap, "Registry instruction"), instruction.registryInstruction().member())));
}
return new IRBackendExecutableFunction(
function.fileId(),
@@ -214,7 +226,7 @@ public class IRBackend implements IRBackendReader {
function.returnSlots(),
function.maxStackSlots(),
ReadOnlyList.wrap(remappedInstructions),
function.span());
function.span(), function.metadata());
}
private IRLifecycleFileDeclaration remapLifecycleFileDeclaration(
@@ -367,6 +379,7 @@ public class IRBackend implements IRBackendReader {
public IRBackend emit() {
return IRBackend
.builder()
.registry(new IRRegistry(ReadOnlyList.wrap(registryTypes), ReadOnlyList.wrap(registryImplementations)))
.entryPointCallableName(resolveEntryPointCallableName())
.entryPointModuleId(resolveEntryPointModuleId())
.functions(ReadOnlyList.wrap(functions))
@@ -10,6 +10,20 @@ import p.studio.utilities.structures.ReadOnlyList;
import java.util.Objects;
public record IRBackendExecutableFunction(
FileId fileId,
ModuleId moduleId,
String callableName,
CallableId callableId,
int sourceStart,
int sourceEnd,
int paramSlots,
int localSlots,
int returnSlots,
int maxStackSlots,
ReadOnlyList<Instruction> instructions,
Span span,
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
public IRBackendExecutableFunction(
FileId fileId,
ModuleId moduleId,
String callableName,
@@ -22,6 +36,9 @@ public record IRBackendExecutableFunction(
int maxStackSlots,
ReadOnlyList<Instruction> instructions,
Span span) {
this(fileId, moduleId, callableName, callableId, sourceStart, sourceEnd, paramSlots, localSlots, returnSlots, maxStackSlots, instructions, span, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
}
public IRBackendExecutableFunction(
final FileId fileId,
@@ -82,6 +99,19 @@ public record IRBackendExecutableFunction(
}
public record Instruction(
InstructionKind kind,
ModuleId calleeModuleId,
String calleeCallableName,
CallableId calleeCallableId,
HostCallMetadata hostCall,
IntrinsicCallMetadata intrinsicCall,
String label,
String targetLabel,
Integer expectedArgSlots,
Integer expectedRetSlots,
Span span,
p.studio.compiler.models.registry.IRRegistryInstruction registryInstruction) {
public Instruction(
InstructionKind kind,
ModuleId calleeModuleId,
String calleeCallableName,
@@ -93,6 +123,9 @@ public record IRBackendExecutableFunction(
Integer expectedArgSlots,
Integer expectedRetSlots,
Span span) {
this(kind, calleeModuleId, calleeCallableName, calleeCallableId, hostCall, intrinsicCall, label, targetLabel, expectedArgSlots, expectedRetSlots, span, null);
}
public Instruction(
final InstructionKind kind,
final ModuleId calleeModuleId,
@@ -217,7 +250,11 @@ public record IRBackendExecutableFunction(
if (expectedRetSlots != null && expectedRetSlots < 0) {
throw new IllegalArgumentException("expectedRetSlots must be non-negative");
}
if ((kind == InstructionKind.REGISTRY) != (registryInstruction != null)) throw new IllegalArgumentException("Registry operand mismatch");
switch (kind) {
case REGISTRY -> {
if (hostCall != null || intrinsicCall != null || calleeCallableId != null || !label.isEmpty() || !targetLabel.isEmpty()) throw new IllegalArgumentException("unexpected Registry instruction metadata");
}
case PUSH_I32 -> {
if (label.isBlank()) {
throw new IllegalArgumentException("PUSH_I32 requires immediate label payload");
@@ -387,6 +424,7 @@ public record IRBackendExecutableFunction(
}
public enum InstructionKind {
REGISTRY,
PUSH_I32,
PUSH_BOOL,
PUSH_CONST,
@@ -7,8 +7,22 @@ import p.studio.compiler.source.tables.ModuleReference;
import p.studio.utilities.structures.ReadOnlyList;
import java.util.Objects;
import p.studio.compiler.models.registry.IRRegistry;
public record IRBackendFile(
FileId fileId,
ReadOnlyList<IRFunction> functions,
ReadOnlyList<IRSyntheticFunction> syntheticFunctions,
ReadOnlyList<IRGlobal> globals,
ReadOnlyList<IRBackendExecutableFunction> executableFunctions,
IRReservedMetadata reservedMetadata,
ReadOnlyList<ModuleReference> modulePool,
ReadOnlyList<CallableSignatureRef> callableSignatures,
ReadOnlyList<IntrinsicReference> intrinsicPool,
IRLifecycleFileDeclaration lifecycleDeclaration,
IRRegistry registry) {
public IRBackendFile(
FileId fileId,
ReadOnlyList<IRFunction> functions,
ReadOnlyList<IRSyntheticFunction> syntheticFunctions,
@@ -19,7 +33,11 @@ public record IRBackendFile(
ReadOnlyList<CallableSignatureRef> callableSignatures,
ReadOnlyList<IntrinsicReference> intrinsicPool,
IRLifecycleFileDeclaration lifecycleDeclaration) {
this(fileId, functions, syntheticFunctions, globals, executableFunctions, reservedMetadata, modulePool, callableSignatures, intrinsicPool, lifecycleDeclaration, IRRegistry.empty());
}
public IRBackendFile {
registry = registry == null ? IRRegistry.empty() : registry;
fileId = Objects.requireNonNull(fileId, "fileId");
functions = functions == null ? ReadOnlyList.empty() : functions;
syntheticFunctions = syntheticFunctions == null ? ReadOnlyList.empty() : syntheticFunctions;
@@ -7,6 +7,7 @@ import p.studio.compiler.source.tables.ModuleReference;
import p.studio.utilities.structures.ReadOnlyList;
public interface IRBackendReader {
p.studio.compiler.models.registry.IRRegistry getRegistry();
String getEntryPointCallableName();
ModuleId getEntryPointModuleId();
ReadOnlyList<IRFunction> getFunctions();
@@ -0,0 +1,82 @@
package p.studio.compiler.models.registry;
import p.studio.compiler.source.identifiers.CallableId;
import p.studio.utilities.structures.ReadOnlyList;
import java.util.Objects;
/** Frontend-neutral declarations. References are stable keys, never Java object identity or image IDs. */
public record IRRegistry(ReadOnlyList<TypeDeclaration> types, ReadOnlyList<Implementation> implementations) {
public IRRegistry {
types = copy(types);
implementations = copy(implementations);
}
public static IRRegistry empty() { return new IRRegistry(null, null); }
private static <T> ReadOnlyList<T> copy(ReadOnlyList<T> values) {
return values == null ? ReadOnlyList.empty() : ReadOnlyList.wrap(values.asList());
}
public record TypeRef(String declarationKey) {
public TypeRef {
Objects.requireNonNull(declarationKey, "declarationKey");
if (declarationKey.isBlank()) throw new IllegalArgumentException("empty type declaration key");
}
public static TypeRef builtin(int id) { return new TypeRef("builtin:" + id); }
}
public record TypeDeclaration(TypeRef identity, String name, TypeBody body) {
public TypeDeclaration { Objects.requireNonNull(identity); Objects.requireNonNull(body); }
}
public sealed interface TypeBody permits Builtin, Struct, Array, Trait, Fn {}
public record Builtin(int id) implements TypeBody {
public Builtin {
switch (id) {
case 1, 2, 3, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16 -> { }
default -> throw new IllegalArgumentException("unsupported BuiltinId: " + id);
}
}
}
public record Struct(ReadOnlyList<NamedType> fields) implements TypeBody {
public Struct { fields = copy(fields); }
}
public record Array(TypeRef element) implements TypeBody {
public Array { Objects.requireNonNull(element); }
}
public record Trait(ReadOnlyList<Method> methods) implements TypeBody {
public Trait { methods = copy(methods); }
}
public record Fn(Signature signature) implements TypeBody {
public Fn { Objects.requireNonNull(signature); }
}
public record NamedType(TypeRef type, String name) {
public NamedType { Objects.requireNonNull(type); }
}
public record Signature(ReadOnlyList<NamedType> parameters, ReadOnlyList<TypeRef> results) {
public Signature { parameters = copy(parameters); results = copy(results); }
public static Signature empty() { return new Signature(null, null); }
}
public record Method(String name, Signature signature) {
public Method { Objects.requireNonNull(signature); }
}
public record Implementation(String declarationKey, TypeRef owner, TypeRef trait, ReadOnlyList<CallableId> methods) {
public Implementation {
Objects.requireNonNull(declarationKey);
if (declarationKey.isBlank()) throw new IllegalArgumentException("empty implementation key");
Objects.requireNonNull(owner); Objects.requireNonNull(trait);
methods = copy(methods);
}
}
public enum Role {
NONE(0), BEFORE_PAUSE(1), BEFORE_RESUME(2), MEMCARD_BOOT(3);
private final int wire;
Role(int wire) { this.wire = wire; }
public int wire() { return wire; }
}
public record FunctionMetadata(Signature signature, ReadOnlyList<TypeRef> captures, Role role, String name) {
public FunctionMetadata(Signature signature, ReadOnlyList<TypeRef> captures, Role role) {
this(signature, captures, role, null);
}
public FunctionMetadata {
Objects.requireNonNull(signature); captures = copy(captures); Objects.requireNonNull(role);
}
public static FunctionMetadata ordinary(Signature signature) { return new FunctionMetadata(signature, null, Role.NONE); }
public static FunctionMetadata empty() { return ordinary(Signature.empty()); }
}
}
@@ -0,0 +1,30 @@
package p.studio.compiler.models.registry;
import p.studio.compiler.source.identifiers.CallableId;
import java.util.Objects;
/** Semantic Registry operands before image linking. No frontend syntax is exposed. */
public record IRRegistryInstruction(Kind kind, IRRegistry.TypeRef type, IRRegistry.TypeRef targetType,
CallableId function, int member) {
public enum Kind {
MAKE_CLOSURE(0x52), CALL_CLOSURE(0x53), GET_CAPTURE(0x57), CALL_TRAIT(0x58),
DYN_PACK(0x90), DYN_UNPACK(0x91), DYN_CAN_UNPACK(0x92),
REF_IS_EXACT(0x93), REF_IMPLEMENTS(0x94), REF_TO_TRAIT(0x95), REF_CAST(0x96),
STRUCT_NEW(0xa0), STRUCT_GET(0xa1), STRUCT_SET(0xa2),
ARRAY_NEW(0xa3), ARRAY_LEN(0xa4), ARRAY_GET(0xa5), ARRAY_SET(0xa6);
private final int opcode;
Kind(int opcode) { this.opcode=opcode; }
public int opcode(){return opcode;}
}
public IRRegistryInstruction {
Objects.requireNonNull(kind);
if(member<0)throw new IllegalArgumentException("negative member index");
if(kind!=Kind.GET_CAPTURE)Objects.requireNonNull(type,"type");
if(kind==Kind.MAKE_CLOSURE) {Objects.requireNonNull(function,"function");function.getIndex();}
else if(function!=null)throw new IllegalArgumentException("unexpected function operand");
boolean pair=kind.opcode()>=0x93&&kind.opcode()<=0x96;
if(pair)Objects.requireNonNull(targetType,"targetType");
else if(targetType!=null)throw new IllegalArgumentException("unexpected target type");
if(kind==Kind.GET_CAPTURE&&type!=null)throw new IllegalArgumentException("capture index is not a TypeId");
}
}
@@ -1,6 +1,8 @@
package p.studio.compiler.models;
import org.junit.jupiter.api.Test;
import p.studio.compiler.models.registry.IRRegistry;
import p.studio.compiler.models.registry.IRRegistryInstruction;
import p.studio.compiler.source.Span;
import p.studio.compiler.source.identifiers.CallableId;
import p.studio.compiler.source.identifiers.FileId;
@@ -25,6 +27,26 @@ import java.util.function.Function;
import static org.junit.jupiter.api.Assertions.*;
class IRBackendExecutableContractTest {
@Test
void registryAggregationRemapsImplementationCallablesWithoutChangingTypeIdentity() {
var owner = new IRRegistry.TypeRef("shared:owner");
var trait = new IRRegistry.TypeRef("shared:trait");
var types = ReadOnlyList.from(new IRRegistry.TypeDeclaration(owner, "Owner", new IRRegistry.Struct(null)),
new IRRegistry.TypeDeclaration(trait, "Trait", new IRRegistry.Trait(null)));
var aggregator = IRBackend.aggregator();
for (int file = 0; file < 2; file++) {
var registry = new IRRegistry(types, ReadOnlyList.from(new IRRegistry.Implementation(
"implementation:" + file, owner, trait, ReadOnlyList.from(new CallableId(0)))));
aggregator.merge(new IRBackendFile(new FileId(file), null, null, null, null, null, null,
ReadOnlyList.from(new CallableSignatureRef("method" + file, 0, "() -> unit")), null, null, registry));
}
var registry = aggregator.emit().getRegistry();
assertEquals(4, registry.types().size()); // Linking, not aggregation, canonicalizes declarations.
assertEquals(registry.types().get(0), registry.types().get(2));
assertEquals(new CallableId(0), registry.implementations().get(0).methods().getFirst());
assertEquals(new CallableId(1), registry.implementations().get(1).methods().getFirst());
assertEquals(owner, registry.implementations().get(1).owner());
}
private static final String FORBIDDEN_PBS_PACKAGE = "p.studio.compiler.pbs";
private static final Set<Class<?>> AUDITED_SUPPORT_TYPES = Set.of(
FileId.class,
@@ -41,6 +63,23 @@ class IRBackendExecutableContractTest {
Long.class,
Boolean.class);
private static final List<Class<?>> PUBLIC_CONTRACT_TYPES = List.of(
IRRegistry.class,
IRRegistryInstruction.class,
IRRegistryInstruction.Kind.class,
IRRegistry.TypeRef.class,
IRRegistry.TypeDeclaration.class,
IRRegistry.TypeBody.class,
IRRegistry.Builtin.class,
IRRegistry.Struct.class,
IRRegistry.Array.class,
IRRegistry.Trait.class,
IRRegistry.Fn.class,
IRRegistry.NamedType.class,
IRRegistry.Signature.class,
IRRegistry.Method.class,
IRRegistry.Implementation.class,
IRRegistry.Role.class,
IRRegistry.FunctionMetadata.class,
IRBackend.class,
IRBackend.IRBackendAggregator.class,
IRBackendReader.class,
@@ -7,8 +7,12 @@ public final class AutomationException extends IllegalStateException {
private final AutomationFailure failure;
public AutomationException(AutomationFailure.Code code, String message) {
super(message);
failure = new AutomationFailure(Objects.requireNonNull(code), message);
this(new AutomationFailure(Objects.requireNonNull(code), message));
}
public AutomationException(AutomationFailure failure) {
super(Objects.requireNonNull(failure).message());
this.failure = failure;
}
public AutomationFailure failure() { return failure; }
@@ -1,9 +1,14 @@
package p.studio.api.automation;
import java.util.Objects;
import java.util.Map;
import java.util.List;
import java.util.LinkedHashMap;
import java.util.Collections;
/** Stable domain failure codes, independent of HTTP status and JSON-RPC errors. */
public record AutomationFailure(Code code, String message) {
public record AutomationFailure(Code code, String message, Map<String, Object> runtimeError) {
public AutomationFailure(Code code, String message) { this(code, message, Map.of()); }
public enum Code {
PROJECT_NOT_OPEN, NOT_GRANTED, INVALID_TASK, BUSY, CLOSED, CLEANUP_FAILED,
UNSUPPORTED_LANGUAGE, UNSUPPORTED_CAPABILITY, REFERENCE_NOT_FOUND, DOCUMENTATION_UNAVAILABLE,
@@ -11,5 +16,19 @@ public record AutomationFailure(Code code, String message) {
RUNTIME_UNAVAILABLE, RUNTIME_INCOMPATIBLE, SESSION_NOT_FOUND, BUILD_NOT_FOUND,
NO_FRAME_AVAILABLE, LIMIT_EXCEEDED, NOT_CANCELABLE, INFRASTRUCTURE_FAILURE
}
public AutomationFailure { Objects.requireNonNull(code); Objects.requireNonNull(message); }
public AutomationFailure {
Objects.requireNonNull(code); Objects.requireNonNull(message);
var copy = new LinkedHashMap<String, Object>();
if (runtimeError != null) runtimeError.forEach((key, value) -> copy.put(key, freeze(value)));
runtimeError = Collections.unmodifiableMap(copy);
}
private static Object freeze(Object value) {
if (value instanceof Map<?, ?> map) {
var copy = new LinkedHashMap<Object, Object>();
map.forEach((key, item) -> copy.put(key, freeze(item)));
return Collections.unmodifiableMap(copy);
}
if (value instanceof List<?> list) return list.stream().map(AutomationFailure::freeze).toList();
return value;
}
}
+47
View File
@@ -0,0 +1,47 @@
# Studio PBX producer conformance
Owner: compiler/general. This harness uses the production runtime loader and VM;
it does not implement a Java approximation of runtime admission.
From the Studio root:
```sh
./gradlew :prometeu-compiler:prometeu-build-pipeline:test --tests '*LinkedRegistryTest' --tests '*PbxRegistryConformanceTest' --tests '*TypedCommonPipelineTest'
python3 tools/pbx-conformance/run.py --runtime ../runtime
```
The Java tests generate PBX in the build-pipeline module's
`build/registry-conformance`. `run.py` creates a temporary Cargo project, copies
the runtime lockfile and references its bytecode/hal/vm/system crates by path.
It never modifies the runtime checkout. Add `--online` for an uncached machine;
by default Cargo uses only cached dependencies. Java 25, Rust and the runtime
checkout's dependencies are required.
The output identifies runtime revision, OS/architecture and load configuration.
Positive fixtures distinguish loading from execution. The vec2 signature fixture
is deliberately load-only because its entry requires arguments. Native binding
uses SYSTEM for the positive case and NONE for missing-capability rejection;
all other cases use NONE. Resource tests explicitly lower default limits.
Negative fixtures mutate Java-produced bytes independently of the writer to
exercise malformed input that the writer correctly refuses to construct.
Initialization JSON comes from `prometeu_system::CrashReport`, and is checked
against the runtime's published headless schema. Saved JSON files remain under
the generated fixture directory. Studio process tests replay selected observed
payloads from `prometeu-app/src/test/resources/runtime` and assert unchanged MCP
projection, including null executed PC and nested verifier context.
## Known upstream schema defect
At runtime `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`, the LoadReason Rust enum
includes LifecycleRole, LifecycleSignature, DuplicateLifecycleRole and
NonFiniteValue, absent from the published JSON schema enum. This corpus
observes LifecycleRole and NonFiniteValue. The harness prints an explicit
`UPSTREAM SCHEMA GAP` for those two reasons and verifies that adding exactly
the missing enum value makes the original payload valid. It never alters the
payload or reports those cases as conforming to the unmodified schema. Any
other schema failure remains fatal. Track the runtime correction in AGD75;
remove the exception once the schema is corrected.
This corpus qualifies common backend production, not PBS lowering, lifecycle
selection, memcard persistence or installed-runtime deployment.
+357
View File
@@ -0,0 +1,357 @@
use prometeu_bytecode::Value;
use prometeu_bytecode::load_limits::{LoadBudget, LoadLimits};
use prometeu_hal::syscalls::caps;
use prometeu_vm::{
HostContext, HostReturn, NativeInterface, ValidatedProgramImage, VirtualMachine,
};
struct HasCart(usize);
impl NativeInterface for HasCart {
fn syscall(
&mut self,
id: u32,
args: &[Value],
ret: &mut HostReturn,
_: &mut HostContext,
) -> Result<(), prometeu_hal::vm_fault::VmFault> {
assert_eq!(id, 1);
assert!(args.is_empty());
self.0 += 1;
ret.push_bool(false);
Ok(())
}
}
struct NoNative;
impl NativeInterface for NoNative {
fn syscall(
&mut self,
_: u32,
_: &[Value],
_: &mut HostReturn,
_: &mut HostContext,
) -> Result<(), prometeu_hal::vm_fault::VmFault> {
panic!("fixture unexpectedly invoked native code")
}
}
fn main() {
let root = std::path::PathBuf::from(std::env::args().nth(1).expect("fixture directory"));
for name in [
"minimal",
"jump-end",
"vec2-signature",
"closure-capture",
"trait-call",
"array-dynamic",
"common-pipeline",
"roles",
"join",
"memcard-role",
"intrinsic-dot",
] {
let bytes = std::fs::read(root.join(format!("{name}.pbx"))).unwrap();
let image = ValidatedProgramImage::load(
&bytes,
&mut LoadBudget::new(LoadLimits::default()),
caps::NONE,
)
.unwrap_or_else(|e| panic!("{name}: {e:?}"));
println!("LOAD PASS {name}");
if name != "vec2-signature" {
let mut vm = VirtualMachine::from_image(image, Default::default());
vm.prepare_boot_call().unwrap();
let report = vm
.run_budget(1000, &mut NoNative, &mut HostContext::new(None))
.unwrap();
let reason = format!("{:?}", report.reason);
assert!(
reason == "Halted" || reason == "EndOfRom",
"{name}: {report:?}"
);
if name == "closure-capture" || name == "trait-call" {
assert!(matches!(vm.operand_stack_top(1)[0], Value::Int32(42)));
}
if name == "intrinsic-dot" {
assert!(
matches!(vm.operand_stack_top(1)[0], Value::Float(value) if value.to_bits() == 11f64.to_bits())
);
}
if name == "array-dynamic" {
assert!(matches!(vm.operand_stack_top(1)[0], Value::Int32(0)));
}
println!(
"EXEC PASS {name}: {reason}, {} steps",
report.steps_executed
);
}
}
for name in ["native", "unknown-native", "native-abi", "unused-native"] {
let bytes = std::fs::read(root.join(format!("{name}.pbx"))).unwrap();
if name == "native" {
let image = ValidatedProgramImage::load(
&bytes,
&mut LoadBudget::new(LoadLimits::default()),
caps::SYSTEM,
)
.unwrap();
let mut vm = VirtualMachine::from_image(image, Default::default());
vm.prepare_boot_call().unwrap();
let mut native = HasCart(0);
let report = vm
.run_budget(1000, &mut native, &mut HostContext::new(None))
.unwrap();
assert_eq!(native.0, 1);
assert_eq!(format!("{:?}", report.reason), "Halted");
println!("BIND + EXEC PASS native with SYSTEM");
}
let capabilities = if name == "native" {
caps::NONE
} else {
caps::SYSTEM
};
let error = ValidatedProgramImage::load(
&bytes,
&mut LoadBudget::new(LoadLimits::default()),
capabilities,
)
.unwrap_err();
record_fault(&root, name, error);
}
let baseline = std::fs::read(root.join("minimal.pbx")).unwrap();
for (name, offset, value) in [
("version", 4, 1),
("unsupported-section", 116, 9),
("unknown-builtin", 178, 0),
("invalid-role", 136, 4),
("retired-opcode", 162, 0x54),
] {
let mut bytes = baseline.clone();
bytes[offset] = value;
reject(&root, name, &bytes, LoadLimits::default());
}
for (name, code, max) in [
("raw-syscall", vec![0x70, 0, 1, 0, 0, 0, 1, 0], 0),
("hostcall-index", vec![0x71, 0, 0, 0, 0, 0, 1, 0], 0),
(
"retired-spawn",
vec![0x54, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0],
0,
),
("retired-yield", vec![0x55, 0, 1, 0], 0),
("retired-sleep", vec![0x56, 0, 0, 0, 0, 0, 1, 0], 0),
(
"unreachable-reference",
vec![1, 0, 0x50, 0, 255, 255, 255, 255],
0,
),
(
"nonfinite",
vec![1, 0, 0x15, 0, 0, 0, 0, 0, 0, 0, 0xf0, 0x7f],
0,
),
("jump-immediate", vec![2, 0, 3, 0, 0, 0, 1, 0], 0),
("zero-max", vec![0x17, 0, 0, 0, 0, 0, 1, 0], 0),
("return-height", vec![0x17, 0, 0, 0, 0, 0, 0x51, 0], 1),
("fallthrough", vec![0, 0], 0),
] {
let mut functions = section(&baseline, 1).to_vec();
functions[18..22].copy_from_slice(&(code.len() as u32).to_le_bytes());
functions[26..30].copy_from_slice(&(max as u32).to_le_bytes());
let bytes = replace(&replace(&baseline, 1, &functions), 2, &code);
reject(&root, name, &bytes, LoadLimits::default());
}
// Two mutually recursive Array declarations: a structural cycle, with valid IDs.
reject(
&root,
"array-cycle",
&replace(
&baseline,
6,
&[2, 0, 0, 0, 2, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0],
),
LoadLimits::default(),
);
reject(
&root,
"structural-cycle",
&replace(
&baseline,
6,
&[
2, 0, 0, 0, 4, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0,
],
),
LoadLimits::default(),
);
reject(
&root,
"invalid-type-reference",
&replace(&baseline, 6, &[1, 0, 0, 0, 2, 0, 99, 0, 0, 0]),
LoadLimits::default(),
);
reject(
&root,
"invalid-utf8",
&replace(
&baseline,
6,
&[1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 255, 0x13, 0, 0, 0],
),
LoadLimits::default(),
);
reject(
&root,
"removed-input",
&replace(&baseline, 6, &[1, 0, 0, 0, 0, 0, 0, 1, 0, 0]),
LoadLimits::default(),
);
let mut overlap = baseline.clone();
overlap[36..40].copy_from_slice(&0u32.to_le_bytes());
reject(&root, "overlap", &overlap, LoadLimits::default());
reject(
&root,
"truncated",
&baseline[..baseline.len() - 1],
LoadLimits::default(),
);
let mut duplicate = baseline.clone();
duplicate[44..48].copy_from_slice(&0u32.to_le_bytes());
reject(
&root,
"duplicate-section",
&duplicate,
LoadLimits::default(),
);
reject(
&root,
"empty-entry",
&replace(&replace(&baseline, 1, &[0, 0, 0, 0]), 8, &[0, 0, 0, 0]),
LoadLimits::default(),
);
reject(
&root,
"resource-types",
&baseline,
LoadLimits {
types: 0,
..Default::default()
},
);
reject(
&root,
"resource-scratch",
&baseline,
LoadLimits {
temporary_bytes: 0,
..Default::default()
},
);
reject(
&root,
"resource",
&baseline,
LoadLimits {
work: 0,
..Default::default()
},
);
}
fn reject(root: &std::path::Path, name: &str, bytes: &[u8], limits: LoadLimits) {
let error =
ValidatedProgramImage::load(bytes, &mut LoadBudget::new(limits), caps::NONE).unwrap_err();
record_fault(root, name, error);
}
fn record_fault(root: &std::path::Path, name: &str, error: prometeu_vm::VmInitError) {
if let prometeu_vm::VmInitError::RegistryLoadFailed(ref failure) = error {
let expected = match name {
"array-cycle" => Some("ArrayCycle"),
"structural-cycle" => Some("StructuralTypeCycle"),
"invalid-type-reference" => Some("TypeId"),
"invalid-utf8" => Some("Utf8"),
"removed-input" => Some("Builtin"),
"retired-spawn" | "retired-yield" | "retired-sleep" => Some("RemovedOpcode"),
"nonfinite" => Some("NonFiniteValue"),
"unreachable-reference" => Some("FunctionId"),
"return-height" => Some("ReturnHeight"),
"fallthrough" => Some("UnterminatedPath"),
"zero-max" => Some("StackOverflow"),
"duplicate-section" => Some("DuplicateSection"),
_ => None,
};
if let Some(reason) = expected {
assert_eq!(format!("{:?}", failure.reason), reason, "{name}");
}
}
let report = prometeu_system::CrashReport::VmInit { error };
assert_eq!(report.pc(), None);
let fault = serde_json::json!({"kind":"vm_init", "pc":null, "summary":"Conformance fixture", "initError":report.init_diagnostic().unwrap()});
let full: serde_json::Value = serde_json::from_slice(
&std::fs::read(std::env::args().nth(2).expect("schema path")).unwrap(),
)
.unwrap();
let schema = jsonschema::validator_for(
&serde_json::json!({"$defs":full["$defs"],"$ref":"#/$defs/fault"}),
)
.unwrap();
if !schema.is_valid(&fault) {
let reason = fault["initError"]["load"]["reason"].as_str().unwrap();
assert!(
["LifecycleRole", "NonFiniteValue"].contains(&reason),
"unexpected schema failure: {fault}"
);
// Validate the ORIGINAL payload with only the observed missing enum value added.
// This is an explicit upstream defect report, not a claim of published-schema conformance.
let mut corrected = full.clone();
corrected["$defs"]["loadFailure"]["properties"]["reason"]["enum"]
.as_array_mut()
.unwrap()
.push(serde_json::json!(reason));
jsonschema::validator_for(
&serde_json::json!({"$defs":corrected["$defs"],"$ref":"#/$defs/fault"}),
)
.unwrap()
.validate(&fault)
.unwrap();
println!(
"UPSTREAM SCHEMA GAP: {reason} missing from published enum; original payload preserved"
);
}
std::fs::write(
root.join(format!("fault-{name}.json")),
serde_json::to_vec_pretty(&fault).unwrap(),
)
.unwrap();
println!("REJECT PASS {name}: {}", fault["initError"]);
}
fn word(bytes: &[u8], at: usize) -> usize {
u32::from_le_bytes(bytes[at..at + 4].try_into().unwrap()) as usize
}
fn section(bytes: &[u8], kind: usize) -> &[u8] {
for i in 0..word(bytes, 8) {
let at = 32 + 12 * i;
if word(bytes, at) == kind {
let offset = word(bytes, at + 4);
return &bytes[offset..offset + word(bytes, at + 8)];
}
}
panic!("missing section {kind}")
}
fn replace(bytes: &[u8], kind: usize, payload: &[u8]) -> Vec<u8> {
let count = word(bytes, 8);
let mut output = bytes[..32 + 12 * count].to_vec();
for i in 0..count {
let at = 32 + 12 * i;
let body = if word(bytes, at) == kind {
payload
} else {
section(bytes, word(bytes, at))
};
let offset = output.len() as u32;
output[at + 4..at + 8].copy_from_slice(&offset.to_le_bytes());
output[at + 8..at + 12].copy_from_slice(&(body.len() as u32).to_le_bytes());
output.extend_from_slice(body);
}
output
}
+27
View File
@@ -0,0 +1,27 @@
"""Build a production-loader harness without writing into the runtime checkout."""
import argparse
import json
import pathlib
import platform
import subprocess
import tempfile
parser = argparse.ArgumentParser()
parser.add_argument("--runtime", type=pathlib.Path, default=pathlib.Path("../runtime"))
parser.add_argument("--fixtures", type=pathlib.Path, default=pathlib.Path("prometeu-compiler/prometeu-build-pipeline/build/registry-conformance"))
parser.add_argument("--online", action="store_true", help="Allow Cargo to fetch uncached dependencies")
args = parser.parse_args()
runtime = args.runtime.resolve()
print("runtime", subprocess.check_output(["git", "-C", str(runtime), "rev-parse", "HEAD"], text=True).strip(), flush=True)
print("platform", platform.platform(), "limits=default capabilities=NONE", flush=True)
with tempfile.TemporaryDirectory(prefix="pbx-conformance-") as directory:
root = pathlib.Path(directory)
manifest = '[package]\nname="studio-pbx-conformance"\nversion="0.1.0"\nedition="2024"\n[dependencies]\n'
for crate in ["prometeu-bytecode", "prometeu-hal", "prometeu-vm", "prometeu-system"]:
manifest += f'{crate}={{path={json.dumps(str(runtime / "crates/console" / crate))}}}\n'
manifest += 'serde_json="1.0.149"\njsonschema={version="0.33",default-features=false}\n'
(root / "Cargo.toml").write_text(manifest)
(root / "Cargo.lock").write_text((runtime / "Cargo.lock").read_text())
(root / "src").mkdir()
(root / "src/main.rs").write_text(pathlib.Path(__file__).with_name("main.rs").read_text())
subprocess.run(["cargo", "run", *([] if args.online else ["--offline"]), "--manifest-path", str(root / "Cargo.toml"), "--", str(args.fixtures.resolve()), str(runtime / "devtools/headless-protocol/response.schema.json")], check=True)