feat(compiler): implement typed registry and production PBX conformance (AGD76)
This commit is contained in:
+145
@@ -0,0 +1,145 @@
|
||||
---
|
||||
id: LSN-0079
|
||||
ticket: compiler-pbx-registry-conformance
|
||||
title: Typed common IR through production PBX admission
|
||||
created: 2026-10-14
|
||||
tags: [runtime-integration]
|
||||
---
|
||||
|
||||
## Original problem and ownership
|
||||
|
||||
Owner: compiler/general; interfaces: compiler/pbs and studio.
|
||||
|
||||
The old producer carried parameter/result slot counts without complete semantic
|
||||
types and emitted a retired PBX function layout. A writer-only repair could not
|
||||
recover signatures, distinguish nominal identities or describe closures and
|
||||
trait implementations. The fix had to cross the entire common producer chain.
|
||||
|
||||
This lesson consolidates AGD-0076, accepted DEC-0063 and completed PLN-0142
|
||||
(DSC-0074). Those workflow artifacts were removed by housekeeping after their
|
||||
normative content and evidence were propagated. The active integration umbrella
|
||||
is [AGD75](../../workflow/agendas/AGD-0075-studio-runtime-readiness.md).
|
||||
|
||||
## Consolidated decisions
|
||||
|
||||
Use stable declaration references before assigning deterministic dense image
|
||||
IDs. Preserve Struct/Trait nominal identity; canonicalize Builtin imports,
|
||||
Array elements and Fn signatures. Names are descriptive, not nominal keys.
|
||||
Parameters, results, captures, implementations and lifecycle roles are explicit
|
||||
common values, preserved through IRVM, optimization and emission.
|
||||
|
||||
The common IR remains serializable by design, but a concrete codec was
|
||||
explicitly deferred. Complete representation does not require selecting a
|
||||
transport format. References avoid recursive Java object graphs and the public
|
||||
contract audit rejects mutable or frontend-specific shapes.
|
||||
|
||||
PBS is one frontend. Optional and Result already belong to PBS's normative
|
||||
language contract and have parser/semantic implementations. Defects in their
|
||||
executable lowering do not justify new backend categories or rejection rules.
|
||||
Their implementation and carrier choices remain frontend-owned. The user
|
||||
explicitly scoped consumer migration to subsequent tickets and a single AGD75
|
||||
integrated delivery. No PBS semantics were added to this backend change.
|
||||
|
||||
## Final implementation
|
||||
|
||||
- IRRegistry and FunctionMetadata represent all five type categories, ordered
|
||||
signatures/captures, roles and implementations. Aggregation remaps callable
|
||||
references while preserving stable type identity.
|
||||
- LinkedRegistry canonicalizes imports/structural declarations deterministically
|
||||
with an iterative dependency traversal. Nominal anchors permit recursion;
|
||||
purely structural cycles reject.
|
||||
- Lowering derives widths from semantic signatures, links typed operands and
|
||||
remaps implementation targets. IRVM coherence and optimization preserve the
|
||||
Registry and function metadata.
|
||||
- The PBX writer emits the 32-byte header and all eight mandatory sections,
|
||||
optional debug, strict UTF-8, exact function records and explicit empty capture
|
||||
environments. Width caches are not serialized. Constant deduplication now
|
||||
relocates PUSH_CONST indices rather than silently changing their meanings.
|
||||
- Local verification decodes all function ranges, validates references and
|
||||
operand categories, forbids retired/raw opcodes and analyzes reachable stack
|
||||
and type facts to convergence. Conflicts become Unknown; exact semantic groups
|
||||
retain provenance when available. Unknown remains subject to runtime checks.
|
||||
- Production loading remains authoritative for native identity/capability/ABI,
|
||||
runtime budgets and executable admission.
|
||||
- Studio previously discarded error.details when a headless response rejected
|
||||
boot. AutomationFailure now preserves the entire runtime error envelope,
|
||||
including initialization, binding and resource details, through MCP output.
|
||||
|
||||
## Examples and pitfalls
|
||||
|
||||
A `vec2 -> vec2` function has logical arity one/one and derived slot widths two/two.
|
||||
The function record writes the two TypeId lists, not the numbers two/two. A test
|
||||
compares that record with independent fixed bytes from the runtime contract.
|
||||
|
||||
A capturing body is not a direct-call target. MAKE_CLOSURE names the body and
|
||||
expected Fn type, consumes its explicit capture groups and produces a callable.
|
||||
CALL_CLOSURE consumes that callable after the explicit arguments. The synthetic
|
||||
frontend test proves this path without PBS source, SDK or language ASTs.
|
||||
|
||||
Jumps contain absolute offsets from the current function start. They are not
|
||||
relative to the next PC. A taken jump to function end is a return boundary;
|
||||
ordinary fallthrough is invalid. These details were checked against the actual
|
||||
runtime verifier, not inferred from older emitter behavior.
|
||||
|
||||
Do not derive type identity from slot width, spelling or display names. Do not
|
||||
claim a self-round-trip proves external conformance. Do not treat parser tests
|
||||
as execution evidence or a load-only argument-taking entry as a boot test.
|
||||
|
||||
Initialization diagnostics have null outer executed PC. A verifier PC remains
|
||||
nested in initError.load.pc; file offsets remain offsets. Preserve the original
|
||||
JSON structure rather than flattening it into human-readable messages.
|
||||
|
||||
## Qualification evidence
|
||||
|
||||
Runtime: `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`.
|
||||
Platform: macOS 27.0.1 arm64. Rust harness uses the runtime Cargo.lock, default
|
||||
load limits and NONE capabilities except the positive native case (SYSTEM).
|
||||
Resource-negative cases explicitly lower Types, TemporaryBytes or Work.
|
||||
Runtime checkout remained unchanged.
|
||||
|
||||
| Gate | Result |
|
||||
| --- | --- |
|
||||
| Build-pipeline suite | 178 tests: 163 passed, 15 skipped, no failures |
|
||||
| Frontend-api suite | 28 passed |
|
||||
| PBS suite | 245 passed; not a claim of complete executable migration |
|
||||
| Headless process suite | 9 passed, including six observed runtime fault payloads |
|
||||
| Architecture suite | 17 passed |
|
||||
| Independent binary oracle | Exact 194-byte minimal image and fixed vec2 function bytes |
|
||||
| Production loader positives | 12 admitted images, including native binding |
|
||||
| Genuine execution | 11 programs: direct/common path, closures/captures, trait, Array/Dynamic, joins, roles, intrinsic and native; vec2 signature is load-only |
|
||||
| Production negative corpus | 32 rejected malformed/binding/code/resource fixtures |
|
||||
| Workflow and patch integrity | discussion validate and git diff --check passed |
|
||||
|
||||
The 15 skipped tests are older conditional integration adapters. The dedicated
|
||||
production harness above ran explicitly; their skip state is not substituted
|
||||
for runtime evidence. The intrinsic/native catalogue goldens are serialization
|
||||
fixtures and do not claim current PBS catalogue compatibility.
|
||||
|
||||
Reproduce with the commands in
|
||||
[the harness README](../../../tools/pbx-conformance/README.md). The harness
|
||||
creates its Cargo build outside the runtime checkout and saves generated PBX and
|
||||
observed JSON under the build-pipeline build directory.
|
||||
|
||||
## Remaining integration work
|
||||
|
||||
The runtime LoadReason enum has four values missing from the published headless
|
||||
schema: LifecycleRole, LifecycleSignature, DuplicateLifecycleRole and
|
||||
NonFiniteValue. The corpus observes the first and last. Both payloads fail the
|
||||
original schema solely because of the missing enum value. The harness explicitly
|
||||
reports this upstream gap, validates the original payload with that single enum
|
||||
addition, and leaves the payload unchanged. It does not claim those two cases
|
||||
conform to the unmodified schema. Runtime must correct the schema under AGD75.
|
||||
|
||||
PBS typed producer migration, Optional/Result executable lowering, catalogue
|
||||
alignment, installed runtime delivery, hook selection, memcard gating and
|
||||
persistence remain with their consumer tickets. The common schema and synthetic
|
||||
producer work without those features. There is no legacy PBX fallback and no
|
||||
slot-to-type inference to conceal an incomplete handoff.
|
||||
|
||||
## References
|
||||
|
||||
- [Typed common Registry/PBX specification](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md)
|
||||
- [Backend conformance matrix](../../../docs/specs/compiler/22.%20Backend%20Spec-to-Test%20Conformance%20Matrix.md)
|
||||
- [Runtime qualification harness](../../../tools/pbx-conformance/README.md)
|
||||
- Runtime sources: registry_v0.rs, program_image.rs, registry_verifier, and
|
||||
devtools/headless-protocol/response.schema.json at the revision above.
|
||||
@@ -0,0 +1,182 @@
|
||||
# Typed common Registry and PBX v0
|
||||
|
||||
Status: normative. Owner: `compiler/general`. Runtime contract reference:
|
||||
`runtime` revision `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`,
|
||||
`docs/specs/runtime/02c-program-registry-and-pbx-v0.md` and
|
||||
`docs/vm-arch/ISA_CORE.md` in that repository.
|
||||
|
||||
## Ownership and handoff
|
||||
|
||||
Every executable frontend MUST provide its semantic types through the common
|
||||
`IRRegistry` model and typed `FunctionMetadata`. The backend MUST NOT inspect
|
||||
language ASTs, PBS names, Optional/Result syntax, or frontend-specific classes.
|
||||
Language carriers and their lowering remain owned by the frontend. A carrier
|
||||
can use ordinary Struct, Array, Trait, Fn and Builtin declarations without
|
||||
introducing a language category in the backend.
|
||||
|
||||
The public model consists of immutable values and ordered `ReadOnlyList`
|
||||
collections. References use explicit declaration keys and callable IDs, not
|
||||
object identity or recursive object graphs. It is serializable by design;
|
||||
no concrete IR codec, wire format or persistence protocol is selected.
|
||||
|
||||
A function declares ordered named parameters, ordered logical results, ordered
|
||||
captures, a lifecycle role and an optional name. Empty names differ from absent
|
||||
names. Captures are an environment, never hidden parameters. Slot counts are
|
||||
derived from types; existing count fields are checked caches, not semantic
|
||||
input. Missing nonempty type metadata is an invalid handoff, not a request to
|
||||
infer types from counts. Optimizers MUST preserve metadata and typed operands.
|
||||
|
||||
## Identity and linking
|
||||
|
||||
TypeId, FunctionId and ImplementationId are separate dense image-local index
|
||||
domains. Index zero is valid. Trait references use TypeIds; field and method
|
||||
indices are local to their owner. Source declaration keys MUST identify the
|
||||
same declaration across repeated module imports and distinguish unrelated
|
||||
nominal declarations, even when names and shapes coincide.
|
||||
|
||||
Linking is deterministic. The selected physical entry is FunctionId zero;
|
||||
remaining functions follow canonical module/callable/source ordering. Struct
|
||||
and Trait retain nominal identity. Builtin imports deduplicate by BuiltinId,
|
||||
Array by canonical element identity, and Fn by ordered parameter/result type
|
||||
identities, excluding names. Forward references and recursion through nominal
|
||||
anchors are valid. Pure Array/Fn cycles are invalid. Every reference must
|
||||
resolve before emission. Frontends should supply only used imports.
|
||||
|
||||
| BuiltinId | Type | Slots |
|
||||
| --- | --- | --- |
|
||||
| `0x0001` | color | 1 |
|
||||
| `0x0002` | vec2 | 2 |
|
||||
| `0x0003` | pixel | 3 |
|
||||
| `0x0010` | Int32 | 1 |
|
||||
| `0x0011` | Int64 | 1 |
|
||||
| `0x0012` | Float | 1 |
|
||||
| `0x0013` | Bool | 1 |
|
||||
| `0x0014` | String | 1 |
|
||||
| `0x0015` | Dynamic | 2 |
|
||||
| `0x0016` | Random | 1 |
|
||||
|
||||
The retired input IDs `0x0100`–`0x0102` have no aliases. Struct, Array and Fn
|
||||
values occupy one reference slot; Trait values occupy two slots. Width does
|
||||
not establish type identity or authorize conversion.
|
||||
|
||||
## Binary contract
|
||||
|
||||
All fields are little-endian. The header is 32 bytes: `PBX\0`, u16 version zero,
|
||||
zero endian/reserved bytes, u32 section count and twenty zero reserved bytes.
|
||||
Directory entries are u32 kind, file-relative offset and length. Ranges must
|
||||
not overlap or overflow. The version remains zero; no legacy fallback exists.
|
||||
|
||||
Mandatory sections are constants (0), functions (1), Code (2), exports (4),
|
||||
syscalls (5), types (6), implementations (7) and captures (8). Debug (3) is
|
||||
optional. Empty tables contain a u32 zero count. Code is raw bytes, not a
|
||||
counted table. An empty table set is serializable but is not an executable
|
||||
program: production loading requires a valid ordinary entry.
|
||||
|
||||
Lists have u32 counts. Optional names use u8 presence 0/1, then u32 UTF-8 byte
|
||||
length and bytes when present. UTF-8 is strict. Named types contain TypeId then
|
||||
optional name; results and captures contain TypeIds without names.
|
||||
|
||||
| Record | Ordered payload |
|
||||
| --- | --- |
|
||||
| Type | u8 tag, optional name, category payload |
|
||||
| Builtin (tag 0) | u32 BuiltinId |
|
||||
| Struct (tag 1) | named fields |
|
||||
| Array (tag 2) | u32 element TypeId |
|
||||
| Trait (tag 3) | methods: optional name, parameters, results |
|
||||
| Fn (tag 4) | parameters, results |
|
||||
| Function | u8 role, optional name, parameters, results, u32 Code offset, length, extra locals, max temporaries |
|
||||
| Implementation | u32 owner TypeId, trait TypeId, method count, ordered FunctionIds |
|
||||
| Captures | function count and one ordered TypeId list per function, including empty lists |
|
||||
|
||||
Function Code offsets are Code-relative. Parameter/result slot counts MUST NOT
|
||||
be serialized. Max temporaries zero means zero. Signature names are independent
|
||||
of debug names. Constants retain tags Null 0, Int64 1, Float64 2, Boolean 3,
|
||||
String 4 and Int32 5. Float payloads must be finite. Constant deduplication
|
||||
requires relocating every PUSH_CONST index; changing only the table is invalid.
|
||||
|
||||
Roles are None 0, BeforePause 1, BeforeResume 2 and MemcardBoot 3. Nonzero roles
|
||||
are unique, cannot target entry zero and cannot capture. Pause/resume have empty
|
||||
signatures; MemcardBoot has `(Int64) -> Int32`. This metadata does not implement
|
||||
frontend hook selection, mem.bootstrap gating or persistence.
|
||||
|
||||
Implementations are unique per Struct/Trait pair, complete and ordered. Each
|
||||
method target has an exact signature with a concrete Struct receiver first,
|
||||
followed by the trait method's explicit parameters. Targets cannot capture.
|
||||
|
||||
## Instructions and verification
|
||||
|
||||
Instructions use u16 opcodes. Jump operands are absolute u32 byte offsets from
|
||||
the current function start, not displacements from the next instruction.
|
||||
A taken jump to the current function end is a return boundary; ordinary
|
||||
fallthrough past the end is invalid.
|
||||
|
||||
| Opcode | Typed operands |
|
||||
| --- | --- |
|
||||
| MAKE_CLOSURE `0x52` | FunctionId, Fn TypeId |
|
||||
| CALL_CLOSURE `0x53` | expected Fn TypeId |
|
||||
| GET_CAPTURE `0x57` | capture group index |
|
||||
| CALL_TRAIT `0x58` | Trait TypeId, method index |
|
||||
| DYN_PACK/UNPACK/CAN_UNPACK `0x90`–`0x92` | TypeId |
|
||||
| Reference operations `0x93`–`0x96` | source TypeId, target TypeId |
|
||||
| STRUCT_NEW `0xa0` | Struct TypeId |
|
||||
| STRUCT_GET/SET `0xa1`–`0xa2` | Struct TypeId, field index |
|
||||
| Array operations `0xa3`–`0xa6` | Array TypeId |
|
||||
|
||||
Direct CALL cannot target a capturing body. Closure signatures match exactly.
|
||||
Retired SPAWN/YIELD/SLEEP (`0x54`–`0x56`) and raw producer SYSCALL are invalid,
|
||||
even when unreachable. All function ranges must decode completely; references,
|
||||
categories, operand boundaries, finite immediates and branch destinations are
|
||||
checked even in unreachable instructions.
|
||||
|
||||
Reachable analysis uses a bounded worklist. Parameters initialize local facts;
|
||||
extra locals start Null and may change type. Joins require equal temporary
|
||||
heights. Equal facts survive; conflicting facts widen to Unknown. Type errors
|
||||
are diagnosed after convergence. Unknown alone neither proves compatibility
|
||||
nor justifies rejection. Runtime group checks remain authoritative. RET and
|
||||
jump-to-end require exact result width and compatible known facts. GET/SET_LOCAL
|
||||
address only parameters plus extra locals. No implicit numeric, trait or Dynamic
|
||||
conversion is introduced by linking, writing or verification.
|
||||
|
||||
HOSTCALL references a declared canonical `(module, name, version)` identity
|
||||
with exact argument/result slot ABI. Duplicate identities with conflicting ABI,
|
||||
unused declarations and invalid indices are rejected. Only the runtime loader
|
||||
binds capabilities and rewrites HOSTCALL to SYSCALL. Intrinsic signatures are
|
||||
explicit compiler metadata; their IDs and definitive ABI remain runtime-owned.
|
||||
|
||||
## Bounds and diagnostics
|
||||
|
||||
Product ceilings: PBX 64 MiB, Code 32 MiB, 65,536 types and functions each,
|
||||
262,144 implementations, 1,048,576 nested records, names 4,096 bytes individually
|
||||
and 4 MiB aggregate, frame 16,384 slots and results 1,024 slots. Runtime admission
|
||||
also owns retained Registry 64 MiB, scratch 128 MiB and work 268,435,456. A guest
|
||||
cannot raise these limits. Java validation is not a substitute for runtime
|
||||
allocation accounting or final `ValidatedProgramImage::load` admission.
|
||||
|
||||
Compilation failures use common LOWER_IRVM/MARSHAL stage diagnostics. Load,
|
||||
binding and resource failures remain distinct from executed traps. Studio
|
||||
preserves the full runtime error envelope in `AutomationFailure.runtimeError`,
|
||||
including null fields and native binding details. The outer initialization
|
||||
fault PC remains null. A verifier PC stays in `initError.load.pc`; a file offset
|
||||
MUST NOT become an executed PC.
|
||||
|
||||
## Reproducible conformance
|
||||
|
||||
Run the focused common tests, then the production harness:
|
||||
|
||||
```sh
|
||||
./gradlew :prometeu-compiler:prometeu-build-pipeline:test --tests '*LinkedRegistryTest' --tests '*PbxRegistryConformanceTest' --tests '*TypedCommonPipelineTest'
|
||||
python3 tools/pbx-conformance/run.py --runtime ../runtime
|
||||
```
|
||||
|
||||
The Java tests write `build/registry-conformance` inside the build-pipeline
|
||||
module. The harness builds outside the runtime checkout using its Cargo.lock,
|
||||
loads Java-produced bytes, executes admitted zero-argument fixtures, validates
|
||||
structured failures against the published schema and records the runtime hash,
|
||||
platform, capabilities and limits. Parsing, loading and execution are separate
|
||||
claims. The vec2 signature fixture is load-only; it is not a zero-argument boot.
|
||||
|
||||
The fixed 194-byte minimal image comes from the independent runtime test oracle.
|
||||
Synthetic common IR additionally exercises linking, lowering, optimization and
|
||||
emission without PBS. Frontend migration and integrated release readiness are
|
||||
tracked separately in AGD75; passing common conformance does not claim complete
|
||||
PBS lowering or complete Studio/runtime integration.
|
||||
@@ -1,5 +1,10 @@
|
||||
# Bytecode and PBX Mapping Specification
|
||||
|
||||
The current typed Registry/PBX v0 contract is normative in
|
||||
[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md).
|
||||
Its complete typed function records, mandatory sections and verification rules
|
||||
replace the earlier slot-only artifact baseline.
|
||||
|
||||
Status: Draft v1 (Backend Baseline)
|
||||
Applies to: mapping from lowered/optimized backend programs into PBX sections, bytecode-facing artifacts, and source-to-artifact invariants required by loader/verifier/runtime
|
||||
|
||||
@@ -74,7 +79,8 @@ Emitter output must map to a `BytecodeModule` shape containing:
|
||||
3. `code`,
|
||||
4. `exports`,
|
||||
5. `syscalls`,
|
||||
6. optional `debug_info` that still satisfies v1 minimum debug obligations.
|
||||
6. typed Registry declarations, implementations and per-function capture lists,
|
||||
7. optional `debug_info` that still satisfies v1 minimum debug obligations.
|
||||
|
||||
### 6.2 Function ordering and IDs
|
||||
|
||||
@@ -168,7 +174,6 @@ At minimum, artifact conformance checks should assert:
|
||||
The following remain deferred:
|
||||
|
||||
- richer optional debug/source-map formats,
|
||||
- additional PBX section-level contracts beyond current baseline,
|
||||
- and profile-specific binary compatibility policy details beyond current v1 baseline.
|
||||
|
||||
## 11. Non-Goals
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Compiler Verification and Safety Checks Specification
|
||||
|
||||
The current typed Registry/PBX v0 contract is normative in
|
||||
[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md).
|
||||
Its complete typed function records, mandatory sections and verification rules
|
||||
replace the earlier slot-only artifact baseline.
|
||||
|
||||
Status: Draft v1 (Quality-Gate Baseline)
|
||||
Applies to: test-backed safety gates for common compiler stages, validation ownership across executable frontends, and integration regression safety checks against runtime
|
||||
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# IRBackend to IRVM Lowering Specification
|
||||
|
||||
The current typed Registry/PBX v0 contract is normative in
|
||||
[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md).
|
||||
Its complete typed function records, mandatory sections and verification rules
|
||||
replace the earlier slot-only artifact baseline.
|
||||
|
||||
Status: Draft v1 (Backend Baseline)
|
||||
Applies to: executable-backend lowering from the common `IRBackend` handoff into `IRVM`
|
||||
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Backend Spec-to-Test Conformance Matrix
|
||||
|
||||
The current typed Registry/PBX v0 contract is normative in
|
||||
[Typed common Registry and PBX v0](../../compiler/general/specs/typed-registry-and-pbx-v0.md).
|
||||
Its complete typed function records, mandatory sections and verification rules
|
||||
replace the earlier slot-only artifact baseline.
|
||||
|
||||
Status: Draft v1 (Traceability Baseline)
|
||||
Applies to: compiler/backend conformance traceability for canonical stage order and entrypoint-specific contracts
|
||||
|
||||
@@ -140,3 +145,19 @@ Matrix integrity is enforced by automated document lint:
|
||||
- if `PROMETEU_MATRIX_HARD_GATE=true` and `PROMETEU_CHANGED_FILES` contains a path under conformance-relevant FE/BE prefixes,
|
||||
- then this matrix file MUST be present in `PROMETEU_CHANGED_FILES`, otherwise the gate fails.
|
||||
- CI policy uses strict Gate I mode via `PROMETEU_GATE_I_STRICT=true` (or `CI=true`) and requires `PROMETEU_RUNTIME_CHECK_CMD` for runtime-backed evidence.
|
||||
|
||||
## 7. Typed Registry/PBX v0 qualification (2026-10-14)
|
||||
|
||||
Owner: compiler/general. These rows supersede slot-only PBX coverage claims;
|
||||
frontend syntax and lowering remain consumer-owned.
|
||||
|
||||
| Requirement | Evidence | Status | Scope |
|
||||
| --- | --- | --- | --- |
|
||||
| Stable declarations, nominal identity, structural deduplication and cycles | LinkedRegistryTest; IRBackendExecutableContractTest | pass | Common API and linking |
|
||||
| Complete typed signatures, captures, roles and implementations | PbxRegistryConformanceTest; TypedCommonPipelineTest | pass | Common IR through emission |
|
||||
| Exact PBX mandatory tables and multi-slot signatures | Independent 194-byte oracle; fixed vec2 function bytes | pass | Wire format, not a self round-trip |
|
||||
| Runtime final admission and execution | tools/pbx-conformance/run.py | pass | Closure, trait, array, Dynamic, intrinsic, roles, joins, native binding; vec2 load-only |
|
||||
| Invalid structures, references, code and resource refusal | tools/pbx-conformance/main.rs mutation corpus | pass | Production loader, default/lowered limits |
|
||||
| Structured initialization and binding diagnostics | HeadlessProcessSessionTest; observed runtime JSON fixtures | pass | Preserves full payload through MCP, including null outer PC |
|
||||
| Published schema covers current runtime reasons | Actual LifecycleRole/NonFiniteValue payloads against runtime schema | partial | Runtime enum/schema mismatch recorded in AGD75; no payload rewriting |
|
||||
| PBS migration and integrated delivery | AGD75 and consumer tickets | partial | Existing PBS suite passing is not complete executable lowering evidence |
|
||||
|
||||
@@ -245,7 +245,9 @@ final class HeadlessProcessSession implements HeadlessGateway.Session {
|
||||
default -> AutomationFailure.Code.INFRASTRUCTURE_FAILURE;
|
||||
};
|
||||
pending.remove(item.id);
|
||||
item.result.completeExceptionally(new AutomationException(code, "Runtime " + error.path("code").asText() + ": " + error.path("message").asText()));
|
||||
Map<String, Object> runtimeError = mapper.convertValue(error, new com.fasterxml.jackson.core.type.TypeReference<>() {});
|
||||
item.result.completeExceptionally(new AutomationException(new AutomationFailure(code,
|
||||
"Runtime " + error.path("code").asText() + ": " + error.path("message").asText(), runtimeError)));
|
||||
} else {
|
||||
var result = message.get("result");
|
||||
if (item.command.equals("cancel") && (!result.path("requested").isBoolean() || !result.path("requested").booleanValue()
|
||||
|
||||
@@ -50,7 +50,15 @@ public final class HeadlessFixture {
|
||||
var request = MAPPER.readTree(line); String id = request.path("requestId").asText(), command = request.path("command").asText();
|
||||
switch (command) {
|
||||
case "initialize" -> response(id, MAPPER.createObjectNode().setAll(Map.of("protocolVersion", hello.path("protocolVersion"), "capabilities", hello.path("capabilities"))));
|
||||
case "boot", "reset" -> response(id, state());
|
||||
case "boot", "reset" -> {
|
||||
if (mode.startsWith("load-fault-")) {
|
||||
var resource = "/runtime/fault-" + mode.substring("load-fault-".length()) + ".json";
|
||||
var fault = MAPPER.readTree(HeadlessFixture.class.getResourceAsStream(resource));
|
||||
var error = MAPPER.createObjectNode().put("code", "boot_failed").put("message", "Runtime rejected cartridge");
|
||||
error.putObject("details").set("fault", fault);
|
||||
emit(MAPPER.createObjectNode().put("type", "response").put("sessionId", "fixture").put("requestId", id).set("error", error));
|
||||
} else response(id, state());
|
||||
}
|
||||
case "get_state" -> { if (!mode.equals("hang-call")) response(id, state()); }
|
||||
case "get_fault" -> response(id, MAPPER.nullNode());
|
||||
case "advance" -> {
|
||||
|
||||
@@ -18,6 +18,20 @@ class HeadlessProcessSessionTest {
|
||||
var session = new HeadlessProcessSession(process, new ObjectMapper(), HeadlessFixture.limits(mode), "build"); sessions.add(session); return session;
|
||||
}
|
||||
@AfterEach void cleanup() { sessions.forEach(session -> session.abort("test cleanup")); }
|
||||
@Test void productionLoadDiagnosticsSurviveBootFailureAndMcpSerialization() throws Exception {
|
||||
var mapper = new ObjectMapper();
|
||||
for (String fixture : List.of("resource", "invalid-role", "retired-opcode", "native", "native-abi", "unknown-native")) {
|
||||
var session = start("load-fault-" + fixture);
|
||||
var failure = assertThrows(AutomationException.class, () -> session.initializeAndBoot(root)).failure();
|
||||
var fault = mapper.readTree(getClass().getResourceAsStream("/runtime/fault-" + fixture + ".json"));
|
||||
var error = mapper.valueToTree(failure.runtimeError());
|
||||
assertEquals(fault, error.path("details").path("fault"));
|
||||
assertTrue(error.path("details").path("fault").path("pc").isNull());
|
||||
var wire = p.studio.app.mcp.McpToolResult.failure(mapper, failure).wire();
|
||||
assertEquals(fault, wire.path("structuredContent").path("error").path("runtimeError").path("details").path("fault"));
|
||||
assertThrows(UnsupportedOperationException.class, () -> failure.runtimeError().put("code", "changed"));
|
||||
}
|
||||
}
|
||||
@Test void fragmentedProtocolPreservesUnsignedFrameIdentityLayoutAndSeparateStderr() throws Exception {
|
||||
var session = start("normal"); session.initializeAndBoot(root);
|
||||
var outcome = session.advance(3, ignored -> {});
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"initError": {
|
||||
"code": "registry_load_failed",
|
||||
"load": {
|
||||
"code": "LOAD_INVALID_REGISTRY",
|
||||
"function": 0,
|
||||
"id_domain": null,
|
||||
"limit": null,
|
||||
"member": null,
|
||||
"offset": 136,
|
||||
"pc": null,
|
||||
"reason": "LifecycleRole",
|
||||
"record": 0,
|
||||
"referenced_id": null,
|
||||
"requested": null,
|
||||
"resource": null,
|
||||
"section": 1
|
||||
},
|
||||
"message": "RegistryLoadFailed(LoadFailure { code: InvalidRegistry, reason: LifecycleRole, section: Some(1), offset: Some(136), record: Some(0), member: None, id_domain: None, referenced_id: None, function: Some(0), pc: None, resource: None, requested: None, limit: None })"
|
||||
},
|
||||
"kind": "vm_init",
|
||||
"pc": null,
|
||||
"summary": "Conformance fixture"
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"initError": {
|
||||
"binding": {
|
||||
"declared_arg_slots": 1,
|
||||
"declared_ret_slots": 1,
|
||||
"expected_arg_slots": 0,
|
||||
"expected_ret_slots": 1,
|
||||
"module": "system",
|
||||
"name": "has_cart",
|
||||
"reason": "abi_mismatch",
|
||||
"version": 1
|
||||
},
|
||||
"code": "loader_patch_failed",
|
||||
"message": "LoaderPatchFailed(ResolveFailed(AbiMismatch { module: \"system\", name: \"has_cart\", version: 1, declared_arg_slots: 1, declared_ret_slots: 1, expected_arg_slots: 0, expected_ret_slots: 1 }))"
|
||||
},
|
||||
"kind": "vm_init",
|
||||
"pc": null,
|
||||
"summary": "Conformance fixture"
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"initError": {
|
||||
"binding": {
|
||||
"module": "system",
|
||||
"name": "has_cart",
|
||||
"provided": 0,
|
||||
"reason": "missing_capability",
|
||||
"required": 1,
|
||||
"version": 1
|
||||
},
|
||||
"code": "loader_patch_failed",
|
||||
"message": "LoaderPatchFailed(ResolveFailed(MissingCapability { required: 1, provided: 0, module: \"system\", name: \"has_cart\", version: 1 }))"
|
||||
},
|
||||
"kind": "vm_init",
|
||||
"pc": null,
|
||||
"summary": "Conformance fixture"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"initError": {
|
||||
"code": "registry_load_failed",
|
||||
"load": {
|
||||
"code": "LOAD_RESOURCE_LIMIT",
|
||||
"function": null,
|
||||
"id_domain": null,
|
||||
"limit": 0,
|
||||
"member": null,
|
||||
"offset": null,
|
||||
"pc": null,
|
||||
"reason": "Limit",
|
||||
"record": null,
|
||||
"referenced_id": null,
|
||||
"requested": 32,
|
||||
"resource": "Work",
|
||||
"section": null
|
||||
},
|
||||
"message": "RegistryLoadFailed(LoadFailure { code: ResourceLimit, reason: Limit, section: None, offset: None, record: None, member: None, id_domain: None, referenced_id: None, function: None, pc: None, resource: Some(Work), requested: Some(32), limit: Some(0) })"
|
||||
},
|
||||
"kind": "vm_init",
|
||||
"pc": null,
|
||||
"summary": "Conformance fixture"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"initError": {
|
||||
"code": "registry_load_failed",
|
||||
"load": {
|
||||
"code": "LOAD_INVALID_CODE",
|
||||
"function": 0,
|
||||
"id_domain": null,
|
||||
"limit": null,
|
||||
"member": null,
|
||||
"offset": null,
|
||||
"pc": 0,
|
||||
"reason": "Truncated",
|
||||
"record": null,
|
||||
"referenced_id": null,
|
||||
"requested": null,
|
||||
"resource": null,
|
||||
"section": 2
|
||||
},
|
||||
"message": "RegistryLoadFailed(LoadFailure { code: InvalidCode, reason: Truncated, section: Some(2), offset: None, record: None, member: None, id_domain: None, referenced_id: None, function: Some(0), pc: Some(0), resource: None, requested: None, limit: None })"
|
||||
},
|
||||
"kind": "vm_init",
|
||||
"pc": null,
|
||||
"summary": "Conformance fixture"
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"initError": {
|
||||
"binding": {
|
||||
"module": "system",
|
||||
"name": "missing",
|
||||
"reason": "unknown_syscall",
|
||||
"version": 1
|
||||
},
|
||||
"code": "loader_patch_failed",
|
||||
"message": "LoaderPatchFailed(ResolveFailed(UnknownSyscall { module: \"system\", name: \"missing\", version: 1 }))"
|
||||
},
|
||||
"kind": "vm_init",
|
||||
"pc": null,
|
||||
"summary": "Conformance fixture"
|
||||
}
|
||||
+50
-3
@@ -6,6 +6,7 @@ import java.io.ByteArrayOutputStream;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.nio.ByteOrder;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.Objects;
|
||||
@@ -48,8 +49,11 @@ public class BytecodeEmitter {
|
||||
final EmissionPlan plan) {
|
||||
final var inputPlan = plan == null ? EmissionPlan.empty() : plan;
|
||||
final var canonicalConstPool = dedupeConstPool(inputPlan.constPool());
|
||||
final var constantIndices = new HashMap<BytecodeModule.ConstantPoolEntry, Integer>();
|
||||
for (int i=0;i<canonicalConstPool.size();i++) constantIndices.put(canonicalConstPool.get(i),i);
|
||||
final var canonicalExports = dedupeExports(inputPlan.exports());
|
||||
final var functionFragments = new ArrayList<BytecodeFunctionLayoutBuilder.FunctionFragment>(inputPlan.functions().size());
|
||||
final var intrinsicSignatures = new LinkedHashMap<Integer, BytecodeModule.IntrinsicSignature>();
|
||||
final var orderedSyscalls = new LinkedHashMap<SyscallIdentity, BytecodeModule.SyscallDecl>();
|
||||
final var syscallIndexByIdentity = new LinkedHashMap<SyscallIdentity, Integer>();
|
||||
|
||||
@@ -59,6 +63,12 @@ public class BytecodeEmitter {
|
||||
for (final var op : function.operations()) {
|
||||
final var pc = code.size();
|
||||
switch (op.kind()) {
|
||||
case REGISTRY -> {
|
||||
var instruction = Objects.requireNonNull(op.registryInstruction());
|
||||
writeOpU32(code, instruction.op().opcode(), instruction.immediate());
|
||||
if (instruction.secondImmediate() != null) code.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(instruction.secondImmediate()).array());
|
||||
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
|
||||
}
|
||||
case HALT -> {
|
||||
writeOpNoImm(code, OP_HALT);
|
||||
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
|
||||
@@ -72,7 +82,7 @@ public class BytecodeEmitter {
|
||||
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
|
||||
}
|
||||
case PUSH_CONST -> {
|
||||
writeOpU32(code, OP_PUSH_CONST, op.immediate());
|
||||
writeOpU32(code, OP_PUSH_CONST, constantIndices.get(inputPlan.constPool().get(op.immediate())));
|
||||
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
|
||||
}
|
||||
case POP -> {
|
||||
@@ -176,6 +186,10 @@ public class BytecodeEmitter {
|
||||
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
|
||||
}
|
||||
case INTRINSIC -> {
|
||||
if(op.expectedArgSlots()==null||op.expectedRetSlots()==null) throw new IllegalArgumentException("intrinsic signature is required");
|
||||
var signature = new BytecodeModule.IntrinsicSignature(op.immediate(),op.expectedArgSlots(),op.expectedRetSlots());
|
||||
var previous = intrinsicSignatures.putIfAbsent(op.immediate(),signature);
|
||||
if(previous!=null&&!previous.equals(signature)) throw new IllegalArgumentException("conflicting intrinsic ABI");
|
||||
writeOpU32(code, OP_INTRINSIC, op.immediate());
|
||||
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
|
||||
}
|
||||
@@ -197,6 +211,7 @@ public class BytecodeEmitter {
|
||||
syscallIndexByIdentity.put(identity, syscallIndexByIdentity.size());
|
||||
orderedSyscalls.put(identity, decl);
|
||||
}
|
||||
if (!orderedSyscalls.get(identity).equals(decl)) throw new IllegalArgumentException("conflicting HOSTCALL ABI for " + identity);
|
||||
final var index = syscallIndexByIdentity.get(identity);
|
||||
writeOpU32(code, OP_HOSTCALL, index);
|
||||
spans.add(new BytecodeFunctionLayoutBuilder.InstructionSpan(pc, spanOrUnknown(op.span())));
|
||||
@@ -213,7 +228,7 @@ public class BytecodeEmitter {
|
||||
function.localSlots(),
|
||||
function.returnSlots(),
|
||||
function.maxStackSlots(),
|
||||
ReadOnlyList.wrap(spans)));
|
||||
ReadOnlyList.wrap(spans), function.metadata()));
|
||||
}
|
||||
|
||||
final var layout = BytecodeFunctionLayoutBuilder.build(ReadOnlyList.wrap(functionFragments));
|
||||
@@ -224,7 +239,7 @@ public class BytecodeEmitter {
|
||||
layout.code(),
|
||||
layout.debugInfo(),
|
||||
canonicalExports,
|
||||
ReadOnlyList.wrap(orderedSyscalls.values()));
|
||||
ReadOnlyList.wrap(orderedSyscalls.values()), inputPlan.registry(), ReadOnlyList.wrap(intrinsicSignatures.values()));
|
||||
}
|
||||
|
||||
private static void writeOpNoImm(final ByteArrayOutputStream out, final int opcode) {
|
||||
@@ -283,10 +298,19 @@ public class BytecodeEmitter {
|
||||
}
|
||||
|
||||
public record EmissionPlan(
|
||||
int version,
|
||||
ReadOnlyList<BytecodeModule.ConstantPoolEntry> constPool,
|
||||
ReadOnlyList<BytecodeModule.Export> exports,
|
||||
ReadOnlyList<FunctionPlan> functions,
|
||||
p.studio.compiler.models.registry.IRRegistry registry) {
|
||||
public EmissionPlan(
|
||||
int version,
|
||||
ReadOnlyList<BytecodeModule.ConstantPoolEntry> constPool,
|
||||
ReadOnlyList<BytecodeModule.Export> exports,
|
||||
ReadOnlyList<FunctionPlan> functions) {
|
||||
this(version, constPool, exports, functions, p.studio.compiler.models.registry.IRRegistry.empty());
|
||||
}
|
||||
|
||||
public EmissionPlan {
|
||||
constPool = constPool == null ? ReadOnlyList.empty() : constPool;
|
||||
exports = exports == null ? ReadOnlyList.empty() : exports;
|
||||
@@ -299,12 +323,23 @@ public class BytecodeEmitter {
|
||||
}
|
||||
|
||||
public record FunctionPlan(
|
||||
String name,
|
||||
int paramSlots,
|
||||
int localSlots,
|
||||
int returnSlots,
|
||||
int maxStackSlots,
|
||||
ReadOnlyList<Operation> operations,
|
||||
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
|
||||
public FunctionPlan(
|
||||
String name,
|
||||
int paramSlots,
|
||||
int localSlots,
|
||||
int returnSlots,
|
||||
int maxStackSlots,
|
||||
ReadOnlyList<Operation> operations) {
|
||||
this(name, paramSlots, localSlots, returnSlots, maxStackSlots, operations, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
|
||||
}
|
||||
|
||||
public FunctionPlan {
|
||||
Objects.requireNonNull(name, "name");
|
||||
operations = operations == null ? ReadOnlyList.empty() : operations;
|
||||
@@ -312,6 +347,7 @@ public class BytecodeEmitter {
|
||||
}
|
||||
|
||||
public enum OperationKind {
|
||||
REGISTRY,
|
||||
HALT,
|
||||
RET,
|
||||
PUSH_CONST,
|
||||
@@ -347,12 +383,23 @@ public class BytecodeEmitter {
|
||||
}
|
||||
|
||||
public record Operation(
|
||||
OperationKind kind,
|
||||
int immediate,
|
||||
BytecodeModule.SyscallDecl syscallDecl,
|
||||
Integer expectedArgSlots,
|
||||
Integer expectedRetSlots,
|
||||
BytecodeModule.SourceSpan span,
|
||||
p.studio.compiler.backend.irvm.IRVMInstruction registryInstruction) {
|
||||
public Operation(
|
||||
OperationKind kind,
|
||||
int immediate,
|
||||
BytecodeModule.SyscallDecl syscallDecl,
|
||||
Integer expectedArgSlots,
|
||||
Integer expectedRetSlots,
|
||||
BytecodeModule.SourceSpan span) {
|
||||
this(kind, immediate, syscallDecl, expectedArgSlots, expectedRetSlots, span, null);
|
||||
}
|
||||
|
||||
public Operation {
|
||||
Objects.requireNonNull(kind, "kind");
|
||||
}
|
||||
|
||||
+13
-1
@@ -25,7 +25,7 @@ public final class BytecodeFunctionLayoutBuilder {
|
||||
fn.paramSlots(),
|
||||
fn.localSlots(),
|
||||
fn.returnSlots(),
|
||||
fn.maxStackSlots()));
|
||||
fn.maxStackSlots(), fn.metadata()));
|
||||
names.add(new BytecodeModule.FunctionName(i, fn.name()));
|
||||
|
||||
for (final var span : fn.instructionSpans()) {
|
||||
@@ -58,6 +58,15 @@ public final class BytecodeFunctionLayoutBuilder {
|
||||
}
|
||||
|
||||
public record FunctionFragment(
|
||||
String name,
|
||||
byte[] code,
|
||||
int paramSlots,
|
||||
int localSlots,
|
||||
int returnSlots,
|
||||
int maxStackSlots,
|
||||
ReadOnlyList<InstructionSpan> instructionSpans,
|
||||
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
|
||||
public FunctionFragment(
|
||||
String name,
|
||||
byte[] code,
|
||||
int paramSlots,
|
||||
@@ -65,6 +74,9 @@ public final class BytecodeFunctionLayoutBuilder {
|
||||
int returnSlots,
|
||||
int maxStackSlots,
|
||||
ReadOnlyList<InstructionSpan> instructionSpans) {
|
||||
this(name, code, paramSlots, localSlots, returnSlots, maxStackSlots, instructionSpans, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
|
||||
}
|
||||
|
||||
public FunctionFragment {
|
||||
Objects.requireNonNull(name, "name");
|
||||
code = code == null ? new byte[0] : code.clone();
|
||||
|
||||
+7
-4
@@ -46,16 +46,18 @@ public class BytecodeLinkPrecheckService {
|
||||
}
|
||||
|
||||
private void validateHostcallIndices(final BytecodeModule module) {
|
||||
var pc = 0;
|
||||
while (pc < module.code().length) {
|
||||
if (pc + 2 > module.code().length) {
|
||||
for (var function : module.functions()) {
|
||||
int end = function.codeOffset() + function.codeLen();
|
||||
var pc = function.codeOffset();
|
||||
while (pc < end) {
|
||||
if (pc + 2 > end) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION,
|
||||
"truncated opcode at pc=" + pc);
|
||||
}
|
||||
final var opcode = readU16(module.code(), pc);
|
||||
final var size = sizeForOpcode(opcode, pc);
|
||||
if (pc + size > module.code().length) {
|
||||
if (pc + size > end) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION,
|
||||
"truncated instruction at pc=" + pc);
|
||||
@@ -70,6 +72,7 @@ public class BytecodeLinkPrecheckService {
|
||||
}
|
||||
pc += size;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private int sizeForOpcode(
|
||||
|
||||
+2
@@ -1,6 +1,8 @@
|
||||
package p.studio.compiler.backend.bytecode;
|
||||
|
||||
public enum BytecodeMarshalingErrorCode {
|
||||
MARSHAL_FORMAT_REGISTRY_INVALID,
|
||||
MARSHAL_VERIFY_REGISTRY_INVALID,
|
||||
MARSHAL_FORMAT_STRING_TOO_LONG,
|
||||
MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG,
|
||||
MARSHAL_FORMAT_SYSC_NAME_TOO_LONG,
|
||||
|
||||
+41
-196
@@ -2,11 +2,7 @@ package p.studio.compiler.backend.bytecode;
|
||||
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.nio.ByteOrder;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Objects;
|
||||
|
||||
public record BytecodeModule(
|
||||
@@ -16,16 +12,38 @@ public record BytecodeModule(
|
||||
byte[] code,
|
||||
DebugInfo debugInfo,
|
||||
ReadOnlyList<Export> exports,
|
||||
ReadOnlyList<SyscallDecl> syscalls) {
|
||||
ReadOnlyList<SyscallDecl> syscalls,
|
||||
p.studio.compiler.models.registry.IRRegistry registry,
|
||||
ReadOnlyList<IntrinsicSignature> intrinsicSignatures) {
|
||||
public BytecodeModule(
|
||||
int version,
|
||||
ReadOnlyList<ConstantPoolEntry> constPool,
|
||||
ReadOnlyList<FunctionMeta> functions,
|
||||
byte[] code,
|
||||
DebugInfo debugInfo,
|
||||
ReadOnlyList<Export> exports,
|
||||
ReadOnlyList<SyscallDecl> syscalls,
|
||||
p.studio.compiler.models.registry.IRRegistry registry) {
|
||||
this(version, constPool, functions, code, debugInfo, exports, syscalls, registry, ReadOnlyList.empty());
|
||||
}
|
||||
|
||||
private static final int SECTION_KIND_CONST_POOL = 0;
|
||||
private static final int SECTION_KIND_FUNCTIONS = 1;
|
||||
private static final int SECTION_KIND_CODE = 2;
|
||||
private static final int SECTION_KIND_DEBUG = 3;
|
||||
private static final int SECTION_KIND_EXPORTS = 4;
|
||||
private static final int SECTION_KIND_SYSCALLS = 5;
|
||||
public BytecodeModule(
|
||||
int version,
|
||||
ReadOnlyList<ConstantPoolEntry> constPool,
|
||||
ReadOnlyList<FunctionMeta> functions,
|
||||
byte[] code,
|
||||
DebugInfo debugInfo,
|
||||
ReadOnlyList<Export> exports,
|
||||
ReadOnlyList<SyscallDecl> syscalls) {
|
||||
this(version, constPool, functions, code, debugInfo, exports, syscalls, p.studio.compiler.models.registry.IRRegistry.empty());
|
||||
}
|
||||
|
||||
|
||||
public record IntrinsicSignature(int id, int argumentSlots, int resultSlots) {}
|
||||
|
||||
public BytecodeModule {
|
||||
intrinsicSignatures = intrinsicSignatures == null ? ReadOnlyList.empty() : ReadOnlyList.wrap(intrinsicSignatures.asList());
|
||||
registry = registry == null ? p.studio.compiler.models.registry.IRRegistry.empty() : registry;
|
||||
constPool = constPool == null ? ReadOnlyList.empty() : constPool;
|
||||
functions = functions == null ? ReadOnlyList.empty() : functions;
|
||||
code = code == null ? new byte[0] : code.clone();
|
||||
@@ -45,191 +63,7 @@ public record BytecodeModule(
|
||||
}
|
||||
|
||||
public byte[] serialize() {
|
||||
final var constPoolData = serializeConstPool();
|
||||
final var functionData = serializeFunctions();
|
||||
final var codeData = code();
|
||||
final var debugData = serializeDebug();
|
||||
final var exportData = serializeExports();
|
||||
final var syscallData = serializeSyscalls();
|
||||
|
||||
final var sections = new ArrayList<SectionData>();
|
||||
if (constPoolData.length > 0) {
|
||||
sections.add(new SectionData(SECTION_KIND_CONST_POOL, constPoolData));
|
||||
}
|
||||
if (functionData.length > 0) {
|
||||
sections.add(new SectionData(SECTION_KIND_FUNCTIONS, functionData));
|
||||
}
|
||||
if (codeData.length > 0) {
|
||||
sections.add(new SectionData(SECTION_KIND_CODE, codeData));
|
||||
}
|
||||
if (debugData.length > 0) {
|
||||
sections.add(new SectionData(SECTION_KIND_DEBUG, debugData));
|
||||
}
|
||||
if (exportData.length > 0) {
|
||||
sections.add(new SectionData(SECTION_KIND_EXPORTS, exportData));
|
||||
}
|
||||
// SYSC section is mandatory, even when empty.
|
||||
sections.add(new SectionData(SECTION_KIND_SYSCALLS, syscallData));
|
||||
|
||||
final var out = new ByteArrayOutputStream();
|
||||
|
||||
out.writeBytes(new byte[] { 'P', 'B', 'X', 0 });
|
||||
writeU16(out, version & 0xFFFF);
|
||||
out.write(0);
|
||||
out.write(0);
|
||||
writeU32(out, sections.size());
|
||||
out.writeBytes(new byte[20]);
|
||||
|
||||
var currentOffset = 32 + (sections.size() * 12);
|
||||
for (final var section : sections) {
|
||||
writeU32(out, section.kind);
|
||||
writeU32(out, currentOffset);
|
||||
writeU32(out, section.data.length);
|
||||
currentOffset += section.data.length;
|
||||
}
|
||||
|
||||
for (final var section : sections) {
|
||||
out.writeBytes(section.data);
|
||||
}
|
||||
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
private byte[] serializeConstPool() {
|
||||
if (constPool.isEmpty()) {
|
||||
return new byte[0];
|
||||
}
|
||||
final var out = new ByteArrayOutputStream();
|
||||
writeU32(out, constPool.size());
|
||||
for (final var entry : constPool) {
|
||||
switch (entry) {
|
||||
case NullConstant ignored -> out.write(0);
|
||||
case Int64Constant value -> {
|
||||
out.write(1);
|
||||
writeI64(out, value.value());
|
||||
}
|
||||
case Float64Constant value -> {
|
||||
out.write(2);
|
||||
writeF64(out, value.value());
|
||||
}
|
||||
case BooleanConstant value -> {
|
||||
out.write(3);
|
||||
out.write(value.value() ? 1 : 0);
|
||||
}
|
||||
case StringConstant value -> {
|
||||
out.write(4);
|
||||
writeU32(out, value.utf8().length);
|
||||
out.writeBytes(value.utf8());
|
||||
}
|
||||
case Int32Constant value -> {
|
||||
out.write(5);
|
||||
writeI32(out, value.value());
|
||||
}
|
||||
}
|
||||
}
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
private byte[] serializeFunctions() {
|
||||
if (functions.isEmpty()) {
|
||||
return new byte[0];
|
||||
}
|
||||
final var out = new ByteArrayOutputStream();
|
||||
writeU32(out, functions.size());
|
||||
for (final var function : functions) {
|
||||
writeU32(out, function.codeOffset());
|
||||
writeU32(out, function.codeLen());
|
||||
writeU16(out, function.paramSlots());
|
||||
writeU16(out, function.localSlots());
|
||||
writeU16(out, function.returnSlots());
|
||||
writeU16(out, function.maxStackSlots());
|
||||
}
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
private byte[] serializeDebug() {
|
||||
if (debugInfo == null || (debugInfo.pcToSpan().isEmpty() && debugInfo.functionNames().isEmpty())) {
|
||||
return new byte[0];
|
||||
}
|
||||
final var out = new ByteArrayOutputStream();
|
||||
writeU32(out, debugInfo.pcToSpan().size());
|
||||
for (final var entry : debugInfo.pcToSpan()) {
|
||||
writeU32(out, entry.pc());
|
||||
writeU32(out, entry.span().fileId());
|
||||
writeU32(out, entry.span().start());
|
||||
writeU32(out, entry.span().end());
|
||||
}
|
||||
writeU32(out, debugInfo.functionNames().size());
|
||||
for (final var fn : debugInfo.functionNames()) {
|
||||
writeU32(out, fn.funcIdx());
|
||||
writeU32(out, fn.utf8Name().length);
|
||||
out.writeBytes(fn.utf8Name());
|
||||
}
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
private byte[] serializeExports() {
|
||||
if (exports.isEmpty()) {
|
||||
return new byte[0];
|
||||
}
|
||||
final var out = new ByteArrayOutputStream();
|
||||
writeU32(out, exports.size());
|
||||
for (final var export : exports) {
|
||||
writeU32(out, export.funcIdx());
|
||||
writeU32(out, export.utf8Symbol().length);
|
||||
out.writeBytes(export.utf8Symbol());
|
||||
}
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
private byte[] serializeSyscalls() {
|
||||
final var out = new ByteArrayOutputStream();
|
||||
writeU32(out, syscalls.size());
|
||||
for (final var syscall : syscalls) {
|
||||
if (syscall.utf8Module().length > 0xFFFF) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG,
|
||||
"syscall module name exceeds u16: " + syscall.module());
|
||||
}
|
||||
if (syscall.utf8Name().length > 0xFFFF) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_NAME_TOO_LONG,
|
||||
"syscall name exceeds u16: " + syscall.name());
|
||||
}
|
||||
writeU16(out, syscall.utf8Module().length);
|
||||
out.writeBytes(syscall.utf8Module());
|
||||
writeU16(out, syscall.utf8Name().length);
|
||||
out.writeBytes(syscall.utf8Name());
|
||||
writeU16(out, syscall.version());
|
||||
writeU16(out, syscall.argSlots());
|
||||
writeU16(out, syscall.retSlots());
|
||||
}
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
private static void writeU16(final ByteArrayOutputStream out, final int value) {
|
||||
out.writeBytes(ByteBuffer.allocate(2).order(ByteOrder.LITTLE_ENDIAN).putShort((short) (value & 0xFFFF)).array());
|
||||
}
|
||||
|
||||
private static void writeU32(final ByteArrayOutputStream out, final int value) {
|
||||
out.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(value).array());
|
||||
}
|
||||
|
||||
private static void writeI32(final ByteArrayOutputStream out, final int value) {
|
||||
out.writeBytes(ByteBuffer.allocate(4).order(ByteOrder.LITTLE_ENDIAN).putInt(value).array());
|
||||
}
|
||||
|
||||
private static void writeI64(final ByteArrayOutputStream out, final long value) {
|
||||
out.writeBytes(ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN).putLong(value).array());
|
||||
}
|
||||
|
||||
private static void writeF64(final ByteArrayOutputStream out, final double value) {
|
||||
out.writeBytes(ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN).putDouble(value).array());
|
||||
}
|
||||
|
||||
private record SectionData(
|
||||
int kind,
|
||||
byte[] data) {
|
||||
return PbxRegistryWriter.write(this);
|
||||
}
|
||||
|
||||
public sealed interface ConstantPoolEntry permits NullConstant, Int64Constant, Float64Constant, BooleanConstant, StringConstant, Int32Constant {
|
||||
@@ -271,12 +105,23 @@ public record BytecodeModule(
|
||||
}
|
||||
|
||||
public record FunctionMeta(
|
||||
int codeOffset,
|
||||
int codeLen,
|
||||
int paramSlots,
|
||||
int localSlots,
|
||||
int returnSlots,
|
||||
int maxStackSlots,
|
||||
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
|
||||
public FunctionMeta(
|
||||
int codeOffset,
|
||||
int codeLen,
|
||||
int paramSlots,
|
||||
int localSlots,
|
||||
int returnSlots,
|
||||
int maxStackSlots) {
|
||||
this(codeOffset, codeLen, paramSlots, localSlots, returnSlots, maxStackSlots, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
public record SourceSpan(
|
||||
|
||||
+12
-53
@@ -1,58 +1,17 @@
|
||||
package p.studio.compiler.backend.bytecode;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
final class BytecodeOpcodeLayout {
|
||||
private static final Map<Integer, Integer> SIZE_BY_OPCODE = Map.ofEntries(
|
||||
Map.entry(0x01, 2),
|
||||
Map.entry(0x02, 6),
|
||||
Map.entry(0x03, 6),
|
||||
Map.entry(0x04, 6),
|
||||
Map.entry(0x10, 6),
|
||||
Map.entry(0x11, 2),
|
||||
Map.entry(0x14, 10),
|
||||
Map.entry(0x15, 10),
|
||||
Map.entry(0x16, 3),
|
||||
Map.entry(0x17, 6),
|
||||
Map.entry(0x18, 6),
|
||||
Map.entry(0x20, 2),
|
||||
Map.entry(0x21, 2),
|
||||
Map.entry(0x22, 2),
|
||||
Map.entry(0x23, 2),
|
||||
Map.entry(0x24, 2),
|
||||
Map.entry(0x30, 2),
|
||||
Map.entry(0x31, 2),
|
||||
Map.entry(0x32, 2),
|
||||
Map.entry(0x33, 2),
|
||||
Map.entry(0x34, 2),
|
||||
Map.entry(0x35, 2),
|
||||
Map.entry(0x36, 2),
|
||||
Map.entry(0x3C, 2),
|
||||
Map.entry(0x3D, 2),
|
||||
Map.entry(0x3E, 2),
|
||||
Map.entry(0x40, 6),
|
||||
Map.entry(0x41, 6),
|
||||
Map.entry(0x42, 6),
|
||||
Map.entry(0x43, 6),
|
||||
Map.entry(0x50, 6),
|
||||
Map.entry(0x51, 2),
|
||||
Map.entry(0x52, 10),
|
||||
Map.entry(0x54, 10),
|
||||
Map.entry(0x56, 6),
|
||||
Map.entry(0x70, 6),
|
||||
Map.entry(0x71, 6),
|
||||
Map.entry(0x72, 6));
|
||||
|
||||
private BytecodeOpcodeLayout() {
|
||||
}
|
||||
|
||||
static int sizeForOpcode(final int opcode) {
|
||||
final var size = SIZE_BY_OPCODE.get(opcode);
|
||||
if (size == null) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE,
|
||||
"unknown opcode 0x%04X".formatted(opcode));
|
||||
}
|
||||
return size;
|
||||
private BytecodeOpcodeLayout() {}
|
||||
static int sizeForOpcode(int opcode) {
|
||||
return switch(opcode) {
|
||||
case 0x00,0x01,0x05,0x11,0x12,0x13,0x20,0x21,0x22,0x23,0x24,
|
||||
0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0x3a,0x3b,0x3c,0x3d,0x3e,0x51,0x80 -> 2;
|
||||
case 0x16 -> 3;
|
||||
case 0x02,0x03,0x04,0x10,0x17,0x18,0x40,0x41,0x42,0x43,0x50,0x53,0x57,0x70,0x71,0x72,
|
||||
0x90,0x91,0x92,0xa0,0xa3,0xa4,0xa5,0xa6 -> 6;
|
||||
case 0x14,0x15,0x52,0x58,0x93,0x94,0x95,0x96,0xa1,0xa2 -> 10;
|
||||
default -> throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE,
|
||||
"unknown or retired opcode 0x%04X".formatted(opcode));
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
+223
-57
@@ -1,70 +1,236 @@
|
||||
package p.studio.compiler.backend.bytecode;
|
||||
|
||||
import java.nio.ByteBuffer;
|
||||
import java.nio.ByteOrder;
|
||||
import java.util.Set;
|
||||
import p.studio.compiler.backend.registry.LinkedRegistry;
|
||||
import p.studio.compiler.models.registry.IRRegistry.*;
|
||||
import java.nio.*;
|
||||
import java.util.*;
|
||||
|
||||
/** Producer verification. Native binding and final admission remain loader responsibilities. */
|
||||
public class BytecodePreloadVerifierService {
|
||||
private static final Set<Integer> TERMINATORS = Set.of(0x01, 0x02, 0x51);
|
||||
|
||||
public void verify(final BytecodeModule module) {
|
||||
final var input = module == null ? BytecodeModule.empty() : module;
|
||||
for (final var function : input.functions()) {
|
||||
verifyFunction(input.code(), function);
|
||||
public void verify(BytecodeModule module) {
|
||||
try { verifyRegistry(module); }
|
||||
catch (IllegalArgumentException | ArithmeticException error) {
|
||||
throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_REGISTRY_INVALID, error.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private void verifyFunction(final byte[] code, final BytecodeModule.FunctionMeta function) {
|
||||
final var start = function.codeOffset();
|
||||
final var end = start + function.codeLen();
|
||||
if (end <= start) {
|
||||
return;
|
||||
}
|
||||
|
||||
var pc = start;
|
||||
var lastOpcode = -1;
|
||||
while (pc < end) {
|
||||
if (pc + 2 > end) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION,
|
||||
"truncated opcode in function at pc=" + pc);
|
||||
private void verifyRegistry(BytecodeModule module) {
|
||||
Objects.requireNonNull(module);
|
||||
// Validate the complete declarative schema without accepting an old slot-only image.
|
||||
module.serialize();
|
||||
var registry=LinkedRegistry.link(module.registry());
|
||||
var used=new boolean[module.syscalls().size()];
|
||||
for(int i=0;i<module.functions().size();i++)verifyFunction(module,registry,i,used);
|
||||
for(int i=0;i<used.length;i++)check(used[i],"unused syscall declaration "+i);
|
||||
}
|
||||
private record Instruction(int pc,int opcode,int first,int second,int next) {}
|
||||
private void verifyFunction(BytecodeModule m,LinkedRegistry r,int index,boolean[] used) {
|
||||
var f=m.functions().get(index);byte[] code=m.code();int start=f.codeOffset(),end=start+f.codeLen();
|
||||
var instructions=new TreeMap<Integer,Instruction>();
|
||||
for(int pc=start;pc<end;) {
|
||||
check(pc+2<=end,"truncated opcode");int op=u16(code,pc);
|
||||
if(op==0x70)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_LINKAGE_RAW_SYSCALL_IN_PRELOAD,"raw SYSCALL in producer image");int size=BytecodeOpcodeLayout.sizeForOpcode(op);check((long)pc+size<=end,"truncated instruction");
|
||||
int a=size>=6?i32(code,pc+2):0,b=size==10?i32(code,pc+6):0;
|
||||
var instruction=new Instruction(pc,op,a,b,pc+size);instructions.put(pc,instruction);
|
||||
switch(op) {
|
||||
case 0x10 -> check(a>=0&&a<m.constPool().size(),"constant index");
|
||||
case 0x16 -> check(code[pc+2]==0||code[pc+2]==1,"Boolean immediate");
|
||||
case 0x15 -> check(Double.isFinite(Double.longBitsToDouble(ByteBuffer.wrap(code,pc+2,8).order(ByteOrder.LITTLE_ENDIAN).getLong())),"nonfinite PushF64");
|
||||
case 0x18,0x40,0x41 -> check(a>=0,"negative count/index");
|
||||
case 0x42,0x43 -> check(a>=0&&(long)a<f.paramSlots()+f.localSlots(),"local index");
|
||||
case 0x50 -> {check(a>=0&&a<m.functions().size(),"FunctionId");check(m.functions().get(a).metadata().captures().isEmpty(),"direct call into capturing function");}
|
||||
case 0x71 -> {check(a>=0&&a<m.syscalls().size(),"HOSTCALL declaration index");used[a]=true;}
|
||||
case 0x72 -> check(m.intrinsicSignatures().stream().anyMatch(n->n.id()==a&&n.argumentSlots()>=0&&n.resultSlots()>=0),"intrinsic signature required");
|
||||
case 0x52 -> {check(a>=0&&a<m.functions().size(),"closure FunctionId");var target=m.functions().get(a).metadata();var fn=fn(r,b);check(r.sameSignature(fn.signature(),target.signature()),"closure signature");}
|
||||
case 0x53 -> fn(r,a);
|
||||
case 0x57 -> check(a>=0&&a<f.metadata().captures().size(),"capture index");
|
||||
case 0x58 -> method(r,a,b);
|
||||
case 0x90,0x91,0x92 -> type(r,a);
|
||||
case 0x93,0x94,0x95,0x96 -> {var src=type(r,a).body();var dst=type(r,b).body();check(src instanceof Struct||src instanceof Array||src instanceof Trait,"reference source category");check(op==0x94||op==0x95?dst instanceof Trait:dst instanceof Struct||dst instanceof Array,"reference target category");}
|
||||
case 0xa0 -> check(type(r,a).body() instanceof Struct,"Struct category");
|
||||
case 0xa1,0xa2 -> field(r,a,b);
|
||||
case 0xa3,0xa4,0xa5,0xa6 -> array(r,a);
|
||||
default -> { }
|
||||
}
|
||||
final var opcode = readU16(code, pc);
|
||||
if (opcode == 0x70) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_LINKAGE_RAW_SYSCALL_IN_PRELOAD,
|
||||
"raw syscall is forbidden in pre-load artifact");
|
||||
pc+=size;
|
||||
}
|
||||
for(var instruction:instructions.values())if(instruction.opcode()>=2&&instruction.opcode()<=4) {
|
||||
// Jump operands are absolute offsets within the current function.
|
||||
long target=(long)start+instruction.first();
|
||||
check(target>=start&&target<=end&&(target==end||instructions.containsKey((int)target)),"jump outside function or into immediate");
|
||||
}
|
||||
analyze(m,r,index,instructions,end);
|
||||
}
|
||||
private record Fact(String carrier,Integer type,int component,int width,Long origin,Long integer) {
|
||||
Fact(String carrier,Integer type,int component,int width) {this(carrier,type,component,width,null,null);}
|
||||
static final Fact UNKNOWN=new Fact("?",null,0,1),NULL=new Fact("null",null,0,1);
|
||||
}
|
||||
private static final class State {
|
||||
Fact[] locals;ArrayList<Fact> stack;
|
||||
State(Fact[] locals,ArrayList<Fact> stack){this.locals=locals;this.stack=stack;}
|
||||
State copy(){return new State(locals.clone(),new ArrayList<>(stack));}
|
||||
}
|
||||
private List<Fact> facts(LinkedRegistry r,TypeRef ref) {
|
||||
int id=r.id(ref).index(),width=r.width(ref);var body=r.type(ref).body();String carrier;
|
||||
if(body instanceof Builtin b)carrier=switch(b.id()){case 1,3,0x11->"i64";case 2,0x12->"f64";case 0x10->"i32";case 0x13->"bool";case 0x14->"string";case 0x15->"?";default->"ref";};
|
||||
else carrier=body instanceof Trait?"?":"ref";
|
||||
var result=new ArrayList<Fact>();for(int i=0;i<width;i++) {
|
||||
String componentCarrier=carrier;
|
||||
if(body instanceof Builtin b&&(b.id()==1||b.id()==3&&i==2))componentCarrier="color";
|
||||
if(body instanceof Trait)componentCarrier=i==0?"ref|null":"i64|null";
|
||||
if(body instanceof Builtin b&&b.id()==0x15)componentCarrier=i==0?"i64|null":"?";
|
||||
result.add(new Fact(componentCarrier,id,i,width));
|
||||
}return result;
|
||||
}
|
||||
private List<Fact> parameters(LinkedRegistry r,Signature signature){var result=new ArrayList<Fact>();for(var p:signature.parameters())result.addAll(facts(r,p.type()));return result;}
|
||||
private List<Fact> results(LinkedRegistry r,Signature signature){var result=new ArrayList<Fact>();for(var t:signature.results())result.addAll(facts(r,t));return result;}
|
||||
private void analyze(BytecodeModule m,LinkedRegistry r,int index,TreeMap<Integer,Instruction> code,int end) {
|
||||
var f=m.functions().get(index);var locals=new Fact[f.paramSlots()+f.localSlots()];Arrays.fill(locals,Fact.NULL);
|
||||
var parameters=parameters(r,f.metadata().signature());for(int i=0;i<parameters.size();i++)locals[i]=stamp(parameters.get(i),-1L-(i-parameters.get(i).component()));
|
||||
var states=new HashMap<Integer,State>();var queue=new ArrayDeque<Integer>();states.put(f.codeOffset(),new State(locals,new ArrayList<>()));queue.add(f.codeOffset());
|
||||
long work=0;
|
||||
while(!queue.isEmpty()) {
|
||||
int pc=queue.remove();var state=states.get(pc).copy();var instruction=code.get(pc);
|
||||
check((work+=1L+state.locals.length+state.stack.size())<=268_435_456L,"verification work limit");
|
||||
transfer(m,r,index,instruction,state,false);
|
||||
for(int successor:successors(instruction,f.codeOffset(),end)) {
|
||||
if(successor==end) {check(state.stack.size()==f.returnSlots(),"function-end result height");continue;}
|
||||
var old=states.get(successor);if(old==null){check((long)(states.size()+1)*(locals.length+f.maxStackSlots())*24<=128L*1024*1024,"verification scratch limit");states.put(successor,state.copy());queue.add(successor);}
|
||||
else {check(old.stack.size()==state.stack.size(),"join height mismatch");boolean changed=false;
|
||||
for(int i=0;i<old.locals.length;i++)if(!old.locals[i].equals(state.locals[i])&&!old.locals[i].equals(Fact.UNKNOWN)){old.locals[i]=Fact.UNKNOWN;changed=true;}
|
||||
for(int i=0;i<old.stack.size();i++)if(!old.stack.get(i).equals(state.stack.get(i))&&!old.stack.get(i).equals(Fact.UNKNOWN)){old.stack.set(i,Fact.UNKNOWN);changed=true;}
|
||||
if(changed)queue.add(successor);
|
||||
}
|
||||
}
|
||||
final var size = sizeForOpcode(opcode, pc);
|
||||
if (pc + size > end) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_TRUNCATED_INSTRUCTION,
|
||||
"truncated instruction in function at pc=" + pc);
|
||||
}
|
||||
// Diagnose proven type errors after widening has converged, not in traversal order.
|
||||
for(var entry:states.entrySet()) {
|
||||
var state=entry.getValue().copy();var instruction=code.get(entry.getKey());transfer(m,r,index,instruction,state,true);
|
||||
for(int successor:successors(instruction,f.codeOffset(),end))if(successor==end)compatible(state.stack,results(r,f.metadata().signature()),true);
|
||||
}
|
||||
}
|
||||
private List<Integer> successors(Instruction i,int start,int end) {
|
||||
if(i.opcode()==0x01||i.opcode()==0x05||i.opcode()==0x51)return List.of();
|
||||
if(i.opcode()==0x02)return List.of(start+i.first());
|
||||
if(i.opcode()==0x03||i.opcode()==0x04) {
|
||||
if(i.next()==end)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION,"conditional fallthrough at function end");
|
||||
return List.of(i.next(),start+i.first());
|
||||
}
|
||||
if(i.next()>=end)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION,"reachable fallthrough at function end");return List.of(i.next());
|
||||
}
|
||||
private void transfer(BytecodeModule m,LinkedRegistry r,int index,Instruction i,State s,boolean typed) {
|
||||
int a=i.first(),b=i.second(),op=i.opcode();var f=m.functions().get(index);
|
||||
switch(op) {
|
||||
case 0x00,0x01,0x02,0x05,0x80 -> { }
|
||||
case 0x03,0x04 -> consume(s,List.of(carrier("bool")),typed);
|
||||
case 0x10 -> {var c=m.constPool().get(a);s.stack.add(switch(c){case BytecodeModule.NullConstant n->Fact.NULL;case BytecodeModule.Int32Constant n->carrier("i32");case BytecodeModule.Int64Constant n->integer(n.value());case BytecodeModule.Float64Constant n->carrier("f64");case BytecodeModule.BooleanConstant n->carrier("bool");case BytecodeModule.StringConstant n->carrier("string");});}
|
||||
case 0x11 -> pop(s,1);
|
||||
case 0x12 -> {var value=pop(s,1);s.stack.addAll(value);s.stack.addAll(value);}
|
||||
case 0x13 -> {var values=pop(s,2);s.stack.add(values.get(1));s.stack.add(values.get(0));}
|
||||
case 0x14 -> s.stack.add(integer(Integer.toUnsignedLong(a)|((long)b<<32)));case 0x15->s.stack.add(carrier("f64"));case 0x16->s.stack.add(carrier("bool"));case 0x17->s.stack.add(carrier("i32"));case 0x18->pop(s,a);
|
||||
case 0x40 -> s.stack.add(Fact.UNKNOWN);case 0x41->pop(s,1);
|
||||
case 0x42 -> s.stack.add(s.locals[a]);case 0x43 -> {var value=pop(s,1).getFirst();s.locals[a]=value;}
|
||||
case 0x50 -> {var metadata=m.functions().get(a).metadata();consume(s,parameters(r,metadata.signature()),typed);produce(s,results(r,metadata.signature()),i.pc());}
|
||||
case 0x51 -> {check(s.stack.size()==f.returnSlots(),"RET result height");compatible(s.stack,results(r,f.metadata().signature()),typed);}
|
||||
case 0x52 -> {var target=m.functions().get(a).metadata();var expected=new ArrayList<Fact>();for(var t:target.captures())expected.addAll(facts(r,t));consume(s,expected,typed);produce(s,facts(r,type(r,b).identity()),i.pc());}
|
||||
case 0x53 -> {consume(s,facts(r,type(r,a).identity()),typed);var signature=fn(r,a).signature();consume(s,parameters(r,signature),typed);produce(s,results(r,signature),i.pc());}
|
||||
case 0x57 -> produce(s,facts(r,f.metadata().captures().get(a)),i.pc());
|
||||
case 0x58 -> {consume(s,facts(r,type(r,a).identity()),typed);var signature=method(r,a,b).signature();consume(s,parameters(r,signature),typed);produce(s,results(r,signature),i.pc());}
|
||||
case 0x71 -> {var decl=m.syscalls().get(a);pop(s,decl.argSlots());for(int n=0;n<decl.retSlots();n++)s.stack.add(Fact.UNKNOWN);}
|
||||
case 0x72 -> {
|
||||
var signature=m.intrinsicSignatures().stream().filter(n->n.id()==a).findFirst().orElseThrow(()->new IllegalArgumentException("intrinsic signature required"));
|
||||
pop(s,signature.argumentSlots());for(int n=0;n<signature.resultSlots();n++)s.stack.add(Fact.UNKNOWN);
|
||||
}
|
||||
case 0x90 -> {
|
||||
consume(s,facts(r,type(r,a).identity()),typed);
|
||||
s.stack.add(new Fact("i64|null",-1,0,2,(long)i.pc(),null));
|
||||
s.stack.add(new Fact("?",-1,1,2,(long)i.pc(),null));
|
||||
}
|
||||
case 0x91 -> {dynamic(pop(s,2),typed);produce(s,facts(r,type(r,a).identity()),i.pc());}
|
||||
case 0x92 -> {dynamic(pop(s,2),typed);s.stack.add(carrier("bool"));}
|
||||
case 0x93,0x94,0x95,0x96 -> {consume(s,facts(r,type(r,a).identity()),typed);if(op==0x93||op==0x94)s.stack.add(carrier("bool"));else produce(s,facts(r,type(r,b).identity()),i.pc());}
|
||||
case 0xa0 -> produce(s,facts(r,type(r,a).identity()),i.pc());
|
||||
case 0xa1 -> {consume(s,facts(r,type(r,a).identity()),typed);produce(s,facts(r,field(r,a,b).type()),i.pc());}
|
||||
case 0xa2 -> {consume(s,facts(r,field(r,a,b).type()),typed);consume(s,facts(r,type(r,a).identity()),typed);}
|
||||
case 0xa3 -> {consume(s,List.of(carrier("i64")),typed);produce(s,facts(r,type(r,a).identity()),i.pc());}
|
||||
case 0xa4 -> {consume(s,facts(r,type(r,a).identity()),typed);s.stack.add(carrier("i64"));}
|
||||
case 0xa5 -> {consume(s,List.of(carrier("i64")),typed);consume(s,facts(r,type(r,a).identity()),typed);produce(s,facts(r,array(r,a).element()),i.pc());}
|
||||
case 0xa6 -> {consume(s,facts(r,array(r,a).element()),typed);consume(s,List.of(carrier("i64")),typed);consume(s,facts(r,type(r,a).identity()),typed);}
|
||||
case 0x34,0x35 -> {consume(s,List.of(carrier("bool"),carrier("bool")),typed);s.stack.add(carrier("bool"));}
|
||||
case 0x36 -> {consume(s,List.of(carrier("bool")),typed);s.stack.add(carrier("bool"));}
|
||||
case 0x3e -> {
|
||||
var value=pop(s,1).getFirst();
|
||||
if(typed)check(value.carrier().equals("?")||numeric(value.carrier()),"numeric operand required");
|
||||
s.stack.add(carrier(value.carrier()));
|
||||
}
|
||||
default -> {
|
||||
var operands=pop(s,2);String left=operands.get(0).carrier(),right=operands.get(1).carrier();
|
||||
boolean comparison=op>=0x30&&op<=0x33||op==0x3c||op==0x3d;
|
||||
boolean unknown=left.equals("?")||right.equals("?")||left.contains("|")||right.contains("|");
|
||||
boolean stringAdd=op==0x20&&(left.equals("string")||right.equals("string"));
|
||||
if(typed&&!unknown&&op!=0x30&&op!=0x31) {
|
||||
if(comparison)check(numeric(left)&&numeric(right)||left.equals(right)&&(left.equals("string")||left.equals("bool")),"comparison carrier mismatch");
|
||||
else if(!stringAdd)check(numeric(left)&&numeric(right)&&(op<0x37||!left.equals("f64")&&!right.equals("f64")),"numeric operand mismatch");
|
||||
}
|
||||
String result="?";
|
||||
if(stringAdd)result="string";
|
||||
else if(numeric(left)&&numeric(right))result=left.equals("f64")||right.equals("f64")?"f64":left.equals("i64")||right.equals("i64")?"i64":"i32";
|
||||
s.stack.add(carrier(comparison?"bool":result));
|
||||
}
|
||||
lastOpcode = opcode;
|
||||
pc += size;
|
||||
}
|
||||
|
||||
if (!TERMINATORS.contains(lastOpcode)) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNTERMINATED_FUNCTION,
|
||||
"function ends without terminator opcode");
|
||||
check(s.stack.size()<=f.maxStackSlots(),"maximum temporary slots exceeded (zero is literal)");
|
||||
}
|
||||
private static boolean numeric(String carrier){return carrier.equals("i32")||carrier.equals("i64")||carrier.equals("f64")||carrier.equals("color");}
|
||||
private static Fact carrier(String name){return new Fact(name,null,0,1);}
|
||||
private static List<Fact> pop(State s,int n){check(n>=0&&n<=s.stack.size(),"temporary stack underflow");var result=new ArrayList<>(s.stack.subList(s.stack.size()-n,s.stack.size()));s.stack.subList(s.stack.size()-n,s.stack.size()).clear();return result;}
|
||||
private static void consume(State s,List<Fact> expected,boolean typed){compatible(pop(s,expected.size()),expected,typed);}
|
||||
private static Fact stamp(Fact fact,long origin) {
|
||||
return new Fact(fact.carrier(),fact.type(),fact.component(),fact.width(),origin,fact.integer());
|
||||
}
|
||||
private static Fact integer(long value){return new Fact("i64",null,0,1,null,value);}
|
||||
private static void produce(State state,List<Fact> values,int pc) {
|
||||
for(int n=0;n<values.size();n++)state.stack.add(stamp(values.get(n),((long)pc<<32)|(n-values.get(n).component())));
|
||||
}
|
||||
private static int mask(String carrier) {
|
||||
int result=0;
|
||||
for(String part:carrier.split("\\|"))result|=switch(part) {
|
||||
case "i32"->1;case "i64","color"->2;case "f64"->4;case "bool"->8;case "string"->16|64;case "ref"->32|64;case "null"->64;default->127;
|
||||
};
|
||||
return result;
|
||||
}
|
||||
private static void dynamic(List<Fact> values,boolean typed) {
|
||||
if(!typed)return;
|
||||
check((mask(values.get(0).carrier())&(2|64))!=0,"Dynamic tag carrier");
|
||||
if(values.get(0).equals(Fact.NULL))check((mask(values.get(1).carrier())&64)!=0,"invalid null Dynamic pair");
|
||||
}
|
||||
private static void compatible(List<Fact> actual,List<Fact> expected,boolean typed) {
|
||||
if(!typed)return;
|
||||
check(actual.size()==expected.size(),"group height");
|
||||
for(int i=0;i<actual.size();i++) {
|
||||
var a=actual.get(i);var e=expected.get(i);
|
||||
if(e.component()==0&&e.type()!=null&&a.type()!=null&&a.origin()!=null&&a.component()==0&&a.width()==e.width()) {
|
||||
boolean complete=true;
|
||||
for(int n=0;n<e.width();n++) {
|
||||
var member=actual.get(i+n);
|
||||
complete&=Objects.equals(member.type(),a.type())&&Objects.equals(member.origin(),a.origin())&&member.component()==n;
|
||||
}
|
||||
// A full proven semantic group cannot silently become another type.
|
||||
if(complete) {
|
||||
if(a.type()==-1)check(e.carrier().equals("i64|null")&&expected.get(i+1).carrier().equals("?"),"Dynamic semantic group mismatch");
|
||||
else check(a.type().equals(e.type()),"proven semantic group mismatch");
|
||||
}
|
||||
}
|
||||
check((mask(a.carrier())&mask(e.carrier()))!=0,"proven carrier mismatch");
|
||||
if(e.carrier().equals("color")&&a.integer()!=null)check(a.integer()>=0&&a.integer()<=0xffffffffL,"color range");
|
||||
if(e.component()==0&&e.width()==2&&e.carrier().contains("|" )&&a.equals(Fact.NULL))
|
||||
check((mask(actual.get(i+1).carrier())&64)!=0,"invalid null group pair");
|
||||
}
|
||||
}
|
||||
|
||||
private int sizeForOpcode(
|
||||
final int opcode,
|
||||
final int pc) {
|
||||
try {
|
||||
return BytecodeOpcodeLayout.sizeForOpcode(opcode);
|
||||
} catch (BytecodeMarshalingException ignored) {
|
||||
throw new BytecodeMarshalingException(
|
||||
BytecodeMarshalingErrorCode.MARSHAL_VERIFY_PRECHECK_UNKNOWN_OPCODE,
|
||||
"unknown opcode 0x%04X at pc=%d".formatted(opcode, pc));
|
||||
}
|
||||
}
|
||||
|
||||
private int readU16(final byte[] code, final int offset) {
|
||||
return ByteBuffer.wrap(code, offset, 2).order(ByteOrder.LITTLE_ENDIAN).getShort() & 0xFFFF;
|
||||
}
|
||||
private static TypeDeclaration type(LinkedRegistry r,int id){check(id>=0&&id<r.types().size(),"TypeId");return r.types().get(id);}
|
||||
private static Fn fn(LinkedRegistry r,int id){check(type(r,id).body() instanceof Fn,"Fn category");return (Fn)type(r,id).body();}
|
||||
private static Method method(LinkedRegistry r,int id,int member){check(type(r,id).body() instanceof Trait,"Trait category");var t=(Trait)type(r,id).body();check(member>=0&&member<t.methods().size(),"method index");return t.methods().get(member);}
|
||||
private static NamedType field(LinkedRegistry r,int id,int member){check(type(r,id).body() instanceof Struct,"Struct category");var t=(Struct)type(r,id).body();check(member>=0&&member<t.fields().size(),"field index");return t.fields().get(member);}
|
||||
private static Array array(LinkedRegistry r,int id){check(type(r,id).body() instanceof Array,"Array category");return (Array)type(r,id).body();}
|
||||
private static int u16(byte[] code,int offset){return ByteBuffer.wrap(code,offset,2).order(ByteOrder.LITTLE_ENDIAN).getShort()&65535;}
|
||||
private static int i32(byte[] code,int offset){return ByteBuffer.wrap(code,offset,4).order(ByteOrder.LITTLE_ENDIAN).getInt();}
|
||||
private static void check(boolean ok,String message){if(!ok)throw new IllegalArgumentException(message);}
|
||||
}
|
||||
|
||||
+154
@@ -0,0 +1,154 @@
|
||||
package p.studio.compiler.backend.bytecode;
|
||||
|
||||
import p.studio.compiler.backend.registry.LinkedRegistry;
|
||||
import p.studio.compiler.models.registry.IRRegistry.*;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.nio.*;
|
||||
import java.nio.charset.*;
|
||||
import java.util.*;
|
||||
|
||||
/** Current PBX v0 field encoding. Never serialize a native layout or infer a semantic type. */
|
||||
final class PbxRegistryWriter {
|
||||
private final BytecodeModule module;
|
||||
private final LinkedRegistry registry;
|
||||
private final Set<Role> roles = EnumSet.noneOf(Role.class);
|
||||
private long nested, names;
|
||||
private PbxRegistryWriter(BytecodeModule module) {
|
||||
this.module=module;
|
||||
this.registry=LinkedRegistry.link(module.registry());
|
||||
}
|
||||
static byte[] write(BytecodeModule module) {
|
||||
try { return new PbxRegistryWriter(module).write(); }
|
||||
catch (IllegalArgumentException | ArithmeticException error) {
|
||||
throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_REGISTRY_INVALID, error.getMessage());
|
||||
}
|
||||
}
|
||||
private byte[] write() {
|
||||
require(module.version()==0,"PBX version must be zero");
|
||||
require(module.code().length<=32*1024*1024,"Code limit");
|
||||
require(module.functions().size()<=65_536,"function limit");
|
||||
require(module.registry().implementations().size()<=262_144,"implementation limit");
|
||||
var sections=new TreeMap<Integer,byte[]>();
|
||||
sections.put(0,constants()); sections.put(1,functions()); sections.put(2,module.code());
|
||||
if(module.debugInfo()!=null) sections.put(3,debug());
|
||||
sections.put(4,exports()); sections.put(5,syscalls()); sections.put(6,types());
|
||||
sections.put(7,implementations()); sections.put(8,captures());
|
||||
require(nested<=1_048_576,"nested record limit"); require(names<=4*1024*1024,"Registry name limit");
|
||||
long total=32+12L*sections.size(); for(var bytes:sections.values()) total+=bytes.length;
|
||||
require(total<=64*1024*1024,"PBX file limit");
|
||||
var out=new Bytes();out.raw(new byte[]{'P','B','X',0});out.u16(0);out.u8(0);out.u8(0);out.u32(sections.size());out.raw(new byte[20]);
|
||||
int offset=32+12*sections.size();
|
||||
for(var section:sections.entrySet()) {out.u32(section.getKey());out.u32(offset);out.u32(section.getValue().length);offset+=section.getValue().length;}
|
||||
sections.values().forEach(out::raw);return out.bytes();
|
||||
}
|
||||
private void optionalName(Bytes out,String name) {
|
||||
names+=LinkedRegistry.nameBytes(name);out.u8(name==null?0:1);if(name!=null)out.text32(name);
|
||||
}
|
||||
private void parameters(Bytes out,ReadOnlyList<NamedType> values) {
|
||||
nested+=values.size();out.u32(values.size());for(var p:values) {out.u32(registry.id(p.type()).index());optionalName(out,p.name());}
|
||||
}
|
||||
private void results(Bytes out,ReadOnlyList<TypeRef> values) {
|
||||
nested+=values.size();out.u32(values.size());for(var r:values)out.u32(registry.id(r).index());
|
||||
}
|
||||
private void signature(Bytes out,Signature signature) {
|
||||
require(registry.parameterSlots(signature)<=16_384,"signature parameter limit");
|
||||
require(registry.resultSlots(signature)<=1024,"signature result limit");
|
||||
parameters(out,signature.parameters());results(out,signature.results());
|
||||
}
|
||||
private byte[] types() {
|
||||
var out=new Bytes();out.u32(registry.types().size());
|
||||
for(var t:registry.types()) {
|
||||
int tag=switch(t.body()){case Builtin b->0;case Struct s->1;case Array a->2;case Trait t1->3;case Fn f->4;};
|
||||
out.u8(tag);optionalName(out,t.name());
|
||||
switch(t.body()) {
|
||||
case Builtin b -> out.u32(b.id());
|
||||
case Struct s -> parameters(out,s.fields());
|
||||
case Array a -> out.u32(registry.id(a.element()).index());
|
||||
case Fn f -> signature(out,f.signature());
|
||||
case Trait t1 -> {nested+=t1.methods().size();out.u32(t1.methods().size());for(var m:t1.methods()){optionalName(out,m.name());signature(out,m.signature());}}
|
||||
}
|
||||
}
|
||||
return out.bytes();
|
||||
}
|
||||
private byte[] functions() {
|
||||
var out=new Bytes();out.u32(module.functions().size());long previousEnd=0;
|
||||
for(int i=0;i<module.functions().size();i++) {
|
||||
var f=module.functions().get(i);var m=f.metadata();require(m!=null,"typed function metadata required");
|
||||
int p=registry.parameterSlots(m.signature()),r=registry.resultSlots(m.signature());
|
||||
require(p==f.paramSlots()&&r==f.returnSlots(),"slot cache differs from typed signature");
|
||||
require(f.localSlots()>=0&&f.maxStackSlots()>=0&&(long)p+f.localSlots()+f.maxStackSlots()<=16_384&&r<=1024,"frame/result limit");
|
||||
long end=(long)f.codeOffset()+f.codeLen();
|
||||
require(f.codeOffset()>=previousEnd&&f.codeLen()>0&&end<=module.code().length,"invalid function Code range");previousEnd=end;
|
||||
if(i==0)require(m.captures().isEmpty(),"boot function must have no captures");
|
||||
if(m.role()!=Role.NONE) {
|
||||
require(roles.add(m.role()),"duplicate lifecycle role");require(i!=0&&m.captures().isEmpty(),"invalid lifecycle target");
|
||||
if(m.role()==Role.MEMCARD_BOOT) require(m.signature().parameters().size()==1&&m.signature().results().size()==1
|
||||
&&isBuiltin(m.signature().parameters().getFirst().type(),0x11)&&isBuiltin(m.signature().results().getFirst(),0x10),"MemcardBoot signature");
|
||||
else require(p==0&&r==0,"pause/resume signature");
|
||||
}
|
||||
out.u8(m.role().wire());optionalName(out,m.name());signature(out,m.signature());
|
||||
out.u32(f.codeOffset());out.u32(f.codeLen());out.u32(f.localSlots());out.u32(f.maxStackSlots());
|
||||
}
|
||||
return out.bytes();
|
||||
}
|
||||
private boolean isBuiltin(TypeRef type,int expected) {return registry.type(type).body() instanceof Builtin b&&b.id()==expected;}
|
||||
private byte[] captures(){var out=new Bytes();out.u32(module.functions().size());for(var f:module.functions())results(out,f.metadata().captures());return out.bytes();}
|
||||
private byte[] implementations() {
|
||||
var out=new Bytes();var declarations=new TreeMap<String,Implementation>();
|
||||
for(var impl:module.registry().implementations()) {
|
||||
var old=declarations.putIfAbsent(impl.declarationKey(),impl);require(old==null||old.equals(impl),"conflicting implementation declaration");
|
||||
}
|
||||
out.u32(declarations.size());var pairs=new HashSet<List<Integer>>();
|
||||
for(var impl:declarations.values()) {
|
||||
require(registry.type(impl.owner()).body() instanceof Struct,"implementation owner must be Struct");
|
||||
require(registry.type(impl.trait()).body() instanceof Trait,"implementation trait category");
|
||||
var trait=(Trait)registry.type(impl.trait()).body();int owner=registry.id(impl.owner()).index(),traitId=registry.id(impl.trait()).index();
|
||||
require(pairs.add(List.of(owner,traitId)),"duplicate implementation pair");
|
||||
require(trait.methods().size()==impl.methods().size(),"implementation method count");
|
||||
out.u32(owner);out.u32(traitId);out.u32(impl.methods().size());nested+=impl.methods().size();
|
||||
for(int i=0;i<impl.methods().size();i++) {
|
||||
int function=impl.methods().get(i).getIndex();require(function<module.functions().size(),"implementation FunctionId");
|
||||
var metadata=module.functions().get(function).metadata();var actual=metadata.signature();var expected=trait.methods().get(i).signature();
|
||||
require(metadata.captures().isEmpty()&&!actual.parameters().isEmpty()&®istry.id(actual.parameters().getFirst().type()).index()==owner,"implementation receiver/captures");
|
||||
require(registry.sameSignature(new Signature(actual.parameters().immutableSubList(1,actual.parameters().size()),actual.results()),expected),"implementation signature mismatch");out.u32(function);
|
||||
}
|
||||
}
|
||||
return out.bytes();
|
||||
}
|
||||
private byte[] constants(){
|
||||
var out=new Bytes();require(module.constPool().size()<=1_048_576,"constant count");out.u32(module.constPool().size());long strings=0;
|
||||
for(var c:module.constPool())switch(c){
|
||||
case BytecodeModule.NullConstant n->out.u8(0);
|
||||
case BytecodeModule.Int64Constant n->{out.u8(1);out.i64(n.value());}
|
||||
case BytecodeModule.Float64Constant n->{require(Double.isFinite(n.value()),"nonfinite Float constant");out.u8(2);out.i64(Double.doubleToRawLongBits(n.value()));}
|
||||
case BytecodeModule.BooleanConstant n->{out.u8(3);out.u8(n.value()?1:0);}
|
||||
case BytecodeModule.StringConstant n->{var bytes=Bytes.utf8(n.value());require(bytes.length<=4*1024*1024,"individual string limit");strings+=bytes.length;require(strings<=64*1024*1024,"constant string limit");out.u8(4);out.u32(bytes.length);out.raw(bytes);}
|
||||
case BytecodeModule.Int32Constant n->{out.u8(5);out.i32(n.value());}
|
||||
}return out.bytes();
|
||||
}
|
||||
private byte[] exports(){var out=new Bytes();out.u32(module.exports().size());for(var e:module.exports()){require(e.funcIdx()>=0&&e.funcIdx()<module.functions().size(),"export FunctionId");out.u32(e.funcIdx());out.text32(e.symbol());}return out.bytes();}
|
||||
private byte[] debug(){
|
||||
var out=new Bytes();var d=module.debugInfo();out.u32(d.pcToSpan().size());
|
||||
for(var p:d.pcToSpan()){require(p.pc()>=0&&p.pc()<module.code().length&&p.span().start()>=0&&p.span().end()>=p.span().start(),"debug span range");out.u32(p.pc());out.i32(p.span().fileId());out.u32(p.span().start());out.u32(p.span().end());}
|
||||
out.u32(d.functionNames().size());for(var n:d.functionNames()){require(n.funcIdx()>=0&&n.funcIdx()<module.functions().size(),"debug FunctionId");out.u32(n.funcIdx());out.text32(n.name());}return out.bytes();
|
||||
}
|
||||
private byte[] syscalls(){var out=new Bytes();var identities=new HashSet<List<Object>>();out.u32(module.syscalls().size());
|
||||
for(var s:module.syscalls()){require(identities.add(List.of(s.module(),s.name(),s.version())),"duplicate syscall declaration");if(Bytes.utf8(s.module()).length>65535)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_MODULE_TOO_LONG,"syscall module name exceeds u16");
|
||||
if(Bytes.utf8(s.name()).length>65535)throw new BytecodeMarshalingException(BytecodeMarshalingErrorCode.MARSHAL_FORMAT_SYSC_NAME_TOO_LONG,"syscall name exceeds u16");
|
||||
out.text16(s.module());out.text16(s.name());out.u16(s.version());out.u16(s.argSlots());out.u16(s.retSlots());}return out.bytes();}
|
||||
static void require(boolean condition,String message){if(!condition)throw new IllegalArgumentException(message);}
|
||||
private static final class Bytes {
|
||||
final ByteArrayOutputStream out=new ByteArrayOutputStream();
|
||||
void u8(int v){require(v>=0&&v<=255,"u8 range");out.write(v);}
|
||||
void u16(int v){require(v>=0&&v<=65535,"u16 range");out.write(v);out.write(v>>>8);}
|
||||
void u32(int v){require(v>=0,"u32 range/product limit");i32(v);}
|
||||
void i32(int v){for(int i=0;i<4;i++)out.write(v>>>(8*i));}
|
||||
void i64(long v){for(int i=0;i<8;i++)out.write((int)(v>>>(8*i)));}
|
||||
void raw(byte[] v){out.writeBytes(v);}
|
||||
void text32(String v){var b=utf8(v);u32(b.length);raw(b);}
|
||||
void text16(String v){var b=utf8(v);u16(b.length);raw(b);}
|
||||
byte[] bytes(){return out.toByteArray();}
|
||||
static byte[] utf8(String text){try{var b=StandardCharsets.UTF_8.newEncoder().encode(CharBuffer.wrap(text));var bytes=new byte[b.remaining()];b.get(bytes);return bytes;}catch(CharacterCodingException e){throw new IllegalArgumentException("invalid UTF-8 source",e);}}
|
||||
}
|
||||
}
|
||||
+11
@@ -5,12 +5,23 @@ import p.studio.utilities.structures.ReadOnlyList;
|
||||
import java.util.Objects;
|
||||
|
||||
public record IRVMFunction(
|
||||
String name,
|
||||
int paramSlots,
|
||||
int localSlots,
|
||||
int returnSlots,
|
||||
int maxStackSlots,
|
||||
ReadOnlyList<IRVMInstruction> instructions,
|
||||
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
|
||||
public IRVMFunction(
|
||||
String name,
|
||||
int paramSlots,
|
||||
int localSlots,
|
||||
int returnSlots,
|
||||
int maxStackSlots,
|
||||
ReadOnlyList<IRVMInstruction> instructions) {
|
||||
this(name, paramSlots, localSlots, returnSlots, maxStackSlots, instructions, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
|
||||
}
|
||||
|
||||
public IRVMFunction {
|
||||
Objects.requireNonNull(name, "name");
|
||||
instructions = instructions == null ? ReadOnlyList.empty() : instructions;
|
||||
|
||||
+13
@@ -3,10 +3,23 @@ package p.studio.compiler.backend.irvm;
|
||||
import java.util.Objects;
|
||||
|
||||
public record IRVMInstruction(
|
||||
IRVMOp op,
|
||||
Integer immediate,
|
||||
Integer secondImmediate) {
|
||||
public IRVMInstruction(
|
||||
IRVMOp op,
|
||||
Integer immediate) {
|
||||
this(op, immediate, null);
|
||||
}
|
||||
|
||||
public IRVMInstruction {
|
||||
Objects.requireNonNull(op, "op");
|
||||
if (op.name().startsWith("REGISTRY_") && ((op.immediateSize() == 8) != (secondImmediate != null))) {
|
||||
throw new IllegalArgumentException("registry operand count differs from opcode layout");
|
||||
}
|
||||
if (!op.name().startsWith("REGISTRY_") && secondImmediate != null) {
|
||||
throw new IllegalArgumentException("unexpected second immediate");
|
||||
}
|
||||
if (op.immediateSize() > 0 && immediate == null) {
|
||||
throw new IllegalArgumentException("immediate is required for opcode: " + op.name());
|
||||
}
|
||||
|
||||
+7
@@ -3,8 +3,15 @@ package p.studio.compiler.backend.irvm;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
|
||||
public record IRVMModule(
|
||||
String vmProfile,
|
||||
ReadOnlyList<IRVMFunction> functions,
|
||||
p.studio.compiler.models.registry.IRRegistry registry) {
|
||||
public IRVMModule(
|
||||
String vmProfile,
|
||||
ReadOnlyList<IRVMFunction> functions) {
|
||||
this(vmProfile, functions, p.studio.compiler.models.registry.IRRegistry.empty());
|
||||
}
|
||||
|
||||
public IRVMModule {
|
||||
vmProfile = vmProfile == null || vmProfile.isBlank() ? "core-v1" : vmProfile;
|
||||
functions = functions == null ? ReadOnlyList.empty() : functions;
|
||||
|
||||
+2
-1
@@ -91,6 +91,7 @@ public final class IRVMProfileFeatureGate {
|
||||
final String profile,
|
||||
final IRVMOp op) {
|
||||
final var allowed = allowedByProfile.get(normalizeProfile(profile));
|
||||
return allowed != null && allowed.contains(op);
|
||||
return allowed != null && (allowed.contains(op) ||
|
||||
op.name().startsWith("REGISTRY_") && java.util.Arrays.stream(p.studio.compiler.models.registry.IRRegistryInstruction.Kind.values()).anyMatch(k -> k.opcode() == op.opcode()));
|
||||
}
|
||||
}
|
||||
|
||||
+12
-5
@@ -54,13 +54,17 @@ public record IRVMProgram(
|
||||
private void validateCoherence(
|
||||
final IRVMModule module,
|
||||
final BytecodeEmitter.EmissionPlan emissionPlan) {
|
||||
if (!module.registry().equals(emissionPlan.registry())) {
|
||||
throw new IllegalArgumentException("irvm/emission plan registry mismatch");
|
||||
}
|
||||
if (module.functions().size() != emissionPlan.functions().size()) {
|
||||
throw new IllegalArgumentException("irvm module/function-plan size mismatch");
|
||||
}
|
||||
for (var fnIndex = 0; fnIndex < module.functions().size(); fnIndex++) {
|
||||
final var irvmFunction = module.functions().get(fnIndex);
|
||||
final var planFunction = emissionPlan.functions().get(fnIndex);
|
||||
if (!irvmFunction.name().equals(planFunction.name())
|
||||
if (!java.util.Objects.equals(irvmFunction.metadata(), planFunction.metadata())
|
||||
|| !irvmFunction.name().equals(planFunction.name())
|
||||
|| irvmFunction.paramSlots() != planFunction.paramSlots()
|
||||
|| irvmFunction.localSlots() != planFunction.localSlots()
|
||||
|| irvmFunction.returnSlots() != planFunction.returnSlots()
|
||||
@@ -84,6 +88,7 @@ public record IRVMProgram(
|
||||
private boolean coherentPair(
|
||||
final IRVMInstruction instruction,
|
||||
final BytecodeEmitter.Operation operation) {
|
||||
if (operation.kind() == BytecodeEmitter.OperationKind.REGISTRY) return instruction.equals(operation.registryInstruction());
|
||||
if (instruction.op().internal()) {
|
||||
return true;
|
||||
}
|
||||
@@ -137,13 +142,13 @@ public record IRVMProgram(
|
||||
function.localSlots(),
|
||||
function.returnSlots(),
|
||||
function.maxStackSlots(),
|
||||
ReadOnlyList.wrap(operations)));
|
||||
ReadOnlyList.wrap(operations), function.metadata()));
|
||||
}
|
||||
return new BytecodeEmitter.EmissionPlan(
|
||||
0,
|
||||
ReadOnlyList.empty(),
|
||||
ReadOnlyList.empty(),
|
||||
ReadOnlyList.wrap(functions));
|
||||
ReadOnlyList.wrap(functions), module.registry());
|
||||
}
|
||||
|
||||
private boolean containsRawSyscall(final BytecodeEmitter.EmissionPlan plan) {
|
||||
@@ -171,6 +176,7 @@ public record IRVMProgram(
|
||||
}
|
||||
|
||||
private BytecodeEmitter.Operation deriveOperation(final IRVMInstruction instruction) {
|
||||
if (instruction.op().name().startsWith("REGISTRY_")) return new BytecodeEmitter.Operation(BytecodeEmitter.OperationKind.REGISTRY, 0, null, null, null, null, instruction);
|
||||
final var immediate = instruction.immediate() == null ? 0 : instruction.immediate();
|
||||
return switch (instruction.op().opcode()) {
|
||||
case 0x01 -> BytecodeEmitter.Operation.halt();
|
||||
@@ -240,13 +246,14 @@ public record IRVMProgram(
|
||||
functionPlan.localSlots(),
|
||||
functionPlan.returnSlots(),
|
||||
functionPlan.maxStackSlots(),
|
||||
ReadOnlyList.wrap(instructions)));
|
||||
ReadOnlyList.wrap(instructions), functionPlan.metadata()));
|
||||
}
|
||||
return new IRVMModule("core-v1", ReadOnlyList.wrap(functions));
|
||||
return new IRVMModule("core-v1", ReadOnlyList.wrap(functions), inputPlan.registry());
|
||||
}
|
||||
|
||||
private static IRVMInstruction operationToInstruction(final BytecodeEmitter.Operation operation) {
|
||||
return switch (operation.kind()) {
|
||||
case REGISTRY -> operation.registryInstruction();
|
||||
case HALT -> new IRVMInstruction(IRVMOp.HALT, null);
|
||||
case RET -> new IRVMInstruction(IRVMOp.RET, null);
|
||||
case PUSH_CONST -> new IRVMInstruction(IRVMOp.PUSH_CONST, operation.immediate());
|
||||
|
||||
+34
-12
@@ -41,10 +41,12 @@ public class LowerToIRVMService {
|
||||
}
|
||||
|
||||
validateCommonLifecycleStructure(backend);
|
||||
final var registry = p.studio.compiler.backend.registry.LinkedRegistry.link(backend.getRegistry());
|
||||
final var ordered = orderFunctions(backend);
|
||||
final var funcIdByCallableId = new HashMap<CallableId, Integer>();
|
||||
for (var i = 0; i < ordered.size(); i++) {
|
||||
final var fn = ordered.get(i);
|
||||
if (fn.metadata() == null) throw new IllegalArgumentException("typed function metadata required: " + fn.callableName());
|
||||
if (funcIdByCallableId.putIfAbsent(fn.callableId(), i) != null) {
|
||||
throw loweringError(
|
||||
fn,
|
||||
@@ -60,6 +62,9 @@ public class LowerToIRVMService {
|
||||
|
||||
for (var i = 0; i < ordered.size(); i++) {
|
||||
final var fn = ordered.get(i);
|
||||
if (fn.metadata() == null) throw new IllegalArgumentException("typed function metadata required: " + fn.callableName());
|
||||
final var parameterSlots = registry.parameterSlots(fn.metadata().signature());
|
||||
final var resultSlots = registry.resultSlots(fn.metadata().signature());
|
||||
final var instructions = new ArrayList<IRVMInstruction>(fn.instructions().size());
|
||||
final var operations = new ArrayList<BytecodeEmitter.Operation>(fn.instructions().size());
|
||||
final var labelToPc = new HashMap<String, Integer>();
|
||||
@@ -68,6 +73,12 @@ public class LowerToIRVMService {
|
||||
for (final var instr : fn.instructions()) {
|
||||
final var sourceSpan = toBytecodeSpan(fn.fileId().getId(), instr.span());
|
||||
switch (instr.kind()) {
|
||||
case REGISTRY -> {
|
||||
var typed = p.studio.compiler.backend.registry.RegistryInstructionLowerer.lower(instr.registryInstruction(), registry, fn, ordered, funcIdByCallableId);
|
||||
instructions.add(typed);
|
||||
operations.add(new BytecodeEmitter.Operation(BytecodeEmitter.OperationKind.REGISTRY, 0, null, null, null, sourceSpan, typed));
|
||||
functionPc += typed.encodedSize();
|
||||
}
|
||||
case PUSH_I32 -> {
|
||||
final int value;
|
||||
try {
|
||||
@@ -271,7 +282,7 @@ public class LowerToIRVMService {
|
||||
}
|
||||
final var calleeFunction = ordered.get(calleeId);
|
||||
if (instr.expectedArgSlots() != null
|
||||
&& instr.expectedArgSlots() != calleeFunction.paramSlots()) {
|
||||
&& instr.expectedArgSlots() != registry.parameterSlots(calleeFunction.metadata().signature())) {
|
||||
throw loweringError(
|
||||
fn,
|
||||
instr,
|
||||
@@ -279,10 +290,10 @@ public class LowerToIRVMService {
|
||||
"call arg_slots mismatch for %s. expected=%d actual=%d".formatted(
|
||||
instr.calleeCallableName(),
|
||||
instr.expectedArgSlots(),
|
||||
calleeFunction.paramSlots()));
|
||||
registry.parameterSlots(calleeFunction.metadata().signature())));
|
||||
}
|
||||
if (instr.expectedRetSlots() != null
|
||||
&& instr.expectedRetSlots() != calleeFunction.returnSlots()) {
|
||||
&& instr.expectedRetSlots() != registry.resultSlots(calleeFunction.metadata().signature())) {
|
||||
throw loweringError(
|
||||
fn,
|
||||
instr,
|
||||
@@ -290,7 +301,7 @@ public class LowerToIRVMService {
|
||||
"call ret_slots mismatch for %s. expected=%d actual=%d".formatted(
|
||||
instr.calleeCallableName(),
|
||||
instr.expectedRetSlots(),
|
||||
calleeFunction.returnSlots()));
|
||||
registry.resultSlots(calleeFunction.metadata().signature())));
|
||||
}
|
||||
instructions.add(new IRVMInstruction(IRVMOp.CALL, calleeId));
|
||||
operations.add(BytecodeEmitter.Operation.callFunc(calleeId, sourceSpan));
|
||||
@@ -404,27 +415,38 @@ public class LowerToIRVMService {
|
||||
}
|
||||
loweredFunctions.add(new IRVMFunction(
|
||||
fn.callableName(),
|
||||
fn.paramSlots(),
|
||||
parameterSlots,
|
||||
fn.localSlots(),
|
||||
fn.returnSlots(),
|
||||
resultSlots,
|
||||
fn.maxStackSlots(),
|
||||
ReadOnlyList.wrap(instructions)));
|
||||
ReadOnlyList.wrap(instructions), fn.metadata()));
|
||||
emissionFunctions.add(new BytecodeEmitter.FunctionPlan(
|
||||
fn.callableName(),
|
||||
fn.paramSlots(),
|
||||
parameterSlots,
|
||||
fn.localSlots(),
|
||||
fn.returnSlots(),
|
||||
resultSlots,
|
||||
fn.maxStackSlots(),
|
||||
ReadOnlyList.wrap(operations)));
|
||||
ReadOnlyList.wrap(operations), fn.metadata()));
|
||||
}
|
||||
|
||||
final var imageImplementations = new ArrayList<p.studio.compiler.models.registry.IRRegistry.Implementation>();
|
||||
for (var impl : backend.getRegistry().implementations()) {
|
||||
var methods = new ArrayList<CallableId>();
|
||||
for (var method : impl.methods()) {
|
||||
var target = funcIdByCallableId.get(method);
|
||||
if (target == null) throw new IllegalArgumentException("undefined implementation function: " + method);
|
||||
methods.add(new CallableId(target));
|
||||
}
|
||||
imageImplementations.add(new p.studio.compiler.models.registry.IRRegistry.Implementation(impl.declarationKey(), impl.owner(), impl.trait(), ReadOnlyList.wrap(methods)));
|
||||
}
|
||||
var imageRegistry = new p.studio.compiler.models.registry.IRRegistry(backend.getRegistry().types(), ReadOnlyList.wrap(imageImplementations));
|
||||
final var program = new IRVMProgram(
|
||||
new IRVMModule(vmProfile, ReadOnlyList.wrap(loweredFunctions)),
|
||||
new IRVMModule(vmProfile, ReadOnlyList.wrap(loweredFunctions), imageRegistry),
|
||||
new BytecodeEmitter.EmissionPlan(
|
||||
0,
|
||||
ReadOnlyList.wrap(constPool),
|
||||
ReadOnlyList.empty(),
|
||||
ReadOnlyList.wrap(emissionFunctions)));
|
||||
ReadOnlyList.wrap(emissionFunctions), imageRegistry));
|
||||
validator.validate(program, false);
|
||||
return program;
|
||||
}
|
||||
|
||||
+6
-6
@@ -124,12 +124,12 @@ public class OptimizeIRVMService {
|
||||
}
|
||||
|
||||
return new IRVMProgram(
|
||||
new IRVMModule(input.module().vmProfile(), ReadOnlyList.wrap(rewrittenFunctions)),
|
||||
new IRVMModule(input.module().vmProfile(), ReadOnlyList.wrap(rewrittenFunctions), input.module().registry()),
|
||||
new BytecodeEmitter.EmissionPlan(
|
||||
input.emissionPlan().version(),
|
||||
input.emissionPlan().constPool(),
|
||||
input.emissionPlan().exports(),
|
||||
ReadOnlyList.wrap(rewrittenPlans)));
|
||||
ReadOnlyList.wrap(rewrittenPlans), input.emissionPlan().registry()));
|
||||
}
|
||||
|
||||
private static boolean isJump(final IRVMOp op) {
|
||||
@@ -218,14 +218,14 @@ public class OptimizeIRVMService {
|
||||
function.localSlots(),
|
||||
function.returnSlots(),
|
||||
function.maxStackSlots(),
|
||||
ReadOnlyList.wrap(newInstructions));
|
||||
ReadOnlyList.wrap(newInstructions), function.metadata());
|
||||
final var rewrittenPlan = new BytecodeEmitter.FunctionPlan(
|
||||
functionPlan.name(),
|
||||
functionPlan.paramSlots(),
|
||||
functionPlan.localSlots(),
|
||||
functionPlan.returnSlots(),
|
||||
functionPlan.maxStackSlots(),
|
||||
ReadOnlyList.wrap(newOperations));
|
||||
ReadOnlyList.wrap(newOperations), functionPlan.metadata());
|
||||
return new FunctionRewrite(rewrittenFunction, rewrittenPlan, changed);
|
||||
}
|
||||
|
||||
@@ -383,14 +383,14 @@ public class OptimizeIRVMService {
|
||||
function.localSlots(),
|
||||
function.returnSlots(),
|
||||
function.maxStackSlots(),
|
||||
ReadOnlyList.wrap(instructions)),
|
||||
ReadOnlyList.wrap(instructions), function.metadata()),
|
||||
new BytecodeEmitter.FunctionPlan(
|
||||
functionPlan.name(),
|
||||
functionPlan.paramSlots(),
|
||||
functionPlan.localSlots(),
|
||||
functionPlan.returnSlots(),
|
||||
functionPlan.maxStackSlots(),
|
||||
ReadOnlyList.wrap(operations)),
|
||||
ReadOnlyList.wrap(operations), functionPlan.metadata()),
|
||||
true);
|
||||
}
|
||||
|
||||
|
||||
+130
@@ -0,0 +1,130 @@
|
||||
package p.studio.compiler.backend.registry;
|
||||
|
||||
import p.studio.compiler.models.registry.IRRegistry;
|
||||
import p.studio.compiler.models.registry.IRRegistry.*;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
import java.util.*;
|
||||
|
||||
/** Immutable image type table and reference resolution. No language-specific type inference. */
|
||||
public final class LinkedRegistry {
|
||||
public record TypeId(int index) { public TypeId { if (index < 0) throw new IllegalArgumentException("negative TypeId"); } }
|
||||
public record FunctionId(int index) { public FunctionId { if (index < 0) throw new IllegalArgumentException("negative FunctionId"); } }
|
||||
public record ImplementationId(int index) { public ImplementationId { if (index < 0) throw new IllegalArgumentException("negative ImplementationId"); } }
|
||||
private final ReadOnlyList<TypeDeclaration> types;
|
||||
private final Map<TypeRef, TypeId> references;
|
||||
|
||||
private LinkedRegistry(List<TypeDeclaration> types, Map<TypeRef, TypeId> references) {
|
||||
this.types = ReadOnlyList.wrap(types);
|
||||
this.references = Map.copyOf(references);
|
||||
}
|
||||
public ReadOnlyList<TypeDeclaration> types() { return types; }
|
||||
public TypeId id(TypeRef reference) {
|
||||
final var id = references.get(reference);
|
||||
if (id == null) throw new IllegalArgumentException("undefined type declaration: " + reference);
|
||||
return id;
|
||||
}
|
||||
public TypeDeclaration type(TypeRef reference) { return types.get(id(reference).index()); }
|
||||
public int width(TypeRef reference) {
|
||||
return switch (type(reference).body()) {
|
||||
case Builtin b -> switch (b.id()) { case 2, 0x15 -> 2; case 3 -> 3; default -> 1; };
|
||||
case Trait ignored -> 2;
|
||||
default -> 1;
|
||||
};
|
||||
}
|
||||
public int parameterSlots(Signature signature) {
|
||||
var slots = 0;
|
||||
for (var p : signature.parameters()) slots = Math.addExact(slots, width(p.type()));
|
||||
return slots;
|
||||
}
|
||||
public int resultSlots(Signature signature) {
|
||||
var slots = 0;
|
||||
for (var r : signature.results()) slots = Math.addExact(slots, width(r));
|
||||
return slots;
|
||||
}
|
||||
public boolean sameSignature(Signature a, Signature b) {
|
||||
if (a.parameters().size() != b.parameters().size() || a.results().size() != b.results().size()) return false;
|
||||
for (int i=0; i<a.parameters().size(); i++) if (!id(a.parameters().get(i).type()).equals(id(b.parameters().get(i).type()))) return false;
|
||||
for (int i=0; i<a.results().size(); i++) if (!id(a.results().get(i)).equals(id(b.results().get(i)))) return false;
|
||||
return true;
|
||||
}
|
||||
public static LinkedRegistry link(IRRegistry source) {
|
||||
final var declarations = new TreeMap<String, TypeDeclaration>();
|
||||
for (var declaration : source.types()) {
|
||||
final var old = declarations.putIfAbsent(declaration.identity().declarationKey(), declaration);
|
||||
if (old != null && !old.equals(declaration)) throw new IllegalArgumentException("conflicting type declaration: " + declaration.identity());
|
||||
}
|
||||
if (declarations.size() > 65_536) throw new IllegalArgumentException("type record limit exceeded");
|
||||
final var output = new ArrayList<TypeDeclaration>();
|
||||
final var ids = new HashMap<TypeRef, TypeId>();
|
||||
final var structuralKeys = new HashMap<Object, TypeId>();
|
||||
final var remaining = new TreeMap<String, TypeDeclaration>();
|
||||
long names = 0, nested = 0;
|
||||
for (var d : declarations.values()) {
|
||||
names += nameBytes(d.name());
|
||||
for (var ref : references(d.body())) {
|
||||
if (!declarations.containsKey(ref.declarationKey())) throw new IllegalArgumentException("undefined type: " + ref);
|
||||
}
|
||||
switch (d.body()) {
|
||||
case Builtin b -> assign(d, List.of("builtin", b.id()), structuralKeys, output, ids);
|
||||
case Struct s -> { assign(d, d.identity(), structuralKeys, output, ids); nested += s.fields().size(); for(var f:s.fields()) names += nameBytes(f.name()); }
|
||||
case Trait t -> {
|
||||
assign(d, d.identity(), structuralKeys, output, ids); nested += t.methods().size();
|
||||
for(var m:t.methods()) { names += nameBytes(m.name()); nested += m.signature().parameters().size()+m.signature().results().size(); for(var p:m.signature().parameters()) names += nameBytes(p.name()); }
|
||||
}
|
||||
case Array ignored -> remaining.put(d.identity().declarationKey(), d);
|
||||
case Fn f -> { remaining.put(d.identity().declarationKey(), d); nested += f.signature().parameters().size()+f.signature().results().size(); for(var p:f.signature().parameters()) names += nameBytes(p.name()); }
|
||||
}
|
||||
}
|
||||
if (names > 4L*1024*1024 || nested > 1_048_576) throw new IllegalArgumentException("registry name/nested-record limit exceeded");
|
||||
// Kahn traversal avoids recursive expansion and accepts nominal anchors.
|
||||
final var pending = new HashMap<TypeRef, Integer>();
|
||||
final var dependents = new HashMap<TypeRef, List<TypeRef>>();
|
||||
final var ready = new PriorityQueue<TypeRef>(Comparator.comparing(TypeRef::declarationKey));
|
||||
for(var d:remaining.values()) {
|
||||
int count=0;
|
||||
for(var r:references(d.body())) if(!ids.containsKey(r)) { count++; dependents.computeIfAbsent(r, ignored -> new ArrayList<>()).add(d.identity()); }
|
||||
pending.put(d.identity(),count);
|
||||
if(count==0) ready.add(d.identity());
|
||||
}
|
||||
while(!ready.isEmpty()) {
|
||||
var reference=ready.remove(); var d=remaining.get(reference.declarationKey());
|
||||
Object key;
|
||||
if(d.body() instanceof Array a) key=List.of("array",ids.get(a.element()));
|
||||
else {
|
||||
var signature=((Fn)d.body()).signature();
|
||||
key=List.of("fn",signature.parameters().stream().map(p->ids.get(p.type())).toList(),signature.results().stream().map(ids::get).toList());
|
||||
}
|
||||
assign(d,key,structuralKeys,output,ids);
|
||||
for(var dependent:dependents.getOrDefault(reference,List.of())) if(pending.compute(dependent,(k,v)->v-1)==0) ready.add(dependent);
|
||||
}
|
||||
if(ids.size()!=declarations.size()) throw new IllegalArgumentException("structural Array/Fn cycle");
|
||||
return new LinkedRegistry(output,ids);
|
||||
}
|
||||
private static void assign(TypeDeclaration d,Object key,Map<Object,TypeId> keys,List<TypeDeclaration> output,Map<TypeRef,TypeId> ids) {
|
||||
var id=keys.get(key);
|
||||
if(id==null) { id=new TypeId(output.size()); output.add(d); keys.put(key,id); }
|
||||
ids.put(d.identity(),id);
|
||||
}
|
||||
private static List<TypeRef> references(TypeBody body) {
|
||||
final var result=new ArrayList<TypeRef>();
|
||||
switch(body) {
|
||||
case Struct s -> s.fields().forEach(f->result.add(f.type()));
|
||||
case Array a -> result.add(a.element());
|
||||
case Fn f -> signatureReferences(f.signature(),result);
|
||||
case Trait t -> t.methods().forEach(m->signatureReferences(m.signature(),result));
|
||||
case Builtin ignored -> { }
|
||||
}
|
||||
return result;
|
||||
}
|
||||
private static void signatureReferences(Signature signature,List<TypeRef> result) {
|
||||
signature.parameters().forEach(p->result.add(p.type())); result.addAll(signature.results().asList());
|
||||
}
|
||||
public static int nameBytes(String name) {
|
||||
if(name==null) return 0;
|
||||
try {
|
||||
int size=java.nio.charset.StandardCharsets.UTF_8.newEncoder().encode(java.nio.CharBuffer.wrap(name)).remaining();
|
||||
if(size>4096) throw new IllegalArgumentException("name exceeds 4096 UTF-8 bytes");
|
||||
return size;
|
||||
} catch(java.nio.charset.CharacterCodingException e) { throw new IllegalArgumentException("invalid UTF-8 source string",e); }
|
||||
}
|
||||
}
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
package p.studio.compiler.backend.registry;
|
||||
|
||||
import p.studio.compiler.models.IRBackendExecutableFunction;
|
||||
import p.studio.compiler.models.registry.IRRegistry.*;
|
||||
import p.studio.compiler.models.registry.IRRegistryInstruction;
|
||||
import p.studio.compiler.backend.irvm.*;
|
||||
import p.studio.compiler.source.identifiers.CallableId;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
import java.util.Map;
|
||||
|
||||
public final class RegistryInstructionLowerer {
|
||||
private RegistryInstructionLowerer() {}
|
||||
public static IRVMInstruction lower(IRRegistryInstruction instruction, LinkedRegistry registry,
|
||||
IRBackendExecutableFunction current, ReadOnlyList<IRBackendExecutableFunction> functions, Map<CallableId,Integer> functionIds) {
|
||||
int first=instruction.type()==null?instruction.member():registry.id(instruction.type()).index();
|
||||
Integer second=null;int pops=0,pushes=0;
|
||||
var body=instruction.type()==null?null:registry.type(instruction.type()).body();
|
||||
switch(instruction.kind()) {
|
||||
case MAKE_CLOSURE -> {
|
||||
if(!(body instanceof Fn type))throw new IllegalArgumentException("closure type must be Fn");
|
||||
var target=functionIds.get(instruction.function());if(target==null)throw new IllegalArgumentException("undefined closure function");
|
||||
var metadata=functions.get(target).metadata();
|
||||
if(metadata==null||!registry.sameSignature(type.signature(),metadata.signature()))throw new IllegalArgumentException("closure signature mismatch");
|
||||
for(var capture:metadata.captures())pops=Math.addExact(pops,registry.width(capture));
|
||||
second=first;first=target;pushes=1;
|
||||
}
|
||||
case CALL_CLOSURE -> {
|
||||
if(!(body instanceof Fn type))throw new IllegalArgumentException("call type must be Fn");
|
||||
pops=registry.parameterSlots(type.signature())+1;pushes=registry.resultSlots(type.signature());
|
||||
}
|
||||
case GET_CAPTURE -> {if(instruction.member()>=current.metadata().captures().size())throw new IllegalArgumentException("capture index");pushes=registry.width(current.metadata().captures().get(instruction.member()));}
|
||||
case CALL_TRAIT -> {
|
||||
if(!(body instanceof Trait trait)||instruction.member()>=trait.methods().size())throw new IllegalArgumentException("trait/method operand");
|
||||
var signature=trait.methods().get(instruction.member()).signature();second=instruction.member();pops=registry.parameterSlots(signature)+2;pushes=registry.resultSlots(signature);
|
||||
}
|
||||
case DYN_PACK -> {pops=registry.width(instruction.type());pushes=2;}
|
||||
case DYN_UNPACK -> {pops=2;pushes=registry.width(instruction.type());}
|
||||
case DYN_CAN_UNPACK -> {pops=2;pushes=1;}
|
||||
case REF_IS_EXACT,REF_IMPLEMENTS,REF_TO_TRAIT,REF_CAST -> {
|
||||
if(!(body instanceof Struct||body instanceof Array||body instanceof Trait))throw new IllegalArgumentException("reference source category");
|
||||
var target=registry.type(instruction.targetType()).body();boolean trait=instruction.kind()==IRRegistryInstruction.Kind.REF_IMPLEMENTS||instruction.kind()==IRRegistryInstruction.Kind.REF_TO_TRAIT;
|
||||
if(trait?!(target instanceof Trait):!(target instanceof Struct||target instanceof Array))throw new IllegalArgumentException("reference target category");
|
||||
second=registry.id(instruction.targetType()).index();pops=registry.width(instruction.type());pushes=instruction.kind()==IRRegistryInstruction.Kind.REF_TO_TRAIT?2:1;
|
||||
}
|
||||
case STRUCT_NEW,STRUCT_GET,STRUCT_SET -> {
|
||||
if(!(body instanceof Struct struct))throw new IllegalArgumentException("Struct category required");
|
||||
if(instruction.kind()==IRRegistryInstruction.Kind.STRUCT_NEW)pushes=1;
|
||||
else {
|
||||
if(instruction.member()>=struct.fields().size())throw new IllegalArgumentException("field index");
|
||||
second=instruction.member();int width=registry.width(struct.fields().get(instruction.member()).type());
|
||||
pops=instruction.kind()==IRRegistryInstruction.Kind.STRUCT_SET?1+width:1;pushes=instruction.kind()==IRRegistryInstruction.Kind.STRUCT_GET?width:0;
|
||||
}
|
||||
}
|
||||
case ARRAY_NEW,ARRAY_LEN,ARRAY_GET,ARRAY_SET -> {
|
||||
if(!(body instanceof Array array))throw new IllegalArgumentException("Array category required");
|
||||
int width=registry.width(array.element());
|
||||
switch(instruction.kind()) {case ARRAY_NEW,ARRAY_LEN->{pops=1;pushes=1;}case ARRAY_GET->{pops=2;pushes=width;}case ARRAY_SET->pops=2+width;default->throw new AssertionError();}
|
||||
}
|
||||
}
|
||||
return new IRVMInstruction(new IRVMOp("REGISTRY_"+instruction.kind(),instruction.kind().opcode(),second==null?4:8,pops,pushes,false,false,false),first,second);
|
||||
}
|
||||
}
|
||||
+1
@@ -201,6 +201,7 @@ public final class LifecycleAssemblerService {
|
||||
frameExecutable.span()));
|
||||
|
||||
return IRBackend.builder()
|
||||
.registry(backend.getRegistry())
|
||||
.entryPointCallableName(entryPointCallableName)
|
||||
.entryPointModuleId(entryPointModuleId)
|
||||
.functions(backend.getFunctions())
|
||||
|
||||
+4
@@ -54,6 +54,10 @@ public class LowerToIRVMPipelineStage implements PipelineStage {
|
||||
.pc(e.pc())
|
||||
.message("[BUILD]: lower to irvm validation failed: " + e.getMessage())
|
||||
.exception(e));
|
||||
} catch (IllegalArgumentException e) {
|
||||
return BuildingIssueSink.empty().report(builder -> builder.error(true)
|
||||
.phase("BACKEND_LOWER_TO_IRVM").code("LOWER_IRVM_INVALID_TYPED_REGISTRY")
|
||||
.message("[BUILD]: invalid common typed IR: " + e.getMessage()).exception(e));
|
||||
}
|
||||
return BuildingIssueSink.empty();
|
||||
}
|
||||
|
||||
+2
-2
@@ -106,7 +106,7 @@ class BytecodeEmitterTest {
|
||||
0,
|
||||
1,
|
||||
ReadOnlyList.from(
|
||||
BytecodeEmitter.Operation.intrinsic(0x2000),
|
||||
BytecodeEmitter.Operation.intrinsic(0x2000, 0, 0, null),
|
||||
BytecodeEmitter.Operation.halt())))));
|
||||
|
||||
assertTrue(module.syscalls().isEmpty());
|
||||
@@ -131,7 +131,7 @@ class BytecodeEmitterTest {
|
||||
ReadOnlyList.from(
|
||||
BytecodeEmitter.Operation.halt(new BytecodeModule.SourceSpan(1, 0, 1)),
|
||||
BytecodeEmitter.Operation.callFunc(0, new BytecodeModule.SourceSpan(1, 2, 3)),
|
||||
BytecodeEmitter.Operation.intrinsic(0x2000, new BytecodeModule.SourceSpan(1, 4, 5)),
|
||||
BytecodeEmitter.Operation.intrinsic(0x2000, 0, 0, new BytecodeModule.SourceSpan(1, 4, 5)),
|
||||
BytecodeEmitter.Operation.ret(new BytecodeModule.SourceSpan(1, 6, 7)))))));
|
||||
|
||||
assertEquals(4, module.debugInfo().pcToSpan().size());
|
||||
|
||||
+5
-5
@@ -22,13 +22,13 @@ class BytecodeModuleTest {
|
||||
assertEquals(0, bytes[3]);
|
||||
|
||||
final var sectionCount = readU32(bytes, 8);
|
||||
assertEquals(1, sectionCount);
|
||||
assertEquals(8, sectionCount);
|
||||
|
||||
final var sectionKind = readU32(bytes, 32);
|
||||
final var sectionOffset = readU32(bytes, 36);
|
||||
final var sectionLength = readU32(bytes, 40);
|
||||
final var sectionKind = readU32(bytes, 32 + 4 * 12);
|
||||
final var sectionOffset = readU32(bytes, 36 + 4 * 12);
|
||||
final var sectionLength = readU32(bytes, 40 + 4 * 12);
|
||||
assertEquals(5, sectionKind);
|
||||
assertEquals(44, sectionOffset);
|
||||
assertEquals(140, sectionOffset);
|
||||
assertEquals(4, sectionLength);
|
||||
|
||||
final var syscCount = readU32(bytes, sectionOffset);
|
||||
|
||||
+53
@@ -0,0 +1,53 @@
|
||||
package p.studio.compiler.backend.registry;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import p.studio.compiler.models.registry.IRRegistry;
|
||||
import p.studio.compiler.models.registry.IRRegistry.*;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
class LinkedRegistryTest {
|
||||
private static TypeRef ref(String key) { return new TypeRef(key); }
|
||||
private static TypeDeclaration decl(String key, TypeBody body) { return new TypeDeclaration(ref(key), "same display name", body); }
|
||||
private static IRRegistry source(TypeDeclaration... types) { return new IRRegistry(ReadOnlyList.from(types), null); }
|
||||
|
||||
@Test void nominalIdentityIsNotNameOrShapeAndRepeatedDeclarationsConverge() {
|
||||
var a=decl("module:a",new Struct(null)); var b=decl("module:b",new Struct(null));
|
||||
var linked=LinkedRegistry.link(source(a,b,a));
|
||||
assertEquals(2,linked.types().size());
|
||||
assertNotEquals(linked.id(a.identity()),linked.id(b.identity()));
|
||||
assertEquals(linked.types(),LinkedRegistry.link(source(b,a)).types());
|
||||
}
|
||||
@Test void importsArraysAndFnSignaturesCanonicalizeWithoutUsingParameterNames() {
|
||||
var signature=new Signature(ReadOnlyList.from(new NamedType(ref("v1"),"x")),ReadOnlyList.from(ref("v2")));
|
||||
var renamed=new Signature(ReadOnlyList.from(new NamedType(ref("v2"),"renamed")),ReadOnlyList.from(ref("v1")));
|
||||
var linked=LinkedRegistry.link(source(decl("v1",new Builtin(2)),decl("v2",new Builtin(2)),
|
||||
decl("a1",new Array(ref("v1"))),decl("a2",new Array(ref("v2"))),
|
||||
decl("f1",new Fn(signature)),decl("f2",new Fn(renamed))));
|
||||
assertEquals(3,linked.types().size());
|
||||
assertEquals(linked.id(ref("v1")),linked.id(ref("v2")));
|
||||
assertEquals(linked.id(ref("a1")),linked.id(ref("a2")));
|
||||
assertEquals(linked.id(ref("f1")),linked.id(ref("f2")));
|
||||
assertEquals(2,linked.parameterSlots(signature));
|
||||
assertEquals(2,linked.resultSlots(signature));
|
||||
}
|
||||
@Test void nominalAnchorsAllowForwardReferencesButStructuralCyclesReject() {
|
||||
var signature=new Signature(ReadOnlyList.from(new NamedType(ref("array"),null)),null);
|
||||
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source(
|
||||
decl("fn",new Fn(signature)),decl("array",new Array(ref("fn"))))));
|
||||
assertDoesNotThrow(()->LinkedRegistry.link(source(
|
||||
decl("fn",new Fn(signature)),decl("array",new Array(ref("node"))),
|
||||
decl("node",new Struct(ReadOnlyList.from(new NamedType(ref("fn"),null)))))));
|
||||
}
|
||||
@Test void missingReferencesConflictingDeclarationsAndRemovedImportsReject() {
|
||||
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source(decl("a",new Array(ref("missing"))))));
|
||||
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.link(source(decl("a",new Builtin(1)),decl("a",new Builtin(2)))));
|
||||
assertThrows(IllegalArgumentException.class,()->new Builtin(0x100));
|
||||
assertEquals(1,LinkedRegistry.link(source(decl("random",new Builtin(0x16)))).width(ref("random")));
|
||||
}
|
||||
@Test void namesAreStrictUtf8AndBounded() {
|
||||
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.nameBytes("\uD800"));
|
||||
assertThrows(IllegalArgumentException.class,()->LinkedRegistry.nameBytes("é".repeat(2049)));
|
||||
assertEquals(4096,LinkedRegistry.nameBytes("é".repeat(2048)));
|
||||
}
|
||||
}
|
||||
+188
@@ -0,0 +1,188 @@
|
||||
package p.studio.compiler.backend.registry;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import p.studio.compiler.backend.bytecode.*;
|
||||
import p.studio.compiler.models.registry.IRRegistry;
|
||||
import p.studio.compiler.models.registry.IRRegistry.*;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
import java.nio.*;
|
||||
import java.nio.file.*;
|
||||
import java.util.*;
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
class PbxRegistryConformanceTest {
|
||||
private static byte[] instructions(int[]... instructions) {
|
||||
var out=new java.io.ByteArrayOutputStream();
|
||||
for(var instruction:instructions) {
|
||||
out.write(instruction[0]);out.write(instruction[0]>>>8);
|
||||
for(int n=1;n<instruction.length;n++)for(int b=0;b<4;b++)out.write(instruction[n]>>>(8*b));
|
||||
}
|
||||
return out.toByteArray();
|
||||
}
|
||||
private static int[] op(int... values) { return values; }
|
||||
private static final BytecodePreloadVerifierService VERIFIER = new BytecodePreloadVerifierService();
|
||||
private static IRRegistry registry(int builtin) {
|
||||
return new IRRegistry(ReadOnlyList.from(new TypeDeclaration(TypeRef.builtin(builtin), null, new Builtin(builtin))), null);
|
||||
}
|
||||
private static byte[] hex(String value) { return HexFormat.of().parseHex(value.replaceAll("\\s", "")); }
|
||||
private static BytecodeModule module(IRRegistry registry, byte[] code, int parameters, int results, int max, FunctionMetadata metadata) {
|
||||
return new BytecodeModule(0, null, ReadOnlyList.from(new BytecodeModule.FunctionMeta(0, code.length, parameters, 0, results, max, metadata)), code, null, null, null, registry);
|
||||
}
|
||||
private static BytecodeModule emptyFunction(byte[] code, int max) {
|
||||
return module(registry(0x13), code, 0, 0, max, FunctionMetadata.empty());
|
||||
}
|
||||
private static void fixture(String name, BytecodeModule module) throws Exception {
|
||||
Path directory=Path.of("build/registry-conformance"); Files.createDirectories(directory);
|
||||
Files.write(directory.resolve(name+".pbx"), module.serialize());
|
||||
}
|
||||
private static byte[] section(byte[] bytes, int kind) {
|
||||
var b=ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN);
|
||||
for(int i=0;i<b.getInt(8);i++) {int at=32+12*i; if(b.getInt(at)==kind)return Arrays.copyOfRange(bytes,b.getInt(at+4),b.getInt(at+4)+b.getInt(at+8));}
|
||||
throw new AssertionError("missing section "+kind);
|
||||
}
|
||||
@Test void matchesIndependent194ByteOracle() throws Exception {
|
||||
var module=emptyFunction(hex("01 00"),0);
|
||||
String source=Files.readString(Path.of("src/test/resources/registry/minimal-runtime-v0.hex"));
|
||||
byte[] expected=hex(source.substring(source.indexOf('\n')+1));
|
||||
assertEquals(194, expected.length);
|
||||
assertArrayEquals(expected,module.serialize());
|
||||
VERIFIER.verify(module);fixture("minimal",module);
|
||||
}
|
||||
@Test void vec2SignatureHasLogicalArityAndNoWireWidthCaches() throws Exception {
|
||||
var type=TypeRef.builtin(2);
|
||||
var metadata=FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(type,null)),ReadOnlyList.from(type)));
|
||||
var module=module(registry(2),hex("42 00 00 00 00 00 42 00 01 00 00 00 51 00"),2,2,2,metadata);
|
||||
assertArrayEquals(hex("01 00 00 00 00 00 01 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 0e 00 00 00 00 00 00 00 02 00 00 00"),section(module.serialize(),1));
|
||||
VERIFIER.verify(module);fixture("vec2-signature",module);
|
||||
assertThrows(BytecodeMarshalingException.class,()->module(registry(2),module.code(),1,1,2,metadata).serialize());
|
||||
}
|
||||
@Test void absoluteJumpToFunctionEndAndUnreachableOperands() throws Exception {
|
||||
var module=emptyFunction(hex("02 00 08 00 00 00 00 00"),0);
|
||||
VERIFIER.verify(module);fixture("jump-end",module);
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("02 00 03 00 00 00 01 00"),0)));
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("01 00 50 00 ff ff ff ff"),0)));
|
||||
for(int op:new int[]{0x54,0x55,0x56,0x70})assertThrows(RuntimeException.class,()->VERIFIER.verify(emptyFunction(new byte[]{1,0,(byte)op,0,0,0,0,0},0)));
|
||||
}
|
||||
@Test void literalZeroAndExactReturnHeight() {
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 11 00 51 00"),0)));
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 51 00"),1)));
|
||||
assertDoesNotThrow(()->VERIFIER.verify(emptyFunction(hex("17 00 00 00 00 00 11 00 51 00"),1)));
|
||||
}
|
||||
@Test void nonfiniteUnreachableImmediateAndConstantReject() {
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("01 00 15 00 00 00 00 00 00 00 f0 7f"),0)));
|
||||
var m=emptyFunction(hex("01 00"),0);
|
||||
var bad=new BytecodeModule(0,ReadOnlyList.from(new BytecodeModule.Float64Constant(Double.NaN)),m.functions(),m.code(),null,null,null,m.registry());
|
||||
assertThrows(BytecodeMarshalingException.class,bad::serialize);
|
||||
}
|
||||
@Test void signatureNamesAreIndependentFromDebugAndStrictUtf8() {
|
||||
var metadata=new FunctionMetadata(Signature.empty(),null,Role.NONE,"");
|
||||
var module=module(registry(0x13),hex("01 00"),0,0,0,metadata);
|
||||
byte[] function=section(module.serialize(),1);
|
||||
assertEquals(1,function[5]);assertEquals(34,function.length);
|
||||
var bad=new FunctionMetadata(Signature.empty(),null,Role.NONE,"\uD800");
|
||||
assertThrows(BytecodeMarshalingException.class,()->module(registry(0x13),hex("01 00"),0,0,0,bad).serialize());
|
||||
}
|
||||
@Test void unusedAndConflictingNativeDeclarationsRejectWithoutBindingFallback() {
|
||||
var m=emptyFunction(hex("01 00"),0);
|
||||
var declaration=new BytecodeModule.SyscallDecl("test","native",0,0,0);
|
||||
var unused=new BytecodeModule(0,null,m.functions(),m.code(),null,null,ReadOnlyList.from(declaration),m.registry());
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(unused));
|
||||
var duplicate=new BytecodeModule(0,null,m.functions(),m.code(),null,null,ReadOnlyList.from(declaration,declaration),m.registry());
|
||||
assertThrows(BytecodeMarshalingException.class,duplicate::serialize);
|
||||
}
|
||||
@Test void closuresCarryExplicitEnvironmentAndExactFnIdentity() throws Exception {
|
||||
var integer=TypeRef.builtin(0x10);var fn=new TypeRef("fixture:fn");
|
||||
var signature=new Signature(null,ReadOnlyList.from(integer));
|
||||
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(fn,null,new Fn(signature))),null);
|
||||
int fnId=LinkedRegistry.link(registry).id(fn).index();
|
||||
byte[] boot=instructions(op(0x17,42),op(0x52,1,fnId),op(0x53,fnId),op(1));
|
||||
byte[] body=instructions(op(0x57,0),op(0x51));
|
||||
var bytes=new java.io.ByteArrayOutputStream();bytes.writeBytes(boot);bytes.writeBytes(body);
|
||||
var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,boot.length,0,0,0,1,FunctionMetadata.empty()),
|
||||
new BytecodeModule.FunctionMeta(boot.length,body.length,0,0,1,1,new FunctionMetadata(signature,ReadOnlyList.from(integer),Role.NONE)));
|
||||
var m=new BytecodeModule(0,null,functions,bytes.toByteArray(),null,null,null,registry);
|
||||
VERIFIER.verify(m);fixture("closure-capture",m);
|
||||
byte[] direct=m.code().clone();direct[6]=0x50;
|
||||
var invalid=new BytecodeModule(0,null,functions,direct,null,null,null,registry);
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(invalid));
|
||||
}
|
||||
@Test void traitImplementationUsesConcreteReceiverAndOrderedMethods() throws Exception {
|
||||
var integer=TypeRef.builtin(0x10);var owner=new TypeRef("fixture:owner");var trait=new TypeRef("fixture:trait");
|
||||
var result=new Signature(null,ReadOnlyList.from(integer));
|
||||
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),
|
||||
new TypeDeclaration(owner,null,new Struct(null)),new TypeDeclaration(trait,null,new Trait(ReadOnlyList.from(new Method("answer",result))))),
|
||||
ReadOnlyList.from(new Implementation("fixture:impl",owner,trait,ReadOnlyList.from(new p.studio.compiler.source.identifiers.CallableId(1)))));
|
||||
var linked=LinkedRegistry.link(registry);int ownerId=linked.id(owner).index(),traitId=linked.id(trait).index();
|
||||
byte[] boot=instructions(op(0xa0,ownerId),op(0x95,ownerId,traitId),op(0x58,traitId,0),op(1));
|
||||
byte[] body=instructions(op(0x17,42),op(0x51));
|
||||
var bytes=new java.io.ByteArrayOutputStream();bytes.writeBytes(boot);bytes.writeBytes(body);
|
||||
var implementation=FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(owner,"self")),ReadOnlyList.from(integer)));
|
||||
var m=new BytecodeModule(0,null,ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,boot.length,0,0,0,2,FunctionMetadata.empty()),
|
||||
new BytecodeModule.FunctionMeta(boot.length,body.length,1,0,1,1,implementation)),bytes.toByteArray(),null,null,null,registry);
|
||||
VERIFIER.verify(m);fixture("trait-call",m);
|
||||
}
|
||||
@Test void arrayAndDynamicOperationsUseRegistryOperands() throws Exception {
|
||||
var integer=TypeRef.builtin(0x10);var array=new TypeRef("fixture:array");
|
||||
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(array,null,new Array(integer))),null);
|
||||
var linked=LinkedRegistry.link(registry);int arrayId=linked.id(array).index(),intId=linked.id(integer).index();
|
||||
byte[] code=instructions(op(0x14,1,0),op(0xa3,arrayId),op(0x14,0,0),op(0xa5,arrayId),op(0x90,intId),op(0x91,intId),op(1));
|
||||
var m=module(registry,code,0,0,2,FunctionMetadata.empty());
|
||||
VERIFIER.verify(m);fixture("array-dynamic",m);
|
||||
}
|
||||
@Test void bindingFixturesKeepExactCanonicalIdentityAndAbi() throws Exception {
|
||||
var m=emptyFunction(instructions(op(0x71,0),op(0x11),op(1)),1);
|
||||
for(String name:List.of("native", "unknown-native", "native-abi", "unused-native")) {
|
||||
var declaration=new BytecodeModule.SyscallDecl("system",name.equals("unknown-native")?"missing":"has_cart",1,name.equals("native-abi")?1:0,1);
|
||||
byte[] code=name.equals("unused-native")?instructions(op(1)):m.code();
|
||||
var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,code.length,0,0,0,1,FunctionMetadata.empty()));
|
||||
var module=new BytecodeModule(0,null,functions,code,null,null,ReadOnlyList.from(declaration),m.registry());
|
||||
fixture(name,module);
|
||||
if(name.equals("native"))VERIFIER.verify(module);
|
||||
}
|
||||
}
|
||||
@Test void rolesStayMetadataAndValidateSignatureAndUniqueness() throws Exception {
|
||||
var pause=new FunctionMetadata(Signature.empty(),null,Role.BEFORE_PAUSE);
|
||||
var resume=new FunctionMetadata(Signature.empty(),null,Role.BEFORE_RESUME);
|
||||
var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,2,0,0,0,0,FunctionMetadata.empty()),
|
||||
new BytecodeModule.FunctionMeta(2,2,0,0,0,0,pause),new BytecodeModule.FunctionMeta(4,2,0,0,0,0,resume));
|
||||
var m=new BytecodeModule(0,null,functions,instructions(op(1),op(0x51),op(0x51)),null,null,null,IRRegistry.empty());
|
||||
VERIFIER.verify(m);fixture("roles",m);
|
||||
assertThrows(BytecodeMarshalingException.class,()->module(IRRegistry.empty(),instructions(op(1)),0,0,0,pause).serialize());
|
||||
var duplicated=ReadOnlyList.from(functions.get(0),functions.get(1),new BytecodeModule.FunctionMeta(4,2,0,0,0,0,pause));
|
||||
assertThrows(BytecodeMarshalingException.class,()->new BytecodeModule(0,null,duplicated,m.code(),null,null,null,m.registry()).serialize());
|
||||
}
|
||||
@Test void joinsWidenFactsAndLoopsReachFixedPoint() throws Exception {
|
||||
// bool; branch to alternate scalar; equal-height join becomes Unknown, then POP.
|
||||
byte[] code=instructions(op(0x10,0),op(0x03,24),op(0x17,1),op(0x02,30),op(0x10,1),op(0x11),op(1));
|
||||
var base=emptyFunction(code,1);
|
||||
var m=new BytecodeModule(0,ReadOnlyList.from(new BytecodeModule.BooleanConstant(false),new BytecodeModule.StringConstant("alternate")),base.functions(),code,null,null,null,base.registry());
|
||||
VERIFIER.verify(m);fixture("join",m);
|
||||
byte[] loop=instructions(op(0x40,0),op(0x03,18),op(0x02,0),op(1));
|
||||
assertDoesNotThrow(()->VERIFIER.verify(emptyFunction(loop,1)));
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(emptyFunction(hex("16 00 01 16 00 00 20 00 01 00"),2)));
|
||||
}
|
||||
@Test void memcardRoleHasExactTypedSignatureWithoutLanguagePolicy() throws Exception {
|
||||
var input=TypeRef.builtin(0x11);var output=TypeRef.builtin(0x10);
|
||||
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(input,null,new Builtin(0x11)),new TypeDeclaration(output,null,new Builtin(0x10))),null);
|
||||
var metadata=new FunctionMetadata(new Signature(ReadOnlyList.from(new NamedType(input,"status")),ReadOnlyList.from(output)),null,Role.MEMCARD_BOOT);
|
||||
var code=instructions(op(1),op(0x17,0),op(0x51));
|
||||
var functions=ReadOnlyList.from(new BytecodeModule.FunctionMeta(0,2,0,0,0,0,FunctionMetadata.empty()),new BytecodeModule.FunctionMeta(2,8,1,0,1,1,metadata));
|
||||
var module=new BytecodeModule(0,null,functions,code,null,null,null,registry);
|
||||
VERIFIER.verify(module);fixture("memcard-role",module);
|
||||
}
|
||||
@Test void knownSemanticGroupsCannotSilentlyChangeType() {
|
||||
var color=TypeRef.builtin(1);var integer=TypeRef.builtin(0x11);
|
||||
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(color,null,new Builtin(1)),new TypeDeclaration(integer,null,new Builtin(0x11))),null);
|
||||
var metadata=FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(color,null)),ReadOnlyList.from(integer)));
|
||||
var module=module(registry,instructions(op(0x42,0),op(0x51)),1,1,1,metadata);
|
||||
assertThrows(BytecodeMarshalingException.class,()->VERIFIER.verify(module));
|
||||
}
|
||||
@Test void intrinsicFixtureUsesExplicitAbiAndKeepsNativeBindingSeparate() throws Exception {
|
||||
// Runtime vec2.dot@1 (0x1000): four Float carriers -> one Float.
|
||||
byte[] code=instructions(op(0x15,0,0x3ff00000),op(0x15,0,0x40000000),
|
||||
op(0x15,0,0x40080000),op(0x15,0,0x40100000),op(0x72,0x1000),op(1));
|
||||
var base=emptyFunction(code,4);
|
||||
var module=new BytecodeModule(0,null,base.functions(),code,null,null,null,base.registry(),ReadOnlyList.from(new BytecodeModule.IntrinsicSignature(0x1000,4,1)));
|
||||
VERIFIER.verify(module);fixture("intrinsic-dot",module);
|
||||
}
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
package p.studio.compiler.backend.registry;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import p.studio.compiler.backend.bytecode.*;
|
||||
import p.studio.compiler.backend.irvm.*;
|
||||
import p.studio.compiler.models.*;
|
||||
import p.studio.compiler.models.registry.*;
|
||||
import p.studio.compiler.models.registry.IRRegistry.*;
|
||||
import p.studio.compiler.source.identifiers.*;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
import java.nio.file.*;
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
class TypedCommonPipelineTest {
|
||||
private static IRBackendExecutableFunction.Instruction plain(IRBackendExecutableFunction.InstructionKind kind, String literal) {
|
||||
return new IRBackendExecutableFunction.Instruction(kind,null,null,null,null,null,literal,null,null,null,null);
|
||||
}
|
||||
private static IRBackendExecutableFunction.Instruction typed(IRRegistryInstruction operand) {
|
||||
return new IRBackendExecutableFunction.Instruction(IRBackendExecutableFunction.InstructionKind.REGISTRY,null,null,null,null,null,null,null,null,null,null,operand);
|
||||
}
|
||||
@Test void syntheticFrontendClosureSurvivesLinkLowerOptimizeAndEmit() throws Exception {
|
||||
var integer=TypeRef.builtin(0x10);var fn=new TypeRef("synthetic:fn");
|
||||
var signature=new Signature(null,ReadOnlyList.from(integer));
|
||||
var registry=new IRRegistry(ReadOnlyList.from(new TypeDeclaration(integer,null,new Builtin(0x10)),new TypeDeclaration(fn,null,new Fn(signature))),null);
|
||||
var capture=new CallableId(71);
|
||||
var main=new IRBackendExecutableFunction(new FileId(0),new ModuleId(0),"main",new CallableId(91),0,0,0,0,0,1,
|
||||
ReadOnlyList.from(plain(IRBackendExecutableFunction.InstructionKind.PUSH_I32,"42"),
|
||||
typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.MAKE_CLOSURE,fn,null,capture,0)),
|
||||
typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.CALL_CLOSURE,fn,null,null,0)),
|
||||
plain(IRBackendExecutableFunction.InstructionKind.POP,null),plain(IRBackendExecutableFunction.InstructionKind.RET,null)),null,FunctionMetadata.empty());
|
||||
var body=new IRBackendExecutableFunction(new FileId(0),new ModuleId(0),"captured",capture,0,0,0,0,1,1,
|
||||
ReadOnlyList.from(typed(new IRRegistryInstruction(IRRegistryInstruction.Kind.GET_CAPTURE,null,null,null,0)),
|
||||
plain(IRBackendExecutableFunction.InstructionKind.RET,null)),null,new FunctionMetadata(signature,ReadOnlyList.from(integer),Role.NONE));
|
||||
var backend=IRBackend.builder().entryPointModuleId(new ModuleId(0)).registry(registry).executableFunctions(ReadOnlyList.from(body,main)).build();
|
||||
var lowered=new LowerToIRVMService().lower(backend);
|
||||
var optimized=new OptimizeIRVMService().optimize(lowered);
|
||||
assertEquals(registry,optimized.module().registry());
|
||||
assertEquals(body.metadata(),optimized.module().functions().get(1).metadata());
|
||||
var module=new BytecodeEmitter().emit(optimized.coherentEmissionPlan());
|
||||
new BytecodePreloadVerifierService().verify(module);
|
||||
var repeated=new BytecodeEmitter().emit(new OptimizeIRVMService().optimize(new LowerToIRVMService().lower(backend)).coherentEmissionPlan());
|
||||
assertArrayEquals(module.serialize(),repeated.serialize());
|
||||
Files.createDirectories(Path.of("build/registry-conformance"));
|
||||
Files.write(Path.of("build/registry-conformance/common-pipeline.pbx"),module.serialize());
|
||||
}
|
||||
}
|
||||
+4
-1
@@ -1,6 +1,8 @@
|
||||
package p.studio.compiler.workspaces.stages;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import p.studio.compiler.models.registry.IRRegistry;
|
||||
import p.studio.compiler.models.registry.IRRegistry.*;
|
||||
import p.studio.compiler.messages.BuilderPipelineConfig;
|
||||
import p.studio.compiler.models.BuilderPipelineContext;
|
||||
import p.studio.compiler.models.IRBackend;
|
||||
@@ -144,6 +146,7 @@ class LowerToIRVMPipelineStageTest {
|
||||
final var ctx = BuilderPipelineContext.fromConfig(new BuilderPipelineConfig(false, "."));
|
||||
ctx.irBackend = IRBackend.builder()
|
||||
.entryPointModuleId(new p.studio.compiler.source.identifiers.ModuleId(0))
|
||||
.registry(new IRRegistry(ReadOnlyList.from(new TypeDeclaration(TypeRef.builtin(0x10),null,new Builtin(0x10))),null))
|
||||
.executableFunctions(ReadOnlyList.from(
|
||||
new IRBackendExecutableFunction(
|
||||
new FileId(0),
|
||||
@@ -162,7 +165,7 @@ class LowerToIRVMPipelineStageTest {
|
||||
null,
|
||||
null,
|
||||
Span.none())),
|
||||
Span.none()),
|
||||
Span.none(), FunctionMetadata.ordinary(new Signature(ReadOnlyList.from(new NamedType(TypeRef.builtin(0x10),null)),null))),
|
||||
new IRBackendExecutableFunction(
|
||||
new FileId(0),
|
||||
new p.studio.compiler.source.identifiers.ModuleId(0),
|
||||
|
||||
+1
-1
@@ -1 +1 @@
|
||||
504258000000000004000000000000000000000000000000000000000000000001000000500000001400000002000000640000000e00000003000000720000004400000005000000b60000001d00000001000000000000000e000000000000000000020071000000000072000020000051000300000000000000010000000000000000000000060000000100000000000000000000000c0000000100000000000000000000000100000000000000040000006d61696e01000000050067667832640a00647261775f706978656c010000000000
|
||||
5042580000000000090000000000000000000000000000000000000000000000000000008c0000000400000001000000900000001e00000002000000ae0000000e00000003000000bc0000004400000004000000000100000400000005000000040100001d000000060000002101000004000000070000002501000004000000080000002901000008000000000000000100000000000000000000000000000000000e000000000000000200000071000000000072000020000051000300000000000000010000000000000000000000060000000100000000000000000000000c0000000100000000000000000000000100000000000000040000006d61696e0000000001000000050067667832640a00647261775f706978656c01000000000000000000000000000100000000000000
|
||||
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
# Independent runtime registry_v0.rs minimal_bytes oracle (194 bytes)
|
||||
50 42 58 00 00 00 00 00 08 00 00 00
|
||||
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|
||||
00 00 00 00 80 00 00 00 04 00 00 00
|
||||
01 00 00 00 84 00 00 00 1e 00 00 00
|
||||
02 00 00 00 a2 00 00 00 02 00 00 00
|
||||
04 00 00 00 a4 00 00 00 04 00 00 00
|
||||
05 00 00 00 a8 00 00 00 04 00 00 00
|
||||
06 00 00 00 ac 00 00 00 0a 00 00 00
|
||||
07 00 00 00 b6 00 00 00 04 00 00 00
|
||||
08 00 00 00 ba 00 00 00 08 00 00 00
|
||||
00 00 00 00
|
||||
01 00 00 00 00 00 00 00 00 00 00 00 00 00
|
||||
00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00
|
||||
01 00
|
||||
00 00 00 00
|
||||
00 00 00 00
|
||||
01 00 00 00 00 00 13 00 00 00
|
||||
00 00 00 00
|
||||
01 00 00 00 00 00 00 00
|
||||
+15
-2
@@ -1,6 +1,7 @@
|
||||
package p.studio.compiler.models;
|
||||
|
||||
import lombok.Builder;
|
||||
import p.studio.compiler.models.registry.IRRegistry;
|
||||
import lombok.Getter;
|
||||
import p.studio.compiler.source.identifiers.CallableId;
|
||||
import p.studio.compiler.source.identifiers.IntrinsicId;
|
||||
@@ -34,6 +35,8 @@ public class IRBackend implements IRBackendReader {
|
||||
private final ReadOnlyList<IntrinsicReference> intrinsicPool = ReadOnlyList.empty();
|
||||
@Builder.Default
|
||||
private final IRReservedMetadata reservedMetadata = IRReservedMetadata.empty();
|
||||
@Builder.Default
|
||||
private final IRRegistry registry = IRRegistry.empty();
|
||||
private final IRLifecycleDeclaration lifecycleDeclaration;
|
||||
|
||||
public static IRBackendAggregator aggregator() {
|
||||
@@ -41,6 +44,8 @@ public class IRBackend implements IRBackendReader {
|
||||
}
|
||||
|
||||
public static final class IRBackendAggregator {
|
||||
private final ArrayList<IRRegistry.TypeDeclaration> registryTypes = new ArrayList<>();
|
||||
private final ArrayList<IRRegistry.Implementation> registryImplementations = new ArrayList<>();
|
||||
private String entryPointCallableName;
|
||||
private ModuleId entryPointModuleId = ModuleId.none();
|
||||
private final ArrayList<IRFunction> functions = new ArrayList<>();
|
||||
@@ -90,6 +95,11 @@ public class IRBackend implements IRBackendReader {
|
||||
}
|
||||
final var callableRemap = reindexCallables(backendFile.callableSignatures(), moduleRemap);
|
||||
final var intrinsicRemap = reindexIntrinsics(backendFile.intrinsicPool());
|
||||
registryTypes.addAll(backendFile.registry().types().asList());
|
||||
for (final var impl : backendFile.registry().implementations()) {
|
||||
registryImplementations.add(new IRRegistry.Implementation(impl.declarationKey(), impl.owner(), impl.trait(),
|
||||
ReadOnlyList.wrap(impl.methods().stream().map(id -> remapCallableId(id, callableRemap, "implementation")).toList())));
|
||||
}
|
||||
for (final var function : backendFile.executableFunctions()) {
|
||||
executableFunctions.add(remapExecutableFunction(function, moduleRemap, callableRemap, intrinsicRemap));
|
||||
}
|
||||
@@ -200,7 +210,9 @@ public class IRBackend implements IRBackendReader {
|
||||
instruction.targetLabel(),
|
||||
instruction.expectedArgSlots(),
|
||||
instruction.expectedRetSlots(),
|
||||
instruction.span()));
|
||||
instruction.span(), instruction.registryInstruction() == null ? null :
|
||||
new p.studio.compiler.models.registry.IRRegistryInstruction(instruction.registryInstruction().kind(), instruction.registryInstruction().type(), instruction.registryInstruction().targetType(),
|
||||
instruction.registryInstruction().function() == null ? null : remapCallableId(instruction.registryInstruction().function(), callableRemap, "Registry instruction"), instruction.registryInstruction().member())));
|
||||
}
|
||||
return new IRBackendExecutableFunction(
|
||||
function.fileId(),
|
||||
@@ -214,7 +226,7 @@ public class IRBackend implements IRBackendReader {
|
||||
function.returnSlots(),
|
||||
function.maxStackSlots(),
|
||||
ReadOnlyList.wrap(remappedInstructions),
|
||||
function.span());
|
||||
function.span(), function.metadata());
|
||||
}
|
||||
|
||||
private IRLifecycleFileDeclaration remapLifecycleFileDeclaration(
|
||||
@@ -367,6 +379,7 @@ public class IRBackend implements IRBackendReader {
|
||||
public IRBackend emit() {
|
||||
return IRBackend
|
||||
.builder()
|
||||
.registry(new IRRegistry(ReadOnlyList.wrap(registryTypes), ReadOnlyList.wrap(registryImplementations)))
|
||||
.entryPointCallableName(resolveEntryPointCallableName())
|
||||
.entryPointModuleId(resolveEntryPointModuleId())
|
||||
.functions(ReadOnlyList.wrap(functions))
|
||||
|
||||
+38
@@ -10,6 +10,20 @@ import p.studio.utilities.structures.ReadOnlyList;
|
||||
import java.util.Objects;
|
||||
|
||||
public record IRBackendExecutableFunction(
|
||||
FileId fileId,
|
||||
ModuleId moduleId,
|
||||
String callableName,
|
||||
CallableId callableId,
|
||||
int sourceStart,
|
||||
int sourceEnd,
|
||||
int paramSlots,
|
||||
int localSlots,
|
||||
int returnSlots,
|
||||
int maxStackSlots,
|
||||
ReadOnlyList<Instruction> instructions,
|
||||
Span span,
|
||||
p.studio.compiler.models.registry.IRRegistry.FunctionMetadata metadata) {
|
||||
public IRBackendExecutableFunction(
|
||||
FileId fileId,
|
||||
ModuleId moduleId,
|
||||
String callableName,
|
||||
@@ -22,6 +36,9 @@ public record IRBackendExecutableFunction(
|
||||
int maxStackSlots,
|
||||
ReadOnlyList<Instruction> instructions,
|
||||
Span span) {
|
||||
this(fileId, moduleId, callableName, callableId, sourceStart, sourceEnd, paramSlots, localSlots, returnSlots, maxStackSlots, instructions, span, paramSlots == 0 && returnSlots == 0 ? p.studio.compiler.models.registry.IRRegistry.FunctionMetadata.empty() : null);
|
||||
}
|
||||
|
||||
|
||||
public IRBackendExecutableFunction(
|
||||
final FileId fileId,
|
||||
@@ -82,6 +99,19 @@ public record IRBackendExecutableFunction(
|
||||
}
|
||||
|
||||
public record Instruction(
|
||||
InstructionKind kind,
|
||||
ModuleId calleeModuleId,
|
||||
String calleeCallableName,
|
||||
CallableId calleeCallableId,
|
||||
HostCallMetadata hostCall,
|
||||
IntrinsicCallMetadata intrinsicCall,
|
||||
String label,
|
||||
String targetLabel,
|
||||
Integer expectedArgSlots,
|
||||
Integer expectedRetSlots,
|
||||
Span span,
|
||||
p.studio.compiler.models.registry.IRRegistryInstruction registryInstruction) {
|
||||
public Instruction(
|
||||
InstructionKind kind,
|
||||
ModuleId calleeModuleId,
|
||||
String calleeCallableName,
|
||||
@@ -93,6 +123,9 @@ public record IRBackendExecutableFunction(
|
||||
Integer expectedArgSlots,
|
||||
Integer expectedRetSlots,
|
||||
Span span) {
|
||||
this(kind, calleeModuleId, calleeCallableName, calleeCallableId, hostCall, intrinsicCall, label, targetLabel, expectedArgSlots, expectedRetSlots, span, null);
|
||||
}
|
||||
|
||||
public Instruction(
|
||||
final InstructionKind kind,
|
||||
final ModuleId calleeModuleId,
|
||||
@@ -217,7 +250,11 @@ public record IRBackendExecutableFunction(
|
||||
if (expectedRetSlots != null && expectedRetSlots < 0) {
|
||||
throw new IllegalArgumentException("expectedRetSlots must be non-negative");
|
||||
}
|
||||
if ((kind == InstructionKind.REGISTRY) != (registryInstruction != null)) throw new IllegalArgumentException("Registry operand mismatch");
|
||||
switch (kind) {
|
||||
case REGISTRY -> {
|
||||
if (hostCall != null || intrinsicCall != null || calleeCallableId != null || !label.isEmpty() || !targetLabel.isEmpty()) throw new IllegalArgumentException("unexpected Registry instruction metadata");
|
||||
}
|
||||
case PUSH_I32 -> {
|
||||
if (label.isBlank()) {
|
||||
throw new IllegalArgumentException("PUSH_I32 requires immediate label payload");
|
||||
@@ -387,6 +424,7 @@ public record IRBackendExecutableFunction(
|
||||
}
|
||||
|
||||
public enum InstructionKind {
|
||||
REGISTRY,
|
||||
PUSH_I32,
|
||||
PUSH_BOOL,
|
||||
PUSH_CONST,
|
||||
|
||||
+18
@@ -7,8 +7,22 @@ import p.studio.compiler.source.tables.ModuleReference;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
|
||||
import java.util.Objects;
|
||||
import p.studio.compiler.models.registry.IRRegistry;
|
||||
|
||||
public record IRBackendFile(
|
||||
FileId fileId,
|
||||
ReadOnlyList<IRFunction> functions,
|
||||
ReadOnlyList<IRSyntheticFunction> syntheticFunctions,
|
||||
ReadOnlyList<IRGlobal> globals,
|
||||
ReadOnlyList<IRBackendExecutableFunction> executableFunctions,
|
||||
IRReservedMetadata reservedMetadata,
|
||||
ReadOnlyList<ModuleReference> modulePool,
|
||||
ReadOnlyList<CallableSignatureRef> callableSignatures,
|
||||
ReadOnlyList<IntrinsicReference> intrinsicPool,
|
||||
IRLifecycleFileDeclaration lifecycleDeclaration,
|
||||
IRRegistry registry) {
|
||||
|
||||
public IRBackendFile(
|
||||
FileId fileId,
|
||||
ReadOnlyList<IRFunction> functions,
|
||||
ReadOnlyList<IRSyntheticFunction> syntheticFunctions,
|
||||
@@ -19,7 +33,11 @@ public record IRBackendFile(
|
||||
ReadOnlyList<CallableSignatureRef> callableSignatures,
|
||||
ReadOnlyList<IntrinsicReference> intrinsicPool,
|
||||
IRLifecycleFileDeclaration lifecycleDeclaration) {
|
||||
this(fileId, functions, syntheticFunctions, globals, executableFunctions, reservedMetadata, modulePool, callableSignatures, intrinsicPool, lifecycleDeclaration, IRRegistry.empty());
|
||||
}
|
||||
|
||||
public IRBackendFile {
|
||||
registry = registry == null ? IRRegistry.empty() : registry;
|
||||
fileId = Objects.requireNonNull(fileId, "fileId");
|
||||
functions = functions == null ? ReadOnlyList.empty() : functions;
|
||||
syntheticFunctions = syntheticFunctions == null ? ReadOnlyList.empty() : syntheticFunctions;
|
||||
|
||||
+1
@@ -7,6 +7,7 @@ import p.studio.compiler.source.tables.ModuleReference;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
|
||||
public interface IRBackendReader {
|
||||
p.studio.compiler.models.registry.IRRegistry getRegistry();
|
||||
String getEntryPointCallableName();
|
||||
ModuleId getEntryPointModuleId();
|
||||
ReadOnlyList<IRFunction> getFunctions();
|
||||
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
package p.studio.compiler.models.registry;
|
||||
|
||||
import p.studio.compiler.source.identifiers.CallableId;
|
||||
import p.studio.utilities.structures.ReadOnlyList;
|
||||
import java.util.Objects;
|
||||
|
||||
/** Frontend-neutral declarations. References are stable keys, never Java object identity or image IDs. */
|
||||
public record IRRegistry(ReadOnlyList<TypeDeclaration> types, ReadOnlyList<Implementation> implementations) {
|
||||
public IRRegistry {
|
||||
types = copy(types);
|
||||
implementations = copy(implementations);
|
||||
}
|
||||
public static IRRegistry empty() { return new IRRegistry(null, null); }
|
||||
private static <T> ReadOnlyList<T> copy(ReadOnlyList<T> values) {
|
||||
return values == null ? ReadOnlyList.empty() : ReadOnlyList.wrap(values.asList());
|
||||
}
|
||||
public record TypeRef(String declarationKey) {
|
||||
public TypeRef {
|
||||
Objects.requireNonNull(declarationKey, "declarationKey");
|
||||
if (declarationKey.isBlank()) throw new IllegalArgumentException("empty type declaration key");
|
||||
}
|
||||
public static TypeRef builtin(int id) { return new TypeRef("builtin:" + id); }
|
||||
}
|
||||
public record TypeDeclaration(TypeRef identity, String name, TypeBody body) {
|
||||
public TypeDeclaration { Objects.requireNonNull(identity); Objects.requireNonNull(body); }
|
||||
}
|
||||
public sealed interface TypeBody permits Builtin, Struct, Array, Trait, Fn {}
|
||||
public record Builtin(int id) implements TypeBody {
|
||||
public Builtin {
|
||||
switch (id) {
|
||||
case 1, 2, 3, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16 -> { }
|
||||
default -> throw new IllegalArgumentException("unsupported BuiltinId: " + id);
|
||||
}
|
||||
}
|
||||
}
|
||||
public record Struct(ReadOnlyList<NamedType> fields) implements TypeBody {
|
||||
public Struct { fields = copy(fields); }
|
||||
}
|
||||
public record Array(TypeRef element) implements TypeBody {
|
||||
public Array { Objects.requireNonNull(element); }
|
||||
}
|
||||
public record Trait(ReadOnlyList<Method> methods) implements TypeBody {
|
||||
public Trait { methods = copy(methods); }
|
||||
}
|
||||
public record Fn(Signature signature) implements TypeBody {
|
||||
public Fn { Objects.requireNonNull(signature); }
|
||||
}
|
||||
public record NamedType(TypeRef type, String name) {
|
||||
public NamedType { Objects.requireNonNull(type); }
|
||||
}
|
||||
public record Signature(ReadOnlyList<NamedType> parameters, ReadOnlyList<TypeRef> results) {
|
||||
public Signature { parameters = copy(parameters); results = copy(results); }
|
||||
public static Signature empty() { return new Signature(null, null); }
|
||||
}
|
||||
public record Method(String name, Signature signature) {
|
||||
public Method { Objects.requireNonNull(signature); }
|
||||
}
|
||||
public record Implementation(String declarationKey, TypeRef owner, TypeRef trait, ReadOnlyList<CallableId> methods) {
|
||||
public Implementation {
|
||||
Objects.requireNonNull(declarationKey);
|
||||
if (declarationKey.isBlank()) throw new IllegalArgumentException("empty implementation key");
|
||||
Objects.requireNonNull(owner); Objects.requireNonNull(trait);
|
||||
methods = copy(methods);
|
||||
}
|
||||
}
|
||||
public enum Role {
|
||||
NONE(0), BEFORE_PAUSE(1), BEFORE_RESUME(2), MEMCARD_BOOT(3);
|
||||
private final int wire;
|
||||
Role(int wire) { this.wire = wire; }
|
||||
public int wire() { return wire; }
|
||||
}
|
||||
public record FunctionMetadata(Signature signature, ReadOnlyList<TypeRef> captures, Role role, String name) {
|
||||
public FunctionMetadata(Signature signature, ReadOnlyList<TypeRef> captures, Role role) {
|
||||
this(signature, captures, role, null);
|
||||
}
|
||||
public FunctionMetadata {
|
||||
Objects.requireNonNull(signature); captures = copy(captures); Objects.requireNonNull(role);
|
||||
}
|
||||
public static FunctionMetadata ordinary(Signature signature) { return new FunctionMetadata(signature, null, Role.NONE); }
|
||||
public static FunctionMetadata empty() { return ordinary(Signature.empty()); }
|
||||
}
|
||||
}
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
package p.studio.compiler.models.registry;
|
||||
|
||||
import p.studio.compiler.source.identifiers.CallableId;
|
||||
import java.util.Objects;
|
||||
|
||||
/** Semantic Registry operands before image linking. No frontend syntax is exposed. */
|
||||
public record IRRegistryInstruction(Kind kind, IRRegistry.TypeRef type, IRRegistry.TypeRef targetType,
|
||||
CallableId function, int member) {
|
||||
public enum Kind {
|
||||
MAKE_CLOSURE(0x52), CALL_CLOSURE(0x53), GET_CAPTURE(0x57), CALL_TRAIT(0x58),
|
||||
DYN_PACK(0x90), DYN_UNPACK(0x91), DYN_CAN_UNPACK(0x92),
|
||||
REF_IS_EXACT(0x93), REF_IMPLEMENTS(0x94), REF_TO_TRAIT(0x95), REF_CAST(0x96),
|
||||
STRUCT_NEW(0xa0), STRUCT_GET(0xa1), STRUCT_SET(0xa2),
|
||||
ARRAY_NEW(0xa3), ARRAY_LEN(0xa4), ARRAY_GET(0xa5), ARRAY_SET(0xa6);
|
||||
private final int opcode;
|
||||
Kind(int opcode) { this.opcode=opcode; }
|
||||
public int opcode(){return opcode;}
|
||||
}
|
||||
public IRRegistryInstruction {
|
||||
Objects.requireNonNull(kind);
|
||||
if(member<0)throw new IllegalArgumentException("negative member index");
|
||||
if(kind!=Kind.GET_CAPTURE)Objects.requireNonNull(type,"type");
|
||||
if(kind==Kind.MAKE_CLOSURE) {Objects.requireNonNull(function,"function");function.getIndex();}
|
||||
else if(function!=null)throw new IllegalArgumentException("unexpected function operand");
|
||||
boolean pair=kind.opcode()>=0x93&&kind.opcode()<=0x96;
|
||||
if(pair)Objects.requireNonNull(targetType,"targetType");
|
||||
else if(targetType!=null)throw new IllegalArgumentException("unexpected target type");
|
||||
if(kind==Kind.GET_CAPTURE&&type!=null)throw new IllegalArgumentException("capture index is not a TypeId");
|
||||
}
|
||||
}
|
||||
+39
@@ -1,6 +1,8 @@
|
||||
package p.studio.compiler.models;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import p.studio.compiler.models.registry.IRRegistry;
|
||||
import p.studio.compiler.models.registry.IRRegistryInstruction;
|
||||
import p.studio.compiler.source.Span;
|
||||
import p.studio.compiler.source.identifiers.CallableId;
|
||||
import p.studio.compiler.source.identifiers.FileId;
|
||||
@@ -25,6 +27,26 @@ import java.util.function.Function;
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
class IRBackendExecutableContractTest {
|
||||
@Test
|
||||
void registryAggregationRemapsImplementationCallablesWithoutChangingTypeIdentity() {
|
||||
var owner = new IRRegistry.TypeRef("shared:owner");
|
||||
var trait = new IRRegistry.TypeRef("shared:trait");
|
||||
var types = ReadOnlyList.from(new IRRegistry.TypeDeclaration(owner, "Owner", new IRRegistry.Struct(null)),
|
||||
new IRRegistry.TypeDeclaration(trait, "Trait", new IRRegistry.Trait(null)));
|
||||
var aggregator = IRBackend.aggregator();
|
||||
for (int file = 0; file < 2; file++) {
|
||||
var registry = new IRRegistry(types, ReadOnlyList.from(new IRRegistry.Implementation(
|
||||
"implementation:" + file, owner, trait, ReadOnlyList.from(new CallableId(0)))));
|
||||
aggregator.merge(new IRBackendFile(new FileId(file), null, null, null, null, null, null,
|
||||
ReadOnlyList.from(new CallableSignatureRef("method" + file, 0, "() -> unit")), null, null, registry));
|
||||
}
|
||||
var registry = aggregator.emit().getRegistry();
|
||||
assertEquals(4, registry.types().size()); // Linking, not aggregation, canonicalizes declarations.
|
||||
assertEquals(registry.types().get(0), registry.types().get(2));
|
||||
assertEquals(new CallableId(0), registry.implementations().get(0).methods().getFirst());
|
||||
assertEquals(new CallableId(1), registry.implementations().get(1).methods().getFirst());
|
||||
assertEquals(owner, registry.implementations().get(1).owner());
|
||||
}
|
||||
private static final String FORBIDDEN_PBS_PACKAGE = "p.studio.compiler.pbs";
|
||||
private static final Set<Class<?>> AUDITED_SUPPORT_TYPES = Set.of(
|
||||
FileId.class,
|
||||
@@ -41,6 +63,23 @@ class IRBackendExecutableContractTest {
|
||||
Long.class,
|
||||
Boolean.class);
|
||||
private static final List<Class<?>> PUBLIC_CONTRACT_TYPES = List.of(
|
||||
IRRegistry.class,
|
||||
IRRegistryInstruction.class,
|
||||
IRRegistryInstruction.Kind.class,
|
||||
IRRegistry.TypeRef.class,
|
||||
IRRegistry.TypeDeclaration.class,
|
||||
IRRegistry.TypeBody.class,
|
||||
IRRegistry.Builtin.class,
|
||||
IRRegistry.Struct.class,
|
||||
IRRegistry.Array.class,
|
||||
IRRegistry.Trait.class,
|
||||
IRRegistry.Fn.class,
|
||||
IRRegistry.NamedType.class,
|
||||
IRRegistry.Signature.class,
|
||||
IRRegistry.Method.class,
|
||||
IRRegistry.Implementation.class,
|
||||
IRRegistry.Role.class,
|
||||
IRRegistry.FunctionMetadata.class,
|
||||
IRBackend.class,
|
||||
IRBackend.IRBackendAggregator.class,
|
||||
IRBackendReader.class,
|
||||
|
||||
@@ -7,8 +7,12 @@ public final class AutomationException extends IllegalStateException {
|
||||
private final AutomationFailure failure;
|
||||
|
||||
public AutomationException(AutomationFailure.Code code, String message) {
|
||||
super(message);
|
||||
failure = new AutomationFailure(Objects.requireNonNull(code), message);
|
||||
this(new AutomationFailure(Objects.requireNonNull(code), message));
|
||||
}
|
||||
|
||||
public AutomationException(AutomationFailure failure) {
|
||||
super(Objects.requireNonNull(failure).message());
|
||||
this.failure = failure;
|
||||
}
|
||||
|
||||
public AutomationFailure failure() { return failure; }
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
package p.studio.api.automation;
|
||||
|
||||
import java.util.Objects;
|
||||
import java.util.Map;
|
||||
import java.util.List;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Collections;
|
||||
|
||||
/** Stable domain failure codes, independent of HTTP status and JSON-RPC errors. */
|
||||
public record AutomationFailure(Code code, String message) {
|
||||
public record AutomationFailure(Code code, String message, Map<String, Object> runtimeError) {
|
||||
public AutomationFailure(Code code, String message) { this(code, message, Map.of()); }
|
||||
public enum Code {
|
||||
PROJECT_NOT_OPEN, NOT_GRANTED, INVALID_TASK, BUSY, CLOSED, CLEANUP_FAILED,
|
||||
UNSUPPORTED_LANGUAGE, UNSUPPORTED_CAPABILITY, REFERENCE_NOT_FOUND, DOCUMENTATION_UNAVAILABLE,
|
||||
@@ -11,5 +16,19 @@ public record AutomationFailure(Code code, String message) {
|
||||
RUNTIME_UNAVAILABLE, RUNTIME_INCOMPATIBLE, SESSION_NOT_FOUND, BUILD_NOT_FOUND,
|
||||
NO_FRAME_AVAILABLE, LIMIT_EXCEEDED, NOT_CANCELABLE, INFRASTRUCTURE_FAILURE
|
||||
}
|
||||
public AutomationFailure { Objects.requireNonNull(code); Objects.requireNonNull(message); }
|
||||
public AutomationFailure {
|
||||
Objects.requireNonNull(code); Objects.requireNonNull(message);
|
||||
var copy = new LinkedHashMap<String, Object>();
|
||||
if (runtimeError != null) runtimeError.forEach((key, value) -> copy.put(key, freeze(value)));
|
||||
runtimeError = Collections.unmodifiableMap(copy);
|
||||
}
|
||||
private static Object freeze(Object value) {
|
||||
if (value instanceof Map<?, ?> map) {
|
||||
var copy = new LinkedHashMap<Object, Object>();
|
||||
map.forEach((key, item) -> copy.put(key, freeze(item)));
|
||||
return Collections.unmodifiableMap(copy);
|
||||
}
|
||||
if (value instanceof List<?> list) return list.stream().map(AutomationFailure::freeze).toList();
|
||||
return value;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# Studio PBX producer conformance
|
||||
|
||||
Owner: compiler/general. This harness uses the production runtime loader and VM;
|
||||
it does not implement a Java approximation of runtime admission.
|
||||
|
||||
From the Studio root:
|
||||
|
||||
```sh
|
||||
./gradlew :prometeu-compiler:prometeu-build-pipeline:test --tests '*LinkedRegistryTest' --tests '*PbxRegistryConformanceTest' --tests '*TypedCommonPipelineTest'
|
||||
python3 tools/pbx-conformance/run.py --runtime ../runtime
|
||||
```
|
||||
|
||||
The Java tests generate PBX in the build-pipeline module's
|
||||
`build/registry-conformance`. `run.py` creates a temporary Cargo project, copies
|
||||
the runtime lockfile and references its bytecode/hal/vm/system crates by path.
|
||||
It never modifies the runtime checkout. Add `--online` for an uncached machine;
|
||||
by default Cargo uses only cached dependencies. Java 25, Rust and the runtime
|
||||
checkout's dependencies are required.
|
||||
|
||||
The output identifies runtime revision, OS/architecture and load configuration.
|
||||
Positive fixtures distinguish loading from execution. The vec2 signature fixture
|
||||
is deliberately load-only because its entry requires arguments. Native binding
|
||||
uses SYSTEM for the positive case and NONE for missing-capability rejection;
|
||||
all other cases use NONE. Resource tests explicitly lower default limits.
|
||||
|
||||
Negative fixtures mutate Java-produced bytes independently of the writer to
|
||||
exercise malformed input that the writer correctly refuses to construct.
|
||||
Initialization JSON comes from `prometeu_system::CrashReport`, and is checked
|
||||
against the runtime's published headless schema. Saved JSON files remain under
|
||||
the generated fixture directory. Studio process tests replay selected observed
|
||||
payloads from `prometeu-app/src/test/resources/runtime` and assert unchanged MCP
|
||||
projection, including null executed PC and nested verifier context.
|
||||
|
||||
## Known upstream schema defect
|
||||
|
||||
At runtime `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`, the LoadReason Rust enum
|
||||
includes LifecycleRole, LifecycleSignature, DuplicateLifecycleRole and
|
||||
NonFiniteValue, absent from the published JSON schema enum. This corpus
|
||||
observes LifecycleRole and NonFiniteValue. The harness prints an explicit
|
||||
`UPSTREAM SCHEMA GAP` for those two reasons and verifies that adding exactly
|
||||
the missing enum value makes the original payload valid. It never alters the
|
||||
payload or reports those cases as conforming to the unmodified schema. Any
|
||||
other schema failure remains fatal. Track the runtime correction in AGD75;
|
||||
remove the exception once the schema is corrected.
|
||||
|
||||
This corpus qualifies common backend production, not PBS lowering, lifecycle
|
||||
selection, memcard persistence or installed-runtime deployment.
|
||||
@@ -0,0 +1,357 @@
|
||||
use prometeu_bytecode::Value;
|
||||
use prometeu_bytecode::load_limits::{LoadBudget, LoadLimits};
|
||||
use prometeu_hal::syscalls::caps;
|
||||
use prometeu_vm::{
|
||||
HostContext, HostReturn, NativeInterface, ValidatedProgramImage, VirtualMachine,
|
||||
};
|
||||
|
||||
struct HasCart(usize);
|
||||
impl NativeInterface for HasCart {
|
||||
fn syscall(
|
||||
&mut self,
|
||||
id: u32,
|
||||
args: &[Value],
|
||||
ret: &mut HostReturn,
|
||||
_: &mut HostContext,
|
||||
) -> Result<(), prometeu_hal::vm_fault::VmFault> {
|
||||
assert_eq!(id, 1);
|
||||
assert!(args.is_empty());
|
||||
self.0 += 1;
|
||||
ret.push_bool(false);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
struct NoNative;
|
||||
impl NativeInterface for NoNative {
|
||||
fn syscall(
|
||||
&mut self,
|
||||
_: u32,
|
||||
_: &[Value],
|
||||
_: &mut HostReturn,
|
||||
_: &mut HostContext,
|
||||
) -> Result<(), prometeu_hal::vm_fault::VmFault> {
|
||||
panic!("fixture unexpectedly invoked native code")
|
||||
}
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let root = std::path::PathBuf::from(std::env::args().nth(1).expect("fixture directory"));
|
||||
for name in [
|
||||
"minimal",
|
||||
"jump-end",
|
||||
"vec2-signature",
|
||||
"closure-capture",
|
||||
"trait-call",
|
||||
"array-dynamic",
|
||||
"common-pipeline",
|
||||
"roles",
|
||||
"join",
|
||||
"memcard-role",
|
||||
"intrinsic-dot",
|
||||
] {
|
||||
let bytes = std::fs::read(root.join(format!("{name}.pbx"))).unwrap();
|
||||
let image = ValidatedProgramImage::load(
|
||||
&bytes,
|
||||
&mut LoadBudget::new(LoadLimits::default()),
|
||||
caps::NONE,
|
||||
)
|
||||
.unwrap_or_else(|e| panic!("{name}: {e:?}"));
|
||||
println!("LOAD PASS {name}");
|
||||
if name != "vec2-signature" {
|
||||
let mut vm = VirtualMachine::from_image(image, Default::default());
|
||||
vm.prepare_boot_call().unwrap();
|
||||
let report = vm
|
||||
.run_budget(1000, &mut NoNative, &mut HostContext::new(None))
|
||||
.unwrap();
|
||||
let reason = format!("{:?}", report.reason);
|
||||
assert!(
|
||||
reason == "Halted" || reason == "EndOfRom",
|
||||
"{name}: {report:?}"
|
||||
);
|
||||
if name == "closure-capture" || name == "trait-call" {
|
||||
assert!(matches!(vm.operand_stack_top(1)[0], Value::Int32(42)));
|
||||
}
|
||||
if name == "intrinsic-dot" {
|
||||
assert!(
|
||||
matches!(vm.operand_stack_top(1)[0], Value::Float(value) if value.to_bits() == 11f64.to_bits())
|
||||
);
|
||||
}
|
||||
if name == "array-dynamic" {
|
||||
assert!(matches!(vm.operand_stack_top(1)[0], Value::Int32(0)));
|
||||
}
|
||||
println!(
|
||||
"EXEC PASS {name}: {reason}, {} steps",
|
||||
report.steps_executed
|
||||
);
|
||||
}
|
||||
}
|
||||
for name in ["native", "unknown-native", "native-abi", "unused-native"] {
|
||||
let bytes = std::fs::read(root.join(format!("{name}.pbx"))).unwrap();
|
||||
if name == "native" {
|
||||
let image = ValidatedProgramImage::load(
|
||||
&bytes,
|
||||
&mut LoadBudget::new(LoadLimits::default()),
|
||||
caps::SYSTEM,
|
||||
)
|
||||
.unwrap();
|
||||
let mut vm = VirtualMachine::from_image(image, Default::default());
|
||||
vm.prepare_boot_call().unwrap();
|
||||
let mut native = HasCart(0);
|
||||
let report = vm
|
||||
.run_budget(1000, &mut native, &mut HostContext::new(None))
|
||||
.unwrap();
|
||||
assert_eq!(native.0, 1);
|
||||
assert_eq!(format!("{:?}", report.reason), "Halted");
|
||||
println!("BIND + EXEC PASS native with SYSTEM");
|
||||
}
|
||||
let capabilities = if name == "native" {
|
||||
caps::NONE
|
||||
} else {
|
||||
caps::SYSTEM
|
||||
};
|
||||
let error = ValidatedProgramImage::load(
|
||||
&bytes,
|
||||
&mut LoadBudget::new(LoadLimits::default()),
|
||||
capabilities,
|
||||
)
|
||||
.unwrap_err();
|
||||
record_fault(&root, name, error);
|
||||
}
|
||||
let baseline = std::fs::read(root.join("minimal.pbx")).unwrap();
|
||||
for (name, offset, value) in [
|
||||
("version", 4, 1),
|
||||
("unsupported-section", 116, 9),
|
||||
("unknown-builtin", 178, 0),
|
||||
("invalid-role", 136, 4),
|
||||
("retired-opcode", 162, 0x54),
|
||||
] {
|
||||
let mut bytes = baseline.clone();
|
||||
bytes[offset] = value;
|
||||
reject(&root, name, &bytes, LoadLimits::default());
|
||||
}
|
||||
for (name, code, max) in [
|
||||
("raw-syscall", vec![0x70, 0, 1, 0, 0, 0, 1, 0], 0),
|
||||
("hostcall-index", vec![0x71, 0, 0, 0, 0, 0, 1, 0], 0),
|
||||
(
|
||||
"retired-spawn",
|
||||
vec![0x54, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0],
|
||||
0,
|
||||
),
|
||||
("retired-yield", vec![0x55, 0, 1, 0], 0),
|
||||
("retired-sleep", vec![0x56, 0, 0, 0, 0, 0, 1, 0], 0),
|
||||
(
|
||||
"unreachable-reference",
|
||||
vec![1, 0, 0x50, 0, 255, 255, 255, 255],
|
||||
0,
|
||||
),
|
||||
(
|
||||
"nonfinite",
|
||||
vec![1, 0, 0x15, 0, 0, 0, 0, 0, 0, 0, 0xf0, 0x7f],
|
||||
0,
|
||||
),
|
||||
("jump-immediate", vec![2, 0, 3, 0, 0, 0, 1, 0], 0),
|
||||
("zero-max", vec![0x17, 0, 0, 0, 0, 0, 1, 0], 0),
|
||||
("return-height", vec![0x17, 0, 0, 0, 0, 0, 0x51, 0], 1),
|
||||
("fallthrough", vec![0, 0], 0),
|
||||
] {
|
||||
let mut functions = section(&baseline, 1).to_vec();
|
||||
functions[18..22].copy_from_slice(&(code.len() as u32).to_le_bytes());
|
||||
functions[26..30].copy_from_slice(&(max as u32).to_le_bytes());
|
||||
let bytes = replace(&replace(&baseline, 1, &functions), 2, &code);
|
||||
reject(&root, name, &bytes, LoadLimits::default());
|
||||
}
|
||||
// Two mutually recursive Array declarations: a structural cycle, with valid IDs.
|
||||
reject(
|
||||
&root,
|
||||
"array-cycle",
|
||||
&replace(
|
||||
&baseline,
|
||||
6,
|
||||
&[2, 0, 0, 0, 2, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0],
|
||||
),
|
||||
LoadLimits::default(),
|
||||
);
|
||||
reject(
|
||||
&root,
|
||||
"structural-cycle",
|
||||
&replace(
|
||||
&baseline,
|
||||
6,
|
||||
&[
|
||||
2, 0, 0, 0, 4, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0,
|
||||
],
|
||||
),
|
||||
LoadLimits::default(),
|
||||
);
|
||||
reject(
|
||||
&root,
|
||||
"invalid-type-reference",
|
||||
&replace(&baseline, 6, &[1, 0, 0, 0, 2, 0, 99, 0, 0, 0]),
|
||||
LoadLimits::default(),
|
||||
);
|
||||
reject(
|
||||
&root,
|
||||
"invalid-utf8",
|
||||
&replace(
|
||||
&baseline,
|
||||
6,
|
||||
&[1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 255, 0x13, 0, 0, 0],
|
||||
),
|
||||
LoadLimits::default(),
|
||||
);
|
||||
reject(
|
||||
&root,
|
||||
"removed-input",
|
||||
&replace(&baseline, 6, &[1, 0, 0, 0, 0, 0, 0, 1, 0, 0]),
|
||||
LoadLimits::default(),
|
||||
);
|
||||
let mut overlap = baseline.clone();
|
||||
overlap[36..40].copy_from_slice(&0u32.to_le_bytes());
|
||||
reject(&root, "overlap", &overlap, LoadLimits::default());
|
||||
reject(
|
||||
&root,
|
||||
"truncated",
|
||||
&baseline[..baseline.len() - 1],
|
||||
LoadLimits::default(),
|
||||
);
|
||||
let mut duplicate = baseline.clone();
|
||||
duplicate[44..48].copy_from_slice(&0u32.to_le_bytes());
|
||||
reject(
|
||||
&root,
|
||||
"duplicate-section",
|
||||
&duplicate,
|
||||
LoadLimits::default(),
|
||||
);
|
||||
reject(
|
||||
&root,
|
||||
"empty-entry",
|
||||
&replace(&replace(&baseline, 1, &[0, 0, 0, 0]), 8, &[0, 0, 0, 0]),
|
||||
LoadLimits::default(),
|
||||
);
|
||||
reject(
|
||||
&root,
|
||||
"resource-types",
|
||||
&baseline,
|
||||
LoadLimits {
|
||||
types: 0,
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
reject(
|
||||
&root,
|
||||
"resource-scratch",
|
||||
&baseline,
|
||||
LoadLimits {
|
||||
temporary_bytes: 0,
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
reject(
|
||||
&root,
|
||||
"resource",
|
||||
&baseline,
|
||||
LoadLimits {
|
||||
work: 0,
|
||||
..Default::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
fn reject(root: &std::path::Path, name: &str, bytes: &[u8], limits: LoadLimits) {
|
||||
let error =
|
||||
ValidatedProgramImage::load(bytes, &mut LoadBudget::new(limits), caps::NONE).unwrap_err();
|
||||
record_fault(root, name, error);
|
||||
}
|
||||
fn record_fault(root: &std::path::Path, name: &str, error: prometeu_vm::VmInitError) {
|
||||
if let prometeu_vm::VmInitError::RegistryLoadFailed(ref failure) = error {
|
||||
let expected = match name {
|
||||
"array-cycle" => Some("ArrayCycle"),
|
||||
"structural-cycle" => Some("StructuralTypeCycle"),
|
||||
"invalid-type-reference" => Some("TypeId"),
|
||||
"invalid-utf8" => Some("Utf8"),
|
||||
"removed-input" => Some("Builtin"),
|
||||
"retired-spawn" | "retired-yield" | "retired-sleep" => Some("RemovedOpcode"),
|
||||
"nonfinite" => Some("NonFiniteValue"),
|
||||
"unreachable-reference" => Some("FunctionId"),
|
||||
"return-height" => Some("ReturnHeight"),
|
||||
"fallthrough" => Some("UnterminatedPath"),
|
||||
"zero-max" => Some("StackOverflow"),
|
||||
"duplicate-section" => Some("DuplicateSection"),
|
||||
_ => None,
|
||||
};
|
||||
if let Some(reason) = expected {
|
||||
assert_eq!(format!("{:?}", failure.reason), reason, "{name}");
|
||||
}
|
||||
}
|
||||
let report = prometeu_system::CrashReport::VmInit { error };
|
||||
assert_eq!(report.pc(), None);
|
||||
let fault = serde_json::json!({"kind":"vm_init", "pc":null, "summary":"Conformance fixture", "initError":report.init_diagnostic().unwrap()});
|
||||
let full: serde_json::Value = serde_json::from_slice(
|
||||
&std::fs::read(std::env::args().nth(2).expect("schema path")).unwrap(),
|
||||
)
|
||||
.unwrap();
|
||||
let schema = jsonschema::validator_for(
|
||||
&serde_json::json!({"$defs":full["$defs"],"$ref":"#/$defs/fault"}),
|
||||
)
|
||||
.unwrap();
|
||||
if !schema.is_valid(&fault) {
|
||||
let reason = fault["initError"]["load"]["reason"].as_str().unwrap();
|
||||
assert!(
|
||||
["LifecycleRole", "NonFiniteValue"].contains(&reason),
|
||||
"unexpected schema failure: {fault}"
|
||||
);
|
||||
// Validate the ORIGINAL payload with only the observed missing enum value added.
|
||||
// This is an explicit upstream defect report, not a claim of published-schema conformance.
|
||||
let mut corrected = full.clone();
|
||||
corrected["$defs"]["loadFailure"]["properties"]["reason"]["enum"]
|
||||
.as_array_mut()
|
||||
.unwrap()
|
||||
.push(serde_json::json!(reason));
|
||||
jsonschema::validator_for(
|
||||
&serde_json::json!({"$defs":corrected["$defs"],"$ref":"#/$defs/fault"}),
|
||||
)
|
||||
.unwrap()
|
||||
.validate(&fault)
|
||||
.unwrap();
|
||||
println!(
|
||||
"UPSTREAM SCHEMA GAP: {reason} missing from published enum; original payload preserved"
|
||||
);
|
||||
}
|
||||
std::fs::write(
|
||||
root.join(format!("fault-{name}.json")),
|
||||
serde_json::to_vec_pretty(&fault).unwrap(),
|
||||
)
|
||||
.unwrap();
|
||||
println!("REJECT PASS {name}: {}", fault["initError"]);
|
||||
}
|
||||
|
||||
fn word(bytes: &[u8], at: usize) -> usize {
|
||||
u32::from_le_bytes(bytes[at..at + 4].try_into().unwrap()) as usize
|
||||
}
|
||||
fn section(bytes: &[u8], kind: usize) -> &[u8] {
|
||||
for i in 0..word(bytes, 8) {
|
||||
let at = 32 + 12 * i;
|
||||
if word(bytes, at) == kind {
|
||||
let offset = word(bytes, at + 4);
|
||||
return &bytes[offset..offset + word(bytes, at + 8)];
|
||||
}
|
||||
}
|
||||
panic!("missing section {kind}")
|
||||
}
|
||||
fn replace(bytes: &[u8], kind: usize, payload: &[u8]) -> Vec<u8> {
|
||||
let count = word(bytes, 8);
|
||||
let mut output = bytes[..32 + 12 * count].to_vec();
|
||||
for i in 0..count {
|
||||
let at = 32 + 12 * i;
|
||||
let body = if word(bytes, at) == kind {
|
||||
payload
|
||||
} else {
|
||||
section(bytes, word(bytes, at))
|
||||
};
|
||||
let offset = output.len() as u32;
|
||||
output[at + 4..at + 8].copy_from_slice(&offset.to_le_bytes());
|
||||
output[at + 8..at + 12].copy_from_slice(&(body.len() as u32).to_le_bytes());
|
||||
output.extend_from_slice(body);
|
||||
}
|
||||
output
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Build a production-loader harness without writing into the runtime checkout."""
|
||||
import argparse
|
||||
import json
|
||||
import pathlib
|
||||
import platform
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--runtime", type=pathlib.Path, default=pathlib.Path("../runtime"))
|
||||
parser.add_argument("--fixtures", type=pathlib.Path, default=pathlib.Path("prometeu-compiler/prometeu-build-pipeline/build/registry-conformance"))
|
||||
parser.add_argument("--online", action="store_true", help="Allow Cargo to fetch uncached dependencies")
|
||||
args = parser.parse_args()
|
||||
runtime = args.runtime.resolve()
|
||||
print("runtime", subprocess.check_output(["git", "-C", str(runtime), "rev-parse", "HEAD"], text=True).strip(), flush=True)
|
||||
print("platform", platform.platform(), "limits=default capabilities=NONE", flush=True)
|
||||
with tempfile.TemporaryDirectory(prefix="pbx-conformance-") as directory:
|
||||
root = pathlib.Path(directory)
|
||||
manifest = '[package]\nname="studio-pbx-conformance"\nversion="0.1.0"\nedition="2024"\n[dependencies]\n'
|
||||
for crate in ["prometeu-bytecode", "prometeu-hal", "prometeu-vm", "prometeu-system"]:
|
||||
manifest += f'{crate}={{path={json.dumps(str(runtime / "crates/console" / crate))}}}\n'
|
||||
manifest += 'serde_json="1.0.149"\njsonschema={version="0.33",default-features=false}\n'
|
||||
(root / "Cargo.toml").write_text(manifest)
|
||||
(root / "Cargo.lock").write_text((runtime / "Cargo.lock").read_text())
|
||||
(root / "src").mkdir()
|
||||
(root / "src/main.rs").write_text(pathlib.Path(__file__).with_name("main.rs").read_text())
|
||||
subprocess.run(["cargo", "run", *([] if args.online else ["--offline"]), "--manifest-path", str(root / "Cargo.toml"), "--", str(args.fixtures.resolve()), str(runtime / "devtools/headless-protocol/response.schema.json")], check=True)
|
||||
Reference in New Issue
Block a user