feat(pbs): add mutable lists and stable snapshot cursors

Implement SDK list<T> and Cursor<T> with trusted compiler bindings and generic storage publication across IR and PBX. Pin the runtime provider and include specs, conformance coverage, installed examples, and DSC-0090 lessons.
This commit is contained in:
2026-10-23 00:46:39 +01:00
parent 76a8b074e8
commit c410e35001
46 changed files with 2683 additions and 165 deletions
+66 -47
View File
@@ -2,7 +2,7 @@ Prometeu VM Runtime — Canonical Architecture
Status: canonical
This document is the authoritative architectural reference for the Prometeu VM runtime. It reflects the implementation as it exists today and defines the invariants that govern architectural changes in the VM layer.
This document is the authoritative architectural reference for the Prometeu VM runtime. It defines the invariants that govern architectural changes in the VM layer.
Scope boundary:
@@ -14,8 +14,8 @@ Scope boundary:
Document roles:
- This file is normative for VM/runtime architecture.
- Detailed domain specifications may live under `docs/runtime/specs/`, but they must not contradict this document where VM/runtime invariants are concerned.
- Roadmaps, agendas, and PR proposals may discuss future changes, but they are not authoritative until this document is updated.
- Detailed domain specifications may live under `docs/specs/runtime/`, but they must not contradict this document where VM/runtime invariants are concerned.
- Future architectural changes become authoritative when this document is updated.
- The machine-wide fantasy console framing lives in the runtime specs manual and related domain specs; those documents are complementary, not competing VM architecture sources.
Maintenance rule:
@@ -30,10 +30,10 @@ Maintenance rule:
- Operand stack + call frames; bytecode is fetched from a ROM/program image with a separate constant pool.
- GC‑managed heap
- Non‑compacting mark–sweep collector; stable object handles (`HeapRef`) while live. Sweep invalidates unreachable handles; objects are never moved.
- Closures (Model B)
- First‑class closures with a heap‑allocated environment. The closure object is passed to the callee as a hidden `arg0` when invoking a closure.
- Cooperative coroutines
- Deterministic, cooperative scheduling. Switching and GC occur only at explicit safepoints (`FRAME_SYNC`).
- Typed first-class Fn
- Structural logical parameter/result signatures and heap Closure environments retained by CallFrames, without hidden arg0.
- Single execution context
- VM directly owns PC, stacks and globals; no heap Coroutine or guest scheduler. GC opportunities are FRAME_SYNC and successful last-root RET after result rooting.
- Unified syscall ABI
- PBX pre-load artifacts declare canonical host bindings in `SYSC` and encode call sites as `HOSTCALL <sysc_index>`. The loader resolves and patches them to numeric `SYSCALL <id>` before verification/execution. Capability gating is enforced at load and checked again defensively at runtime. Syscalls are not first‑class values.
@@ -48,29 +48,28 @@ Maintenance rule:
- The VM’s current operand stack and frames are GC roots.
- Heap
- The heap stores runtime objects that require identity and reachability tracking. Handles are `HeapRef` indices into an internal object store.
- The heap stores runtime objects that require identity and reachability tracking. Handles are `HeapRef { index: u32, generation: u32 }` values into an internal object store. Every access checks the live generation; reclaimed maximum-generation slots are retired permanently.
- The collector is mark–sweep, non‑moving: it marks from roots, then reclaims unreachable objects without relocating survivors. Indices for live objects remain stable across collections.
2.2 Heap Object Kinds (as used today)
- Arrays of `Value`
- Variable‑length arrays whose elements may contain further `HeapRef`s.
- Typed arrays
- Variable-length arrays use the Registry element layout and defaults. Primitive/builtin values are stored directly; Struct/Array elements contain generational references. Native String/Dynamic slots retain their ownership rules. Stack/local execution continues to use `Value`.
- Closures
- Carry a function identifier and a captured environment (a slice/vector of `Value`s stored with the closure). Captured `HeapRef`s are traversed by the GC.
- Coroutines
- Heap‑resident coroutine records (state + wake time + suspended operand stack and call frames). These act as GC roots when suspended.
- Structs and Dynamic boxes
- Typed payloads use resolved reference/ownership maps. Private Struct cells can share mutable captured bindings.
Notes:
- Literals like strings and numbers are sourced from the constant pool in the program image; heap allocation is only used for runtime objects (closures, arrays, coroutine records, and any future heap kinds). The constant pool never embeds raw `HeapRef`s.
- Literals like strings and numbers are sourced from the constant pool in the program image; heap allocation is only used for runtime objects (closures, arrays, Structs and Dynamic boxes). The constant pool never embeds raw `HeapRef`s.
2.3 GC Roots
- VM roots
- Current operand stack and call frames of the running coroutine (or main context).
- Suspended coroutines
- All heap‑resident, suspended coroutine objects are treated as roots. Their saved stacks/frames are scanned during marking.
- Root traversal
- The VM exposes a root‑visitor that walks the operand stack, frames, and coroutine records to feed the collector. The collector then follows children from each object kind (e.g., array elements, closure environments, coroutine stacks).
- Roots comprise globals, all live operand values, Closure owners in every active
CallFrame and legally retained native/host roots. Inspection snapshots are not
guest roots. Tracing visits complete typed fields, arrays, captures and cells.
- Root results are transferred/rooted before the last frame is removed or GC runs,
and remain owned until consumed/replaced/reset/unloaded. Unreachable cycles collect.
3. Execution Model
@@ -85,20 +84,22 @@ Notes:
3.2 Safepoints
- `FRAME_SYNC` is the only safepoint.
- At `FRAME_SYNC`, the VM performs two actions in a well‑defined order:
1) Garbage‑collection opportunity: root enumeration + mark–sweep.
2) Scheduler handoff: the currently running coroutine may yield/sleep, and a next ready coroutine is selected deterministically.
- No other opcode constitutes a GC or scheduling safepoint. Syscalls do not implicitly trigger GC or rescheduling.
- Threshold GC opportunities are FRAME_SYNC and successful RET of the last root,
after result rooting. Ordinary callee RET, entry preparation, syscalls and
allocation/quota refusal do not collect. GC adds no guest callback, clock/frame
advance, budget grant or partial sweep on scratch failure.
3.3 Scheduler Behavior (Cooperative Coroutines)
3.3 Single Context and Host Continuation
- Coroutines are cooperative and scheduled deterministically (FIFO among ready coroutines).
- `YIELD` and `SLEEP` take effect at `FRAME_SYNC`:
- `YIELD` places the current coroutine at the end of the ready queue.
- `SLEEP` parks the current coroutine until its exact `wake_tick`, after which it re‑enters the ready queue at the correct point.
- `SPAWN` creates a new coroutine with its own stack/frames recorded in the heap and enqueues it deterministically.
- No preemption: the VM never interrupts a coroutine between safepoints.
- FRAME_SYNC may occur in any body, has no stack effect and resumes next PC with
live frames/locals/environments. It closes the logical frame but does not restart
entry. Budget exhaustion/breakpoints preserve the same open logical frame.
- Last-root RET completes entry. Recurring host prepares FunctionId 0 next frame,
preserving globals/init guard; preparation validates/reserves before replacing
previous owners and allocates no Coroutine. HALT/trap/reset/unload retain their
contracts. Render/asset workers remain independent of guest execution.
- SPAWN/YIELD/SLEEP are retired: numbers 0x54–0x56 stay reserved and all uses reject
at load, including unreachable code. No compatibility scheduler remains.
4. Verification Model
@@ -119,25 +120,33 @@ The verifier statically checks bytecode for structural safety and stack‑shape
- Syscalls
- The verifier runs only on the patched executable image. `HOSTCALL` is invalid at verification time. Final `SYSCALL` IDs must exist per `SyscallMeta`, and arity/declared return slot counts must match metadata.
- Closures
- `CALL_CLOSURE` is only allowed on closure values; the callee function must be known; argument counts for closure calls must match.
- Coroutines
- `YIELD` context must be valid; `SPAWN` argument counts are validated.
- Expected FnTypeId determines complete parameter/result groups even when the concrete FunctionId is unknown. Known mismatch rejects; Unknown retains runtime checks. Capture indices and target metadata are checked, and direct entry into capturing bodies rejects.
- Trait calls
- Exact method signatures and complete views resolve through immutable tables; concrete receiver is the first callee parameter, with no capture environment.
4.2 Runtime vs Verifier Guarantees
- The verifier guarantees structural correctness and stack‑shape invariants. It does not perform full type checking of value contents; dynamic checks (e.g., numeric domain checks, polymorphic comparisons, concrete syscall argument validation) occur at runtime and may trap.
- The Registry verifier checks structure and stack shape and conservatively tracks known types through control flow. It rejects proved incompatibilities; Unknown values retain runtime checks at typed boundaries. Numeric domain checks, polymorphic comparisons and concrete syscall argument validation may still trap at runtime.
- Capability gating for syscalls is enforced at load from cartridge capability flags and checked again at runtime by the VM/native interface.
5. Closures (Model B) — Calling Convention
-------------------------------------------
5. Typed Fn and Environments
----------------------------
- Creation
- `MAKE_CLOSURE` captures N values from the operand stack into a heap‑allocated environment alongside a function identifier. The opcode _pushes a `HeapRef` to the new closure.
- Call
- `CALL_CLOSURE` invokes a closure. The closure object itself is supplied to the callee as a hidden `arg0`. User‑visible arguments follow the function’s declared arity.
- Access to captures
- The callee can access captured values via the closure’s environment. Captured `HeapRef`s are traced by the GC.
- Fn identity is ordered logical parameter/result TypeIds; names, target and captures
are metadata. Pure Array/Fn structural cycles reject; nominal anchors stop expansion.
- MAKE_CLOSURE takes FunctionId/FnTypeId, captures full declared groups and pushes a
distinct one-slot instance. Immutable bindings copy values; mutable bindings can
share private typed Struct cells. Metadata lives in required PBX section 8.
- CALL_CLOSURE takes expected FnTypeId and retains the Closure in CallFrame, without
public arg0. GET_CAPTURE takes an index and copies a complete group. Direct calls
to capturing bodies reject; bound receivers use explicit adapter bodies.
- Fn is valid in Struct/Array/Dynamic, defaults Null and compares instance identity.
It gains no REF_* casts, Trait implementations, fault isolation or separate budget.
- Validate/reserve before consume/publication; faults preserve input owners and expose
no partial frame/object. Costs are 8+C / 6 / 2+W including traps, charged once.
- CALL_TRAIT takes TraitTypeId/method index, consumes explicit args and a complete
view, then passes concrete receiver first. Exact immutable-table dispatch costs 6.
6. Unified Syscall ABI
@@ -160,11 +169,11 @@ The verifier statically checks bytecode for structural safety and stack‑shape
- Collector
- Non‑moving mark–sweep.
- Triggers
- GC runs only at `FRAME_SYNC` safepoints.
- Threshold GC runs at FRAME_SYNC and successful last-root RET after result rooting.
- Liveness
- Roots comprise: the live VM stack/frames and all suspended coroutines. The collector traverses object‑specific children (array elements, closure environments, coroutine stacks).
- Globals, live operand values and active-frame Closures root complete typed object graphs, including cells and legally retained host/native owners.
- Determinism
- GC opportunities and scheduling order are tied to `FRAME_SYNC`, ensuring repeatable execution traces across runs with the same inputs.
- Opportunities depend on execution boundaries and thresholds, preserving deterministic guest execution with the same inputs.
8. Non‑Goals
@@ -180,6 +189,16 @@ The verifier statically checks bytecode for structural safety and stack‑shape
--------------------------
- Keep the public surface minimal and metadata‑driven (e.g., syscalls via `SyscallMeta`).
- Do not assume implicit safepoints; schedule and GC only at `FRAME_SYNC`.
- Preserve explicit GC opportunities and root results before root-return collection.
- When adding new opcodes or object kinds, extend the verifier and GC traversal accordingly (children enumeration, environment scanning, root sets).
- Update this document alongside any architectural change that affects runtime invariants.
## Program Registry
The [Program Registry contract](../specs/runtime/02c-program-registry-and-pbx-v0.md) defines immutable image-owned Type/Function metadata linked by dense IDs. Descriptors are native metadata outside guest GC; nominal identity does not depend on names. Stack/local execution retains Value, while typed storage uses resolved aligned layouts. PBX v0 is the sole target format; publication requires full bounded validation. Fn tag 4 and mandatory captures section 8 extend the Registry; no old-schema fallback is permitted.
`ValidatedProgramImage::load` publishes a shared image only after bounded decoding, native binding and Registry verification; `VirtualMachine::from_image` creates sessions that consume its typed signatures, constants and conversion instructions. Production and internal generators must use this sole sealed-v1 path. Typed Fn integration and legacy removal are implemented together; publication alone is not proof that every consumer has migrated. [Registry Boundaries and Owned Values](../../discussion/lessons/DSC-0058-program-registry-type-and-function-metadata/LSN-0059-registry-boundaries-and-owned-values.md) explains the published mechanisms. The [producer handoff](registry-producer-handoff.md) documents schema, input IDs and named-argument order, while [layout evidence](registry-layout-evidence.md) records reproducible load/access/GC measurements and the current publication boundary.
## VM-owned random core
See the [random contract](../specs/runtime/02e-vm-owned-random-core.md) for Random BuiltinId 0x0016, ObjectKind tag 9, intrinsics 0x4000–0x4003, atomic state mutation, bounded 4+8*N costs and rejection-limit trap 0x00000015. Random is managed by the VM heap and existing GC, without host entropy or PBX changes.