feat(vm)!: implement typed callables and single-context execution
Intrepid/Prometeu/Runtime/pipeline/pr-master This commit looks good

Add structural Fn signatures, typed capture environments, indirect calls and Trait dispatch while preserving logical parameter and result groups, ownership and budgets.

Remove guest coroutine execution and legacy loading. Activate bounded sealed PBX v1 with mandatory capture metadata, preserve frame continuations and root return results, and migrate producers and shipped cartridges.

Publish coordinated specs, handoffs and the completed runtime lesson. Validation: 880 workspace tests passed, 9 existing tests ignored; both cartridges completed six production-host frames.
This commit is contained in:
2026-10-09 05:47:24 +01:00
parent 86bade3e1f
commit ec84a6bba5
96 changed files with 3656 additions and 6075 deletions
@@ -0,0 +1,177 @@
---
id: LSN-0061
ticket: vm-closures-captures-and-dynamic-calls
title: Typed Callables and Frame-Owned Continuations
created: 2026-10-08
tags: [vm, bytecode, closures, ownership, callbacks]
---
## Context
DEC-0047 established typed first-class function values, explicit capture metadata,
trait dispatch and a single directly owned VM execution context. Its runtime
delivery also activated the existing sealed Registry provider in production and
migrated internal producers. This lesson records the implemented result of
DSC-0052; the workflow scaffolding is disposable, while these explanations and the
[discussion index](../../index.ndjson) persist.
The normative references are [function values and closures](../../../docs/specs/runtime/02b-vm-function-values-and-closures.md),
[Registry and PBX v1](../../../docs/specs/runtime/02c-program-registry-and-pbx-v1.md),
[frame continuation](../../../docs/specs/runtime/09a-coroutines-and-cooperative-scheduling.md)
and [memory ownership](../../../docs/specs/runtime/03-memory-stack-heap-and-allocation.md).
The earlier [Registry lesson](../DSC-0058-program-registry-type-and-function-metadata/LSN-0059-registry-boundaries-and-owned-values.md)
explains the provider foundation; its historical production handoff is now
implemented in this runtime.
## Key Decisions
### Logical signatures survive flattening
A function value's structural identity includes both ordered parameter TypeIds
and ordered result TypeIds. Names, concrete FunctionId, capture types and physical
slot counts do not define that identity. Consequently, one Vec2 result and two
Float results remain distinct even though each occupies two execution slots.
Returning two callbacks likewise means two logical Fn results, each carried by
one handle. Tuple flattening needs to retain the logical lists before deriving
widths; counting slots cannot recover the signature afterward.
The Registry rejects duplicate Fn keys and structural cycles through Array/Fn
edges. Nominal Struct/Trait anchors end that structural traversal. Recursive
function calls, capturing another Fn and cyclic runtime objects remain useful
and legal: they are different kinds of recursion from an infinitely expanding
structural type descriptor.
### Environments belong to call frames
Every MAKE_CLOSURE creates a distinct heap instance, including an empty
environment. Copies preserve that instance's identity. The environment's declared
capture groups are separate from the public function signature; no hidden arg0
is inserted. CALL_CLOSURE consumes the caller's Fn operand and makes the active
CallFrame retain its environment. GET_CAPTURE copies a complete declared group,
while GET_LOCAL addresses only public parameters and ordinary locals.
Immutable bindings can be copied into environments. A shared mutable binding can
use a private typed Struct cell captured by reference. In the implemented factory
tests, two callbacks update/read one cell, while a second factory allocates an
independent cell. This reuses existing field access, reference maps and GC rather
than introducing a second mutation mechanism. Lexical mutability and fresh-binding
rules still belong to frontend lowering. Capturing a reference preserves object
sharing; it does not freeze the referenced object.
Bound methods can use explicit adapters that capture a receiver or complete Trait
view. CALL_TRAIT itself resolves an immutable implementation-table entry: the
caller places explicit arguments before the view; the callee receives its concrete
receiver first and receives no ImplementationId parameter or closure environment.
Capturing implementation targets are rejected because that dispatch cannot
provide their environment.
### A frame boundary differs from entry completion
VM owns its PC, operand stack, call stack and globals directly. Preparing an entry
allocates no Coroutine object. Guest coroutine instructions and ready/sleeping
queues were removed; the independent render scheduler and asset workers retain
their own protocols.
FRAME_SYNC ends a logical frame and preserves the next PC, live operands, locals
and environments. It can occur inside an ordinary callee or a closure. Preparing
boot again after this boundary would replay the caller and abandon that
continuation. Successful RET of the last root instead completes the entry, so a
recurring host can prepare FunctionId 0 on the next frame while preserving the
global init guard. Budget exhaustion preserves the current open frame and does
not submit another visual packet or give callbacks an extra budget.
These distinctions let consumers advance persistent operations through short
ordinary frame steps and call Fn values when appropriate. The runtime does not
need suspended coroutine instances to represent callbacks. UI/tween consumers
and native callback registration/delivery need their own integration contracts;
this delivery created neither API.
## Patterns and Algorithms
The sealed image resolves ordered group layouts and capture offsets once. PBX v1
adds Fn tag 4 and requires captures section 8, including explicit empty lists for
noncapturing bodies. The function-table encoding stays unchanged. Mandatory
metadata provides a deterministic rejection boundary for older images rather
than guessing captures from public argument widths. Internal producers and both
shipped cartridges now use the same bounded writer and production loader.
Callable boundaries validate complete groups and reserve capacity before
consuming operands or publishing a frame/object. Their attempt costs are 8+C
for creation, 6 for an indirect call, 2+W for capture access and 6 for Trait dispatch.
Faults retain the charged attempt cost and preserve callable source operands and
owners. A positive slice admits one whole instruction even when its cost exceeds
the remaining budget; budget zero admits no instruction. Allocation refusal does
not trigger rescue GC.
GC roots include every live operand/global and the Closure retained by every
active frame. Environments trace whole groups, including Trait/Dynamic components,
references, shared cells and String owners. Successful root return transfers
results to the operand stack before threshold collection. Those results remain
roots until consumption, replacement, reset or unload. Threshold opportunities
are FRAME_SYNC and successful last-root RET; an ordinary callee return or entry
preparation adds no collection event. Unreachable closure/cell cycles are collected
by the existing mark-sweep algorithm.
An inspection ValueSnapshot can retain native String content, but copying a raw
HeapRef does not establish a guest root or authorize cross-session import.
Compact handle bits cannot identify foreign provenance if numerical handles
collide. Trusted native callers must preserve session provenance; the runtime
does not gain a generic Fn retention/delivery API from inspection snapshots.
## Pitfalls
- Comparing physical widths loses logical argument/result identity.
- Adding captures to public parameters makes function types depend on lowering
details and shifts ordinary local addressing.
- Rooting only the caller's Fn operand loses an active environment after that
operand is consumed. Rooting frame environments fixes its full lifetime.
- Restarting entry after FRAME_SYNC loses nested continuations; retaining context
after completed root RET fails to implement recurring entry execution.
- Treating callback calls as another scheduling phase changes frame/budget
semantics. These calls are ordinary execution in the same context.
- Keeping an unchecked fixture constructor for valid programs leaves a second
execution authority. Valid fixtures now use official sealed images; test-only
corrupted-state injection remains limited to defensive failure probes.
## Published Evidence
The final `cargo test --workspace` run passed 880 tests with zero failures and
9 existing ignored tests. It exercised the bytecode, VM, HAL, System, firmware,
layer, both host, generator and CLI packages. The ignored cases are three explicit
layout/load/ownership measurement probes, five desktop localhost transport tests
and one packaged-installation smoke requiring an extracted distribution archive.
None is claimed as executed. `cargo fmt --all -- --check`, `git diff --check` and
`discussion validate` passed; edited documentation and published lesson links
were checked for local targets and workflow references were removed from docs.
Discriminating evidence includes:
- Registry v1: 37 passed / 1 ignored, independent 193-byte minimal module,
independent Fn/capture payload bytes, mandatory-section failures, logical
signature identity and finite structural graph checks. Registry ISA: 7 passed.
- VM callable unit probes: 13 passed, including attempt costs, zero budget,
overshoot, quota rollback, active/result roots, mixed capture ownership,
malformed Trait views, bounded recursive calls and unreachable cycles.
- VM `typed_closures`: 13 passed, including unknown selected targets, Fn
pass/return/Struct/Array/Dynamic storage, independent factories, bound adapters,
direct captured-target rejection and 3000 recurring entries with zero execution
heap allocations under a zero handle quota.
- Layer tuple-returned callbacks and frame continuation: one passing case each.
Existing typed collection/native/string/reference cases also pass through the
sealed execution path.
- Headless `typed_runtime`: 5 passed. Nested callee/closure sync retains local 73
and init count 1 across six frames; a long nested Fn spans budget slices without
opening another frame. Native type and null-Fn traps plus removed-opcode load
rejection retain machine context. Wire tests additionally cover executed closure
quota refusal and an active native type fault.
- Both generator binaries regenerated their program.pbx files. Dummy Boy and
Stress each complete six production-host frames with progressing visuals;
direct sealed-image side-effect observation confirms six frame-command
executions and exactly one Stress scene bind. Manifests/assets are unchanged.
This is bounded functional regression evidence, not long-run soak or game
headroom certification. The [producer handoff](../../../docs/vm-arch/registry-producer-handoff.md)
names external compiler targets and acceptance cases: exact v1 metadata/bytes,
logical signatures, named-argument evaluation order, capture lowering, callable
behavior and structured failures. Studio/FE adaptation remains consumer work and
has not been executed by this runtime delivery.