feat(vm)!: implement typed callables and single-context execution
Intrepid/Prometeu/Runtime/pipeline/pr-master This commit looks good
Intrepid/Prometeu/Runtime/pipeline/pr-master This commit looks good
Add structural Fn signatures, typed capture environments, indirect calls and Trait dispatch while preserving logical parameter and result groups, ownership and budgets. Remove guest coroutine execution and legacy loading. Activate bounded sealed PBX v1 with mandatory capture metadata, preserve frame continuations and root return results, and migrate producers and shipped cartridges. Publish coordinated specs, handoffs and the completed runtime lesson. Validation: 880 workspace tests passed, 9 existing tests ignored; both cartridges completed six production-host frames.
This commit is contained in:
+177
@@ -0,0 +1,177 @@
|
||||
---
|
||||
id: LSN-0061
|
||||
ticket: vm-closures-captures-and-dynamic-calls
|
||||
title: Typed Callables and Frame-Owned Continuations
|
||||
created: 2026-10-08
|
||||
tags: [vm, bytecode, closures, ownership, callbacks]
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
DEC-0047 established typed first-class function values, explicit capture metadata,
|
||||
trait dispatch and a single directly owned VM execution context. Its runtime
|
||||
delivery also activated the existing sealed Registry provider in production and
|
||||
migrated internal producers. This lesson records the implemented result of
|
||||
DSC-0052; the workflow scaffolding is disposable, while these explanations and the
|
||||
[discussion index](../../index.ndjson) persist.
|
||||
|
||||
The normative references are [function values and closures](../../../docs/specs/runtime/02b-vm-function-values-and-closures.md),
|
||||
[Registry and PBX v1](../../../docs/specs/runtime/02c-program-registry-and-pbx-v1.md),
|
||||
[frame continuation](../../../docs/specs/runtime/09a-coroutines-and-cooperative-scheduling.md)
|
||||
and [memory ownership](../../../docs/specs/runtime/03-memory-stack-heap-and-allocation.md).
|
||||
The earlier [Registry lesson](../DSC-0058-program-registry-type-and-function-metadata/LSN-0059-registry-boundaries-and-owned-values.md)
|
||||
explains the provider foundation; its historical production handoff is now
|
||||
implemented in this runtime.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
### Logical signatures survive flattening
|
||||
|
||||
A function value's structural identity includes both ordered parameter TypeIds
|
||||
and ordered result TypeIds. Names, concrete FunctionId, capture types and physical
|
||||
slot counts do not define that identity. Consequently, one Vec2 result and two
|
||||
Float results remain distinct even though each occupies two execution slots.
|
||||
Returning two callbacks likewise means two logical Fn results, each carried by
|
||||
one handle. Tuple flattening needs to retain the logical lists before deriving
|
||||
widths; counting slots cannot recover the signature afterward.
|
||||
|
||||
The Registry rejects duplicate Fn keys and structural cycles through Array/Fn
|
||||
edges. Nominal Struct/Trait anchors end that structural traversal. Recursive
|
||||
function calls, capturing another Fn and cyclic runtime objects remain useful
|
||||
and legal: they are different kinds of recursion from an infinitely expanding
|
||||
structural type descriptor.
|
||||
|
||||
### Environments belong to call frames
|
||||
|
||||
Every MAKE_CLOSURE creates a distinct heap instance, including an empty
|
||||
environment. Copies preserve that instance's identity. The environment's declared
|
||||
capture groups are separate from the public function signature; no hidden arg0
|
||||
is inserted. CALL_CLOSURE consumes the caller's Fn operand and makes the active
|
||||
CallFrame retain its environment. GET_CAPTURE copies a complete declared group,
|
||||
while GET_LOCAL addresses only public parameters and ordinary locals.
|
||||
|
||||
Immutable bindings can be copied into environments. A shared mutable binding can
|
||||
use a private typed Struct cell captured by reference. In the implemented factory
|
||||
tests, two callbacks update/read one cell, while a second factory allocates an
|
||||
independent cell. This reuses existing field access, reference maps and GC rather
|
||||
than introducing a second mutation mechanism. Lexical mutability and fresh-binding
|
||||
rules still belong to frontend lowering. Capturing a reference preserves object
|
||||
sharing; it does not freeze the referenced object.
|
||||
|
||||
Bound methods can use explicit adapters that capture a receiver or complete Trait
|
||||
view. CALL_TRAIT itself resolves an immutable implementation-table entry: the
|
||||
caller places explicit arguments before the view; the callee receives its concrete
|
||||
receiver first and receives no ImplementationId parameter or closure environment.
|
||||
Capturing implementation targets are rejected because that dispatch cannot
|
||||
provide their environment.
|
||||
|
||||
### A frame boundary differs from entry completion
|
||||
|
||||
VM owns its PC, operand stack, call stack and globals directly. Preparing an entry
|
||||
allocates no Coroutine object. Guest coroutine instructions and ready/sleeping
|
||||
queues were removed; the independent render scheduler and asset workers retain
|
||||
their own protocols.
|
||||
|
||||
FRAME_SYNC ends a logical frame and preserves the next PC, live operands, locals
|
||||
and environments. It can occur inside an ordinary callee or a closure. Preparing
|
||||
boot again after this boundary would replay the caller and abandon that
|
||||
continuation. Successful RET of the last root instead completes the entry, so a
|
||||
recurring host can prepare FunctionId 0 on the next frame while preserving the
|
||||
global init guard. Budget exhaustion preserves the current open frame and does
|
||||
not submit another visual packet or give callbacks an extra budget.
|
||||
|
||||
These distinctions let consumers advance persistent operations through short
|
||||
ordinary frame steps and call Fn values when appropriate. The runtime does not
|
||||
need suspended coroutine instances to represent callbacks. UI/tween consumers
|
||||
and native callback registration/delivery need their own integration contracts;
|
||||
this delivery created neither API.
|
||||
|
||||
## Patterns and Algorithms
|
||||
|
||||
The sealed image resolves ordered group layouts and capture offsets once. PBX v1
|
||||
adds Fn tag 4 and requires captures section 8, including explicit empty lists for
|
||||
noncapturing bodies. The function-table encoding stays unchanged. Mandatory
|
||||
metadata provides a deterministic rejection boundary for older images rather
|
||||
than guessing captures from public argument widths. Internal producers and both
|
||||
shipped cartridges now use the same bounded writer and production loader.
|
||||
|
||||
Callable boundaries validate complete groups and reserve capacity before
|
||||
consuming operands or publishing a frame/object. Their attempt costs are 8+C
|
||||
for creation, 6 for an indirect call, 2+W for capture access and 6 for Trait dispatch.
|
||||
Faults retain the charged attempt cost and preserve callable source operands and
|
||||
owners. A positive slice admits one whole instruction even when its cost exceeds
|
||||
the remaining budget; budget zero admits no instruction. Allocation refusal does
|
||||
not trigger rescue GC.
|
||||
|
||||
GC roots include every live operand/global and the Closure retained by every
|
||||
active frame. Environments trace whole groups, including Trait/Dynamic components,
|
||||
references, shared cells and String owners. Successful root return transfers
|
||||
results to the operand stack before threshold collection. Those results remain
|
||||
roots until consumption, replacement, reset or unload. Threshold opportunities
|
||||
are FRAME_SYNC and successful last-root RET; an ordinary callee return or entry
|
||||
preparation adds no collection event. Unreachable closure/cell cycles are collected
|
||||
by the existing mark-sweep algorithm.
|
||||
|
||||
An inspection ValueSnapshot can retain native String content, but copying a raw
|
||||
HeapRef does not establish a guest root or authorize cross-session import.
|
||||
Compact handle bits cannot identify foreign provenance if numerical handles
|
||||
collide. Trusted native callers must preserve session provenance; the runtime
|
||||
does not gain a generic Fn retention/delivery API from inspection snapshots.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Comparing physical widths loses logical argument/result identity.
|
||||
- Adding captures to public parameters makes function types depend on lowering
|
||||
details and shifts ordinary local addressing.
|
||||
- Rooting only the caller's Fn operand loses an active environment after that
|
||||
operand is consumed. Rooting frame environments fixes its full lifetime.
|
||||
- Restarting entry after FRAME_SYNC loses nested continuations; retaining context
|
||||
after completed root RET fails to implement recurring entry execution.
|
||||
- Treating callback calls as another scheduling phase changes frame/budget
|
||||
semantics. These calls are ordinary execution in the same context.
|
||||
- Keeping an unchecked fixture constructor for valid programs leaves a second
|
||||
execution authority. Valid fixtures now use official sealed images; test-only
|
||||
corrupted-state injection remains limited to defensive failure probes.
|
||||
|
||||
## Published Evidence
|
||||
|
||||
The final `cargo test --workspace` run passed 880 tests with zero failures and
|
||||
9 existing ignored tests. It exercised the bytecode, VM, HAL, System, firmware,
|
||||
layer, both host, generator and CLI packages. The ignored cases are three explicit
|
||||
layout/load/ownership measurement probes, five desktop localhost transport tests
|
||||
and one packaged-installation smoke requiring an extracted distribution archive.
|
||||
None is claimed as executed. `cargo fmt --all -- --check`, `git diff --check` and
|
||||
`discussion validate` passed; edited documentation and published lesson links
|
||||
were checked for local targets and workflow references were removed from docs.
|
||||
|
||||
Discriminating evidence includes:
|
||||
|
||||
- Registry v1: 37 passed / 1 ignored, independent 193-byte minimal module,
|
||||
independent Fn/capture payload bytes, mandatory-section failures, logical
|
||||
signature identity and finite structural graph checks. Registry ISA: 7 passed.
|
||||
- VM callable unit probes: 13 passed, including attempt costs, zero budget,
|
||||
overshoot, quota rollback, active/result roots, mixed capture ownership,
|
||||
malformed Trait views, bounded recursive calls and unreachable cycles.
|
||||
- VM `typed_closures`: 13 passed, including unknown selected targets, Fn
|
||||
pass/return/Struct/Array/Dynamic storage, independent factories, bound adapters,
|
||||
direct captured-target rejection and 3000 recurring entries with zero execution
|
||||
heap allocations under a zero handle quota.
|
||||
- Layer tuple-returned callbacks and frame continuation: one passing case each.
|
||||
Existing typed collection/native/string/reference cases also pass through the
|
||||
sealed execution path.
|
||||
- Headless `typed_runtime`: 5 passed. Nested callee/closure sync retains local 73
|
||||
and init count 1 across six frames; a long nested Fn spans budget slices without
|
||||
opening another frame. Native type and null-Fn traps plus removed-opcode load
|
||||
rejection retain machine context. Wire tests additionally cover executed closure
|
||||
quota refusal and an active native type fault.
|
||||
- Both generator binaries regenerated their program.pbx files. Dummy Boy and
|
||||
Stress each complete six production-host frames with progressing visuals;
|
||||
direct sealed-image side-effect observation confirms six frame-command
|
||||
executions and exactly one Stress scene bind. Manifests/assets are unchanged.
|
||||
|
||||
This is bounded functional regression evidence, not long-run soak or game
|
||||
headroom certification. The [producer handoff](../../../docs/vm-arch/registry-producer-handoff.md)
|
||||
names external compiler targets and acceptance cases: exact v1 metadata/bytes,
|
||||
logical signatures, named-argument evaluation order, capture lowering, callable
|
||||
behavior and structured failures. Studio/FE adaptation remains consumer work and
|
||||
has not been executed by this runtime delivery.
|
||||
Reference in New Issue
Block a user