Files
prometeu-runtime/docs/vm-arch/collection-producer-handoff.md
bquarkz dcf6561ea7
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
refactor(vm)!: remove publication and logical guard opcodes
Remove INDEX_GUARD, ARRAY_CHECK_LENGTH, STORAGE_PUBLISH and FIELDS_PUBLISH
from decoding, metadata, tooling, verification and execution. Keep ARRAY_LEN
and overlap-safe ARRAY_COPY with complete-group ownership and early charging
of calculable copy work, including refused attempts.

Update specifications, producer handoff, qualification evidence and lessons.
Share collection test support and remove redundant Copy clones for Clippy.

Validation: make ci-domains passed with 1,095 workspace tests, five separately
executed debugger socket tests and all 15 unchanged coverage domain gates.
VM release, resource diagnostics and layer/quality suites also passed.

BREAKING CHANGE: PBX using opcodes 0xA7, 0xA8, 0xAA or 0xAB is rejected,
including unreachable instructions. Consumers must adapt their emission and
collection algorithms or operational guarantees.
2026-10-23 23:37:56 +01:00

9.4 KiB

Typed collection producer and consumer handoff

The collection ISA, memory contract and attempt charging are normative. This interface has executable evidence through locally serialized PBX v0, ValidatedProgramImage::load and VirtualMachine::from_image. Production initialize, internal test producers and both example generators use the same sealed v0 path. External SDK/frontend adaptation remains a separate consumer obligation.

Encoding and complete groups

Opcodes are little-endian u16, followed by little-endian u32 immediates. TypeId is the receiver descriptor, independent of BuiltinId and element TypeId. Struct field indices are immediate, owner-local declaration positions. Array lengths/indices and LEN results use exact Int64 carriers; length must fit u32. References are exact nominal/invariant Registry types. There is no implicit conversion or array covariance.

Operation Code Payload Stack bottom → top
STRUCT_NEW 0xA0 TypeId [] → ref
STRUCT_GET 0xA1 TypeId, field ref → complete field group
STRUCT_SET 0xA2 TypeId, field ref, complete field group → []
ARRAY_NEW 0xA3 TypeId Int64 length → ref
ARRAY_LEN 0xA4 TypeId ref → Int64 length
ARRAY_GET 0xA5 TypeId ref, Int64 index → complete element group
ARRAY_SET 0xA6 TypeId ref, Int64 index, complete element group → []
ARRAY_COPY 0xA9 TypeId source, sourceStart, target, targetStart, count → []

The single-immediate forms are six bytes; field forms are ten. Width is one for scalar/String/color/concrete references, two for vec2/Trait/Dynamic, three for pixel. A Trait contains receiver then ImplementationId; Dynamic contains TypeId then payload. Preserve the whole group, including absent [Null, Null]. ImplementationId zero is valid. GET does not implicitly box; Dynamic copies share immutable boxes. NEW initializes canonical defaults without constructing referenced instances. Arrays have fixed lengths. SDKs implement List, Set and Map algorithms using these runtime capabilities.

Evaluate source expressions under the language's ordering rules into temporaries, then load receiver, index (if applicable) and components in Registry order. Runtime validation does not define source expression evaluation order: it checks receiver first, index carrier/bounds next, and a SET group last.

Default-then-store construction oracle

With TypeId 0 = Int32 and TypeId 1 = Struct{value: TypeId 0}, the frontend can emit:

STRUCT_NEW 1
SET_GLOBAL 0
GET_GLOBAL 0
PUSH_I32 42
STRUCT_SET type=1, field=0
GET_GLOBAL 0
STRUCT_GET type=1, field=0
HALT

The default value before the store is Int32 zero; the final value is Int32 42. A language-specific constructor can be called after rooting/storing under its own rules. NEW does not call it. Global/local slots used by this example remain the producer's responsibility.

With TypeId 11 = Array, independent code bytes for length two followed by LEN and HALT are:

14 00 02 00 00 00 00 00 00 00
A3 00 0B 00 00 00
A4 00 0B 00 00 00
01 00

The final stack is [Int64(2)]. Generic independent immediate oracles use T=0x01020304 and f=0x05060708: NEW is A0 00 04 03 02 01; GET is A1 00 04 03 02 01 08 07 06 05; SET starts A2 00 with that same payload. Array NEW/LEN/GET/SET start A3 00, A4 00, A5 00, A6 00, each with 04 03 02 01. Their descriptor tables must actually contain the referenced IDs for execution; these large-ID bytes are encoding oracles only.

Costs, budgets and faults

Full costs are NEW 8 plus initialized execution slots, GET 3 plus value width, SET 4 plus value width, LEN 3. Struct sums are resolved/cached at load. ARRAY_NEW with valid Int64 N in 0..u32::MAX charges 8+N*w(E), including allocation refusal; unavailable/wrong/negative/above-u32 length charges eight. Every decoded attempt charges once, including traps. OpCode::cycles() is only the base for collection operations.

Ten vec2 elements cost 28; pixel GET/SET cost 6/7; empty construction costs 8. With positive remaining budget, the whole instruction executes, possibly overshooting. Budget zero does nothing. run_budget(1) at ARRAY_NEW ten vec2 with continuation returns BudgetExhausted, 28 cycles, one successful step; refused construction returns Trap, 28 cycles, zero successful target steps. No intra-instruction GC/tick/switch occurs.

Static bad IDs/categories/fields reject load with typed LOAD_INVALID_REFERENCE context; proven heights/types reject with LOAD_INVALID_CODE. Runtime faults retain attempted opcode/PC/span: incomplete temporary operands use STACK_UNDERFLOW; absence NULL_REFERENCE 0x0E; invalid/dead/stale handle INVALID_HANDLE 0x0F; wrong carrier/category/Registry/exact type/group TYPE 0x04; negative/out-of-range index or negative length OOB 0x03; above-u32 length, checked overflow and quotas/allocation refusal RESOURCE_LIMIT 0x10. Resource reason/requested/limit remain attached. Failed attempts preserve source operands and previous complete storage/owners, publish no partial handle and release provisional resources without refunding cycles.

Consumer integration and evidence

Consumers must recognize all eight encodings, invoke Registry-derived effects in bounded structural/height/type verification and use validated-image execution. The metadata-free verifier and legacy decoder have been removed. Reuse compact typed storage, default initialization, native ownership maps and generational handles; do not replace them with raw Value arrays or SDK container implementations.

Executable oracles live in bytecode tests/registry_isa.rs, VM tests/collection_conformance.rs, VM virtual_machine/collections_tests.rs and layer tests/typed_collections.rs. They cover independent bytes, default/alias/group operations, attempt costs and refusals, malformed/proven types, ownership and actual FRAME_SYNC graph reclamation. VM unit tests inject otherwise unreachable invalid state only inside tests. Optimized ownership/conformance runs complement debug evidence.

Dummy Boy and Stress emit sealed PBX v0, including empty captures sections, and run through the production host. Fn is a one-slot nullable HeapRef value in fields and arrays; Dynamic can carry it with its exact Fn TypeId. Closure environments and shared private cells participate in existing reference maps. Default construction never invokes callbacks. Threshold GC runs at FRAME_SYNC and successful last-root RET after rooting results. Quota refusal cannot invoke rescue GC.

The published patterns and validation limits are explained in Collection Attempts Preserve Groups and Charge Work Before Publication.

Range-copy migration contract

ARRAY_LEN stays 0xA4. ARRAY_COPY is six bytes: A9 00 04 03 02 01 for encoding oracle TypeId 0x01020304. Actual execution needs an existing exact Array descriptor. The five operands use Int64 offsets/count and exact Array receivers. Physical ranges are strict; an empty range may start at length. Copy snapshots complete source groups before writing, preserving tags, native owners and identity under overlap without boxing. Reserve complete TemporaryBytes scratch first; release it on success/failure. Failed attempts preserve operands/storage; no request token, guest object, callback, GC or scheduler occurs inside the instruction.

For available five temporary operands and exact Int64 count N in 0..u32::MAX, precharge representable C=4+2NW before receiver/range/group validation or scratch reservation. Retain it on refusal: N=10/W=1 costs 24 for success, bad receiver, bad range or scratch quota. Other counts/underflow cost base 4 with normal fault order. Checked cost overflow costs 4 and traps SizeOverflow before ordinary validation. Empty copies validate receivers/ranges and cost 4; aliases give no discount. See the copy contract for exact precedence. Positive budget admits the full attempt with overshoot; Trap wins on refusal and only successful steps count. Zero budget does no work.

Encodings 0xA7, 0xA8, 0xAA and 0xAB are unsupported in actual instruction positions, including dead code. They are not remapped or emulated. Producers must remove their emission, adapt logical checks/error paths and re-evaluate algorithms, representations and collection guarantees. Sequential stores do not preserve the former multi-owner publication guarantee. Prepare fallible values before mutation and explicitly define the remaining source contract.

Compiler/SDK migration, provider pinning/catalog import and source/optimized/ installed-resource conformance belong to the consumer. Prior List/Linked evidence qualifies earlier providers only. Identity/hash catalog schema 2 import is a separate consumer dependency, not a storage or new-collection implementation here. See virtual_machine/array_copy_tests.rs and collection/Registry conformance for current executable oracles. Qualification provenance is recorded in storage qualification; an uncommitted implementation must not be attributed to an earlier committed provider SHA.