docs(pbs): consolidate collection lessons and failure guarantees

This commit is contained in:
2026-10-27 13:40:38 +00:00
parent 0016d92ac3
commit eb7d6a2398
3 changed files with 326 additions and 118 deletions
@@ -0,0 +1,163 @@
---
id: LSN-0096
ticket: pbs-sdk-collections
title: Shared foundations and separate collection contracts
created: 2026-10-27
tags: [compiler-pbs, sdk, collections, umbrella, navigation, failure-phases]
---
# Shared foundations and separate collection contracts
**Worker:** Lesson Worker. **Owner domain:** `compiler/pbs`.
DSC-0087 / AGD89 coordinated eight completed child discussions. Their accepted
decisions/plans remain the sources of technical choices, consolidated in the
specifications and lessons below. This umbrella closes coordination and the
previously authorized editorial reconciliation; it introduces no new collection
contract or implementation scope.
## Original problem and resolution
Array syntax, generic language mechanisms, identity policies and several collection
APIs were initially mixed into one discussion. A monolithic decision would have
made prerequisites, product choices and execution gates hard to distinguish.
The user chose an umbrella with separate domain-owned children, ordered as shown
below, then added generic structs as a prerequisite and Stack/Queue as one further
child. Order of execution did not prescribe one collection's implementation over
another.
The result is a qualified source SDK catalog over shared language/storage
mechanisms. Each child completed its own agenda → decision → plan → implementation
→ lesson flow. The final umbrella task reconciled List/Linked lessons with the
already accepted sequential-storage migration and repaired their provider links.
Original evidence files retain their original bytes and provider provenance.
## Reading and ownership map
Read the foundations first, then the collection needed by the caller. Specifications
are normative; lessons teach the choices and retain implementation/evidence context.
| Sequence / discussion | Durable lesson | Normative specification |
| --- | --- | --- |
| 1. Array, DSC-0088 | [LSN-0087: contextual parametric arrays](../DSC-0088-pbs-array-type-syntax/LSN-0087-parametric-arrays-and-contextual-construction.md) | [Array/optional](../../../docs/compiler/pbs/specs/typed-arrays-and-optional-execution.md) |
| 2. Generic contracts, DSC-0089 | [LSN-0088: concrete contracts](../DSC-0089-pbs-generic-contracts/LSN-0088-concrete-generic-contracts-from-source-to-vm.md) | [Generic contracts](../../../docs/compiler/pbs/specs/generic-contracts.md) |
| 3. Generic structs/functions, DSC-0094 | [LSN-0089: concrete state/bodies](../DSC-0094-pbs-generic-structs/LSN-0089-concrete-generic-state-and-bodies.md) | [Generic structs/functions](../../../docs/compiler/pbs/specs/generic-structs-and-free-functions.md) |
| 4. List/Cursor, DSC-0090 | [LSN-0090: mutable positions/stable snapshots](../DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md) | [List/shared Cursor](../../../docs/compiler/pbs/specs/sdk-list-and-cursor.md) |
| 5. Linked, DSC-0091 | [LSN-0091: private nodes/sequential phases](../DSC-0091-pbs-sdk-linked/LSN-0091-private-linked-nodes-and-atomic-publication.md) | [Linked](../../../docs/compiler/pbs/specs/sdk-linked-and-cursor.md) |
| 6. Set/Identity, DSC-0092 | [LSN-0093: reusable identity/buckets](../DSC-0092-pbs-sdk-set/LSN-0093-reusable-identity-and-source-set-buckets.md) | [Set/Identity](../../../docs/compiler/pbs/specs/sdk-set-and-identity.md) |
| 7. Map/Keyval, DSC-0093 | [LSN-0094: presence/representative keys](../DSC-0093-pbs-sdk-map-keyval/LSN-0094-map-presence-key-identity-and-captured-associations.md) | [Map/Keyval](../../../docs/compiler/pbs/specs/sdk-map-and-keyval.md) |
| 8. Stack/Queue, DSC-0097 | [LSN-0095: dense/circular arrays](../DSC-0097-pbs-sdk-stack-queue/LSN-0095-private-array-stacks-and-circular-queues.md) | [Stack/Queue](../../../docs/compiler/pbs/specs/sdk-stack-and-queue.md) |
The supporting [LSN-0092 / DSC-0096](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md)
explains accepted DEC-0077 / PLN-0155: source checks, ordinary stores/ARRAY_COPY,
failure phases and current-provider provenance. It supersedes the early compound
publication guarantees without changing the delivered List/Linked APIs or Cursor.
Compiler/general owns neutral IR/PBX; the provider owns storage, ISA, quotas and GC.
## Similar syntax does not imply identical collection semantics
Concrete generic applications are invariant. Explicit and contextual lowercase
construction follow the accepted language rules; argument values do not infer
generic parameters. Shared object instances remain shared through collection
insertion, mutation and snapshots. No deep freezing or structural equality is
silently introduced.
| Surface | Logical content and traversal | Important distinction |
| --- | --- | --- |
| array<T> | Fixed length; positions may be empty | Physical length is not a resizable collection contract |
| list<T> | Logical positions include holes; snapshot skips holes in index order | Size and capacity differ; clearAt keeps a position while removeAt removes it |
| linked<T> | Dense private doubly linked nodes; forward/reverse snapshots | No capacity, holes or Array u32 size bound |
| set<T> | Unique members under selected Identity; unordered snapshot | Hash filters; equality resolves collisions; private buckets are not a shared universal table |
| map<K,V> | Key associations; unordered read-only Keyval snapshots | Lookup/previous-value optional describes entry existence independently of optional V |
| stack<T> | Dense array; LIFO removal/snapshot order | Initial capacity does not cap collection size |
| queue<T> | Circular array; FIFO removal/snapshot order | Wrapped growth linearizes occupied segments; dequeue does not shift values |
Array/List/Linked/Set/Stack/Queue reject directly optional elements. Map rejects a
directly optional K but admits optional V; an independent Keyval can hold optional
components. Valid objects containing optional fields and optional collections
remain distinct from those restrictions. Zero/false are present values; faults
are not absence.
Set/Map reuse Identity/hash policies and helpers, with separate private nodes and
algorithms. Eligible object keys use instance identity by default; explicit content
strategies must remain coherent while keys are stored. Hashing does not supply an
implicit structural equality or freeze mutable key content.
## One Cursor protocol, collection-specific capture
The shared Cursor captures complete values in immutable private singly linked,
non-circular nodes, independent of source slots/editable nodes. It has no public
node, live iterator, reset or removal operation. Captured references remain shared
objects; later edits to collection membership do not rewrite the captured sequence.
A nonempty Cursor begins on first; hasNext tests successor; next without successor
is a no-op; only an empty-born Cursor has absent current. Process current before
testing whether to advance. Consumed prefixes can become collectible, but final
current remains retained. List, Linked, Set, Map, Stack and Queue select their
payload/order while reusing this exact state table.
## Current failure guarantees and historical evidence
Prepare checks, complete values, return carriers and candidates before visible
mutation, then execute source-selected ordinary stores and copies. Before the
first visible write, refusal preserves the prior collection. Success restores
all invariants. A later internal/resource fault may retain earlier writes and
incomplete invariants; no rollback, safe reuse, retry or recovery is promised.
Per-instruction preservation does not establish whole-method atomicity. Budget/
debugger pauses remain resumable instruction boundaries.
The List/Linked lessons now teach this contract and link to LSN-0092. Their early
INDEX_GUARD/ARRAY_CHECK_LENGTH/STORAGE_PUBLISH/FIELDS_PUBLISH mechanisms are marked
retired or historical. Their original evidence JSONs still describe their original
provider pins and stronger publication results, not current guarantees. The old
missing provider-publication links now point to existing typed-storage/copy sources.
The historical Linked filename is retained to preserve existing references.
The current selected [catalog pin](../../../docs/vm-arch/runtime-catalog.properties)
is `dcf6561ea73446e0a3e1cf1c73eda20923b38df0`. The latest
[complete qualification record](../DSC-0097-pbs-sdk-stack-queue/evidence/README.md)
includes all existing collection corpora in debug/release and both producer paths,
1,091 standard JUnit cases with no failures/errors and 13 existing skips, plus
two passing installed-JAR tests. Those are retained implementation results;
this documentation-only closure validates workflow/links/provenance and does not
claim a new test run, benchmark, other-host qualification or game readiness.
## Candidate backlog retained at closure
These remain candidates, without accepted APIs, delivery commitments or new
discussions created by this closure:
| Candidate | Questions requiring a separate discussion |
| --- | --- |
| Deque | Two-ended API, order/capacity and whether to reuse a circular mechanism |
| Pools | Distinguish capacity from object creation; acquire/release/reset, full state, duplicate returns and stale references/generations |
| Priority queue | Priority/order/equality policy, storage and resource behavior |
| Bitset | Representation, bounds and intended operations |
| Sparse set | Sparse/dense representation, identity and removal policy |
Audio, guest lifecycle, memcard, game qualification, installed Build/Play/headless
readiness and handoffs to AGD75/AGD88 are outside this umbrella's closure, as
explicitly directed by the user. General documentation migration is a separate
discussion; the local reconciliation does not require completing that migration.
## Closure and maintenance guidance
All eight children and the supporting migration are done, with only lessons in
their workflow records. The remaining List/Linked editorial task is complete.
DSC-0087 closes through `discussion housekeep`, retaining this English navigation
lesson and preserving candidate backlog and scope boundaries. The CLI owns index
updates and workflow removal; no manual index edits or runtime writes are involved.
Avoid these maintenance errors:
- Do not turn an umbrella's delivery order into a physical implementation dependency.
- Do not reinterpret a child decision while planning another collection.
- Do not inherit capacity, optional eligibility, equality or traversal order by analogy.
- Do not teach a historical provider guarantee as current merely because its evidence passed.
- Do not edit historical evidence to make its pin/counts match later implementations.
- Do not turn backlog candidates or adjacent runtime-readiness work into accepted scope.
The CLI currently has no dedicated umbrella/dependency relationship or standalone
lesson-ID allocation command. Existing tags and durable links express relationships;
the read-only model accessor supplies the next lesson ID and housekeeping owns
registration/counter advancement. These tooling gaps remain explicit rather than
being filled with manual index conventions.
@@ -3,7 +3,7 @@ id: LSN-0090
ticket: pbs-sdk-list
title: Mutable Lists and Stable Snapshot Cursors
created: 2026-10-22
tags: [compiler-pbs, collections, snapshot-cursor, atomic-publication, runtime-conformance]
tags: [compiler-pbs, collections, snapshot-cursor, failure-phases, runtime-conformance]
---
## Original problem and final result
@@ -19,8 +19,15 @@ PBS needed a resizable collection whose existing references remain valid, togeth
with a traversal that survives later collection edits. Both now live in the
executable SDK: ordinary generic `list<T>`, the shared `Cursor<T>` contract and
private snapshot nodes. The compiler provides trusted private bindings and source
projections; generic storage instructions support complete publication in the VM.
Linked, Set and Map remain separate discussions.
projections; ordinary typed storage and ARRAY_COPY preserve complete stored groups.
This lesson was reconciled during AGD89 closure on 2026-10-27 with accepted
DEC-0077 / PLN-0155, consolidated in
[LSN-0092](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md).
That migration replaced the original guard/publication instructions and universal
whole-method refusal guarantee. The current explanation below follows the
sequential-store contract; the original evidence remains explicitly historical.
Other collection APIs are documented in their own specifications and lessons.
## Separate collection identity, logical positions and physical storage
@@ -84,44 +91,47 @@ returns 42. Current still contains 22 afterwards.
The outer optional describes whether a node exists. A producer with optional T
must preserve `some(none)` as a present node whose payload is none. A synthetic
non-List producer qualifies that distinction independently of List's element rule.
Future Linked/Set producers use Cursor<T>; Map entries use Cursor<keyval<K,V>>.
Their construction rules and ordering guarantees belong to their own discussions.
Linked/Set produce Cursor<T>; Map entries use Cursor<keyval<K,V>>, and Stack/Queue
capture in removal order. Their construction rules and ordering guarantees belong
to their own specifications. These producers reuse the same Cursor state table.
## Prepare the whole mutation before publishing
## Prepare values, then perform sequential updates
An insert without growth can shift many slots and update size. A remove also
prepares an optional result and clears its old tail. Each operation needs a single
failure boundary for all observable changes.
Logical index/length checks are executable PBS comparisons and branches ending in
the trusted private terminal TRAP_EXPLICIT binding. They are not INDEX_GUARD or
ARRAY_CHECK_LENGTH instructions. ARRAY_COPY remains a neutral complete-group,
overlap-safe transfer; STORAGE_PUBLISH and FIELDS_PUBLISH were retired by the
migration in LSN-0092. No compound publication instruction is available.
The generic provider adds four operations, carried through common IR and PBX:
For insertion with growth, allocate a private candidate and copy prefix/suffix
directly into their final positions, including holes and presence tags. Store
the new value privately, then assign backing and length in sequence. Without
growth, first prepare the inserted stored group in a private one-slot array,
shift the suffix with ARRAY_COPY, copy the prepared slot and update length.
The temporary array and both copies retain ordinary allocation, scratch,
validation and charging costs, including empty copies.
| Operation | Purpose |
Removal prepares its optional return, shifts the suffix, clears the old tail and
updates length. Clear and in-capacity resize clear the selected interval in index
order before assigning length. Growing resize prepares/copies a candidate before
backing/length installation; reserve/trim prepare/copy candidates and replace only
backing. Indexed replacement/clearAt update one checked position. Complete stored
groups move without unpacking/reboxing every shifted element.
| Outcome | Current guarantee |
| --- | --- |
| INDEX_GUARD | Check a signed-long logical index against an exclusive limit |
| ARRAY_CHECK_LENGTH | Validate a structural length without allocating |
| ARRAY_COPY | Transfer complete stored groups with overlap-safe prestate semantics |
| STORAGE_PUBLISH | Prepare and commit source-selected copy/fill ranges and two typed fields |
| Logical rejection or preparation refusal before the first visible write | Previous backing, capacity, size and slots survive; argument effects and provisional allocations are not undone |
| Successful return | All collection, cleanup, identity, group and snapshot invariants hold |
| Internal/resource fault after visible writes start | Earlier writes may remain and invariants may be incomplete; no rollback, safe reuse, retry or recovery is promised |
The SDK chooses its indices, ranges, fill value, target backing and resulting size.
The generic layer validates those operands and resource charges. It snapshots
complete source groups into charged scratch before any write, then commits the
selected copy, fill and field updates without allocation, callbacks, scheduler or
GC interleaving. Its four immediate operands identify the Array/Struct types and
two distinct fields. There is no exported preparation token or general transaction
syntax.
Preparing values/returns does not make every subsequent instruction infallible.
A refused instruction preserves its own entry state, not earlier successful
stores. BudgetExhausted/debugger pauses are resumable boundaries; a method is not
an atomic host observation region. No callback or explicit suspension is inserted
between mutation updates. Candidates, temporary arrays and scratch follow normal
ownership/quotas; immediate GC or quota return is not promised.
For growth, a private candidate array is allocated and populated first. For a
no-growth edit, scratch and all argument/result carriers are prepared first.
`removeAt` obtains its previous optional value before publication. Raw transfer
keeps presence tags, full value groups and existing reference/carrier identity.
Per-element public optional unpack/repack would add failure points and allocation
costs while shifting.
Refusal leaves backing identity, capacity, size and complete stored groups
unchanged. Argument effects evaluated before the call remain ordinary source
effects. Unpublished candidates and scratch follow normal ownership and budgets.
The provider has no List/Cursor opcode, category or syscall; collection algorithms
remain in
The provider has no List/Cursor opcode, category or syscall; algorithms remain in
[SDK source](../../../prometeu-compiler/frontends/prometeu-frontend-pbs/src/main/resources/game/stdlib/1/sdk/collections/main.pbs).
## Compiler and tooling integration
@@ -140,41 +150,52 @@ tuples, nested arrays, contracts and function references.
List index/length arguments accept localized lossless int-to-long promotion.
Receiver and arguments execute once in source order, including named arguments.
General numeric conversion rules stay unchanged. Common aggregation, optimization,
emission and preload verification retain all four publication operands; malformed
types or fields are rejected even in unreachable instructions.
emission and preload verification carry ordinary typed stores, ARRAY_COPY and the
terminal operation. Retired encodings are rejected even in dead instructions;
matching byte values inside ordinary immediates are not retired instructions.
Editor projections show concrete public List/Cursor methods, optional results,
constructor overloads and SDK definitions. Hover/completion hide private adapters
and backing fields. Installed JAR tests load the actual packaged SDK, normative
spec and example outside the checkout, generate PBX and execute it in the provider.
## Evidence and practical limits
## Current qualification and historical evidence
The selected provider is clean commit
The current selected pin is [the catalog revision](../../../docs/vm-arch/runtime-catalog.properties),
`dcf6561ea73446e0a3e1cf1c73eda20923b38df0`. The
[migration evidence](../DSC-0096-runtime-storage-isa-consumer-migration/runtime-conformance-evidence.json)
qualifies sequential stores, logical traps and mutation phases. Later complete
[debug/release evidence](../DSC-0097-pbs-sdk-stack-queue/evidence/README.md) includes
the current List/Cursor corpus with both producers. This editorial reconciliation
does not rerun those suites or change their recorded scope.
The following table and the original JSON are historical delivery evidence for
the superseded storage design, not proof of whole-method preservation today.
The original selected provider was clean commit
`d9b9eeb171d2861ec660d4cf5703b0795eac49a9`. Studio changes were tested over base
`76a8b074e89e17c7425220888eeec38427261d46`. Qualification used macOS 27.0.1 arm64,
Rust 1.98.1/LLVM 22.1.8, the dev runtime profile with resource-diagnostics,
capabilities NONE, Gradle 9.3.1 and the repository's Java 21 toolchain
(Corretto 21.0.4). Ordinary and optimized producer paths were exercised.
| Gate | Observed result |
| Historical delivery gate | Result on the original pin |
| --- | --- |
| Provider bytecode and VM libraries | 6 + 302 passed; 2 existing measurement tests ignored |
| Full Studio regression and installed knowledge | Passed; existing skipped tests retained |
| Final frontend/editor, LSP, source generation and installed checks | Passed; both installed tests passed |
| PBS List/Cursor through production verifier/VM | 27 cases in each producer mode: 54 passed |
| Non-PBS common IR | 2 storage fixtures executed through production VM |
| Whole-operation quota refusals | 71 attempts; independently observed prior List state preserved |
| Compound-publication refusals, now retired | 71 attempts; prior List state preserved on the original pin |
| Cursor GC | Nodes/items: 1200/1200 → 1/1 → 1/1 → 0/0 |
| Reexports and installed example execution | Both passed; final captured value 22 retained |
| Provider catalogs and canonical mirrors | Clean pinned provider; catalog parity and all four mirrors match |
The refusal matrix lowers GuestBytes, HandleSlots, AuxiliaryBytes and TemporaryBytes
That historical refusal matrix lowered GuestBytes, HandleSlots, AuxiliaryBytes and TemporaryBytes
around candidate allocation, no-growth publication scratch, optional return
preparation, node creation and frame/operand preparation. It compares the typed
List fields and every backing group before and after each refusal, and verifies
resource accounting returns to zero after VM drop. Generic provider tests also
cover overlapping multi-slot transfers, late invalid fields/ranges, immutable
preparation, node creation and frame/operand preparation. It compared the typed
List fields and every backing group before and after each refusal, and verified
resource accounting returned to zero after VM drop. The original provider tests also
covered overlapping multi-slot transfers, late invalid fields/ranges, immutable
Dynamic-box identity and malformed encodings.
Controlled normal GC safepoints prove consumed-prefix reclamation, retention of
@@ -184,35 +205,37 @@ safepoints. Optional carriers and other legitimate owners still have their norma
allocation and retention costs. Capturing n positions with k present values takes
O(n) time and O(k) additional nodes; next follows one link in O(1).
[Machine-readable evidence](runtime-conformance-evidence.json) records commands,
[Historical machine-readable evidence](runtime-conformance-evidence.json) records commands,
source and PBX hashes, default ceilings, every lowered refusal quota, GC observations
and tested provenance. New cases are mandatory; missing fixtures fail the harness.
The full harness also reports two pre-existing response-schema gaps, LifecycleRole
and NonFiniteValue, with original payloads preserved. These were not changed by
this delivery. Qualification makes no release-profile or cross-platform claim.
this original delivery. That historical run makes no release-profile or cross-platform
claim; later debug/release qualification is linked separately above.
R1–R5 are covered by type/construction/bounds/mutation/capacity fixtures; R6–R9 by
snapshot states, identity, non-List optional production and GC; R10–R11 by direct
provider tests, common IR and the refusal matrix; R12 by editor/import/distribution
gates; R13 by resource accounting and retention observations; R14 by the mandatory
The original R1–R5 mapping used type/construction/bounds/mutation/capacity fixtures; R6–R9 used
snapshot states, identity, non-List optional production and GC; R10–R11 used direct
provider tests, common IR and the refusal matrix; R12 used editor/import/distribution
gates; R13 used resource accounting and retention observations; R14 used the mandatory
production corpus, full regressions and pinned provenance.
## Pitfalls to preserve in future work
- Treat logical size and capacity separately. An index inside physical capacity
can still require TRAP_OOB; a hole is an optional result only at a valid index.
can still fail the SDK's logical check with TRAP_EXPLICIT; a hole is an optional
result only at a valid index. Physical Array instructions retain their own faults.
- Keep faults distinct from absence. Long overflow, invalid bounds and resource
refusal cannot become none or a truncated index.
- Prepare optional returns and temporary groups before publication. Individual
Array instruction atomicity alone cannot prove whole-mutator atomicity.
- Prepare optional returns and temporary groups before visible writes. Preparation
and individual instruction preservation do not provide whole-method rollback.
- Preserve complete stored groups while shifting. Public optional reboxing can
change identity, roots, costs and refusal boundaries.
- Follow the Cursor state table. Never advance a nonempty cursor past its last
node or retain an initial-head root just to support an unrequested reset.
- Account for node and optional-carrier allocation. A linked snapshot has no
promise of lower memory use than an array or immediate quota recovery.
- Carry all publication operands through linking and optimization. A lost second
field identity can invalidate otherwise well-typed SDK calls after imports.
- Preserve source-selected store/copy order through linking and optimization, and
compare refusal observations with the successful prefix at the actual failure phase.
- Keep privileged bindings origin-based and private. Names and copied attributes
are insufficient authority.
@@ -220,15 +243,18 @@ production corpus, full regressions and pinned provenance.
- [List/Cursor specification](../../../docs/compiler/pbs/specs/sdk-list-and-cursor.md)
and [Array/optional foundation](../../../docs/compiler/pbs/specs/typed-arrays-and-optional-execution.md).
- [Common Registry/PBX specification](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md)
and [canonical provider publication code](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/storage_publication.rs).
- [Common Registry/PBX specification](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md),
[current provider copy code](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/array_copy.rs)
and [ordinary typed storage](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/collections.rs).
- [Source fixture generator](../../../prometeu-architecture-tests/src/test/java/p/studio/compiler/backend/irvm/PbsListCursorTest.java),
[neutral IR fixtures](../../../prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/StoragePublicationPipelineTest.java)
and [production harness instructions](../../../tools/pbx-conformance/README.md).
- [Generic structs lesson](../DSC-0094-pbs-generic-structs/LSN-0089-concrete-generic-state-and-bodies.md)
- [Mutation-phase migration](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md),
[generic structs lesson](../DSC-0094-pbs-generic-structs/LSN-0089-concrete-generic-state-and-bodies.md)
and [generic contracts lesson](../DSC-0089-pbs-generic-contracts/LSN-0088-concrete-generic-contracts-from-source-to-vm.md).
Run `./gradlew test :prometeu-app:installedKnowledgeTest --console=plain` before
`python3 tools/pbx-conformance/run.py --runtime ../runtime` so the complete fixture
corpus, including the installed example, exists. Provider and catalog commands,
their working directories and the tested configuration are in the evidence file.
corpus, including the installed example, exists. Select the current pin and use
the migration/later evidence for current reproduction. The original JSON retains
its original provider, commands and configuration unchanged.
@@ -1,9 +1,9 @@
---
id: LSN-0091
ticket: pbs-sdk-linked
title: Private Linked Nodes and Atomic Publication
title: Private Linked Nodes and Sequential Mutation Phases
created: 2026-10-23
tags: [compiler-pbs, collections, snapshot-cursor, atomic-publication, runtime-conformance]
tags: [compiler-pbs, collections, snapshot-cursor, failure-phases, runtime-conformance]
---
## Original problem and final result
@@ -17,14 +17,18 @@ is the durable normative reference.
PBS needed a dense sequence with private doubly linked nodes and useful end
operations, while preserving the shared snapshot traversal already delivered for
List. `linked<T>` now supplies that sequence through executable SDK source.
Forward and reverse captures both return the existing Cursor<T> contract. A
neutral FIELDS_PUBLISH operation provides complete publication across the collection
and its neighbors, without moving node algorithms into the VM.
Forward and reverse captures both return the existing Cursor<T> contract. Source
algorithms prepare nodes/returns, then use ordinary typed field stores with
explicit failure phases; node algorithms remain outside the VM.
The selected provider is clean commit `57992cf214e8a5f9de53a40dba9aaa7fab038c91`.
Studio changes were qualified over base `c410e3500150de5e6d55c1da8f9b22076ca84251`.
[Machine-readable evidence](runtime-conformance-evidence.json) records source/PBX
hashes, commands, configuration, lowered refusal limits and observed GC states.
This lesson was reconciled during AGD89 closure on 2026-10-27 with accepted
DEC-0077 / PLN-0155, consolidated in
[LSN-0092](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md).
That migration retired FIELDS_PUBLISH and replaced the original whole-method
preservation guarantee. The filename preserves the original delivery's provenance;
its atomic-publication wording does not describe today's contract. The original
[evidence JSON](runtime-conformance-evidence.json) remains unchanged and is discussed
separately as historical evidence below.
## Dense positions do not change the optional API
@@ -86,38 +90,35 @@ The [distributed example](../../../prometeu-compiler/frontends/prometeu-frontend
captures both orders, then replaces a source element and clears Linked. Both
traversals still sum to 42, retaining final values 22 and 20 respectively.
## Prepare every object before changing any object
## Prepare nodes and returns, then update links sequentially
Insertion can update the collection's head/tail/size and two existing neighbors.
Removal also prepares the returned optional value. Separate successful field
stores are insufficient: a later allocation or frame/operand refusal could leave
half of a neighbor pair updated.
Insertion can update head/tail/size and two existing neighbors. Removal also
prepares the returned optional value. Prepare bounds, checked next size, candidate
nodes, neighbor values and complete groups before visible mutation. This preparation
does not make every later instruction infallible: a later allocation or frame/operand
refusal can leave a neighbor pair only partly updated.
The existing range-based STORAGE_PUBLISH operation covered an Array and two
fields of one owner. Its guarantee did not automatically extend to Linked.
The new generic FIELDS_PUBLISH uses a normal Struct TypeId as a schema: each
schema field names one destination Struct type. The stack supplies each nullable
destination followed by all its complete field groups. No request object, new
Registry category or public preparation token is needed.
Logical checks run as PBS comparisons/branches ending in the trusted private
TRAP_EXPLICIT binding. Insertion first updates previous.next or head, then
next.previous or tail, then length. Removal prepares its result/neighbors and
updates those two sides in the same order before length. Indexed replacement
changes only value. Clear assigns head, tail and length in that order. These are
ordinary typed stores, without a publication schema, compound opcode, callbacks
or explicit suspension between updates. Do not reconstruct the whole list on
every mutation to simulate the retired mechanism.
All non-null destinations must have the exact concrete type and live provenance.
All supplied value groups are validated, including those paired with a null
(skipped) destination. The original operands stay rooted and unconsumed until the
whole attempt is admitted. The VM then commits in schema order without allocation,
validation, callbacks or GC/scheduler interleaving, and consumes the operands.
Repeated destinations have ordered last-write semantics using pre-captured inputs.
| Outcome | Current guarantee |
| --- | --- |
| Rejection/preparation refusal before visible mutation | Old roots, size, payloads and neighbor links survive; argument effects/provisional allocations are not undone |
| Successful return | All reciprocal-link, size, group, identity, cleanup and snapshot invariants hold |
| Internal/resource fault after visible mutation starts | Earlier stores may remain and topology/size can be incomplete; no rollback, safe reuse, retry or recovery is promised |
The SDK selects topology and values. It allocates and initializes a private
candidate node before publishing an insertion. Removal prepares its optional
result first. Private helper bindings lower those source-selected values to the
neutral operation. A refusal preserves the old roots, size, values and reciprocal
links; already evaluated caller argument effects remain ordinary source effects.
Publication itself needs no heap object or scratch. This does not make a source
mutation allocation-free: optional carriers, candidate nodes, packing and call
preparation still have their normal costs. Provider tests execute publication at
already exhausted allocation ceilings, preserve native String/reference identity
and multi-slot groups, and reject a late invalid group before every shared write.
Individual instruction refusal preserves its own entry state, not earlier stores.
This is not a basic-consistency guarantee for a faulted method. BudgetExhausted
and debugger pauses are resumable instruction boundaries; host observation is
not atomic. No catch, poison flag or recovery API is introduced. Candidate nodes,
optional carriers, packing and call/operand preparation retain their normal
allocation, rooting and quota costs; immediate reclamation is not promised.
## GC proofs require an actual collection, not only a safepoint
@@ -162,38 +163,55 @@ The full harness also needed its original scalar/input corpus selector to exclud
the new Linked fixture prefix. Linked cases remain mandatory in their dedicated
validator; they are not silently treated as older scalar fixtures or skipped.
## Qualification and limits
## Current qualification and historical delivery evidence
| Gate | Observed result |
The current pin is [the catalog revision](../../../docs/vm-arch/runtime-catalog.properties),
`dcf6561ea73446e0a3e1cf1c73eda20923b38df0`. The
[migration evidence](../DSC-0096-runtime-storage-isa-consumer-migration/runtime-conformance-evidence.json)
qualifies source checks, sequential stores and failure phases. Later complete
[debug/release evidence](../DSC-0097-pbs-sdk-stack-queue/evidence/README.md) includes
the current Linked corpus and both producers. No suites were rerun by this
editorial reconciliation.
The table below records the original delivery on clean provider
`57992cf214e8a5f9de53a40dba9aaa7fab038c91`, over Studio base
`c410e3500150de5e6d55c1da8f9b22076ca84251`. Its original JSON preserves source/PBX
hashes, commands, lowered limits and GC observations unchanged. Compound publication
and whole-method refusal results in that evidence were superseded by LSN-0092;
they are not current guarantees or current-provider reproduction results.
| Historical delivery gate | Result on the original pin |
| --- | --- |
| Full Studio regression | 1,030 tests; no failures/errors; 13 existing skips |
| Installed knowledge | Both tests passed, including standalone Linked PBX generation |
| Provider bytecode/VM libraries | 6 + 305 passed; 2 existing measurement tests ignored |
| Linked source to production VM | 26 cases in each producer mode: 52 passed |
| Neutral storage/publication | Four common-IR fixtures executed, including ordinary/optimized FIELDS_PUBLISH |
| Whole-operation refusals | 231; all ten mutation/capture boundaries preserved previous source state |
| Retired publication fixtures | Four common-IR fixtures, including ordinary/optimized FIELDS_PUBLISH on the original pin |
| Original whole-operation refusals | 231; original mutation/capture boundaries preserved previous source state on that pin |
| Linked removal/clear GC | 1,200 → 1,197 → 0 → 0 nodes and payloads |
| Snapshot GC | 1,200 → 1 → 1 → 0 captured nodes/payloads, with no retained Linked topology |
| Imports/installed traversal | Passed; installed forward/reverse final values 22/20 |
| Catalogs and canonical mirrors | Clean selected provider; catalog/exporter parity and all four mirrors match |
The refusal matrix observed GuestBytes, HandleSlots and AuxiliaryBytes refusals.
TemporaryBytes was checked against the measured setup/success peaks; no additional
peak required a lowered refusal case in this Linked corpus. FIELDS_PUBLISH has no
scratch allocation, and the direct provider test proves publication succeeds with
TemporaryBytes already capped at current usage. This is not a claim that general
source preparation never needs temporary resources.
The original refusal matrix observed GuestBytes, HandleSlots and AuxiliaryBytes
refusals. TemporaryBytes was checked against its measured setup/success peaks;
no additional peak required a lowered refusal case in that historical corpus.
The former no-scratch FIELDS_PUBLISH tests belong only to the retired design.
Current source preparation, carriers and sequential stores retain normal quota
costs and use the phase-specific evidence linked above.
Qualification used macOS 27.0.1 arm64, Rust 1.98.1/LLVM 22.1.8, the dev runtime
profile with resource-diagnostics, capabilities NONE, Gradle 9.3.1 and the Java 21
toolchain; the installed JVM was Corretto 21.0.4. No release or cross-platform claim
is made. Two pre-existing full-harness response-schema gaps, LifecycleRole and
toolchain; the installed JVM was Corretto 21.0.4. That historical run made no
release or cross-platform claim; later debug/release qualification is linked
separately above. Two pre-existing full-harness response-schema gaps, LifecycleRole and
NonFiniteValue, remain explicitly reported with original payloads.
## Pitfalls to preserve
- Do not implement a multi-object mutation as a sequence of fallible stores.
- Prepare the optional removed-value carrier before publishing the removal.
- Prepare values and use source-selected sequential stores with explicit failure
phases. Do not claim whole-method atomicity or safe reuse after a later fault.
- Prepare the optional removed-value carrier before unlinking the node.
- Preserve every field group and native/reference identity, including unchanged
payloads when updating neighbor links.
- Never expose source nodes to support traversal. Snapshot and collection links
@@ -209,15 +227,16 @@ NonFiniteValue, remain explicitly reported with original payloads.
- [Linked specification](../../../docs/compiler/pbs/specs/sdk-linked-and-cursor.md),
[List/shared Cursor](../../../docs/compiler/pbs/specs/sdk-list-and-cursor.md),
[generic structs](../../../docs/compiler/pbs/specs/generic-structs-and-free-functions.md)
and [generic Registry/PBX publication](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md).
and [generic Registry/PBX storage](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md).
- [SDK source](../../../prometeu-compiler/frontends/prometeu-frontend-pbs/src/main/resources/game/stdlib/1/sdk/collections/main.pbs),
[source corpus](../../../prometeu-architecture-tests/src/test/java/p/studio/compiler/backend/irvm/PbsLinkedTest.java),
[neutral producer](../../../prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/StoragePublicationPipelineTest.java)
and [production harness instructions](../../../tools/pbx-conformance/README.md).
- [Provider publication implementation](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/storage_publication.rs)
and [previous List/Cursor lesson](../DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md).
- [Current provider typed storage](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/collections.rs),
[sequential mutation migration](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md)
and [List/Cursor lesson](../DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md).
Run `./gradlew test :prometeu-app:installedKnowledgeTest --console=plain`, followed
by `python3 tools/pbx-conformance/run.py --runtime ../runtime`, after selecting the
recorded provider. The evidence file retains command/configuration details and
R1–R12 traceability after removal of the completed workflow controls.
by `python3 tools/pbx-conformance/run.py --runtime ../runtime`, using the current
pin and the migration/later evidence for current qualification. The original JSON
retains its original provider/configuration and R1–R12 delivery traceability.