From eb7d6a23988b2e13ab1e91b7663abbd7ab983f4c Mon Sep 17 00:00:00 2001 From: bQUARKz Date: Tue, 27 Oct 2026 13:40:38 +0000 Subject: [PATCH] docs(pbs): consolidate collection lessons and failure guarantees --- ...tions-and-separate-collection-contracts.md | 163 ++++++++++++++++++ ...table-lists-and-stable-snapshot-cursors.md | 148 +++++++++------- ...ate-linked-nodes-and-atomic-publication.md | 133 ++++++++------ 3 files changed, 326 insertions(+), 118 deletions(-) create mode 100644 discussion/lessons/DSC-0087-pbs-sdk-collections/LSN-0096-shared-foundations-and-separate-collection-contracts.md diff --git a/discussion/lessons/DSC-0087-pbs-sdk-collections/LSN-0096-shared-foundations-and-separate-collection-contracts.md b/discussion/lessons/DSC-0087-pbs-sdk-collections/LSN-0096-shared-foundations-and-separate-collection-contracts.md new file mode 100644 index 00000000..e6c0c689 --- /dev/null +++ b/discussion/lessons/DSC-0087-pbs-sdk-collections/LSN-0096-shared-foundations-and-separate-collection-contracts.md @@ -0,0 +1,163 @@ +--- +id: LSN-0096 +ticket: pbs-sdk-collections +title: Shared foundations and separate collection contracts +created: 2026-10-27 +tags: [compiler-pbs, sdk, collections, umbrella, navigation, failure-phases] +--- + +# Shared foundations and separate collection contracts + +**Worker:** Lesson Worker. **Owner domain:** `compiler/pbs`. +DSC-0087 / AGD89 coordinated eight completed child discussions. Their accepted +decisions/plans remain the sources of technical choices, consolidated in the +specifications and lessons below. This umbrella closes coordination and the +previously authorized editorial reconciliation; it introduces no new collection +contract or implementation scope. + +## Original problem and resolution + +Array syntax, generic language mechanisms, identity policies and several collection +APIs were initially mixed into one discussion. A monolithic decision would have +made prerequisites, product choices and execution gates hard to distinguish. +The user chose an umbrella with separate domain-owned children, ordered as shown +below, then added generic structs as a prerequisite and Stack/Queue as one further +child. Order of execution did not prescribe one collection's implementation over +another. + +The result is a qualified source SDK catalog over shared language/storage +mechanisms. Each child completed its own agenda → decision → plan → implementation +→ lesson flow. The final umbrella task reconciled List/Linked lessons with the +already accepted sequential-storage migration and repaired their provider links. +Original evidence files retain their original bytes and provider provenance. + +## Reading and ownership map + +Read the foundations first, then the collection needed by the caller. Specifications +are normative; lessons teach the choices and retain implementation/evidence context. + +| Sequence / discussion | Durable lesson | Normative specification | +| --- | --- | --- | +| 1. Array, DSC-0088 | [LSN-0087: contextual parametric arrays](../DSC-0088-pbs-array-type-syntax/LSN-0087-parametric-arrays-and-contextual-construction.md) | [Array/optional](../../../docs/compiler/pbs/specs/typed-arrays-and-optional-execution.md) | +| 2. Generic contracts, DSC-0089 | [LSN-0088: concrete contracts](../DSC-0089-pbs-generic-contracts/LSN-0088-concrete-generic-contracts-from-source-to-vm.md) | [Generic contracts](../../../docs/compiler/pbs/specs/generic-contracts.md) | +| 3. Generic structs/functions, DSC-0094 | [LSN-0089: concrete state/bodies](../DSC-0094-pbs-generic-structs/LSN-0089-concrete-generic-state-and-bodies.md) | [Generic structs/functions](../../../docs/compiler/pbs/specs/generic-structs-and-free-functions.md) | +| 4. List/Cursor, DSC-0090 | [LSN-0090: mutable positions/stable snapshots](../DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md) | [List/shared Cursor](../../../docs/compiler/pbs/specs/sdk-list-and-cursor.md) | +| 5. Linked, DSC-0091 | [LSN-0091: private nodes/sequential phases](../DSC-0091-pbs-sdk-linked/LSN-0091-private-linked-nodes-and-atomic-publication.md) | [Linked](../../../docs/compiler/pbs/specs/sdk-linked-and-cursor.md) | +| 6. Set/Identity, DSC-0092 | [LSN-0093: reusable identity/buckets](../DSC-0092-pbs-sdk-set/LSN-0093-reusable-identity-and-source-set-buckets.md) | [Set/Identity](../../../docs/compiler/pbs/specs/sdk-set-and-identity.md) | +| 7. Map/Keyval, DSC-0093 | [LSN-0094: presence/representative keys](../DSC-0093-pbs-sdk-map-keyval/LSN-0094-map-presence-key-identity-and-captured-associations.md) | [Map/Keyval](../../../docs/compiler/pbs/specs/sdk-map-and-keyval.md) | +| 8. Stack/Queue, DSC-0097 | [LSN-0095: dense/circular arrays](../DSC-0097-pbs-sdk-stack-queue/LSN-0095-private-array-stacks-and-circular-queues.md) | [Stack/Queue](../../../docs/compiler/pbs/specs/sdk-stack-and-queue.md) | + +The supporting [LSN-0092 / DSC-0096](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md) +explains accepted DEC-0077 / PLN-0155: source checks, ordinary stores/ARRAY_COPY, +failure phases and current-provider provenance. It supersedes the early compound +publication guarantees without changing the delivered List/Linked APIs or Cursor. +Compiler/general owns neutral IR/PBX; the provider owns storage, ISA, quotas and GC. + +## Similar syntax does not imply identical collection semantics + +Concrete generic applications are invariant. Explicit and contextual lowercase +construction follow the accepted language rules; argument values do not infer +generic parameters. Shared object instances remain shared through collection +insertion, mutation and snapshots. No deep freezing or structural equality is +silently introduced. + +| Surface | Logical content and traversal | Important distinction | +| --- | --- | --- | +| array | Fixed length; positions may be empty | Physical length is not a resizable collection contract | +| list | Logical positions include holes; snapshot skips holes in index order | Size and capacity differ; clearAt keeps a position while removeAt removes it | +| linked | Dense private doubly linked nodes; forward/reverse snapshots | No capacity, holes or Array u32 size bound | +| set | Unique members under selected Identity; unordered snapshot | Hash filters; equality resolves collisions; private buckets are not a shared universal table | +| map | Key associations; unordered read-only Keyval snapshots | Lookup/previous-value optional describes entry existence independently of optional V | +| stack | Dense array; LIFO removal/snapshot order | Initial capacity does not cap collection size | +| queue | Circular array; FIFO removal/snapshot order | Wrapped growth linearizes occupied segments; dequeue does not shift values | + +Array/List/Linked/Set/Stack/Queue reject directly optional elements. Map rejects a +directly optional K but admits optional V; an independent Keyval can hold optional +components. Valid objects containing optional fields and optional collections +remain distinct from those restrictions. Zero/false are present values; faults +are not absence. + +Set/Map reuse Identity/hash policies and helpers, with separate private nodes and +algorithms. Eligible object keys use instance identity by default; explicit content +strategies must remain coherent while keys are stored. Hashing does not supply an +implicit structural equality or freeze mutable key content. + +## One Cursor protocol, collection-specific capture + +The shared Cursor captures complete values in immutable private singly linked, +non-circular nodes, independent of source slots/editable nodes. It has no public +node, live iterator, reset or removal operation. Captured references remain shared +objects; later edits to collection membership do not rewrite the captured sequence. + +A nonempty Cursor begins on first; hasNext tests successor; next without successor +is a no-op; only an empty-born Cursor has absent current. Process current before +testing whether to advance. Consumed prefixes can become collectible, but final +current remains retained. List, Linked, Set, Map, Stack and Queue select their +payload/order while reusing this exact state table. + +## Current failure guarantees and historical evidence + +Prepare checks, complete values, return carriers and candidates before visible +mutation, then execute source-selected ordinary stores and copies. Before the +first visible write, refusal preserves the prior collection. Success restores +all invariants. A later internal/resource fault may retain earlier writes and +incomplete invariants; no rollback, safe reuse, retry or recovery is promised. +Per-instruction preservation does not establish whole-method atomicity. Budget/ +debugger pauses remain resumable instruction boundaries. + +The List/Linked lessons now teach this contract and link to LSN-0092. Their early +INDEX_GUARD/ARRAY_CHECK_LENGTH/STORAGE_PUBLISH/FIELDS_PUBLISH mechanisms are marked +retired or historical. Their original evidence JSONs still describe their original +provider pins and stronger publication results, not current guarantees. The old +missing provider-publication links now point to existing typed-storage/copy sources. +The historical Linked filename is retained to preserve existing references. + +The current selected [catalog pin](../../../docs/vm-arch/runtime-catalog.properties) +is `dcf6561ea73446e0a3e1cf1c73eda20923b38df0`. The latest +[complete qualification record](../DSC-0097-pbs-sdk-stack-queue/evidence/README.md) +includes all existing collection corpora in debug/release and both producer paths, +1,091 standard JUnit cases with no failures/errors and 13 existing skips, plus +two passing installed-JAR tests. Those are retained implementation results; +this documentation-only closure validates workflow/links/provenance and does not +claim a new test run, benchmark, other-host qualification or game readiness. + +## Candidate backlog retained at closure + +These remain candidates, without accepted APIs, delivery commitments or new +discussions created by this closure: + +| Candidate | Questions requiring a separate discussion | +| --- | --- | +| Deque | Two-ended API, order/capacity and whether to reuse a circular mechanism | +| Pools | Distinguish capacity from object creation; acquire/release/reset, full state, duplicate returns and stale references/generations | +| Priority queue | Priority/order/equality policy, storage and resource behavior | +| Bitset | Representation, bounds and intended operations | +| Sparse set | Sparse/dense representation, identity and removal policy | + +Audio, guest lifecycle, memcard, game qualification, installed Build/Play/headless +readiness and handoffs to AGD75/AGD88 are outside this umbrella's closure, as +explicitly directed by the user. General documentation migration is a separate +discussion; the local reconciliation does not require completing that migration. + +## Closure and maintenance guidance + +All eight children and the supporting migration are done, with only lessons in +their workflow records. The remaining List/Linked editorial task is complete. +DSC-0087 closes through `discussion housekeep`, retaining this English navigation +lesson and preserving candidate backlog and scope boundaries. The CLI owns index +updates and workflow removal; no manual index edits or runtime writes are involved. + +Avoid these maintenance errors: + +- Do not turn an umbrella's delivery order into a physical implementation dependency. +- Do not reinterpret a child decision while planning another collection. +- Do not inherit capacity, optional eligibility, equality or traversal order by analogy. +- Do not teach a historical provider guarantee as current merely because its evidence passed. +- Do not edit historical evidence to make its pin/counts match later implementations. +- Do not turn backlog candidates or adjacent runtime-readiness work into accepted scope. + +The CLI currently has no dedicated umbrella/dependency relationship or standalone +lesson-ID allocation command. Existing tags and durable links express relationships; +the read-only model accessor supplies the next lesson ID and housekeeping owns +registration/counter advancement. These tooling gaps remain explicit rather than +being filled with manual index conventions. diff --git a/discussion/lessons/DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md b/discussion/lessons/DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md index b6ecfa73..c103cb6a 100644 --- a/discussion/lessons/DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md +++ b/discussion/lessons/DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md @@ -3,7 +3,7 @@ id: LSN-0090 ticket: pbs-sdk-list title: Mutable Lists and Stable Snapshot Cursors created: 2026-10-22 -tags: [compiler-pbs, collections, snapshot-cursor, atomic-publication, runtime-conformance] +tags: [compiler-pbs, collections, snapshot-cursor, failure-phases, runtime-conformance] --- ## Original problem and final result @@ -19,8 +19,15 @@ PBS needed a resizable collection whose existing references remain valid, togeth with a traversal that survives later collection edits. Both now live in the executable SDK: ordinary generic `list`, the shared `Cursor` contract and private snapshot nodes. The compiler provides trusted private bindings and source -projections; generic storage instructions support complete publication in the VM. -Linked, Set and Map remain separate discussions. +projections; ordinary typed storage and ARRAY_COPY preserve complete stored groups. + +This lesson was reconciled during AGD89 closure on 2026-10-27 with accepted +DEC-0077 / PLN-0155, consolidated in +[LSN-0092](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md). +That migration replaced the original guard/publication instructions and universal +whole-method refusal guarantee. The current explanation below follows the +sequential-store contract; the original evidence remains explicitly historical. +Other collection APIs are documented in their own specifications and lessons. ## Separate collection identity, logical positions and physical storage @@ -84,44 +91,47 @@ returns 42. Current still contains 22 afterwards. The outer optional describes whether a node exists. A producer with optional T must preserve `some(none)` as a present node whose payload is none. A synthetic non-List producer qualifies that distinction independently of List's element rule. -Future Linked/Set producers use Cursor; Map entries use Cursor>. -Their construction rules and ordering guarantees belong to their own discussions. +Linked/Set produce Cursor; Map entries use Cursor>, and Stack/Queue +capture in removal order. Their construction rules and ordering guarantees belong +to their own specifications. These producers reuse the same Cursor state table. -## Prepare the whole mutation before publishing +## Prepare values, then perform sequential updates -An insert without growth can shift many slots and update size. A remove also -prepares an optional result and clears its old tail. Each operation needs a single -failure boundary for all observable changes. +Logical index/length checks are executable PBS comparisons and branches ending in +the trusted private terminal TRAP_EXPLICIT binding. They are not INDEX_GUARD or +ARRAY_CHECK_LENGTH instructions. ARRAY_COPY remains a neutral complete-group, +overlap-safe transfer; STORAGE_PUBLISH and FIELDS_PUBLISH were retired by the +migration in LSN-0092. No compound publication instruction is available. -The generic provider adds four operations, carried through common IR and PBX: +For insertion with growth, allocate a private candidate and copy prefix/suffix +directly into their final positions, including holes and presence tags. Store +the new value privately, then assign backing and length in sequence. Without +growth, first prepare the inserted stored group in a private one-slot array, +shift the suffix with ARRAY_COPY, copy the prepared slot and update length. +The temporary array and both copies retain ordinary allocation, scratch, +validation and charging costs, including empty copies. -| Operation | Purpose | +Removal prepares its optional return, shifts the suffix, clears the old tail and +updates length. Clear and in-capacity resize clear the selected interval in index +order before assigning length. Growing resize prepares/copies a candidate before +backing/length installation; reserve/trim prepare/copy candidates and replace only +backing. Indexed replacement/clearAt update one checked position. Complete stored +groups move without unpacking/reboxing every shifted element. + +| Outcome | Current guarantee | | --- | --- | -| INDEX_GUARD | Check a signed-long logical index against an exclusive limit | -| ARRAY_CHECK_LENGTH | Validate a structural length without allocating | -| ARRAY_COPY | Transfer complete stored groups with overlap-safe prestate semantics | -| STORAGE_PUBLISH | Prepare and commit source-selected copy/fill ranges and two typed fields | +| Logical rejection or preparation refusal before the first visible write | Previous backing, capacity, size and slots survive; argument effects and provisional allocations are not undone | +| Successful return | All collection, cleanup, identity, group and snapshot invariants hold | +| Internal/resource fault after visible writes start | Earlier writes may remain and invariants may be incomplete; no rollback, safe reuse, retry or recovery is promised | -The SDK chooses its indices, ranges, fill value, target backing and resulting size. -The generic layer validates those operands and resource charges. It snapshots -complete source groups into charged scratch before any write, then commits the -selected copy, fill and field updates without allocation, callbacks, scheduler or -GC interleaving. Its four immediate operands identify the Array/Struct types and -two distinct fields. There is no exported preparation token or general transaction -syntax. +Preparing values/returns does not make every subsequent instruction infallible. +A refused instruction preserves its own entry state, not earlier successful +stores. BudgetExhausted/debugger pauses are resumable boundaries; a method is not +an atomic host observation region. No callback or explicit suspension is inserted +between mutation updates. Candidates, temporary arrays and scratch follow normal +ownership/quotas; immediate GC or quota return is not promised. -For growth, a private candidate array is allocated and populated first. For a -no-growth edit, scratch and all argument/result carriers are prepared first. -`removeAt` obtains its previous optional value before publication. Raw transfer -keeps presence tags, full value groups and existing reference/carrier identity. -Per-element public optional unpack/repack would add failure points and allocation -costs while shifting. - -Refusal leaves backing identity, capacity, size and complete stored groups -unchanged. Argument effects evaluated before the call remain ordinary source -effects. Unpublished candidates and scratch follow normal ownership and budgets. -The provider has no List/Cursor opcode, category or syscall; collection algorithms -remain in +The provider has no List/Cursor opcode, category or syscall; algorithms remain in [SDK source](../../../prometeu-compiler/frontends/prometeu-frontend-pbs/src/main/resources/game/stdlib/1/sdk/collections/main.pbs). ## Compiler and tooling integration @@ -140,41 +150,52 @@ tuples, nested arrays, contracts and function references. List index/length arguments accept localized lossless int-to-long promotion. Receiver and arguments execute once in source order, including named arguments. General numeric conversion rules stay unchanged. Common aggregation, optimization, -emission and preload verification retain all four publication operands; malformed -types or fields are rejected even in unreachable instructions. +emission and preload verification carry ordinary typed stores, ARRAY_COPY and the +terminal operation. Retired encodings are rejected even in dead instructions; +matching byte values inside ordinary immediates are not retired instructions. Editor projections show concrete public List/Cursor methods, optional results, constructor overloads and SDK definitions. Hover/completion hide private adapters and backing fields. Installed JAR tests load the actual packaged SDK, normative spec and example outside the checkout, generate PBX and execute it in the provider. -## Evidence and practical limits +## Current qualification and historical evidence -The selected provider is clean commit +The current selected pin is [the catalog revision](../../../docs/vm-arch/runtime-catalog.properties), +`dcf6561ea73446e0a3e1cf1c73eda20923b38df0`. The +[migration evidence](../DSC-0096-runtime-storage-isa-consumer-migration/runtime-conformance-evidence.json) +qualifies sequential stores, logical traps and mutation phases. Later complete +[debug/release evidence](../DSC-0097-pbs-sdk-stack-queue/evidence/README.md) includes +the current List/Cursor corpus with both producers. This editorial reconciliation +does not rerun those suites or change their recorded scope. + +The following table and the original JSON are historical delivery evidence for +the superseded storage design, not proof of whole-method preservation today. +The original selected provider was clean commit `d9b9eeb171d2861ec660d4cf5703b0795eac49a9`. Studio changes were tested over base `76a8b074e89e17c7425220888eeec38427261d46`. Qualification used macOS 27.0.1 arm64, Rust 1.98.1/LLVM 22.1.8, the dev runtime profile with resource-diagnostics, capabilities NONE, Gradle 9.3.1 and the repository's Java 21 toolchain (Corretto 21.0.4). Ordinary and optimized producer paths were exercised. -| Gate | Observed result | +| Historical delivery gate | Result on the original pin | | --- | --- | | Provider bytecode and VM libraries | 6 + 302 passed; 2 existing measurement tests ignored | | Full Studio regression and installed knowledge | Passed; existing skipped tests retained | | Final frontend/editor, LSP, source generation and installed checks | Passed; both installed tests passed | | PBS List/Cursor through production verifier/VM | 27 cases in each producer mode: 54 passed | | Non-PBS common IR | 2 storage fixtures executed through production VM | -| Whole-operation quota refusals | 71 attempts; independently observed prior List state preserved | +| Compound-publication refusals, now retired | 71 attempts; prior List state preserved on the original pin | | Cursor GC | Nodes/items: 1200/1200 → 1/1 → 1/1 → 0/0 | | Reexports and installed example execution | Both passed; final captured value 22 retained | | Provider catalogs and canonical mirrors | Clean pinned provider; catalog parity and all four mirrors match | -The refusal matrix lowers GuestBytes, HandleSlots, AuxiliaryBytes and TemporaryBytes +That historical refusal matrix lowered GuestBytes, HandleSlots, AuxiliaryBytes and TemporaryBytes around candidate allocation, no-growth publication scratch, optional return -preparation, node creation and frame/operand preparation. It compares the typed -List fields and every backing group before and after each refusal, and verifies -resource accounting returns to zero after VM drop. Generic provider tests also -cover overlapping multi-slot transfers, late invalid fields/ranges, immutable +preparation, node creation and frame/operand preparation. It compared the typed +List fields and every backing group before and after each refusal, and verified +resource accounting returned to zero after VM drop. The original provider tests also +covered overlapping multi-slot transfers, late invalid fields/ranges, immutable Dynamic-box identity and malformed encodings. Controlled normal GC safepoints prove consumed-prefix reclamation, retention of @@ -184,35 +205,37 @@ safepoints. Optional carriers and other legitimate owners still have their norma allocation and retention costs. Capturing n positions with k present values takes O(n) time and O(k) additional nodes; next follows one link in O(1). -[Machine-readable evidence](runtime-conformance-evidence.json) records commands, +[Historical machine-readable evidence](runtime-conformance-evidence.json) records commands, source and PBX hashes, default ceilings, every lowered refusal quota, GC observations and tested provenance. New cases are mandatory; missing fixtures fail the harness. The full harness also reports two pre-existing response-schema gaps, LifecycleRole and NonFiniteValue, with original payloads preserved. These were not changed by -this delivery. Qualification makes no release-profile or cross-platform claim. +this original delivery. That historical run makes no release-profile or cross-platform +claim; later debug/release qualification is linked separately above. -R1–R5 are covered by type/construction/bounds/mutation/capacity fixtures; R6–R9 by -snapshot states, identity, non-List optional production and GC; R10–R11 by direct -provider tests, common IR and the refusal matrix; R12 by editor/import/distribution -gates; R13 by resource accounting and retention observations; R14 by the mandatory +The original R1–R5 mapping used type/construction/bounds/mutation/capacity fixtures; R6–R9 used +snapshot states, identity, non-List optional production and GC; R10–R11 used direct +provider tests, common IR and the refusal matrix; R12 used editor/import/distribution +gates; R13 used resource accounting and retention observations; R14 used the mandatory production corpus, full regressions and pinned provenance. ## Pitfalls to preserve in future work - Treat logical size and capacity separately. An index inside physical capacity - can still require TRAP_OOB; a hole is an optional result only at a valid index. + can still fail the SDK's logical check with TRAP_EXPLICIT; a hole is an optional + result only at a valid index. Physical Array instructions retain their own faults. - Keep faults distinct from absence. Long overflow, invalid bounds and resource refusal cannot become none or a truncated index. -- Prepare optional returns and temporary groups before publication. Individual - Array instruction atomicity alone cannot prove whole-mutator atomicity. +- Prepare optional returns and temporary groups before visible writes. Preparation + and individual instruction preservation do not provide whole-method rollback. - Preserve complete stored groups while shifting. Public optional reboxing can change identity, roots, costs and refusal boundaries. - Follow the Cursor state table. Never advance a nonempty cursor past its last node or retain an initial-head root just to support an unrequested reset. - Account for node and optional-carrier allocation. A linked snapshot has no promise of lower memory use than an array or immediate quota recovery. -- Carry all publication operands through linking and optimization. A lost second - field identity can invalidate otherwise well-typed SDK calls after imports. +- Preserve source-selected store/copy order through linking and optimization, and + compare refusal observations with the successful prefix at the actual failure phase. - Keep privileged bindings origin-based and private. Names and copied attributes are insufficient authority. @@ -220,15 +243,18 @@ production corpus, full regressions and pinned provenance. - [List/Cursor specification](../../../docs/compiler/pbs/specs/sdk-list-and-cursor.md) and [Array/optional foundation](../../../docs/compiler/pbs/specs/typed-arrays-and-optional-execution.md). -- [Common Registry/PBX specification](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md) - and [canonical provider publication code](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/storage_publication.rs). +- [Common Registry/PBX specification](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md), + [current provider copy code](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/array_copy.rs) + and [ordinary typed storage](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/collections.rs). - [Source fixture generator](../../../prometeu-architecture-tests/src/test/java/p/studio/compiler/backend/irvm/PbsListCursorTest.java), [neutral IR fixtures](../../../prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/StoragePublicationPipelineTest.java) and [production harness instructions](../../../tools/pbx-conformance/README.md). -- [Generic structs lesson](../DSC-0094-pbs-generic-structs/LSN-0089-concrete-generic-state-and-bodies.md) +- [Mutation-phase migration](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md), + [generic structs lesson](../DSC-0094-pbs-generic-structs/LSN-0089-concrete-generic-state-and-bodies.md) and [generic contracts lesson](../DSC-0089-pbs-generic-contracts/LSN-0088-concrete-generic-contracts-from-source-to-vm.md). Run `./gradlew test :prometeu-app:installedKnowledgeTest --console=plain` before `python3 tools/pbx-conformance/run.py --runtime ../runtime` so the complete fixture -corpus, including the installed example, exists. Provider and catalog commands, -their working directories and the tested configuration are in the evidence file. +corpus, including the installed example, exists. Select the current pin and use +the migration/later evidence for current reproduction. The original JSON retains +its original provider, commands and configuration unchanged. diff --git a/discussion/lessons/DSC-0091-pbs-sdk-linked/LSN-0091-private-linked-nodes-and-atomic-publication.md b/discussion/lessons/DSC-0091-pbs-sdk-linked/LSN-0091-private-linked-nodes-and-atomic-publication.md index a0ace1e1..6ff54118 100644 --- a/discussion/lessons/DSC-0091-pbs-sdk-linked/LSN-0091-private-linked-nodes-and-atomic-publication.md +++ b/discussion/lessons/DSC-0091-pbs-sdk-linked/LSN-0091-private-linked-nodes-and-atomic-publication.md @@ -1,9 +1,9 @@ --- id: LSN-0091 ticket: pbs-sdk-linked -title: Private Linked Nodes and Atomic Publication +title: Private Linked Nodes and Sequential Mutation Phases created: 2026-10-23 -tags: [compiler-pbs, collections, snapshot-cursor, atomic-publication, runtime-conformance] +tags: [compiler-pbs, collections, snapshot-cursor, failure-phases, runtime-conformance] --- ## Original problem and final result @@ -17,14 +17,18 @@ is the durable normative reference. PBS needed a dense sequence with private doubly linked nodes and useful end operations, while preserving the shared snapshot traversal already delivered for List. `linked` now supplies that sequence through executable SDK source. -Forward and reverse captures both return the existing Cursor contract. A -neutral FIELDS_PUBLISH operation provides complete publication across the collection -and its neighbors, without moving node algorithms into the VM. +Forward and reverse captures both return the existing Cursor contract. Source +algorithms prepare nodes/returns, then use ordinary typed field stores with +explicit failure phases; node algorithms remain outside the VM. -The selected provider is clean commit `57992cf214e8a5f9de53a40dba9aaa7fab038c91`. -Studio changes were qualified over base `c410e3500150de5e6d55c1da8f9b22076ca84251`. -[Machine-readable evidence](runtime-conformance-evidence.json) records source/PBX -hashes, commands, configuration, lowered refusal limits and observed GC states. +This lesson was reconciled during AGD89 closure on 2026-10-27 with accepted +DEC-0077 / PLN-0155, consolidated in +[LSN-0092](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md). +That migration retired FIELDS_PUBLISH and replaced the original whole-method +preservation guarantee. The filename preserves the original delivery's provenance; +its atomic-publication wording does not describe today's contract. The original +[evidence JSON](runtime-conformance-evidence.json) remains unchanged and is discussed +separately as historical evidence below. ## Dense positions do not change the optional API @@ -86,38 +90,35 @@ The [distributed example](../../../prometeu-compiler/frontends/prometeu-frontend captures both orders, then replaces a source element and clears Linked. Both traversals still sum to 42, retaining final values 22 and 20 respectively. -## Prepare every object before changing any object +## Prepare nodes and returns, then update links sequentially -Insertion can update the collection's head/tail/size and two existing neighbors. -Removal also prepares the returned optional value. Separate successful field -stores are insufficient: a later allocation or frame/operand refusal could leave -half of a neighbor pair updated. +Insertion can update head/tail/size and two existing neighbors. Removal also +prepares the returned optional value. Prepare bounds, checked next size, candidate +nodes, neighbor values and complete groups before visible mutation. This preparation +does not make every later instruction infallible: a later allocation or frame/operand +refusal can leave a neighbor pair only partly updated. -The existing range-based STORAGE_PUBLISH operation covered an Array and two -fields of one owner. Its guarantee did not automatically extend to Linked. -The new generic FIELDS_PUBLISH uses a normal Struct TypeId as a schema: each -schema field names one destination Struct type. The stack supplies each nullable -destination followed by all its complete field groups. No request object, new -Registry category or public preparation token is needed. +Logical checks run as PBS comparisons/branches ending in the trusted private +TRAP_EXPLICIT binding. Insertion first updates previous.next or head, then +next.previous or tail, then length. Removal prepares its result/neighbors and +updates those two sides in the same order before length. Indexed replacement +changes only value. Clear assigns head, tail and length in that order. These are +ordinary typed stores, without a publication schema, compound opcode, callbacks +or explicit suspension between updates. Do not reconstruct the whole list on +every mutation to simulate the retired mechanism. -All non-null destinations must have the exact concrete type and live provenance. -All supplied value groups are validated, including those paired with a null -(skipped) destination. The original operands stay rooted and unconsumed until the -whole attempt is admitted. The VM then commits in schema order without allocation, -validation, callbacks or GC/scheduler interleaving, and consumes the operands. -Repeated destinations have ordered last-write semantics using pre-captured inputs. +| Outcome | Current guarantee | +| --- | --- | +| Rejection/preparation refusal before visible mutation | Old roots, size, payloads and neighbor links survive; argument effects/provisional allocations are not undone | +| Successful return | All reciprocal-link, size, group, identity, cleanup and snapshot invariants hold | +| Internal/resource fault after visible mutation starts | Earlier stores may remain and topology/size can be incomplete; no rollback, safe reuse, retry or recovery is promised | -The SDK selects topology and values. It allocates and initializes a private -candidate node before publishing an insertion. Removal prepares its optional -result first. Private helper bindings lower those source-selected values to the -neutral operation. A refusal preserves the old roots, size, values and reciprocal -links; already evaluated caller argument effects remain ordinary source effects. - -Publication itself needs no heap object or scratch. This does not make a source -mutation allocation-free: optional carriers, candidate nodes, packing and call -preparation still have their normal costs. Provider tests execute publication at -already exhausted allocation ceilings, preserve native String/reference identity -and multi-slot groups, and reject a late invalid group before every shared write. +Individual instruction refusal preserves its own entry state, not earlier stores. +This is not a basic-consistency guarantee for a faulted method. BudgetExhausted +and debugger pauses are resumable instruction boundaries; host observation is +not atomic. No catch, poison flag or recovery API is introduced. Candidate nodes, +optional carriers, packing and call/operand preparation retain their normal +allocation, rooting and quota costs; immediate reclamation is not promised. ## GC proofs require an actual collection, not only a safepoint @@ -162,38 +163,55 @@ The full harness also needed its original scalar/input corpus selector to exclud the new Linked fixture prefix. Linked cases remain mandatory in their dedicated validator; they are not silently treated as older scalar fixtures or skipped. -## Qualification and limits +## Current qualification and historical delivery evidence -| Gate | Observed result | +The current pin is [the catalog revision](../../../docs/vm-arch/runtime-catalog.properties), +`dcf6561ea73446e0a3e1cf1c73eda20923b38df0`. The +[migration evidence](../DSC-0096-runtime-storage-isa-consumer-migration/runtime-conformance-evidence.json) +qualifies source checks, sequential stores and failure phases. Later complete +[debug/release evidence](../DSC-0097-pbs-sdk-stack-queue/evidence/README.md) includes +the current Linked corpus and both producers. No suites were rerun by this +editorial reconciliation. + +The table below records the original delivery on clean provider +`57992cf214e8a5f9de53a40dba9aaa7fab038c91`, over Studio base +`c410e3500150de5e6d55c1da8f9b22076ca84251`. Its original JSON preserves source/PBX +hashes, commands, lowered limits and GC observations unchanged. Compound publication +and whole-method refusal results in that evidence were superseded by LSN-0092; +they are not current guarantees or current-provider reproduction results. + +| Historical delivery gate | Result on the original pin | | --- | --- | | Full Studio regression | 1,030 tests; no failures/errors; 13 existing skips | | Installed knowledge | Both tests passed, including standalone Linked PBX generation | | Provider bytecode/VM libraries | 6 + 305 passed; 2 existing measurement tests ignored | | Linked source to production VM | 26 cases in each producer mode: 52 passed | -| Neutral storage/publication | Four common-IR fixtures executed, including ordinary/optimized FIELDS_PUBLISH | -| Whole-operation refusals | 231; all ten mutation/capture boundaries preserved previous source state | +| Retired publication fixtures | Four common-IR fixtures, including ordinary/optimized FIELDS_PUBLISH on the original pin | +| Original whole-operation refusals | 231; original mutation/capture boundaries preserved previous source state on that pin | | Linked removal/clear GC | 1,200 → 1,197 → 0 → 0 nodes and payloads | | Snapshot GC | 1,200 → 1 → 1 → 0 captured nodes/payloads, with no retained Linked topology | | Imports/installed traversal | Passed; installed forward/reverse final values 22/20 | | Catalogs and canonical mirrors | Clean selected provider; catalog/exporter parity and all four mirrors match | -The refusal matrix observed GuestBytes, HandleSlots and AuxiliaryBytes refusals. -TemporaryBytes was checked against the measured setup/success peaks; no additional -peak required a lowered refusal case in this Linked corpus. FIELDS_PUBLISH has no -scratch allocation, and the direct provider test proves publication succeeds with -TemporaryBytes already capped at current usage. This is not a claim that general -source preparation never needs temporary resources. +The original refusal matrix observed GuestBytes, HandleSlots and AuxiliaryBytes +refusals. TemporaryBytes was checked against its measured setup/success peaks; +no additional peak required a lowered refusal case in that historical corpus. +The former no-scratch FIELDS_PUBLISH tests belong only to the retired design. +Current source preparation, carriers and sequential stores retain normal quota +costs and use the phase-specific evidence linked above. Qualification used macOS 27.0.1 arm64, Rust 1.98.1/LLVM 22.1.8, the dev runtime profile with resource-diagnostics, capabilities NONE, Gradle 9.3.1 and the Java 21 -toolchain; the installed JVM was Corretto 21.0.4. No release or cross-platform claim -is made. Two pre-existing full-harness response-schema gaps, LifecycleRole and +toolchain; the installed JVM was Corretto 21.0.4. That historical run made no +release or cross-platform claim; later debug/release qualification is linked +separately above. Two pre-existing full-harness response-schema gaps, LifecycleRole and NonFiniteValue, remain explicitly reported with original payloads. ## Pitfalls to preserve -- Do not implement a multi-object mutation as a sequence of fallible stores. -- Prepare the optional removed-value carrier before publishing the removal. +- Prepare values and use source-selected sequential stores with explicit failure + phases. Do not claim whole-method atomicity or safe reuse after a later fault. +- Prepare the optional removed-value carrier before unlinking the node. - Preserve every field group and native/reference identity, including unchanged payloads when updating neighbor links. - Never expose source nodes to support traversal. Snapshot and collection links @@ -209,15 +227,16 @@ NonFiniteValue, remain explicitly reported with original payloads. - [Linked specification](../../../docs/compiler/pbs/specs/sdk-linked-and-cursor.md), [List/shared Cursor](../../../docs/compiler/pbs/specs/sdk-list-and-cursor.md), [generic structs](../../../docs/compiler/pbs/specs/generic-structs-and-free-functions.md) - and [generic Registry/PBX publication](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md). + and [generic Registry/PBX storage](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md). - [SDK source](../../../prometeu-compiler/frontends/prometeu-frontend-pbs/src/main/resources/game/stdlib/1/sdk/collections/main.pbs), [source corpus](../../../prometeu-architecture-tests/src/test/java/p/studio/compiler/backend/irvm/PbsLinkedTest.java), [neutral producer](../../../prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/StoragePublicationPipelineTest.java) and [production harness instructions](../../../tools/pbx-conformance/README.md). -- [Provider publication implementation](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/storage_publication.rs) - and [previous List/Cursor lesson](../DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md). +- [Current provider typed storage](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/collections.rs), + [sequential mutation migration](../DSC-0096-runtime-storage-isa-consumer-migration/LSN-0092-sequential-collection-mutations-need-explicit-failure-boundaries.md) + and [List/Cursor lesson](../DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md). Run `./gradlew test :prometeu-app:installedKnowledgeTest --console=plain`, followed -by `python3 tools/pbx-conformance/run.py --runtime ../runtime`, after selecting the -recorded provider. The evidence file retains command/configuration details and -R1–R12 traceability after removal of the completed workflow controls. +by `python3 tools/pbx-conformance/run.py --runtime ../runtime`, using the current +pin and the migration/later evidence for current qualification. The original JSON +retains its original provider/configuration and R1–R12 delivery traceability.