feat(pbs): add linked collections and bidirectional snapshots

Implement SDK linked<T> with forward and reverse Cursor snapshots, trusted compiler integration, and neutral FIELDS_PUBLISH producer support. Pin the tested runtime revision and include specs, editor and installed knowledge coverage, conformance fixtures, and DSC-0091 lessons and evidence.
This commit is contained in:
2026-10-23 10:28:44 +01:00
parent c410e35001
commit 77aada6259
38 changed files with 3040 additions and 57 deletions
@@ -0,0 +1,223 @@
---
id: LSN-0091
ticket: pbs-sdk-linked
title: Private Linked Nodes and Atomic Publication
created: 2026-10-23
tags: [compiler-pbs, collections, snapshot-cursor, atomic-publication, runtime-conformance]
---
## Original problem and final result
Owner: `compiler/pbs`. Transversal owners: `compiler/general` for neutral IR/PBX
and the provider for typed storage, faults, ownership, GC and resource accounting.
DSC-0091 executed accepted DEC-0076 through PLN-0154, covering R1–R12. Their
workflow control artifacts are disposable; the [Linked specification](../../../docs/compiler/pbs/specs/sdk-linked-and-cursor.md)
is the durable normative reference.
PBS needed a dense sequence with private doubly linked nodes and useful end
operations, while preserving the shared snapshot traversal already delivered for
List. `linked<T>` now supplies that sequence through executable SDK source.
Forward and reverse captures both return the existing Cursor<T> contract. A
neutral FIELDS_PUBLISH operation provides complete publication across the collection
and its neighbors, without moving node algorithms into the VM.
The selected provider is clean commit `57992cf214e8a5f9de53a40dba9aaa7fab038c91`.
Studio changes were qualified over base `c410e3500150de5e6d55c1da8f9b22076ca84251`.
[Machine-readable evidence](runtime-conformance-evidence.json) records source/PBX
hashes, commands, configuration, lowered refusal limits and observed GC states.
## Dense positions do not change the optional API
A Linked contains one present value per private node. Its size counts nodes;
capacity, holes, reserve, trim, resize and clearAt do not exist. Valid indexed
reads/removals return a present optional, and invalid indices fault even when
empty. First/last and their removal operations return none for an empty collection.
This preserves the accepted API rather than changing return types because a
particular representation cannot contain holes.
Construction is empty and takes no capacity. Explicit and contextual applications
follow the existing generic struct rules. Directly optional elements are rejected,
including through grouping, just as in Array/List. An optional collection and an
otherwise valid element containing optional fields remain valid.
The executable API includes size, first/last, bracket read/write, add/addFirst,
insert, removeAt/removeFirst/removeLast, clear and both capture methods. Adding
appends, while addFirst prepends. Replacement preserves size; insertion/removal
changes following logical indices. Read/replace/remove require an index below
size; insert also accepts size. Long indices are not truncated, int arguments have
the same localized lossless promotion as List, and growth is checked before signed
long overflow without inheriting Array's structural size maximum.
Preserving inserted object identity is separate from choosing a comparison policy.
There is no remove(value), generic equality or uniqueness rule in this delivery.
Repeated occurrences, including repeated references to one instance, remain
independent sequence positions. List similarly leaves value search/removal outside
its current contract.
## Keep collection topology separate from traversal topology
Linked nodes have reciprocal previous/next links. The SDK locates indexed values
from the nearer end, so search remains O(n) in the worst case; end operations do
not search. A constant number of link updates does not make indexed access O(1).
Snapshot nodes are different objects: private immutable singly linked nodes with
no circular links or editable source association. Forward capture walks Linked
backwards and prepends; reverse capture walks it forwards and prepends. Both cost
O(n) work and additional nodes, and both finish before returning a Cursor.
For a source `[20, 22]`, forward starts at 20 and reverse starts at 22. Replacing
source values, removing elements or clearing the collection cannot change either
captured sequence. Referenced objects remain shared, so later field mutations
remain visible. Snapshots preserve occurrences and object identity, not deep copies.
The existing state table remains the only traversal protocol:
| State | hasNext() | current() | next() |
| --- | --- | --- | --- |
| Empty | false | none | No change |
| Current node has a successor | true | Present current value | Follow successor |
| Last node, including a singleton | false | Present final value | No change |
Process current before testing whether to advance. A hasNext-only loop omits the
last element; waiting for current to become absent never finishes a nonempty
snapshot. There is no removal/reset through Cursor.
The [distributed example](../../../prometeu-compiler/frontends/prometeu-frontend-pbs/src/main/resources/pbs-knowledge/examples/game/1/src/linked.pbs)
captures both orders, then replaces a source element and clears Linked. Both
traversals still sum to 42, retaining final values 22 and 20 respectively.
## Prepare every object before changing any object
Insertion can update the collection's head/tail/size and two existing neighbors.
Removal also prepares the returned optional value. Separate successful field
stores are insufficient: a later allocation or frame/operand refusal could leave
half of a neighbor pair updated.
The existing range-based STORAGE_PUBLISH operation covered an Array and two
fields of one owner. Its guarantee did not automatically extend to Linked.
The new generic FIELDS_PUBLISH uses a normal Struct TypeId as a schema: each
schema field names one destination Struct type. The stack supplies each nullable
destination followed by all its complete field groups. No request object, new
Registry category or public preparation token is needed.
All non-null destinations must have the exact concrete type and live provenance.
All supplied value groups are validated, including those paired with a null
(skipped) destination. The original operands stay rooted and unconsumed until the
whole attempt is admitted. The VM then commits in schema order without allocation,
validation, callbacks or GC/scheduler interleaving, and consumes the operands.
Repeated destinations have ordered last-write semantics using pre-captured inputs.
The SDK selects topology and values. It allocates and initializes a private
candidate node before publishing an insertion. Removal prepares its optional
result first. Private helper bindings lower those source-selected values to the
neutral operation. A refusal preserves the old roots, size, values and reciprocal
links; already evaluated caller argument effects remain ordinary source effects.
Publication itself needs no heap object or scratch. This does not make a source
mutation allocation-free: optional carriers, candidate nodes, packing and call
preparation still have their normal costs. Provider tests execute publication at
already exhausted allocation ceilings, preserve native String/reference identity
and multi-slot groups, and reject a late invalid group before every shared write.
## GC proofs require an actual collection, not only a safepoint
Removal disconnects a node from the collection. Clear removes the collection's
roots to the whole chain. Unreachable double-link cycles are handled by the normal
tracing collector; neither operation promises immediate collection or quota return.
A retained snapshot owns captured values through its own nodes and does not need
the editable Linked nodes.
A controlled fixture uses normal GC safepoints and independent non-owning storage
observations. With no snapshot, nodes and payloads fall from 1,200 to 1,197 after
three removals, then to zero after clear. A second fixture clears the source while
retaining its snapshot: original Linked nodes disappear, captured nodes/items go
from 1,200/1,200 to 1/1 after traversal, stay at 1/1 after repeated next, and reach
0/0 after the final Cursor owner is dropped.
A safepoint alone does not force this collector to run. The removal/clear fixture
therefore creates ordinary garbage pressure and asserts that the collection
counter advances at each observed round. Without that check, a test can mistake
an uncollected unreachable node for a retention bug. The observer introduces no
guest root or mutation.
## Compiler, tooling and distribution
Linked reuses ordinary generic specialization, methods, field groups and imports.
Origin-checked SDK metadata adds constructor/index integration and the same
localized int-to-long argument promotion as List. User homonyms or copied
attributes do not acquire those privileges.
Public projections show concrete Linked methods, optional results and source SDK
locations. They hide head/tail/length, private nodes and storage adapters, including
the internal nearest-end lookup method. Negative workspace cases reject optional
elements, capacity construction, unsupported APIs and private access. Invalid
constructor selection now reports diagnostics instead of throwing from lowering.
The normative English spec and example ship with installed knowledge. An isolated
installDist JVM, outside the checkout, loads the actual JAR resources and compiles
the Linked example. The production harness executes its emitted PBX and observes
both completed traversals. Imports/reexports are qualified separately.
The full harness also needed its original scalar/input corpus selector to exclude
the new Linked fixture prefix. Linked cases remain mandatory in their dedicated
validator; they are not silently treated as older scalar fixtures or skipped.
## Qualification and limits
| Gate | Observed result |
| --- | --- |
| Full Studio regression | 1,030 tests; no failures/errors; 13 existing skips |
| Installed knowledge | Both tests passed, including standalone Linked PBX generation |
| Provider bytecode/VM libraries | 6 + 305 passed; 2 existing measurement tests ignored |
| Linked source to production VM | 26 cases in each producer mode: 52 passed |
| Neutral storage/publication | Four common-IR fixtures executed, including ordinary/optimized FIELDS_PUBLISH |
| Whole-operation refusals | 231; all ten mutation/capture boundaries preserved previous source state |
| Linked removal/clear GC | 1,200 → 1,197 → 0 → 0 nodes and payloads |
| Snapshot GC | 1,200 → 1 → 1 → 0 captured nodes/payloads, with no retained Linked topology |
| Imports/installed traversal | Passed; installed forward/reverse final values 22/20 |
| Catalogs and canonical mirrors | Clean selected provider; catalog/exporter parity and all four mirrors match |
The refusal matrix observed GuestBytes, HandleSlots and AuxiliaryBytes refusals.
TemporaryBytes was checked against the measured setup/success peaks; no additional
peak required a lowered refusal case in this Linked corpus. FIELDS_PUBLISH has no
scratch allocation, and the direct provider test proves publication succeeds with
TemporaryBytes already capped at current usage. This is not a claim that general
source preparation never needs temporary resources.
Qualification used macOS 27.0.1 arm64, Rust 1.98.1/LLVM 22.1.8, the dev runtime
profile with resource-diagnostics, capabilities NONE, Gradle 9.3.1 and the Java 21
toolchain; the installed JVM was Corretto 21.0.4. No release or cross-platform claim
is made. Two pre-existing full-harness response-schema gaps, LifecycleRole and
NonFiniteValue, remain explicitly reported with original payloads.
## Pitfalls to preserve
- Do not implement a multi-object mutation as a sequence of fallible stores.
- Prepare the optional removed-value carrier before publishing the removal.
- Preserve every field group and native/reference identity, including unchanged
payloads when updating neighbor links.
- Never expose source nodes to support traversal. Snapshot and collection links
have different mutability and lifetime responsibilities.
- Keep optional results despite dense positions, and faults distinct from absence.
- Keep private SDK authority tied to origin through aliases and specialization.
- Check GC collection counters when asserting reclamation at a safepoint.
- Record clean provider provenance, explicit lowered limits and mandatory fixture
counts; metadata defaults or small unit tests alone do not qualify execution.
## References and reproduction
- [Linked specification](../../../docs/compiler/pbs/specs/sdk-linked-and-cursor.md),
[List/shared Cursor](../../../docs/compiler/pbs/specs/sdk-list-and-cursor.md),
[generic structs](../../../docs/compiler/pbs/specs/generic-structs-and-free-functions.md)
and [generic Registry/PBX publication](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md).
- [SDK source](../../../prometeu-compiler/frontends/prometeu-frontend-pbs/src/main/resources/game/stdlib/1/sdk/collections/main.pbs),
[source corpus](../../../prometeu-architecture-tests/src/test/java/p/studio/compiler/backend/irvm/PbsLinkedTest.java),
[neutral producer](../../../prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/StoragePublicationPipelineTest.java)
and [production harness instructions](../../../tools/pbx-conformance/README.md).
- [Provider publication implementation](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/storage_publication.rs)
and [previous List/Cursor lesson](../DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md).
Run `./gradlew test :prometeu-app:installedKnowledgeTest --console=plain`, followed
by `python3 tools/pbx-conformance/run.py --runtime ../runtime`, after selecting the
recorded provider. The evidence file retains command/configuration details and
R1–R12 traceability after removal of the completed workflow controls.
File diff suppressed because it is too large Load Diff