feat(pbs): add linked collections and bidirectional snapshots
Implement SDK linked<T> with forward and reverse Cursor snapshots, trusted compiler integration, and neutral FIELDS_PUBLISH producer support. Pin the tested runtime revision and include specs, editor and installed knowledge coverage, conformance fixtures, and DSC-0091 lessons and evidence.
This commit is contained in:
+223
@@ -0,0 +1,223 @@
|
||||
---
|
||||
id: LSN-0091
|
||||
ticket: pbs-sdk-linked
|
||||
title: Private Linked Nodes and Atomic Publication
|
||||
created: 2026-10-23
|
||||
tags: [compiler-pbs, collections, snapshot-cursor, atomic-publication, runtime-conformance]
|
||||
---
|
||||
|
||||
## Original problem and final result
|
||||
|
||||
Owner: `compiler/pbs`. Transversal owners: `compiler/general` for neutral IR/PBX
|
||||
and the provider for typed storage, faults, ownership, GC and resource accounting.
|
||||
DSC-0091 executed accepted DEC-0076 through PLN-0154, covering R1–R12. Their
|
||||
workflow control artifacts are disposable; the [Linked specification](../../../docs/compiler/pbs/specs/sdk-linked-and-cursor.md)
|
||||
is the durable normative reference.
|
||||
|
||||
PBS needed a dense sequence with private doubly linked nodes and useful end
|
||||
operations, while preserving the shared snapshot traversal already delivered for
|
||||
List. `linked<T>` now supplies that sequence through executable SDK source.
|
||||
Forward and reverse captures both return the existing Cursor<T> contract. A
|
||||
neutral FIELDS_PUBLISH operation provides complete publication across the collection
|
||||
and its neighbors, without moving node algorithms into the VM.
|
||||
|
||||
The selected provider is clean commit `57992cf214e8a5f9de53a40dba9aaa7fab038c91`.
|
||||
Studio changes were qualified over base `c410e3500150de5e6d55c1da8f9b22076ca84251`.
|
||||
[Machine-readable evidence](runtime-conformance-evidence.json) records source/PBX
|
||||
hashes, commands, configuration, lowered refusal limits and observed GC states.
|
||||
|
||||
## Dense positions do not change the optional API
|
||||
|
||||
A Linked contains one present value per private node. Its size counts nodes;
|
||||
capacity, holes, reserve, trim, resize and clearAt do not exist. Valid indexed
|
||||
reads/removals return a present optional, and invalid indices fault even when
|
||||
empty. First/last and their removal operations return none for an empty collection.
|
||||
This preserves the accepted API rather than changing return types because a
|
||||
particular representation cannot contain holes.
|
||||
|
||||
Construction is empty and takes no capacity. Explicit and contextual applications
|
||||
follow the existing generic struct rules. Directly optional elements are rejected,
|
||||
including through grouping, just as in Array/List. An optional collection and an
|
||||
otherwise valid element containing optional fields remain valid.
|
||||
|
||||
The executable API includes size, first/last, bracket read/write, add/addFirst,
|
||||
insert, removeAt/removeFirst/removeLast, clear and both capture methods. Adding
|
||||
appends, while addFirst prepends. Replacement preserves size; insertion/removal
|
||||
changes following logical indices. Read/replace/remove require an index below
|
||||
size; insert also accepts size. Long indices are not truncated, int arguments have
|
||||
the same localized lossless promotion as List, and growth is checked before signed
|
||||
long overflow without inheriting Array's structural size maximum.
|
||||
|
||||
Preserving inserted object identity is separate from choosing a comparison policy.
|
||||
There is no remove(value), generic equality or uniqueness rule in this delivery.
|
||||
Repeated occurrences, including repeated references to one instance, remain
|
||||
independent sequence positions. List similarly leaves value search/removal outside
|
||||
its current contract.
|
||||
|
||||
## Keep collection topology separate from traversal topology
|
||||
|
||||
Linked nodes have reciprocal previous/next links. The SDK locates indexed values
|
||||
from the nearer end, so search remains O(n) in the worst case; end operations do
|
||||
not search. A constant number of link updates does not make indexed access O(1).
|
||||
|
||||
Snapshot nodes are different objects: private immutable singly linked nodes with
|
||||
no circular links or editable source association. Forward capture walks Linked
|
||||
backwards and prepends; reverse capture walks it forwards and prepends. Both cost
|
||||
O(n) work and additional nodes, and both finish before returning a Cursor.
|
||||
|
||||
For a source `[20, 22]`, forward starts at 20 and reverse starts at 22. Replacing
|
||||
source values, removing elements or clearing the collection cannot change either
|
||||
captured sequence. Referenced objects remain shared, so later field mutations
|
||||
remain visible. Snapshots preserve occurrences and object identity, not deep copies.
|
||||
|
||||
The existing state table remains the only traversal protocol:
|
||||
|
||||
| State | hasNext() | current() | next() |
|
||||
| --- | --- | --- | --- |
|
||||
| Empty | false | none | No change |
|
||||
| Current node has a successor | true | Present current value | Follow successor |
|
||||
| Last node, including a singleton | false | Present final value | No change |
|
||||
|
||||
Process current before testing whether to advance. A hasNext-only loop omits the
|
||||
last element; waiting for current to become absent never finishes a nonempty
|
||||
snapshot. There is no removal/reset through Cursor.
|
||||
|
||||
The [distributed example](../../../prometeu-compiler/frontends/prometeu-frontend-pbs/src/main/resources/pbs-knowledge/examples/game/1/src/linked.pbs)
|
||||
captures both orders, then replaces a source element and clears Linked. Both
|
||||
traversals still sum to 42, retaining final values 22 and 20 respectively.
|
||||
|
||||
## Prepare every object before changing any object
|
||||
|
||||
Insertion can update the collection's head/tail/size and two existing neighbors.
|
||||
Removal also prepares the returned optional value. Separate successful field
|
||||
stores are insufficient: a later allocation or frame/operand refusal could leave
|
||||
half of a neighbor pair updated.
|
||||
|
||||
The existing range-based STORAGE_PUBLISH operation covered an Array and two
|
||||
fields of one owner. Its guarantee did not automatically extend to Linked.
|
||||
The new generic FIELDS_PUBLISH uses a normal Struct TypeId as a schema: each
|
||||
schema field names one destination Struct type. The stack supplies each nullable
|
||||
destination followed by all its complete field groups. No request object, new
|
||||
Registry category or public preparation token is needed.
|
||||
|
||||
All non-null destinations must have the exact concrete type and live provenance.
|
||||
All supplied value groups are validated, including those paired with a null
|
||||
(skipped) destination. The original operands stay rooted and unconsumed until the
|
||||
whole attempt is admitted. The VM then commits in schema order without allocation,
|
||||
validation, callbacks or GC/scheduler interleaving, and consumes the operands.
|
||||
Repeated destinations have ordered last-write semantics using pre-captured inputs.
|
||||
|
||||
The SDK selects topology and values. It allocates and initializes a private
|
||||
candidate node before publishing an insertion. Removal prepares its optional
|
||||
result first. Private helper bindings lower those source-selected values to the
|
||||
neutral operation. A refusal preserves the old roots, size, values and reciprocal
|
||||
links; already evaluated caller argument effects remain ordinary source effects.
|
||||
|
||||
Publication itself needs no heap object or scratch. This does not make a source
|
||||
mutation allocation-free: optional carriers, candidate nodes, packing and call
|
||||
preparation still have their normal costs. Provider tests execute publication at
|
||||
already exhausted allocation ceilings, preserve native String/reference identity
|
||||
and multi-slot groups, and reject a late invalid group before every shared write.
|
||||
|
||||
## GC proofs require an actual collection, not only a safepoint
|
||||
|
||||
Removal disconnects a node from the collection. Clear removes the collection's
|
||||
roots to the whole chain. Unreachable double-link cycles are handled by the normal
|
||||
tracing collector; neither operation promises immediate collection or quota return.
|
||||
A retained snapshot owns captured values through its own nodes and does not need
|
||||
the editable Linked nodes.
|
||||
|
||||
A controlled fixture uses normal GC safepoints and independent non-owning storage
|
||||
observations. With no snapshot, nodes and payloads fall from 1,200 to 1,197 after
|
||||
three removals, then to zero after clear. A second fixture clears the source while
|
||||
retaining its snapshot: original Linked nodes disappear, captured nodes/items go
|
||||
from 1,200/1,200 to 1/1 after traversal, stay at 1/1 after repeated next, and reach
|
||||
0/0 after the final Cursor owner is dropped.
|
||||
|
||||
A safepoint alone does not force this collector to run. The removal/clear fixture
|
||||
therefore creates ordinary garbage pressure and asserts that the collection
|
||||
counter advances at each observed round. Without that check, a test can mistake
|
||||
an uncollected unreachable node for a retention bug. The observer introduces no
|
||||
guest root or mutation.
|
||||
|
||||
## Compiler, tooling and distribution
|
||||
|
||||
Linked reuses ordinary generic specialization, methods, field groups and imports.
|
||||
Origin-checked SDK metadata adds constructor/index integration and the same
|
||||
localized int-to-long argument promotion as List. User homonyms or copied
|
||||
attributes do not acquire those privileges.
|
||||
|
||||
Public projections show concrete Linked methods, optional results and source SDK
|
||||
locations. They hide head/tail/length, private nodes and storage adapters, including
|
||||
the internal nearest-end lookup method. Negative workspace cases reject optional
|
||||
elements, capacity construction, unsupported APIs and private access. Invalid
|
||||
constructor selection now reports diagnostics instead of throwing from lowering.
|
||||
|
||||
The normative English spec and example ship with installed knowledge. An isolated
|
||||
installDist JVM, outside the checkout, loads the actual JAR resources and compiles
|
||||
the Linked example. The production harness executes its emitted PBX and observes
|
||||
both completed traversals. Imports/reexports are qualified separately.
|
||||
|
||||
The full harness also needed its original scalar/input corpus selector to exclude
|
||||
the new Linked fixture prefix. Linked cases remain mandatory in their dedicated
|
||||
validator; they are not silently treated as older scalar fixtures or skipped.
|
||||
|
||||
## Qualification and limits
|
||||
|
||||
| Gate | Observed result |
|
||||
| --- | --- |
|
||||
| Full Studio regression | 1,030 tests; no failures/errors; 13 existing skips |
|
||||
| Installed knowledge | Both tests passed, including standalone Linked PBX generation |
|
||||
| Provider bytecode/VM libraries | 6 + 305 passed; 2 existing measurement tests ignored |
|
||||
| Linked source to production VM | 26 cases in each producer mode: 52 passed |
|
||||
| Neutral storage/publication | Four common-IR fixtures executed, including ordinary/optimized FIELDS_PUBLISH |
|
||||
| Whole-operation refusals | 231; all ten mutation/capture boundaries preserved previous source state |
|
||||
| Linked removal/clear GC | 1,200 → 1,197 → 0 → 0 nodes and payloads |
|
||||
| Snapshot GC | 1,200 → 1 → 1 → 0 captured nodes/payloads, with no retained Linked topology |
|
||||
| Imports/installed traversal | Passed; installed forward/reverse final values 22/20 |
|
||||
| Catalogs and canonical mirrors | Clean selected provider; catalog/exporter parity and all four mirrors match |
|
||||
|
||||
The refusal matrix observed GuestBytes, HandleSlots and AuxiliaryBytes refusals.
|
||||
TemporaryBytes was checked against the measured setup/success peaks; no additional
|
||||
peak required a lowered refusal case in this Linked corpus. FIELDS_PUBLISH has no
|
||||
scratch allocation, and the direct provider test proves publication succeeds with
|
||||
TemporaryBytes already capped at current usage. This is not a claim that general
|
||||
source preparation never needs temporary resources.
|
||||
|
||||
Qualification used macOS 27.0.1 arm64, Rust 1.98.1/LLVM 22.1.8, the dev runtime
|
||||
profile with resource-diagnostics, capabilities NONE, Gradle 9.3.1 and the Java 21
|
||||
toolchain; the installed JVM was Corretto 21.0.4. No release or cross-platform claim
|
||||
is made. Two pre-existing full-harness response-schema gaps, LifecycleRole and
|
||||
NonFiniteValue, remain explicitly reported with original payloads.
|
||||
|
||||
## Pitfalls to preserve
|
||||
|
||||
- Do not implement a multi-object mutation as a sequence of fallible stores.
|
||||
- Prepare the optional removed-value carrier before publishing the removal.
|
||||
- Preserve every field group and native/reference identity, including unchanged
|
||||
payloads when updating neighbor links.
|
||||
- Never expose source nodes to support traversal. Snapshot and collection links
|
||||
have different mutability and lifetime responsibilities.
|
||||
- Keep optional results despite dense positions, and faults distinct from absence.
|
||||
- Keep private SDK authority tied to origin through aliases and specialization.
|
||||
- Check GC collection counters when asserting reclamation at a safepoint.
|
||||
- Record clean provider provenance, explicit lowered limits and mandatory fixture
|
||||
counts; metadata defaults or small unit tests alone do not qualify execution.
|
||||
|
||||
## References and reproduction
|
||||
|
||||
- [Linked specification](../../../docs/compiler/pbs/specs/sdk-linked-and-cursor.md),
|
||||
[List/shared Cursor](../../../docs/compiler/pbs/specs/sdk-list-and-cursor.md),
|
||||
[generic structs](../../../docs/compiler/pbs/specs/generic-structs-and-free-functions.md)
|
||||
and [generic Registry/PBX publication](../../../docs/compiler/general/specs/typed-registry-and-pbx-v0.md).
|
||||
- [SDK source](../../../prometeu-compiler/frontends/prometeu-frontend-pbs/src/main/resources/game/stdlib/1/sdk/collections/main.pbs),
|
||||
[source corpus](../../../prometeu-architecture-tests/src/test/java/p/studio/compiler/backend/irvm/PbsLinkedTest.java),
|
||||
[neutral producer](../../../prometeu-compiler/prometeu-build-pipeline/src/test/java/p/studio/compiler/backend/registry/StoragePublicationPipelineTest.java)
|
||||
and [production harness instructions](../../../tools/pbx-conformance/README.md).
|
||||
- [Provider publication implementation](../../../../runtime/crates/console/prometeu-vm/src/virtual_machine/storage_publication.rs)
|
||||
and [previous List/Cursor lesson](../DSC-0090-pbs-sdk-list/LSN-0090-mutable-lists-and-stable-snapshot-cursors.md).
|
||||
|
||||
Run `./gradlew test :prometeu-app:installedKnowledgeTest --console=plain`, followed
|
||||
by `python3 tools/pbx-conformance/run.py --runtime ../runtime`, after selecting the
|
||||
recorded provider. The evidence file retains command/configuration details and
|
||||
R1–R12 traceability after removal of the completed workflow controls.
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user