feat(pbs): add mutable lists and stable snapshot cursors

Implement SDK list<T> and Cursor<T> with trusted compiler bindings and generic storage publication across IR and PBX. Pin the runtime provider and include specs, conformance coverage, installed examples, and DSC-0090 lessons.
This commit is contained in:
2026-10-23 00:46:39 +01:00
parent 76a8b074e8
commit c410e35001
46 changed files with 2683 additions and 165 deletions
@@ -1,7 +1,7 @@
# Typed common Registry and PBX v0
Status: normative. Owner: `compiler/general`. Runtime contract reference:
`runtime` revision `f7d7c167531bb662a1af9d73fc326bb476bf8aa7`,
`runtime` revision `d9b9eeb171d2861ec660d4cf5703b0795eac49a9`,
`docs/specs/runtime/02c-program-registry-and-pbx-v0.md` and
`docs/vm-arch/ISA_CORE.md` in that repository.
@@ -186,3 +186,61 @@ Synthetic common IR additionally exercises linking, lowering, optimization and
emission without PBS. Frontend migration and integrated release readiness are
tracked separately in AGD75; passing common conformance does not claim complete
PBS lowering or complete Studio/runtime integration.
## Generic logical guards and atomic storage publication
The core includes the following generic operations, with u16 opcodes and
little-endian u32 immediates. They introduce no new Registry category, syscall,
source-language collection or ambient transaction. Operand order below is bottom
to top; all integer operands are Int64.
| Opcode | Immediate | Stack operands consumed |
| --- | --- | --- |
| INDEX_GUARD `0xA7` | none | index, exclusive limit |
| ARRAY_CHECK_LENGTH `0xA8` | Array TypeId | length |
| ARRAY_COPY `0xA9` | Array TypeId | source, sourceStart, target, targetStart, count |
| STORAGE_PUBLISH `0xAA` | Array TypeId, Struct TypeId, first field, second field | source, sourceStart, target, targetStart, count, fillStart, fillCount, element group, owner, first field group, second field group |
All operations return no values. INDEX_GUARD accepts exactly
`0 <= index < limit`, otherwise TRAP_OOB. It does not use physical storage length.
ARRAY_CHECK_LENGTH accepts zero through u32::MAX without allocating. Negative
lengths produce TRAP_OOB; values above u32::MAX produce the same resource
SizeOverflow failure as ARRAY_NEW. This check does not reserve future allocation.
ARRAY_COPY and STORAGE_PUBLISH require exact Array receiver identities and complete
stored groups. Every range must have nonnegative start/count and fit its physical
array; an empty range may start at length. Invalid ranges produce TRAP_OOB, never
clamping or truncation. Copy reads the pre-attempt source sequence, including for
overlapping source/target ranges. Presence tags, immutable boxes, native owners and
object identities are preserved without payload unpacking or repacking.
STORAGE_PUBLISH first copies the selected source range, then fills the selected
target range with the supplied element group, then writes the two selected fields
on the supplied Struct owner. Fields must be distinct valid indices on that Struct;
their types and widths come from the immutable Registry. Source code/producer owns
selection of every range, value, owner and field. The VM implements these generic
ordered effects and does not select collection algorithms.
Loading validates all immediate identities/categories/fields even in unreachable
code. Verification requires complete typed operands. ARRAY_COPY consumes five
slots; STORAGE_PUBLISH consumes `8 + elementWidth + firstWidth + secondWidth`.
INDEX_GUARD consumes two slots and ARRAY_CHECK_LENGTH consumes one.
Execution validates all receivers, lengths/ranges, supplied groups and source
groups, and reserves the complete charged copy scratch before writing. Validation,
quota or host allocation refusal leaves all input operands and destinations
unchanged, including both fields and native owners. Scratch uses TemporaryBytes,
is refunded on success/refusal and does not relax any resource ceiling. Existing
operands remain roots during preparation. There is no GC, callback, scheduler or
other interleaving inside this single instruction. Commit uses already validated
complete groups, performs no fallible allocation and consumes operands only after
all selected effects have been published. No prepared token escapes the attempt.
Base cycle costs are 2 for either guard, 4 for ARRAY_COPY and 6 for STORAGE_PUBLISH.
Copy adds `2 * count * elementWidth`; publication additionally adds
`fillCount * elementWidth + firstWidth + secondWidth`. Work arithmetic is checked
before mutation. A refused preparation still charges the attempted work. Neither
operation is a safepoint. Consumers must prepare any potentially failing return
carrier and all call operands before publication; a series of ordinary stores is
not a substitute for whole-operation safety.
@@ -0,0 +1,175 @@
# SDK List and snapshot Cursor
Owner: `compiler/pbs`. List and Cursor algorithms belong to the executable PBS
SDK. Logical guards, complete stored-group transfers and atomic publication are
language-neutral compiler/runtime mechanisms. This contract builds on
[arrays and optional](typed-arrays-and-optional-execution.md),
[generic structs](generic-structs-and-free-functions.md) and
[generic contracts](generic-contracts.md).
## Type, identity and construction
The SDK exports the ordinary generic struct `list<T>` and contract `Cursor<T>`
from its collections module. Applications are invariant. A directly optional
element, including through grouping, is invalid; `optional (list<T>)` is valid.
Nested valid arrays, structs, contracts, functions and value groups keep their
normal types and identities. An unrelated user declaration named list does not
acquire SDK behavior.
`new list<T>()` creates size zero and capacity 16. `new list<T>(capacity)` creates
size zero with exactly that capacity, including zero. Contextual `new list()`
and `new list(capacity)` require an expected concrete List type. Constructor
values do not infer T. Only the backing array is allocated; elements are absent.
List has stable reference identity. Assignment and parameter passing share that
identity; aliases observe mutations even when growth replaces its private backing
array. Storage, internal nodes and private compiler bindings are inaccessible to
clients. No public Dynamic or universal element type is introduced.
## Logical positions and operations
Every observable List satisfies `0 <= size <= capacity`. Size includes holes.
Capacity is physical storage and never makes additional indices accessible.
Zero and false are present values, distinct from holes.
| Operation | Result | Behavior |
| --- | --- | --- |
| `size()` | long | Logical positions, including holes |
| `capacity()` | long | Physical backing length |
| `xs[i]` | optional T | Present value or hole at `0 <= i < size` |
| `xs[i] = value` | unit | Replace one valid position without extending size |
| `add(value)` | unit | Append at size; do not fill earlier holes |
| `insert(i, value)` | unit | Insert at `0 <= i <= size`; shift slots and holes right |
| `clearAt(i)` | unit | Clear a valid position without shifting or changing size |
| `removeAt(i)` | optional T | Return old content, shift left, clear old tail, decrement size |
| `clear()` | unit | Clear all contents and size, preserve capacity |
| `resize(n)` | unit | Grow with holes or clear discarded suffix; set logical size |
| `reserve(n)` | unit | Grow exactly to n only when n exceeds capacity |
| `trim()` | unit | Set capacity exactly to size, including zero |
| `cursor()` | Cursor<T> | Complete immediate snapshot of present occurrences |
Removing a hole still removes its position and decrements size. Shrink, removal
and clear discard all hidden suffix references: regrowing must not resurrect old
values. Physical shrinking occurs only through trim. `resize(0)` preserves capacity.
Receiver and arguments are evaluated once in source order. Indices, lengths and
capacities accept int and long with localized lossless promotion; this does not
add general numeric conversion. No long value may be truncated.
Invalid logical indices raise TRAP_OOB, including indices inside physical capacity.
Negative lengths, structural overflow and quota refusals preserve the applicable
Array/provider fault classification. Failures are never translated into none.
## Capacity policy
When required size r exceeds capacity c, automatic growth is:
```text
candidate = max(r, c + max(1, ceil(c / 2)))
new capacity = min(Array structural maximum, candidate)
```
Calculate without overflow. Clamp only if r itself fits the structural maximum;
otherwise raise the applicable provider failure. Explicit reserve uses exactly n,
never shrinks and preserves size. Trim uses exactly size. Old and candidate arrays
may coexist; neither discarded references nor replaced storage promise immediate
GC or immediate quota recovery.
## Complete mutation attempts
Private storage transfer preserves whole existing groups, presence tags and
reference identities. Shifting must not unpack each public Array optional and
repack/rebox every payload. Prepare potentially failing validation, packing,
allocations, return carriers, scratch and publication operands before modifying
observable storage. Then publish the complete selected slot/range and field
changes without a fallible allocation, callback, scheduler or GC boundary.
Any refused attempt leaves size, backing reference, capacity, slots and roots
unchanged. This includes insert/remove without growth and preparation of
removeAt's optional result. Individually atomic Array instructions do not by
themselves establish this guarantee. Previously evaluated argument effects are
not rolled back. Unpublished candidates obey normal ownership and budgets.
## Cursor protocol
```pbs
declare contract Cursor<T> {
fn hasNext() -> bool;
fn next();
fn current() -> optional T;
}
```
The shared SDK implementation captures one private immutable singly linked,
non-circular node per present occurrence. It uses no snapshot array and retains
no source-slot or editable source-node association. List capture scans logical
positions and preserves increasing index order, duplicates and complete values.
Capture finishes before returning; failure exposes no partial Cursor.
A nonempty Cursor starts on its first node. `hasNext()` tests for a successor.
`next()` advances only when a successor exists; otherwise it does nothing.
`current()` reports the current node without advancing. Only a snapshot born
empty reports none. Repeated calls never advance beyond the last node.
| Snapshot and position | hasNext() | current() |
| --- | --- | --- |
| Empty, initially or after any next | false | none |
| [A], initially or after any next | false | some(A) |
| [A, B], initially | true | some(A) |
| [A, B], after one or any further next | false | some(B) |
Consume the current value before deciding whether to advance:
```pbs
while true {
if let value = cursor.current() {
// Consume value.
} else {
break;
}
if cursor.hasNext() {
cursor.next();
} else {
break;
}
}
```
A hasNext-only loop skips the last element. Waiting for current to become none
does not terminate a nonempty snapshot. There is no index, reset, removal, clone
or new for-in protocol. Another complete traversal creates another snapshot.
Captured object references preserve identity and observe subsequent field mutation.
Later List insertions, replacements, removals, clear, resize and backing changes
do not alter the captured sequence. Snapshot links and captured values are not
rewritten. The Cursor retains only its current node and suffix, allowing consumed
prefixes to become unreachable. It keeps the final node/value while reachable.
Other owners remain independent; reclamation happens through normal GC.
The outer optional describes node existence. A future producer with optional T
must distinguish `some(none)` from an empty Cursor. Type/resource faults remain
faults. List itself rejects directly optional elements.
## Reuse, costs and conformance
The shared implementation is independent of List representation. Future Linked<T>
and Set<T> producers use Cursor<T>; Map<K,V> uses Cursor<keyval<K,V>>. Set/Map gain
no ordering guarantee between captures. A captured keyval association is read-only
and does not follow later association replacement; referenced objects stay shared.
Those collections and keyval are separate deliveries.
For n logical List positions and k present occurrences, capture takes O(n) work
and O(k) additional nodes; next follows one link in O(1). Each node costs allocation,
ownership and eventual GC work. Do not promise lower memory than arrays, zero
allocation or allocation-free optional results. All complete groups remain rooted
through allocations and calls; provider quotas remain unchanged.
Conformance must execute real PBS through common IR and PBX in the production VM.
It covers every operation, constructor/type rule, logical bounds, exact growth,
holes and full groups, aliases, the entire Cursor table, snapshot independence,
shared object mutation, non-List producers, full-operation refusals with and without
growth, prefix reclamation, last-node retention, editor projections and installed
SDK/spec/example distribution. Generic publication also requires non-PBS tests.
Value search/removal, sorting, bulk APIs, new equality, public nodes, deep freezing,
general effect/transaction syntax and Linked/Set/Map implementations are out of scope.
@@ -81,7 +81,10 @@ Conformance must compile real PBS through typed IR and production PBX into the p
- [Dynamic semantics](../../../specs/compiler-languages/pbs/9.%20Dynamic%20Semantics%20Specification.md)
- [Existing type runtime alignment](input-and-existing-type-runtime-alignment.md)
List/Set/Map, growth, iteration, equality and hashing policies belong to the separate SDK collections discussion. A future list capacity does not create logical elements; fixed array length does.
[SDK List and Cursor](sdk-list-and-cursor.md) define logical size, growth and
snapshot traversal above private fixed-length arrays. List capacity does not
create logical elements; fixed array length does. Set/Map, equality and hashing
policies remain separate concerns.
See [Runtime Random integration](runtime-random-integration.md) for opaque generator aliases in typed storage and captures.