feat(vm): add reference identity and deterministic value hashing

This commit is contained in:
2026-10-23 14:29:00 +01:00
parent 57992cf214
commit 69ad1931de
41 changed files with 2521 additions and 121 deletions
@@ -192,3 +192,13 @@ remain unchanged. CPU timings never change guest-visible cycle weights.
## Random core integration
The [random core](02e-vm-owned-random-core.md) charges fixed calls before checks and bounded range calls as 4+8*N (at most 64 draws), retaining fault cycles and whole-instruction overshoot.
## Identity and hash attempts
The [identity/hash contract](02f-vm-identity-and-value-hashing.md) charges
8+w(T) for reference hash, 4+w(A)+w(B) for identity comparison, eight for scalar
numeric hashes, and 8+n resident UTF-8 bytes for String. Known lengths charge
before traversal/refusal; null/wrong/unavailable String carriers charge eight.
Checked cost overflow charges base and faults. These are total attempted costs,
without a generic extra INTRINSIC cycle or refunds. Existing whole-instruction
overshoot and zero-budget rules apply; the operations introduce no safepoint.
@@ -253,3 +253,11 @@ entry and one per destination field execution slot for the complete declared
attempt, including skipped destinations. Quotas are unchanged.
This is a generic typed storage mechanism; collection algorithms and node topology
remain source-owned, with no collection-specific VM instruction or syscall.
## Identity and value hashing
[Identity and value hashing](02f-vm-identity-and-value-hashing.md) defines
REF_IDENTITY_HASH 0x0097 and REF_IDENTITY_EQ 0x0098, complete typed source
groups, one/two TypeId immediates, exact attempted costs and operand-preserving
faults. It also defines intrinsics 0x4100–0x4102 and their shared bounded
invocation profile. Old EQ and REF_* categories retain their contracts.
@@ -315,3 +315,13 @@ STORAGE_PUBLISH (0xAA) provide generic logical validation, complete stored-group
transfer and prepared atomic publication. Their encoding, typed operands, exact
faults, overlap semantics, costs and preflight/commit guarantees are defined by
the [canonical instruction contract](02-vm-instruction-set.md#logical-guards-and-atomic-storage-publication). No new Registry category, syscall, collection algorithm or safepoint is introduced.
## Identity queries
[Identity queries](02f-vm-identity-and-value-hashing.md) add opcodes 0x0097
(source TypeId, six bytes) and 0x0098 (left/right TypeIds, ten bytes). Sources
are Struct, Array, Fn, Trait and Random; validate complete groups before
consulting handles. They add no PBX section, value type or layout change. The
older REF_IS_EXACT/IMPLEMENTS/TO_TRAIT/CAST category restrictions remain intact.
Intrinsic argument nullability is explicit per entry: hash.string admits known
null at load without relaxing the static masks of existing String intrinsics.
@@ -0,0 +1,178 @@
# VM Identity, Value Hashing and Bounded Intrinsic Invocation
Domain: VM values/intrinsics. Function: normative.
## Operations
| Operation | ID | Immediate/inputs, bottom to top | Result | Attempt cycles |
| --- | --- | --- | --- | --- |
| REF_IDENTITY_HASH | opcode 0x0097 | source:TypeId; complete source group | Int64 | 8+w(source) |
| REF_IDENTITY_EQ | opcode 0x0098 | left:TypeId,right:TypeId; left group,right group | Bool | 4+w(left)+w(right) |
| hash.i64@1 | intrinsic 0x4100 | value:Int64 | Int64 | 8 |
| hash.float@1 | intrinsic 0x4101 | value:Float | Int64 | 8 |
| hash.string@1 | intrinsic 0x4102 | value:String, nullable | Int64 | 8+n |
Opcodes are u16 LE, immediate TypeIds u32 LE. The reference instructions occupy
six/ten bytes. INTRINSIC remains opcode 0x0072 followed by u32 ID, six bytes.
w(T) is resolved Registry slot width; n is present UTF-8 byte length, zero for
null. Success consumes complete groups and publishes one result; failure retains
inputs/owners. TypeIds are static metadata, never arbitrary guest selectors.
## Identity and equality domains
Identity sources MUST be Struct, Array, Fn, Trait or Random. String, Dynamic,
internal boxes and other builtins are invalid categories. The old REF_* source
rules and global EQ are unchanged. Validate the complete left group before the
right, and both before comparing: exact type/category, liveness/generation and
Registry. An invalid right input cannot be hidden by an unequal/absent left.
Struct/Array identity is instance identity, without traversing fields/elements.
Fn identity is Closure identity, including empty captures; distinct creation
stays distinct even with identical code/captures. Random identity is generator
identity, independent of seed/state; the query neither advances nor hashes state.
Trait identity is the receiver after validating the entire receiver and
ImplementationId against owner/trait metadata. ImplementationId zero is valid
for a coherent present view. Static trait type and implementation do not enter
the hash. Valid views through different traits and their concrete reference
agree. This adds no implicit trait implementation to Fn/Random.
Canonical stack absence is Null for one-slot references/String and [Null,Null]
for Trait. Two eligible absences compare true, absence/present false; absence
hashes to Int64(0). Zero is not exclusive to absence. HeapRef(0,0) is not stack
absence: existing validators classify it TYPE for Struct/Array/Fn/Trait and
INVALID_HANDLE for Random. Malformed/stale values never become false/null.
The producer controls comparisons across static types and the equality used by
collections. Within the selected domain/context, equals(a,b) implies equal
hashes; equal hashes do not imply equality. No universal cross-numeric, Dynamic
box or object-content equality is introduced. A producer joining integer/Float
equality must define coherent canonicalization. Int32 can widen exactly to Int64,
Bool map to 0/1; SDK component mixing must preserve order and boundaries.
Fold to 32 bits is explicit, not abs or lossy numeric conversion. Equality/hash
must remain stable while used as a key; the VM cannot prove custom callbacks.
## Hash-v1
Use XXH64, seed u64 zero, per the
[upstream v0.8.3 specification](https://github.com/Cyan4973/xxHash/blob/v0.8.3/doc/xxhash_spec.md).
Return the digest's identical 64-bit pattern in Int64, including negative values.
All algorithm arithmetic wraps modulo 2^64; host word size/endianness is irrelevant.
| Input | Exact message |
| --- | --- |
| Int64 | Eight LE two's-complement bytes. |
| Finite Float zero | Eight zero bytes for either sign. |
| Other finite Float | Eight LE IEEE 754 binary64 bit-pattern bytes. |
| Present String | Exact resident UTF-8 bytes and their message length, without terminator/prefix/normalization. |
| Present reference | index:u32 LE followed by generation:u32 LE after validation. |
| Accepted absence | Return zero directly; do not hash an empty message. |
Float MUST NOT be numerically converted to integer. NaN/infinity trap NON_FINITE.
Empty String hashes the empty message and remains unequal to null. String owners,
origin/retention and allocation history do not affect content hashing. Reference
hash excludes fields, Random state, FunctionId, captures, view type and native
addresses. There is no cryptographic, uniqueness, capability or confidentiality
guarantee. The hash cannot be converted into a guest reference and creates no root.
Same scalar data/content and contract produce identical bits across hosts and
sessions. An object's hash remains stable while alive, independently of mutation;
reproduction depends on the same VM revision and allocation/collection sequence.
Objects have no shared/persistent identity across sessions/revisions. Time, native
addresses and host entropy are excluded. Existing entry boundaries preserve heap/
session provenance; the handle encoding gains no global session identifier.
## Shared bounded invocation
The three scalar hashes declare invocation_model=borrowed_inline_v1,
scratch_model=none, context_access=none, deterministic=true, may_allocate=false,
mutates_guest_state=false. Protocol/cost selection follows descriptors, not the
owner name. Native Rust handler functions are allowed; guest calls/reentry are not.
1. Resolve metadata and charge using safe carrier inspection.
2. Check temporaries above the frame floor and validate borrowed inputs in order.
3. Admit resources declared by that entry's profile.
4. Compute into bounded local output storage derived from ret_layout.
5. Validate the complete result's width and carriers.
6. Replace/consume inputs and publish output in an infallible commit.
Inputs remain original stack roots/owners until commit. No argument BudgetVec,
Vec-returning callback or mutable HostContext implements this profile. Unsupported
inline result capacity or inconsistent descriptor/handler is rejected without
dynamic-buffer fallback. Each new hash replaces one slot with one and requires
no stack growth, guest allocation or dynamic scratch. No GC, safepoint, guest
callback or scheduling occurs inside the operation. Faults retain inputs and
discard partial results. Stateful handlers retain their own preparation/commit;
Random keeps its existing candidate-state, allocation and bounded-attempt rules.
## Costs, faults and admission
Charge each decoded attempt once, including refusal, without an extra generic
INTRINSIC cycle. Numeric hashes use fixed cost eight. String costs 8+n for an
available String carrier, before traversal or provenance/ownership refusal.
Null/wrong/unavailable carrier and underflow cost eight. Checked cost overflow
charges base and traps RESOURCE_LIMIT without wrap or traversal. Reference costs
use resolved widths even on underflow/refusal. Charge before height/group checks;
validate left then right, and Float finitude after its exact carrier. No refunds.
| Cause | Outcome |
| --- | --- |
| Invalid immediate TypeId/category | LOAD_INVALID_REFERENCE. |
| Invalid encoding/unknown intrinsic/proven static conflict | Existing structured load rejection, including dead code, no guest cycles. |
| Incomplete temporary inputs | STACK_UNDERFLOW 0x08. |
| Wrong carrier/type/malformed group/view | TYPE 0x04. |
| Dead/stale/invalid reference | INVALID_HANDLE 0x0F under existing typed validation. |
| Non-finite Float reaching execution | NON_FINITE 0x14. |
| Checked cost/resource overflow | RESOURCE_LIMIT 0x10 with applicable resource detail. |
| Handler result violates descriptor | INVALID_INTRINSIC 0x0D; original operands retained. |
Accepted null does not trap NULL_REFERENCE; Null to a numeric hash is TYPE.
TrapInfo retains attempted PC/opcode, available source span and resource context.
Budget zero executes nothing; positive fuel admits a whole instruction and can
overshoot. There is no per-byte preemption, quota-rescue GC, hash cache/table,
new String limit or relaxed ownership/resource quota.
## Intrinsic catalog schema 2
Schema version is independent of canonical intrinsic version. INTRINSICS.csv is
generated from descriptors, ordered by ID, UTF-8/LF. The exact header is:
```csv
final_id_hex,final_id_dec,canonical_name,canonical_version,owner,name,arg_slots,ret_slots,arg_layout,ret_layout,deterministic,may_allocate,parameter_names,cost_cycles,mutates_guest_state,cost_model,attempt_cost_cycles,max_attempts,schema_version,arg_nullable,invocation_model,scratch_model,context_access,cost_arg_index,cost_per_byte
```
schema_version is 2 on every row. arg_nullable is one lowercase boolean per
argument slot, pipe-separated, empty only for no arguments; widths must match
arg_layout. cost_arg_index is a zero-based String slot for the byte model and
empty for other models. cost_per_byte is one for String hashing and zero for
fixed/random_rejection_v1. Base/fixed cost remains cost_cycles. Unsupported or
inconsistent schema, profile, model, layout, index or capacity fails explicitly.
```csv
0x4100,16640,hash.i64,1,hash,i64,1,1,Int64,Int64,true,false,value,8,false,fixed,0,0,2,false,borrowed_inline_v1,none,none,,0
0x4101,16641,hash.float,1,hash,float,1,1,Float,Int64,true,false,value,8,false,fixed,0,0,2,false,borrowed_inline_v1,none,none,,0
0x4102,16642,hash.string,1,hash,string,1,1,String,Int64,true,false,value,8,false,argument_utf8_bytes_v1,0,0,2,true,borrowed_inline_v1,none,none,0,1
```
Static admission and native validation use per-entry arg_nullable. Known null
is valid for hash.string; other String intrinsics retain their old static masks.
Nullable structural admission does not remove domain preconditions such as a
Random mutation requiring a present generator.
Old Host entries use host_buffered_v1/existing_contract_v1/host; old VM-owned
entries vm_owned_v1/existing_contract_v1/vm_state. These retain existing effects,
accounting, limits, IDs, signatures, versions and costs, not the new profile's
guarantees. Old scalar/color args have false nullability, Random args true;
no-arg lists are empty. Legacy cost_arg_index is empty and cost_per_byte zero.
No broad migration is required. The builtin constant catalog remains unchanged.
## Producer boundary
Publish exact provider revision, schema and catalog digests. Consumers explicitly
port metadata, admission, lowering/emission, mirrors, packaged provenance and
normal/optimized conformance. CSV-only updates are not completed integration.
Reject unsupported data without inferring purity/nullability/zero scratch from
deterministic or names. Preserve faults, ordering and charged work even when
results are discarded. Collections, defaults and custom/Dynamic strategies remain
consumer-owned. Set/Map, storage publication, Lua FE, arbitrary byte strings,
a new Float domain and general structural hashing are outside this contract.
@@ -232,3 +232,12 @@ roots throughout preparation; the commit phase has no allocation, GC, scheduler
callback boundary. Refusal changes neither destinations nor input groups, and
releases temporary reservations. Consumer return carriers must be prepared before
publication. See [the instruction contract](02c-program-registry-and-pbx-v0.md#logical-guards-and-atomic-storage-publication).
## Borrowed intrinsic invocation
The [bounded intrinsic profile](02f-vm-identity-and-value-hashing.md) keeps
original arguments/owners rooted until result validation and stack commit. Hash
intrinsics use no guest allocation, dynamic scratch or stack growth; native
output storage is bounded by the declared result layout. They do not collect,
reenter or call guest code. Hash values create no roots. Legacy intrinsic
resource/effect contracts remain explicit and do not inherit these guarantees.
+4
View File
@@ -132,3 +132,7 @@ In those cases:
## Random core integration
The [VM-owned random core](02e-vm-owned-random-core.md) defines opaque Random references, PCG64 and the random intrinsic surface.
## Identity and hashing integration
The [identity and value hashing contract](02f-vm-identity-and-value-hashing.md) defines typed reference queries, scalar/content hash-v1, bounded intrinsic invocation and catalog schema 2. See the [producer handoff](../../vm-arch/identity-hashing-producer-handoff.md) and [qualification record](../../vm-arch/IDENTITY_HASHING_QUALIFICATION.md) for implementation evidence and consumer integration targets.