Files
prometeu-runtime/docs/vm-arch/resource-stability-qualification.md
T
bquarkz a2275932a5
Intrepid/Prometeu/Runtime/pipeline/pr-master There was a failure building this commit
Keep PBX at v0 and simplify Jenkins quality gate
2026-10-14 14:35:44 +01:00

261 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# VM Resource Safety and Stability Qualification
The local functional and stationary workload checks described below passed on
macOS/arm64. This delivery was **accepted as complete on 2026-10-14** using those
checks and the observed extended-run evidence. The planned eight-hour active
release soak was interrupted by host disk exhaustion after approximately 3h22
elapsed; the user explicitly waived the eight-hour completion requirement for
this delivery. Eight-hour endurance was not demonstrated. Linux and other
architectures have not been tested by this qualification.
The normative contracts remain [Registry/PBX v0](../specs/runtime/02c-program-registry-and-pbx-v0.md),
[memory/GC](../specs/runtime/03-memory-stack-heap-and-allocation.md),
[typed callables](../specs/runtime/02b-vm-function-values-and-closures.md), and
[single-context lifecycle](../specs/runtime/09a-coroutines-and-cooperative-scheduling.md).
This report does not change quotas, safepoints, instruction costs or scheduler
policy. Audio and representative-game headroom are outside this qualification.
## Environment and Reproduction
Recorded 2026-10-13: Apple M2 Pro, macOS 27.0.1 (26A434), arm64, Rust
1.98.1 (48a229cea 2026-09-01). Base repository revision:
`6c2a07e579fce6eda15bff89dc0e9c9cf02e7dbf`, with the resource diagnostics and
qualification changes in the working tree. No result below implies x86-64 or
Linux execution.
```sh
cargo test --workspace
cargo test --workspace --features prometeu-host-headless/resource-diagnostics
cargo test --release --workspace --features prometeu-host-headless/resource-diagnostics
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --frames 216000
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --seconds 28800
cargo test --release -p prometeu-vm --features resource-diagnostics --lib sealed_image_load_probe -- --ignored --nocapture
cargo test --release -p prometeu-vm --features resource-diagnostics --lib storage_and_gc_probe -- --ignored --nocapture
cargo test --release -p prometeu-bytecode reference_map_probe -- --ignored --nocapture
cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored
cargo test -p prometeu-host-desktop-winit --lib audio::control_tests::desktop_audio_device_smoke -- --ignored --exact --nocapture
```
The reproduction commands name manual probes individually. The original recorded
desktop qualification grouped ignored tests; current automation selects only the
five debugger socket cases and leaves device audio explicit.
The `resource-diagnostics` feature adds engineering observations through Rust
only. It adds no guest syscall. The eight-hour example is a manual command;
the repository CI, release workflow and Jenkins pipeline do not invoke it. Normal builds omit timing histograms, additional
budget counters and depth sampling. Identical-input replay compares logical
outputs/cycles, never wall-clock durations.
## Resource and Enforcement Matrix
All sizes below are binary MiB. `LOAD_RESOURCE_LIMIT` is a load-time refusal;
`TRAP_RESOURCE_LIMIT` is runtime code `0x10`. `TRAP_INVALID_HANDLE` is `0x0F`.
Each failure must preserve its domain, requested amount and limit. Instruction
and function context are checked where applicable. VM quotas are not RSS caps.
| Resource / invariant | Accepted limit / rule | Enforcement and evidence |
| --- | --- | --- |
| Input PBX | 64 MiB | HAL `cartridge_loader::read_program` checks metadata before read and detects extra data without unchecked growth. `program_admission_accepts_exact_limit_and_rejects_growth_after_metadata`, metadata-before-read, short/interrupted-read and real sparse-file rejection tests. |
| Code | 32 MiB | Bytecode envelope/owned decoder uses `LoadBudget` before materialization; `registry_v0` covers oversized declarations, truncation, code ranges and transactional payload refusal. |
| Type/function records | 65,536 each | Registry decoder/resolver bounded count admission; `oversized_counts_fail_before_dependent_allocation` and invalid reference tests. |
| Implementation/nested records | 262,144 / 1,048,576 | Registry resolution/codec count checks; multi-slot/trait records, aggregate checks and owned-payload rollback coverage in `registry_v0`. |
| Names | 4,096 bytes each; 4 MiB aggregate | `name_limits_include_aggregate_and_empty_is_not_absent`; invalid UTF-8/flags/tails rejected before publication. |
| Retained Registry | 64 MiB/image | Charged native layouts/capacities during resolution; `work_retained_and_temporary_limits_refuse_before_publication`; explicit reference-map probe records native representation. |
| Load scratch | 128 MiB | Parser/writer/verifier temporary buffers charged and refunded; `scratch_is_refunded_per_function_and_preserves_prior_load_ownership`, `stack_analysis_refunds_all_state_buffers_and_preserves_prior_ownership`, sealed-image load probe. |
| Load/verifier work | 268,435,456 units | `LoadBudget::work`, CFG/type reprocessing and native lookup charging; `late_work_refusal_releases_all_new_owners`, `native_lookup_work_refusal_retains_the_actual_instruction_pc`, work-exhaustion tests. |
| Declared frame/results | 16,384 slots / 1,024 slots | Registry layout/verification before narrowed execution widths; `literal_zero_max_stack_is_preserved_and_frame_limits_are_not_truncated`, call/result-group verification tests. |
| Constants | 1,048,576 entries; 64 MiB string content/image; 4 MiB individual string | Owned decoder/writer checks before materialization; `complete_module_writer_checks_constant_quotas_before_payload_allocation`, `constant_string_limits_are_checked_before_materialization`, shared image ownership and final-owner tests. |
| Guest objects | 256 MiB/session | Heap `StoredAllocation`, typed payloads and captures reserve before publication. Header/payload boundary, partial construction and provisional allocation tests; stationary graph/closure workloads. |
| Handle slots | 1,048,576 including free/retired | Heap admission and persistent slot charge; `handle_limit_counts_free_and_retired_slots_and_reuse_does_not_charge_twice`. |
| Generations | No wrap; stale handles trap | `maximum_generation_is_retired_without_wraparound`, stale typed reference/collection/callable tests. Controlled generation injection tests retirement without billions of allocations. |
| Persistent auxiliary storage | 64 MiB/session | `BudgetVec` capacities for stacks, frames, globals, tables, free lists and string ledger. Atomic table growth, entry/call quota refusal, capacity transfer and rollback tests. |
| Operation temporaries | 64 MiB/session | Provisional coexistence, native results and GC worklist reservation. `temporary_growth_refusal_keeps_old_buffer_and_refunds_provisional_charge`, `mark_worklist_refusal_changes_no_marks_or_objects`, collection/callable rollback tests. |
| Runtime string content/count | 64 MiB / 262,144 allocations | Weak `StringLedger` admission; UTF-8 byte/count boundaries, empty strings, pressure reconciliation and failed ledger growth tests. |
| Individual runtime string | 4 MiB | `content_and_individual_limits_are_checked_before_materialization`, bounded native writer tests; refusal precedes content creation/callback. |
| Call depth | 1,024 including entry | `call_depth_includes_entry_and_traps_before_frame_growth` covers 1/3/1,024; zero-entry setup and closure call refusal preserve owners and attempt costs. |
| GC roots/safepoints | FRAME_SYNC and successful last-root RET, threshold only | Active frame environments, transitively rooted Trait/Dynamic/native groups, cycles and rooted results. Actual layer test allocates 1,031 closures and verifies only the active closure survives sync collection. |
| Native provenance/retention | Session ownership retained explicitly | Native string import/snapshot and cross-session isolation tests; an external string snapshot keeps its accounting domain alive after VM disposal and its final drop refunds it. Raw handle bits/snapshots do not establish guest roots. |
| Lifecycle | Same stacks/quotas; existing hook budgets | Nested pause/resume hook, breakpoint/budget, forbidden FRAME_SYNC, entry refusal and result ownership tests; repeated host reset and desktop Home/resume validation. |
| No-sync loop | Existing slice/fuel semantics | 1,000 finite slices of a sealed `JMP 0` program return BudgetExhausted, preserve PC/depth/capacities, perform no GC, and remain breakpoint-controllable. No frame completion is fabricated. |
Relevant test sources:
[Registry](../../crates/console/prometeu-bytecode/tests/registry_v0.rs),
[HAL cartridge admission](../../crates/console/prometeu-hal/src/cartridge_loader.rs),
[heap](../../crates/console/prometeu-vm/src/heap.rs),
[string ledger](../../crates/console/prometeu-vm/src/string_ledger/tests.rs),
[typed execution](../../crates/console/prometeu-vm/src/virtual_machine/typed_tests.rs),
[callables](../../crates/console/prometeu-vm/src/virtual_machine/callable_tests.rs),
[collections](../../crates/console/prometeu-vm/src/virtual_machine/collections_tests.rs),
[integrated resource probes](../../crates/console/prometeu-vm/tests/resource_stability.rs).
The matrix identifies enforcement and discriminating evidence; it does not claim
exhaustive coverage of all bytecode states or recovery from physical allocator
abort. A blocked native call cannot be preempted by VM fuel. LoadBudget records
one admission attempt; its retained counters are not RAII session reservations.
Image/Registry/constant owners are checked separately through owner lifetimes.
## Bounded Observations
The budget snapshot uses the discriminant order
`GuestBytes, HandleSlots, AuxiliaryBytes, TemporaryBytes, StringBytes,
StringAllocations, IndividualStringBytes, CallDepth`. Current usage is reserved
storage; individual size/depth are admission checks and therefore have a peak
but no current reservation. Refusal counters include speculative geometric
capacity attempts that may succeed through an exact-capacity fallback. They
must not be interpreted as guest trap counts.
GC mark, sweep, total and string reconciliation use fixed 64-bucket logarithmic
nanosecond histograms. Percentiles are **bucket upper bounds**, capped by observed
maximum, not exact percentiles. Count and maximum are exact observations.
Snapshots and observers retain no guest root. A budget observer keeps accounting
available after VM destruction without retaining its guest allocations.
## Stationary Workloads
The [versioned fixtures](../../crates/console/prometeu-vm/tests/support/qualification.rs)
are serialized through the official bounded PBX writer and loaded through the
sealed production image boundary. Fixed input seed/value is 73. Each family has
an independent session and a second identical session used for lockstep replay:
- **Collections:** self-referencing typed nodes and containing arrays, alias
reads and numeric output verification; 128 objects per completed entry.
- **Closures:** 127 discarded closures and one executing captured closure,
FRAME_SYNC inside that closure, then result 73 returned through its caller;
128 objects per two completed frames.
- **Strings/Dynamic:** UTF-8 concatenation, typed string arrays, Dynamic
pack/unpack and equality against an independent expected constant.
The default GC allocation threshold remains 1,024. Each frame has a finite
1,000-slice harness guard. No history of individual instructions/frames is kept.
Equivalent checkpoints occur every 10,000 frames, starting at the end of the
10,000-frame warm-up. Logical output and cycle digest are compared every frame;
resource snapshots are compared at checkpoints. Timed mode runs the replay
alongside the primary sessions, not as a second eight-hour job.
Observed stationary result (216,000 frames **per family and per replay**, release):
| Family | Logical cycles | Digest | Guest bytes at checkpoint | Auxiliary bytes | Handle slots | GC collections |
| --- | ---: | --- | ---: | ---: | ---: | ---: |
| Collections | 983,664,000 | `f81812a99fda45e5` | 0 | 61,672 | 1,024 | 27,000 |
| Closures | 167,724,000 | `ca8215da0bc457a5` | 332 | 123,056 | 1,025 | 13,500 |
| Strings/Dynamic | 734,832,000 | `9711182bb9169a65` | 0 | 127,320 | 1,024 | 13,500 |
All replays matched and every disposed session reported zero reserved usage.
The run took approximately 70.652 seconds of active wall time; this is not a
real-time frame-rate promise. Temporary/string reservations were zero at the
stationary checkpoints. Peak guest storage was 344,064 / 340,300 / 319,488 bytes,
respectively. Peak tracked runtime string content was 11,264 bytes in 1,024
allocations. These are synthetic small workloads, not near-limit game headroom.
One closure remains allocated after its root return: it was reachable at the
preceding FRAME_SYNC collection, then became unreachable, with insufficient new
allocation to trigger another collection at RET. It is reclaimed by the next
threshold sweep. The stable 332 bytes are expected safepoint policy, not a leak;
final session disposal releases them. The free-list capacity and 1,025 published
slots are also legitimately retained. Retirement has a separate injected-state
oracle because retired slots must accumulate by contract.
## Host and Functional Evidence
- Baseline workspace: 984 passed, zero failed, 10 ignored.
- Instrumented workspace debug and release before the final diagnostic-depth and
image-owner regression additions: 990 passed each, zero failed, 10 ignored.
- Final default workspace: 986 passed, zero failed, 10 ignored. Final instrumented
workspace debug: 993 passed, zero failed, 10 ignored. Final instrumented VM
release unit/resource subset: 291 passed, zero failed, 2 ignored. The final
desktop lifecycle regression also passed separately in release.
- Formatting, whitespace and documentation reference checks passed.
- Headless: 1,000 production boot/GC/frame/pause/resume/reset cycles passed;
session count and all session reservations returned to zero after every reset.
- Desktop runner: 32 cold-reset/Home/resume cycles with the actual desktop
hardware and render worker passed in debug and release.
- Real desktop window: 32 pause/stepFrame/resume cycles via port 17959 passed,
progressing from logical frame 2,668 to 2,937; no fault event was observed.
The initial harness incorrectly assumed completion within a fixed 60 ms.
Replacing that assumption with bounded polling of logical progress passed;
no runtime timing contract was changed.
- All five explicit localhost debugger transport tests passed.
- Three explicit release engineering probes passed: sealed image load,
storage/GC and Registry reference-map measurements. The sealed-image example
used 77,906 peak scratch bytes and retained zero scratch after publication.
- Remaining ignored default-suite cases are the independent file-host memcard
latency measurement and packaged-installation smoke; neither is claimed as
executed here or required for VM resource qualification.
Desktop smoke reproduction (two terminals, from repository root):
```sh
cargo run -p prometeu-host-desktop-winit -- --debug test-cartridges/dummy-boy --port 17959
python3 crates/host/prometeu-host-desktop-winit/tests/resource_control_smoke.py
```
## Interrupted Extended Run and Accepted Scope
The interrupted run used a frozen release executable with SHA-256
`d9265cd90c085965220a581623023a84bb6d4d2efe9cb91ad12ebf501e15e388`.
It counts batches lasting at most two seconds as active wall time; longer gaps
are conservatively excluded, so suspension/descheduling can extend the run.
It requires at least 28,800 credited seconds and prints final replay and
disposal results only after that duration. Do not infer completion from an
intermediate checkpoint or process survival.
Checked on 2026-10-14: the process exited with code 101 after stdout failed with
`No space left on device (os error 28)`. The last log modification was
2026-10-13 18:32:24 UTC, approximately 3 hours 22 minutes after launch. This is
observed elapsed time, not a completed measurement of credited active time.
There are no `load_complete`, `replay_pass`, or `qualification_pass` records;
final disposal checks were not reached.
The last common checkpoint completed 36,420,000 frames per family and replay;
collections additionally recorded 36,430,000. All recorded equivalent-phase
envelopes remained stable and no replay assertion failed before the stdout
failure. The 369 external RSS samples span 15:28:13–18:32:21 UTC and range from
3,296 to 3,408 KiB, with a final sample of 3,328 KiB. Sampling started about
18 minutes after launch, so it does not measure startup peak RSS.
The log is 5,934,947 bytes; its size alone does not explain the host volume
exhaustion. On inspection the volume reported only 1.3 GiB available. This
failure is in host log output, not a recorded guest resource-limit violation.
The user accepted the observed evidence on 2026-10-14 and removed the requirement
to repeat eight hours before closing this delivery. The interrupted attempt is
not a successful eight-hour run. Final disposal and successful runner completion
are evidenced by the separate 216,000-frame run and ownership/reset tests above,
not inferred from the interrupted process. Further duration, workload diversity
and platform testing remain future work with their own acceptance criteria.
The [compact evidence record](evidence/resource-stability-2026-10-13.json)
preserves the last counters, digests, GC distributions, RSS summary, completed
stationary-run records and source-log hashes independently of temporary files.
| Family | Last recorded frames | GC collections | GC p50 upper bound (ms) | GC p95 / p99 upper bounds (ms) | GC maximum (ms) |
| --- | ---: | ---: | ---: | ---: | ---: |
| Collections | 36,430,000 | 4,553,750 | 0.065535 | 0.065535 / 0.131071 | 128.828583 |
| Closures | 36,420,000 | 2,276,250 | 0.065535 | 0.131071 / 0.131071 | 46.940041 |
| Strings/Dynamic | 36,420,000 | 2,276,250 | 0.065535 | 0.131071 / 0.131071 | 51.491375 |
These are instrumented host wall-time observations and may include scheduling
delays; the maxima are not a universal latency guarantee. They do not qualify
representative-game headroom or a near-limit workload.
Session artifacts currently reside in `/tmp/resource-qualification-8h.log`,
`/tmp/resource-qualification-rss.csv`, and
`/tmp/resource-qualification-environment.txt`. The external RSS monitor also
exited successfully after the workload stopped. RSS is not a guest allocation
counter. These artifacts preserve the interrupted attempt for diagnosis.
The frozen executable predates a diagnostic-only correction that counts an
entry returning on its first instruction in peak call depth. These workloads
already execute multiple instructions before return and their depth evidence
is unaffected; the immediate-return case is checked by a separate regression.
## Future Platform Qualification
Run the same corpus, seed, commands and logical oracles on Linux/x86-64 and other
supported targets when available. Record native layouts and quota boundaries
locally; native-capacity counts can differ by ABI. Cross-platform numerical,
resource-fault and timing comparisons must not be inferred from this macOS run.
The [resource stability lesson](../../discussion/lessons/DSC-0056-vm-resource-limits-handle-safety-and-long-run-stability/LSN-0068-resource-stability-needs-phase-aware-evidence.md)
explains phase-aware memory observations, ownership checks and the distinction
between an accepted delivery and an interrupted test run.