Intrepid/Prometeu/Runtime/pipeline/pr-master There was a failure building this commit
261 lines
19 KiB
Markdown
261 lines
19 KiB
Markdown
# VM Resource Safety and Stability Qualification
|
||
|
||
The local functional and stationary workload checks described below passed on
|
||
macOS/arm64. This delivery was **accepted as complete on 2026-10-14** using those
|
||
checks and the observed extended-run evidence. The planned eight-hour active
|
||
release soak was interrupted by host disk exhaustion after approximately 3h22
|
||
elapsed; the user explicitly waived the eight-hour completion requirement for
|
||
this delivery. Eight-hour endurance was not demonstrated. Linux and other
|
||
architectures have not been tested by this qualification.
|
||
|
||
The normative contracts remain [Registry/PBX v0](../specs/runtime/02c-program-registry-and-pbx-v0.md),
|
||
[memory/GC](../specs/runtime/03-memory-stack-heap-and-allocation.md),
|
||
[typed callables](../specs/runtime/02b-vm-function-values-and-closures.md), and
|
||
[single-context lifecycle](../specs/runtime/09a-coroutines-and-cooperative-scheduling.md).
|
||
This report does not change quotas, safepoints, instruction costs or scheduler
|
||
policy. Audio and representative-game headroom are outside this qualification.
|
||
|
||
## Environment and Reproduction
|
||
|
||
Recorded 2026-10-13: Apple M2 Pro, macOS 27.0.1 (26A434), arm64, Rust
|
||
1.98.1 (48a229cea 2026-09-01). Base repository revision:
|
||
`6c2a07e579fce6eda15bff89dc0e9c9cf02e7dbf`, with the resource diagnostics and
|
||
qualification changes in the working tree. No result below implies x86-64 or
|
||
Linux execution.
|
||
|
||
```sh
|
||
cargo test --workspace
|
||
cargo test --workspace --features prometeu-host-headless/resource-diagnostics
|
||
cargo test --release --workspace --features prometeu-host-headless/resource-diagnostics
|
||
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --frames 216000
|
||
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --seconds 28800
|
||
cargo test --release -p prometeu-vm --features resource-diagnostics --lib sealed_image_load_probe -- --ignored --nocapture
|
||
cargo test --release -p prometeu-vm --features resource-diagnostics --lib storage_and_gc_probe -- --ignored --nocapture
|
||
cargo test --release -p prometeu-bytecode reference_map_probe -- --ignored --nocapture
|
||
cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored
|
||
cargo test -p prometeu-host-desktop-winit --lib audio::control_tests::desktop_audio_device_smoke -- --ignored --exact --nocapture
|
||
```
|
||
|
||
The reproduction commands name manual probes individually. The original recorded
|
||
desktop qualification grouped ignored tests; current automation selects only the
|
||
five debugger socket cases and leaves device audio explicit.
|
||
|
||
The `resource-diagnostics` feature adds engineering observations through Rust
|
||
only. It adds no guest syscall. The eight-hour example is a manual command;
|
||
the repository CI, release workflow and Jenkins pipeline do not invoke it. Normal builds omit timing histograms, additional
|
||
budget counters and depth sampling. Identical-input replay compares logical
|
||
outputs/cycles, never wall-clock durations.
|
||
|
||
## Resource and Enforcement Matrix
|
||
|
||
All sizes below are binary MiB. `LOAD_RESOURCE_LIMIT` is a load-time refusal;
|
||
`TRAP_RESOURCE_LIMIT` is runtime code `0x10`. `TRAP_INVALID_HANDLE` is `0x0F`.
|
||
Each failure must preserve its domain, requested amount and limit. Instruction
|
||
and function context are checked where applicable. VM quotas are not RSS caps.
|
||
|
||
| Resource / invariant | Accepted limit / rule | Enforcement and evidence |
|
||
| --- | --- | --- |
|
||
| Input PBX | 64 MiB | HAL `cartridge_loader::read_program` checks metadata before read and detects extra data without unchecked growth. `program_admission_accepts_exact_limit_and_rejects_growth_after_metadata`, metadata-before-read, short/interrupted-read and real sparse-file rejection tests. |
|
||
| Code | 32 MiB | Bytecode envelope/owned decoder uses `LoadBudget` before materialization; `registry_v0` covers oversized declarations, truncation, code ranges and transactional payload refusal. |
|
||
| Type/function records | 65,536 each | Registry decoder/resolver bounded count admission; `oversized_counts_fail_before_dependent_allocation` and invalid reference tests. |
|
||
| Implementation/nested records | 262,144 / 1,048,576 | Registry resolution/codec count checks; multi-slot/trait records, aggregate checks and owned-payload rollback coverage in `registry_v0`. |
|
||
| Names | 4,096 bytes each; 4 MiB aggregate | `name_limits_include_aggregate_and_empty_is_not_absent`; invalid UTF-8/flags/tails rejected before publication. |
|
||
| Retained Registry | 64 MiB/image | Charged native layouts/capacities during resolution; `work_retained_and_temporary_limits_refuse_before_publication`; explicit reference-map probe records native representation. |
|
||
| Load scratch | 128 MiB | Parser/writer/verifier temporary buffers charged and refunded; `scratch_is_refunded_per_function_and_preserves_prior_load_ownership`, `stack_analysis_refunds_all_state_buffers_and_preserves_prior_ownership`, sealed-image load probe. |
|
||
| Load/verifier work | 268,435,456 units | `LoadBudget::work`, CFG/type reprocessing and native lookup charging; `late_work_refusal_releases_all_new_owners`, `native_lookup_work_refusal_retains_the_actual_instruction_pc`, work-exhaustion tests. |
|
||
| Declared frame/results | 16,384 slots / 1,024 slots | Registry layout/verification before narrowed execution widths; `literal_zero_max_stack_is_preserved_and_frame_limits_are_not_truncated`, call/result-group verification tests. |
|
||
| Constants | 1,048,576 entries; 64 MiB string content/image; 4 MiB individual string | Owned decoder/writer checks before materialization; `complete_module_writer_checks_constant_quotas_before_payload_allocation`, `constant_string_limits_are_checked_before_materialization`, shared image ownership and final-owner tests. |
|
||
| Guest objects | 256 MiB/session | Heap `StoredAllocation`, typed payloads and captures reserve before publication. Header/payload boundary, partial construction and provisional allocation tests; stationary graph/closure workloads. |
|
||
| Handle slots | 1,048,576 including free/retired | Heap admission and persistent slot charge; `handle_limit_counts_free_and_retired_slots_and_reuse_does_not_charge_twice`. |
|
||
| Generations | No wrap; stale handles trap | `maximum_generation_is_retired_without_wraparound`, stale typed reference/collection/callable tests. Controlled generation injection tests retirement without billions of allocations. |
|
||
| Persistent auxiliary storage | 64 MiB/session | `BudgetVec` capacities for stacks, frames, globals, tables, free lists and string ledger. Atomic table growth, entry/call quota refusal, capacity transfer and rollback tests. |
|
||
| Operation temporaries | 64 MiB/session | Provisional coexistence, native results and GC worklist reservation. `temporary_growth_refusal_keeps_old_buffer_and_refunds_provisional_charge`, `mark_worklist_refusal_changes_no_marks_or_objects`, collection/callable rollback tests. |
|
||
| Runtime string content/count | 64 MiB / 262,144 allocations | Weak `StringLedger` admission; UTF-8 byte/count boundaries, empty strings, pressure reconciliation and failed ledger growth tests. |
|
||
| Individual runtime string | 4 MiB | `content_and_individual_limits_are_checked_before_materialization`, bounded native writer tests; refusal precedes content creation/callback. |
|
||
| Call depth | 1,024 including entry | `call_depth_includes_entry_and_traps_before_frame_growth` covers 1/3/1,024; zero-entry setup and closure call refusal preserve owners and attempt costs. |
|
||
| GC roots/safepoints | FRAME_SYNC and successful last-root RET, threshold only | Active frame environments, transitively rooted Trait/Dynamic/native groups, cycles and rooted results. Actual layer test allocates 1,031 closures and verifies only the active closure survives sync collection. |
|
||
| Native provenance/retention | Session ownership retained explicitly | Native string import/snapshot and cross-session isolation tests; an external string snapshot keeps its accounting domain alive after VM disposal and its final drop refunds it. Raw handle bits/snapshots do not establish guest roots. |
|
||
| Lifecycle | Same stacks/quotas; existing hook budgets | Nested pause/resume hook, breakpoint/budget, forbidden FRAME_SYNC, entry refusal and result ownership tests; repeated host reset and desktop Home/resume validation. |
|
||
| No-sync loop | Existing slice/fuel semantics | 1,000 finite slices of a sealed `JMP 0` program return BudgetExhausted, preserve PC/depth/capacities, perform no GC, and remain breakpoint-controllable. No frame completion is fabricated. |
|
||
|
||
Relevant test sources:
|
||
[Registry](../../crates/console/prometeu-bytecode/tests/registry_v0.rs),
|
||
[HAL cartridge admission](../../crates/console/prometeu-hal/src/cartridge_loader.rs),
|
||
[heap](../../crates/console/prometeu-vm/src/heap.rs),
|
||
[string ledger](../../crates/console/prometeu-vm/src/string_ledger/tests.rs),
|
||
[typed execution](../../crates/console/prometeu-vm/src/virtual_machine/typed_tests.rs),
|
||
[callables](../../crates/console/prometeu-vm/src/virtual_machine/callable_tests.rs),
|
||
[collections](../../crates/console/prometeu-vm/src/virtual_machine/collections_tests.rs),
|
||
[integrated resource probes](../../crates/console/prometeu-vm/tests/resource_stability.rs).
|
||
|
||
The matrix identifies enforcement and discriminating evidence; it does not claim
|
||
exhaustive coverage of all bytecode states or recovery from physical allocator
|
||
abort. A blocked native call cannot be preempted by VM fuel. LoadBudget records
|
||
one admission attempt; its retained counters are not RAII session reservations.
|
||
Image/Registry/constant owners are checked separately through owner lifetimes.
|
||
|
||
## Bounded Observations
|
||
|
||
The budget snapshot uses the discriminant order
|
||
`GuestBytes, HandleSlots, AuxiliaryBytes, TemporaryBytes, StringBytes,
|
||
StringAllocations, IndividualStringBytes, CallDepth`. Current usage is reserved
|
||
storage; individual size/depth are admission checks and therefore have a peak
|
||
but no current reservation. Refusal counters include speculative geometric
|
||
capacity attempts that may succeed through an exact-capacity fallback. They
|
||
must not be interpreted as guest trap counts.
|
||
|
||
GC mark, sweep, total and string reconciliation use fixed 64-bucket logarithmic
|
||
nanosecond histograms. Percentiles are **bucket upper bounds**, capped by observed
|
||
maximum, not exact percentiles. Count and maximum are exact observations.
|
||
Snapshots and observers retain no guest root. A budget observer keeps accounting
|
||
available after VM destruction without retaining its guest allocations.
|
||
|
||
## Stationary Workloads
|
||
|
||
The [versioned fixtures](../../crates/console/prometeu-vm/tests/support/qualification.rs)
|
||
are serialized through the official bounded PBX writer and loaded through the
|
||
sealed production image boundary. Fixed input seed/value is 73. Each family has
|
||
an independent session and a second identical session used for lockstep replay:
|
||
|
||
- **Collections:** self-referencing typed nodes and containing arrays, alias
|
||
reads and numeric output verification; 128 objects per completed entry.
|
||
- **Closures:** 127 discarded closures and one executing captured closure,
|
||
FRAME_SYNC inside that closure, then result 73 returned through its caller;
|
||
128 objects per two completed frames.
|
||
- **Strings/Dynamic:** UTF-8 concatenation, typed string arrays, Dynamic
|
||
pack/unpack and equality against an independent expected constant.
|
||
|
||
The default GC allocation threshold remains 1,024. Each frame has a finite
|
||
1,000-slice harness guard. No history of individual instructions/frames is kept.
|
||
Equivalent checkpoints occur every 10,000 frames, starting at the end of the
|
||
10,000-frame warm-up. Logical output and cycle digest are compared every frame;
|
||
resource snapshots are compared at checkpoints. Timed mode runs the replay
|
||
alongside the primary sessions, not as a second eight-hour job.
|
||
|
||
Observed stationary result (216,000 frames **per family and per replay**, release):
|
||
|
||
| Family | Logical cycles | Digest | Guest bytes at checkpoint | Auxiliary bytes | Handle slots | GC collections |
|
||
| --- | ---: | --- | ---: | ---: | ---: | ---: |
|
||
| Collections | 983,664,000 | `f81812a99fda45e5` | 0 | 61,672 | 1,024 | 27,000 |
|
||
| Closures | 167,724,000 | `ca8215da0bc457a5` | 332 | 123,056 | 1,025 | 13,500 |
|
||
| Strings/Dynamic | 734,832,000 | `9711182bb9169a65` | 0 | 127,320 | 1,024 | 13,500 |
|
||
|
||
All replays matched and every disposed session reported zero reserved usage.
|
||
The run took approximately 70.652 seconds of active wall time; this is not a
|
||
real-time frame-rate promise. Temporary/string reservations were zero at the
|
||
stationary checkpoints. Peak guest storage was 344,064 / 340,300 / 319,488 bytes,
|
||
respectively. Peak tracked runtime string content was 11,264 bytes in 1,024
|
||
allocations. These are synthetic small workloads, not near-limit game headroom.
|
||
|
||
One closure remains allocated after its root return: it was reachable at the
|
||
preceding FRAME_SYNC collection, then became unreachable, with insufficient new
|
||
allocation to trigger another collection at RET. It is reclaimed by the next
|
||
threshold sweep. The stable 332 bytes are expected safepoint policy, not a leak;
|
||
final session disposal releases them. The free-list capacity and 1,025 published
|
||
slots are also legitimately retained. Retirement has a separate injected-state
|
||
oracle because retired slots must accumulate by contract.
|
||
|
||
## Host and Functional Evidence
|
||
|
||
- Baseline workspace: 984 passed, zero failed, 10 ignored.
|
||
- Instrumented workspace debug and release before the final diagnostic-depth and
|
||
image-owner regression additions: 990 passed each, zero failed, 10 ignored.
|
||
- Final default workspace: 986 passed, zero failed, 10 ignored. Final instrumented
|
||
workspace debug: 993 passed, zero failed, 10 ignored. Final instrumented VM
|
||
release unit/resource subset: 291 passed, zero failed, 2 ignored. The final
|
||
desktop lifecycle regression also passed separately in release.
|
||
- Formatting, whitespace and documentation reference checks passed.
|
||
- Headless: 1,000 production boot/GC/frame/pause/resume/reset cycles passed;
|
||
session count and all session reservations returned to zero after every reset.
|
||
- Desktop runner: 32 cold-reset/Home/resume cycles with the actual desktop
|
||
hardware and render worker passed in debug and release.
|
||
- Real desktop window: 32 pause/stepFrame/resume cycles via port 17959 passed,
|
||
progressing from logical frame 2,668 to 2,937; no fault event was observed.
|
||
The initial harness incorrectly assumed completion within a fixed 60 ms.
|
||
Replacing that assumption with bounded polling of logical progress passed;
|
||
no runtime timing contract was changed.
|
||
- All five explicit localhost debugger transport tests passed.
|
||
- Three explicit release engineering probes passed: sealed image load,
|
||
storage/GC and Registry reference-map measurements. The sealed-image example
|
||
used 77,906 peak scratch bytes and retained zero scratch after publication.
|
||
- Remaining ignored default-suite cases are the independent file-host memcard
|
||
latency measurement and packaged-installation smoke; neither is claimed as
|
||
executed here or required for VM resource qualification.
|
||
|
||
Desktop smoke reproduction (two terminals, from repository root):
|
||
|
||
```sh
|
||
cargo run -p prometeu-host-desktop-winit -- --debug test-cartridges/dummy-boy --port 17959
|
||
python3 crates/host/prometeu-host-desktop-winit/tests/resource_control_smoke.py
|
||
```
|
||
|
||
## Interrupted Extended Run and Accepted Scope
|
||
|
||
The interrupted run used a frozen release executable with SHA-256
|
||
`d9265cd90c085965220a581623023a84bb6d4d2efe9cb91ad12ebf501e15e388`.
|
||
It counts batches lasting at most two seconds as active wall time; longer gaps
|
||
are conservatively excluded, so suspension/descheduling can extend the run.
|
||
It requires at least 28,800 credited seconds and prints final replay and
|
||
disposal results only after that duration. Do not infer completion from an
|
||
intermediate checkpoint or process survival.
|
||
|
||
Checked on 2026-10-14: the process exited with code 101 after stdout failed with
|
||
`No space left on device (os error 28)`. The last log modification was
|
||
2026-10-13 18:32:24 UTC, approximately 3 hours 22 minutes after launch. This is
|
||
observed elapsed time, not a completed measurement of credited active time.
|
||
There are no `load_complete`, `replay_pass`, or `qualification_pass` records;
|
||
final disposal checks were not reached.
|
||
|
||
The last common checkpoint completed 36,420,000 frames per family and replay;
|
||
collections additionally recorded 36,430,000. All recorded equivalent-phase
|
||
envelopes remained stable and no replay assertion failed before the stdout
|
||
failure. The 369 external RSS samples span 15:28:13–18:32:21 UTC and range from
|
||
3,296 to 3,408 KiB, with a final sample of 3,328 KiB. Sampling started about
|
||
18 minutes after launch, so it does not measure startup peak RSS.
|
||
|
||
The log is 5,934,947 bytes; its size alone does not explain the host volume
|
||
exhaustion. On inspection the volume reported only 1.3 GiB available. This
|
||
failure is in host log output, not a recorded guest resource-limit violation.
|
||
The user accepted the observed evidence on 2026-10-14 and removed the requirement
|
||
to repeat eight hours before closing this delivery. The interrupted attempt is
|
||
not a successful eight-hour run. Final disposal and successful runner completion
|
||
are evidenced by the separate 216,000-frame run and ownership/reset tests above,
|
||
not inferred from the interrupted process. Further duration, workload diversity
|
||
and platform testing remain future work with their own acceptance criteria.
|
||
|
||
The [compact evidence record](evidence/resource-stability-2026-10-13.json)
|
||
preserves the last counters, digests, GC distributions, RSS summary, completed
|
||
stationary-run records and source-log hashes independently of temporary files.
|
||
|
||
| Family | Last recorded frames | GC collections | GC p50 upper bound (ms) | GC p95 / p99 upper bounds (ms) | GC maximum (ms) |
|
||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||
| Collections | 36,430,000 | 4,553,750 | 0.065535 | 0.065535 / 0.131071 | 128.828583 |
|
||
| Closures | 36,420,000 | 2,276,250 | 0.065535 | 0.131071 / 0.131071 | 46.940041 |
|
||
| Strings/Dynamic | 36,420,000 | 2,276,250 | 0.065535 | 0.131071 / 0.131071 | 51.491375 |
|
||
|
||
These are instrumented host wall-time observations and may include scheduling
|
||
delays; the maxima are not a universal latency guarantee. They do not qualify
|
||
representative-game headroom or a near-limit workload.
|
||
|
||
Session artifacts currently reside in `/tmp/resource-qualification-8h.log`,
|
||
`/tmp/resource-qualification-rss.csv`, and
|
||
`/tmp/resource-qualification-environment.txt`. The external RSS monitor also
|
||
exited successfully after the workload stopped. RSS is not a guest allocation
|
||
counter. These artifacts preserve the interrupted attempt for diagnosis.
|
||
|
||
The frozen executable predates a diagnostic-only correction that counts an
|
||
entry returning on its first instruction in peak call depth. These workloads
|
||
already execute multiple instructions before return and their depth evidence
|
||
is unaffected; the immediate-return case is checked by a separate regression.
|
||
|
||
## Future Platform Qualification
|
||
|
||
Run the same corpus, seed, commands and logical oracles on Linux/x86-64 and other
|
||
supported targets when available. Record native layouts and quota boundaries
|
||
locally; native-capacity counts can differ by ABI. Cross-platform numerical,
|
||
resource-fault and timing comparisons must not be inferred from this macOS run.
|
||
|
||
The [resource stability lesson](../../discussion/lessons/DSC-0056-vm-resource-limits-handle-safety-and-long-run-stability/LSN-0068-resource-stability-needs-phase-aware-evidence.md)
|
||
explains phase-aware memory observations, ownership checks and the distinction
|
||
between an accepted delivery and an interrupted test run.
|