Files
prometeu-runtime/docs/vm-arch/resource-stability-qualification.md
T
bquarkz a2275932a5
Intrepid/Prometeu/Runtime/pipeline/pr-master There was a failure building this commit
Keep PBX at v0 and simplify Jenkins quality gate
2026-10-14 14:35:44 +01:00

19 KiB
Raw Blame History

VM Resource Safety and Stability Qualification

The local functional and stationary workload checks described below passed on macOS/arm64. This delivery was accepted as complete on 2026-10-14 using those checks and the observed extended-run evidence. The planned eight-hour active release soak was interrupted by host disk exhaustion after approximately 3h22 elapsed; the user explicitly waived the eight-hour completion requirement for this delivery. Eight-hour endurance was not demonstrated. Linux and other architectures have not been tested by this qualification.

The normative contracts remain Registry/PBX v0, memory/GC, typed callables, and single-context lifecycle. This report does not change quotas, safepoints, instruction costs or scheduler policy. Audio and representative-game headroom are outside this qualification.

Environment and Reproduction

Recorded 2026-10-13: Apple M2 Pro, macOS 27.0.1 (26A434), arm64, Rust 1.98.1 (48a229cea 2026-09-01). Base repository revision: 6c2a07e579fce6eda15bff89dc0e9c9cf02e7dbf, with the resource diagnostics and qualification changes in the working tree. No result below implies x86-64 or Linux execution.

cargo test --workspace
cargo test --workspace --features prometeu-host-headless/resource-diagnostics
cargo test --release --workspace --features prometeu-host-headless/resource-diagnostics
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --frames 216000
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --seconds 28800
cargo test --release -p prometeu-vm --features resource-diagnostics --lib sealed_image_load_probe -- --ignored --nocapture
cargo test --release -p prometeu-vm --features resource-diagnostics --lib storage_and_gc_probe -- --ignored --nocapture
cargo test --release -p prometeu-bytecode reference_map_probe -- --ignored --nocapture
cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored
cargo test -p prometeu-host-desktop-winit --lib audio::control_tests::desktop_audio_device_smoke -- --ignored --exact --nocapture

The reproduction commands name manual probes individually. The original recorded desktop qualification grouped ignored tests; current automation selects only the five debugger socket cases and leaves device audio explicit.

The resource-diagnostics feature adds engineering observations through Rust only. It adds no guest syscall. The eight-hour example is a manual command; the repository CI, release workflow and Jenkins pipeline do not invoke it. Normal builds omit timing histograms, additional budget counters and depth sampling. Identical-input replay compares logical outputs/cycles, never wall-clock durations.

Resource and Enforcement Matrix

All sizes below are binary MiB. LOAD_RESOURCE_LIMIT is a load-time refusal; TRAP_RESOURCE_LIMIT is runtime code 0x10. TRAP_INVALID_HANDLE is 0x0F. Each failure must preserve its domain, requested amount and limit. Instruction and function context are checked where applicable. VM quotas are not RSS caps.

Resource / invariant Accepted limit / rule Enforcement and evidence
Input PBX 64 MiB HAL cartridge_loader::read_program checks metadata before read and detects extra data without unchecked growth. program_admission_accepts_exact_limit_and_rejects_growth_after_metadata, metadata-before-read, short/interrupted-read and real sparse-file rejection tests.
Code 32 MiB Bytecode envelope/owned decoder uses LoadBudget before materialization; registry_v0 covers oversized declarations, truncation, code ranges and transactional payload refusal.
Type/function records 65,536 each Registry decoder/resolver bounded count admission; oversized_counts_fail_before_dependent_allocation and invalid reference tests.
Implementation/nested records 262,144 / 1,048,576 Registry resolution/codec count checks; multi-slot/trait records, aggregate checks and owned-payload rollback coverage in registry_v0.
Names 4,096 bytes each; 4 MiB aggregate name_limits_include_aggregate_and_empty_is_not_absent; invalid UTF-8/flags/tails rejected before publication.
Retained Registry 64 MiB/image Charged native layouts/capacities during resolution; work_retained_and_temporary_limits_refuse_before_publication; explicit reference-map probe records native representation.
Load scratch 128 MiB Parser/writer/verifier temporary buffers charged and refunded; scratch_is_refunded_per_function_and_preserves_prior_load_ownership, stack_analysis_refunds_all_state_buffers_and_preserves_prior_ownership, sealed-image load probe.
Load/verifier work 268,435,456 units LoadBudget::work, CFG/type reprocessing and native lookup charging; late_work_refusal_releases_all_new_owners, native_lookup_work_refusal_retains_the_actual_instruction_pc, work-exhaustion tests.
Declared frame/results 16,384 slots / 1,024 slots Registry layout/verification before narrowed execution widths; literal_zero_max_stack_is_preserved_and_frame_limits_are_not_truncated, call/result-group verification tests.
Constants 1,048,576 entries; 64 MiB string content/image; 4 MiB individual string Owned decoder/writer checks before materialization; complete_module_writer_checks_constant_quotas_before_payload_allocation, constant_string_limits_are_checked_before_materialization, shared image ownership and final-owner tests.
Guest objects 256 MiB/session Heap StoredAllocation, typed payloads and captures reserve before publication. Header/payload boundary, partial construction and provisional allocation tests; stationary graph/closure workloads.
Handle slots 1,048,576 including free/retired Heap admission and persistent slot charge; handle_limit_counts_free_and_retired_slots_and_reuse_does_not_charge_twice.
Generations No wrap; stale handles trap maximum_generation_is_retired_without_wraparound, stale typed reference/collection/callable tests. Controlled generation injection tests retirement without billions of allocations.
Persistent auxiliary storage 64 MiB/session BudgetVec capacities for stacks, frames, globals, tables, free lists and string ledger. Atomic table growth, entry/call quota refusal, capacity transfer and rollback tests.
Operation temporaries 64 MiB/session Provisional coexistence, native results and GC worklist reservation. temporary_growth_refusal_keeps_old_buffer_and_refunds_provisional_charge, mark_worklist_refusal_changes_no_marks_or_objects, collection/callable rollback tests.
Runtime string content/count 64 MiB / 262,144 allocations Weak StringLedger admission; UTF-8 byte/count boundaries, empty strings, pressure reconciliation and failed ledger growth tests.
Individual runtime string 4 MiB content_and_individual_limits_are_checked_before_materialization, bounded native writer tests; refusal precedes content creation/callback.
Call depth 1,024 including entry call_depth_includes_entry_and_traps_before_frame_growth covers 1/3/1,024; zero-entry setup and closure call refusal preserve owners and attempt costs.
GC roots/safepoints FRAME_SYNC and successful last-root RET, threshold only Active frame environments, transitively rooted Trait/Dynamic/native groups, cycles and rooted results. Actual layer test allocates 1,031 closures and verifies only the active closure survives sync collection.
Native provenance/retention Session ownership retained explicitly Native string import/snapshot and cross-session isolation tests; an external string snapshot keeps its accounting domain alive after VM disposal and its final drop refunds it. Raw handle bits/snapshots do not establish guest roots.
Lifecycle Same stacks/quotas; existing hook budgets Nested pause/resume hook, breakpoint/budget, forbidden FRAME_SYNC, entry refusal and result ownership tests; repeated host reset and desktop Home/resume validation.
No-sync loop Existing slice/fuel semantics 1,000 finite slices of a sealed JMP 0 program return BudgetExhausted, preserve PC/depth/capacities, perform no GC, and remain breakpoint-controllable. No frame completion is fabricated.

Relevant test sources: Registry, HAL cartridge admission, heap, string ledger, typed execution, callables, collections, integrated resource probes.

The matrix identifies enforcement and discriminating evidence; it does not claim exhaustive coverage of all bytecode states or recovery from physical allocator abort. A blocked native call cannot be preempted by VM fuel. LoadBudget records one admission attempt; its retained counters are not RAII session reservations. Image/Registry/constant owners are checked separately through owner lifetimes.

Bounded Observations

The budget snapshot uses the discriminant order GuestBytes, HandleSlots, AuxiliaryBytes, TemporaryBytes, StringBytes, StringAllocations, IndividualStringBytes, CallDepth. Current usage is reserved storage; individual size/depth are admission checks and therefore have a peak but no current reservation. Refusal counters include speculative geometric capacity attempts that may succeed through an exact-capacity fallback. They must not be interpreted as guest trap counts.

GC mark, sweep, total and string reconciliation use fixed 64-bucket logarithmic nanosecond histograms. Percentiles are bucket upper bounds, capped by observed maximum, not exact percentiles. Count and maximum are exact observations. Snapshots and observers retain no guest root. A budget observer keeps accounting available after VM destruction without retaining its guest allocations.

Stationary Workloads

The versioned fixtures are serialized through the official bounded PBX writer and loaded through the sealed production image boundary. Fixed input seed/value is 73. Each family has an independent session and a second identical session used for lockstep replay:

  • Collections: self-referencing typed nodes and containing arrays, alias reads and numeric output verification; 128 objects per completed entry.
  • Closures: 127 discarded closures and one executing captured closure, FRAME_SYNC inside that closure, then result 73 returned through its caller; 128 objects per two completed frames.
  • Strings/Dynamic: UTF-8 concatenation, typed string arrays, Dynamic pack/unpack and equality against an independent expected constant.

The default GC allocation threshold remains 1,024. Each frame has a finite 1,000-slice harness guard. No history of individual instructions/frames is kept. Equivalent checkpoints occur every 10,000 frames, starting at the end of the 10,000-frame warm-up. Logical output and cycle digest are compared every frame; resource snapshots are compared at checkpoints. Timed mode runs the replay alongside the primary sessions, not as a second eight-hour job.

Observed stationary result (216,000 frames per family and per replay, release):

Family Logical cycles Digest Guest bytes at checkpoint Auxiliary bytes Handle slots GC collections
Collections 983,664,000 f81812a99fda45e5 0 61,672 1,024 27,000
Closures 167,724,000 ca8215da0bc457a5 332 123,056 1,025 13,500
Strings/Dynamic 734,832,000 9711182bb9169a65 0 127,320 1,024 13,500

All replays matched and every disposed session reported zero reserved usage. The run took approximately 70.652 seconds of active wall time; this is not a real-time frame-rate promise. Temporary/string reservations were zero at the stationary checkpoints. Peak guest storage was 344,064 / 340,300 / 319,488 bytes, respectively. Peak tracked runtime string content was 11,264 bytes in 1,024 allocations. These are synthetic small workloads, not near-limit game headroom.

One closure remains allocated after its root return: it was reachable at the preceding FRAME_SYNC collection, then became unreachable, with insufficient new allocation to trigger another collection at RET. It is reclaimed by the next threshold sweep. The stable 332 bytes are expected safepoint policy, not a leak; final session disposal releases them. The free-list capacity and 1,025 published slots are also legitimately retained. Retirement has a separate injected-state oracle because retired slots must accumulate by contract.

Host and Functional Evidence

  • Baseline workspace: 984 passed, zero failed, 10 ignored.
  • Instrumented workspace debug and release before the final diagnostic-depth and image-owner regression additions: 990 passed each, zero failed, 10 ignored.
  • Final default workspace: 986 passed, zero failed, 10 ignored. Final instrumented workspace debug: 993 passed, zero failed, 10 ignored. Final instrumented VM release unit/resource subset: 291 passed, zero failed, 2 ignored. The final desktop lifecycle regression also passed separately in release.
  • Formatting, whitespace and documentation reference checks passed.
  • Headless: 1,000 production boot/GC/frame/pause/resume/reset cycles passed; session count and all session reservations returned to zero after every reset.
  • Desktop runner: 32 cold-reset/Home/resume cycles with the actual desktop hardware and render worker passed in debug and release.
  • Real desktop window: 32 pause/stepFrame/resume cycles via port 17959 passed, progressing from logical frame 2,668 to 2,937; no fault event was observed. The initial harness incorrectly assumed completion within a fixed 60 ms. Replacing that assumption with bounded polling of logical progress passed; no runtime timing contract was changed.
  • All five explicit localhost debugger transport tests passed.
  • Three explicit release engineering probes passed: sealed image load, storage/GC and Registry reference-map measurements. The sealed-image example used 77,906 peak scratch bytes and retained zero scratch after publication.
  • Remaining ignored default-suite cases are the independent file-host memcard latency measurement and packaged-installation smoke; neither is claimed as executed here or required for VM resource qualification.

Desktop smoke reproduction (two terminals, from repository root):

cargo run -p prometeu-host-desktop-winit -- --debug test-cartridges/dummy-boy --port 17959
python3 crates/host/prometeu-host-desktop-winit/tests/resource_control_smoke.py

Interrupted Extended Run and Accepted Scope

The interrupted run used a frozen release executable with SHA-256 d9265cd90c085965220a581623023a84bb6d4d2efe9cb91ad12ebf501e15e388. It counts batches lasting at most two seconds as active wall time; longer gaps are conservatively excluded, so suspension/descheduling can extend the run. It requires at least 28,800 credited seconds and prints final replay and disposal results only after that duration. Do not infer completion from an intermediate checkpoint or process survival.

Checked on 2026-10-14: the process exited with code 101 after stdout failed with No space left on device (os error 28). The last log modification was 2026-10-13 18:32:24 UTC, approximately 3 hours 22 minutes after launch. This is observed elapsed time, not a completed measurement of credited active time. There are no load_complete, replay_pass, or qualification_pass records; final disposal checks were not reached.

The last common checkpoint completed 36,420,000 frames per family and replay; collections additionally recorded 36,430,000. All recorded equivalent-phase envelopes remained stable and no replay assertion failed before the stdout failure. The 369 external RSS samples span 15:28:13–18:32:21 UTC and range from 3,296 to 3,408 KiB, with a final sample of 3,328 KiB. Sampling started about 18 minutes after launch, so it does not measure startup peak RSS.

The log is 5,934,947 bytes; its size alone does not explain the host volume exhaustion. On inspection the volume reported only 1.3 GiB available. This failure is in host log output, not a recorded guest resource-limit violation. The user accepted the observed evidence on 2026-10-14 and removed the requirement to repeat eight hours before closing this delivery. The interrupted attempt is not a successful eight-hour run. Final disposal and successful runner completion are evidenced by the separate 216,000-frame run and ownership/reset tests above, not inferred from the interrupted process. Further duration, workload diversity and platform testing remain future work with their own acceptance criteria.

The compact evidence record preserves the last counters, digests, GC distributions, RSS summary, completed stationary-run records and source-log hashes independently of temporary files.

Family Last recorded frames GC collections GC p50 upper bound (ms) GC p95 / p99 upper bounds (ms) GC maximum (ms)
Collections 36,430,000 4,553,750 0.065535 0.065535 / 0.131071 128.828583
Closures 36,420,000 2,276,250 0.065535 0.131071 / 0.131071 46.940041
Strings/Dynamic 36,420,000 2,276,250 0.065535 0.131071 / 0.131071 51.491375

These are instrumented host wall-time observations and may include scheduling delays; the maxima are not a universal latency guarantee. They do not qualify representative-game headroom or a near-limit workload.

Session artifacts currently reside in /tmp/resource-qualification-8h.log, /tmp/resource-qualification-rss.csv, and /tmp/resource-qualification-environment.txt. The external RSS monitor also exited successfully after the workload stopped. RSS is not a guest allocation counter. These artifacts preserve the interrupted attempt for diagnosis.

The frozen executable predates a diagnostic-only correction that counts an entry returning on its first instruction in peak call depth. These workloads already execute multiple instructions before return and their depth evidence is unaffected; the immediate-return case is checked by a separate regression.

Future Platform Qualification

Run the same corpus, seed, commands and logical oracles on Linux/x86-64 and other supported targets when available. Record native layouts and quota boundaries locally; native-capacity counts can differ by ABI. Cross-platform numerical, resource-fault and timing comparisons must not be inferred from this macOS run.

The resource stability lesson explains phase-aware memory observations, ownership checks and the distinction between an accepted delivery and an interrupted test run.