Files
prometeu-runtime/docs/vm-arch/COVERAGE_DOMAINS.md
T

18 KiB

Coverage domains

Function: repository coverage ownership and CI operations.

Background lesson: Domain gates must be owned by the repository.

Coverage domains assign crate source code to a responsible area. Unit, integration and cross-layer tests keep their Cargo targets and contribute to the domains containing the implementation they exercise. Test files do not need to move into domain directories.

The authorities are runtime-edge-coverage-domains.json, the ownership auditor and the evidence/gate helper. The configuration retains its original runtime-edge filename and includes the core workspace responsibilities as well.

Ownership

Every Rust file under a repository crate src tree has exactly one owner. Inclusive paths use repository-relative prefixes; optional exclude_paths use the same semantics and win within their domain. A file without a domain or with two domains fails the ownership audit. Sources with no executable LLVM lines still need an owner.

Domain Source responsibility
system/os OS facades, process and task services
system/runtime System-side VM lifecycle, scheduling and render integration
system/fs Runtime filesystem, memcard.rs and the complete memcard/ module tree
system/programs Resident system programs
system/core Remaining system sources, excluding the four specialized owners
asset/bank HAL asset/cartridge prefixes, driver asset.rs and the complete driver asset/ module tree
firmware Firmware state and transitions
host-dependent Desktop host
host/headless Headless host and automation implementation
vm/core VM execution, values, heap, verifier, admission, intrinsics and math
bytecode PBX/Registry encodings, ISA, assembler, decoder and binary admission
hal/core HAL contracts and helpers, excluding existing asset/bank ownership
drivers/core Peripheral/render implementations, excluding the asset driver
tooling CLI and dummy/stress PBX producers
dev/test-support Shared test-program construction and development infrastructure source stubs

Existing specialized paths retain their original ownership. Both module entry files and sibling module directories need explicit prefixes. Memcard and asset-driver descendants, including future nested modules, retain their specialized owner even when absent from the coverage report or reporting zero executable lines. The corresponding residual exclusions cover both forms. Residual domains use broad crate source prefixes with explicit exclusions, so a new module inherits the appropriate owner. A new crate or conflicting prefix must be assigned before the ownership audit can pass.

How test suites contribute

VM unit/integration tests, including numeric_contract and the portable MPFR-generated fixture, exercise vm/core. Bytecode round-trip and Registry admission tests exercise bytecode; end-to-end loading can also exercise vm/core. HAL/drivers suites contribute to their corresponding implementation owners. Headless/CLI integration and development cross-layer suites can contribute to several domains in one run.

Quality tests remain in prometeu-quality-checks/tests. They enforce the map and drive the auditor/helper with independent source/report fixtures, including omitted/overlapping owners, exclusions, absolute/relative filenames, absent/empty evidence, invalid statistics and percentage regression. Expected-owner tests use the auditor's ownership function for entry files, every existing descendant and future nested paths. Synthetic reports also check absent and zero-line descendants. Exclusive residual fallback is not sufficient: it can silently transfer specialized code into the wrong gate. The empty quality/layer library stubs inherit infrastructure ownership; they do not create separate gates with zero executable lines.

Only crate src paths participate in this domain denominator; integration test directories, examples, benches, external dependencies and standalone Python oracle code are outside it. The LLVM instrumentation/report remains the existing workspace all-features default-profile measurement. Unit tests colocated in src retain the tool's existing reporting behavior.

The standalone MPFR precision/bit comparison remains complementary numeric qualification. It is not replaced by line coverage and is not newly required by make ci-domains. See numeric qualification.

Evidence and gate rules

Reports may use absolute or repository-relative source paths. The auditor normalizes these names before assigning ownership; evidence display and percentage checks consume its same domain statistics. This prevents path aliases or separate matching implementations from disagreeing about totals. The verbose file lists use normalized repository-relative names.

Each report source within scope must exist in the repository and appear once. Every configured domain must have a positive aggregate instrumented-line count. Missing reports, unknown sources, duplicate records, invalid counts and empty domains fail validation. A zero baseline cannot make missing or empty evidence pass.

The gate compares covered instrumented lines to total instrumented lines, truncating the percentage downward to two decimals. A domain passes when this percentage is at least baseline_percent. Counts for functions and regions remain supporting evidence. Scenario/contract assertions continue to govern behavior; a percentage does not replace them.

Measured coverage and the acceptance floor are separate values. The historical margin revision gave every domain at least 5.00 percentage points of headroom from that recorded local measurement. Its rule was new_floor = min(original_floor, measured_percent - 5.00), using the same downward truncation. Floors already providing that margin remain unchanged. Five percentage points means an 88.56% measurement permits an 83.56% floor; it is not a relative five-percent reduction. That historical margin is not a guarantee about later measurements or corrected ownership. The current map restores the reviewed system/core floor of 87.71% and preserves every other floor.

Floors remain fixed during CI. A failing run does not automatically lower them. An intentional scope or floor revision requires explicit review with fresh measured evidence. New domains also require an explicit floor supported by a fresh measurement.

Commands and CI

make ci-domains is the authoritative repository quality gate. Both GitHub CI and Jenkins provision dependencies and invoke that same target. make ci is an alias. Invoke one gate per checkout at a time.

The Makefile supplies this ordered sequence to the serial evidence runner:

  1. Clean LLVM coverage instrumentation from the previous run.
  2. Check formatting with cargo fmt --all -- --check.
  3. Run cargo clippy --workspace --all-targets --all-features -- -D warnings.
  4. Test gate failure/report handling with the Python regression suite.
  5. Run cargo test --workspace --all-targets --all-features --no-fail-fast.
  6. Run the five ignored debugger socket tests through make test-debugger-socket.
  7. Generate fresh workspace/all-feature LLVM coverage, HTML, full JSON, Cobertura XML and summary JSON.
  8. Audit source ownership/evidence and check every fixed coverage domain floor.

The socket command is deliberately restricted:

cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored

It selects test_debug_port_opens, test_debug_reconnection, test_debug_refuse_second_connection, test_get_state_returns_response and test_debug_resume_on_disconnect. It does not select the audio-device test. Local sandbox TCP permissions must permit bind/connect; a permission denial is not evidence of a debugger behavior regression.

The runner executes child Make targets with -j1 even if the outer invocation uses -j. CI invokes make ci-domains serially. Do not run a separate Cargo, coverage-clean or report process concurrently in the same checkout: serialized steps inside one gate cannot protect against an independent process.

Prerequisites and reports

Both pipelines use bash scripts/provision-quality-gate.sh. A Debian/Ubuntu agent needs root or sudo for apt provisioning, and an existing Rust/rustup installation. GitHub installs stable Rust first; Jenkins must supply a supported Rust toolchain and rustup in CARGO_HOME/bin. The script installs rustfmt, Clippy, llvm-tools-preview and pinned cargo-llvm-cov 0.8.5 (--locked).

Linux packages include build-essential, pkg-config, libasound2-dev, libudev-dev, libxkbcommon-dev, libwayland-dev, libx11-dev, libxrandr-dev, libxi-dev, Python 3, jq, ripgrep, Git, curl and CA certificates. ALSA development files satisfy CPAL's Linux build dependency; a running audio device is not needed. Non-Linux local hosts must already provide native compiler/SDK tools, Bash, Make, pkg-config, Python 3, jq, rg and Git. The provisioning script verifies these commands.

target/quality-gate/metadata.json records revision, dirty working tree, host, Rust/coverage-tool versions, configuration/source fingerprints, ordered step exit codes and report hashes. Each step has a separate combined-output log. The runner stops on the first failed Make target and returns its nonzero exit status; the log preserves underlying Cargo diagnostics. Three regression cases check failure logs/identity, fresh success evidence and partial-report retention when a later gate fails.

Both pipelines discard old generated report directories before provisioning and publish current artifacts even after gate failure. An early failed gate has metadata/logs but no old coverage report. Failure after generation preserves the partial current reports. Provisioning failure can leave no report at all and still fails CI. Reports are archived from target/quality-gate/** and target/llvm-cov/**; Jenkins also publishes current HTML when present. Artifact publication does not turn a failed gate into success.

List domains or inspect ownership and existing evidence:

make coverage-domain-list
python3 scripts/audit-coverage-domains.py
make coverage-domain-audit
make coverage-domain-evidence DOMAIN=vm/core
scripts/coverage-domain-evidence.sh --check --verbose hal/core

Explicit manual qualification

Ordinary all-target tests and coverage leave ignored device/measurement tests ignored. The shared target only opts in to the five socket tests above. It does not certify the following manual workloads:

# Physical desktop audio, exact named test.
cargo test -p prometeu-host-desktop-winit --lib audio::control_tests::desktop_audio_device_smoke -- --ignored --exact --nocapture
# Physical callback duration under host I/O pressure.
cargo test --release -p prometeu-host-desktop-winit --test audio_callback_allocations physical_callback_duration_16_voices_with_io_pressure -- --ignored --exact --nocapture
# Eight-hour active release soak.
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --seconds 28800
# Packaged installation; set PROMETEU_PROTOCOL_TEST_INSTALL to an extracted distribution first.
cargo test -p prometeu-cli --test automation_process packaged_installation_smoke -- --ignored --exact --nocapture
# File-host commit latency with the production asset decoder.
cargo test --release -p prometeu-system --all-features qualify_file_commits_with_production_asset_decoder -- --ignored --nocapture
# Explicit layout observation.
cargo test --release -p prometeu-bytecode --test registry_v1 reference_map_probe -- --ignored --exact --nocapture

Other ignored timing probes remain opt-in by their individual test names. All-target execution can print ordinary benchmark observations; this does not establish completion of ignored manual qualification. For scope and prior measured results see resource stability and audio callback qualification.

Historical local baseline

The initial measured evidence records configuration, source/report fingerprints, command/toolchain/host, line totals and gate results for all 15 domains before the margin revision. It preserves the eight earlier domain definitions and records these seven additional starting floors on macOS/aarch64:

New domain Baseline lines
vm/core 89.12%
bytecode 93.23%
hal/core 88.56%
drivers/core 84.51%
system/core 92.71%
tooling 64.75%
dev/test-support 85.96%

These are historical starting floors; current acceptance floors appear below.

Historical margin revision

The margin evidence records fresh measurements, original and revised floors, toolchain, fingerprints and executed validation. Only domains with less than 5.00 percentage points of headroom were adjusted. The historical evidence above remains unchanged.

Domain Measured lines Original floor Revised floor Margin (points)
system/os 82.24% 77.24% 77.24% 5.00
system/runtime 81.19% 76.50% 76.19% 5.00
system/fs 81.08% 76.08% 76.08% 5.00
system/programs 98.05% 93.05% 93.05% 5.00
asset/bank 91.76% 86.76% 86.76% 5.00
firmware 88.26% 83.20% 83.20% 5.06
host-dependent 45.05% 40.05% 40.05% 5.00
host/headless 88.39% 83.39% 83.39% 5.00
vm/core 89.29% 89.12% 84.29% 5.00
bytecode 93.26% 93.23% 88.26% 5.00
hal/core 88.56% 88.56% 83.56% 5.00
drivers/core 84.51% 84.51% 79.51% 5.00
system/core 92.71% 92.71% 87.71% 5.00
tooling 64.75% 64.75% 59.75% 5.00
dev/test-support 85.96% 85.96% 80.96% 5.00

The full local make ci-domains validation passed with 939 tests passed and 9 ignored. system/runtime measured 81.26% in the initial run and 81.19% in verification; its final floor uses the lower measurement. All gates were rechecked against the verification report after that final adjustment.

Ownership-repair measurement

The ownership-repair evidence records the repaired module-tree ownership, fixed floors, normalized source paths, actual line counts and source/config/report fingerprints. Both historical JSON records above remain unchanged. The measurement uses the implementation working tree identified by its base HEAD and content fingerprints.

Domain Covered / total lines Measured lines Fixed floor Margin (points)
system/os 1410 / 1711 82.40% 77.24% 5.16
system/runtime 2762 / 3309 83.46% 76.19% 7.27
system/fs 1944 / 2469 78.73% 76.08% 2.65
system/programs 556 / 567 98.05% 93.05% 5.00
asset/bank 3096 / 3420 90.52% 86.76% 3.76
firmware 1990 / 2314 85.99% 83.20% 2.79
host-dependent 1196 / 2059 58.08% 40.05% 18.03
host/headless 739 / 855 86.43% 83.39% 3.04
vm/core 8308 / 9294 89.39% 84.29% 5.10
bytecode 3085 / 3303 93.39% 88.26% 5.13
hal/core 2093 / 2369 88.34% 83.56% 4.78
drivers/core 2728 / 3059 89.17% 79.51% 9.66
system/core 1353 / 1444 93.69% 87.71% 5.98
tooling 396 / 612 64.70% 59.75% 4.95
dev/test-support 99 / 115 86.08% 80.96% 5.12

The fresh macOS/aarch64 make ci-domains run passed all 15 coverage gates with 1,021 tests passed and 11 ignored. The 10 focused ownership/evidence tests and strict package Clippy also passed. No floor reduction or additional runtime behavior test was needed to pass the repaired map.

At the time of that ownership measurement the target used non-strict workspace Clippy, and a separate strict invocation failed on VM useless_vec, cloned_ref_to_slice_refs and items_after_test_module diagnostics. That historical record qualifies the repaired coverage map, not the later strict gate. The lint sources have since been corrected without blanket suppression or floor changes.

Platform-specific compilation and host scenarios can change report totals. Local evidence does not certify an unexecuted Linux or Intel Mac run; evaluate those runs against the configured floors and investigate differences.

Strict shared-gate qualification

On 2026-10-14, macOS/aarch64 with rustc 1.98.1, the shared target passed strict workspace/all-target/all-feature Clippy, all 1,039 ordinary tests (13 ignored), the five explicitly selected socket tests, three gate-runner regression tests, and all 15 fixed coverage domain gates. The separate fresh coverage execution also passed 1,039 tests with 13 ignored. Those repeated runs are not distinct behavioral tests and must not be added into a unique-test count.

The baseline strict invocation failed on the concrete VM lint sources above; the repaired sources pass without new lint allowances. Socket tests initially failed because the sandbox denied localhost TCP operations; the authorized outside-sandbox run passed all five. Gate failure fixtures preserve nonzero Make status, logs and commit/config identity, discard stale coverage, and retain fresh partial reports when a later gate fails. Interrupted runners record failure rather than success.

GitHub's YAML was parsed locally, and both pipeline definitions were inspected for shared provisioning, make ci-domains and unconditional failure-artifact publication. No remote GitHub or Jenkins build was launched. No Linux gate ran: the local Docker API was denied by the sandbox and the unrestricted daemon query did not return a usable result before interruption. Local macOS coverage does not certify Linux compilation or domain percentages. Such failures must be investigated on that platform without lowering the existing floors.

No physical audio, eight-hour soak, packaged-installation or ignored manual latency/layout qualification was selected by this gate. Prior manual observations remain separate evidence. Current executable metadata and reports are emitted under target/quality-gate/ and target/llvm-cov/ by every invocation.