18 KiB
Coverage domains
Function: repository coverage ownership and CI operations.
Background lesson: Domain gates must be owned by the repository.
Coverage domains assign crate source code to a responsible area. Unit, integration and cross-layer tests keep their Cargo targets and contribute to the domains containing the implementation they exercise. Test files do not need to move into domain directories.
The authorities are runtime-edge-coverage-domains.json, the ownership auditor and the evidence/gate helper. The configuration retains its original runtime-edge filename and includes the core workspace responsibilities as well.
Ownership
Every Rust file under a repository crate src tree has exactly one owner. Inclusive paths use repository-relative prefixes; optional exclude_paths use the same semantics and win within their domain. A file without a domain or with two domains fails the ownership audit. Sources with no executable LLVM lines still need an owner.
| Domain | Source responsibility |
|---|---|
| system/os | OS facades, process and task services |
| system/runtime | System-side VM lifecycle, scheduling and render integration |
| system/fs | Runtime filesystem, memcard.rs and the complete memcard/ module tree |
| system/programs | Resident system programs |
| system/core | Remaining system sources, excluding the four specialized owners |
| asset/bank | HAL asset/cartridge prefixes, driver asset.rs and the complete driver asset/ module tree |
| firmware | Firmware state and transitions |
| host-dependent | Desktop host |
| host/headless | Headless host and automation implementation |
| vm/core | VM execution, values, heap, verifier, admission, intrinsics and math |
| bytecode | PBX/Registry encodings, ISA, assembler, decoder and binary admission |
| hal/core | HAL contracts and helpers, excluding existing asset/bank ownership |
| drivers/core | Peripheral/render implementations, excluding the asset driver |
| tooling | CLI and dummy/stress PBX producers |
| dev/test-support | Shared test-program construction and development infrastructure source stubs |
Existing specialized paths retain their original ownership. Both module entry files and sibling module directories need explicit prefixes. Memcard and asset-driver descendants, including future nested modules, retain their specialized owner even when absent from the coverage report or reporting zero executable lines. The corresponding residual exclusions cover both forms. Residual domains use broad crate source prefixes with explicit exclusions, so a new module inherits the appropriate owner. A new crate or conflicting prefix must be assigned before the ownership audit can pass.
How test suites contribute
VM unit/integration tests, including numeric_contract and the portable MPFR-generated fixture, exercise vm/core. Bytecode round-trip and Registry admission tests exercise bytecode; end-to-end loading can also exercise vm/core. HAL/drivers suites contribute to their corresponding implementation owners. Headless/CLI integration and development cross-layer suites can contribute to several domains in one run.
Quality tests remain in prometeu-quality-checks/tests. They enforce the map and drive the auditor/helper with independent source/report fixtures, including omitted/overlapping owners, exclusions, absolute/relative filenames, absent/empty evidence, invalid statistics and percentage regression. Expected-owner tests use the auditor's ownership function for entry files, every existing descendant and future nested paths. Synthetic reports also check absent and zero-line descendants. Exclusive residual fallback is not sufficient: it can silently transfer specialized code into the wrong gate. The empty quality/layer library stubs inherit infrastructure ownership; they do not create separate gates with zero executable lines.
Only crate src paths participate in this domain denominator; integration test directories, examples, benches, external dependencies and standalone Python oracle code are outside it. The LLVM instrumentation/report remains the existing workspace all-features default-profile measurement. Unit tests colocated in src retain the tool's existing reporting behavior.
The standalone MPFR precision/bit comparison remains complementary numeric qualification. It is not replaced by line coverage and is not newly required by make ci-domains. See numeric qualification.
Evidence and gate rules
Reports may use absolute or repository-relative source paths. The auditor normalizes these names before assigning ownership; evidence display and percentage checks consume its same domain statistics. This prevents path aliases or separate matching implementations from disagreeing about totals. The verbose file lists use normalized repository-relative names.
Each report source within scope must exist in the repository and appear once. Every configured domain must have a positive aggregate instrumented-line count. Missing reports, unknown sources, duplicate records, invalid counts and empty domains fail validation. A zero baseline cannot make missing or empty evidence pass.
The gate compares covered instrumented lines to total instrumented lines, truncating the percentage downward to two decimals. A domain passes when this percentage is at least baseline_percent. Counts for functions and regions remain supporting evidence. Scenario/contract assertions continue to govern behavior; a percentage does not replace them.
Measured coverage and the acceptance floor are separate values. The historical
margin revision gave every domain at least 5.00 percentage points of headroom
from that recorded local measurement. Its rule was
new_floor = min(original_floor, measured_percent - 5.00), using the same
downward truncation. Floors already providing that margin remain unchanged.
Five percentage points means an 88.56% measurement permits an 83.56% floor;
it is not a relative five-percent reduction.
That historical margin is not a guarantee about later measurements or corrected
ownership. The current map restores the reviewed system/core floor of 87.71%
and preserves every other floor.
Floors remain fixed during CI. A failing run does not automatically lower them. An intentional scope or floor revision requires explicit review with fresh measured evidence. New domains also require an explicit floor supported by a fresh measurement.
Commands and CI
Jenkins orchestrates the quality gate through explicit
stages invoking Makefile targets. make ci-domains runs the same checks locally
and in GitHub CI; make ci is an alias.
Invoke one gate per checkout at a time.
The ordered checks are:
- Clean previous LLVM coverage instrumentation and reports.
- Check formatting.
- Run workspace Clippy with warnings denied.
- Run workspace tests across all targets and features.
- Run the five ignored debugger socket tests.
- Generate workspace coverage, HTML, full JSON, Cobertura and summary JSON.
- Audit source ownership/evidence and check every fixed coverage domain floor.
The socket command is deliberately restricted:
cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored
It selects test_debug_port_opens, test_debug_reconnection,
test_debug_refuse_second_connection, test_get_state_returns_response and
test_debug_resume_on_disconnect. It does not select the audio-device test.
Local sandbox TCP permissions must permit bind/connect; a permission denial is
not evidence of a debugger behavior regression.
The local Make recipe executes child targets serially with -j1, even if the outer
invocation uses -j. Jenkins executes its stages sequentially. Do not run a separate Cargo,
coverage-clean or report process concurrently in the same checkout: serialized
steps inside one gate cannot protect against an independent process.
Prerequisites and reports
Jenkins uses bash scripts/check-quality-gate-prerequisites.sh to validate an
already provisioned agent. The job does not install system packages, Rust
components or Cargo tools and does not require root or sudo. The agent image
must supply a supported Rust toolchain and rustup in CARGO_HOME/bin, rustfmt,
Clippy, llvm-tools-preview and pinned cargo-llvm-cov 0.8.5.
Provision the Jenkins agent before running jobs. The standalone
bash scripts/provision-quality-gate.sh helper requires root or sudo on
Debian/Ubuntu and an existing Rust/rustup installation; run it during agent
setup, with Rust tools installed for the job user. GitHub's disposable hosted
runner still uses this helper after installing stable Rust.
Linux packages include build-essential, pkg-config, libasound2-dev, libudev-dev, libxkbcommon-dev, libwayland-dev, libx11-dev, libxrandr-dev, libxi-dev, Python 3, jq, Git, curl and CA certificates. ALSA development files satisfy CPAL's Linux build dependency; a running audio device is not needed. Non-Linux local hosts must already provide native compiler/SDK tools, Bash, Make, pkg-config, Python 3, jq and Git. The provisioning script verifies these commands.
Jenkins owns stage status, console logs, build identity and artifact fingerprints. GitHub retains step logs and uploaded coverage artifacts. Local execution streams command output and stops at the first failed target; no separate Python runner or metadata report is required.
Both pipelines discard old coverage reports before prerequisite checks or setup,
and archive current target/llvm-cov/** reports even after failure. Early failures
may leave no report; later failures retain partial current reports. Jenkins also
publishes HTML when present. Artifact publication does not mask a failed gate.
List domains or inspect ownership and existing evidence:
make coverage-domain-list
python3 scripts/audit-coverage-domains.py
make coverage-domain-audit
make coverage-domain-evidence DOMAIN=vm/core
scripts/coverage-domain-evidence.sh --check --verbose hal/core
Explicit manual qualification
Ordinary all-target tests and coverage leave ignored device/measurement tests ignored. The shared target only opts in to the five socket tests above. It does not certify the following manual workloads:
# Physical desktop audio, exact named test.
cargo test -p prometeu-host-desktop-winit --lib audio::control_tests::desktop_audio_device_smoke -- --ignored --exact --nocapture
# Physical callback duration under host I/O pressure.
cargo test --release -p prometeu-host-desktop-winit --test audio_callback_allocations physical_callback_duration_16_voices_with_io_pressure -- --ignored --exact --nocapture
# Eight-hour active release soak.
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --seconds 28800
# Packaged installation; set PROMETEU_PROTOCOL_TEST_INSTALL to an extracted distribution first.
cargo test -p prometeu-cli --test automation_process packaged_installation_smoke -- --ignored --exact --nocapture
# File-host commit latency with the production asset decoder.
cargo test --release -p prometeu-system --all-features qualify_file_commits_with_production_asset_decoder -- --ignored --nocapture
# Explicit layout observation.
cargo test --release -p prometeu-bytecode --test registry_v0 reference_map_probe -- --ignored --exact --nocapture
Other ignored timing probes remain opt-in by their individual test names. All-target execution can print ordinary benchmark observations; this does not establish completion of ignored manual qualification. For scope and prior measured results see resource stability and audio callback qualification.
Historical local baseline
The initial measured evidence records configuration, source/report fingerprints, command/toolchain/host, line totals and gate results for all 15 domains before the margin revision. It preserves the eight earlier domain definitions and records these seven additional starting floors on macOS/aarch64:
| New domain | Baseline lines |
|---|---|
| vm/core | 89.12% |
| bytecode | 93.23% |
| hal/core | 88.56% |
| drivers/core | 84.51% |
| system/core | 92.71% |
| tooling | 64.75% |
| dev/test-support | 85.96% |
These are historical starting floors; current acceptance floors appear below.
Historical margin revision
The margin evidence records fresh measurements, original and revised floors, toolchain, fingerprints and executed validation. Only domains with less than 5.00 percentage points of headroom were adjusted. The historical evidence above remains unchanged.
| Domain | Measured lines | Original floor | Revised floor | Margin (points) |
|---|---|---|---|---|
| system/os | 82.24% | 77.24% | 77.24% | 5.00 |
| system/runtime | 81.19% | 76.50% | 76.19% | 5.00 |
| system/fs | 81.08% | 76.08% | 76.08% | 5.00 |
| system/programs | 98.05% | 93.05% | 93.05% | 5.00 |
| asset/bank | 91.76% | 86.76% | 86.76% | 5.00 |
| firmware | 88.26% | 83.20% | 83.20% | 5.06 |
| host-dependent | 45.05% | 40.05% | 40.05% | 5.00 |
| host/headless | 88.39% | 83.39% | 83.39% | 5.00 |
| vm/core | 89.29% | 89.12% | 84.29% | 5.00 |
| bytecode | 93.26% | 93.23% | 88.26% | 5.00 |
| hal/core | 88.56% | 88.56% | 83.56% | 5.00 |
| drivers/core | 84.51% | 84.51% | 79.51% | 5.00 |
| system/core | 92.71% | 92.71% | 87.71% | 5.00 |
| tooling | 64.75% | 64.75% | 59.75% | 5.00 |
| dev/test-support | 85.96% | 85.96% | 80.96% | 5.00 |
The full local make ci-domains validation passed with 939 tests passed and
9 ignored. system/runtime measured 81.26% in the initial run and 81.19%
in verification; its final floor uses the lower measurement. All gates were
rechecked against the verification report after that final adjustment.
Current coverage measurement
The current evidence identifies the committed source by HEAD and content fingerprints, records the strict shared gate and retains the earlier ownership-repair measurement as historical evidence. Both original historical JSON records above remain unchanged. The latest measurement uses the repaired module-tree map with all 15 fixed floors.
| Domain | Covered / total lines | Measured lines | Fixed floor | Margin (points) |
|---|---|---|---|---|
| system/os | 1410 / 1711 | 82.40% | 77.24% | 5.16 |
| system/runtime | 2757 / 3309 | 83.31% | 76.19% | 7.12 |
| system/fs | 1946 / 2469 | 78.81% | 76.08% | 2.73 |
| system/programs | 556 / 567 | 98.05% | 93.05% | 5.00 |
| asset/bank | 3176 / 3434 | 92.48% | 86.76% | 5.72 |
| firmware | 1990 / 2314 | 85.99% | 83.20% | 2.79 |
| host-dependent | 1531 / 2393 | 63.97% | 40.05% | 23.92 |
| host/headless | 739 / 855 | 86.43% | 83.39% | 3.04 |
| vm/core | 8522 / 9526 | 89.46% | 84.29% | 5.17 |
| bytecode | 3085 / 3303 | 93.39% | 88.26% | 5.13 |
| hal/core | 2093 / 2369 | 88.34% | 83.56% | 4.78 |
| drivers/core | 2775 / 3107 | 89.31% | 79.51% | 9.80 |
| system/core | 1353 / 1444 | 93.69% | 87.71% | 5.98 |
| tooling | 396 / 612 | 64.70% | 59.75% | 4.95 |
| dev/test-support | 99 / 115 | 86.08% | 80.96% | 5.12 |
The fresh macOS/aarch64 make ci-domains run passed all 15 domain gates,
1,060 ordinary tests (13 ignored), five explicit socket tests and three
runner regression tests. The fresh coverage replay also passed 1,060 tests;
it is not an additional set of unique behavioral tests. All ten focused
ownership/evidence tests passed again on the committed source. No floor
was lowered. The retained ownership-repair measurement used the earlier
non-strict gate; the current measurement passes strict workspace Clippy.
Platform-specific compilation and host scenarios can change report totals. Local evidence does not certify an unexecuted Linux or Intel Mac run; evaluate those runs against the configured floors and investigate differences.
Strict shared-gate qualification
On 2026-10-14, macOS/aarch64 with rustc 1.98.1, the shared target passed strict workspace/all-target/all-feature Clippy, all 1,060 ordinary tests (13 ignored), the five explicitly selected socket tests, three gate-runner regression tests, and all 15 fixed coverage domain gates. The separate fresh coverage execution also passed 1,060 tests with 13 ignored. Those repeated runs are not distinct behavioral tests and must not be added into a unique-test count.
The baseline strict invocation failed on the concrete VM lint sources above; the repaired sources pass without new lint allowances. Socket tests initially failed because the sandbox denied localhost TCP operations; the authorized outside-sandbox run passed all five. Gate failure fixtures preserve nonzero Make status, logs and commit/config identity, discard stale coverage, and retain fresh partial reports when a later gate fails. Interrupted runners record failure rather than success.
GitHub's YAML was parsed locally, and both pipeline definitions were inspected
for shared provisioning, make ci-domains and unconditional failure-artifact
publication. No remote GitHub or Jenkins build was launched. No Linux gate ran:
the local Docker API was denied by the sandbox and the unrestricted daemon query
did not return a usable result before interruption. Local macOS coverage does not
certify Linux compilation or domain percentages. Such failures must be investigated
on that platform without lowering the existing floors.
No physical audio, eight-hour soak, packaged-installation or ignored manual
latency/layout qualification was selected by this gate. Prior manual observations
remain separate evidence. Current coverage reports are emitted under target/llvm-cov/; CI retains execution
logs and build status.