Files
prometeu-runtime/docs/vm-arch/COVERAGE_DOMAINS.md
T
bquarkz 3e39a90083
Intrepid/Prometeu/Runtime/pipeline/pr-master This commit looks good
feat(vm): add VM-owned PCG64 random and coverage headroom
Implement GC-managed random generators, next/nextf intrinsics, bounded ranges and deterministic conformance coverage. Publish the runtime contract and producer guidance.

Give coverage domains at least five percentage points of measured headroom while preserving sufficient floors. Record local qualification: make ci-domains passes with 939 tests and all domain gates green.
2026-10-09 12:51:05 +01:00

189 lines
8.9 KiB
Markdown

# Coverage domains
Function: repository coverage ownership and CI operations.
Background lesson: [Domain gates must be owned by the repository](../../discussion/lessons/DSC-0002-runtime-edge-test-plan/LSN-0037-domain-gates-must-be-owned-by-the-repository.md).
Coverage domains assign crate source code to a responsible area. Unit,
integration and cross-layer tests keep their Cargo targets and contribute to
the domains containing the implementation they exercise. Test files do not
need to move into domain directories.
The authorities are
[runtime-edge-coverage-domains.json](../../files/config/runtime-edge-coverage-domains.json),
[the ownership auditor](../../scripts/audit-coverage-domains.py) and
[the evidence/gate helper](../../scripts/coverage-domain-evidence.sh).
The configuration retains its original runtime-edge filename and includes
the core workspace responsibilities as well.
## Ownership
Every Rust file under a repository crate src tree has exactly one owner.
Inclusive paths use repository-relative prefixes; optional exclude_paths
use the same semantics and win within their domain. A file without a domain
or with two domains fails the ownership audit. Sources with no executable
LLVM lines still need an owner.
| Domain | Source responsibility |
| --- | --- |
| system/os | OS facades, process and task services |
| system/runtime | System-side VM lifecycle, scheduling and render integration |
| system/fs | Runtime filesystem and memcard service |
| system/programs | Resident system programs |
| system/core | Remaining system sources, excluding the four specialized owners |
| asset/bank | Existing asset, cartridge and asset-driver prefixes |
| firmware | Firmware state and transitions |
| host-dependent | Desktop host |
| host/headless | Headless host and automation implementation |
| vm/core | VM execution, values, heap, verifier, admission, intrinsics and math |
| bytecode | PBX/Registry encodings, ISA, assembler, decoder and binary admission |
| hal/core | HAL contracts and helpers, excluding existing asset/bank ownership |
| drivers/core | Peripheral/render implementations, excluding the asset driver |
| tooling | CLI and dummy/stress PBX producers |
| dev/test-support | Shared test-program construction and development infrastructure source stubs |
Existing specialized paths retain their original ownership.
Residual domains use broad crate source prefixes with explicit exclusions,
so a new module inherits the appropriate owner. A new crate or conflicting
prefix must be assigned before the ownership audit can pass.
## How test suites contribute
VM unit/integration tests, including numeric_contract and the portable
MPFR-generated fixture, exercise vm/core. Bytecode round-trip and Registry
admission tests exercise bytecode; end-to-end loading can also exercise
vm/core. HAL/drivers suites contribute to their corresponding implementation
owners. Headless/CLI integration and development cross-layer suites can
contribute to several domains in one run.
Quality tests remain in prometeu-quality-checks/tests. They enforce the map
and drive the auditor/helper with independent source/report fixtures,
including omitted/overlapping owners, exclusions, absolute/relative filenames,
absent/empty evidence, invalid statistics and percentage regression.
The empty quality/layer library stubs inherit infrastructure ownership;
they do not create separate gates with zero executable lines.
Only crate src paths participate in this domain denominator; integration
test directories, examples, benches, external dependencies and standalone
Python oracle code are outside it. The LLVM instrumentation/report remains
the existing workspace all-features default-profile measurement. Unit tests
colocated in src retain the tool's existing reporting behavior.
The standalone MPFR precision/bit comparison remains complementary numeric
qualification. It is not replaced by line coverage and is not newly required
by make ci-domains. See [numeric qualification](NUMERIC_QUALIFICATION.md).
## Evidence and gate rules
Reports may use absolute or repository-relative source paths. The auditor
normalizes these names before assigning ownership; evidence display and
percentage checks consume its same domain statistics. This prevents path
aliases or separate matching implementations from disagreeing about totals.
The verbose file lists use normalized repository-relative names.
Each report source within scope must exist in the repository and appear once.
Every configured domain must have a positive aggregate instrumented-line
count. Missing reports, unknown sources, duplicate records, invalid counts
and empty domains fail validation. A zero baseline cannot make missing or
empty evidence pass.
The gate compares covered instrumented lines to total instrumented lines,
truncating the percentage downward to two decimals. A domain passes when
this percentage is at least baseline_percent. Counts for functions and
regions remain supporting evidence. Scenario/contract assertions continue
to govern behavior; a percentage does not replace them.
Measured coverage and the acceptance floor are separate values. This explicitly
reviewed revision gives every domain at least 5.00 percentage points of headroom
from the recorded local measurement. Its rule is
`new_floor = min(original_floor, measured_percent - 5.00)`, using the same
downward truncation. Floors already providing that margin remain unchanged.
Five percentage points means an 88.56% measurement permits an 83.56% floor;
it is not a relative five-percent reduction.
Floors remain fixed during CI. A failing run does not automatically lower them.
An intentional scope or floor revision requires explicit review with fresh
measured evidence. New domains also require an explicit floor supported by a
fresh measurement.
## Commands and CI
List domains, inspect source ownership and validate coverage evidence:
make coverage-domain-list
python3 scripts/audit-coverage-domains.py
make coverage-domain-audit
python3 scripts/audit-coverage-domains.py --report target/llvm-cov/report.json --json
Produce a fresh report and run all gates:
make ci-domains
The existing Jenkins configuration invokes this repository target. Its
coverage-domain-gate step also performs the ownership/evidence audit.
GitHub's workspace-test step discovers the quality regression tests; it
does not separately produce a domain coverage report.
Inspect a domain:
make coverage-domain-evidence DOMAIN=vm/core
scripts/coverage-domain-evidence.sh --check --verbose hal/core
scripts/coverage-domain-evidence.sh --check-all
Dependencies are the existing Rust/LLVM coverage tooling, Bash, Git, jq and rg,
plus Python 3 for the standard-library ownership auditor.
## Historical local baseline
The [initial measured evidence](COVERAGE_DOMAINS_EVIDENCE.json) records configuration,
source/report fingerprints, command/toolchain/host, line totals and gate
results for all 15 domains before the margin revision. It preserves the eight
earlier domain definitions and records these seven additional starting floors
on macOS/aarch64:
| New domain | Baseline lines |
| --- | ---: |
| vm/core | 89.12% |
| bytecode | 93.23% |
| hal/core | 88.56% |
| drivers/core | 84.51% |
| system/core | 92.71% |
| tooling | 64.75% |
| dev/test-support | 85.96% |
These are historical starting floors; current acceptance floors appear below.
## Current local margins
The [margin evidence](COVERAGE_DOMAINS_MARGIN.json) records fresh measurements,
original and revised floors, toolchain, fingerprints and executed validation.
Only domains with less than 5.00 percentage points of headroom were adjusted.
The historical evidence above remains unchanged.
| Domain | Measured lines | Original floor | Current floor | Margin (points) |
| --- | ---: | ---: | ---: | ---: |
| system/os | 82.24% | 77.24% | 77.24% | 5.00 |
| system/runtime | 81.19% | 76.50% | 76.19% | 5.00 |
| system/fs | 81.08% | 76.08% | 76.08% | 5.00 |
| system/programs | 98.05% | 93.05% | 93.05% | 5.00 |
| asset/bank | 91.76% | 86.76% | 86.76% | 5.00 |
| firmware | 88.26% | 83.20% | 83.20% | 5.06 |
| host-dependent | 45.05% | 40.05% | 40.05% | 5.00 |
| host/headless | 88.39% | 83.39% | 83.39% | 5.00 |
| vm/core | 89.29% | 89.12% | 84.29% | 5.00 |
| bytecode | 93.26% | 93.23% | 88.26% | 5.00 |
| hal/core | 88.56% | 88.56% | 83.56% | 5.00 |
| drivers/core | 84.51% | 84.51% | 79.51% | 5.00 |
| system/core | 92.71% | 92.71% | 87.71% | 5.00 |
| tooling | 64.75% | 64.75% | 59.75% | 5.00 |
| dev/test-support | 85.96% | 85.96% | 80.96% | 5.00 |
The full local `make ci-domains` validation passed with 939 tests passed and
9 ignored. `system/runtime` measured 81.26% in the initial run and 81.19%
in verification; its final floor uses the lower measurement. All gates were
rechecked against the verification report after that final adjustment.
Platform-specific compilation and host scenarios can change report totals.
Local evidence does not certify an unexecuted Linux or Intel Mac run; evaluate
those runs against the configured floors and investigate differences.