Intrepid/Prometeu/Runtime/pipeline/pr-master There was a failure building this commit
148 lines
7.0 KiB
Markdown
148 lines
7.0 KiB
Markdown
# Coverage domains
|
|
|
|
Function: repository coverage ownership and CI operations.
|
|
|
|
Background lesson: [Domain gates must be owned by the repository](../../discussion/lessons/DSC-0002-runtime-edge-test-plan/LSN-0037-domain-gates-must-be-owned-by-the-repository.md).
|
|
|
|
Coverage domains assign crate source code to a responsible area. Unit,
|
|
integration and cross-layer tests keep their Cargo targets and contribute to
|
|
the domains containing the implementation they exercise. Test files do not
|
|
need to move into domain directories.
|
|
|
|
The authorities are
|
|
[runtime-edge-coverage-domains.json](../../files/config/runtime-edge-coverage-domains.json),
|
|
[the ownership auditor](../../scripts/audit-coverage-domains.py) and
|
|
[the evidence/gate helper](../../scripts/coverage-domain-evidence.sh).
|
|
The configuration retains its original runtime-edge filename and includes
|
|
the core workspace responsibilities as well.
|
|
|
|
## Ownership
|
|
|
|
Every Rust file under a repository crate src tree has exactly one owner.
|
|
Inclusive paths use repository-relative prefixes; optional exclude_paths
|
|
use the same semantics and win within their domain. A file without a domain
|
|
or with two domains fails the ownership audit. Sources with no executable
|
|
LLVM lines still need an owner.
|
|
|
|
| Domain | Source responsibility |
|
|
| --- | --- |
|
|
| system/os | OS facades, process and task services |
|
|
| system/runtime | System-side VM lifecycle, scheduling and render integration |
|
|
| system/fs | Runtime filesystem and memcard service |
|
|
| system/programs | Resident system programs |
|
|
| system/core | Remaining system sources, excluding the four specialized owners |
|
|
| asset/bank | Existing asset, cartridge and asset-driver prefixes |
|
|
| firmware | Firmware state and transitions |
|
|
| host-dependent | Desktop host |
|
|
| host/headless | Headless host and automation implementation |
|
|
| vm/core | VM execution, values, heap, verifier, admission, intrinsics and math |
|
|
| bytecode | PBX/Registry encodings, ISA, assembler, decoder and binary admission |
|
|
| hal/core | HAL contracts and helpers, excluding existing asset/bank ownership |
|
|
| drivers/core | Peripheral/render implementations, excluding the asset driver |
|
|
| tooling | CLI and dummy/stress PBX producers |
|
|
| dev/test-support | Shared test-program construction and development infrastructure source stubs |
|
|
|
|
Existing specialized paths and baselines retain their original values.
|
|
Residual domains use broad crate source prefixes with explicit exclusions,
|
|
so a new module inherits the appropriate owner. A new crate or conflicting
|
|
prefix must be assigned before the ownership audit can pass.
|
|
|
|
## How test suites contribute
|
|
|
|
VM unit/integration tests, including numeric_contract and the portable
|
|
MPFR-generated fixture, exercise vm/core. Bytecode round-trip and Registry
|
|
admission tests exercise bytecode; end-to-end loading can also exercise
|
|
vm/core. HAL/drivers suites contribute to their corresponding implementation
|
|
owners. Headless/CLI integration and development cross-layer suites can
|
|
contribute to several domains in one run.
|
|
|
|
Quality tests remain in prometeu-quality-checks/tests. They enforce the map
|
|
and drive the auditor/helper with independent source/report fixtures,
|
|
including omitted/overlapping owners, exclusions, absolute/relative filenames,
|
|
absent/empty evidence, invalid statistics and percentage regression.
|
|
The empty quality/layer library stubs inherit infrastructure ownership;
|
|
they do not create separate gates with zero executable lines.
|
|
|
|
Only crate src paths participate in this domain denominator; integration
|
|
test directories, examples, benches, external dependencies and standalone
|
|
Python oracle code are outside it. The LLVM instrumentation/report remains
|
|
the existing workspace all-features default-profile measurement. Unit tests
|
|
colocated in src retain the tool's existing reporting behavior.
|
|
|
|
The standalone MPFR precision/bit comparison remains complementary numeric
|
|
qualification. It is not replaced by line coverage and is not newly required
|
|
by make ci-domains. See [numeric qualification](NUMERIC_QUALIFICATION.md).
|
|
|
|
## Evidence and gate rules
|
|
|
|
Reports may use absolute or repository-relative source paths. The auditor
|
|
normalizes these names before assigning ownership; evidence display and
|
|
percentage checks consume its same domain statistics. This prevents path
|
|
aliases or separate matching implementations from disagreeing about totals.
|
|
The verbose file lists use normalized repository-relative names.
|
|
|
|
Each report source within scope must exist in the repository and appear once.
|
|
Every configured domain must have a positive aggregate instrumented-line
|
|
count. Missing reports, unknown sources, duplicate records, invalid counts
|
|
and empty domains fail validation. A zero baseline cannot make missing or
|
|
empty evidence pass.
|
|
|
|
The gate compares covered instrumented lines to total instrumented lines,
|
|
truncating the percentage downward to two decimals. A domain passes when
|
|
this percentage is at least baseline_percent. Counts for functions and
|
|
regions remain supporting evidence. Scenario/contract assertions continue
|
|
to govern behavior; a percentage does not replace them.
|
|
|
|
New domains start from a fresh measured baseline, using the same truncation.
|
|
Do not lower existing thresholds to hide a failing run. An intentional scope
|
|
or baseline revision requires explicit review with measured evidence.
|
|
|
|
## Commands and CI
|
|
|
|
List domains, inspect source ownership and validate coverage evidence:
|
|
|
|
make coverage-domain-list
|
|
python3 scripts/audit-coverage-domains.py
|
|
make coverage-domain-audit
|
|
python3 scripts/audit-coverage-domains.py --report target/llvm-cov/report.json --json
|
|
|
|
Produce a fresh report and run all gates:
|
|
|
|
make ci-domains
|
|
|
|
The existing Jenkins configuration invokes this repository target. Its
|
|
coverage-domain-gate step also performs the ownership/evidence audit.
|
|
GitHub's workspace-test step discovers the quality regression tests; it
|
|
does not separately produce a domain coverage report.
|
|
|
|
Inspect a domain:
|
|
|
|
make coverage-domain-evidence DOMAIN=vm/core
|
|
scripts/coverage-domain-evidence.sh --check --verbose hal/core
|
|
scripts/coverage-domain-evidence.sh --check-all
|
|
|
|
Dependencies are the existing Rust/LLVM coverage tooling, Bash, Git, jq and rg,
|
|
plus Python 3 for the standard-library ownership auditor.
|
|
|
|
## Published local baseline
|
|
|
|
The [measured evidence](COVERAGE_DOMAINS_EVIDENCE.json) records configuration,
|
|
source/report fingerprints, command/toolchain/host, line totals and gate
|
|
results for all 15 domains. The eight earlier domain definitions are
|
|
preserved. The seven additional baselines measured on macOS/aarch64 are:
|
|
|
|
| New domain | Baseline lines |
|
|
| --- | ---: |
|
|
| vm/core | 89.12% |
|
|
| bytecode | 93.23% |
|
|
| hal/core | 88.56% |
|
|
| drivers/core | 84.51% |
|
|
| system/core | 92.71% |
|
|
| tooling | 64.75% |
|
|
| dev/test-support | 85.96% |
|
|
|
|
These are observed starting baselines, not uniform target percentages.
|
|
Platform-specific compilation and host scenarios can change report totals.
|
|
Local evidence does not certify an unexecuted Linux or Intel Mac run; evaluate
|
|
those runs using the unchanged repository policy and investigate differences.
|