feat(vm): add deterministic gameplay math and domain coverage gates
Intrepid/Prometeu/Runtime/pipeline/pr-master There was a failure building this commit
Intrepid/Prometeu/Runtime/pipeline/pr-master There was a failure building this commit
This commit is contained in:
@@ -0,0 +1,147 @@
|
||||
# Coverage domains
|
||||
|
||||
Function: repository coverage ownership and CI operations.
|
||||
|
||||
Background lesson: [Domain gates must be owned by the repository](../../discussion/lessons/DSC-0002-runtime-edge-test-plan/LSN-0037-domain-gates-must-be-owned-by-the-repository.md).
|
||||
|
||||
Coverage domains assign crate source code to a responsible area. Unit,
|
||||
integration and cross-layer tests keep their Cargo targets and contribute to
|
||||
the domains containing the implementation they exercise. Test files do not
|
||||
need to move into domain directories.
|
||||
|
||||
The authorities are
|
||||
[runtime-edge-coverage-domains.json](../../files/config/runtime-edge-coverage-domains.json),
|
||||
[the ownership auditor](../../scripts/audit-coverage-domains.py) and
|
||||
[the evidence/gate helper](../../scripts/coverage-domain-evidence.sh).
|
||||
The configuration retains its original runtime-edge filename and includes
|
||||
the core workspace responsibilities as well.
|
||||
|
||||
## Ownership
|
||||
|
||||
Every Rust file under a repository crate src tree has exactly one owner.
|
||||
Inclusive paths use repository-relative prefixes; optional exclude_paths
|
||||
use the same semantics and win within their domain. A file without a domain
|
||||
or with two domains fails the ownership audit. Sources with no executable
|
||||
LLVM lines still need an owner.
|
||||
|
||||
| Domain | Source responsibility |
|
||||
| --- | --- |
|
||||
| system/os | OS facades, process and task services |
|
||||
| system/runtime | System-side VM lifecycle, scheduling and render integration |
|
||||
| system/fs | Runtime filesystem and memcard service |
|
||||
| system/programs | Resident system programs |
|
||||
| system/core | Remaining system sources, excluding the four specialized owners |
|
||||
| asset/bank | Existing asset, cartridge and asset-driver prefixes |
|
||||
| firmware | Firmware state and transitions |
|
||||
| host-dependent | Desktop host |
|
||||
| host/headless | Headless host and automation implementation |
|
||||
| vm/core | VM execution, values, heap, verifier, admission, intrinsics and math |
|
||||
| bytecode | PBX/Registry encodings, ISA, assembler, decoder and binary admission |
|
||||
| hal/core | HAL contracts and helpers, excluding existing asset/bank ownership |
|
||||
| drivers/core | Peripheral/render implementations, excluding the asset driver |
|
||||
| tooling | CLI and dummy/stress PBX producers |
|
||||
| dev/test-support | Shared test-program construction and development infrastructure source stubs |
|
||||
|
||||
Existing specialized paths and baselines retain their original values.
|
||||
Residual domains use broad crate source prefixes with explicit exclusions,
|
||||
so a new module inherits the appropriate owner. A new crate or conflicting
|
||||
prefix must be assigned before the ownership audit can pass.
|
||||
|
||||
## How test suites contribute
|
||||
|
||||
VM unit/integration tests, including numeric_contract and the portable
|
||||
MPFR-generated fixture, exercise vm/core. Bytecode round-trip and Registry
|
||||
admission tests exercise bytecode; end-to-end loading can also exercise
|
||||
vm/core. HAL/drivers suites contribute to their corresponding implementation
|
||||
owners. Headless/CLI integration and development cross-layer suites can
|
||||
contribute to several domains in one run.
|
||||
|
||||
Quality tests remain in prometeu-quality-checks/tests. They enforce the map
|
||||
and drive the auditor/helper with independent source/report fixtures,
|
||||
including omitted/overlapping owners, exclusions, absolute/relative filenames,
|
||||
absent/empty evidence, invalid statistics and percentage regression.
|
||||
The empty quality/layer library stubs inherit infrastructure ownership;
|
||||
they do not create separate gates with zero executable lines.
|
||||
|
||||
Only crate src paths participate in this domain denominator; integration
|
||||
test directories, examples, benches, external dependencies and standalone
|
||||
Python oracle code are outside it. The LLVM instrumentation/report remains
|
||||
the existing workspace all-features default-profile measurement. Unit tests
|
||||
colocated in src retain the tool's existing reporting behavior.
|
||||
|
||||
The standalone MPFR precision/bit comparison remains complementary numeric
|
||||
qualification. It is not replaced by line coverage and is not newly required
|
||||
by make ci-domains. See [numeric qualification](NUMERIC_QUALIFICATION.md).
|
||||
|
||||
## Evidence and gate rules
|
||||
|
||||
Reports may use absolute or repository-relative source paths. The auditor
|
||||
normalizes these names before assigning ownership; evidence display and
|
||||
percentage checks consume its same domain statistics. This prevents path
|
||||
aliases or separate matching implementations from disagreeing about totals.
|
||||
The verbose file lists use normalized repository-relative names.
|
||||
|
||||
Each report source within scope must exist in the repository and appear once.
|
||||
Every configured domain must have a positive aggregate instrumented-line
|
||||
count. Missing reports, unknown sources, duplicate records, invalid counts
|
||||
and empty domains fail validation. A zero baseline cannot make missing or
|
||||
empty evidence pass.
|
||||
|
||||
The gate compares covered instrumented lines to total instrumented lines,
|
||||
truncating the percentage downward to two decimals. A domain passes when
|
||||
this percentage is at least baseline_percent. Counts for functions and
|
||||
regions remain supporting evidence. Scenario/contract assertions continue
|
||||
to govern behavior; a percentage does not replace them.
|
||||
|
||||
New domains start from a fresh measured baseline, using the same truncation.
|
||||
Do not lower existing thresholds to hide a failing run. An intentional scope
|
||||
or baseline revision requires explicit review with measured evidence.
|
||||
|
||||
## Commands and CI
|
||||
|
||||
List domains, inspect source ownership and validate coverage evidence:
|
||||
|
||||
make coverage-domain-list
|
||||
python3 scripts/audit-coverage-domains.py
|
||||
make coverage-domain-audit
|
||||
python3 scripts/audit-coverage-domains.py --report target/llvm-cov/report.json --json
|
||||
|
||||
Produce a fresh report and run all gates:
|
||||
|
||||
make ci-domains
|
||||
|
||||
The existing Jenkins configuration invokes this repository target. Its
|
||||
coverage-domain-gate step also performs the ownership/evidence audit.
|
||||
GitHub's workspace-test step discovers the quality regression tests; it
|
||||
does not separately produce a domain coverage report.
|
||||
|
||||
Inspect a domain:
|
||||
|
||||
make coverage-domain-evidence DOMAIN=vm/core
|
||||
scripts/coverage-domain-evidence.sh --check --verbose hal/core
|
||||
scripts/coverage-domain-evidence.sh --check-all
|
||||
|
||||
Dependencies are the existing Rust/LLVM coverage tooling, Bash, Git, jq and rg,
|
||||
plus Python 3 for the standard-library ownership auditor.
|
||||
|
||||
## Published local baseline
|
||||
|
||||
The [measured evidence](COVERAGE_DOMAINS_EVIDENCE.json) records configuration,
|
||||
source/report fingerprints, command/toolchain/host, line totals and gate
|
||||
results for all 15 domains. The eight earlier domain definitions are
|
||||
preserved. The seven additional baselines measured on macOS/aarch64 are:
|
||||
|
||||
| New domain | Baseline lines |
|
||||
| --- | ---: |
|
||||
| vm/core | 89.12% |
|
||||
| bytecode | 93.23% |
|
||||
| hal/core | 88.56% |
|
||||
| drivers/core | 84.51% |
|
||||
| system/core | 92.71% |
|
||||
| tooling | 64.75% |
|
||||
| dev/test-support | 85.96% |
|
||||
|
||||
These are observed starting baselines, not uniform target percentages.
|
||||
Platform-specific compilation and host scenarios can change report totals.
|
||||
Local evidence does not certify an unexecuted Linux or Intel Mac run; evaluate
|
||||
those runs using the unchanged repository policy and investigate differences.
|
||||
Reference in New Issue
Block a user