210 lines
7.1 KiB
Markdown
210 lines
7.1 KiB
Markdown
---
|
|
id: DEC-0020
|
|
ticket: runtime-edge-test-plan
|
|
title: Decision - Runtime Edge Coverage Governance by Domain
|
|
status: in_progress
|
|
created: 2026-04-20
|
|
accepted: 2026-04-20
|
|
agenda: AGD-0001
|
|
plans: [PLN-0037, PLN-0038, PLN-0039]
|
|
tags: [tests, coverage, runtime, firmware, host]
|
|
---
|
|
|
|
## Status
|
|
|
|
In progress.
|
|
|
|
Derived from `AGD-0001`.
|
|
|
|
This decision records the normative direction for runtime edge coverage governance.
|
|
It has been accepted and now drives one or more execution plans.
|
|
|
|
## Contexto
|
|
|
|
PROMETEU no longer has the same problem that originally motivated `AGD-0001`.
|
|
The runtime edge is no longer broadly untested. The repository now already contains meaningful coverage for runtime lifecycle paths, `VirtualFS`, asset/preload behavior, status-first surfaces, and part of the desktop host integration.
|
|
|
|
The remaining problem is governance, not raw test absence.
|
|
|
|
Global coverage metrics already exist through `cargo llvm-cov`, and CI already enforces a workspace-wide minimum gate. That is useful, but insufficient on its own. A good aggregated number does not guarantee that firmware transitions, boot flow, host-dependent surfaces, or domain-specific status-first contracts are covered at the right level.
|
|
|
|
The project therefore needs an explicit domain-based acceptance model for tests:
|
|
|
|
- a small canonical domain set;
|
|
- mandatory scenarios per domain;
|
|
- a rule for when a change must add or adjust tests;
|
|
- a clear relationship between global coverage and domain-oriented evidence.
|
|
|
|
## Decisao
|
|
|
|
PROMETEU SHALL govern runtime-edge test sufficiency through a two-layer model:
|
|
|
|
1. a mandatory global coverage gate enforced in CI; and
|
|
2. a domain-based acceptance gate driven by mandatory scenarios plus domain-scoped coverage evidence.
|
|
|
|
The canonical domains SHALL be:
|
|
|
|
1. `system/runtime`
|
|
2. `fs`
|
|
3. `asset/bank`
|
|
4. `firmware`
|
|
5. `host-dependent`
|
|
|
|
`host-dependent` SHALL be treated as a transversal execution/testability slice, not as a product subsystem parallel to `fs`, `asset/bank`, or `firmware`.
|
|
|
|
Domain gates SHALL be scenario-first.
|
|
Coverage percentages by domain MAY exist, but they SHALL start from an initial baseline of `0` and SHALL be tightened incrementally over time as segmentation and suites mature.
|
|
|
|
No PR that changes the observable contract of a canonical domain SHALL be accepted without one of the following:
|
|
|
|
- tests added or adjusted for that domain; or
|
|
- an explicit justification that the observable contract did not change.
|
|
|
|
Improved aggregate coverage SHALL NOT be accepted as sufficient evidence on its own when the changed behavior belongs to a canonical domain with mandatory scenarios.
|
|
|
|
## Rationale
|
|
|
|
### Why not global-only coverage
|
|
|
|
A single global percentage is easy to automate, but too weak as a risk control.
|
|
It allows firmware and host-sensitive regressions to hide behind unrelated line execution elsewhere in the workspace.
|
|
|
|
### Why not rigid per-domain percentages immediately
|
|
|
|
The project does not yet have a stable enough domain segmentation baseline to enforce hard per-domain minimums from day one without adding friction disproportionate to the benefit.
|
|
|
|
Starting from `0` preserves forward motion while still making domain evidence mandatory during review.
|
|
|
|
### Why scenario-first
|
|
|
|
The platform risk lies in behavioral contracts:
|
|
|
|
- lifecycle boundaries,
|
|
- boot transitions,
|
|
- status-first surfaces,
|
|
- filesystem normalization and invalid-state handling,
|
|
- preload/bootstrap versus runtime operational failures,
|
|
- host-only integration boundaries.
|
|
|
|
Those are not safely governed by percentages alone.
|
|
|
|
### Why keep coverage in the model
|
|
|
|
Coverage remains useful as:
|
|
|
|
- an operational CI signal;
|
|
- a regression detector;
|
|
- a way to inspect whether the changed domain actually exercised the intended area.
|
|
|
|
The correct role of coverage is supporting evidence, not sole authority.
|
|
|
|
## Invariantes / Contrato
|
|
|
|
### Global Coverage Contract
|
|
|
|
- CI MUST keep a workspace-wide coverage gate.
|
|
- The current `llvm-cov` based global gate remains valid as the operational baseline.
|
|
- Changing the tooling is out of scope for this decision.
|
|
|
|
### Domain Gate Contract
|
|
|
|
Each canonical domain MUST define mandatory scenarios and acceptance expectations.
|
|
|
|
#### `system/runtime`
|
|
|
|
Mandatory scenarios:
|
|
|
|
- initialization;
|
|
- reset and cleanup;
|
|
- trap versus panic behavior;
|
|
- pause and breakpoint behavior;
|
|
- `FRAME_SYNC` and budget boundary behavior.
|
|
|
|
#### `fs`
|
|
|
|
Mandatory scenarios:
|
|
|
|
- happy path;
|
|
- invalid path / traversal rejection;
|
|
- unhealthy backend behavior;
|
|
- cleanup of handles and state after reset/unmount when applicable.
|
|
|
|
#### `asset/bank`
|
|
|
|
Mandatory scenarios:
|
|
|
|
- preload behavior;
|
|
- `load / status / commit / cancel`;
|
|
- missing asset handling;
|
|
- invalid slot handling;
|
|
- structural bootstrap failure versus operational runtime failure.
|
|
|
|
#### `firmware`
|
|
|
|
Mandatory scenarios:
|
|
|
|
- cartridge load flow;
|
|
- `AppMode` branch behavior;
|
|
- VM/runtime initialization failure leading to crash path;
|
|
- basic boot-target and state-transition coordination.
|
|
|
|
#### `host-dependent`
|
|
|
|
Mandatory scenarios:
|
|
|
|
- host-only behaviors that require real socket, window, or desktop integration MUST be isolated;
|
|
- deterministic parts of those rules SHOULD exist below the host layer whenever feasible.
|
|
|
|
### Review Contract
|
|
|
|
- Any PR touching a canonical domain MUST review the domain gate explicitly.
|
|
- Domain-scoped coverage inspection MUST be part of review evidence when the domain is materially touched.
|
|
- Domain percentage baselines MAY start at `0`, but they MUST be allowed to rise over time rather than remain permanently undefined.
|
|
|
|
### Organization Contract
|
|
|
|
- Test suites SHOULD move toward domain-oriented ownership.
|
|
- Monolithic suites MAY be split incrementally as touched by real work.
|
|
- This decision does NOT require a one-shot refactor of the entire current test tree.
|
|
|
|
## Impactos
|
|
|
|
### Spec
|
|
|
|
- The testing policy now has a normative direction suitable for a future plan and eventual publication in the repository's process/spec surfaces if desired.
|
|
|
|
### Runtime
|
|
|
|
- Runtime-area changes gain an explicit expectation that domain tests move with behavior changes.
|
|
|
|
### Firmware
|
|
|
|
- Firmware transitions become a first-class governed testing domain rather than an implicit gap.
|
|
|
|
### Host
|
|
|
|
- Host-dependent tests become explicitly governed and justified, rather than ad hoc exceptions.
|
|
|
|
### Tooling / CI
|
|
|
|
- Existing `llvm-cov` and Jenkins integration remain valid.
|
|
- Future plans may add domain reporting, thresholds, or helper commands without changing this baseline decision.
|
|
|
|
## Referencias
|
|
|
|
- `AGD-0001`
|
|
- `Makefile`
|
|
- `files/config/Jenkinsfile`
|
|
- `docs/specs/runtime/12-firmware-pos-and-prometeuhub.md`
|
|
- `crates/console/prometeu-system/src/virtual_machine_runtime/tests.rs`
|
|
- `crates/console/prometeu-system/src/services/fs/virtual_fs.rs`
|
|
- `crates/host/prometeu-host-desktop-winit/src/runner.rs`
|
|
|
|
## Propagacao Necessaria
|
|
|
|
- Create an execution plan that turns the domain model into concrete work items.
|
|
- Define how domain evidence will be gathered during review.
|
|
- Decide whether to add helper commands or scripts for domain-oriented coverage inspection.
|
|
- Prioritize firmware and host-dependent expansion first.
|
|
- Define an incremental split strategy for the monolithic runtime test suite.
|