Runtime Edge Test Plan

This commit is contained in:
2026-04-20 17:57:16 +01:00
parent cdb8dfb2ac
commit cbc94768a5
6 changed files with 793 additions and 36 deletions
@@ -0,0 +1,131 @@
---
id: PLN-0037
ticket: runtime-edge-test-plan
title: Plan - Runtime Edge Coverage Governance Foundation
status: open
created: 2026-04-20
completed:
tags: [tests, coverage, governance, ci]
---
## Briefing
This plan operationalizes the governance layer of `DEC-0020`.
Its goal is to turn the decision into concrete review and CI mechanics without changing the underlying coverage toolchain.
## Decisions de Origem
- `DEC-0020`
## Alvo
Define the repository-level mechanics for:
- global coverage gate continuity;
- domain-scoped coverage evidence during review;
- baseline tracking that can start at `0` per domain and tighten later;
- reviewer-visible rules for when domain tests are mandatory.
## Escopo
- Add repository commands or scripts for domain-oriented coverage evidence collection.
- Add repository-facing documentation that explains the global gate plus domain evidence workflow.
- Align CI/coverage outputs with the new governance model without replacing `llvm-cov`.
## Fora de Escopo
- Expanding firmware tests.
- Refactoring the monolithic runtime suite.
- Introducing hard domain coverage percentages above baseline `0`.
- Replacing Jenkins or `cargo llvm-cov`.
## Plano de Execucao
### Step 1 - Define the domain evidence interface
**What:**
Define how contributors and reviewers gather coverage evidence for `system/runtime`, `fs`, `asset/bank`, `firmware`, and `host-dependent`.
**How:**
Add one repository-visible command path that produces:
- existing global HTML/XML/JSON coverage artifacts;
- a documented procedure for mapping changed files/modules to one of the canonical domains;
- an initial baseline format that records per-domain coverage starting from `0`.
**File(s):**
- `Makefile`
- `scripts/`
- `README.md` or another repository-facing process document if more appropriate
### Step 2 - Add domain evidence helpers
**What:**
Introduce helper commands or scripts that make domain evidence collection reproducible.
**How:**
Prefer simple wrappers over new tooling. For example:
- capture `llvm-cov` JSON/HTML consistently;
- filter or summarize target files/modules for a domain;
- emit review-friendly evidence artifacts without changing the authoritative global gate.
**File(s):**
- `Makefile`
- `scripts/`
### Step 3 - Document the PR acceptance rule
**What:**
Publish the decision’s review contract in a place visible during development.
**How:**
Document that any PR touching a canonical domain must either:
- update/add tests for that domain; or
- justify why the observable contract did not change.
Document that domain coverage evidence is mandatory review input even when domain thresholds are still at baseline `0`.
**File(s):**
- `README.md`
- optional repo process document if the repository has a better canonical location
### Step 4 - Align CI outputs with the governance model
**What:**
Make the existing coverage pipeline clearly compatible with domain review.
**How:**
Keep the current global Jenkins gate unchanged while ensuring produced artifacts are sufficient for domain inspection.
If needed, archive additional summaries or helper outputs generated by the new scripts.
**File(s):**
- `files/config/Jenkinsfile`
- `Makefile`
## Criterios de Aceite
- [ ] The repository exposes a documented way to collect domain-oriented coverage evidence without replacing the global `llvm-cov` gate.
- [ ] Review guidance explicitly states when domain tests are mandatory.
- [ ] Domain baselines are represented in a form that can start at `0` and increase later.
- [ ] Existing global coverage enforcement remains intact.
## Tests / Validacao
### Automated
- Validate all added helper commands locally.
- Run the existing coverage pipeline end to end after the helper changes.
### Evidence
- Show that the global coverage artifacts still build.
- Show an example of domain evidence collection for at least one domain.
## Riscos
- Overdesigning domain evidence collection before real reviewer usage patterns exist.
- Accidentally creating a second competing coverage authority instead of a reviewer aid.
- Spreading policy text across too many documents and making the rule hard to find.
@@ -0,0 +1,136 @@
---
id: PLN-0038
ticket: runtime-edge-test-plan
title: Plan - Firmware and Host-Dependent Domain Coverage Expansion
status: open
created: 2026-04-20
completed:
tags: [tests, firmware, host, coverage]
---
## Briefing
This plan executes the priority expansion areas mandated by `DEC-0020`: `firmware` and `host-dependent`.
The objective is not blanket test growth, but closing the most important domain gaps first.
## Decisions de Origem
- `DEC-0020`
## Alvo
Expand automated evidence for:
- firmware load and state-transition rules;
- `AppMode` branching;
- crash-path transitions from VM/runtime initialization and runtime execution;
- host-dependent desktop/debugger behaviors that legitimately require socket or window integration.
## Escopo
- Add or refine firmware tests in `prometeu-firmware`.
- Add or refine isolated host-dependent tests in the desktop host crate.
- Clarify which host behaviors must remain ignored/integration-style and which deterministic pieces should move below the host.
## Fora de Escopo
- Full test-tree reorganization across all runtime domains.
- Hard domain thresholds above baseline `0`.
- Reworking the public debugger protocol itself unless required by testability.
## Plano de Execucao
### Step 1 - Close firmware state-transition gaps
**What:**
Expand firmware coverage around canonical state behavior, not only isolated happy paths.
**How:**
Add tests around:
- `Reset` to boot-target-driven transitions;
- `LaunchHub` behavior for `BootTarget::Hub` versus `BootTarget::Cartridge`;
- `LoadCartridge` branch behavior for `AppMode::Game` versus `AppMode::System`;
- transition to `AppCrashes` when initialization or runtime execution fails.
**File(s):**
- `crates/console/prometeu-firmware/src/firmware/firmware.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_reset.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_launch_hub.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_load_cartridge.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_game_running.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_crash_screen.rs`
### Step 2 - Delimit host-dependent versus deterministic behavior
**What:**
Make the boundary between true host integration and lower-layer deterministic behavior explicit in tests.
**How:**
Review existing desktop-host tests and classify them:
- tests that truly require socket bind/connect or window integration remain host-dependent and may stay `#[ignore]`;
- deterministic logic should be covered in firmware/runtime/host-internal tests that do not require real integration.
**File(s):**
- `crates/host/prometeu-host-desktop-winit/src/runner.rs`
- `crates/host/prometeu-host-desktop-winit/src/debugger.rs`
### Step 3 - Strengthen the host-dependent suite without widening its scope
**What:**
Keep real integration tests focused on behaviors the lower layers cannot prove.
**How:**
Prioritize cases such as:
- debugger socket open/handshake lifecycle;
- reconnect/refuse-second-connection behavior;
- resume/pause/disconnect coordination when the real host wiring is part of the rule.
Avoid pushing general deterministic state logic into ignored desktop-only tests if it can be covered below the host.
**File(s):**
- `crates/host/prometeu-host-desktop-winit/src/runner.rs`
- `crates/host/prometeu-host-desktop-winit/src/debugger.rs`
### Step 4 - Capture evidence for the two priority domains
**What:**
Produce explicit review evidence showing that `firmware` and `host-dependent` are now governed domains rather than implied gaps.
**How:**
Use the governance helpers from `PLN-0037` if available, or temporary documented evidence if it lands first.
**File(s):**
- firmware and host test targets
- coverage/report artifacts as defined by the governance plan
## Criterios de Aceite
- [ ] Firmware has explicit automated coverage for boot-target transitions, `AppMode` branching, and crash-path transitions.
- [ ] Host-dependent tests are clearly limited to behaviors that genuinely require desktop/socket integration.
- [ ] Deterministic behavior that does not need real host integration is covered below the host when feasible.
- [ ] Review evidence can point to firmware and host-dependent coverage separately from the global aggregate.
## Tests / Validacao
### Unit / Integration
- Run `cargo test -p prometeu-firmware`.
- Run the relevant desktop host tests.
### Host-Dependent
- Run `cargo test -p prometeu-host-desktop-winit --lib -- --ignored`.
### Evidence
- Produce coverage evidence for touched firmware files and host-dependent integration paths.
## Riscos
- Adding host-only tests for behavior that belongs below the host boundary.
- Expanding firmware tests without aligning them to the state machine contract in the specs.
- Overfitting ignored integration tests to desktop timing details.
@@ -0,0 +1,133 @@
---
id: PLN-0039
ticket: runtime-edge-test-plan
title: Plan - Incremental Runtime Domain Suite Split and Baselines
status: open
created: 2026-04-20
completed:
tags: [tests, runtime, fs, asset, organization]
---
## Briefing
This plan turns the current monolithic runtime-edge suite into clearer domain-owned test surfaces over time, while preserving the existing breadth of coverage already present in `prometeu-system`.
## Decisions de Origem
- `DEC-0020`
## Alvo
Establish clearer test ownership and baseline evidence for:
- `system/runtime`
- `fs`
- `asset/bank`
Do this incrementally, without a one-shot refactor of the whole runtime test tree.
## Escopo
- Split the current runtime-edge suite along canonical domain lines as work touches those areas.
- Preserve or improve current behavioral coverage while improving maintainability.
- Establish explicit baseline evidence for the non-priority domains, starting from `0` if needed.
## Fora de Escopo
- Rewriting all tests in one pass.
- Large behavioral refactors unrelated to test structure.
- Introducing strict non-zero domain thresholds immediately.
## Plano de Execucao
### Step 1 - Define the split targets for the existing monolithic suite
**What:**
Map the current `virtual_machine_runtime/tests.rs` coverage into the canonical runtime domains.
**How:**
Create a target decomposition for the current file, separating at least:
- `system/runtime` lifecycle and tick behavior;
- `asset/bank` status-first and preload behavior;
- `fs` and memcard-adjacent runtime cases when they belong to runtime orchestration rather than `VirtualFS` internals.
Do not move files blindly. First define the intended ownership map.
**File(s):**
- `crates/console/prometeu-system/src/virtual_machine_runtime/tests.rs`
- `crates/console/prometeu-system/src/virtual_machine_runtime/`
### Step 2 - Split the runtime suite incrementally
**What:**
Reduce the monolithic test concentration without destabilizing current coverage.
**How:**
As each domain is touched, move or extract tests into smaller modules with domain ownership.
Prefer incremental Rust module splits such as domain-focused test modules over an all-at-once rewrite.
**File(s):**
- `crates/console/prometeu-system/src/virtual_machine_runtime/tests.rs`
- new sibling test modules under `crates/console/prometeu-system/src/virtual_machine_runtime/`
### Step 3 - Preserve and expose `fs` and `asset/bank` baselines
**What:**
Make existing stronger areas explicitly visible in the new governance model.
**How:**
Use:
- `crates/console/prometeu-system/src/services/fs/virtual_fs.rs` for filesystem contract evidence;
- runtime and loader tests for `asset/bank` scenario evidence.
Capture baseline evidence even if the numeric threshold remains `0` initially.
**File(s):**
- `crates/console/prometeu-system/src/services/fs/virtual_fs.rs`
- `crates/console/prometeu-system/src/virtual_machine_runtime/tests.rs`
- `crates/console/prometeu-hal/src/cartridge_loader.rs`
- `crates/console/prometeu-drivers/src/asset.rs`
### Step 4 - Align future review with the new runtime-domain ownership
**What:**
Ensure later PRs can tell which runtime-domain tests should move with the code.
**How:**
Document or encode enough structure that maintainers can tell where to add tests for:
- lifecycle/tick behavior;
- filesystem normalization and runtime cleanup;
- asset/bootstrap/status-first flows.
**File(s):**
- runtime test modules
- repository guidance introduced by `PLN-0037`
## Criterios de Aceite
- [ ] The current monolithic runtime suite has an explicit decomposition into canonical domains.
- [ ] New or moved runtime tests follow domain ownership rather than returning to a single catch-all file.
- [ ] `fs` and `asset/bank` have baseline evidence captured under the governance model.
- [ ] The split is incremental and does not require a one-shot rewrite to start delivering value.
## Tests / Validacao
### Automated
- Run `cargo test -p prometeu-system`.
- Run any focused tests for `prometeu-hal` and `prometeu-drivers` touched during the split.
### Evidence
- Show that the domain-oriented split preserves existing runtime behaviors.
- Produce baseline coverage evidence for `system/runtime`, `fs`, and `asset/bank`.
## Riscos
- Moving tests mechanically without improving domain clarity.
- Creating fragmented helpers that are harder to reuse than the current monolith.
- Treating structural cleanup as more important than preserving behavioral coverage.