Runtime Edge Test Plan

This commit is contained in:
bQUARKz 2026-04-20 17:57:16 +01:00
parent cdb8dfb2ac
commit cbc94768a5
Signed by: bquarkz
SSH Key Fingerprint: SHA256:Z7dgqoglWwoK6j6u4QC87OveEq74WOhFN+gitsxtkf8
6 changed files with 793 additions and 36 deletions

View File

@ -1,10 +1,10 @@
{"type":"meta","next_id":{"DSC":29,"AGD":29,"DEC":20,"PLN":37,"LSN":37,"CLSN":1}}
{"type":"meta","next_id":{"DSC":29,"AGD":29,"DEC":21,"PLN":40,"LSN":37,"CLSN":1}}
{"type":"discussion","id":"DSC-0023","status":"done","ticket":"perf-full-migration-to-atomic-telemetry","title":"Agenda - [PERF] Full Migration to Atomic Telemetry","created_at":"2026-04-10","updated_at":"2026-04-10","tags":["perf","runtime","telemetry"],"agendas":[{"id":"AGD-0021","file":"workflow/agendas/AGD-0021-full-migration-to-atomic-telemetry.md","status":"done","created_at":"2026-04-10","updated_at":"2026-04-10"}],"decisions":[{"id":"DEC-0008","file":"workflow/decisions/DEC-0008-full-migration-to-atomic-telemetry.md","status":"accepted","created_at":"2026-04-10","updated_at":"2026-04-10"}],"plans":[{"id":"PLN-0007","file":"workflow/plans/PLN-0007-full-migration-to-atomic-telemetry.md","status":"done","created_at":"2026-04-10","updated_at":"2026-04-10"}],"lessons":[{"id":"LSN-0028","file":"lessons/DSC-0023-perf-full-migration-to-atomic-telemetry/LSN-0028-converging-to-single-atomic-telemetry-source.md","status":"done","created_at":"2026-04-10","updated_at":"2026-04-10"}]}
{"type":"discussion","id":"DSC-0020","status":"done","ticket":"jenkins-gitea-integration","title":"Jenkins Gitea Integration and Relocation","created_at":"2026-04-07","updated_at":"2026-04-07","tags":["ci","jenkins","gitea"],"agendas":[{"id":"AGD-0018","file":"workflow/agendas/AGD-0018-jenkins-gitea-integration-and-relocation.md","status":"done","created_at":"2026-04-07","updated_at":"2026-04-07"}],"decisions":[{"id":"DEC-0003","file":"workflow/decisions/DEC-0003-jenkins-gitea-strategy.md","status":"accepted","created_at":"2026-04-07","updated_at":"2026-04-07"}],"plans":[{"id":"PLN-0003","file":"workflow/plans/PLN-0003-jenkins-gitea-execution.md","status":"done","created_at":"2026-04-07","updated_at":"2026-04-07"}],"lessons":[{"id":"LSN-0021","file":"lessons/DSC-0020-jenkins-gitea-integration/LSN-0021-jenkins-gitea-integration.md","status":"done","created_at":"2026-04-07","updated_at":"2026-04-07"}]}
{"type":"discussion","id":"DSC-0021","status":"done","ticket":"asset-entry-codec-enum-with-metadata","title":"Asset Entry Codec Enum Contract","created_at":"2026-04-09","updated_at":"2026-04-09","tags":["asset","runtime","codec","metadata"],"agendas":[],"decisions":[],"plans":[],"lessons":[{"id":"LSN-0024","file":"lessons/DSC-0021-asset-entry-codec-enum-contract/LSN-0024-string-on-the-wire-enum-in-runtime.md","status":"done","created_at":"2026-04-09","updated_at":"2026-04-09"}]}
{"type":"discussion","id":"DSC-0022","status":"done","ticket":"tile-bank-vs-glyph-bank-domain-naming","title":"Glyph Bank Domain Naming Contract","created_at":"2026-04-09","updated_at":"2026-04-10","tags":["gfx","runtime","naming","domain-model"],"agendas":[],"decisions":[],"plans":[],"lessons":[{"id":"LSN-0025","file":"lessons/DSC-0022-glyph-bank-domain-naming/LSN-0025-rename-artifact-by-meaning-not-by-token.md","status":"done","created_at":"2026-04-10","updated_at":"2026-04-10"}]}
{"type":"discussion","id":"DSC-0001","status":"done","ticket":"legacy-runtime-learn-import","title":"Import legacy runtime learn into discussion lessons","created_at":"2026-03-27","updated_at":"2026-03-27","tags":["migration","tech-debt"],"agendas":[],"decisions":[],"plans":[],"lessons":[{"id":"LSN-0001","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0001-prometeu-learn-index.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0002","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0002-historical-asset-status-first-fault-and-return-contract.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0003","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0003-historical-audio-status-first-fault-and-return-contract.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0004","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0004-historical-cartridge-boot-protocol-and-manifest-authority.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0005","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0005-historical-game-memcard-slots-surface-and-semantics.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0006","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0006-historical-gfx-status-first-fault-and-return-contract.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0007","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0007-historical-retired-fault-and-input-decisions.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0008","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0008-historical-vm-core-and-assets.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0009","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0009-mental-model-asset-management.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0010","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0010-mental-model-audio.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0011","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0011-mental-model-gfx.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0012","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0012-mental-model-input.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0013","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0013-mental-model-observability-and-debugging.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0014","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0014-mental-model-portability-and-cross-platform.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0015","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0015-mental-model-save-memory-and-memcard.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0016","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0016-mental-model-status-first-and-fault-thinking.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0017","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0017-mental-model-time-and-cycles.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"},{"id":"LSN-0018","file":"lessons/DSC-0001-runtime-learn-legacy-import/LSN-0018-mental-model-touch.md","status":"done","created_at":"2026-03-27","updated_at":"2026-03-27"}]}
{"type":"discussion","id":"DSC-0002","status":"open","ticket":"runtime-edge-test-plan","title":"Agenda - Runtime Edge Test Plan","created_at":"2026-03-27","updated_at":"2026-03-27","tags":[],"agendas":[{"id":"AGD-0001","file":"workflow/agendas/AGD-0001-runtime-edge-test-plan.md","status":"open","created_at":"2026-03-27","updated_at":"2026-03-27"}],"decisions":[],"plans":[],"lessons":[]}
{"type":"discussion","id":"DSC-0002","status":"open","ticket":"runtime-edge-test-plan","title":"Agenda - Runtime Edge Test Plan","created_at":"2026-03-27","updated_at":"2026-04-20","tags":[],"agendas":[{"id":"AGD-0001","file":"workflow/agendas/AGD-0001-runtime-edge-test-plan.md","status":"done","created_at":"2026-03-27","updated_at":"2026-04-20"}],"decisions":[{"id":"DEC-0020","file":"workflow/decisions/DEC-0020-runtime-edge-coverage-governance-by-domain.md","status":"in_progress","created_at":"2026-04-20","updated_at":"2026-04-20","ref_agenda":"AGD-0001"}],"plans":[{"id":"PLN-0037","file":"workflow/plans/PLN-0037-runtime-edge-coverage-governance-foundation.md","status":"open","created_at":"2026-04-20","updated_at":"2026-04-20","ref_decisions":["DEC-0020"]},{"id":"PLN-0038","file":"workflow/plans/PLN-0038-firmware-and-host-dependent-domain-coverage-expansion.md","status":"open","created_at":"2026-04-20","updated_at":"2026-04-20","ref_decisions":["DEC-0020"]},{"id":"PLN-0039","file":"workflow/plans/PLN-0039-incremental-runtime-domain-suite-split-and-baselines.md","status":"open","created_at":"2026-04-20","updated_at":"2026-04-20","ref_decisions":["DEC-0020"]}],"lessons":[]}
{"type":"discussion","id":"DSC-0003","status":"open","ticket":"packed-cartridge-loader-pmc","title":"Agenda - Packed Cartridge Loader PMC","created_at":"2026-03-27","updated_at":"2026-03-27","tags":[],"agendas":[{"id":"AGD-0002","file":"workflow/agendas/AGD-0002-packed-cartridge-loader-pmc.md","status":"open","created_at":"2026-03-27","updated_at":"2026-03-27"}],"decisions":[],"plans":[],"lessons":[]}
{"type":"discussion","id":"DSC-0004","status":"open","ticket":"system-run-cart","title":"Agenda - System Run Cart","created_at":"2026-03-27","updated_at":"2026-03-27","tags":[],"agendas":[{"id":"AGD-0003","file":"workflow/agendas/AGD-0003-system-run-cart.md","status":"open","created_at":"2026-03-27","updated_at":"2026-03-27"}],"decisions":[],"plans":[],"lessons":[]}
{"type":"discussion","id":"DSC-0005","status":"open","ticket":"system-fault-semantics-and-control-surface","title":"Agenda - System Fault Semantics and Control Surface","created_at":"2026-03-27","updated_at":"2026-03-27","tags":[],"agendas":[{"id":"AGD-0004","file":"workflow/agendas/AGD-0004-system-fault-semantics-and-control-surface.md","status":"open","created_at":"2026-03-27","updated_at":"2026-03-27"}],"decisions":[],"plans":[],"lessons":[]}

View File

@ -2,10 +2,10 @@
id: AGD-0001
ticket: runtime-edge-test-plan
title: Agenda - Runtime Edge Test Plan
status: open
status: done
created: 2026-03-27
resolved:
decision:
resolved: 2026-04-20
decision: DEC-0020
tags: []
---
@ -13,36 +13,36 @@ tags: []
## Problema
O core da VM está muito melhor testado do que a borda do runtime.
A borda do runtime deixou de ser "quase sem testes", mas continua sem contrato claro de suficiência por dominio.
O miolo já cobre verificador, scheduler, GC, patching e execução básica. Já a camada de syscalls de sistema, FS, assets, bank e fluxos de firmware ainda tem cobertura desigual e rasa.
Hoje já existe cobertura relevante para runtime edge, `VirtualFS`, asset/preload, status-first surfaces e parte do host. O problema atual não é ausência total de testes, e sim falta de uma régua explícita que diga quando um domínio está aceitavelmente coberto e que tipo de mudança deve obrigar novos testes.
## Dor
- A parte mais sujeita a integração com host é justamente a menos protegida.
- Regressões na borda podem passar com o core inteiro verde.
- O projeto ganha falsa sensação de maturidade porque o núcleo está sólido, mas a superfície pública do runtime ainda pode quebrar com facilidade.
- Sem matriz de testes da borda, futuras PRs tendem a cobrir só o happy path.
- Coverage global do workspace não prova que os domínios de maior risco estão realmente cobertos.
- Regressões de firmware, boot, host integration e superfícies status-first ainda podem passar com números globais "bons".
- Sem gates por domínio, futuras PRs tendem a melhorar percentuais agregados sem fechar cenários obrigatórios.
- A suíte atual está funcional, mas concentrada demais, o que dificulta enxergar lacunas e manter responsabilidade por domínio.
## Alvo da Discussao
Definir um plano de testes para a borda do runtime proporcional ao risco da plataforma.
Definir uma política mínima de cobertura para a borda do runtime proporcional ao risco da plataforma.
Essa agenda deve produzir uma visão clara de:
- quais domínios precisam de testes;
- qual nível de teste pertence a cada domínio;
- quais cenários são mandatórios antes de chamar o runtime de estável.
- quais domínios têm gate próprio;
- quais cenários mínimos cada domínio deve cobrir;
- como coverage entra como evidência operacional, sem substituir a matriz qualitativa;
- que mudanças devem obrigar teste novo antes de merge.
## O Que Precisa Ser Definido
1. Matriz de cobertura por domínio.
- system
- system/runtime
- fs
- log
- asset
- bank
- asset/bank
- firmware transitions
- host-dependent surfaces quando aplicável
2. Tipos de teste.
- unitário puro
@ -50,37 +50,185 @@ Essa agenda deve produzir uma visão clara de:
- testes de firmware
- testes host-dependentes isolados
3. Casos obrigatórios.
Cada domínio precisa definir:
- happy path;
- erro de contrato do guest;
- indisponibilidade/estado inválido do host;
- persistência/cleanup de estado quando aplicável.
3. Gates de aceitação por domínio.
Cada domínio precisa definir ao menos:
- cenários obrigatórios;
- tipo de evidência esperada;
- leitura de coverage por domínio como evidência obrigatória de revisão;
- como o gate por domínio evolui ao longo do tempo sem bloquear a adoção inicial.
4. Critério de severidade.
Que mudanças exigem teste novo obrigatório antes de merge.
Que mudanças exigem teste novo obrigatório antes de merge, mesmo que o percentual global de coverage continue aceitável.
5. Organização.
Onde os testes vivem para nao voltar a concentrar tudo em arquivos gigantes.
Onde os testes vivem para nao voltar a concentrar tudo em arquivos gigantes e para manter ownership claro por domínio.
## O Que Necessita Para Resolver
## O Que Ainda Precisa Ser Fechado
- inventário da cobertura atual;
- definição dos cenários obrigatórios por domínio;
- política mínima para testes ignorados ou dependentes de host;
- decisão sobre helpers compartilhados para fixtures de runtime.
- definição dos domínios canônicos e seus respectivos gates;
- definição dos cenários mínimos obrigatórios por domínio;
- política mínima para testes host-dependentes e `#[ignore]`;
- regra de convivência entre gate global de coverage e leitura segmentada por domínio;
- diretriz de organização para quebrar suites monolíticas de runtime.
## Opcoes
### Opcao A - Gate apenas global de coverage
- **Abordagem:** manter somente o percentual agregado do workspace como barra de aceitação.
- **Pro:** simples de operar e fácil de automatizar no CI.
- **Con:** permite que domínios sensíveis continuem subcobertos enquanto o número global sobe.
- **Manutenibilidade:** boa operacionalmente, ruim como governança de risco.
### Opcao B - Gate percentual rígido por domínio desde o início
- **Abordagem:** exigir percentuais mínimos por domínio já na primeira versão da política.
- **Pro:** força disciplina quantitativa explícita por área.
- **Con:** cria atrito alto de adoção, exige segmentação mais sofisticada e pode travar a convergência inicial por falta de baseline confiável.
- **Manutenibilidade:** média; forte no longo prazo, pesada demais para o ponto atual do projeto.
### Opcao C - Gate global de coverage + gate por domínio com coverage como evidência incremental
- **Abordagem:** manter gate global obrigatório no CI e adotar gate por domínio baseado em cenários mandatórios e leitura de coverage segmentada como evidência. Baselines percentuais por domínio podem começar em `0` e subir com o tempo conforme a suíte amadurece.
- **Pro:** combina disciplina prática imediata com caminho realista para endurecer a barra por domínio sem bloquear a adoção.
- **Con:** exige revisão mais criteriosa, porque a aceitação não fica reduzida a um único número.
- **Manutenibilidade:** alta; permite endurecimento progressivo sem perder governança qualitativa.
## Sugestao / Recomendacao
Fechar esta agenda com a `Opcao C`: uma matriz pequena de domínios canônicos, gate global obrigatório de coverage e gate por domínio baseado em cenários mandatórios, usando leitura segmentada de `llvm-cov` como evidência de revisão e baseline quantitativo incremental.
### Domínios canônicos propostos
1. `system/runtime`
2. `fs`
3. `asset/bank`
4. `firmware`
5. `host-dependent`
### Gates propostos por domínio
#### `system/runtime`
Cenários mínimos:
- `init`
- `reset / cleanup`
- `trap vs panic`
- `pause / breakpoint`
- fronteira de `FRAME_SYNC` e budget
Regra de aceitação:
- qualquer mudança em lifecycle, tick, crash/report, pause/debug flow ou syscall de sistema deve adicionar ou ajustar testes desse domínio;
- coverage global não substitui a obrigação de cobrir o cenário alterado.
#### `fs`
Cenários mínimos:
- happy path;
- path inválido / traversal;
- backend unhealthy;
- cleanup de handles e estado após reset/unmount quando aplicável.
Regra de aceitação:
- qualquer mudança em normalização de path, mount/unmount, handles ou contrato guest-host de filesystem deve adicionar ou ajustar testes desse domínio.
#### `asset/bank`
Cenários mínimos:
- preload;
- `load / status / commit / cancel`;
- asset ausente;
- slot inválido;
- erro estrutural de bootstrap vs erro operacional em runtime.
Regra de aceitação:
- qualquer mudança em cartridge loader, preload, asset bridge, slot telemetry ou semântica status-first deve adicionar ou ajustar testes desse domínio.
#### `firmware`
Cenários mínimos:
- fluxo de load do cart;
- branch por `AppMode`;
- falha de init levando ao crash path;
- coordenação básica de boot target e transição de estado.
Regra de aceitação:
- qualquer mudança em boot flow, cartridge launch, state transition ou crash path deve adicionar ou ajustar testes desse domínio.
Observação:
- este é o domínio com maior lacuna atual e deve ser a prioridade principal de expansão da suíte.
#### `host-dependent`
Cenários mínimos:
- superfícies que dependem de socket, janela ou integração desktop devem ficar isoladas;
- quando possível, a parte determinística da regra deve existir também em teste console/runtime.
Regra de aceitação:
- testes `#[ignore]` são permitidos quando dependem de infraestrutura local real, mas devem trazer justificativa explícita;
- mudanças host-only não podem empurrar comportamento determinístico para testes exclusivamente desktop quando esse comportamento puder ser validado abaixo do host.
Observação:
- `host-dependent` não é um subsistema funcional paralelo a `fs` ou `firmware`; é um corte transversal de testabilidade e execução que governa onde certos testes podem viver e como devem ser justificados.
### Uso de coverage
- `coverage` global via `cargo llvm-cov` continua como gate operacional mínimo do CI;
- leitura segmentada por domínio deve ser usada como evidência obrigatória de revisão quando uma mudança tocar área sensível;
- percentuais por domínio podem começar em baseline `0` e subir progressivamente conforme o projeto estabiliza a segmentação e a matriz de testes;
- percentual global aceitável não dispensa cobertura dos cenários mandatórios do domínio afetado.
### Prioridades sugeridas
1. expandir cobertura de `firmware`;
2. consolidar política de `host-dependent`;
3. quebrar a suíte monolítica de `system/runtime` por responsabilidade, conforme os domínios forem sendo tocados;
4. manter `fs` e `asset/bank` como domínios já relativamente encaminhados, mas ainda sob gate explícito.
### Barra sugerida para PRs
- nenhum PR que altere domínio canônico deve entrar sem tocar testes do domínio afetado, ou sem justificar explicitamente por que o contrato observável não mudou;
- melhoria de coverage agregada é desejável, mas não conta como evidência suficiente sem cenário alinhado ao domínio alterado.
## Perguntas em Aberto
- Nenhuma ambiguidade arquitetural substantiva restante para fechar a política base.
- O endurecimento quantitativo por domínio fica explicitamente como evolução incremental a partir de baseline inicial `0`, sem bloquear a adoção imediata da política.
## Estado Atual Relevante
- O repositório já possui targets de coverage com `cargo llvm-cov` no `Makefile`.
- O CI já aplica gate global mínimo de coverage via Jenkins.
- O runtime já possui cobertura relevante para traps, status-first, assets, composer, audio, memcard e reset.
- `VirtualFS` já cobre parte importante do contrato de path normalization e rejeições antes do backend.
- Ainda falta tratar firmware transitions e superfícies host-dependent como domínios explicitamente governados pela agenda.
## Fora de Escopo
- benchmark/performance suite;
- fuzzing completo de toda a ISA;
- infraestrutura externa de CI além do necessário para os testes definidos.
- troca de ferramenta de coverage;
- perseguir 100% global como objetivo primário;
- transformar coverage percentual em substituto de cenários normativos.
## Critério de Saida Desta Agenda
Pode virar PR quando houver:
- matriz de cobertura aprovada;
- matriz de gates por domínio aprovada;
- lista de cenários mínimos por domínio;
- decisão de organização dos testes;
- barra clara para aceitar novas syscalls e mudanças de runtime.
- política explícita para testes host-dependent / `#[ignore]`;
- uso de `llvm-cov` encaixado como evidência operacional;
- barra clara para aceitar novas syscalls, mudanças de runtime e transições de firmware.

View File

@ -0,0 +1,209 @@
---
id: DEC-0020
ticket: runtime-edge-test-plan
title: Decision - Runtime Edge Coverage Governance by Domain
status: in_progress
created: 2026-04-20
accepted: 2026-04-20
agenda: AGD-0001
plans: [PLN-0037, PLN-0038, PLN-0039]
tags: [tests, coverage, runtime, firmware, host]
---
## Status
In progress.
Derived from `AGD-0001`.
This decision records the normative direction for runtime edge coverage governance.
It has been accepted and now drives one or more execution plans.
## Contexto
PROMETEU no longer has the same problem that originally motivated `AGD-0001`.
The runtime edge is no longer broadly untested. The repository now already contains meaningful coverage for runtime lifecycle paths, `VirtualFS`, asset/preload behavior, status-first surfaces, and part of the desktop host integration.
The remaining problem is governance, not raw test absence.
Global coverage metrics already exist through `cargo llvm-cov`, and CI already enforces a workspace-wide minimum gate. That is useful, but insufficient on its own. A good aggregated number does not guarantee that firmware transitions, boot flow, host-dependent surfaces, or domain-specific status-first contracts are covered at the right level.
The project therefore needs an explicit domain-based acceptance model for tests:
- a small canonical domain set;
- mandatory scenarios per domain;
- a rule for when a change must add or adjust tests;
- a clear relationship between global coverage and domain-oriented evidence.
## Decisao
PROMETEU SHALL govern runtime-edge test sufficiency through a two-layer model:
1. a mandatory global coverage gate enforced in CI; and
2. a domain-based acceptance gate driven by mandatory scenarios plus domain-scoped coverage evidence.
The canonical domains SHALL be:
1. `system/runtime`
2. `fs`
3. `asset/bank`
4. `firmware`
5. `host-dependent`
`host-dependent` SHALL be treated as a transversal execution/testability slice, not as a product subsystem parallel to `fs`, `asset/bank`, or `firmware`.
Domain gates SHALL be scenario-first.
Coverage percentages by domain MAY exist, but they SHALL start from an initial baseline of `0` and SHALL be tightened incrementally over time as segmentation and suites mature.
No PR that changes the observable contract of a canonical domain SHALL be accepted without one of the following:
- tests added or adjusted for that domain; or
- an explicit justification that the observable contract did not change.
Improved aggregate coverage SHALL NOT be accepted as sufficient evidence on its own when the changed behavior belongs to a canonical domain with mandatory scenarios.
## Rationale
### Why not global-only coverage
A single global percentage is easy to automate, but too weak as a risk control.
It allows firmware and host-sensitive regressions to hide behind unrelated line execution elsewhere in the workspace.
### Why not rigid per-domain percentages immediately
The project does not yet have a stable enough domain segmentation baseline to enforce hard per-domain minimums from day one without adding friction disproportionate to the benefit.
Starting from `0` preserves forward motion while still making domain evidence mandatory during review.
### Why scenario-first
The platform risk lies in behavioral contracts:
- lifecycle boundaries,
- boot transitions,
- status-first surfaces,
- filesystem normalization and invalid-state handling,
- preload/bootstrap versus runtime operational failures,
- host-only integration boundaries.
Those are not safely governed by percentages alone.
### Why keep coverage in the model
Coverage remains useful as:
- an operational CI signal;
- a regression detector;
- a way to inspect whether the changed domain actually exercised the intended area.
The correct role of coverage is supporting evidence, not sole authority.
## Invariantes / Contrato
### Global Coverage Contract
- CI MUST keep a workspace-wide coverage gate.
- The current `llvm-cov` based global gate remains valid as the operational baseline.
- Changing the tooling is out of scope for this decision.
### Domain Gate Contract
Each canonical domain MUST define mandatory scenarios and acceptance expectations.
#### `system/runtime`
Mandatory scenarios:
- initialization;
- reset and cleanup;
- trap versus panic behavior;
- pause and breakpoint behavior;
- `FRAME_SYNC` and budget boundary behavior.
#### `fs`
Mandatory scenarios:
- happy path;
- invalid path / traversal rejection;
- unhealthy backend behavior;
- cleanup of handles and state after reset/unmount when applicable.
#### `asset/bank`
Mandatory scenarios:
- preload behavior;
- `load / status / commit / cancel`;
- missing asset handling;
- invalid slot handling;
- structural bootstrap failure versus operational runtime failure.
#### `firmware`
Mandatory scenarios:
- cartridge load flow;
- `AppMode` branch behavior;
- VM/runtime initialization failure leading to crash path;
- basic boot-target and state-transition coordination.
#### `host-dependent`
Mandatory scenarios:
- host-only behaviors that require real socket, window, or desktop integration MUST be isolated;
- deterministic parts of those rules SHOULD exist below the host layer whenever feasible.
### Review Contract
- Any PR touching a canonical domain MUST review the domain gate explicitly.
- Domain-scoped coverage inspection MUST be part of review evidence when the domain is materially touched.
- Domain percentage baselines MAY start at `0`, but they MUST be allowed to rise over time rather than remain permanently undefined.
### Organization Contract
- Test suites SHOULD move toward domain-oriented ownership.
- Monolithic suites MAY be split incrementally as touched by real work.
- This decision does NOT require a one-shot refactor of the entire current test tree.
## Impactos
### Spec
- The testing policy now has a normative direction suitable for a future plan and eventual publication in the repository's process/spec surfaces if desired.
### Runtime
- Runtime-area changes gain an explicit expectation that domain tests move with behavior changes.
### Firmware
- Firmware transitions become a first-class governed testing domain rather than an implicit gap.
### Host
- Host-dependent tests become explicitly governed and justified, rather than ad hoc exceptions.
### Tooling / CI
- Existing `llvm-cov` and Jenkins integration remain valid.
- Future plans may add domain reporting, thresholds, or helper commands without changing this baseline decision.
## Referencias
- `AGD-0001`
- `Makefile`
- `files/config/Jenkinsfile`
- `docs/specs/runtime/12-firmware-pos-and-prometeuhub.md`
- `crates/console/prometeu-system/src/virtual_machine_runtime/tests.rs`
- `crates/console/prometeu-system/src/services/fs/virtual_fs.rs`
- `crates/host/prometeu-host-desktop-winit/src/runner.rs`
## Propagacao Necessaria
- Create an execution plan that turns the domain model into concrete work items.
- Define how domain evidence will be gathered during review.
- Decide whether to add helper commands or scripts for domain-oriented coverage inspection.
- Prioritize firmware and host-dependent expansion first.
- Define an incremental split strategy for the monolithic runtime test suite.

View File

@ -0,0 +1,131 @@
---
id: PLN-0037
ticket: runtime-edge-test-plan
title: Plan - Runtime Edge Coverage Governance Foundation
status: open
created: 2026-04-20
completed:
tags: [tests, coverage, governance, ci]
---
## Briefing
This plan operationalizes the governance layer of `DEC-0020`.
Its goal is to turn the decision into concrete review and CI mechanics without changing the underlying coverage toolchain.
## Decisions de Origem
- `DEC-0020`
## Alvo
Define the repository-level mechanics for:
- global coverage gate continuity;
- domain-scoped coverage evidence during review;
- baseline tracking that can start at `0` per domain and tighten later;
- reviewer-visible rules for when domain tests are mandatory.
## Escopo
- Add repository commands or scripts for domain-oriented coverage evidence collection.
- Add repository-facing documentation that explains the global gate plus domain evidence workflow.
- Align CI/coverage outputs with the new governance model without replacing `llvm-cov`.
## Fora de Escopo
- Expanding firmware tests.
- Refactoring the monolithic runtime suite.
- Introducing hard domain coverage percentages above baseline `0`.
- Replacing Jenkins or `cargo llvm-cov`.
## Plano de Execucao
### Step 1 - Define the domain evidence interface
**What:**
Define how contributors and reviewers gather coverage evidence for `system/runtime`, `fs`, `asset/bank`, `firmware`, and `host-dependent`.
**How:**
Add one repository-visible command path that produces:
- existing global HTML/XML/JSON coverage artifacts;
- a documented procedure for mapping changed files/modules to one of the canonical domains;
- an initial baseline format that records per-domain coverage starting from `0`.
**File(s):**
- `Makefile`
- `scripts/`
- `README.md` or another repository-facing process document if more appropriate
### Step 2 - Add domain evidence helpers
**What:**
Introduce helper commands or scripts that make domain evidence collection reproducible.
**How:**
Prefer simple wrappers over new tooling. For example:
- capture `llvm-cov` JSON/HTML consistently;
- filter or summarize target files/modules for a domain;
- emit review-friendly evidence artifacts without changing the authoritative global gate.
**File(s):**
- `Makefile`
- `scripts/`
### Step 3 - Document the PR acceptance rule
**What:**
Publish the decisions review contract in a place visible during development.
**How:**
Document that any PR touching a canonical domain must either:
- update/add tests for that domain; or
- justify why the observable contract did not change.
Document that domain coverage evidence is mandatory review input even when domain thresholds are still at baseline `0`.
**File(s):**
- `README.md`
- optional repo process document if the repository has a better canonical location
### Step 4 - Align CI outputs with the governance model
**What:**
Make the existing coverage pipeline clearly compatible with domain review.
**How:**
Keep the current global Jenkins gate unchanged while ensuring produced artifacts are sufficient for domain inspection.
If needed, archive additional summaries or helper outputs generated by the new scripts.
**File(s):**
- `files/config/Jenkinsfile`
- `Makefile`
## Criterios de Aceite
- [ ] The repository exposes a documented way to collect domain-oriented coverage evidence without replacing the global `llvm-cov` gate.
- [ ] Review guidance explicitly states when domain tests are mandatory.
- [ ] Domain baselines are represented in a form that can start at `0` and increase later.
- [ ] Existing global coverage enforcement remains intact.
## Tests / Validacao
### Automated
- Validate all added helper commands locally.
- Run the existing coverage pipeline end to end after the helper changes.
### Evidence
- Show that the global coverage artifacts still build.
- Show an example of domain evidence collection for at least one domain.
## Riscos
- Overdesigning domain evidence collection before real reviewer usage patterns exist.
- Accidentally creating a second competing coverage authority instead of a reviewer aid.
- Spreading policy text across too many documents and making the rule hard to find.

View File

@ -0,0 +1,136 @@
---
id: PLN-0038
ticket: runtime-edge-test-plan
title: Plan - Firmware and Host-Dependent Domain Coverage Expansion
status: open
created: 2026-04-20
completed:
tags: [tests, firmware, host, coverage]
---
## Briefing
This plan executes the priority expansion areas mandated by `DEC-0020`: `firmware` and `host-dependent`.
The objective is not blanket test growth, but closing the most important domain gaps first.
## Decisions de Origem
- `DEC-0020`
## Alvo
Expand automated evidence for:
- firmware load and state-transition rules;
- `AppMode` branching;
- crash-path transitions from VM/runtime initialization and runtime execution;
- host-dependent desktop/debugger behaviors that legitimately require socket or window integration.
## Escopo
- Add or refine firmware tests in `prometeu-firmware`.
- Add or refine isolated host-dependent tests in the desktop host crate.
- Clarify which host behaviors must remain ignored/integration-style and which deterministic pieces should move below the host.
## Fora de Escopo
- Full test-tree reorganization across all runtime domains.
- Hard domain thresholds above baseline `0`.
- Reworking the public debugger protocol itself unless required by testability.
## Plano de Execucao
### Step 1 - Close firmware state-transition gaps
**What:**
Expand firmware coverage around canonical state behavior, not only isolated happy paths.
**How:**
Add tests around:
- `Reset` to boot-target-driven transitions;
- `LaunchHub` behavior for `BootTarget::Hub` versus `BootTarget::Cartridge`;
- `LoadCartridge` branch behavior for `AppMode::Game` versus `AppMode::System`;
- transition to `AppCrashes` when initialization or runtime execution fails.
**File(s):**
- `crates/console/prometeu-firmware/src/firmware/firmware.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_reset.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_launch_hub.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_load_cartridge.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_game_running.rs`
- `crates/console/prometeu-firmware/src/firmware/firmware_step_crash_screen.rs`
### Step 2 - Delimit host-dependent versus deterministic behavior
**What:**
Make the boundary between true host integration and lower-layer deterministic behavior explicit in tests.
**How:**
Review existing desktop-host tests and classify them:
- tests that truly require socket bind/connect or window integration remain host-dependent and may stay `#[ignore]`;
- deterministic logic should be covered in firmware/runtime/host-internal tests that do not require real integration.
**File(s):**
- `crates/host/prometeu-host-desktop-winit/src/runner.rs`
- `crates/host/prometeu-host-desktop-winit/src/debugger.rs`
### Step 3 - Strengthen the host-dependent suite without widening its scope
**What:**
Keep real integration tests focused on behaviors the lower layers cannot prove.
**How:**
Prioritize cases such as:
- debugger socket open/handshake lifecycle;
- reconnect/refuse-second-connection behavior;
- resume/pause/disconnect coordination when the real host wiring is part of the rule.
Avoid pushing general deterministic state logic into ignored desktop-only tests if it can be covered below the host.
**File(s):**
- `crates/host/prometeu-host-desktop-winit/src/runner.rs`
- `crates/host/prometeu-host-desktop-winit/src/debugger.rs`
### Step 4 - Capture evidence for the two priority domains
**What:**
Produce explicit review evidence showing that `firmware` and `host-dependent` are now governed domains rather than implied gaps.
**How:**
Use the governance helpers from `PLN-0037` if available, or temporary documented evidence if it lands first.
**File(s):**
- firmware and host test targets
- coverage/report artifacts as defined by the governance plan
## Criterios de Aceite
- [ ] Firmware has explicit automated coverage for boot-target transitions, `AppMode` branching, and crash-path transitions.
- [ ] Host-dependent tests are clearly limited to behaviors that genuinely require desktop/socket integration.
- [ ] Deterministic behavior that does not need real host integration is covered below the host when feasible.
- [ ] Review evidence can point to firmware and host-dependent coverage separately from the global aggregate.
## Tests / Validacao
### Unit / Integration
- Run `cargo test -p prometeu-firmware`.
- Run the relevant desktop host tests.
### Host-Dependent
- Run `cargo test -p prometeu-host-desktop-winit --lib -- --ignored`.
### Evidence
- Produce coverage evidence for touched firmware files and host-dependent integration paths.
## Riscos
- Adding host-only tests for behavior that belongs below the host boundary.
- Expanding firmware tests without aligning them to the state machine contract in the specs.
- Overfitting ignored integration tests to desktop timing details.

View File

@ -0,0 +1,133 @@
---
id: PLN-0039
ticket: runtime-edge-test-plan
title: Plan - Incremental Runtime Domain Suite Split and Baselines
status: open
created: 2026-04-20
completed:
tags: [tests, runtime, fs, asset, organization]
---
## Briefing
This plan turns the current monolithic runtime-edge suite into clearer domain-owned test surfaces over time, while preserving the existing breadth of coverage already present in `prometeu-system`.
## Decisions de Origem
- `DEC-0020`
## Alvo
Establish clearer test ownership and baseline evidence for:
- `system/runtime`
- `fs`
- `asset/bank`
Do this incrementally, without a one-shot refactor of the whole runtime test tree.
## Escopo
- Split the current runtime-edge suite along canonical domain lines as work touches those areas.
- Preserve or improve current behavioral coverage while improving maintainability.
- Establish explicit baseline evidence for the non-priority domains, starting from `0` if needed.
## Fora de Escopo
- Rewriting all tests in one pass.
- Large behavioral refactors unrelated to test structure.
- Introducing strict non-zero domain thresholds immediately.
## Plano de Execucao
### Step 1 - Define the split targets for the existing monolithic suite
**What:**
Map the current `virtual_machine_runtime/tests.rs` coverage into the canonical runtime domains.
**How:**
Create a target decomposition for the current file, separating at least:
- `system/runtime` lifecycle and tick behavior;
- `asset/bank` status-first and preload behavior;
- `fs` and memcard-adjacent runtime cases when they belong to runtime orchestration rather than `VirtualFS` internals.
Do not move files blindly. First define the intended ownership map.
**File(s):**
- `crates/console/prometeu-system/src/virtual_machine_runtime/tests.rs`
- `crates/console/prometeu-system/src/virtual_machine_runtime/`
### Step 2 - Split the runtime suite incrementally
**What:**
Reduce the monolithic test concentration without destabilizing current coverage.
**How:**
As each domain is touched, move or extract tests into smaller modules with domain ownership.
Prefer incremental Rust module splits such as domain-focused test modules over an all-at-once rewrite.
**File(s):**
- `crates/console/prometeu-system/src/virtual_machine_runtime/tests.rs`
- new sibling test modules under `crates/console/prometeu-system/src/virtual_machine_runtime/`
### Step 3 - Preserve and expose `fs` and `asset/bank` baselines
**What:**
Make existing stronger areas explicitly visible in the new governance model.
**How:**
Use:
- `crates/console/prometeu-system/src/services/fs/virtual_fs.rs` for filesystem contract evidence;
- runtime and loader tests for `asset/bank` scenario evidence.
Capture baseline evidence even if the numeric threshold remains `0` initially.
**File(s):**
- `crates/console/prometeu-system/src/services/fs/virtual_fs.rs`
- `crates/console/prometeu-system/src/virtual_machine_runtime/tests.rs`
- `crates/console/prometeu-hal/src/cartridge_loader.rs`
- `crates/console/prometeu-drivers/src/asset.rs`
### Step 4 - Align future review with the new runtime-domain ownership
**What:**
Ensure later PRs can tell which runtime-domain tests should move with the code.
**How:**
Document or encode enough structure that maintainers can tell where to add tests for:
- lifecycle/tick behavior;
- filesystem normalization and runtime cleanup;
- asset/bootstrap/status-first flows.
**File(s):**
- runtime test modules
- repository guidance introduced by `PLN-0037`
## Criterios de Aceite
- [ ] The current monolithic runtime suite has an explicit decomposition into canonical domains.
- [ ] New or moved runtime tests follow domain ownership rather than returning to a single catch-all file.
- [ ] `fs` and `asset/bank` have baseline evidence captured under the governance model.
- [ ] The split is incremental and does not require a one-shot rewrite to start delivering value.
## Tests / Validacao
### Automated
- Run `cargo test -p prometeu-system`.
- Run any focused tests for `prometeu-hal` and `prometeu-drivers` touched during the split.
### Evidence
- Show that the domain-oriented split preserves existing runtime behaviors.
- Produce baseline coverage evidence for `system/runtime`, `fs`, and `asset/bank`.
## Riscos
- Moving tests mechanically without improving domain clarity.
- Creating fragmented helpers that are harder to reuse than the current monolith.
- Treating structural cleanup as more important than preserving behavioral coverage.