From d9911a63dc3e1e62bcedf314a9ba9f59f3274f39 Mon Sep 17 00:00:00 2001 From: bQUARKz Date: Fri, 20 Mar 2026 07:26:14 +0000 Subject: [PATCH] packer (WIP) --- ...zard Studio Decision Propagation Agenda.md | 385 ++++++++++-------- docs/packer/agendas/README.md | 2 + ...ile Bank Packing Materialization Agenda.md | 174 ++++++++ ...izard Pack Execution Semantics Decision.md | 218 ++++++++++ docs/packer/decisions/README.md | 2 + 5 files changed, 618 insertions(+), 163 deletions(-) create mode 100644 docs/packer/agendas/Tile Bank Packing Materialization Agenda.md create mode 100644 docs/packer/decisions/Pack Wizard Pack Execution Semantics Decision.md diff --git a/docs/packer/agendas/Pack Wizard Studio Decision Propagation Agenda.md b/docs/packer/agendas/Pack Wizard Studio Decision Propagation Agenda.md index b91a4945..246c716e 100644 --- a/docs/packer/agendas/Pack Wizard Studio Decision Propagation Agenda.md +++ b/docs/packer/agendas/Pack Wizard Studio Decision Propagation Agenda.md @@ -1,236 +1,295 @@ # Pack Wizard Studio Decision Propagation Agenda -## Status - -Open +Status: Open +Domain Owner: `docs/packer` +Cross-Domain Impact: `docs/studio` ## Purpose -Converge the packer-side discussion required to propagate the Studio `Pack Wizard` decision into packer-owned public contracts. +Converge the packer-side discussion that still remains open after the first `Pack Wizard` propagation wave: the actual `packing` operation. -The immediate goal is not to implement packing behavior yet. -The goal is to close how the packer should expose summary, validation, execution, progress, and result contracts so Studio can remain a shell over packer-owned semantics. +`summary` and `validation` are already closed enough to implement and consume. +What is still unresolved is how `pack execution` should behave as a packer-owned operation when Studio enters the `Packing` and `Result` phases. -This agenda is driven by the Studio-side decision: +This agenda is therefore about execution semantics, artifact emission, progress visibility, and failure boundaries for `packWorkspace(...)`. -- [`../../studio/decisions/Pack Wizard in Assets Workspace Decision.md`](../../studio/decisions/Pack%20Wizard%20in%20Assets%20Workspace%20Decision.md) +## Problem + +The repository already closed the Studio shell boundary and the packer-side preflight boundary, but the actual pack execution contract is still underspecified. + +Today the gap is concrete: + +1. Studio already depends on a four-phase wizard: + `Summary`, `Validation`, `Packing`, `Result`; +2. the public packer API already exposes `packWorkspace(...)`; +3. `prometeu-packer-v1` still throws `UnsupportedOperationException` for that operation; +4. the specs define `assets.pa` authority and deterministic ordering, but they do not yet define the operational semantics for emitting that artifact through the Studio-facing service boundary. + +Without this agenda, implementation risks falling into one of these traps: + +- a direct file-write implementation that leaks partial artifacts; +- a Studio-visible progress model that is too vague to support the wizard honestly; +- an execution path that does not clearly separate validation failure from emission failure; +- or a result contract that is too poor for the `Result` phase even if the pack technically succeeds. ## Context -The new Studio decision already closes several boundary rules: +The upstream state is now clear enough to isolate the remaining question. -- Studio is a modal shell only; -- packer owns `summary`, `validation`, `pack execution`, progress, and result semantics; -- the wizard has four explicit phases: - `Summary`, `Validation`, `Packing`, `Result`; -- the pack set is the current `registered + included in build` set; -- blocking diagnostics in that set fail validation; -- the first wave is non-cancelable; -- `assets.pa` remains the canonical emitted artifact. +Already closed: -What remains open on the packer side is the exact API and DTO shape that should support this flow cleanly. +- Studio is a shell over packer-owned semantics: + [`../../studio/decisions/Pack Wizard in Assets Workspace Decision.md`](../../studio/decisions/Pack%20Wizard%20in%20Assets%20Workspace%20Decision.md) +- packer-side `summary` and `validation` contracts are accepted: + [`../decisions/Pack Wizard Summary and Validation Contracts Decision.md`](../decisions/Pack%20Wizard%20Summary%20and%20Validation%20Contracts%20Decision.md) +- runtime artifact authority is already normative: + [`../specs/4. Build Artifacts and Deterministic Packing Specification.md`](../specs/4.%20Build%20Artifacts%20and%20Deterministic%20Packing%20Specification.md) +- event causality and serialized operation rules are already normative: + [`../specs/5. Diagnostics, Operations, and Studio Integration Specification.md`](../specs/5.%20Diagnostics,%20Operations,%20and%20Studio%20Integration%20Specification.md) -Important current constraints: +Current code context: -1. `PackerWorkspaceService` already exposes multiple operation-specific request/response pairs rather than one giant command surface; -2. the public API already has stable patterns for: - - `...Request` - - `...Result` - - `PackerOperationStatus` - - event-driven progress through `PackerEvent` and `PackerProgress`; -3. the build artifact spec already defines `assets.pa` authority and deterministic `asset_table` rules; -4. the Studio decision explicitly requires three distinct calls: - `summary`, `validation`, and `pack execution`. +- `getPackWorkspaceSummary(...)` already exists in `FileSystemPackerWorkspaceService`; +- `validatePackWorkspace(...)` already exists in `FileSystemPackerWorkspaceService`; +- `packWorkspace(...)` exists in the public API but is not implemented in `prometeu-packer-v1`; +- the current `PackWorkspaceResult` shape is minimal, but the operational meaning of its fields is not fully closed. -This means the packer should extend an existing contract style where possible instead of inventing a one-off build wizard protocol. +That means the current decision gap is no longer about whether packing should exist. +It is about what guarantees `packWorkspace(...)` must provide when it starts emitting runtime-facing artifacts. ## Options -### Option A - Three new public operations with dedicated request/result DTOs +### Option A - Direct in-place pack with minimal lifecycle semantics -Add three explicit public operations to `PackerWorkspaceService`: +Implement `packWorkspace(...)` as one serialized write-lane operation that: -1. summary -2. validation -3. pack execution +- optionally reruns the validation gate; +- emits `assets.pa` and companion artifacts directly into their final locations; +- publishes a small number of lifecycle events; +- returns one final result DTO with success or failure. -Each operation gets dedicated request/result messages, with nested DTOs where needed. +### Option B - Staged pack transaction with explicit lifecycle phases -Progress continues through the existing event path. +Implement `packWorkspace(...)` as one serialized packer-owned execution flow with explicit internal phases: -### Option B - One `pack` command plus phase-rich result objects +1. gate +2. plan/materialize +3. stage emit +4. commit/promote +5. finalize result -Expose one public `pack` call and let Studio infer phases from intermediate result payloads and packer events. +The operation still appears to Studio as one `packWorkspace(...)` call, but the packer gives stronger semantics: -Summary and validation would become either optional preflight flags or internal behavior hidden behind one command. +- validation is rechecked at execution start; +- emitted files are staged before final promotion; +- final artifact visibility changes only at commit; +- progress events map to meaningful lifecycle boundaries; +- the final result distinguishes validation, emission, and persistence failure classes. -### Option C - Composite build session contract +### Option C - Detached build session abstraction -Create a higher-level `PackSession` or similar build-session API that owns summary, validation, execution, and result under one larger protocol object. +Introduce a larger `PackSession`-style model or resumable build-session protocol for packing. -Studio would talk to that session abstraction rather than to simple operation methods. +Studio would create a session, observe step state, and then explicitly finalize or close it. ## Tradeoffs -- Option A matches the existing public API style best and aligns with the Studio decision directly. -- Option A also makes each phase testable and reusable without forcing execution. -- Option A increases DTO count, but the surface stays explicit and predictable. -- Option B reduces top-level API additions, but it hides too much semantic structure from Studio and pushes phase reconstruction into the host. -- Option B also weakens the explicit preflight/value of `summary` and `validation`. -- Option C could become elegant later if the build system grows more modes, but it is too heavy for the current wave and risks introducing a second API style beside the existing `PackerWorkspaceService` request/result pattern. +- Option A is the smallest implementation step, but it leaves too many failure and visibility rules implicit. +- Option A also makes it easier to leak partially written artifacts into final paths if the process fails mid-pack. +- Option B fits the existing service boundary while still making artifact visibility and progress semantics explicit. +- Option B also aligns well with the current specs: + serialized writes, causal events, deterministic outputs, and packer-owned truth. +- Option B adds some implementation structure, but that structure corresponds to real operational risk and is not ceremonial. +- Option C could become attractive later if the build system needs cancellation, resume, remote workers, or richer host orchestration. +- Option C is too heavy for the current repository state and would introduce a second interaction model before the first one is stable. ## Recommendation -Adopt `Option A`. +Adopt `Option B`. -The packer should expose three explicit public operations and keep them aligned with the existing API style. +The packer should keep `packWorkspace(...)` as a single public service operation, but internally define and honor explicit execution phases with staged emission and an atomic final visibility boundary. -Recommended first-wave contract direction: +### Execution Boundary Recommendation -1. add `PackWorkspaceSummaryRequest` / `PackWorkspaceSummaryResult`; -2. add `ValidatePackWorkspaceRequest` / `ValidatePackWorkspaceResult`; -3. add `PackWorkspaceRequest` / `PackWorkspaceResult`; -4. add focused DTOs for: - - pack summary - - per-asset validation status - - validation aggregate - - emitted artifact summary - - final pack result summary -5. reuse existing public primitives where they already fit: - - `PackerProjectContext` - - `PackerOperationStatus` - - `PackerDiagnosticDTO` - - `PackerProgress` - - existing event publication path for operation progress. - -### Boundary Recommendation - -The packer contract should state clearly: - -- summary is read-oriented and non-mutating; -- validation is dry-run-like and non-emitting; -- pack execution is the only operation that actually emits build artifacts. - -### Validation Recommendation - -The validation contract should be build-set-scoped and asset-oriented. +`packWorkspace(...)` should be the only first-wave operation that emits build artifacts. Rules: -- validation runs on the same pack set used by summary: - `registered + included in build`; -- validation is read-only, non-mutating, and snapshot-backed; -- validation may reuse deep inspection or reconcile-grade logic internally; -- a future deep-sync step may run before validation in the Studio flow, but validation itself still evaluates the packer snapshot; -- only diagnostics marked `blocking` fail validation in the first wave; -- if the returned per-asset list contains no blocking diagnostics, validation is green; -- if the returned per-asset list contains any blocking diagnostics, validation is red and packing must not begin. +- it runs through the packer-owned serialized write lane for the project; +- it evaluates the active pack set as `registered + included in build`; +- it must perform an internal gate check before emitting artifacts, even if Studio already ran `validation`; +- it must freeze the execution input set before materialization begins; +- a failing gate returns a terminal result without creating final artifacts. -The first-wave validation payload should prioritize the per-asset diagnostic list over aggregate data. +This avoids trusting host orchestration timing as if it were a correctness guarantee. -Each validation item should expose: +### Frozen Snapshot Recommendation -- asset id; -- asset name; -- asset path; -- `lastModified`; -- diagnostics. +`packWorkspace(...)` should not pack from live filesystem rereads after execution begins. -The diagnostics in this contract are blocking diagnostics only in the first wave. +Recommended first-wave baseline: -Aggregate data is secondary here. -It may still exist, but it should not be treated as the primary UI payload because the Studio can derive aggregate views from the returned per-asset list if needed. +- the packer creates a frozen pack-execution snapshot before materialization starts; +- that frozen snapshot is derived from the current runtime snapshot but enriches build-relevant walk files with in-memory content bytes; +- content bytes are needed only for: + - registered assets included in build; + - artifact files that actually participate in the current build output; +- non-build assets and non-selected artifact files do not need in-memory content in the pack-execution snapshot; +- the current runtime snapshot model should expose file bytes as optional rather than introducing a second snapshot model in the first wave; +- `PackerRuntimeWalkFile` should therefore carry optional content bytes so the packer can freeze build-relevant inputs without forcing all runtime-backed reads to retain content unnecessarily; +- the family walkers may continue to produce full file probes with bytes loaded during probe processing; +- the variation for runtime versus packing should happen when probe results are exported into the runtime walk projection; +- walker materialization must therefore gain clearer execution context/config so build-focused flows can request content bytes intentionally instead of coupling that behavior to every normal read path. -Ordering is not a packer concern for the first-wave validation payload. -The Studio may sort the returned items as it sees fit. +This preserves precision without turning the general runtime snapshot into a blanket in-memory mirror of the workspace. -### Execution Recommendation +### Walker Policy Recommendation -The execution contract should keep final result summary separate from progress. +The current walker baseline should remain family-oriented and generalist. -Progress should continue to flow through the existing packer event system rather than being duplicated into polling-oriented response DTOs for the first wave. +Recommended first-wave baseline: -The final result should describe: +- family walkers continue to discover files relevant to the asset family; +- that family-oriented discovery remains the normal runtime-backed mode; +- packing adds one extra selection layer instead of redefining walker discovery itself; +- the extra selection layer filters the discovered probe results down to the files that actually participate in the current build output; +- the same export layer also decides whether optional content bytes are injected into `PackerRuntimeWalkFile`. + +This means the first-wave split should be: + +1. family-relevant probe discovery in the walkers; +2. export/materialization policy for runtime projection; +3. packing-specific projection filter over discovered probe results; +4. optional byte injection only when the export policy requests it. + +This is intentionally conservative. +It keeps the walkers on known ground and moves the packing-specific behavior into one explicit projection/materialization policy layer. + +Recommended config shape: + +```java +public record PackerRuntimeMaterializationConfig( + PackerWalkMode mode, + Predicate projectionFilter) { + + public static PackerRuntimeMaterializationConfig runtimeDefault() { + return new PackerRuntimeMaterializationConfig(PackerWalkMode.RUNTIME, probe -> true); + } + + public static PackerRuntimeMaterializationConfig packingBuild() { + return new PackerRuntimeMaterializationConfig( + PackerWalkMode.PACKING, + PackerProbePolicies::isIncludedInCurrentBuild); + } +} +``` + +With this baseline: + +- `PackerWalkMode.RUNTIME` means runtime-backed general projection and suppresses optional content bytes during export; +- `PackerWalkMode.PACKING` means packing-focused projection and injects optional content bytes during export; +- `projectionFilter` decides which discovered probe results actually survive into `PackerRuntimeWalkFile`; +- the first-wave packing filter should keep only current-build probe results; +- byte injection behavior is derived from `mode` rather than from a second explicit boolean in the first wave. + +### Emission Recommendation + +Artifact emission should be staged before promotion into final visible locations. + +Recommended first-wave baseline: + +- materialize `assets.pa` and companion artifacts in a staging location owned by the packer operation; +- use `build/.staging//` as the first-wave staging location; +- `` should be one random 20-character alphanumeric identifier; +- if a staging directory collision happens for that identifier, the packer may overwrite that staging directory rather than adding a second collision-resolution protocol in the first wave; +- treat staging output as non-authoritative until commit; +- promote staged files into final `build/` locations only after the operation reaches a coherent success point; +- if emission fails before commit, final artifact locations must remain unchanged from the caller's point of view. + +This is the safest baseline for a filesystem-first packer. +It keeps durable visibility aligned with a packer-owned commit boundary rather than with incidental file-write timing. + +### Progress Recommendation + +Progress should continue to flow through the existing packer event path rather than through a polling-specific result shape. + +The first-wave lifecycle should be observable through summaries and progress updates equivalent to: + +1. `packing_started` +2. `packing_validating` +3. `packing_materializing` +4. `packing_emitting` +5. `packing_finalizing` +6. terminal success or failure + +The exact event kind names can still be finalized in the decision, but the meaning must be explicit enough for Studio to render the `Packing` phase honestly without fake timers or inferred state. + +### Result Recommendation + +The final `PackWorkspaceResult` should stay compact but semantically explicit. + +The primary result summary should expose at least: - final status; -- emitted canonical artifact `assets.pa`; -- emitted companion artifacts where available; -- total assets packed; -- duration or equivalent timing summary when available. +- whether validation failed or execution actually began; +- canonical build-relative `assets.pa` artifact reference; +- build-relative companion artifact references when emitted; +- total packed asset count; +- elapsed time in milliseconds. -### Summary Recommendation +The first-wave `Result` phase in Studio does not need a full build report dump, but it does need enough structure to distinguish successful pack, blocked pack, and failed pack after execution started. -The first-wave summary contract is now closed as a fast snapshot-backed preflight view. +### Failure Recommendation -Rules: +The packer should distinguish at least three terminal classes: -- summary is read-only and non-mutating; -- summary must return quickly from packer-owned snapshot state rather than from a deep recomputation pass; -- packer is the only source of truth for the summary; -- Studio must not reconstruct the summary from local navigator state; -- unregistered assets and excluded assets do not appear as explicit summary items; -- the summary does not anticipate emitted artifacts, companion artifacts, or `assets.pa` details in the first wave. +1. validation gate failure before emission; +2. execution or materialization failure during pack generation; +3. persistence or promotion failure while making artifacts final. -The first-wave summary shape should contain: +These classes matter because they map directly to the Studio `Result` phase already decided upstream. -1. one aggregate count for the current pack set; -2. one per-asset list for the current pack set. +### Build Output Recommendation -The aggregate is intentionally minimal in the first wave: +The packer publication boundary for this flow should remain `build/`. -- total included asset count is enough for now. +Recommended first-wave baseline: -Each per-asset summary entry should include at least: +- `packWorkspace(...)` publishes final outputs to `build/`; +- the canonical runtime-facing artifact is `build/assets.pa`; +- the baseline companion artifacts are: + - `build/asset_table.json` + - `build/preload.json` + - `build/asset_table_metadata.json` +- a future shipper may consume those packer outputs together with `program.pbx`; +- the shipper is mentioned here only to keep the boundary explicit, not because it participates in `packWorkspace(...)`. -- asset name; -- asset family; -- min artifact count for that asset; -- max artifact count for that asset; -- `lastModified` as a `long`. - -`lastModified` is explicitly useful even if the packer cannot yet provide the final real value for every asset in the first implementation wave. -The contract may expose `0` temporarily, but the field should already exist so the Studio can depend on the shape and later sort against real values. - -This per-asset list is the correct place for the developer-facing graphical occupancy preview per asset in the Studio shell. -The aggregate should remain intentionally small. +This keeps the current domain boundary intact: +the packer publishes build artifacts and the shipper assembles cartridges. ## Open Questions -1. Should the final pack result expose artifact paths, artifact labels, or both? -2. Should the pack execution result carry timing directly, or should timing remain event-derived metadata? +None for the first-wave execution shape. +The remaining work is to convert this agenda into a decision and then propagate the agreed materialization policy into code and specs. -## Resolved In This Agenda Wave +## Recommendation Summary -The following points are now closed for `summary`: +This agenda recommends closing the first-wave `packing` contract as: -1. summary is read-only, non-mutating, and snapshot-backed; -2. packer is the sole source of truth for summary data; -3. unregistered and excluded assets do not appear as explicit summary entries; -4. first-wave summary includes one aggregate count only: - total included asset count; -5. first-wave summary also includes a per-asset list for the current pack set; -6. each per-asset summary entry must expose: - asset name, asset family, min artifact count, max artifact count, and `lastModified`; -7. `lastModified` may temporarily be `0` until full support exists, but the field should already exist in the public contract; -8. summary does not anticipate companion artifacts or emitted artifact details in the first wave. +1. one packer-owned `packWorkspace(...)` operation; +2. internally phased and serialized through the project write lane; +3. guarded by a packer-owned validation check at execution start; +4. based on one frozen in-memory execution snapshot for the build-relevant asset bytes; +5. staged under `build/.staging//` before promotion to final artifact visibility in `build/`; +6. observable through causality-preserving progress events; +7. finished with a result that distinguishes blocked, failed, and successful pack outcomes. -The following points are now closed for `validation`: +## Next Suggested Step -1. validation runs on the same `registered + included in build` set used by summary; -2. validation is read-only, non-mutating, and snapshot-backed; -3. only diagnostics marked `blocking` fail validation in the first wave; -4. the primary validation payload is the per-asset diagnostics list, not the aggregate; -5. each per-asset validation entry must expose: - asset id, asset name, asset path, `lastModified`, and diagnostics; -6. the diagnostics returned by validation are blocking diagnostics only in the first wave; -7. if the returned validation list contains no blocking diagnostics, validation is green; -8. if the returned validation list contains any blocking diagnostics, validation is red and packing must not begin; -9. payload ordering is not a packer responsibility in the first wave; -10. a future deep-sync step may be added before validation in the Studio flow, but validation itself still evaluates the packer snapshot. +Convert this agenda into a `docs/packer` decision focused on `pack execution` semantics and propagation targets. -## Expected Follow-up +That decision should then drive: -1. Convert this agenda into a packer `decision` that closes the public API shape. -2. Execute a packer PR that adds the public contracts in `prometeu-packer-api`. -3. Follow with a packer implementation PR in `prometeu-packer-v1` for summary, validation, progress, and pack execution behavior. -4. Follow with a Studio PR to bind the wizard shell to the new packer contracts. +1. one PR for `prometeu-packer-api` result and event-contract adjustments, if needed; +2. one PR for `prometeu-packer-v1` execution, staging, and artifact-promotion behavior; +3. one Studio PR to bind the `Packing` and `Result` phases to the final packer-owned execution contract. diff --git a/docs/packer/agendas/README.md b/docs/packer/agendas/README.md index 97d2bbab..cfc82d71 100644 --- a/docs/packer/agendas/README.md +++ b/docs/packer/agendas/README.md @@ -5,6 +5,8 @@ This directory contains active packer discussion agendas. ## Active Agendas 1. [`Tilemap and Metatile Runtime Binary Layout Agenda.md`](./Tilemap%20and%20Metatile%20Runtime%20Binary%20Layout%20Agenda.md) +2. [`Pack Wizard Studio Decision Propagation Agenda.md`](./Pack%20Wizard%20Studio%20Decision%20Propagation%20Agenda.md) +3. [`Tile Bank Packing Materialization Agenda.md`](./Tile%20Bank%20Packing%20Materialization%20Agenda.md) The first packer agenda wave was consolidated into normative specs under [`../specs/`](../specs/) and didactic material under [`../learn/`](../learn/). diff --git a/docs/packer/agendas/Tile Bank Packing Materialization Agenda.md b/docs/packer/agendas/Tile Bank Packing Materialization Agenda.md new file mode 100644 index 00000000..40f785d0 --- /dev/null +++ b/docs/packer/agendas/Tile Bank Packing Materialization Agenda.md @@ -0,0 +1,174 @@ +# Tile Bank Packing Materialization Agenda + +Status: Open +Domain Owner: `docs/packer` +Cross-Domain Impact: `../runtime`, `docs/studio` + +## Purpose + +Convergir a discussão sobre como um asset `tile bank` deve ser materializado durante o `packing` para produzir payload runtime-facing válido dentro de `assets.pa`. + +Esta agenda não trata do workflow operacional do `Pack Wizard`. +Ela trata do contrato técnico de materialização do formato `TILES/indexed_v1`: + +- quais arquivos entram no pack; +- como eles viram payload binário final; +- quais campos do `asset_table` são derivados; +- quais metadados convergem para `AssetEntry.metadata`; +- e quais invariantes devem falhar o build. + +## Problem + +O repositório já tem base suficiente para: + +1. descobrir arquivos relevantes de `tile bank`; +2. validar metadados mínimos como `tile_size`; +3. construir snapshots runtime-backed; +4. definir `assets.pa` como artefato autoritativo do runtime. + +Mas ainda não existe decisão formal sobre a materialização final de `tile bank` no pack. + +Hoje falta fechar, pelo menos: + +- qual é o payload binário efetivamente emitido para um `tile bank`; +- como múltiplos artifacts selecionados são agregados no payload final; +- como `bank_type`, `codec`, `size`, `decoded_size` e `metadata` são derivados para a entrada runtime; +- quais dados ficam em `AssetEntry.metadata` versus quais permanecem detalhe interno de pipeline; +- quais condições tornam o pack inválido para esse formato. + +Sem isso, o `packWorkspace(...)` pode até ganhar semântica operacional correta, mas ainda ficará sem contrato suficiente para produzir `assets.pa` conformat para `tile bank`. + +## Context + +O contrato upstream já impõe limites claros: + +- `assets.pa` é o artefato runtime-facing autoritativo: + [`../specs/1. Domain and Artifact Boundary Specification.md`](../specs/1.%20Domain%20and%20Artifact%20Boundary%20Specification.md) +- `asset_table` é determinística por `asset_id`, offsets são relativos ao payload region e metadata converge para um sink único: + [`../specs/4. Build Artifacts and Deterministic Packing Specification.md`](../specs/4.%20Build%20Artifacts%20and%20Deterministic%20Packing%20Specification.md) +- o runtime espera `assets.pa` autocontido com `asset_table` e `preload` válidos: + [`../../../runtime/docs/runtime/specs/13-cartridge.md`](../../../runtime/docs/runtime/specs/13-cartridge.md) +- o runtime consome `AssetEntry { asset_id, asset_name, bank_type, offset, size, decoded_size, codec, metadata }`: + [`../../../runtime/docs/runtime/specs/15-asset-management.md`](../../../runtime/docs/runtime/specs/15-asset-management.md) + +Contexto atual de código: + +- `PackerAssetWalker` já reconhece `OutputFormatCatalog.TILES_INDEXED_V1`; +- `PackerTileBankWalker` já produz probes/metadata family-relevant; +- o snapshot atual já consegue expor arquivos candidatos e metadata de walk; +- ainda não existe materialização final de payload de `tile bank` dentro de `assets.pa`. + +Isso significa que o problema agora não é descoberta de arquivos. +É fechamento do contrato `tile bank -> runtime asset entry + payload bytes`. + +## Options + +### Option A - Concatenate selected tile artifacts as a simple raw stream + +Cada artifact selecionado do `tile bank` vira um segmento binário simples, e o payload final do asset é a concatenação determinística desses segmentos. + +`metadata` carrega apenas o mínimo necessário para o runtime interpretar o asset. + +### Option B - Emit one canonical tile-bank payload plus normalized runtime metadata + +Os artifacts selecionados são primeiro normalizados para um modelo canônico de `tile bank`, e então o packer emite: + +- um único payload binário canônico para o asset; +- um conjunto fechado de campos runtime-facing em `AssetEntry.metadata`. + +Qualquer detalhe interno adicional de pipeline fica fora do contrato runtime principal ou vai apenas para tooling metadata. + +### Option C - Preserve rich per-artifact structure directly in runtime metadata + +O packer mantém estrutura mais rica de artifacts individuais no próprio `AssetEntry.metadata`, expondo para o runtime detalhes mais próximos da pipeline de build. + +## Tradeoffs + +- Option A é a implementação mais simples, mas corre risco de deixar semântica demais implícita no consumidor. +- Option A também pode dificultar compatibilidade futura se a concatenação simples não codificar claramente limites, forma lógica ou derivação de `decoded_size`. +- Option B exige fechar um modelo canônico do `tile bank`, mas produz o contrato mais limpo entre packer e runtime. +- Option B também respeita melhor a regra já vigente de convergência para `AssetEntry.metadata` sem transformar metadata runtime em espelho da pipeline. +- Option C pode parecer flexível no curto prazo, mas mistura detalhe de pipeline com contrato runtime e aumenta acoplamento. +- Option C tensiona diretamente o guardrail já documentado de que `asset_table[].metadata` não deve virar depósito arbitrário de estrutura interna. + +## Recommendation + +Adotar `Option B`. + +O primeiro formato de packing a ser fechado deve ter payload canônico e metadata runtime-facing normalizada. + +### Payload Recommendation + +O `tile bank` deve produzir um payload binário único por asset incluído no build. + +Regras recomendadas: + +- o payload é derivado apenas dos artifacts selecionados que realmente entram no build atual; +- a ordem de agregação dos artifacts deve ser determinística; +- o payload final do asset deve ter fronteiras e interpretação definidas pelo próprio contrato do formato, não por convenção incidental de concatenação; +- `size` deve representar o tamanho emitido no payload region; +- `decoded_size` deve representar o tamanho lógico relevante para o runtime quando o codec exigir distinção entre payload armazenado e materialização decodificada. + +### Runtime Entry Recommendation + +Cada `tile bank` emitido para o runtime deve preencher, no mínimo: + +- `asset_id` +- `asset_name` +- `bank_type = TILES` +- `offset` +- `size` +- `decoded_size` +- `codec` +- `metadata` + +O contrato de `bank_type`, `codec` e `decoded_size` não deve ser deixado implícito no packer implementation detail. + +### Metadata Recommendation + +`AssetEntry.metadata` deve receber apenas os campos runtime-consumable e format-relevant. + +Direção inicial recomendada: + +- metadados declarativos como `tile_size` entram no sink runtime; +- metadados derivados necessários para leitura correta do runtime também entram no sink runtime; +- detalhes de pipeline úteis apenas para inspeção e tooling não devem dominar `AssetEntry.metadata`; +- quando um detalhe interno for necessário apenas para tooling, ele deve preferir companion tooling data em vez de inflar o contrato runtime. + +### Failure Recommendation + +O build de `tile bank` deve falhar quando qualquer uma das seguintes condições acontecer: + +1. não existir conjunto suficiente de artifacts selecionados para materialização válida; +2. o metadata declarativo obrigatório do formato estiver ausente ou inválido; +3. a normalização dos artifacts para o payload canônico falhar; +4. houver colisão ou ambiguidade ao convergir metadata runtime-facing; +5. o packer não conseguir derivar de forma determinística os campos exigidos para a entry runtime. + +### Packing Boundary Recommendation + +O seletor de packing para `tile bank` deve operar sobre os probes já descobertos pelo walker family-oriented. + +Regras: + +- o walker continua generalista e orientado à family; +- a seleção do que entra no payload final acontece na policy/materialization layer de packing; +- somente probes do asset registrado, incluído no build, e efetivamente selecionados pelo contrato do formato entram na materialização final; +- quando o `PackerRuntimeMaterializationConfig` estiver em `PACKING`, esses probes relevantes devem carregar bytes opcionais preenchidos para congelar o input do pack. + +## Open Questions + +1. Qual é o payload canônico inicial de `TILES/indexed_v1`: tiles brutos indexados, tiles mais tabela auxiliar, ou outro envelope mínimo? +2. Quais campos derivados além de `tile_size` precisam entrar em `AssetEntry.metadata` já na primeira versão? +3. `decoded_size` para `tile bank` deve refletir bytes lógicos pós-codec, bytes efetivos em bank, ou ambos via combinação `decoded_size + metadata`? +4. O formato inicial do payload precisa carregar múltiplos sub-blocos internos explícitos ou um stream único já é suficiente para `v1`? + +## Expected Follow-up + +1. Converter esta agenda em uma `decision` de `docs/packer` para materialização de `tile bank`. +2. Propagar o contrato resultante para: + - `docs/packer/specs/4. Build Artifacts and Deterministic Packing Specification.md` + - specs runtime relevantes em `../runtime` +3. Planejar PR de implementação do materializador de `tile bank` em `prometeu-packer-v1`. +4. Adicionar testes de conformance do formato: + `artifacts -> payload -> asset_table entry -> runtime-read assumptions`. diff --git a/docs/packer/decisions/Pack Wizard Pack Execution Semantics Decision.md b/docs/packer/decisions/Pack Wizard Pack Execution Semantics Decision.md new file mode 100644 index 00000000..cfaddd1b --- /dev/null +++ b/docs/packer/decisions/Pack Wizard Pack Execution Semantics Decision.md @@ -0,0 +1,218 @@ +# Pack Wizard Pack Execution Semantics Decision + +Status: Accepted +Date: 2026-03-20 +Domain Owner: `docs/packer` +Cross-Domain Impact: `docs/studio` + +## Context + +The Studio-side `Pack Wizard` flow is already closed as a shell over packer-owned operations. + +On the packer side, the repository also already closed the first-wave `summary` and `validation` contracts. + +What remained open was the execution contract for the actual pack operation: + +- how `packWorkspace(...)` should behave when Studio enters `Packing` and `Result`; +- how the packer should avoid filesystem drift while packing; +- how final artifact visibility should be protected during emission; +- and how the operation should report success, blocking failure, and execution failure without pushing semantic reconstruction into Studio. + +The agenda that closed this discussion is: + +- [`../agendas/Pack Wizard Studio Decision Propagation Agenda.md`](../agendas/Pack%20Wizard%20Studio%20Decision%20Propagation%20Agenda.md) + +Relevant upstream references are: + +- [`../decisions/Pack Wizard Summary and Validation Contracts Decision.md`](../decisions/Pack%20Wizard%20Summary%20and%20Validation%20Contracts%20Decision.md) +- [`../specs/1. Domain and Artifact Boundary Specification.md`](../specs/1.%20Domain%20and%20Artifact%20Boundary%20Specification.md) +- [`../specs/4. Build Artifacts and Deterministic Packing Specification.md`](../specs/4.%20Build%20Artifacts%20and%20Deterministic%20Packing%20Specification.md) +- [`../specs/5. Diagnostics, Operations, and Studio Integration Specification.md`](../specs/5.%20Diagnostics,%20Operations,%20and%20Studio%20Integration%20Specification.md) +- [`../../studio/decisions/Pack Wizard in Assets Workspace Decision.md`](../../studio/decisions/Pack%20Wizard%20in%20Assets%20Workspace%20Decision.md) + +## Decision + +The first-wave packer execution contract for the `Pack Wizard` adopts the following direction: + +1. `packWorkspace(...)` is one packer-owned public operation. +2. It executes through the project-scoped serialized write lane. +3. It must rerun a packer-owned gate check before artifact emission begins. +4. It must pack from one frozen execution snapshot rather than from live filesystem rereads. +5. It must stage emitted outputs before promotion to final `build/` locations. +6. It publishes final build artifacts to `build/` only after a coherent success point is reached. +7. It remains the only first-wave operation that emits build artifacts for the Studio `Pack Wizard`. + +## Adopted Constraints + +### 1. Execution Boundary + +- `packWorkspace(...)` evaluates the active pack set as `registered + included in build`; +- it runs under the same packer-owned project write serialization model already adopted for same-project writes and build operations; +- it must not trust Studio call ordering as if that were a correctness guarantee; +- a failing pack gate returns a terminal result without publishing final build artifacts. + +### 2. Frozen Execution Snapshot + +- pack execution must not reread live workspace files once execution materialization begins; +- the packer must create a frozen pack-execution snapshot before payload materialization starts; +- that frozen snapshot is derived from the current runtime snapshot and packer-owned walk/materialization path; +- file content bytes are required only for build-relevant inputs: + - registered assets included in build; + - artifact files that actually participate in the current build output; +- non-build assets and non-selected artifact files do not need in-memory content in the pack-execution snapshot. + +### 3. Runtime Walk Projection Shape + +- the existing runtime walk projection model remains the baseline; +- `PackerRuntimeWalkFile` gains optional content bytes instead of introducing a second snapshot model in the first wave; +- optional content bytes remain absent in normal runtime-backed projection and become present only in packing-focused projection. + +### 4. Materialization Policy Shape + +The adopted first-wave shape is: + +```java +public record PackerRuntimeMaterializationConfig( + PackerWalkMode mode, + Predicate projectionFilter) { + + public static PackerRuntimeMaterializationConfig runtimeDefault() { + return new PackerRuntimeMaterializationConfig(PackerWalkMode.RUNTIME, probe -> true); + } + + public static PackerRuntimeMaterializationConfig packingBuild() { + return new PackerRuntimeMaterializationConfig( + PackerWalkMode.PACKING, + PackerProbePolicies::isIncludedInCurrentBuild); + } +} +``` + +With this contract: + +- `PackerWalkMode.RUNTIME` suppresses optional content bytes during export to runtime walk projection; +- `PackerWalkMode.PACKING` injects optional content bytes during export to runtime walk projection; +- `projectionFilter` decides which discovered probe results survive into `PackerRuntimeWalkFile`; +- the first-wave packing filter keeps only current-build probe results. + +### 5. Walker Boundary + +- family walkers remain family-oriented discovery mechanisms; +- they may continue to produce full file probes with bytes available during probe processing; +- the runtime-versus-packing variation happens during export/materialization into runtime walk projection; +- packing-specific selection is one additional projection/materialization policy layer over the discovered probe results; +- first-wave pack execution must not fork the repository into one discovery model for runtime and another discovery model for packing. + +### 6. Staging and Promotion Boundary + +- pack execution stages outputs under `build/.staging//`; +- `` is one random 20-character alphanumeric identifier; +- if a staging directory collision occurs for that identifier, the packer may overwrite that staging directory in the first wave; +- staged outputs are non-authoritative until promotion; +- promotion into final `build/` locations happens only after the operation reaches a coherent success point; +- if emission fails before promotion, final `build/` outputs must remain unchanged from the caller's point of view. + +### 7. Build Publication Boundary + +- the canonical runtime-facing output is `build/assets.pa`; +- the baseline companion outputs are: + - `build/asset_table.json` + - `build/preload.json` + - `build/asset_table_metadata.json` +- `packWorkspace(...)` stops at build artifact publication; +- shipper behavior remains outside this operation and is mentioned only as a future consumer boundary. + +### 8. Result and Failure Semantics + +The final pack result must distinguish at least: + +1. validation gate failure before emission; +2. execution or materialization failure during pack generation; +3. persistence or promotion failure while making outputs final. + +The first-wave result summary must expose at least: + +- final status; +- whether execution actually began or stopped at gate failure; +- canonical build-relative reference to `assets.pa`; +- build-relative companion artifact references when emitted; +- total packed asset count; +- elapsed time in milliseconds. + +### 9. Minimum Structural Conformance + +The first-wave pack execution is not successful unless the emitted outputs satisfy the already adopted baseline structural guarantees: + +- `asset_table` ordering is deterministic by increasing `asset_id`; +- header serialization is canonical; +- offsets are relative to `payload_offset`; +- preload entries are resolved by `asset_id`; +- invalid preload references or preload slot clashes fail the operation rather than being deferred to host guesswork. + +## Why This Direction Was Chosen + +- It preserves packer ownership of build semantics while keeping Studio as an honest shell. +- It prevents filesystem drift during execution without turning the general runtime snapshot into a wasteful full-memory mirror. +- It aligns build visibility with the same commit-oriented operational model already adopted elsewhere in the packer runtime. +- It keeps the first-wave implementation on known terrain by reusing current walker/materialization structure instead of inventing a second discovery stack. +- It gives the Studio enough structured outcome information to render `Packing` and `Result` without semantic reconstruction. + +## Explicit Non-Decisions + +This decision does not yet define: + +- the format-specific payload materialization rules for each asset output format; +- the canonical payload contract for `tile bank`, `sound bank`, or other family-specific outputs; +- the final event kind vocabulary for pack execution progress beyond requiring causality-preserving observable lifecycle; +- cancellation semantics; +- future incremental or remote build behavior; +- shipper packaging rules, cartridge manifest generation, or final packaged cartridge layout. + +## Implications + +- `packWorkspace(...)` can no longer be implemented as a direct live-filesystem read/write shortcut; +- runtime walk projection and materialization code must gain explicit packing-aware export policy; +- `PackerRuntimeWalkFile` must support optional content bytes; +- pack execution implementation must include frozen-input creation, staging, promotion, and structured terminal outcomes; +- format-specific packing discussions must now happen as follow-up decisions rather than being guessed inside the generic execution path. + +## Propagation Targets + +Specs: + +- [`../specs/4. Build Artifacts and Deterministic Packing Specification.md`](../specs/4.%20Build%20Artifacts%20and%20Deterministic%20Packing%20Specification.md) +- [`../specs/5. Diagnostics, Operations, and Studio Integration Specification.md`](../specs/5.%20Diagnostics,%20Operations,%20and%20Studio%20Integration%20Specification.md) + +Plans: + +- [`../pull-requests/PR-28-pack-wizard-public-contracts-summary-validation-and-execution.md`](../pull-requests/PR-28-pack-wizard-public-contracts-summary-validation-and-execution.md) +- [`../pull-requests/PR-29-pack-wizard-contract-adjustments-for-summary-and-validation.md`](../pull-requests/PR-29-pack-wizard-contract-adjustments-for-summary-and-validation.md) +- future packer PR for pack execution, staging, and result semantics + +Cross-domain references: + +- [`../../studio/decisions/Pack Wizard in Assets Workspace Decision.md`](../../studio/decisions/Pack%20Wizard%20in%20Assets%20Workspace%20Decision.md) +- [`../../studio/pull-requests/PR-11-pack-wizard-shell-and-packer-contract-consumption.md`](../../studio/pull-requests/PR-11-pack-wizard-shell-and-packer-contract-consumption.md) + +Follow-up agenda references: + +- [`../agendas/Tile Bank Packing Materialization Agenda.md`](../agendas/Tile%20Bank%20Packing%20Materialization%20Agenda.md) + +Implementation surfaces: + +- `prometeu-packer-api` pack execution result contract +- `prometeu-packer-v1` runtime materialization/export policy +- `prometeu-packer-v1` frozen execution snapshot creation +- `prometeu-packer-v1` staging and promote pipeline +- Studio `Pack Wizard` packing/result bindings + +## Validation Notes + +This decision is correctly implemented only when all of the following are true: + +- `packWorkspace(...)` does not depend on live filesystem rereads after frozen execution materialization begins; +- only build-relevant selected inputs carry frozen content bytes into the packing path; +- normal runtime-backed reads do not retain optional content bytes by default; +- failed execution before promotion does not leave partial final `build/` artifacts presented as success; +- successful pack publication produces `build/assets.pa` and the expected companion outputs coherently; +- Studio can distinguish blocked, failed, and successful pack outcomes from packer-owned results alone. diff --git a/docs/packer/decisions/README.md b/docs/packer/decisions/README.md index 57566acf..b34aa66a 100644 --- a/docs/packer/decisions/README.md +++ b/docs/packer/decisions/README.md @@ -6,6 +6,8 @@ Retained packer decision records: - [`Concurrency, Observability, and Studio Adapter Boundary Decision.md`](./Concurrency,%20Observability,%20and%20Studio%20Adapter%20Boundary%20Decision.md) - [`Filesystem-First Operational Runtime and Reconcile Boundary Decision.md`](./Filesystem-First%20Operational%20Runtime%20and%20Reconcile%20Boundary%20Decision.md) +- [`Pack Wizard Summary and Validation Contracts Decision.md`](./Pack%20Wizard%20Summary%20and%20Validation%20Contracts%20Decision.md) +- [`Pack Wizard Pack Execution Semantics Decision.md`](./Pack%20Wizard%20Pack%20Execution%20Semantics%20Decision.md) The first packer decision wave was already consolidated into: