packer (WIP)

This commit is contained in:
2026-03-24 13:42:55 +00:00
parent d9911a63dc
commit db8c17864f
12 changed files with 815 additions and 406 deletions
@@ -58,8 +58,39 @@ Contexto atual de código:
- o snapshot atual já consegue expor arquivos candidatos e metadata de walk;
- ainda não existe materialização final de payload de `tile bank` dentro de `assets.pa`.
Consumer baseline now confirmed in `../runtime`:
- `TILES` now uses `codec = NONE` as the runtime-facing v1 baseline;
- serialized tile pixels are packed `u4` palette indices in payload order;
- the runtime expands those packed indices into one `u8` logical index per pixel in memory after decode;
- tile-bank palettes are serialized as `RGB565` `u16` values in little-endian order;
- runtime-facing v1 uses `64` palettes per tile bank;
- the runtime loader currently requires the following metadata fields for tile banks:
- `tile_size`
- `width`
- `height`
- `palette_count`
- for v1, `palette_count` must be `64`;
- for v1, the serialized tile-bank payload is:
1. packed indexed pixels for the full sheet, using `ceil(width * height / 2)` bytes;
2. one palette block of `64 * 16 * 2 = 2048` bytes;
- for v1, the runtime-side size expectations are:
- `size = ceil(width * height / 2) + 2048`
- `decoded_size = (width * height) + 2048`
- for the producer-side contract discussed here, `tile_id = 0` remains valid and must not be reserved away by the packer.
Relevant confirmed runtime references:
- [`../../../runtime/docs/runtime/specs/15-asset-management.md`](../../../runtime/docs/runtime/specs/15-asset-management.md)
- [`../../../runtime/docs/runtime/specs/04-gfx-peripheral.md`](../../../runtime/docs/runtime/specs/04-gfx-peripheral.md)
- [`../../../runtime/crates/console/prometeu-drivers/src/asset.rs`](../../../runtime/crates/console/prometeu-drivers/src/asset.rs)
- [`../../../runtime/crates/console/prometeu-hal/src/tile_bank.rs`](../../../runtime/crates/console/prometeu-hal/src/tile_bank.rs)
- tilemap empty-cell semantics remain under active runtime discussion and must not currently force the packer to reserve `tile_id = 0`:
[`../../../runtime/docs/runtime/agendas/023-tilemap-empty-cell-vs-tile-id-zero.md`](../../../runtime/docs/runtime/agendas/023-tilemap-empty-cell-vs-tile-id-zero.md)
Isso significa que o problema agora não é descoberta de arquivos.
É fechamento do contrato `tile bank -> runtime asset entry + payload bytes`.
O lado consumidor já está suficientemente claro.
O problema agora é fechar o contrato produtor `tile bank -> runtime asset entry + payload bytes` no packer sem contradizer esse baseline runtime.
## Options
@@ -104,10 +135,17 @@ O `tile bank` deve produzir um payload binário único por asset incluído no bu
Regras recomendadas:
- o payload é derivado apenas dos artifacts selecionados que realmente entram no build atual;
- a ordem de agregação dos artifacts deve ser determinística;
- a ordem de agregação dos artifacts deve ser determinística by `artifacts[*].index`;
- for v1, `1 artifact = 1 tile`;
- for the current target, the canonical tile-bank sheet is always `256 x 256`;
- o payload final do asset deve ter fronteiras e interpretação definidas pelo próprio contrato do formato, não por convenção incidental de concatenação;
- para `TILES/indexed_v1`, o payload v1 já deve assumir:
1. plano de pixels packed `u4`;
2. bloco de paletas `64 * 16 * u16`;
- palettes must always be materialized to `RGB565` during pack emission;
- `size` deve representar o tamanho emitido no payload region;
- `decoded_size` deve representar o tamanho lógico relevante para o runtime quando o codec exigir distinção entre payload armazenado e materialização decodificada.
- `decoded_size` deve seguir a convenção runtime já confirmada:
tamanho expandido dos indices em memória mais o bloco de paletas runtime-facing.
### Runtime Entry Recommendation
@@ -124,6 +162,18 @@ Cada `tile bank` emitido para o runtime deve preencher, no mínimo:
O contrato de `bank_type`, `codec` e `decoded_size` não deve ser deixado implícito no packer implementation detail.
Baseline now fixed by the runtime consumer:
- `bank_type = TILES`
- `codec = NONE`
- metadata mínima obrigatória:
- `tile_size`
- `width`
- `height`
- `palette_count = 64`
- `width` and `height` are bank-sheet helpers, not per-artifact dimensions
- with the current v1 target, the emitted bank sheet is fixed at `256 x 256`
### Metadata Recommendation
`AssetEntry.metadata` deve receber apenas os campos runtime-consumable e format-relevant.
@@ -131,7 +181,18 @@ O contrato de `bank_type`, `codec` e `decoded_size` não deve ser deixado implí
Direção inicial recomendada:
- metadados declarativos como `tile_size` entram no sink runtime;
- metadados derivados necessários para leitura correta do runtime também entram no sink runtime;
- metadados derivados necessários para leitura correta do runtime entram no sink runtime, pelo menos:
- `width`
- `height`
- `palette_count`;
- `AssetEntry.metadata` should aggregate normalized maps derived from:
- `asset.json.output.metadata`
- `asset.json.output.codec_configuration`
- `asset.json.output.pipeline`;
- bank palettes are declared in `asset.json.output.pipeline.palettes` using explicit `{ index, palette }` entries and emitted in ascending numeric `index` order;
- any tile in the bank may be rendered with any palette in the bank;
- palette assignment is therefore not a per-artifact packing contract and remains a runtime draw-time concern;
- the packer must nevertheless validate whether the declared bank palette set safely covers the indices used by packed tiles.
- detalhes de pipeline úteis apenas para inspeção e tooling não devem dominar `AssetEntry.metadata`;
- quando um detalhe interno for necessário apenas para tooling, ele deve preferir companion tooling data em vez de inflar o contrato runtime.
@@ -145,6 +206,13 @@ O build de `tile bank` deve falhar quando qualquer uma das seguintes condições
4. houver colisão ou ambiguidade ao convergir metadata runtime-facing;
5. o packer não conseguir derivar de forma determinística os campos exigidos para a entry runtime.
Additional first-wave diagnostic expectations:
- bank without declared palettes in `asset.json.output.pipeline.palettes` must emit a diagnostic;
- tile banks whose declared palette list exceeds `64` must fail;
- tiles that use fragile indices, meaning indices not represented safely across the full declared bank palette set, should emit a `WARNING`;
- that fragile-index warning is advisory in the first wave and should not block pack by itself unless later promoted by decision.
### Packing Boundary Recommendation
O seletor de packing para `tile bank` deve operar sobre os probes já descobertos pelo walker family-oriented.
@@ -155,13 +223,21 @@ Regras:
- a seleção do que entra no payload final acontece na policy/materialization layer de packing;
- somente probes do asset registrado, incluído no build, e efetivamente selecionados pelo contrato do formato entram na materialização final;
- quando o `PackerRuntimeMaterializationConfig` estiver em `PACKING`, esses probes relevantes devem carregar bytes opcionais preenchidos para congelar o input do pack.
- palette declarations in `asset.json.output.pipeline.palettes` carry explicit semantic identity through `index`;
- palette order is ascending numeric `index`, never raw array position;
- palette ids are the normalized declared `index` values from that pipeline palette list;
- all tiles in the bank may use any palette declared in the bank;
- palette selection is a runtime draw concern, not a tile-payload embedding concern.
## Open Questions
1. Qual é o payload canônico inicial de `TILES/indexed_v1`: tiles brutos indexados, tiles mais tabela auxiliar, ou outro envelope mínimo?
2. Quais campos derivados além de `tile_size` precisam entrar em `AssetEntry.metadata` já na primeira versão?
3. `decoded_size` para `tile bank` deve refletir bytes lógicos pós-codec, bytes efetivos em bank, ou ambos via combinação `decoded_size + metadata`?
4. O formato inicial do payload precisa carregar múltiplos sub-blocos internos explícitos ou um stream único já é suficiente para `v1`?
1. Given the fixed `256 x 256` bank target, what is the canonical tile placement rule inside that sheet for `tile_size = 8`, `16`, and `32`?
2. Which normalization diagnostics should be `blocking` versus `warning` specifically for:
- artifact index gaps or duplicates
- sheet capacity overflow
- palette list overflow beyond `64`
- banks without declared palettes
- fragile tile indices across the declared bank palette set?
## Expected Follow-up
@@ -0,0 +1,113 @@
# PR-32 Palette Declarations with Explicit Index Contract
Domain Owner: `docs/packer`
Cross-Domain Impact: `docs/studio`, `../runtime`
## Briefing
The current authoring shape for `asset.json.output.pipeline.palettes` is still too dependent on list order as semantic identity.
That is fragile:
- editorial reordering can silently change meaning;
- merges become harder to review safely;
- the packer has to infer identity from position instead of reading it explicitly.
The repository already depends on stable palette ordering for `tile bank` packing.
That ordering should therefore be explicit in the authoring contract.
## Decisions de Origem
- [`../decisions/Pack Wizard Pack Execution Semantics Decision.md`](../decisions/Pack%20Wizard%20Pack%20Execution%20Semantics%20Decision.md)
- ongoing agenda:
[`../agendas/Tile Bank Packing Materialization Agenda.md`](../agendas/Tile%20Bank%20Packing%20Materialization%20Agenda.md)
## Objective
Replace list-position-defined palette identity with an explicit index-bearing declaration contract for `asset.json.output.pipeline.palettes`.
Recommended shape:
```json
"palettes": [
{
"index": 0,
"palette": { ... }
},
{
"index": 1,
"palette": { ... }
}
]
```
## Dependencies
- tile-bank materialization agenda in `docs/packer/agendas`
- runtime-side tile-bank consumer contract already aligned around stable palette ids
## Scope
- define the normative authoring shape for `output.pipeline.palettes`
- make `index` the semantic identity of each declared palette
- require packer normalization to sort by `index`, not by list position
- define diagnostics for duplicate, missing, invalid, or malformed palette entries
- prepare propagation into:
- packer specs
- parser/runtime-facing normalization code
- Studio editing surfaces if they expose palette management
## Non-Goals
- no immediate migration of `artifacts` in this PR
- no tile-bank payload implementation in this PR
- no runtime code changes in this PR
- no speculative keyed-object JSON contract for palettes in this wave
## Execution Method
1. Update the packer-side documentation/decision chain to make palette identity explicit.
2. Define `output.pipeline.palettes` as a list of records with:
- `index`
- `palette`
3. State that semantic ordering is ascending numeric `index`, never raw list position.
4. Define structural diagnostics:
- duplicate `index` => blocking
- negative or malformed `index` => blocking
- missing `palette` payload => blocking
- palette count above the runtime-facing v1 limit => blocking
5. Define whether index gaps are:
- blocking
- or warning/advisory
6. Prepare follow-up implementation PRs for parser, validation, and Studio surfaces.
## Contract Direction
First-wave packer contract direction:
1. `output.pipeline.palettes` remains a JSON array for authoring ergonomics;
2. each entry must declare an explicit `index`;
3. each entry must declare a `palette` object payload;
4. packer meaning comes from numeric `index`, not from the physical array position;
5. the runtime-facing `palette_id` consumed later by tile/sprite draw paths is the normalized numeric palette `index`.
## Acceptance Criteria
- the packer docs/spec path no longer relies on raw list order as palette identity
- the explicit-index palette shape is documented clearly enough for parser and Studio implementation
- duplicate or malformed palette indices are identified as structural diagnostics
- the contract leaves no ambiguity about whether sorting happens by array order or by `index`
## Validation
- editorial validation across packer agenda/decision/spec material
- parser-oriented review proving the contract is implementable without hidden heuristics
- Studio-facing review confirming the editing UX can preserve stable palette identity
## Affected Artifacts
- `docs/packer/agendas/**`
- `docs/packer/decisions/**`
- `docs/packer/specs/3. Asset Declaration and Virtual Asset Contract Specification.md`
- `docs/packer/specs/4. Build Artifacts and Deterministic Packing Specification.md`
- future parser/runtime materialization code under `prometeu-packer-v1`
@@ -96,6 +96,40 @@ Rules:
- `output.pipeline` carries pipeline-injected metadata kept separate at authoring time;
- codec must remain explicit and must not be hidden inside format naming.
### Explicit Index Collections
When `output.pipeline` carries ordered collections whose members need stable semantic identity, that identity must not depend on raw list position alone.
Rules:
- collections such as `output.pipeline.palettes` may remain JSON arrays for authoring ergonomics;
- when a collection member has runtime- or packing-relevant identity, each entry must carry an explicit `index`;
- `output.pipeline.palettes` entries must use the shape `{ "index": <int>, "palette": { ... } }`;
- the packer must sort such entries by numeric `index`, not by physical array position;
- duplicate or malformed indices are structural errors;
- missing or malformed `palette` payloads are structural errors;
- editorial reordering of the JSON array alone must not change semantic meaning.
Example:
```json
{
"output": {
"pipeline": {
"palettes": [
{
"index": 0,
"palette": {
"originalArgb8888": [4294901760, 4278255360],
"convertedRgb565": [63488, 2016]
}
}
]
}
}
}
```
## Metadata Source Segmentation and Runtime Sink
`asset.json` may keep metadata segmented by concern during authoring.