prometeu-studio/docs/packer/agendas/Variable Tile Bank Palette Serialization Agenda.md

Variable Tile Bank Palette Serialization Agenda

Status

Active

Domain Owner

• docs/packer

Cross-domain dependency:

• ../runtime

Purpose

Decide whether tile bank payloads should keep serializing a fixed block of 64 palettes in v1, or evolve to serialize only the palettes actually authored by the asset.

Context

The current tile bank contract is aligned between packer and runtime:

• pixels are serialized as packed u4;
• palettes are serialized as RGB565 u16;
• palette_count = 64;
• payload size is derived from ceil(width * height / 2) + (64 * 16 * 2).

This means a tile bank with 1 authored palette still serializes 64 palette slots. Unused slots are currently zero-filled, which is valid for the runtime, but wastes cartridge space.

The packer now exposes:

• metadata.palette_count = 64
• metadata.palette_authored = <actual authored count>

This makes the waste explicit, but does not remove it.

Problem

The runtime-facing v1 contract favors fixed-size decode simplicity over cartridge efficiency.

We need to decide whether:

1. fixed 64 palette slots remain the normative contract for tile bank;
2. palette_count becomes variable and payload size shrinks to the authored set;
3. another staged strategy is better.

Options

Option A: Keep Fixed 64-Palette Serialization

Keep the current contract unchanged:

• palette_count remains 64;
• every tile bank serializes 64 * 16 * 2 = 2048 bytes of palettes;
• authored palettes populate their indexed slots;
• unused slots are zero-filled.

Option B: Move to Variable Palette Serialization

Evolve the contract so tile bank serializes only authored palettes:

• palette_count becomes the authored count;
• payload size becomes ceil(width * height / 2) + (palette_count * 16 * 2);
• runtime validates and decodes variable palette blocks.

Option C: Keep V1 Fixed, Design Variable Serialization for V2

Preserve the current runtime-facing v1 contract now, but explicitly open a v2 format/contract:

• v1 stays fixed at 64;
• v2 introduces variable palette serialization;
• packer and runtime keep clear compatibility boundaries.

Tradeoffs

Option A

Pros:

• no further runtime work;
• simplest decode path;
• current specs and tests stay valid.

Cons:

• every tile bank pays the full 2 KB palette cost;
• cartridge size is inflated for small authored palette sets;
• the metadata now exposes a mismatch between authored and serialized counts by design.

Option B

Pros:

• reduces cartridge size;
• makes palette_count semantically tighter;
• removes fixed empty palette padding from the payload.

Cons:

• requires coordinated changes in packer, runtime, specs, and tests;
• changes the size / decoded_size formulas;
• raises compatibility and migration questions for existing assets and cartridges.

Option C

Pros:

• keeps current work stable;
• makes room for a cleaner future contract;
• avoids mixing optimization work into the current tile-bank rollout.

Cons:

• known waste remains in v1;
• adds future format/version management work.

Recommendation

Recommendation for now: Option C.

Reasoning:

• the current fixed-size contract is already implemented and aligned;
• the waste is real, but it is not a correctness bug;
• changing it now would reopen a cross-domain runtime contract that was just stabilized;
• if we want variable palette serialization, it should be introduced deliberately as a versioned follow-up, not as an incidental tweak.

Open Questions

1. Should variable palette serialization be introduced as a new tile bank payload version, or as an incompatible change to the current one?
2. If palette_count becomes variable, should runtime still materialize a 64-slot bank in memory, or truly shrink the resident representation too?
3. Should palette slot identity remain sparse by authored index, or should serialization canonicalize into a dense runtime block?
4. Which domain owns the compatibility story for existing cartridges: packer, runtime, or shipper?

Expected Follow-up

If the team wants to eliminate the fixed 64-palette padding:

1. create a corresponding runtime agenda/decision;
2. define the versioning and compatibility story;
3. update packer and runtime specs together;
4. then plan code changes in both domains.