prometeu-studio/docs/packer/agendas/01.3. Build Artifacts and Deterministic Packing Agenda.md

Build Artifacts and Deterministic Packing Agenda

Status

Closed

Resolved by 003-build-artifacts-and-deterministic-packing-decision.md.

Purpose

Close the contract for packer build outputs and the determinism rules that make them reproducible.

Context

The draft proposes two core build artifacts:

• build/assets.pa
• build/asset_table.json

At least one of these artifacts must form a documented interface that ../runtime can consume safely. That runtime-facing contract should be specified before the implementation hardens around incidental file structure.

It also states strong determinism requirements:

• deterministic asset order,
• deterministic derived outputs,
• explicit materialization of inferred defaults.

This is a strong basis for normative packer output specs.

Source Sections

• 8. Build Artifacts Produced by the Packer
• 9. Determinism Rules

Runtime Normative Inputs

From ../runtime/docs/runtime/specs/13-cartridge.md:

• assets.pa is the runtime-facing artifact;
• assets.pa v1 is an autocontained binary with:
  • fixed binary prelude,
  • JSON header,
  • binary payload region;
• the fixed prelude contains, at minimum:
  • magic,
  • schema_version,
  • header_len,
  • payload_offset;
• the JSON header carries:
  • asset_table,
  • preload.

From ../runtime/docs/runtime/specs/15-asset-management.md:

• asset_table entries currently expose:
  • asset_id,
  • asset_name,
  • bank_type,
  • offset,
  • size,
  • decoded_size,
  • codec,
  • metadata;
• offset is relative to the payload region inside assets.pa, not to the start of the full file;
• preload is boot-time input only;
• runtime loading is based on open_slice over payload slices;
• the runtime must not require the entire payload to stay resident in RAM.

These are already normative runtime-facing requirements and should be treated as baseline constraints for packer output design.

For this agenda, the intended direction is:

• assets.pa is the authoritative runtime-facing artifact;
• build/asset_table.json is a companion tooling artifact derived from the same build result;
• runtime must not depend on build/asset_table.json as its primary source of truth.

Current runtime code now resolves preload by asset_id, but game-facing runtime APIs still refer to assets by asset_name. Therefore this agenda distinguishes:

• asset_id as stable artifact identity for preload/bootstrap and asset-table integrity;
• asset_name as logical API-level identifier used by game/runtime-facing calls today.

This agenda does not introduce a second runtime-only asset identity. The asset_id emitted by the packer is the same asset_id used in the runtime-facing artifact contract.

Key Questions

1. How should the packer spec mirror the already normative assets.pa v1 envelope from runtime specs 13 and 15?
2. Which fields in build/asset_table.json are companion tooling data versus runtime-facing contract data?
3. Should global asset packing order be deterministic by asset_id, or influenced by usage and format-specific heuristics?
4. How should alignment be expressed and validated?
5. What exactly counts as determinism for virtual asset pipelines?
6. Which inferred values must be written back versus only emitted in build outputs?
7. Which packer-side metadata is intentionally excluded from the runtime-facing contract?
8. How should packer document preload lifecycle and offset semantics so they match runtime reader behavior?
9. How should per-asset preload declarations be transformed into the preload section of assets.pa deterministically?
10. How should the packer preserve both asset_id integrity and asset_name lookup semantics in runtime-facing artifacts?
11. Should asset_table be modeled as a deterministically ordered asset list that preserves packer asset_id values end-to-end?
12. Which envelope fields are runtime-validated versus emitted only for Studio/tooling use?

Options

Option A

Keep assets.pa as the authoritative autocontained runtime-facing artifact and emit build/asset_table.json as a derived companion artifact for Studio and tooling.

Option B

Keep runtime-facing structure split across assets.pa and build/asset_table.json.

Tradeoffs

• Option A matches the runtime specs already in force.
• Option A keeps runtime authority in one artifact and prevents contract drift.
• Option A still allows rich tooling through a derived companion file.
• Option B risks splitting authority across artifacts and making runtime/tooling divergence easier.

Recommendation

Adopt Option A with these fixed directions:

• assets.pa is the authoritative runtime-facing artifact;
• the assets.pa header is the authoritative runtime-facing descriptor;
• build/asset_table.json is a derived companion artifact for Studio, debugging, inspection, and builder/tooling integration;
• global packing order is deterministic by increasing asset_id;
• preload emission is derived deterministically from per-asset preload.enabled;
• alignment exists only when explicitly required by spec;
• offsets are always relative to the payload region, never to the start of the full file;
• checksum is not part of the baseline assets.pa envelope contract.

Also adopt this performance direction:

• usage-based hot-first packing is not part of the baseline contract;
• deterministic stable ordering is preferred over heuristic physical reordering;
• future locality optimization may be introduced only through an explicit later decision and spec, not as an implementation shortcut now.

Also adopt this identity direction:

• runtime-facing preload entries should migrate to stable asset_id instead of mutable asset_name;
• runtime-facing preload/bootstrap integrity should treat asset_id as primary identity;
• asset_name remains present in asset_table as logical descriptive and API-facing metadata used by current runtime/game-facing calls;
• asset_table should be generated deterministically on every build from the current managed asset set;
• asset_table should be emitted as a deterministically ordered list of asset entries;
• the runtime-facing asset_id is exactly the stable asset_id allocated by the packer registry;
• stable cross-build identity comes from asset_id, not from table position;
• rename of asset_name is therefore an API-visible/content-visible change, but not an identity change.

Also adopt this companion-artifact direction:

• build/asset_table.json should mirror the runtime-facing header data 1:1 for debugging and inspection;
• if richer tooling artifacts are needed, they should be emitted as separate files instead of overloading build/asset_table.json.

Expected Decisions to Produce

1. v1 artifact boundary and ownership.
2. Deterministic ordering and alignment rules.
3. Descriptor schema responsibilities.
4. Materialization requirements for inferred values.
5. Runtime-facing artifact contract and reader expectations.
6. Packer-side restatement of the runtime-owned assets.pa envelope and slice semantics.
7. Deterministic mapping from asset declarations to runtime preload data.
8. Explicit exclusion of usage-based packing heuristics from the baseline contract.
9. Migration of preload/bootstrap integrity toward asset_id-based identity while preserving asset_name as current API-facing metadata.
10. Deterministically ordered asset_table preserving packer asset_id values end-to-end.
11. Separation between runtime-validated envelope fields and non-envelope tooling concerns.

Expected Spec Follow-up

• Packed ROM artifact spec.
• Asset table descriptor spec.
• Determinism and reproducibility spec.
• Runtime consumption interface spec.

Non-Goals

• Asset workspace mutation commands.
• Quarantine behavior.
• Builder packaging flow.