180 lines
7.8 KiB
Markdown
180 lines
7.8 KiB
Markdown
# Build Artifacts and Deterministic Packing Agenda
|
|
|
|
## Status
|
|
|
|
Closed
|
|
|
|
Resolved by [`003-build-artifacts-and-deterministic-packing-decision.md`](../decisions/003-build-artifacts-and-deterministic-packing-decision.md).
|
|
|
|
## Purpose
|
|
|
|
Close the contract for packer build outputs and the determinism rules that make them reproducible.
|
|
|
|
## Context
|
|
|
|
The draft proposes two core build artifacts:
|
|
|
|
- `build/assets.pa`
|
|
- `build/asset_table.json`
|
|
|
|
At least one of these artifacts must form a documented interface that `../runtime` can consume safely.
|
|
That runtime-facing contract should be specified before the implementation hardens around incidental file structure.
|
|
|
|
It also states strong determinism requirements:
|
|
|
|
- deterministic asset order,
|
|
- deterministic derived outputs,
|
|
- explicit materialization of inferred defaults.
|
|
|
|
This is a strong basis for normative packer output specs.
|
|
|
|
## Source Sections
|
|
|
|
- `8. Build Artifacts Produced by the Packer`
|
|
- `9. Determinism Rules`
|
|
|
|
## Runtime Normative Inputs
|
|
|
|
From `../runtime/docs/runtime/specs/13-cartridge.md`:
|
|
|
|
- `assets.pa` is the runtime-facing artifact;
|
|
- `assets.pa` v1 is an autocontained binary with:
|
|
- fixed binary prelude,
|
|
- JSON header,
|
|
- binary payload region;
|
|
- the fixed prelude contains, at minimum:
|
|
- `magic`,
|
|
- `schema_version`,
|
|
- `header_len`,
|
|
- `payload_offset`;
|
|
- the JSON header carries:
|
|
- `asset_table`,
|
|
- `preload`.
|
|
|
|
From `../runtime/docs/runtime/specs/15-asset-management.md`:
|
|
|
|
- `asset_table` entries currently expose:
|
|
- `asset_id`,
|
|
- `asset_name`,
|
|
- `bank_type`,
|
|
- `offset`,
|
|
- `size`,
|
|
- `decoded_size`,
|
|
- `codec`,
|
|
- `metadata`;
|
|
- `offset` is relative to the payload region inside `assets.pa`, not to the start of the full file;
|
|
- `preload` is boot-time input only;
|
|
- runtime loading is based on `open_slice` over payload slices;
|
|
- the runtime must not require the entire payload to stay resident in RAM.
|
|
|
|
These are already normative runtime-facing requirements and should be treated as baseline constraints for packer output design.
|
|
|
|
For this agenda, the intended direction is:
|
|
|
|
- `assets.pa` is the authoritative runtime-facing artifact;
|
|
- `build/asset_table.json` is a companion tooling artifact derived from the same build result;
|
|
- runtime must not depend on `build/asset_table.json` as its primary source of truth.
|
|
|
|
Current runtime code now resolves preload by `asset_id`, but game-facing runtime APIs still refer to assets by `asset_name`.
|
|
Therefore this agenda distinguishes:
|
|
|
|
- `asset_id` as stable artifact identity for preload/bootstrap and asset-table integrity;
|
|
- `asset_name` as logical API-level identifier used by game/runtime-facing calls today.
|
|
|
|
This agenda does not introduce a second runtime-only asset identity.
|
|
The `asset_id` emitted by the packer is the same `asset_id` used in the runtime-facing artifact contract.
|
|
|
|
## Key Questions
|
|
|
|
1. How should the packer spec mirror the already normative `assets.pa` v1 envelope from runtime specs 13 and 15?
|
|
2. Which fields in `build/asset_table.json` are companion tooling data versus runtime-facing contract data?
|
|
3. Should global asset packing order be deterministic by `asset_id`, or influenced by usage and format-specific heuristics?
|
|
4. How should alignment be expressed and validated?
|
|
5. What exactly counts as determinism for virtual asset pipelines?
|
|
6. Which inferred values must be written back versus only emitted in build outputs?
|
|
7. Which packer-side metadata is intentionally excluded from the runtime-facing contract?
|
|
8. How should packer document `preload` lifecycle and `offset` semantics so they match runtime reader behavior?
|
|
9. How should per-asset preload declarations be transformed into the `preload` section of `assets.pa` deterministically?
|
|
10. How should the packer preserve both `asset_id` integrity and `asset_name` lookup semantics in runtime-facing artifacts?
|
|
11. Should `asset_table` be modeled as a deterministically ordered asset list that preserves packer `asset_id` values end-to-end?
|
|
12. Which envelope fields are runtime-validated versus emitted only for Studio/tooling use?
|
|
|
|
## Options
|
|
|
|
### Option A
|
|
|
|
Keep `assets.pa` as the authoritative autocontained runtime-facing artifact and emit `build/asset_table.json` as a derived companion artifact for Studio and tooling.
|
|
|
|
### Option B
|
|
|
|
Keep runtime-facing structure split across `assets.pa` and `build/asset_table.json`.
|
|
|
|
## Tradeoffs
|
|
|
|
- Option A matches the runtime specs already in force.
|
|
- Option A keeps runtime authority in one artifact and prevents contract drift.
|
|
- Option A still allows rich tooling through a derived companion file.
|
|
- Option B risks splitting authority across artifacts and making runtime/tooling divergence easier.
|
|
|
|
## Recommendation
|
|
|
|
Adopt Option A with these fixed directions:
|
|
|
|
- `assets.pa` is the authoritative runtime-facing artifact;
|
|
- the `assets.pa` header is the authoritative runtime-facing descriptor;
|
|
- `build/asset_table.json` is a derived companion artifact for Studio, debugging, inspection, and builder/tooling integration;
|
|
- global packing order is deterministic by increasing `asset_id`;
|
|
- preload emission is derived deterministically from per-asset `preload.enabled`;
|
|
- alignment exists only when explicitly required by spec;
|
|
- offsets are always relative to the payload region, never to the start of the full file;
|
|
- checksum is not part of the baseline `assets.pa` envelope contract.
|
|
|
|
Also adopt this performance direction:
|
|
|
|
- usage-based hot-first packing is not part of the baseline contract;
|
|
- deterministic stable ordering is preferred over heuristic physical reordering;
|
|
- future locality optimization may be introduced only through an explicit later decision and spec, not as an implementation shortcut now.
|
|
|
|
Also adopt this identity direction:
|
|
|
|
- runtime-facing preload entries should migrate to stable `asset_id` instead of mutable `asset_name`;
|
|
- runtime-facing preload/bootstrap integrity should treat `asset_id` as primary identity;
|
|
- `asset_name` remains present in `asset_table` as logical descriptive and API-facing metadata used by current runtime/game-facing calls;
|
|
- `asset_table` should be generated deterministically on every build from the current managed asset set;
|
|
- `asset_table` should be emitted as a deterministically ordered list of asset entries;
|
|
- the runtime-facing `asset_id` is exactly the stable `asset_id` allocated by the packer registry;
|
|
- stable cross-build identity comes from `asset_id`, not from table position;
|
|
- rename of `asset_name` is therefore an API-visible/content-visible change, but not an identity change.
|
|
|
|
Also adopt this companion-artifact direction:
|
|
|
|
- `build/asset_table.json` should mirror the runtime-facing header data 1:1 for debugging and inspection;
|
|
- if richer tooling artifacts are needed, they should be emitted as separate files instead of overloading `build/asset_table.json`.
|
|
|
|
## Expected Decisions to Produce
|
|
|
|
1. v1 artifact boundary and ownership.
|
|
2. Deterministic ordering and alignment rules.
|
|
3. Descriptor schema responsibilities.
|
|
4. Materialization requirements for inferred values.
|
|
5. Runtime-facing artifact contract and reader expectations.
|
|
6. Packer-side restatement of the runtime-owned `assets.pa` envelope and slice semantics.
|
|
7. Deterministic mapping from asset declarations to runtime preload data.
|
|
8. Explicit exclusion of usage-based packing heuristics from the baseline contract.
|
|
9. Migration of preload/bootstrap integrity toward `asset_id`-based identity while preserving `asset_name` as current API-facing metadata.
|
|
10. Deterministically ordered `asset_table` preserving packer `asset_id` values end-to-end.
|
|
11. Separation between runtime-validated envelope fields and non-envelope tooling concerns.
|
|
|
|
## Expected Spec Follow-up
|
|
|
|
- Packed ROM artifact spec.
|
|
- Asset table descriptor spec.
|
|
- Determinism and reproducibility spec.
|
|
- Runtime consumption interface spec.
|
|
|
|
## Non-Goals
|
|
|
|
- Asset workspace mutation commands.
|
|
- Quarantine behavior.
|
|
- Builder packaging flow.
|