fix structs
This commit is contained in:
@@ -0,0 +1,564 @@
|
||||
# Prometeu Packer (prometeu-packer) — Specification (Draft)
|
||||
|
||||
> **Status:** Draft / Community-facing
|
||||
>
|
||||
> This document specifies the **Prometeu Packer**, the tooling responsible for **asset management** and producing two build artifacts:
|
||||
>
|
||||
> * `build/assets.pa` — the ROM payload containing packed asset bytes
|
||||
> * `build/asset_table.json` — a machine-readable descriptor of the packed assets
|
||||
>
|
||||
> The Packer is deliberately **agnostic of cartridge building**. A separate **Cartridge Builder** (outside the Packer) consumes `build/asset_table.json` to generate the final `cartridge/manifest.json`, copy `assets.pa` to the cartridge directory, and zip the cartridge into a distributable format.
|
||||
|
||||
---
|
||||
|
||||
## 1. Goals and Non-Goals
|
||||
|
||||
### 1.1 Goals
|
||||
|
||||
1. **Be the guardian of sanity** in a constantly mutating `assets/` workspace.
|
||||
|
||||
* Users may be disorganized.
|
||||
* The directory may contain WIP, junk, unused files, duplicates, outdated exports.
|
||||
* The Packer must help users identify and fix mistakes.
|
||||
|
||||
2. Provide a robust, deterministic, **tooling-grade** asset pipeline.
|
||||
|
||||
* Stable asset identity.
|
||||
* Deterministic packing order.
|
||||
* Reproducible output bytes.
|
||||
|
||||
3. Support both **raw (direct) assets** and **virtual assets**.
|
||||
|
||||
* Raw assets: the payload in ROM is exactly the source bytes.
|
||||
* Virtual assets: the payload is derived from multiple inputs via a build pipeline (e.g., PNG + palettes → `TILES` payload).
|
||||
|
||||
4. Produce an output descriptor (`build/asset_table.json`) suitable for:
|
||||
|
||||
* a Cartridge Builder to generate the runtime manifest
|
||||
* CI checks
|
||||
* a future IDE / GUI tooling
|
||||
|
||||
5. Provide an extensive **diagnostics chain** (doctor) with structured error codes and suggested fixes.
|
||||
|
||||
### 1.2 Non-Goals
|
||||
|
||||
* The Packer **does not**:
|
||||
|
||||
* generate `cartridge/manifest.json`
|
||||
* decide preload slots
|
||||
* copy files into `cartridge/`
|
||||
* compile PBS bytecode
|
||||
* zip cartridges
|
||||
|
||||
These responsibilities belong to a separate **Cartridge Builder**.
|
||||
|
||||
---
|
||||
|
||||
## 2. Repository / Project Layout (Golden Pipeline)
|
||||
|
||||
The Prometeu project uses the following canonical structure:
|
||||
|
||||
* `src/` — PBS scripts
|
||||
* `assets/` — mutable asset workspace (WIP allowed)
|
||||
* `build/` — generated artifacts and caches
|
||||
* `cartridge/` — final cartridge directory (produced by Cartridge Builder)
|
||||
* `prometeu.json` — project description (dependencies, version, etc.)
|
||||
* `sdk/` — SDK/tooling and libraries
|
||||
|
||||
The Packer owns the asset workspace metadata under:
|
||||
|
||||
* `assets/.prometeu/` — Packer control directory (registry, cache, quarantine)
|
||||
|
||||
---
|
||||
|
||||
## 3. Crate Topology
|
||||
|
||||
### 3.1 Crates
|
||||
|
||||
* **`prometeu-packer`**
|
||||
|
||||
* **lib**: `prometeu_packer_core`
|
||||
* **bin**: `prometeu-packer`
|
||||
|
||||
* a thin CLI wrapper delegating to `prometeu_packer_core::run()`
|
||||
|
||||
* **`prometeu` dispatcher**
|
||||
|
||||
* provides a wrapper command **`prometeup`** (or integrated subcommand)
|
||||
* delegates to `prometeu-packer` for packer operations
|
||||
|
||||
### 3.2 Design Principle
|
||||
|
||||
Treat the Packer like the compiler: core library + CLI wrapper.
|
||||
|
||||
* The core library enables future integrations (IDE, GUI, watch mode) without shelling out.
|
||||
* CLI is a stable interface for users and CI.
|
||||
|
||||
---
|
||||
|
||||
## 4. Mental Model: A “Git-like” Asset Workspace
|
||||
|
||||
The Packer treats `assets/` like a **dirty working tree**.
|
||||
|
||||
* `assets/` can contain *anything*.
|
||||
* Only the assets registered in the Packer registry are considered part of the build.
|
||||
|
||||
This is analogous to Git:
|
||||
|
||||
* working tree (chaos) vs index (truth)
|
||||
|
||||
The **source of truth** for “what counts” is the registry:
|
||||
|
||||
* `assets/.prometeu/index.json`
|
||||
|
||||
---
|
||||
|
||||
## 5. Core Concepts
|
||||
|
||||
### 5.1 Managed Asset
|
||||
|
||||
A **managed asset** is an entry in `assets/.prometeu/index.json` pointing to an **asset root directory** that contains an anchor file:
|
||||
|
||||
* `asset.json` (the asset specification)
|
||||
|
||||
Everything else is free-form.
|
||||
|
||||
### 5.2 Asset Identity
|
||||
|
||||
Each asset has stable identity:
|
||||
|
||||
* `asset_id: u32` — stable within the project (used by runtime/tooling)
|
||||
* `asset_uuid: string` — globally unique stable identifier (useful for IDE and future migrations)
|
||||
|
||||
Names and paths may change, but identity remains.
|
||||
|
||||
### 5.3 Asset Types (Bank Targets)
|
||||
|
||||
Assets ultimately target a **bank type** in the runtime:
|
||||
|
||||
* `TILES`
|
||||
* `SOUNDS`
|
||||
* (future) `SPRITESHEET`, `MAP`, `FONT`, `RAW_BLOB`, etc.
|
||||
|
||||
The Packer does **not** define bank memory semantics. It defines the *ROM payload* and its metadata.
|
||||
|
||||
### 5.4 Raw vs Virtual Assets
|
||||
|
||||
* **Raw assets**: ROM payload equals the source bytes.
|
||||
* **Virtual assets**: ROM payload is derived from input(s) via deterministic build steps.
|
||||
|
||||
Examples:
|
||||
|
||||
* PNG + palette files → `TILES` payload (indexed pixels + packed palettes)
|
||||
* WAV → PCM16LE payload
|
||||
* Multiple PNGs → atlas spritesheet
|
||||
|
||||
---
|
||||
|
||||
## 6. Directory Structure and Control Files
|
||||
|
||||
### 6.1 Workspace
|
||||
|
||||
`assets/` is a mutable workspace:
|
||||
|
||||
* users may create nested organization trees
|
||||
* junk files are allowed
|
||||
|
||||
### 6.2 Control Directory
|
||||
|
||||
The Packer stores its truth + tools state in:
|
||||
|
||||
```
|
||||
assets/
|
||||
.prometeu/
|
||||
index.json
|
||||
cache/
|
||||
fingerprints.json
|
||||
build-cache.json
|
||||
quarantine/
|
||||
...
|
||||
```
|
||||
|
||||
* `index.json` — registry of managed assets
|
||||
* `cache/` — fingerprints and incremental build cache
|
||||
* `quarantine/` — optional area to move detected junk (only by explicit user action)
|
||||
|
||||
---
|
||||
|
||||
## 7. Asset Specification: `asset.json`
|
||||
|
||||
`asset.json` describes:
|
||||
|
||||
1. **the output ROM payload** expected by runtime
|
||||
2. **the build pipeline** (for virtual assets)
|
||||
3. **metadata** needed by runtime/builder
|
||||
|
||||
This spec is modular: **each asset format** (e.g. `TILES/indexed_v1`) has its own dedicated specification document.
|
||||
|
||||
### 7.1 Common Fields (All Assets)
|
||||
|
||||
* `schema_version`
|
||||
* `name`
|
||||
* `type` (bank type)
|
||||
* `codec` (e.g. `RAW`; future: compression)
|
||||
* `inputs` (for virtual assets)
|
||||
* `output` (format + required metadata)
|
||||
* `build` (optional pipeline configuration)
|
||||
|
||||
### 7.2 Virtual Asset Pipeline Declaration
|
||||
|
||||
Virtual assets must be declared in a way that is:
|
||||
|
||||
* deterministic
|
||||
* fully materialized (no silent inference)
|
||||
* explicit about defaults (defaults may exist, but must be written into `asset.json` or build outputs)
|
||||
|
||||
---
|
||||
|
||||
## 8. Build Artifacts Produced by the Packer
|
||||
|
||||
### 8.1 `build/assets.pa`
|
||||
|
||||
**`assets.pa`** is the ROM asset payload used by the runtime.
|
||||
|
||||
**Definition:** a contiguous binary blob where each managed asset contributes a payload region.
|
||||
|
||||
#### Key Properties
|
||||
|
||||
* Deterministic asset order (by `asset_id`)
|
||||
* Offsets are recorded in `build/asset_table.json`
|
||||
* Alignment rules (configurable by packer, default: no alignment unless required by a format)
|
||||
|
||||
**Note:** `assets.pa` is intentionally simple.
|
||||
|
||||
* No internal header is required.
|
||||
* The authoritative structure comes from the descriptor (`asset_table.json`).
|
||||
|
||||
Future versions may introduce chunk tables, but v1 keeps ROM simple.
|
||||
|
||||
### 8.2 `build/asset_table.json`
|
||||
|
||||
**`asset_table.json`** is the canonical descriptor output of the Packer.
|
||||
|
||||
It contains:
|
||||
|
||||
* `assets_pa` file info (size, hash)
|
||||
* `asset_table[]` entries describing each payload slice
|
||||
* optional diagnostics/warnings
|
||||
|
||||
#### Asset Table Entry
|
||||
|
||||
An entry describes a ROM slice and its runtime meaning:
|
||||
|
||||
* `asset_id` — stable u32
|
||||
* `asset_uuid` — stable UUID string
|
||||
* `asset_name` — stable user-facing name
|
||||
* `bank_type` — e.g. `TILES`, `SOUNDS`
|
||||
* `offset` — byte offset in `assets.pa`
|
||||
* `size` — bytes stored in ROM
|
||||
* `decoded_size` — bytes after decode (equal to `size` when `codec=RAW`)
|
||||
* `codec` — `RAW` (future: compression)
|
||||
* `metadata` — format-specific metadata needed by runtime/builder
|
||||
|
||||
Additional tooling fields:
|
||||
|
||||
* `source_root` — path to asset dir
|
||||
* `inputs` — resolved input paths
|
||||
* `source_hashes` — stable fingerprints of inputs
|
||||
|
||||
`asset_table.json` is machine-readable and designed for:
|
||||
|
||||
* cartridge builder consumption
|
||||
* IDE visualization
|
||||
* debugging
|
||||
|
||||
---
|
||||
|
||||
## 9. Determinism Rules
|
||||
|
||||
1. Asset packing order MUST be deterministic.
|
||||
|
||||
* Default: increasing `asset_id`
|
||||
|
||||
2. All derived outputs MUST be deterministic.
|
||||
|
||||
* No random seeds unless explicitly declared
|
||||
* Any seed must be written to output metadata
|
||||
|
||||
3. Default values MUST be materialized.
|
||||
|
||||
* If the packer infers something, it must be written into `asset.json` (via `--fix`) or recorded in build outputs.
|
||||
|
||||
---
|
||||
|
||||
## 10. Diagnostics and the “Sanity Guardian” Chain
|
||||
|
||||
The Packer provides structured diagnostics:
|
||||
|
||||
* `code` — stable diagnostic code
|
||||
* `severity` — `error | warning | info`
|
||||
* `path` — affected file
|
||||
* `message` — human friendly
|
||||
* `help` — extended context
|
||||
* `fixes[]` — suggested automated or manual fixes
|
||||
|
||||
### 10.1 Diagnostic Classes
|
||||
|
||||
1. **Registered Errors** (break build)
|
||||
|
||||
* registry entry missing anchor file
|
||||
* `asset.json` invalid
|
||||
* missing inputs
|
||||
* format/metadata mismatch
|
||||
|
||||
2. **Workspace Warnings** (does not break build)
|
||||
|
||||
* orphaned `asset.json` (not registered)
|
||||
* unused large files
|
||||
* duplicate inputs by hash
|
||||
|
||||
3. **Policy Hints** (optional)
|
||||
|
||||
* naming conventions
|
||||
* missing preview
|
||||
|
||||
### 10.2 `doctor` Modes
|
||||
|
||||
* `doctor` (default) — validate registry only (fast)
|
||||
* `doctor --workspace` — deep scan workspace (slow)
|
||||
|
||||
---
|
||||
|
||||
## 11. Incremental Build, Cache, and Fingerprints
|
||||
|
||||
The Packer maintains fingerprints of inputs:
|
||||
|
||||
* size
|
||||
* mtime
|
||||
* strong hash (sha256)
|
||||
|
||||
Stored in:
|
||||
|
||||
* `assets/.prometeu/cache/fingerprints.json`
|
||||
|
||||
This enables:
|
||||
|
||||
* detecting changes
|
||||
* rebuild only what changed
|
||||
* producing stable reports
|
||||
|
||||
The cache must never compromise determinism.
|
||||
|
||||
---
|
||||
|
||||
## 12. Quarantine and Garbage Collection
|
||||
|
||||
### 12.1 Quarantine
|
||||
|
||||
The Packer can optionally move suspected junk to:
|
||||
|
||||
* `assets/.prometeu/quarantine/`
|
||||
|
||||
Rules:
|
||||
|
||||
* Quarantine is **never automatic** without user consent.
|
||||
* Packer must explain exactly what will be moved.
|
||||
|
||||
### 12.2 Garbage Collection (`gc`)
|
||||
|
||||
The Packer can report unused files:
|
||||
|
||||
* files not referenced by any registered asset
|
||||
* orphaned asset dirs
|
||||
|
||||
Actions:
|
||||
|
||||
* list candidates
|
||||
* optionally move to quarantine
|
||||
* never delete without explicit user request
|
||||
|
||||
---
|
||||
|
||||
## 13. CLI Commands (Comprehensive)
|
||||
|
||||
> The CLI is a stable interface; all commands are implemented by calling `prometeu_packer_core`.
|
||||
|
||||
### 13.1 `prometeu packer init`
|
||||
|
||||
Creates the control directory and initial registry:
|
||||
|
||||
* creates `assets/.prometeu/index.json`
|
||||
* creates caches directory
|
||||
|
||||
### 13.2 `prometeu packer add <path> [--name <name>] [--type <TILES|SOUNDS|...>]`
|
||||
|
||||
Registers a new managed asset.
|
||||
|
||||
* does not require moving files
|
||||
* can create an asset root directory if desired
|
||||
* generates `asset.json` with explicit defaults
|
||||
* allocates `asset_id` and `asset_uuid`
|
||||
|
||||
Variants:
|
||||
|
||||
* `add --dir` creates a dedicated asset root dir
|
||||
* `add --in-place` anchors next to the file
|
||||
|
||||
### 13.3 `prometeu packer adopt`
|
||||
|
||||
Scans workspace for unregistered `asset.json` anchors and offers to register them.
|
||||
|
||||
* default: dry-run list
|
||||
* `--apply` registers them
|
||||
|
||||
### 13.4 `prometeu packer forget <name|id|uuid>`
|
||||
|
||||
Removes an asset from the registry without deleting files.
|
||||
|
||||
Useful for WIP and cleanup.
|
||||
|
||||
### 13.5 `prometeu packer rm <name|id|uuid> [--delete]`
|
||||
|
||||
Removes the asset from the registry.
|
||||
|
||||
* default: no deletion
|
||||
* `--delete` can remove the asset root dir (dangerous; must confirm in UI tooling, or require a force flag in CLI)
|
||||
|
||||
### 13.6 `prometeu packer list`
|
||||
|
||||
Lists managed assets:
|
||||
|
||||
* id, uuid, name
|
||||
* type
|
||||
* status (ok/error)
|
||||
|
||||
### 13.7 `prometeu packer show <name|id|uuid>`
|
||||
|
||||
Shows detailed information:
|
||||
|
||||
* resolved inputs
|
||||
* metadata
|
||||
* fingerprints
|
||||
* last build summary
|
||||
|
||||
### 13.8 `prometeu packer doctor [--workspace] [--strict] [--fix]`
|
||||
|
||||
Runs diagnostics:
|
||||
|
||||
* `--workspace` deep scan
|
||||
* `--strict` warnings become errors
|
||||
* `--fix` applies safe automatic fixes (materialize defaults, normalize paths)
|
||||
|
||||
### 13.9 `prometeu packer build [--out build/assets.pa] [--table build/asset_table.json]`
|
||||
|
||||
Builds:
|
||||
|
||||
* `build/assets.pa`
|
||||
* `build/asset_table.json`
|
||||
|
||||
Key behaviors:
|
||||
|
||||
* validates registry before packing
|
||||
* packs assets deterministically
|
||||
* for virtual assets, runs build pipelines
|
||||
* records all offsets and metadata
|
||||
|
||||
### 13.10 `prometeu packer watch`
|
||||
|
||||
Watches registered inputs and registry changes.
|
||||
|
||||
* emits events (future)
|
||||
* rebuilds incrementally
|
||||
|
||||
`watch` is optional in v0 but recommended.
|
||||
|
||||
### 13.11 `prometeu packer gc [--workspace] [--quarantine]`
|
||||
|
||||
Reports unused files.
|
||||
|
||||
* default: report only
|
||||
* `--quarantine` moves candidates to quarantine
|
||||
|
||||
### 13.12 `prometeu packer quarantine <path> [--restore]`
|
||||
|
||||
Moves or restores files into/from quarantine.
|
||||
|
||||
---
|
||||
|
||||
## 14. Virtual Assets (Deep Explanation)
|
||||
|
||||
Virtual assets are a major capability.
|
||||
|
||||
### 14.1 Why Virtual Assets
|
||||
|
||||
* Most runtime formats should be derived from human-friendly authoring formats.
|
||||
* Example:
|
||||
|
||||
* author uses `source.png` and palette files
|
||||
* runtime expects indexed pixels + packed RGB565 palettes
|
||||
|
||||
### 14.2 Virtual Asset Contract
|
||||
|
||||
* Inputs are explicit.
|
||||
* Build steps are deterministic.
|
||||
* Outputs match a well-defined runtime payload format.
|
||||
|
||||
### 14.3 Examples of Future Virtual Assets
|
||||
|
||||
* `TILES/indexed_v1`: PNG + palette files → indexed pixels + packed palettes
|
||||
* `SOUNDS/pcm16le_v1`: WAV → PCM16LE
|
||||
* `SPRITESHEET/atlas_v1`: multiple PNG frames → atlas + metadata
|
||||
|
||||
Each `output.format` must have its own dedicated spec.
|
||||
|
||||
---
|
||||
|
||||
## 15. Integration with Cartridge Builder
|
||||
|
||||
The Cartridge Builder should:
|
||||
|
||||
1. Compile PBS into bytecode (e.g. `program.pbc` / `program.pbx`)
|
||||
2. Call `prometeu packer build`
|
||||
3. Consume `build/asset_table.json` and produce `cartridge/manifest.json`
|
||||
4. Copy artifacts into `cartridge/`
|
||||
5. Zip the cartridge into a distributable format (`.crt` / `.rom` / `.pro`)
|
||||
|
||||
The packer never touches `cartridge/`.
|
||||
|
||||
---
|
||||
|
||||
## 16. Compatibility and Versioning
|
||||
|
||||
* `assets/.prometeu/index.json` has `schema_version`
|
||||
* `asset.json` has `schema_version`
|
||||
* `build/asset_table.json` has `schema_version`
|
||||
|
||||
The Packer must be able to migrate older schema versions or emit actionable diagnostics.
|
||||
|
||||
---
|
||||
|
||||
## 17. Security and Trust Model
|
||||
|
||||
* The Packer is offline tooling.
|
||||
* It must never execute untrusted scripts.
|
||||
* It should treat external inputs as untrusted data.
|
||||
|
||||
---
|
||||
|
||||
## 18. Implementation Notes (Non-Normative)
|
||||
|
||||
* Rust implementation with a core crate + CLI wrapper.
|
||||
* Prefer structured JSON serde models.
|
||||
* Use stable diagnostic codes.
|
||||
* Keep the build deterministic.
|
||||
|
||||
---
|
||||
|
||||
## Appendix A — Glossary
|
||||
|
||||
* **ROM (`assets.pa`)**: packed asset payload used by runtime
|
||||
* **Descriptor (`asset_table.json`)**: mapping from logical assets to ROM slices
|
||||
* **Managed asset**: registered asset with stable identity and anchor file
|
||||
* **Virtual asset**: derived asset built from multiple inputs
|
||||
* **Quarantine**: safe area for suspected junk
|
||||
* **Doctor**: diagnostic command to keep sanity
|
||||
Reference in New Issue
Block a user