Intrepid/prometeu-studio

Fork 0

bQUARKz ebbfe311ee

added asset workspace working with packer

2026-03-24 13:42:43 +00:00

14 KiB

Raw Blame History

Prometeu Packer (prometeu-packer) — Specification (Draft)

Status: Draft / Community-facing

This document specifies the Prometeu Packer, the tooling responsible for asset management and producing two build artifacts:

build/assets.pa — the ROM payload containing packed asset bytes

build/asset_table.json — a machine-readable descriptor of the packed assets

The Packer is deliberately agnostic of cartridge shipping. A separate Cartridge Shipper (outside the Packer) consumes build/asset_table.json to generate the final cartridge/manifest.json, copy assets.pa to the cartridge directory, and zip the cartridge into a distributable format.

1. Goals and Non-Goals

1.1 Goals

Be the guardian of sanity in a constantly mutating assets/ workspace.
- Users may be disorganized.
- The directory may contain WIP, junk, unused files, duplicates, outdated exports.
- The Packer must help users identify and fix mistakes.
Provide a robust, deterministic, tooling-grade asset pipeline.
- Stable asset identity.
- Deterministic packing order.
- Reproducible output bytes.
Support both raw (direct) assets and virtual assets.
- Raw assets: the payload in ROM is exactly the source bytes.
- Virtual assets: the payload is derived from multiple inputs via a build pipeline (e.g., PNG + palettes → TILES payload).
Produce an output descriptor (build/asset_table.json) suitable for:
- a Cartridge Shipper to generate the runtime manifest
- CI checks
- a future IDE / GUI tooling
Provide an extensive diagnostics chain (doctor) with structured error codes and suggested fixes.

1.2 Non-Goals

The Packer does not:
- generate cartridge/manifest.json
- decide preload slots
- copy files into cartridge/
- compile PBS bytecode
- zip cartridges

These responsibilities belong to a separate Cartridge Shipper.

2. Repository / Project Layout (Golden Pipeline)

The Prometeu project uses the following canonical structure:

src/ — PBS scripts
assets/ — mutable asset workspace (WIP allowed)
build/ — generated artifacts and caches
cartridge/ — final cartridge directory (produced by Cartridge Shipper)
prometeu.json — project description (dependencies, version, etc.)
sdk/ — SDK/tooling and libraries

The Packer owns the asset workspace metadata under:

assets/.prometeu/ — Packer control directory (registry, cache, quarantine)

3. Crate Topology

3.1 Crates

prometeu-packer
- lib: prometeu_packer_core
- bin: prometeu-packer
  - a thin CLI wrapper delegating to prometeu_packer_core::run()
prometeu dispatcher
- provides a wrapper command prometeup (or integrated subcommand)
- delegates to prometeu-packer for packer operations

3.2 Design Principle

Treat the Packer like the compiler: core library + CLI wrapper.

The core library enables future integrations (IDE, GUI, watch mode) without shelling out.
CLI is a stable interface for users and CI.

4. Mental Model: A “Git-like” Asset Workspace

The Packer treats assets/ like a dirty working tree.

assets/ can contain anything.
Only the assets registered in the Packer registry are considered part of the build.

This is analogous to Git:

working tree (chaos) vs index (truth)

The source of truth for “what counts” is the registry:

assets/.prometeu/index.json

5. Core Concepts

5.1 Managed Asset

A managed asset is an entry in assets/.prometeu/index.json pointing to an asset root directory that contains an anchor file:

asset.json (the asset specification)

Everything else is free-form.

5.2 Asset Identity

Each asset has stable identity:

asset_id: u32 — stable within the project (used by runtime/tooling)
asset_uuid: string — globally unique stable identifier (useful for IDE and future migrations)

Names and paths may change, but identity remains.

5.3 Asset Types (Bank Targets)

Assets ultimately target a bank type in the runtime:

TILES
SOUNDS
(future) SPRITESHEET, MAP, FONT, RAW_BLOB, etc.

The Packer does not define bank memory semantics. It defines the ROM payload and its metadata.

5.4 Raw vs Virtual Assets

Raw assets: ROM payload equals the source bytes.
Virtual assets: ROM payload is derived from input(s) via deterministic build steps.

Examples:

PNG + palette files → TILES payload (indexed pixels + packed palettes)
WAV → PCM16LE payload
Multiple PNGs → atlas spritesheet

6. Directory Structure and Control Files

6.1 Workspace

assets/ is a mutable workspace:

users may create nested organization trees
junk files are allowed

6.2 Control Directory

The Packer stores its truth + tools state in:

assets/
  .prometeu/
    index.json
    cache/
      fingerprints.json
      build-cache.json
    quarantine/
      ...

index.json — registry of managed assets
cache/ — fingerprints and incremental build cache
quarantine/ — optional area to move detected junk (only by explicit user action)

7. Asset Specification: `asset.json`

asset.json describes:

the output ROM payload expected by runtime
the build pipeline (for virtual assets)
metadata needed by runtime/shipper

This spec is modular: each asset format (e.g. TILES/indexed_v1) has its own dedicated specification document.

7.1 Common Fields (All Assets)

schema_version
name
type (bank type)
codec (e.g. RAW; future: compression)
inputs (for virtual assets)
output (format + required metadata)
build (optional pipeline configuration)

7.2 Virtual Asset Pipeline Declaration

Virtual assets must be declared in a way that is:

deterministic
fully materialized (no silent inference)
explicit about defaults (defaults may exist, but must be written into asset.json or build outputs)

8. Build Artifacts Produced by the Packer

8.1 `build/assets.pa`

assets.pa is the ROM asset payload used by the runtime.

Definition: a contiguous binary blob where each managed asset contributes a payload region.

Key Properties

Deterministic asset order (by asset_id)
Offsets are recorded in build/asset_table.json
Alignment rules (configurable by packer, default: no alignment unless required by a format)

Note: assets.pa is intentionally simple.

No internal header is required.
The authoritative structure comes from the descriptor (asset_table.json).

Future versions may introduce chunk tables, but v1 keeps ROM simple.

8.2 `build/asset_table.json`

asset_table.json is the canonical descriptor output of the Packer.

It contains:

assets_pa file info (size, hash)
asset_table[] entries describing each payload slice
optional diagnostics/warnings

Asset Table Entry

An entry describes a ROM slice and its runtime meaning:

asset_id — stable u32
asset_uuid — stable UUID string
asset_name — stable user-facing name
bank_type — e.g. TILES, SOUNDS
offset — byte offset in assets.pa
size — bytes stored in ROM
decoded_size — bytes after decode (equal to size when codec=RAW)
codec — RAW (future: compression)
metadata — format-specific metadata needed by runtime/shipper

Additional tooling fields:

source_root — path to asset dir
inputs — resolved input paths
source_hashes — stable fingerprints of inputs

asset_table.json is machine-readable and designed for:

cartridge shipper consumption
IDE visualization
debugging

9. Determinism Rules

Asset packing order MUST be deterministic.
- Default: increasing asset_id
All derived outputs MUST be deterministic.
- No random seeds unless explicitly declared
- Any seed must be written to output metadata
Default values MUST be materialized.
- If the packer infers something, it must be written into asset.json (via --fix) or recorded in build outputs.

10. Diagnostics and the “Sanity Guardian” Chain

The Packer provides structured diagnostics:

code — stable diagnostic code
severity — error | warning | info
path — affected file
message — human friendly
help — extended context
fixes[] — suggested automated or manual fixes

10.1 Diagnostic Classes

Registered Errors (break build)
- registry entry missing anchor file
- asset.json invalid
- missing inputs
- format/metadata mismatch
Workspace Warnings (does not break build)
- orphaned asset.json (not registered)
- unused large files
- duplicate inputs by hash
Policy Hints (optional)
- naming conventions
- missing preview

10.2 `doctor` Modes

doctor (default) — validate registry only (fast)
doctor --workspace — deep scan workspace (slow)

11. Incremental Build, Cache, and Fingerprints

The Packer maintains fingerprints of inputs:

size
mtime
strong hash (sha256)

Stored in:

assets/.prometeu/cache/fingerprints.json

This enables:

detecting changes
rebuild only what changed
producing stable reports

The cache must never compromise determinism.

12. Quarantine and Garbage Collection

12.1 Quarantine

The Packer can optionally move suspected junk to:

assets/.prometeu/quarantine/

Rules:

Quarantine is never automatic without user consent.
Packer must explain exactly what will be moved.

12.2 Garbage Collection (`gc`)

The Packer can report unused files:

files not referenced by any registered asset
orphaned asset dirs

Actions:

list candidates
optionally move to quarantine
never delete without explicit user request

13. CLI Commands (Comprehensive)

The CLI is a stable interface; all commands are implemented by calling prometeu_packer_core.

13.1 `prometeu packer init`

Creates the control directory and initial registry:

creates assets/.prometeu/index.json
creates caches directory

13.2 `prometeu packer add <path> [--name <name>] [--type <TILES|SOUNDS|...>]`

Registers a new managed asset.

does not require moving files
can create an asset root directory if desired
generates asset.json with explicit defaults
allocates asset_id and asset_uuid

Variants:

add --dir creates a dedicated asset root dir
add --in-place anchors next to the file

13.3 `prometeu packer forget <name|id|uuid>`

Removes an asset from the registry without deleting files.

Useful for WIP and cleanup.

13.4 `prometeu packer rm <name|id|uuid> [--delete]`

Removes the asset from the registry.

default: no deletion
--delete can remove the asset root dir (dangerous; must confirm in UI tooling, or require a force flag in CLI)

13.5 `prometeu packer list`

Lists managed assets:

id, uuid, name
type
status (ok/error)

13.7 `prometeu packer show <name|id|uuid>`

Shows detailed information:

resolved inputs
metadata
fingerprints
last build summary

13.8 `prometeu packer doctor [--workspace] [--strict] [--fix]`

Runs diagnostics:

--workspace deep scan
--strict warnings become errors
--fix applies safe automatic fixes (materialize defaults, normalize paths)

13.9 `prometeu packer build [--out build/assets.pa] [--table build/asset_table.json]`

Builds:

build/assets.pa
build/asset_table.json

Key behaviors:

validates registry before packing
packs assets deterministically
for virtual assets, runs build pipelines
records all offsets and metadata

13.10 `prometeu packer watch`

Watches registered inputs and registry changes.

emits events (future)
rebuilds incrementally

watch is optional in v0 but recommended.

13.11 `prometeu packer gc [--workspace] [--quarantine]`

Reports unused files.

default: report only
--quarantine moves candidates to quarantine

13.12 `prometeu packer quarantine <path> [--restore]`

Moves or restores files into/from quarantine.

14. Virtual Assets (Deep Explanation)

Virtual assets are a major capability.

14.1 Why Virtual Assets

Most runtime formats should be derived from human-friendly authoring formats.
Example:
- author uses source.png and palette files
- runtime expects indexed pixels + packed RGB565 palettes

14.2 Virtual Asset Contract

Inputs are explicit.
Build steps are deterministic.
Outputs match a well-defined runtime payload format.

14.3 Examples of Future Virtual Assets

TILES/indexed_v1: PNG + palette files → indexed pixels + packed palettes
SOUNDS/pcm16le_v1: WAV → PCM16LE
SPRITESHEET/atlas_v1: multiple PNG frames → atlas + metadata

Each output.format must have its own dedicated spec.

15. Integration with Cartridge Shipper

The Cartridge Shipper should:

Compile PBS into bytecode (e.g. program.pbc / program.pbx)
Call prometeu packer build
Consume build/asset_table.json and produce cartridge/manifest.json
Copy artifacts into cartridge/
Zip the cartridge into a distributable format (.crt / .rom / .pro)

The packer never touches cartridge/.

16. Compatibility and Versioning

assets/.prometeu/index.json has schema_version
asset.json has schema_version
build/asset_table.json has schema_version

The Packer must be able to migrate older schema versions or emit actionable diagnostics.

17. Security and Trust Model

The Packer is offline tooling.
It must never execute untrusted scripts.
It should treat external inputs as untrusted data.

18. Implementation Notes (Non-Normative)

Rust implementation with a core crate + CLI wrapper.
Prefer structured JSON serde models.
Use stable diagnostic codes.
Keep the build deterministic.

Appendix A — Glossary

ROM (assets.pa): packed asset payload used by runtime
Descriptor (asset_table.json): mapping from logical assets to ROM slices
Managed asset: registered asset with stable identity and anchor file
Virtual asset: derived asset built from multiple inputs
Quarantine: safe area for suspected junk
Doctor: diagnostic command to keep sanity

14 KiB Raw Blame History