From f27b28c5163d87f66f596960a6d1089abfb2a83c Mon Sep 17 00:00:00 2001 From: bQUARKz Date: Wed, 11 Mar 2026 09:32:22 +0000 Subject: [PATCH] updated packer agendas --- docs/packer/Prometeu Packer.md | 18 +- ... Boundaries, and Golden Pipeline Agenda.md | 16 +- ...sets, and Virtual Asset Contract Agenda.md | 2 +- ...ifacts and Deterministic Packing Agenda.md | 4 +- ...uarantine, and Workspace Hygiene Agenda.md | 4 +- ...al Build, Cache, and Watch Model Agenda.md | 6 +- ... Surface and Mutating Operations Agenda.md | 68 +++-- ...ion, Versioning, and Trust Model Agenda.md | 20 +- docs/packer/agendas/README.md | 13 +- ...arantine-and-workspace-hygiene-decision.md | 217 ++++++++++++++ ...al-build-cache-and-watch-model-decision.md | 201 +++++++++++++ ...-asset-services-and-event-lane-decision.md | 278 ++++++++++++++++++ ...ion-versioning-and-trust-model-decision.md | 223 ++++++++++++++ docs/packer/pull-requests/README.md | 2 +- 14 files changed, 1008 insertions(+), 64 deletions(-) create mode 100644 docs/packer/decisions/004-diagnostics-doctor-quarantine-and-workspace-hygiene-decision.md create mode 100644 docs/packer/decisions/005-incremental-build-cache-and-watch-model-decision.md create mode 100644 docs/packer/decisions/006-studio-first-asset-services-and-event-lane-decision.md create mode 100644 docs/packer/decisions/007-shipper-integration-versioning-and-trust-model-decision.md diff --git a/docs/packer/Prometeu Packer.md b/docs/packer/Prometeu Packer.md index 294e1cfd..39142fb0 100644 --- a/docs/packer/Prometeu Packer.md +++ b/docs/packer/Prometeu Packer.md @@ -7,7 +7,7 @@ > * `build/assets.pa` — the ROM payload containing packed asset bytes > * `build/asset_table.json` — a machine-readable descriptor of the packed assets > -> The Packer is deliberately **agnostic of cartridge building**. A separate **Cartridge Builder** (outside the Packer) consumes `build/asset_table.json` to generate the final `cartridge/manifest.json`, copy `assets.pa` to the cartridge directory, and zip the cartridge into a distributable format. +> The Packer is deliberately **agnostic of cartridge shipping**. A separate **Cartridge Shipper** (outside the Packer) consumes `build/asset_table.json` to generate the final `cartridge/manifest.json`, copy `assets.pa` to the cartridge directory, and zip the cartridge into a distributable format. --- @@ -34,7 +34,7 @@ 4. Produce an output descriptor (`build/asset_table.json`) suitable for: - * a Cartridge Builder to generate the runtime manifest + * a Cartridge Shipper to generate the runtime manifest * CI checks * a future IDE / GUI tooling @@ -50,7 +50,7 @@ * compile PBS bytecode * zip cartridges -These responsibilities belong to a separate **Cartridge Builder**. +These responsibilities belong to a separate **Cartridge Shipper**. --- @@ -61,7 +61,7 @@ The Prometeu project uses the following canonical structure: * `src/` — PBS scripts * `assets/` — mutable asset workspace (WIP allowed) * `build/` — generated artifacts and caches -* `cartridge/` — final cartridge directory (produced by Cartridge Builder) +* `cartridge/` — final cartridge directory (produced by Cartridge Shipper) * `prometeu.json` — project description (dependencies, version, etc.) * `sdk/` — SDK/tooling and libraries @@ -191,7 +191,7 @@ assets/ 1. **the output ROM payload** expected by runtime 2. **the build pipeline** (for virtual assets) -3. **metadata** needed by runtime/builder +3. **metadata** needed by runtime/shipper This spec is modular: **each asset format** (e.g. `TILES/indexed_v1`) has its own dedicated specification document. @@ -258,7 +258,7 @@ An entry describes a ROM slice and its runtime meaning: * `size` — bytes stored in ROM * `decoded_size` — bytes after decode (equal to `size` when `codec=RAW`) * `codec` — `RAW` (future: compression) -* `metadata` — format-specific metadata needed by runtime/builder +* `metadata` — format-specific metadata needed by runtime/shipper Additional tooling fields: @@ -268,7 +268,7 @@ Additional tooling fields: `asset_table.json` is machine-readable and designed for: -* cartridge builder consumption +* cartridge shipper consumption * IDE visualization * debugging @@ -513,9 +513,9 @@ Each `output.format` must have its own dedicated spec. --- -## 15. Integration with Cartridge Builder +## 15. Integration with Cartridge Shipper -The Cartridge Builder should: +The Cartridge Shipper should: 1. Compile PBS into bytecode (e.g. `program.pbc` / `program.pbx`) 2. Call `prometeu packer build` diff --git a/docs/packer/agendas/01.0. Packer Scope, Boundaries, and Golden Pipeline Agenda.md b/docs/packer/agendas/01.0. Packer Scope, Boundaries, and Golden Pipeline Agenda.md index 578872b3..8662598a 100644 --- a/docs/packer/agendas/01.0. Packer Scope, Boundaries, and Golden Pipeline Agenda.md +++ b/docs/packer/agendas/01.0. Packer Scope, Boundaries, and Golden Pipeline Agenda.md @@ -18,7 +18,7 @@ Define the architectural boundary of the `packer` domain before the draft is dec - packer must produce artifacts that can be consumed by `../runtime`, - packer operations must be available as Studio-integrated services, - packer does not build the cartridge, -- cartridge builder consumes packer outputs later. +- shipper consumes packer outputs later. This is a good draft direction, but it still needs to be closed as an explicit domain contract. The interface between packer outputs and `../runtime` should be documented first, before it is treated as implementation detail. @@ -30,7 +30,7 @@ The runtime already exposes normative constraints that the packer must treat as - `1. Goals and Non-Goals` - `2. Repository / Project Layout (Golden Pipeline)` - `3. Crate Topology` -- `15. Integration with Cartridge Builder` +- `15. Integration with Cartridge Shipper` ## Runtime Normative Inputs @@ -56,7 +56,7 @@ These points are already normative on the runtime side and should be treated as ## Key Questions -1. What is the exact ownership boundary between `packer` and `cartridge builder`? +1. What is the exact ownership boundary between `packer` and `shipper`? 2. Should `packer` be defined purely as offline asset tooling, or as a Studio-integrated service surface with optional CLI adapters? 3. Which project directories are normative for packer behavior, and which are merely conventional? 4. Should packer be allowed to read `src/` or compiler outputs at all, or only operate on `assets/` plus project metadata? @@ -77,12 +77,12 @@ Allow packer to absorb more cartridge preparation responsibilities over time. ## Tradeoffs - Option A keeps the architecture cleaner and reduces domain leakage. -- Option A makes builder integration more explicit and easier to spec. -- Option B may feel convenient short-term, but it risks collapsing packer and builder concerns into one blurred tool boundary. +- Option A makes shipper integration more explicit and easier to spec. +- Option B may feel convenient short-term, but it risks collapsing packer and shipper concerns into one blurred tool boundary. ## Recommendation -Adopt Option A and make the packer/builder boundary explicit as a first-class architectural invariant. +Adopt Option A and make the packer/shipper boundary explicit as a first-class architectural invariant. Also adopt this operational direction: @@ -102,7 +102,7 @@ Also adopt this operational direction: ## Expected Spec Follow-up - Packer domain charter or overview spec. -- Builder integration spec. +- Shipper integration spec. - Artifact ownership and responsibility spec. - Runtime artifact interface spec. @@ -110,4 +110,4 @@ Also adopt this operational direction: - Defining the detailed schema of `asset_table.json`. - Defining individual asset formats. -- Designing cartridge builder internals. +- Designing cartridge shipper internals. diff --git a/docs/packer/agendas/01.2. Asset Specification, Raw Assets, and Virtual Asset Contract Agenda.md b/docs/packer/agendas/01.2. Asset Specification, Raw Assets, and Virtual Asset Contract Agenda.md index 1167fa48..24572ca8 100644 --- a/docs/packer/agendas/01.2. Asset Specification, Raw Assets, and Virtual Asset Contract Agenda.md +++ b/docs/packer/agendas/01.2. Asset Specification, Raw Assets, and Virtual Asset Contract Agenda.md @@ -137,4 +137,4 @@ Direction: - Final CLI UX for authoring these files. - Incremental build internals. -- Cartridge builder consumption details. +- Cartridge shipper consumption details. diff --git a/docs/packer/agendas/01.3. Build Artifacts and Deterministic Packing Agenda.md b/docs/packer/agendas/01.3. Build Artifacts and Deterministic Packing Agenda.md index 8eae0309..3808a903 100644 --- a/docs/packer/agendas/01.3. Build Artifacts and Deterministic Packing Agenda.md +++ b/docs/packer/agendas/01.3. Build Artifacts and Deterministic Packing Agenda.md @@ -122,7 +122,7 @@ Adopt Option A with these fixed directions: - `assets.pa` is the authoritative runtime-facing artifact; - the `assets.pa` header is the authoritative runtime-facing descriptor; -- `build/asset_table.json` is a derived companion artifact for Studio, debugging, inspection, and builder/tooling integration; +- `build/asset_table.json` is a derived companion artifact for Studio, debugging, inspection, and shipper/tooling integration; - global packing order is deterministic by increasing `asset_id`; - preload emission is derived deterministically from per-asset `preload.enabled`; - alignment exists only when explicitly required by spec; @@ -176,4 +176,4 @@ Also adopt this companion-artifact direction: - Asset workspace mutation commands. - Quarantine behavior. -- Builder packaging flow. +- Shipper packaging flow. diff --git a/docs/packer/agendas/01.4. Diagnostics, Doctor, Quarantine, and Workspace Hygiene Agenda.md b/docs/packer/agendas/01.4. Diagnostics, Doctor, Quarantine, and Workspace Hygiene Agenda.md index 2e965134..c96116d4 100644 --- a/docs/packer/agendas/01.4. Diagnostics, Doctor, Quarantine, and Workspace Hygiene Agenda.md +++ b/docs/packer/agendas/01.4. Diagnostics, Doctor, Quarantine, and Workspace Hygiene Agenda.md @@ -2,7 +2,9 @@ ## Status -Open +Closed + +Resolved by [`004-diagnostics-doctor-quarantine-and-workspace-hygiene-decision.md`](../decisions/004-diagnostics-doctor-quarantine-and-workspace-hygiene-decision.md). ## Purpose diff --git a/docs/packer/agendas/01.5. Incremental Build, Cache, and Watch Model Agenda.md b/docs/packer/agendas/01.5. Incremental Build, Cache, and Watch Model Agenda.md index dec4782a..25b1dfa8 100644 --- a/docs/packer/agendas/01.5. Incremental Build, Cache, and Watch Model Agenda.md +++ b/docs/packer/agendas/01.5. Incremental Build, Cache, and Watch Model Agenda.md @@ -2,7 +2,9 @@ ## Status -Open +Closed + +Resolved by [`005-incremental-build-cache-and-watch-model-decision.md`](../decisions/005-incremental-build-cache-and-watch-model-decision.md). ## Purpose @@ -74,4 +76,4 @@ Adopt Option A: - Full CLI command catalog. - Asset registry identity rules. -- Cartridge builder integration. +- Cartridge shipper integration. diff --git a/docs/packer/agendas/01.6. CLI Surface and Mutating Operations Agenda.md b/docs/packer/agendas/01.6. CLI Surface and Mutating Operations Agenda.md index 277b6edb..672c1786 100644 --- a/docs/packer/agendas/01.6. CLI Surface and Mutating Operations Agenda.md +++ b/docs/packer/agendas/01.6. CLI Surface and Mutating Operations Agenda.md @@ -2,11 +2,13 @@ ## Status -Open +Closed + +Resolved by [`006-studio-first-asset-services-and-event-lane-decision.md`](../decisions/006-studio-first-asset-services-and-event-lane-decision.md). ## Purpose -Define the stable service and adapter contract for packer operations, especially operations that mutate registry state or user files. +Define the stable Studio-facing service contract for packer operations, especially operations that mutate registry state or user files. ## Context @@ -26,13 +28,15 @@ The draft already lists a substantial command surface: - `quarantine` Historically, packer was framed as a CLI-first tool. -The current direction is different: +The current target is different: - packer should be fully integrated with Studio; -- operations should exist as services; -- CLI, if retained, should be an adapter over those services. +- asset UI and packer services should evolve together; +- operations should exist as Studio-consumable services first; +- asset workflows should be observable through a dedicated background event lane for UI integration; +- CLI, if retained, should be treated as a later adapter rather than the design center. -This is enough surface area that operation semantics need architectural review before turning into normative service and CLI contracts. +This means the architectural work should focus on service semantics and UI integration points first, not on freezing a command-line contract too early. ## Source Sections @@ -40,55 +44,75 @@ This is enough surface area that operation semantics need architectural review b ## Key Questions -1. Which operations are core v1 and which are aspirational? -2. Which identifiers must every command accept: name, id, uuid, path? +1. Which packer operations are core Studio services in v1 and which are later capabilities? +2. Which identifiers should Studio-facing services accept for normal asset workflows: name, id, uuid, root path? 3. Which operations mutate registry only versus mutate workspace files? -4. What requires confirmation, force flags, or dry-run support? -5. How explicit should `--fix` and auto-materialization behavior be? -6. Which parts of the contract belong to Studio services versus optional CLI adapters? -7. Should adapter output contracts be stable for automation, or only JSON/structured output modes? +4. What requires explicit confirmation, preview, or staged apply behavior in Studio? +5. How should safe fix flows be exposed as service operations rather than CLI flags? +6. Which responses must be structured for UI consumption and automation? +7. What, if anything, should remain CLI-specific in the future adapter layer? +8. Should assets have a dedicated background event lane that reports changes, diagnostics, and build activity to the UI? ## Options ### Option A -Define a minimal stable v1 service surface and mark the rest as future or optional. +Define a minimal stable v1 Studio service surface and mark the rest as future or optional. ### Option B -Freeze the larger command catalog immediately. +Freeze a larger command catalog immediately. ## Tradeoffs - Option A lowers the early compatibility burden. -- Option A matches the Studio-integrated direction better because services can stabilize before adapters. +- Option A matches the Studio-integrated direction better because services and UI can stabilize together before any CLI freeze. - Option A gives room to validate mutating operations safely before promising too much. -- Option B gives a bigger product surface up front, but it increases long-term compatibility cost. +- Option B gives a bigger product surface up front, but it increases long-term compatibility cost around the wrong adapter. ## Recommendation Adopt Option A and distinguish: - required core services, -- optional adapters, +- optional future adapters, - future operations not yet in the normative contract. +Current design target: + +- Studio integration is primary; +- asset UI and packer services should be designed together; +- service contracts should describe intent, preview, apply, and result models suitable for UI workflows; +- asset-oriented operations should publish structured background events suitable for logs, progress, and live UI refresh; +- CLI should not drive the core architecture in this phase. + +Also adopt this integration direction: + +- the system should have a dedicated asset workspace event lane; +- this lane observes relevant asset and registry changes, triggers analysis/rebuild workflows as appropriate, and reports structured events to the UI; +- the architectural contract is the event/reporting surface, not a commitment to a specific threading primitive; +- dangerous mutations still require explicit user intent and must not be silently auto-applied by the background lane. + ## Expected Decisions to Produce -1. Core v1 service set. -2. Mutation safety model. +1. Core v1 Studio service set. +2. Mutation safety and preview/apply model. 3. Identifier resolution rules. -4. Adapter and structured output expectations. +4. Structured response contracts for UI and automation. +5. Boundary between core services and future adapters. +6. Asset workspace event lane and structured event model for UI reporting. ## Expected Spec Follow-up - Service operation contract spec. -- CLI adapter contract spec. +- Studio asset workflow integration spec. - Mutation and confirmation policy spec. -- Command output contract spec. +- Structured response/output contract spec. +- Asset event/reporting integration spec. ## Non-Goals - Asset payload binary layout. - Format-specific `asset.json` rules. - Detailed cache internals. +- Freezing a comprehensive CLI contract in the current phase. diff --git a/docs/packer/agendas/01.7. Builder Integration, Versioning, and Trust Model Agenda.md b/docs/packer/agendas/01.7. Builder Integration, Versioning, and Trust Model Agenda.md index 7d65ecfd..c1a1a2c7 100644 --- a/docs/packer/agendas/01.7. Builder Integration, Versioning, and Trust Model Agenda.md +++ b/docs/packer/agendas/01.7. Builder Integration, Versioning, and Trust Model Agenda.md @@ -1,18 +1,20 @@ -# Builder Integration, Versioning, and Trust Model Agenda +# Shipper Integration, Versioning, and Trust Model Agenda ## Status -Open +Closed + +Resolved by [`007-shipper-integration-versioning-and-trust-model-decision.md`](../decisions/007-shipper-integration-versioning-and-trust-model-decision.md). ## Purpose -Close the cross-domain contracts around builder integration, schema versioning, migrations, and the packer trust model. +Close the cross-domain contracts around shipper integration, schema versioning, migrations, and the packer trust model. ## Context The tail of the draft contains several cross-cutting commitments: -- cartridge builder consumes packer outputs, +- shipper consumes packer outputs, - `../runtime` must be able to read the packer-produced artifact contract, - multiple JSON artifacts carry `schema_version`, - packer must migrate older schemas or emit actionable diagnostics, @@ -23,7 +25,7 @@ The first step should be to document the runtime-facing interface explicitly ins ## Source Sections -- `15. Integration with Cartridge Builder` +- `15. Integration with Cartridge Shipper` - `16. Compatibility and Versioning` - `17. Security and Trust Model` @@ -48,7 +50,7 @@ These runtime specs already constrain packer evolution. The packer discussion sh ## Key Questions -1. Which integration points with builder are packer-owned versus shared contracts? +1. Which integration points with shipper are packer-owned versus shared contracts? 2. Which artifacts need explicit migration policy? 3. Is backward compatibility required for all schema versions or only within a support window? 4. What counts as trusted configuration versus untrusted input data? @@ -62,7 +64,7 @@ These runtime specs already constrain packer evolution. The packer discussion sh ### Option A -Keep builder integration narrow, schema versioning explicit, and trust model conservative. +Keep shipper integration narrow, schema versioning explicit, and trust model conservative. ### Option B @@ -80,7 +82,7 @@ Adopt Option A and treat schema evolution plus trust boundaries as first-class c ## Expected Decisions to Produce -1. Builder integration boundary. +1. Shipper integration boundary. 2. Schema versioning and migration policy. 3. Compatibility guarantees. 4. Security and trust invariants. @@ -89,7 +91,7 @@ Adopt Option A and treat schema evolution plus trust boundaries as first-class c ## Expected Spec Follow-up -- Builder integration spec. +- Shipper integration spec. - Schema versioning and migration spec. - Security and trust model spec. - Runtime reader and compatibility interface spec. diff --git a/docs/packer/agendas/README.md b/docs/packer/agendas/README.md index f3224927..e6d33384 100644 --- a/docs/packer/agendas/README.md +++ b/docs/packer/agendas/README.md @@ -4,14 +4,9 @@ This directory contains active packer discussion agendas. ## Active Agendas -1. `01.0. Packer Scope, Boundaries, and Golden Pipeline Agenda.md` -2. `01.1. Asset Workspace, Registry, and Stable Identity Agenda.md` -3. `01.2. Asset Specification, Raw Assets, and Virtual Asset Contract Agenda.md` -4. `01.3. Build Artifacts and Deterministic Packing Agenda.md` -5. `01.4. Diagnostics, Doctor, Quarantine, and Workspace Hygiene Agenda.md` -6. `01.5. Incremental Build, Cache, and Watch Model Agenda.md` -7. `01.6. CLI Surface and Mutating Operations Agenda.md` -8. `01.7. Builder Integration, Versioning, and Trust Model Agenda.md` +There are currently no active packer agendas. + +The first packer agenda wave (`01.0` through `01.7`) has been closed into decision records under [`../decisions/`](../decisions/). ## Purpose @@ -47,7 +42,7 @@ A packer agenda should usually include: - State the concrete decision the discussion is trying to produce. - Prefer explicit tradeoffs over vague option lists. - Avoid drafting large blocks of final normative spec text here. -- Be specific about packer scope: asset workspace, build artifacts, diagnostics, registry, builder boundary, and related contracts. +- Be specific about packer scope: asset workspace, build artifacts, diagnostics, registry, shipper boundary, and related contracts. ## Exit Rule diff --git a/docs/packer/decisions/004-diagnostics-doctor-quarantine-and-workspace-hygiene-decision.md b/docs/packer/decisions/004-diagnostics-doctor-quarantine-and-workspace-hygiene-decision.md new file mode 100644 index 00000000..1b8695c4 --- /dev/null +++ b/docs/packer/decisions/004-diagnostics-doctor-quarantine-and-workspace-hygiene-decision.md @@ -0,0 +1,217 @@ +# Diagnostics, Doctor, Quarantine, and Workspace Hygiene Decision + +## Status + +Accepted + +## Date + +2026-03-11 + +## Context + +This decision closes the architectural questions raised in [`01.4. Diagnostics, Doctor, Quarantine, and Workspace Hygiene Agenda`](../agendas/01.4.%20Diagnostics,%20Doctor,%20Quarantine,%20and%20Workspace%20Hygiene%20Agenda.md). + +The packer is expected to operate as a sanity guardian over a potentially messy `assets/` workspace while still preserving trust and predictability. + +This means the packer must: + +- protect the correctness of managed builds; +- surface hygiene problems in the broader workspace; +- avoid surprising destructive or reorganizing mutations; +- keep recovery and auditability stronger than convenience heuristics. + +## Decision + +The packer adopts a `validate-the-managed-world, inspect-the-workspace, mutate-only-with-consent` model. + +### 1. Diagnostic classes + +Diagnostics are divided into two major classes: + +- build-blocking structural errors; +- advisory workspace hygiene diagnostics. + +Build-blocking structural errors include, at minimum: + +- registered asset root missing its anchor `asset.json`; +- invalid `asset.json`; +- missing required inputs for a managed asset; +- output contract inconsistency; +- invalid preload-related artifact state; +- structural identity collisions or ambiguous ownership. + +Advisory workspace hygiene diagnostics include, at minimum: + +- orphan `asset.json` declarations; +- duplicate source content by hash; +- large unused files; +- assets not currently referenced by managed build inputs; +- naming or organization hints. + +### 2. Registry validation versus workspace scanning + +Registry validation and workspace scanning are distinct surfaces. + +Rules: + +- build-time validation is scoped to the managed world defined by the registry; +- workspace scanning is a broader hygiene pass over `assets/`; +- `doctor` without wider scope flags validates the managed world; +- `doctor --workspace` expands visibility to broader workspace chaos. + +The build must not silently turn into a full workspace scanner. + +### 3. Safety model for automatic fixes + +Automatic fixes are allowed only for corrections that are mechanical, deterministic, and safe. + +Safe automatic fixes include: + +- path normalization; +- explicit materialization of already-determined defaults; +- canonical ordering or formatting of declaration data; +- strictly mechanical regeneration of control-file content when explicitly requested. + +Unsafe automatic fixes are not baseline-safe: + +- moving user content; +- deleting user content; +- deduplicating content by rewriting authoring files; +- renaming assets; +- auto-registering or auto-adopting assets without explicit consent. + +### 4. Explicit consent policy + +Any operation that mutates user-controlled workspace content requires explicit consent. + +This includes: + +- quarantine moves; +- deletion; +- adoption/registration of previously unmanaged assets; +- changes in physical file location; +- changes that alter declared authoring intent rather than only normalizing control data. + +### 5. Quarantine model + +Quarantine is a reversible staging area, not a magic trash bin. + +Rules: + +- quarantine is never automatic by default; +- the packer must explain what is being moved and why; +- recovery must remain possible; +- quarantine exists to support explicit hygiene workflows, not hidden cleanup. + +### 6. Detection aggressiveness + +The packer should be aggressive in detection and conservative in action. + +Meaning: + +- detect broadly when scanning workspace hygiene; +- report clearly and specifically; +- avoid mutating content unless the action is explicitly requested and safe. + +### 7. `doctor --fix` + +`doctor --fix` is limited to safe mechanical corrections in the baseline contract. + +Rules: + +- it must not mutate authoring content as a baseline behavior; +- it must not move or delete content without a stronger explicit operation; +- it must remain predictable enough that users can trust it in Studio and CLI-adapter workflows. + +### 8. Structural conflicts + +Conflicts between registry state, anchor files, and managed identity are structural errors. + +Examples: + +- two asset roots competing for the same identity; +- registered root without anchor; +- duplicated or ambiguous anchor declarations under managed expectations. + +These are not advisory hygiene warnings. +They invalidate the managed build contract. + +### 9. Duplicate content policy + +Duplicate source content by hash is not a structural build error in the baseline contract. + +Rules: + +- it is a hygiene warning or hint; +- it may inform cost, cleanup, or optimization workflows; +- it does not imply semantic invalidity by itself. + +### 10. Orphan asset policy + +Orphan `asset.json` declarations do not block the baseline build. + +Rules: + +- they belong to workspace hygiene surfaces; +- they should be visible in `doctor --workspace`; +- they do not become managed assets automatically; +- stricter policy can be added later, but it is not part of the current baseline. + +## Invariants and Constraints + +The following invariants now apply: + +1. The build validates the managed registry-defined world, not the entire workspace by default. +2. Workspace hygiene scanning is broader than build validation. +3. Structural managed-world errors block builds. +4. Hygiene findings do not block builds by default unless separately elevated by policy. +5. `doctor --fix` is limited to safe mechanical corrections. +6. Quarantine is explicit and reversible. +7. Destructive or relocational mutations require explicit consent. +8. Duplicate content is advisory, not structurally invalid by itself. +9. Orphan anchors are diagnosable, not build-active. +10. Trust and reversibility are preferred over aggressive automatic remediation. + +## Explicit Non-Decisions + +This decision does not yet define: + +- the full taxonomy and numbering of diagnostic codes; +- richer policy knobs such as strict workspace modes by profile; +- detailed UI presentation in Studio; +- watch-mode-specific hygiene behavior; +- detailed garbage collection flows beyond the baseline safety model. + +Those belong to later decisions and specs. + +## Propagation Targets + +This decision must propagate to: + +- packer diagnostics contract spec; +- doctor and workspace scanning spec; +- quarantine and garbage collection spec; +- Studio service behavior for inspection, fix, and cleanup actions; +- future implementation of safe fix classification and explicit-consent workflows. + +## Validation Notes + +Example: orphan anchor + +- `assets/wip/new_bank/asset.json` exists +- it is not registered in the managed registry +- build does not fail by default +- `doctor --workspace` must surface it + +Example: unsafe fix + +- two large duplicate PNGs are detected in workspace +- packer may report them +- packer must not delete or move either file automatically in baseline behavior + +Example: structural error + +- a registered asset root no longer contains its anchor `asset.json` +- this is a managed-world integrity failure +- build validation must fail diff --git a/docs/packer/decisions/005-incremental-build-cache-and-watch-model-decision.md b/docs/packer/decisions/005-incremental-build-cache-and-watch-model-decision.md new file mode 100644 index 00000000..38840d32 --- /dev/null +++ b/docs/packer/decisions/005-incremental-build-cache-and-watch-model-decision.md @@ -0,0 +1,201 @@ +# Incremental Build, Cache, and Watch Model Decision + +## Status + +Accepted + +## Date + +2026-03-11 + +## Context + +This decision closes the architectural questions raised in [`01.5. Incremental Build, Cache, and Watch Model Agenda`](../agendas/01.5.%20Incremental%20Build,%20Cache,%20and%20Watch%20Model%20Agenda.md). + +The packer should support incremental work and future watch-driven workflows, but only under a strict rule: + +- optimization must never redefine build semantics. + +Incremental behavior, cache reuse, and watch mode must remain subordinate to deterministic artifact production. + +## Decision + +The packer adopts a `determinism-first, cache-as-optimization, watch-as-optional-trigger` model. + +### 1. Fingerprint model + +Input tracking uses three levels of signal: + +- `size` +- `mtime` +- `strong_hash` + +Roles: + +- `size` and `mtime` are fast-change hints; +- `strong_hash` is the authoritative content identity signal. + +### 2. Authority of fingerprint signals + +`size` and `mtime` are not sufficient as final truth for cache reuse. + +Rules: + +- they may be used to avoid unnecessary hashing in the fast path; +- they may indicate that a strong hash check is required; +- when reuse affects emitted output, strong hash is the authoritative content check. + +### 3. Cache reuse rule + +Cached work may be reused only when the full effective build inputs remain equivalent. + +This includes, at minimum: + +- source inputs; +- selected output format; +- codec; +- runtime-facing metadata; +- build configuration that affects emitted bytes; +- materialized defaults or effective derived configuration. + +If any of these change, the relevant output must be rebuilt. + +### 4. Incremental equivalence + +Incremental build is an execution optimization only. + +Rules: + +- full build and incremental build must produce the same observable results; +- byte output, header content, offsets, preload data, and other runtime-facing artifacts must match; +- any semantic difference between full and incremental build is a bug. + +### 5. Cache scope + +The packer may persist: + +- fingerprint data; +- derived build results; +- internal cache metadata. + +But: + +- cache internals are not the source of truth; +- runtime-facing artifacts remain authoritative; +- cache structure is not frozen as a normative external contract in the current baseline. + +### 6. Cache explainability + +The packer must be able to explain reuse and rebuild decisions. + +At minimum, the model should support reasons equivalent to: + +- cache hit; +- rebuild because input changed; +- rebuild because config changed; +- rebuild because format or schema contract changed; +- rebuild because cache state is missing or invalid. + +This does not require always-on verbosity, but the capability must exist for diagnostics and Studio tooling. + +### 7. Watch mode + +Watch mode is optional in the current baseline contract. + +Rules: + +- watch may exist as a supported capability; +- watch must trigger rebuild and/or revalidation behavior equivalent to normal packer execution; +- watch is not the place to freeze detailed UI, timing, or orchestration policy yet. + +### 8. Watch safety + +Watch mode must not become a silent mutation engine. + +Rules: + +- baseline watch behavior is observe, detect, validate, and rebuild; +- watch does not imply auto-fix, quarantine, adoption, or destructive cleanup; +- any broader mutation policy requires explicit later design. + +### 9. Strong hash timing + +Strong hash calculation is performed pragmatically, but remains authoritative. + +Rules: + +- fast hints may be checked first; +- when hints indicate possible change, strong hash resolves the question; +- when cache trust is insufficient, strong hash may be recomputed even if hints appear unchanged. + +### 10. Cache portability + +Cross-machine cache portability is not part of the current baseline contract. + +Rules: + +- cache is treated as local to the workspace/runtime environment by default; +- portability, remote reuse, or shareable cache semantics require later explicit design. + +## Invariants and Constraints + +The following invariants now apply: + +1. Deterministic output has priority over cache reuse. +2. `size` and `mtime` are hints, not final truth. +3. Strong hash is the authoritative content identity signal for reuse decisions. +4. Cache reuse is allowed only when effective build inputs are equivalent. +5. Full and incremental builds must be observably equivalent. +6. Cache is an optimization layer, not a semantic authority. +7. The packer must be able to explain rebuild/reuse decisions. +8. Watch mode is optional in the baseline contract. +9. Watch mode does not imply silent mutation of authoring content. +10. Cache portability is deferred. + +## Explicit Non-Decisions + +This decision does not yet define: + +- the exact on-disk schema of every cache file; +- detailed watch UX or scheduling behavior; +- remote/shared cache design; +- background worker orchestration; +- policy for multi-user cache coordination. + +Those belong to later decisions and specs. + +## Propagation Targets + +This decision must propagate to: + +- packer cache and fingerprint specs; +- incremental build guarantees spec; +- watch mode spec or deferred capability note; +- Studio diagnostics for build-reason visibility; +- future implementation of fingerprint persistence and cache invalidation. + +## Validation Notes + +Example: changed input + +- source file timestamp changed +- strong hash also changed +- cached derived result must be invalidated + +Example: misleading timestamp + +- `mtime` changed +- strong hash remains identical +- packer may treat the content as unchanged after authoritative hash confirmation + +Example: config change + +- source inputs are identical +- `output.format` or runtime-facing metadata changed +- cached result must be invalidated because emitted bytes may differ + +Example: watch behavior + +- file changes while watch is active +- packer revalidates or rebuilds +- watch does not automatically move, delete, or adopt workspace content diff --git a/docs/packer/decisions/006-studio-first-asset-services-and-event-lane-decision.md b/docs/packer/decisions/006-studio-first-asset-services-and-event-lane-decision.md new file mode 100644 index 00000000..02e0ef82 --- /dev/null +++ b/docs/packer/decisions/006-studio-first-asset-services-and-event-lane-decision.md @@ -0,0 +1,278 @@ +# Studio-First Asset Services and Event Lane Decision + +## Status + +Accepted + +## Date + +2026-03-11 + +## Context + +This decision closes the architectural questions raised in [`01.6. CLI Surface and Mutating Operations Agenda`](../agendas/01.6.%20CLI%20Surface%20and%20Mutating%20Operations%20Agenda.md). + +The original packer framing was CLI-oriented. +That is no longer the design center. + +The current target is Studio integration: + +- asset UI and packer behavior should evolve together; +- packer operations should exist as Studio-consumable services first; +- background asset activity should be observable in the UI through structured events; +- any future CLI should be an adapter over those services rather than the architecture driver. + +## Decision + +The packer adopts a `Studio-first services + structured event lane` model. + +### 1. Primary integration model + +Studio is the primary integration target. + +Rules: + +- service contracts are the normative operational surface; +- UI workflows are first-class design inputs; +- CLI is not the baseline architecture contract in the current phase; +- if a CLI exists later, it adapts these services rather than redefining them. + +### 2. Core v1 services + +The baseline Studio service surface includes: + +- `init_workspace` +- `register_asset` +- `adopt_asset` +- `forget_asset` +- `remove_asset` +- `list_assets` +- `get_asset_details` +- `doctor` +- `build` + +Later or optional capabilities may include: + +- `watch` +- `gc` +- `quarantine` +- more advanced bulk refactor flows + +### 3. Identifier expectations + +For normal Studio-facing workflows, the preferred identifiers are: + +- `asset_name` +- `asset_root` +- `asset_id` when already resolved internally + +`asset_uuid` may still exist for migration or internal tooling purposes, but it is not required as the primary UX identifier for the baseline flow. + +### 4. Read-only versus mutating services + +The service contract must distinguish clearly between: + +- read-only operations; +- registry mutations; +- workspace mutations. + +Baseline interpretation: + +- read-only: + - `list_assets` + - `get_asset_details` + - `doctor` +- registry mutations: + - `register_asset` + - `adopt_asset` + - `forget_asset` +- workspace mutations: + - `remove_asset` + - future quarantine/move/delete flows + +This distinction is part of the service semantics and must be visible to the UI. + +### 5. Preview/apply model + +Sensitive mutating workflows use staged intent rather than immediate blind mutation. + +Rules: + +- operations with significant mutation potential should support analysis or preview before apply; +- broad, destructive, or relocational actions require explicit user confirmation; +- safe-fix flows should be modeled as services such as: + - analyze + - propose fixes + - apply safe fixes + +This replaces CLI-centric flag thinking as the architectural baseline. + +### 6. Structured service responses + +Service responses must be structured for UI and automation. + +At minimum, the model should support fields equivalent to: + +- `status` +- `summary` +- `affected_assets` +- `diagnostics` +- `proposed_actions` +- `applied_actions` +- `blockers` + +Human-readable strings may still exist, but they must not be the only contract the UI depends on. + +### 7. `forget` versus `remove` + +`forget_asset` and `remove_asset` remain distinct operations. + +Meaning: + +- `forget_asset` leaves the managed set without deleting authoring content; +- `remove_asset` expresses a stronger mutation intent and may affect workspace content depending on policy. + +The UI must present this difference explicitly. + +### 8. Adoption is core + +`adopt_asset` is part of the baseline service set. + +Rationale: + +- orphan `asset.json` declarations are a normal part of the packer model; +- adoption is not an edge-case maintenance feature; +- Studio needs a first-class path from discovery to managed registration. + +### 9. Asset workspace event lane + +The system includes a dedicated asset workspace event lane for Studio integration. + +This is an architectural event/reporting contract, not a commitment to one specific implementation primitive. + +Responsibilities: + +- observe relevant changes in the asset workspace and registry; +- trigger analysis, refresh, rebuild, or rescan workflows when appropriate; +- publish structured events for UI logging, status, and live refresh; +- avoid blocking the UI thread on asset work. + +### 10. Mutation serialization + +Mutating asset operations are serialized semantically. + +Rules: + +- the event lane and service model must not allow conflicting asset mutations to race arbitrarily; +- internal implementation may use controlled parallelism where safe; +- but the observable mutation model should behave as if sensitive workspace operations are serialized. + +This keeps Studio state coherent and avoids hard-to-debug race conditions between watch, doctor, build, and explicit user actions. + +### 11. Event model + +The initial structured event set should include, at minimum: + +- `asset_discovered` +- `asset_changed` +- `diagnostics_updated` +- `build_started` +- `build_finished` +- `cache_hit` +- `cache_miss` +- `preview_ready` +- `action_applied` +- `action_failed` +- `progress_updated` + +`progress_updated` exists to support visible progress reporting such as progress bars and long-running asset activity feedback in the UI. + +### 12. Event lane safety + +The background event lane is not an autonomous mutation engine. + +Rules: + +- it may observe, analyze, report, and trigger safe refresh/rebuild behavior; +- it must not silently apply dangerous mutations; +- explicit user intent remains required for destructive, relocational, or adoption-related changes unless separately designed later. + +### 13. Future CLI boundary + +Any future CLI should remain a secondary adapter. + +Rules: + +- CLI should consume the same service semantics; +- CLI text output is not the canonical contract; +- if stable automation output is needed, it should be derived from the same structured response model used by Studio. + +## Invariants and Constraints + +The following invariants now apply: + +1. Studio integration is the primary target. +2. Services are the normative operational surface. +3. UI and services are designed together. +4. CLI is deferred to adapter status. +5. Core asset operations are service-defined, not command-defined. +6. Sensitive mutations use preview/apply semantics where appropriate. +7. Structured responses are required for UI consumption. +8. `forget_asset` and `remove_asset` remain distinct. +9. Adoption is part of the baseline service model. +10. A dedicated asset event lane exists as an architectural integration surface. +11. Mutating asset workflows are serialized semantically. +12. Initial asset event reporting includes progress updates. +13. Background reporting does not authorize silent dangerous mutation. + +## Explicit Non-Decisions + +This decision does not yet define: + +- the exact transport or IPC mechanism for services; +- the exact threading/runtime primitive used for the event lane; +- final visual design of Studio asset panels; +- detailed watch-mode orchestration beyond the baseline event model; +- a future CLI command grammar. + +Those belong to later decisions and specs. + +## Propagation Targets + +This decision must propagate to: + +- packer service operation spec; +- Studio asset workflow integration spec; +- mutation and confirmation policy spec; +- structured response contract spec; +- asset event/reporting integration spec; +- future implementation of the asset workspace event lane and progress reporting. + +## Validation Notes + +Example: safe inspection flow + +- Studio requests `list_assets` +- service returns structured asset summaries +- event lane may emit `diagnostics_updated` or `asset_changed` as background state evolves + +Example: staged mutation flow + +- Studio requests doctor analysis +- service returns diagnostics plus proposed safe fixes +- user confirms apply +- service performs safe changes +- event lane emits `action_applied` and `progress_updated` + +Example: background rebuild + +- asset source changes +- event lane observes the change +- analysis/rebuild is triggered according to service policy +- UI receives `build_started`, `progress_updated`, and `build_finished` + +Example: dangerous mutation boundary + +- user has not confirmed quarantine or deletion +- background lane may report the issue and propose the action +- background lane must not perform the mutation by itself diff --git a/docs/packer/decisions/007-shipper-integration-versioning-and-trust-model-decision.md b/docs/packer/decisions/007-shipper-integration-versioning-and-trust-model-decision.md new file mode 100644 index 00000000..aa0fc0bb --- /dev/null +++ b/docs/packer/decisions/007-shipper-integration-versioning-and-trust-model-decision.md @@ -0,0 +1,223 @@ +# Shipper Integration, Versioning, and Trust Model Decision + +## Status + +Accepted + +## Date + +2026-03-11 + +## Context + +This decision closes the architectural questions raised in [`01.7. Builder Integration, Versioning, and Trust Model Agenda`](../agendas/01.7.%20Builder%20Integration,%20Versioning,%20and%20Trust%20Model%20Agenda.md). + +The packer must define: + +- how its artifacts are versioned; +- what migration obligations it owns; +- what trust boundaries apply to asset tooling; +- how a future shipper will consume packer outputs without becoming part of the packer architecture now. + +The important clarification is: + +- the shipper does not exist yet as part of the current packer work; +- shipper integration is therefore a future consumer boundary, not a present operational dependency. + +## Decision + +The packer adopts a `future-shipper-consumer, explicit-versioning, conservative-trust` model. + +### 1. Current shipper boundary + +The shipper is not part of the current packer architecture. + +Rules: + +- the packer must not depend on a concrete shipper implementation; +- the packer should emit stable, documented artifacts that a future shipper can consume; +- current packer design must stop at artifact publication, not at cartridge shipping orchestration. + +### 2. Ownership boundary + +Current ownership is split as follows: + +- packer owns artifact production; +- runtime owns runtime-side artifact reading semantics; +- future shipper will own cartridge shipping/publication concerns. + +This means: + +- runtime-facing reading contract for `assets.pa` belongs to runtime specs; +- writer-side production contract for `assets.pa` belongs to packer specs; +- a future shipper consumes these artifacts but does not redefine the asset contract. + +### 3. Packer-owned artifacts + +The packer is responsible for emitting and versioning at least: + +- `assets.pa` +- `build/asset_table.json` +- `build/preload.json` +- `build/asset_table_metadata.json` +- `asset.json` +- `index.json` +- packer-owned cache and control data + +These are the boundaries a future shipper may consume, but they remain packer-owned artifacts. + +### 4. Minimal future shipper contract + +The future shipper contract should remain minimal for now. + +Baseline contract: + +- which packer artifacts are published; +- where they are expected to exist; +- which artifacts are runtime-facing versus tooling-only; +- which artifacts are authoritative versus derived. + +Detailed shipping workflow is deferred until the shipper exists. + +### 5. Versioning policy + +Versioning is explicit and per-artifact/per-schema. + +Rules: + +- `asset.json` has explicit schema versioning; +- `index.json` has explicit schema versioning; +- `assets.pa` has explicit runtime-facing envelope versioning; +- companion artifacts may have their own schema versioning where needed; +- version evolution is tracked per concern, not by one global version number for the whole domain. + +### 6. Migration policy + +The packer migrates only artifacts it owns. + +This includes, at minimum: + +- `asset.json` +- `index.json` +- packer-owned caches and control files + +Rules: + +- migration may be automatic within a supported window; +- outside the supported window, the packer fails with actionable diagnostics; +- the packer does not assume a cooperative migration workflow with a shipper that does not yet exist. + +### 7. Compatibility window + +Backward compatibility is provided through an explicit support window, not an open-ended forever policy. + +Rules: + +- the packer must define which previous versions it can read or migrate; +- unsupported versions fail clearly and early; +- CI and Studio should be able to surface unsupported-version failures deterministically. + +### 8. Runtime compatibility guarantees + +The packer must preserve compatibility guarantees relevant to `../runtime`. + +These include: + +- `assets.pa` emitted according to the runtime-facing reader contract; +- offsets relative to payload region; +- canonical header semantics where required by packer-side production rules; +- stable `asset_id` continuity end-to-end; +- preload emitted according to the current runtime-facing contract. + +The runtime must not depend on companion JSON artifacts for correctness. + +### 9. Trusted versus untrusted inputs + +The packer uses a conservative trust model. + +Untrusted until validated: + +- source asset files; +- manually edited or legacy `asset.json`; +- legacy registry/control data; +- imported artifacts and external project content. + +Trusted only after: + +- strict parse; +- structural validation; +- semantic validation; +- version compatibility check. + +### 10. No plugin/script execution baseline + +The packer must not execute arbitrary external plugins or scripts in the baseline architecture. + +Rules: + +- no script execution as part of normal asset transformation contract; +- no untrusted extension hooks; +- no implicit foreign code execution during build. + +Any future extensibility model requires a separate decision. + +### 11. Shipper as consumer, not co-author + +A future shipper consumes packer outputs but must not silently reinterpret or repair the asset contract. + +Rules: + +- shipper does not redefine `assets.pa`; +- shipper does not become the source of truth for `asset_table` or preload semantics; +- if an asset artifact is invalid, the issue belongs back to packer production, not to shipper-side correction. + +## Invariants and Constraints + +The following invariants now apply: + +1. The shipper is a future consumer, not a current packer dependency. +2. Packer stops at artifact production. +3. Runtime owns reader semantics of the runtime-facing artifact. +4. Packer owns writer semantics and companion artifacts on the production side. +5. Versioning is explicit per artifact/schema. +6. Migration applies only to artifacts the packer owns. +7. Compatibility is governed by a support window, not indefinite backward promises. +8. Inputs are untrusted until validated. +9. Baseline packer architecture does not execute arbitrary plugins or scripts. +10. A future shipper is a consumer, not a co-author of the asset contract. + +## Explicit Non-Decisions + +This decision does not yet define: + +- the concrete shipper implementation; +- cartridge shipping workflow details; +- packaging formats beyond the currently documented runtime forms; +- artifact signing or distribution security policy; +- remote/shared artifact registry behavior. + +Those belong to later decisions and future shipper work. + +## Propagation Targets + +This decision must propagate to: + +- packer versioning and migration specs; +- packer trust/security model spec; +- future shipper integration spec; +- Studio diagnostics for unsupported versions and migration failures; +- future implementation of schema migration boundaries and compatibility checks. + +## Validation Notes + +Example: unsupported registry version + +- `index.json` version is outside the supported migration window +- packer reports explicit version mismatch and migration failure +- build does not continue + +Example: future shipper consumption + +- packer emits `assets.pa` and companion build artifacts +- future shipper consumes those artifacts to assemble a cartridge +- shipper does not reinterpret asset semantics or rewrite packer-owned artifact meaning diff --git a/docs/packer/pull-requests/README.md b/docs/packer/pull-requests/README.md index 76587e1c..2d6358af 100644 --- a/docs/packer/pull-requests/README.md +++ b/docs/packer/pull-requests/README.md @@ -11,7 +11,7 @@ Use this directory when a change: - touches multiple documents or code paths, - changes packer architecture or semantics, - needs explicit acceptance criteria, -- affects diagnostics, build outputs, registry rules, or builder boundaries, +- affects diagnostics, build outputs, registry rules, or shipper boundaries, - should be reviewed before editing the final normative corpus or implementation. ## Required Properties