Files
ai-engineering/docs/protocol.md
T
Nilton Constantino c84bb8d36f feat: store discussion prose in Markdown bodies
Add versioned Markdown discussion documents with legacy YAML reading and explicit migration that backs up originals and preserves semantic history.

Render readable discussion and decision prose in Markdown context, document compatibility, and extend native smoke coverage for migration.

Validated with 29 passing tests, macOS ARM64 native smoke, and context-preserving migration of the Zelda pilot Work.
2026-11-11 19:50:36 +00:00

231 lines
12 KiB
Markdown

# Portable engineering protocol — schema 1
## Project root and configuration
The engineering root is the parent of `.ai-engineering`. Repository paths resolve
against that root, independently of the CLI's current directory. Discovery searches
upward or accepts an explicit `--project` root (or `.ai-engineering` directory).
An engineering project may be a simple source folder without Git, one Git repository,
or a workspace containing several repositories. All three use the same Work protocol.
Initialize directly in the folder that represents the desired project scope. An empty
`repositories: []` is valid; Works need no repository references to persist discussions,
decisions, artifacts and context. Git is invoked only when inspecting registered
repositories. Without repository references, a Work has no Git revision snapshots.
For a single Git repository at the engineering root, register it with `path: "."`
and `type: linked-git`. Registration remains explicit; initialization does not create
or register a Git repository. For multiple repositories, register their relative
subdirectory paths as in the example below.
```yaml
schemaVersion: 1
project:
id: prometeu
name: Prometeu
repositories:
- id: runtime
path: runtime
type: linked-git
language: rust
access:
read: true
edit: false
validationCommands: []
```
IDs use lowercase ASCII letters, digits and hyphens, beginning with a letter.
Repository types are `linked-git` and `git-submodule`. Omitted `access` defaults to
`read: true, edit: false`. If `access` is supplied its booleans are explicit; edit
requires read. Language is optional. `validationCommands` is optional metadata,
never executable through this release. Unknown fields and schema versions fail validation.
Absolute repository paths, `..` components, backslashes and symlinks beneath the root
are rejected, including links that point inside the root. This stricter initial rule
avoids ambiguous link policies. A linked Git repository may be the engineering root;
repositories cannot live inside `.ai-engineering`. Git inspection verifies the registered
path is a repository top-level;
submodule registration additionally verifies its superproject. Paths must exist.
No submodules are automatically created, moved or updated.
Policy controls protocol intent, not OS security. Concurrent malicious filesystem
replacement is outside this CLI's sandbox guarantee; use OS isolation for untrusted
processes. The adapter clears inherited `GIT_*` variables and disables optional Git locks.
## Work and files
```text
.ai-engineering/
ai-engineering.yaml
.gitignore
.local/write.lock # ignored, no semantic state
contracts/ # reserved for project contracts
works/
.sequence.yaml # canonical high-water mark; commit this
WRK-0001-improve-scene-editor/
work.md
discussions/DSC-0001.md
decisions/DEC-0001.md
artifacts/
references/
experiments/
evidence/ART-0001/
artifact.yaml
payload
outputs/
runs/ # ignored temporary execution outputs
```
`work.md`, discussion and decision files contain YAML front matter, delimited by
`---`. Work and draft DEC files retain their typed fields in front matter; trailing
Markdown in those documents remains noncanonical. Discussion files use the Markdown
body for prose as described below. There is no Java object serialization and no
dependency on conversation state.
Work fields: `schemaVersion`, `id`, `title`, `description`, `goal`, `motivation`,
`constraints[]`, `acceptanceCriteria[]`, `status`, `createdAt`, `updatedAt`, `author`,
`repositories[]`, `discussions[]`, `decisions[]`, `artifacts[]`. Each repository snapshot
records `repositoryId`, relative `path`, exact `head` commit and `dirty`. An empty
repository records `UNBORN`, not an invented commit. Snapshots are taken at Work
creation and remain stable; they are not claims about later experiments.
Work IDs are project-wide, monotonic `WRK-` numbers with at least four digits. A
persisted sequence reserves identity before publishing a Work; crashes may leave gaps.
Do not remove or rewind the sequence. Discussion/decision/artifact IDs are **Work-local**;
their full identity is `(project, Work ID, child ID)`. Work reference arrays determine
history order. Directory names are a convenience; ID stability does not depend on title.
Duplicate Work IDs and directory/record mismatches fail validation.
Initial statuses: OPEN, ACTIVE, BLOCKED, COMPLETED, CANCELLED. `work status` permits
reorientation without irreversible lifecycle gates. Completion currently records intent;
it does not prove acceptance criteria. Conformance automation belongs to Milestone 5.
## Discussion and draft decisions
Each discussion entry has `schemaVersion`, `id`, `workId`, `discussionId` (thread name),
`kind`, `author`, `createdAt`, optional `respondsTo`, and `text`. Multiple threads can
coexist. Kinds: NOTE, QUESTION, RESPONSE, SUGGESTION, ASSUMPTION, HYPOTHESIS,
INVESTIGATION, ALTERNATIVE, ISSUE, CONCLUSION. Entries are appended, not rewritten.
Text is supplied by a human or external agent; the CLI never fabricates conclusions.
New discussion files have `documentVersion: 2` in front matter and omit the `text`
field there. The complete Markdown body supplies `text`, including its original
headings, lists, tables, code blocks and whitespace:
```markdown
---
schemaVersion: 1
id: DSC-0002
workId: WRK-0001
discussionId: main
kind: SUGGESTION
author: agent
createdAt: '2026-11-11T16:47:32Z'
respondsTo: DSC-0001
documentVersion: 2
---
## Proposed approach
Use the next small, testable slice.
- State the limits.
- Identify supporting evidence.
```
One blank line separates the closing delimiter from the body; that formatting line
is excluded from `text`. All subsequent whitespace belongs to the prose. LF and CRLF
front matter are accepted; the reader does not normalize the body's line endings.
`schemaVersion: 1` still describes the protocol record. `documentVersion: 2` describes
this file representation, not a change to JSON context fields. Unknown document
versions and simultaneous YAML `text` plus a version 2 body fail validation.
Legacy discussions omit `documentVersion` (or declare 1) and keep canonical `text`
in YAML; their trailing body remains noncanonical notes. The new reader accepts both
representations without automatic writes. The old CLI cannot read document version 2.
`discussion migrate <Work>` explicitly converts legacy entries. Every discussion is
preflighted before any file is replaced. Nonblank legacy trailing notes cause refusal;
incorporate those notes into canonical text first. Original UTF-8 files are backed up
under `.local/backups/discussions/<Work>/<operation-id>/` before atomic replacement.
Migration preserves all semantic fields, Work timestamps and context JSON/Markdown,
and skips already converted entries. It changes formatting, not discussion history.
Interrupted migrations can be resumed; completed files are skipped and existing backups
are retained. This is per-file publication, not an all-or-nothing multi-file transaction.
A response must refer to an earlier entry in the same Work. Outstanding questions
include QUESTION and ISSUE entries without an explicit RESPONSE or CONCLUSION pointing
to them. This mechanical definition does not infer agreement or approval from prose.
Draft DEC fields: `schemaVersion`, `id`, `workId`, `title`, `context`, `approach`,
`alternatives[]`, `rationale`, `risks[]`, `acceptanceCriteria[]`, `approvalStatus`,
`author`, `createdAt`, `evidence[]`. Only DRAFT is accepted in this release. Empty
refinement fields are legal in drafts. The CLI initially supplies context, approach,
rationale and evidence; remaining fields can be refined in the canonical YAML.
No approval, supersession or autonomous operational decision is implemented.
## Evidence and integrity
`artifact add` copies one regular file up to 1 MiB. Its manifest records schema, ID,
Work ID, origin, purpose, Work-relative payload path, SHA-256, byte size, author and UTC
creation time. The origin defaults to the source basename; use `--origin` for a useful
portable provenance reference. Payload bytes are exact. The fixed `payload` filename
avoids filename ambiguity; origin preserves the supplied source identity.
Validation checks required directories, IDs, schemas, repository references, child
ownership, missing records, orphan records and payload integrity. Evidence paths stay
inside the Work. Large resources should remain in their canonical location and be
described in discussion/reference files, rather than copied into this workspace.
`artifacts/outputs` is versionable for deliberately retained outputs; use ignored `runs/`
for large temporary results. No command creates large output files by default.
## Context and continuity
`work context --format markdown|json` loads only canonical project/Work files. It
includes all Work fields (metadata separately in Markdown), stored source revisions,
ordered discussion history/conclusions, explicit draft decisions, unresolved questions
and complete artifact references with integrity metadata. JSON has top-level
`schemaVersion`, `work`, `discussionHistory`, `decisions`, `outstandingQuestions`,
`artifacts`. It does not embed arbitrary evidence payloads.
No export timestamp, current absolute path or live Git state is injected. Unchanged
files yield byte-identical output across new instances and processes. Export rejects
broken references or integrity failures. It requires neither a model nor live Git
inspection; `project validate` separately verifies live repositories.
Markdown context renders discussion and decision prose directly, with IDs, authors,
timestamps and reply/approval metadata. Empty sections use `None recorded.`. JSON
keeps the complete typed records, including discussion `text`, regardless of their
on-disk document version. Evidence remains referenced rather than embedded.
## Write guarantees and recovery
Protocol mutations hold `.local/write.lock`, an OS lock released automatically at
process exit. Individual files publish atomically; create operations reject overwrites.
New project/Work/artifact directories are staged and renamed. Work sequence reservation
precedes directory publication to avoid identity reuse after interruption.
A child file/bundle is published before its reference is atomically added to `work.md`.
The filesystem is not a multi-file transactional database: interruption in that window
can leave an orphan, which validation reports. Inspect its metadata and either add its
ID to the Work reference array or remove it after review. Hidden `.tmp` directories are
incomplete staging outputs, ignored by readers; inspect and clean them manually after
an interrupted write. Concurrent manual editors do not honor the CLI lock. Directory
entry durability under sudden power loss depends on the host filesystem.
## Future contracts (design only)
- AgentAdapter: accept versioned context and return proposed questions, suggestions,
assumptions or draft content with provenance. The caller decides what to persist;
no provider or orchestration dependency is part of schema 1.
- OODA iteration: identify Work and prior observations, orientation, operational
choices and attempts. Phases can repeat or be skipped; no four-stage workflow gate.
- Attempt: stable identity, hypothesis, input references, exact repository revisions,
granted permissions, invocation/environment, observations, outputs, result and
evidence references. Evidence and interpretation remain distinguishable.
- Coordinated execution: independent per-repository worktrees and commit records,
integrated validation, explicit authority for destructive actions, pushes and merges.
No claim of atomic commits across repositories.
- Conformance: evaluate explicit criteria against evidence and canonical code/spec/test
updates. No required Lesson artifact. Later schema changes require explicit migration.