Files
ai-engineering/docs/protocol.md
T
Nilton Constantino c84bb8d36f feat: store discussion prose in Markdown bodies
Add versioned Markdown discussion documents with legacy YAML reading and explicit migration that backs up originals and preserves semantic history.

Render readable discussion and decision prose in Markdown context, document compatibility, and extend native smoke coverage for migration.

Validated with 29 passing tests, macOS ARM64 native smoke, and context-preserving migration of the Zelda pilot Work.
2026-11-11 19:50:36 +00:00

12 KiB

Portable engineering protocol — schema 1

Project root and configuration

The engineering root is the parent of .ai-engineering. Repository paths resolve against that root, independently of the CLI's current directory. Discovery searches upward or accepts an explicit --project root (or .ai-engineering directory).

An engineering project may be a simple source folder without Git, one Git repository, or a workspace containing several repositories. All three use the same Work protocol. Initialize directly in the folder that represents the desired project scope. An empty repositories: [] is valid; Works need no repository references to persist discussions, decisions, artifacts and context. Git is invoked only when inspecting registered repositories. Without repository references, a Work has no Git revision snapshots.

For a single Git repository at the engineering root, register it with path: "." and type: linked-git. Registration remains explicit; initialization does not create or register a Git repository. For multiple repositories, register their relative subdirectory paths as in the example below.

schemaVersion: 1
project:
  id: prometeu
  name: Prometeu
repositories:
  - id: runtime
    path: runtime
    type: linked-git
    language: rust
    access:
      read: true
      edit: false
    validationCommands: []

IDs use lowercase ASCII letters, digits and hyphens, beginning with a letter. Repository types are linked-git and git-submodule. Omitted access defaults to read: true, edit: false. If access is supplied its booleans are explicit; edit requires read. Language is optional. validationCommands is optional metadata, never executable through this release. Unknown fields and schema versions fail validation.

Absolute repository paths, .. components, backslashes and symlinks beneath the root are rejected, including links that point inside the root. This stricter initial rule avoids ambiguous link policies. A linked Git repository may be the engineering root; repositories cannot live inside .ai-engineering. Git inspection verifies the registered path is a repository top-level; submodule registration additionally verifies its superproject. Paths must exist. No submodules are automatically created, moved or updated.

Policy controls protocol intent, not OS security. Concurrent malicious filesystem replacement is outside this CLI's sandbox guarantee; use OS isolation for untrusted processes. The adapter clears inherited GIT_* variables and disables optional Git locks.

Work and files

.ai-engineering/
  ai-engineering.yaml
  .gitignore
  .local/write.lock                 # ignored, no semantic state
  contracts/                        # reserved for project contracts
  works/
    .sequence.yaml                  # canonical high-water mark; commit this
    WRK-0001-improve-scene-editor/
      work.md
      discussions/DSC-0001.md
      decisions/DEC-0001.md
      artifacts/
        references/
        experiments/
        evidence/ART-0001/
          artifact.yaml
          payload
        outputs/
      runs/                         # ignored temporary execution outputs

work.md, discussion and decision files contain YAML front matter, delimited by ---. Work and draft DEC files retain their typed fields in front matter; trailing Markdown in those documents remains noncanonical. Discussion files use the Markdown body for prose as described below. There is no Java object serialization and no dependency on conversation state.

Work fields: schemaVersion, id, title, description, goal, motivation, constraints[], acceptanceCriteria[], status, createdAt, updatedAt, author, repositories[], discussions[], decisions[], artifacts[]. Each repository snapshot records repositoryId, relative path, exact head commit and dirty. An empty repository records UNBORN, not an invented commit. Snapshots are taken at Work creation and remain stable; they are not claims about later experiments.

Work IDs are project-wide, monotonic WRK- numbers with at least four digits. A persisted sequence reserves identity before publishing a Work; crashes may leave gaps. Do not remove or rewind the sequence. Discussion/decision/artifact IDs are Work-local; their full identity is (project, Work ID, child ID). Work reference arrays determine history order. Directory names are a convenience; ID stability does not depend on title. Duplicate Work IDs and directory/record mismatches fail validation.

Initial statuses: OPEN, ACTIVE, BLOCKED, COMPLETED, CANCELLED. work status permits reorientation without irreversible lifecycle gates. Completion currently records intent; it does not prove acceptance criteria. Conformance automation belongs to Milestone 5.

Discussion and draft decisions

Each discussion entry has schemaVersion, id, workId, discussionId (thread name), kind, author, createdAt, optional respondsTo, and text. Multiple threads can coexist. Kinds: NOTE, QUESTION, RESPONSE, SUGGESTION, ASSUMPTION, HYPOTHESIS, INVESTIGATION, ALTERNATIVE, ISSUE, CONCLUSION. Entries are appended, not rewritten. Text is supplied by a human or external agent; the CLI never fabricates conclusions.

New discussion files have documentVersion: 2 in front matter and omit the text field there. The complete Markdown body supplies text, including its original headings, lists, tables, code blocks and whitespace:

---
schemaVersion: 1
id: DSC-0002
workId: WRK-0001
discussionId: main
kind: SUGGESTION
author: agent
createdAt: '2026-11-11T16:47:32Z'
respondsTo: DSC-0001
documentVersion: 2
---

## Proposed approach

Use the next small, testable slice.

- State the limits.
- Identify supporting evidence.

One blank line separates the closing delimiter from the body; that formatting line is excluded from text. All subsequent whitespace belongs to the prose. LF and CRLF front matter are accepted; the reader does not normalize the body's line endings. schemaVersion: 1 still describes the protocol record. documentVersion: 2 describes this file representation, not a change to JSON context fields. Unknown document versions and simultaneous YAML text plus a version 2 body fail validation.

Legacy discussions omit documentVersion (or declare 1) and keep canonical text in YAML; their trailing body remains noncanonical notes. The new reader accepts both representations without automatic writes. The old CLI cannot read document version 2.

discussion migrate <Work> explicitly converts legacy entries. Every discussion is preflighted before any file is replaced. Nonblank legacy trailing notes cause refusal; incorporate those notes into canonical text first. Original UTF-8 files are backed up under .local/backups/discussions/<Work>/<operation-id>/ before atomic replacement. Migration preserves all semantic fields, Work timestamps and context JSON/Markdown, and skips already converted entries. It changes formatting, not discussion history. Interrupted migrations can be resumed; completed files are skipped and existing backups are retained. This is per-file publication, not an all-or-nothing multi-file transaction.

A response must refer to an earlier entry in the same Work. Outstanding questions include QUESTION and ISSUE entries without an explicit RESPONSE or CONCLUSION pointing to them. This mechanical definition does not infer agreement or approval from prose.

Draft DEC fields: schemaVersion, id, workId, title, context, approach, alternatives[], rationale, risks[], acceptanceCriteria[], approvalStatus, author, createdAt, evidence[]. Only DRAFT is accepted in this release. Empty refinement fields are legal in drafts. The CLI initially supplies context, approach, rationale and evidence; remaining fields can be refined in the canonical YAML. No approval, supersession or autonomous operational decision is implemented.

Evidence and integrity

artifact add copies one regular file up to 1 MiB. Its manifest records schema, ID, Work ID, origin, purpose, Work-relative payload path, SHA-256, byte size, author and UTC creation time. The origin defaults to the source basename; use --origin for a useful portable provenance reference. Payload bytes are exact. The fixed payload filename avoids filename ambiguity; origin preserves the supplied source identity.

Validation checks required directories, IDs, schemas, repository references, child ownership, missing records, orphan records and payload integrity. Evidence paths stay inside the Work. Large resources should remain in their canonical location and be described in discussion/reference files, rather than copied into this workspace. artifacts/outputs is versionable for deliberately retained outputs; use ignored runs/ for large temporary results. No command creates large output files by default.

Context and continuity

work context --format markdown|json loads only canonical project/Work files. It includes all Work fields (metadata separately in Markdown), stored source revisions, ordered discussion history/conclusions, explicit draft decisions, unresolved questions and complete artifact references with integrity metadata. JSON has top-level schemaVersion, work, discussionHistory, decisions, outstandingQuestions, artifacts. It does not embed arbitrary evidence payloads.

No export timestamp, current absolute path or live Git state is injected. Unchanged files yield byte-identical output across new instances and processes. Export rejects broken references or integrity failures. It requires neither a model nor live Git inspection; project validate separately verifies live repositories.

Markdown context renders discussion and decision prose directly, with IDs, authors, timestamps and reply/approval metadata. Empty sections use None recorded.. JSON keeps the complete typed records, including discussion text, regardless of their on-disk document version. Evidence remains referenced rather than embedded.

Write guarantees and recovery

Protocol mutations hold .local/write.lock, an OS lock released automatically at process exit. Individual files publish atomically; create operations reject overwrites. New project/Work/artifact directories are staged and renamed. Work sequence reservation precedes directory publication to avoid identity reuse after interruption.

A child file/bundle is published before its reference is atomically added to work.md. The filesystem is not a multi-file transactional database: interruption in that window can leave an orphan, which validation reports. Inspect its metadata and either add its ID to the Work reference array or remove it after review. Hidden .tmp directories are incomplete staging outputs, ignored by readers; inspect and clean them manually after an interrupted write. Concurrent manual editors do not honor the CLI lock. Directory entry durability under sudden power loss depends on the host filesystem.

Future contracts (design only)

  • AgentAdapter: accept versioned context and return proposed questions, suggestions, assumptions or draft content with provenance. The caller decides what to persist; no provider or orchestration dependency is part of schema 1.
  • OODA iteration: identify Work and prior observations, orientation, operational choices and attempts. Phases can repeat or be skipped; no four-stage workflow gate.
  • Attempt: stable identity, hypothesis, input references, exact repository revisions, granted permissions, invocation/environment, observations, outputs, result and evidence references. Evidence and interpretation remain distinguishable.
  • Coordinated execution: independent per-repository worktrees and commit records, integrated validation, explicit authority for destructive actions, pushes and merges. No claim of atomic commits across repositories.
  • Conformance: evaluate explicit criteria against evidence and canonical code/spec/test updates. No required Lesson artifact. Later schema changes require explicit migration.