Files
ai-engineering/README.md
T

492 lines
25 KiB
Markdown

# ai-engineering
An open, agent-agnostic engineering protocol and CLI for simple projects and projects
spanning multiple repositories. Git is optional. Works belong to the engineering
project. Canonical files preserve goals, discussion,
decisions, plans and evidence so another person or agent can continue without chat history.
Milestones 0 and 1 and the operational replacement baseline are implemented.
Discussion refinement now includes topics, surveys and two deterministic adapters
to verify the integration contract. Local Codex/Claude adapters now exercise that
contract with real providers for proposals and synthesis.
The application runs locally in non-web mode;
it needs no model API, database or running service.
## Build and run
Build requirements: JDK 25, Git (for repository inspection and integration fixtures),
and internet access for the first dependency download. The Gradle Wrapper is included.
The recommended local toolchain is GraalVM Community Edition, pinned in `.sdkmanrc`.
With SDKMAN installed, activate it from the project root:
```sh
sdk env install # only needed if the pinned JDK is not installed
sdk env
export GRAALVM_HOME="$JAVA_HOME"
java --version
native-image --version
```
SDKMAN candidate `25.5.5.0+3-graalce` provides GraalVM CE 25.5.5.0.3 based on
JDK 25.0.5.0.3. The Java language/bytecode target remains 25. SDKMAN is optional;
setting `JAVA_HOME` and `GRAALVM_HOME` to an equivalent GraalVM installation also works.
```sh
./gradlew clean test
./gradlew bootJar
java -jar build/libs/ai-engineering-0.1.0.jar --help
java -jar build/libs/ai-engineering-0.1.0.jar --version
```
Pinned stack: Java target 25, Spring Boot 4.0.9, Spring Modulith 2.0.9,
Gradle 9.3.1, Picocli 4.7.7, Native Build Tools 0.11.5, JUnit 5.13.4.
Jackson YAML/JSON and AssertJ versions come from the pinned Spring Boot BOM.
Compatibility sources and rationale are in [architecture](docs/architecture.md).
## Native installation and packaging
A native distribution contains `ai-engineering` (`ai-engineering.exe` on Windows).
Extract the platform archive and put that executable on `PATH`. Running it requires
no JVM. Git must be on `PATH` for commands that inspect registered repositories.
There are no published release downloads yet; packaging tasks create local archives.
Building requires GraalVM for **JDK 25**, Native Image and the host C/C++ toolchain:
Xcode Command Line Tools on macOS, GCC and development libraries on Linux, or the
Visual Studio C++ build tools on Windows. Set `JAVA_HOME` and `GRAALVM_HOME` to the
GraalVM JDK home (`Contents/Home` on macOS).
When SDKMAN's `current` points to another JDK, set both variables explicitly;
Gradle's `org.gradle.java.home` alone does not select the Native Image tool.
```sh
./gradlew nativeCompile
./build/native/nativeCompile/ai-engineering --help
./gradlew nativeDistZip nativeDistTar -PtargetPlatform=macos-arm64
bash scripts/native-smoke.sh "$PWD/build/native/nativeCompile/ai-engineering"
```
Archives are written to `build/distributions/ai-engineering-0.1.0-<platform>.*`.
Build each binary on its target OS/architecture; Native Image is not a general cross
compiler. CI defines Linux x86_64, macOS ARM64 and Windows x86_64 native builds.
macOS x86_64 remains a target with an upstream toolchain availability limitation:
GraalVM 25 dropped that port; see [roadmap](docs/roadmap.md). Local verification
results, including which native binaries were actually tested, are in [experiments](docs/experiments.md).
## Jenkins tests and coverage
The Jenkins pipeline is [files/config/Jenkinsfile](files/config/Jenkinsfile). Configure
that Script Path in a Pipeline from SCM job on a Unix agent with JDK 25 and Git.
It runs tests with JaCoCo and publishes JUnit, coverage XML/HTML and raw execution
data, including available diagnostics after failures. Coverage gates follow Studio.
```sh
./gradlew test jacocoTestReport jenkinsPipelineTest --continue --console=plain
```
Coverage HTML is generated at
`build/reports/jacoco/jacocoTestReport/html/index.html`. See the
[Jenkins setup and verification guide](files/config/README.md) for required plugins,
thresholds, report paths and the boundary between local checks and a controller run.
## Start in a simple project
Run inside the folder you already use for your application, game or library:
```sh
ai-engineering init --name "My Application"
ai-engineering work create "Improve the application" \
--goal "Preserve enough context to continue in another session"
ai-engineering discussion add WRK-0001 --text "Record the current state and next questions."
ai-engineering decision draft WRK-0001 --title "Evaluate the proposed approach"
ai-engineering work context WRK-0001 --format markdown
ai-engineering project validate
```
This creates `.ai-engineering/` beside your existing source files. A project with
`repositories: []` is fully supported; the workflow above does not require Git to be
installed. Its Works have no Git revision snapshots. Git becomes relevant when you
choose to register repositories.
If the project folder is itself an existing Git repository, you can register it:
```sh
ai-engineering repo add app --path .
ai-engineering work create "Investigate a change" --repo app
```
Registration is optional and defaults to read-only. For example, a standalone
`zelda-1/` project can keep `.ai-engineering/` directly inside `zelda-1/`, without
registering the separate runtime or editor repositories. Each Work explicitly selects
any registered repositories it needs with `--repo`.
## Discover and continue with an agent
`init` installs `.ai-engineering/README.md`, `AGENTS.md` and `protocol.json`. These
describe the protocol without a provider-specific skill. It also creates or updates
a marked AIE block in the project's root `AGENTS.md`, preserving rules outside the
block. Changed instruction files are backed up under `.local/backups/onboarding/`.
For an existing project, install the entry contract explicitly:
```sh
aie onboard
aie status --format json
aie next-actions --format json
aie work context WRK-0001
```
Use `ai-engineering` if the optional `aie` alias is unavailable. `onboard` is
idempotent; it leaves Work records unchanged. `aie guide` reads the bundled guide
even outside a project; `aie guide --format json` shows the discovery manifest.
An agent that finds `.ai-engineering` can read its README and manifest directly.
The root instructions point agents to that entry. Automatic discovery still depends
on the agent inspecting project instructions; a CLI cannot force an arbitrary model
to inspect hidden folders or obey a protocol. Exported context includes the guide.
## Inspect compatibility before updating
```sh
aie project compatibility --format json
```
`status` includes a compatibility summary. The diagnostic separates the installed
CLI release, protocol contract revision and actual storage formats, including record
history. `CURRENT` and `LEGACY_COMPATIBLE` both mean readable; neither requires a
migration. `UNSUPPORTED` and `INVALID` return exit 4 and identify affected files.
Validation of references and evidence remains `aie project validate`.
Keep completed Works in their existing formats. Older supported files are interpreted
in memory; reads and diagnostics never normalize them on disk. `aie onboard` updates
only entry instructions and the manifest, with backups. It refuses to overwrite a
newer unsupported contract. There is no required data migration in this release.
See [compatibility and evolution](docs/compatibility.md) for the version policy and
the contract for future migrations; no source-code inspection is needed by consumers.
## Discuss topics and answer surveys
Ideas remain Discussion entries, organized by Work-local topics. Existing discussion
threads appear as implicit topics without rewriting their files. A small Work starts
with topic `main`. Every survey question has suggestions and accepts free text:
```sh
aie topic create WRK-0001 "Rendering"
aie discussion add WRK-0001 --topic TOP-0001 --kind SUGGESTION \
--text "Compare two rendering approaches" --author agent
aie survey create WRK-0001 --title "Choose an investigation"
aie survey question WRK-0001 SVY-0001 --topic TOP-0001 \
--text "Which approach should we investigate?" \
--option "Measure the current renderer" --option "Prototype layers" --recommend 1
aie survey show WRK-0001 SVY-0001
aie survey answer WRK-0001 SVY-0001 QST-0001 --choice 1 --author human
# A new answer supersedes the previous one while preserving it:
aie survey answer WRK-0001 SVY-0001 QST-0001 \
--text "Compare both with a small fixture" --author human
aie survey history WRK-0001 SVY-0001 --format json
```
Add `--multiple` when a question permits several selections, and repeat `--choice`.
`--text-file` accepts UTF-8 answers. `survey ask WRK-0001 SVY-0001` guides a human
at a console; use `--input-file answers.txt` for explicit prepared input. Type option
numbers separated by commas, or write a free-text answer (`text:` forces free text).
Recommendations never count as answers. Revised answers signal review; they do not
silently change accepted DECs or plans. Pending questions appear in next-actions.
## Exercise the agent contract
The `fixture-general` and `fixture-reviewer` adapters perform predictable
test transformations, with no model API or real semantic analysis. Use a disposable
project to exercise the complete refinement flow:
```sh
mkdir aie-fixture
cd aie-fixture
aie init --name "Refinement fixture" --general-adapter fixture-general \
--archive-policy reversible
aie work create "Compare ideas"
printf 'Rendering: measure layers\nInput: compare controls\n' > ideas.md
aie agent run WRK-0001 --task-file ideas.md --request-id first-ideas
aie survey ask WRK-0001 SVY-0001
printf 'Synthetic observation for the contract test\n' > evidence.txt
aie artifact add WRK-0001 --file evidence.txt --purpose "Explicit fixture evidence"
aie agent configure --activity analyze --adapter fixture-reviewer
aie agent run WRK-0001 --task "analyze: compare the fixture proposals" --request-id analysis
aie agent run WRK-0001 --task "summarize current context"
aie agent history WRK-0001 --format json
aie work context WRK-0001
aie project validate
```
General handles activities without a configured specialization; specialized configuration
wins. `--activity` selects explicitly. Fixture classification recognizes `analyze:` and
`summarize`; otherwise refinement groups one idea per line, optionally `Topic: idea`.
Profiles/models are fixture identifiers, not connections to an external provider.
The analyze fixture reads stored answers/evidence, archives one proposal under the
explicit policy and prepares another as a **draft** DEC. It never creates human approval.
`discussion archive WRK DSC --reason TEXT --evidence ART` requires policy `reversible`
and integrity-checked evidence; the default is `review-only`. `discussion restore`
requires a reason and retains the source and disposition history. Run records retain
requester, executor, source fingerprint, proposals and applied references. Identical
completed request IDs reuse their result; changed input is rejected. Context marks
older runs stale when sources change; summaries refresh only on an explicit new run.
See [the refinement contract](docs/discussion-slice.md) for format and failure limits.
## Connect local Codex and Claude CLIs
`codex-cli` and `claude-cli` return typed Discussion/synthesis proposals that AIE
validates before applying. Install/authenticate each provider's CLI separately;
neither is required for ordinary AIE use or automated tests. AIE itself still needs
no JVM or Node at runtime. Check the local interface without contacting a model:
```sh
aie agent doctor --format json
aie agent configure --adapter codex-cli --profile general --model default --timeout 120
aie agent configure --activity analyze --adapter claude-cli --profile reviewer
aie agent run WRK-0001 --activity summarize --task "Summarize the recorded goals and open questions"
aie agent history WRK-0001 --format json
```
Model `default` omits the provider model flag; explicit model names are passed as one
argument. Profile is an AIE role instruction, not a vendor configuration file. General
handles activities without a specialization; provider failure does not silently
switch executors. Without `--activity`, general first classifies the request; ambiguous
requests can yield a clarification survey. Previous RUN outputs travel with context
as derived interpretations. They do not become verified evidence by being summarized.
Calls use stdin, fixed arguments, a private temporary directory, provider tool controls
and timeout/output limits. Source repository working directories are not supplied.
Credentials stay in provider stores/environment. RUN provenance records observed CLI
version, configured role/model, duration and output digest; it does not claim an
unreported actual model. Transport errors leave canonical Work data unchanged.
For a nonstandard executable location, set `AIE_CODEX_EXECUTABLE` or
`AIE_CLAUDE_EXECUTABLE` locally. No arbitrary extra-arguments option is supported.
Guided setup can collect general and activity-specific executors before writing files:
```sh
aie init . --interactive
# Or provide explicit prepared UTF-8 answers:
aie init . --name "My Project" --interactive --input-file setup-answers.txt
```
`init .` without --name starts guidance when a console is present. `init --name TEXT`
stays noninteractive. No terminal/file or incomplete answers produces a diagnostic;
setup never calls a model. See [the CLI adapter contract](docs/cli-adapters.md) for
scope, supported controls, environment handling and actual validation results.
## Review, execute and close
Refine a draft with `decision update` (including `--text-file` for narrative), then
record an actual human approval. The following fixture assumes the human has agreed:
```sh
aie decision update WRK-0001 DEC-0001 \
--context "Resume a small project change" --approach "One bounded slice" \
--rationale "Keep the result verifiable" --by agent --reason "Discussion refined"
aie decision accept WRK-0001 DEC-0001 \
--by owner --reason "Explicitly agreed scope" --human-approved
aie plan create WRK-0001 --title "Deliver the slice" --decision DEC-0001
aie plan step add WRK-0001 PLN-0001 --title "Implement and test"
# Execute the actual change/tests externally, then attach their observed result:
aie artifact add WRK-0001 --file result.txt --purpose "Actual verification result"
aie plan step status WRK-0001 PLN-0001 TSK-0001 COMPLETED \
--by agent --reason "Recorded the observed test outcome" --evidence ART-0001
# The last required step consolidates Plan and Work automatically.
aie work check WRK-0001
aie work reconcile WRK-0001 --requester agent --format json
aie work archive WRK-0001 --by owner --reason "Delivery recorded"
```
The framework consolidates completed Plans and their recorded results, without a
model call or a new Work-level acceptance checklist. Unanswered questions/surveys,
draft DECs and explicit blocking states remain pending. Empty sets never complete.
`plan exclude WRK PLN [--step TSK] --by NAME --reason TEXT` reviews cancelled scope
while retaining its history. See [the contract](docs/plan-consolidation.md).
Inspection-only Works without Plans retain human-reviewed `work complete`, with
criterion evidence and `--integration PATH` or `--no-integration-needed "Reason"`.
Human approval remains an explicit attestation for significant commitments; system
completion is recorded separately and does not prove arbitrary prose semantically.
Plans are proportional: small Works need not have a DEC or plan. Plans can contain
multiple steps, `--depends-on TSK-0001`, acceptance criteria and recorded results.
Step execution requires completed dependencies and accepted referenced decisions;
completed steps require evidence. `plan update`, `plan step update` and status changes
allow explicit replanning. `decision reject` retains rejected proposals;
`decision revise` creates a linked draft whose approval supersedes the prior decision.
Use `decision history` / `plan history` to inspect snapshots and SHA-256 links.
Archived and terminal Works reject edits. Resume with
`aie work reopen WRK-0001 --by owner --reason "New observation"`; previous completion
records and all files remain. `status` hides archived Works unless `--all` is supplied.
No housekeeping command deletes records or requires a Lesson document. Legacy agenda
and Lesson import is deliberately deferred until this replacement is established.
## Coordinate an EPIC
An EPIC is a Work coordinating other Works in the same engineering project. Create
each Work with its own scope/criteria and link it explicitly. All children are
important by default; nested EPICs are allowed, with one parent per Work and no cycles.
```sh
aie work create "Coordinate delivery" --acceptance "The reviewed child results satisfy the delivery"
aie work create "First slice" --acceptance "First bounded result verified"
aie work create "Second slice" --acceptance "Second bounded result verified"
aie epic configure WRK-0001 --mode results --by agent --reason "Coordinate these slices"
aie epic link WRK-0001 WRK-0002 --by agent --reason "First delivery"
aie epic link WRK-0001 WRK-0003 --by agent --reason "Second delivery"
# Only after actual human review of the policy and parent/child scopes:
aie epic authorize WRK-0001 --by owner --reason "These results are sufficient for the parent criteria" --human-approved
aie epic show WRK-0001 --format json
aie epic check WRK-0001
```
Children with Plans complete through deterministic Plan consolidation; inspection-only
children can use human-reviewed `work complete`. Each
lifecycle command checks affected ancestors and returns their status/pending results
to the caller. Once important children and the parent's local checks pass, RESULTS
auto-completes using an owned ART receipt. Its origin is SYSTEM_EPIC and
humanApproved is false; the separate policy delegation remains explicit and traceable.
No LLM performs the state checks, and arbitrary prose criteria are not semantically
proved by the framework. `--requester` / `--request-id` correlate lifecycle reports.
An important cancelled child remains pending. Review a reduced scope with
`epic importance EPIC CHILD --optional --by NAME --reason TEXT`, then authorize the
revised policy explicitly. Optional children may remain open. Scope/link changes
invalidate prior authority. An empty important set never auto-closes. Use mode
`decomposition` when the parent delivers the decomposition itself: ordinary reviewed
completion applies and does not wait for child execution.
Reopening an important child reopens RESULTS ancestors and retains prior delivery
records. Optional changes do not invalidate them. Explicitly reopening an EPIC
suspends its automatic authority so new children/scope can be planned. Inspect
policy revisions with `epic history`; `epic reconcile [WRK]` recovers missed
propagation from files and does not duplicate receipts/completions on repeated calls.
See [the EPIC contract](docs/epics.md) for recovery and publication limits.
## Refine a Work
Update agreed scope without editing YAML manually:
```sh
ai-engineering work update WRK-0001 \
--goal "Define and validate the next small slice" \
--description "Scope refined during discussion" \
--motivation "Continue from recorded evidence" \
--constraint "Preserve existing source behavior" \
--constraint "Use isolated test data" \
--acceptance "A fresh execution reconstructs the context" \
--format json
```
Only supplied fields change. `--title` is also available; changing it preserves the
Work ID and folder. Status, creator, creation time, repository snapshots and all
discussion/decision/artifact references are retained. Identical updates do not rewrite
the file or change its modification timestamp. `work status` remains the lifecycle
command; updating scope does not approve a decision.
**List options replace the whole corresponding list.** Repeat `--constraint` or
`--acceptance` for every value to retain. Omitted lists are preserved. Clear them
explicitly with `--clear-constraints` or `--clear-acceptance`; a clear option cannot be
combined with values for the same list. Empty description/motivation strings clear
those fields; title, goal and individual list values must be nonblank.
For longer discussion entries, write a Markdown draft and import it:
```sh
ai-engineering discussion add WRK-0001 \
--kind SUGGESTION --author agent \
--text-file ./proposal.md
```
Choose exactly one of `--text` and `--text-file`. File paths resolve from the invocation
directory, including when `--project` points elsewhere. Files must contain nonblank
UTF-8 text and be at most 1 MiB. The complete content, including line endings and
Markdown, becomes discussion text; source front matter is not interpreted. The
source is read once and left unchanged. Use `artifact add` separately for evidence.
## Multiple repositories in one project
From an external project directory with existing `runtime/` and `studio/` Git
repositories, run:
```sh
ai-engineering init --name "Prometeu"
ai-engineering repo add runtime --path runtime --language rust
ai-engineering repo add studio --path studio --language java
ai-engineering repo list --format json
ai-engineering work create "Improve scene editor" \
--repo runtime --repo studio \
--goal "Evaluate portable scene rendering" \
--constraint "No external service" \
--acceptance "A fresh execution reconstructs the context"
ai-engineering discussion add WRK-0001 --kind QUESTION \
--text "Should we use layered Canvas?"
ai-engineering discussion add WRK-0001 --kind RESPONSE --responds-to DSC-0001 \
--text "Run a rendering experiment first"
ai-engineering artifact add WRK-0001 --file result.txt \
--purpose "Rendering measurements" --origin "manual experiment"
ai-engineering decision draft WRK-0001 --title "Evaluate layered Canvas" \
--context "Scene editor rendering" --approach "Prototype layers" \
--rationale "Measure before committing" --evidence ART-0001
ai-engineering work context WRK-0001 --format markdown
ai-engineering work context WRK-0001 --format json
ai-engineering project validate
```
The CLI never initializes or modifies Git repositories. To register an existing
submodule use `--type git-submodule`; the default is `linked-git`. Repositories are
read-only unless registration explicitly includes `--allow-edit`. This is policy
metadata; it does not replace an operating-system sandbox. No source edits or
configured validation commands are executed in these milestones.
`--project /path/to/project` works before or after subcommands. Otherwise discovery
searches upward, including when invoked inside a nested source directory.
Work IDs use uppercase hexadecimal with at least four digits:
`WRK-0009`, `WRK-000A`, …, `WRK-000F`, `WRK-0010`. Existing IDs retain their spelling;
legacy counters are upgraded when the next Work is created, without reusing reserved
IDs. Child IDs remain decimal and are scoped to their Work. New discussions can
be grouped with `--discussion <name>`. History remains append-only. `work status
WRK-0001 ACTIVE` changes an open Work's lifecycle status. Completion uses
Plan consolidation or human-reviewed `work complete` for inspection-only Works;
terminal Works return to ACTIVE through `work reopen`.
Every command provides `--help`; inspection commands support `--format json`.
New discussion files keep metadata in YAML front matter (`documentVersion: 2`) and
the complete text in the Markdown body. Headings, lists, tables and code blocks remain
readable directly in an editor. Markdown context/list output also displays prose
directly; JSON continues to expose the same `text` field.
Older discussion files with `text` in YAML are read without being rewritten. To
convert a Work's existing discussions explicitly with the updated CLI:
```sh
ai-engineering discussion migrate WRK-0001
```
Originals are backed up under `.ai-engineering/.local/backups/discussions/`; the
command reports the backup paths. IDs, authors, dates, text, Work metadata and exported
context data are preserved. Running the command again skips converted files. Migration
refuses legacy trailing notes rather than silently dropping them. Use the updated CLI
for files in document version 2; old CLI builds cannot read that new file format.
Exit codes: **0** success, **1** unexpected I/O/runtime failure, **2** invalid input
or unsafe path, **3** missing project/resource, **4** structural validation failure.
Validation commands return 4 with diagnostics even when the underlying issue is a
missing referenced resource or unsafe persisted path.
## Architecture and protocol
One Gradle project, one Spring Boot application, ten verified Modulith modules:
project, work, discussion, decision, planning, artifact, agent, verification, CLI and shared filesystem
infrastructure. Domain state is immutable Java records; persistence uses versioned
YAML and Markdown front matter. Files remain understandable without this CLI.
- [Architecture and reuse decisions](docs/architecture.md)
- [Protocol schema and persistence guarantees](docs/protocol.md)
- [Roadmap and acceptance criteria](docs/roadmap.md)
- [Experiments and actual validation results](docs/experiments.md)
No Agenda, Lesson or autonomous OODA engine is part of this implementation. Local
Codex/Claude integrations are limited to proposals; EPIC/conformance/state checks
run in framework code. The two fixture adapters remain available for offline use.