feat: bootstrap portable engineering protocol CLI

Implement milestones 0 and 1 with project and repository configuration, Work persistence, discussions, draft decisions, evidence integrity, and deterministic context exports.

Add Java 25, Spring Boot and Modulith, native distributions, CI workflows, protocol documentation, and the pinned SDKMAN GraalVM CE toolchain.

Validated with 20 passing tests and the macOS ARM64 native multi-repository continuity smoke.
This commit is contained in:
Nilton Constantino
2026-11-11 16:14:22 +00:00
commit f0e0dd061a
49 changed files with 2367 additions and 0 deletions
+100
View File
@@ -0,0 +1,100 @@
# Architecture
## Technology selection
The project deliberately stays on a compatible maintenance pair: **Spring Boot
4.0.9 and Spring Modulith 2.0.9**. The published Modulith 2.0.9 POM declares
Boot autoconfigure 4.0.9. Java compilation and bytecode target are both **25**,
without preview features. Gradle **9.3.1** supports running on Java 25; its Wrapper
distribution is pinned with SHA-256 verification.
The local development JDK is pinned through `.sdkmanrc` to SDKMAN candidate
`25.5.5.0+3-graalce` (GraalVM CE 25.5.5.0.3, OpenJDK 25.0.5.0.3). This
[GraalVM innovation release](https://www.graalvm.org/release-notes/25.5/) still uses
Java 25. It provides both JVM and Native Image tooling without changing compilation
target or application dependencies. Use `sdk env` for project-scoped activation;
the Gradle Wrapper remains responsible for the Gradle version.
Primary compatibility references checked during implementation:
- [Spring Boot 4.0 requirements](https://docs.spring.io/spring-boot/4.0/system-requirements.html): Java 25, Gradle 9 and GraalVM 25 are supported; Native Build Tools 0.11.5 is the documented version.
- [Published Modulith 2.0.9 dependencies](https://repo.maven.apache.org/maven2/org/springframework/modulith/spring-modulith-core/2.0.9/spring-modulith-core-2.0.9.pom): Boot 4.0.9 dependency.
- [Gradle Java compatibility](https://docs.gradle.org/9.3.1/userguide/compatibility.html): Java 25 support starts at Gradle 9.1.
- [Spring AOT Gradle integration](https://docs.spring.io/spring-boot/4.0/gradle-plugin/aot.html).
- [Picocli native configuration](https://picocli.info/#_graalvm_native_images): annotation-processor-generated reflection metadata.
- [GraalVM 25 release notes](https://www.graalvm.org/release-notes/JDK_25/): macOS x86_64 support was removed.
Picocli **4.7.7** handles argument validation, help and dispatch. Jackson 3 YAML/JSON
comes from Boot's BOM. JUnit is explicitly pinned to **5.13.4** because the requested
test baseline is JUnit 5; the Boot BOM otherwise selects a newer major generation.
AssertJ and Modulith/ArchUnit verify behavior and dependencies. There is no Spring AI,
Embabel, web starter, persistence server or workflow framework.
## Module boundaries
All packages below are under `net.intrepid.aiengineering`. A module's base package is
its explicit public API; implementation-only Git adapters live in `project.internal`.
`package-info.java` declares allowed dependencies. `ArchitectureTest` calls
`ApplicationModules.verify()` to enforce cycles, visibility and dependency rules.
| Module | Owns | May depend on |
|---|---|---|
| filesystem | Protocol codecs, safe paths, atomic file writes, locks, errors | none |
| project | Configuration, discovery, repository policy, read-only Git adapter | filesystem |
| work | Work identity, lifecycle, repository snapshots and reference manifest | project, filesystem |
| discussion | Append-only refinement entries and thread grouping | work, filesystem |
| decision | Draft DEC records | work, filesystem |
| artifact | Small evidence payloads and integrity metadata | work, filesystem |
| verification | Structural conformance and portable context assembly | all protocol modules |
| cli | Picocli commands, formats, exit codes | all APIs above |
Spring owns the process lifecycle and CLI bean, and supplies AOT processing. Protocol
services are ordinary Java classes with explicit path arguments and no retained Work
cache. This keeps direct tests and fresh instances simple. Infrastructure dependencies
remain inside services, outside immutable domain records. The small `GitAdapter` port
is internal to Project and can be replaced without changing Work contracts.
Context assembly belongs to Verification because it combines independently persisted
modules and refuses to export structurally inconsistent state. Work does not depend
on Discussion, Decision or Artifact, avoiding cycles. It owns stable references only.
## Persistence and native execution
Writers take an OS file lock per project. Complete UTF-8 files are written to a sibling
temporary file, forced to disk and published atomically. New files use a hard link
to avoid overwriting an existing destination. Existing files use atomic replacement;
unsupported filesystems fail instead of quietly falling back to partial writes.
New Works, projects and artifact payload bundles are staged before directory rename.
See the protocol document for crash limits and recovery.
Jackson parses explicit records, rejects unknown fields, duplicate YAML keys and
unsupported schema versions. Native hints register those records; Picocli's annotation
processor provides command reflection metadata. The Spring AOT tasks are wired by the
native build plugin. No application arguments are passed through Spring's property
parser. The entry point explicitly chooses non-web mode and closes its context.
## Earlier framework review and migration
Both implementations were inspected read-only:
- The user-identified Python repository at
`/Users/niltonconstantino/personal/workspace.personal/intrepid/discussion-framework`:
README, model/locking/index persistence and structural validation.
- The sibling Java `discussion-framework`: `DiscussionOrchestrator`,
`WorkflowValidator`, `WorkPacket`, `ArtifactDocumentParser` and
`FilesystemDiscussionRepository` concepts and implementations.
Reused concepts: deterministic context as the handoff surface, stable IDs, explicit
references, file locks, atomic replacement, duplicate/reference checks and separating
content from mechanical CLI state management. No Python code or dependency was copied.
The old generic Markdown inference was replaced by typed, versioned front matter;
semantic sections are explicit and cannot silently disappear from context parsing.
No legacy HTTP adapter, NDJSON global index, accepted-agenda gate, automatic housekeeping
deletion or Lesson model was retained. Work is the project-scoped aggregate; Discussion
is refinement within a Work. Source repositories were not changed.
There is no automated legacy importer in Milestone 1. A future migration should create
one Work per coherent agenda, retain original discussion/decision text with origin
references, and verify identifiers and links before publication. Legacy Lessons should
be reviewed for integration into canonical docs, not recreated as a required entity.
+149
View File
@@ -0,0 +1,149 @@
# Initial experiments
Execution host: macOS ARM64, 2026-11-11. JVM: Amazon Corretto 25.0.4+7.
Native toolchain: Oracle GraalVM for JDK 25.0.4+7.1 and Apple C compiler.
Resolved Jackson YAML/Databind: 3.1.7 (Boot BOM). No Python, Node, model API,
database or external runtime service was used.
## H1 — One non-web modular application is enough
Observation: the packaged Spring Boot application prints Picocli help/version and
terminates. Automated process tests enforce a 30-second termination limit. There
are no web/server dependencies. Modulith verifies eight modules and their declared
dependency graph.
Result: verified by `ArchitectureTest`, CLI smoke tests and independent JVM processes.
Spring handles process lifecycle and AOT; the portable protocol stays plain Java.
## H2 — Files alone preserve meaningful engineering state
The main integration fixture initializes an engineering project with two independent
Git repositories (`runtime`, `studio`), creates a Work referencing both, supplies goal,
constraints and acceptance criteria, appends a question/conclusion, persists a draft
DEC and attaches evidence. A fresh CLI instance invoked from `studio/src/editor`
reconstructs exactly the same exported context.
`ProcessContinuityTest` repeats the scenario through separate `java -jar` processes.
Each source repository contains a real fixture commit. The fresh context includes
both revision snapshots, author, discussion, DRAFT status and evidence SHA-256. JSON
exports from independent processes compare byte-for-byte. Project validation succeeds.
Result: verified without shared application memory or conversation history. The native
smoke script reproduces the same scenario with a separate native process per command.
## H3 — Structural safety and integrity are testable without a model
Verified cases: multiple Works, separate artifact workspaces, missing repositories,
unknown configuration fields/schema, malformed YAML and duplicate YAML keys,
traversal and symlink attempts, malformed/missing/duplicate Work identifiers,
missing decision files and evidence payloads, orphan records, required directories,
SHA-256 corruption, explicit question replies, reversible status, default read-only
access, immutable models, atomic no-overwrite writes and persistent ID reservations.
A real submodule fixture additionally verifies registration without rewriting
`.gitmodules`. It initially exposed a path equality bug between macOS `/var` and
`/private/var`. Comparing both real paths fixed it. A numeric-enum negative fixture
initially failed to alter Jackson's quoted YAML; the corrected fixture now verifies
that ordinal lifecycle values are rejected.
Result: final clean build has **20 tests, 0 failures, 0 errors, 0 skipped**.
The first test-first run intentionally failed because implementation classes did not
exist. Intermediate compilation caught interruption handling and a Jackson 2 feature
constant moved to Jackson 3's `EnumFeature`; both were corrected. These failures are
resolved in the final tested source.
## Commands and evidence
```sh
./gradlew clean test bootJar --console=plain
./gradlew dependencies --configuration runtimeClasspath --console=plain
JAVA_HOME=<graalvm25-home> GRAALVM_HOME=<graalvm25-home> \
./gradlew nativeCompile nativeDistZip nativeDistTar \
-PtargetPlatform=macos-arm64 --console=plain
```
The clean test/JAR build passed. Generated test reports are under
`build/reports/tests/test`; raw JUnit XML is under `build/test-results/test`.
Runtime dependency inspection confirmed the pinned Spring pair and Jackson 3.1.7.
An initial Native Image compilation succeeded, but only the final rebuilt binary's
execution results count as native functionality verification.
Native result: **compiled and executed successfully on macOS ARM64**. The initial
verified Native Image build completed in approximately 1m 13s (Gradle invocation:
1m 23s). `ai-engineering` is a Mach-O ARM64 executable. ZIP and tar.gz archives were
generated with executable permissions and inspected.
The native continuity smoke passed with no JDK on the configured execution path:
```sh
JAVA_HOME=/nonexistent PATH=/usr/bin:/bin:/usr/sbin:/sbin AIE_KEEP_FIXTURE=1 \
bash scripts/native-smoke.sh "$PWD/build/native/nativeCompile/ai-engineering"
```
Observed output: `Native multi-repository continuity smoke passed.` Every CLI command
in the script starts a separate process. Help/version, exit code 2 for invalid input,
both repository registrations, Work creation, discussion, evidence, DEC draft,
nested discovery, JSON byte equality, Markdown export and final structural validation
all succeeded. The engineering fixture root had no `.git` directory.
The retained local demonstration fixture is at
`/var/folders/0t/tfsqc14j5wbgh9h4s4dytwb00000gq/T/ai-engineering-smoke.ziQUkw`.
Inspect `.ai-engineering/works/WRK-0001-improve-scene-editor/`, `before.json`,
`after.json`, `context.md` and `validation.json` there. The JSON validation report
contains `valid: true` and an empty issue list. Temporary fixtures are not required
for future use; the committed script can recreate the experiment.
Nonblocking Native Image build warnings: Picocli's generated proxy metadata uses a
deprecated configuration resource option; SLF4J reports no provider during image
analysis. Neither prevented compilation or the full native execution scenario.
An earlier exploratory native invocation was terminated with exit 137 during a clean
rebuild; it was not treated as a functional pass. The final rebuilt binary passed the
full smoke scenario above.
## H4 — SDKMAN GraalVM Community toolchain
The local project JDK is now pinned in `.sdkmanrc` to SDKMAN candidate
`25.5.5.0+3-graalce`. Direct inspection identifies GraalVM Community Edition
25.5.5.0.3+1.1, based on OpenJDK 25.0.5.0.3, with Native Image included. Gradle
9.3.1 recognizes it as both launcher and daemon JVM. A clean Java/AOT/JAR build
and all **20 tests** passed again using this installation. The language and bytecode
target remain Java 25. CI native builds select the Community 25.5 series; CI itself
has not been executed locally.
Native compilation with Native Build Tools 0.11.5 also succeeded (image generation
approximately 1m 8s; complete clean test/JAR/native invocation 1m 40s). The full native
continuity smoke passed with the CE-built executable, including project validation
and byte-identical context exports across processes. An isolated copy containing only
the executable passed the same scenario with both `JAVA_HOME` and `GRAALVM_HOME` set
to nonexistent paths and no JDK on the configured PATH. Native Image also emitted
optional desktop libraries beside the build output; the implemented CLI commands
were verified without those libraries present beside the isolated executable.
The compiler reports nonblocking deprecation warnings for `--no-fallback`,
`FallbackThreshold` and Picocli's proxy configuration. No application dependency or
native plugin upgrade was needed for the tested functionality.
Reproduction after `sdk env`:
```sh
export GRAALVM_HOME="$JAVA_HOME"
./gradlew clean test bootJar nativeCompile --console=plain
JAVA_HOME=/nonexistent GRAALVM_HOME=/nonexistent PATH=/usr/bin:/bin:/usr/sbin:/sbin \
bash scripts/native-smoke.sh "$PWD/build/native/nativeCompile/ai-engineering"
```
## Limits and next experiment
CI workflows are provided but have not been executed from this local workspace.
Linux x86_64 and Windows x86_64 binaries are not locally verified. GraalVM 25 removed
macOS x86_64 support, so that release target has no claimed native artifact.
Filesystem atomicity is per file/directory publication, not a transaction across the
child record and parent manifest. Orphans after interruption are detected, not
automatically repaired. Manual concurrent editors and adversarial path replacement
require external coordination/sandboxing. Artifact attachment is limited to small
files, and no full provenance/environment capture or agent intelligence is implemented.
Next recommended experiment: define a deterministic AgentAdapter test contract and
demonstrate two fake adapters continuing the same Work. Keep that work in Milestone 2;
no adapter implementation is included here.
+167
View File
@@ -0,0 +1,167 @@
# Portable engineering protocol — schema 1
## Project root and configuration
The engineering root is the parent of `.ai-engineering`. Repository paths resolve
against that root, independently of the CLI's current directory. Discovery searches
upward or accepts an explicit `--project` root (or `.ai-engineering` directory).
```yaml
schemaVersion: 1
project:
id: prometeu
name: Prometeu
repositories:
- id: runtime
path: runtime
type: linked-git
language: rust
access:
read: true
edit: false
validationCommands: []
```
IDs use lowercase ASCII letters, digits and hyphens, beginning with a letter.
Repository types are `linked-git` and `git-submodule`. Omitted `access` defaults to
`read: true, edit: false`. If `access` is supplied its booleans are explicit; edit
requires read. Language is optional. `validationCommands` is optional metadata,
never executable through this release. Unknown fields and schema versions fail validation.
Absolute repository paths, `..` components, backslashes and symlinks beneath the root
are rejected, including links that point inside the root. This stricter initial rule
avoids ambiguous link policies. Repositories cannot be the engineering root or live
inside `.ai-engineering`. Git inspection verifies an independent repository top-level;
submodule registration additionally verifies its superproject. Paths must exist.
No submodules are automatically created, moved or updated.
Policy controls protocol intent, not OS security. Concurrent malicious filesystem
replacement is outside this CLI's sandbox guarantee; use OS isolation for untrusted
processes. The adapter clears inherited `GIT_*` variables and disables optional Git locks.
## Work and files
```text
.ai-engineering/
ai-engineering.yaml
.gitignore
.local/write.lock # ignored, no semantic state
contracts/ # reserved for project contracts
works/
.sequence.yaml # canonical high-water mark; commit this
WRK-0001-improve-scene-editor/
work.md
discussions/DSC-0001.md
decisions/DEC-0001.md
artifacts/
references/
experiments/
evidence/ART-0001/
artifact.yaml
payload
outputs/
runs/ # ignored temporary execution outputs
```
`work.md`, discussion and decision files contain YAML front matter, delimited by
`---`. All semantic content lives in the front matter in this version; trailing
Markdown is reserved for noncanonical notes and is not exported. YAML multiline strings
remain human-readable. Edit the typed fields to update canonical content. There is no
Java object serialization and no dependency on conversation state.
Work fields: `schemaVersion`, `id`, `title`, `description`, `goal`, `motivation`,
`constraints[]`, `acceptanceCriteria[]`, `status`, `createdAt`, `updatedAt`, `author`,
`repositories[]`, `discussions[]`, `decisions[]`, `artifacts[]`. Each repository snapshot
records `repositoryId`, relative `path`, exact `head` commit and `dirty`. An empty
repository records `UNBORN`, not an invented commit. Snapshots are taken at Work
creation and remain stable; they are not claims about later experiments.
Work IDs are project-wide, monotonic `WRK-` numbers with at least four digits. A
persisted sequence reserves identity before publishing a Work; crashes may leave gaps.
Do not remove or rewind the sequence. Discussion/decision/artifact IDs are **Work-local**;
their full identity is `(project, Work ID, child ID)`. Work reference arrays determine
history order. Directory names are a convenience; ID stability does not depend on title.
Duplicate Work IDs and directory/record mismatches fail validation.
Initial statuses: OPEN, ACTIVE, BLOCKED, COMPLETED, CANCELLED. `work status` permits
reorientation without irreversible lifecycle gates. Completion currently records intent;
it does not prove acceptance criteria. Conformance automation belongs to Milestone 5.
## Discussion and draft decisions
Each discussion entry has `schemaVersion`, `id`, `workId`, `discussionId` (thread name),
`kind`, `author`, `createdAt`, optional `respondsTo`, and `text`. Multiple threads can
coexist. Kinds: NOTE, QUESTION, RESPONSE, SUGGESTION, ASSUMPTION, HYPOTHESIS,
INVESTIGATION, ALTERNATIVE, ISSUE, CONCLUSION. Entries are appended, not rewritten.
Text is supplied by a human or external agent; the CLI never fabricates conclusions.
A response must refer to an earlier entry in the same Work. Outstanding questions
include QUESTION and ISSUE entries without an explicit RESPONSE or CONCLUSION pointing
to them. This mechanical definition does not infer agreement or approval from prose.
Draft DEC fields: `schemaVersion`, `id`, `workId`, `title`, `context`, `approach`,
`alternatives[]`, `rationale`, `risks[]`, `acceptanceCriteria[]`, `approvalStatus`,
`author`, `createdAt`, `evidence[]`. Only DRAFT is accepted in this release. Empty
refinement fields are legal in drafts. The CLI initially supplies context, approach,
rationale and evidence; remaining fields can be refined in the canonical YAML.
No approval, supersession or autonomous operational decision is implemented.
## Evidence and integrity
`artifact add` copies one regular file up to 1 MiB. Its manifest records schema, ID,
Work ID, origin, purpose, Work-relative payload path, SHA-256, byte size, author and UTC
creation time. The origin defaults to the source basename; use `--origin` for a useful
portable provenance reference. Payload bytes are exact. The fixed `payload` filename
avoids filename ambiguity; origin preserves the supplied source identity.
Validation checks required directories, IDs, schemas, repository references, child
ownership, missing records, orphan records and payload integrity. Evidence paths stay
inside the Work. Large resources should remain in their canonical location and be
described in discussion/reference files, rather than copied into this workspace.
`artifacts/outputs` is versionable for deliberately retained outputs; use ignored `runs/`
for large temporary results. No command creates large output files by default.
## Context and continuity
`work context --format markdown|json` loads only canonical project/Work files. It
includes all Work fields (metadata separately in Markdown), stored source revisions,
ordered discussion history/conclusions, explicit draft decisions, unresolved questions
and complete artifact references with integrity metadata. JSON has top-level
`schemaVersion`, `work`, `discussionHistory`, `decisions`, `outstandingQuestions`,
`artifacts`. It does not embed arbitrary evidence payloads.
No export timestamp, current absolute path or live Git state is injected. Unchanged
files yield byte-identical output across new instances and processes. Export rejects
broken references or integrity failures. It requires neither a model nor live Git
inspection; `project validate` separately verifies live repositories.
## Write guarantees and recovery
Protocol mutations hold `.local/write.lock`, an OS lock released automatically at
process exit. Individual files publish atomically; create operations reject overwrites.
New project/Work/artifact directories are staged and renamed. Work sequence reservation
precedes directory publication to avoid identity reuse after interruption.
A child file/bundle is published before its reference is atomically added to `work.md`.
The filesystem is not a multi-file transactional database: interruption in that window
can leave an orphan, which validation reports. Inspect its metadata and either add its
ID to the Work reference array or remove it after review. Hidden `.tmp` directories are
incomplete staging outputs, ignored by readers; inspect and clean them manually after
an interrupted write. Concurrent manual editors do not honor the CLI lock. Directory
entry durability under sudden power loss depends on the host filesystem.
## Future contracts (design only)
- AgentAdapter: accept versioned context and return proposed questions, suggestions,
assumptions or draft content with provenance. The caller decides what to persist;
no provider or orchestration dependency is part of schema 1.
- OODA iteration: identify Work and prior observations, orientation, operational
choices and attempts. Phases can repeat or be skipped; no four-stage workflow gate.
- Attempt: stable identity, hypothesis, input references, exact repository revisions,
granted permissions, invocation/environment, observations, outputs, result and
evidence references. Evidence and interpretation remain distinguishable.
- Coordinated execution: independent per-repository worktrees and commit records,
integrated validation, explicit authority for destructive actions, pushes and merges.
No claim of atomic commits across repositories.
- Conformance: evaluate explicit criteria against evidence and canonical code/spec/test
updates. No required Lesson artifact. Later schema changes require explicit migration.
+63
View File
@@ -0,0 +1,63 @@
# Roadmap
## Milestone 0 — Bootstrap
Implemented: Java 25, Spring Boot 4, non-web Picocli entry point, one Gradle Kotlin DSL
project with Wrapper, Modulith boundaries, help/version smoke tests, Spring AOT,
Native Image configuration and native archive tasks. Acceptance: clean tests and JAR
build pass, CLI terminates and errors are meaningful. Native results are tracked in
the experiment log, separately from configuration claims.
## Milestone 1 — Portable engineering protocol
Implemented: project discovery/init, schema 1 configuration, linked Git/submodule
registry and default read-only policy, stable Work IDs, local artifact workspaces,
discussion history, draft DECs, evidence integrity, structural validation and stable
Markdown/JSON context. Temporary-fixture tests cover continuity, independent Works,
multiple repositories and malformed/missing/unsafe state.
Acceptance demonstration: initialize a project with two independent Git repositories,
create a Work referencing both, append discussion, persist a draft decision and
evidence, export context, then reconstruct the same context in independent processes
using only files. No framework state or model history may be required.
Platform release targets: Linux x86_64, macOS ARM64, macOS x86_64 and Windows x86_64.
The workflow builds the three available GraalVM 25 platforms. **macOS x86_64 is blocked
by upstream GraalVM 25 support removal**, documented in the
[official release notes](https://www.graalvm.org/release-notes/JDK_25/).
Do not substitute an older Java target or label an ARM binary as Intel. A future
distribution experiment must identify and validate a maintained compatible toolchain
before enabling that native release target. CI configuration is not evidence that CI
has run; see actual local results in experiments.
## Milestone 2 — Intelligent Discussion (design only)
Introduce a replaceable AgentAdapter using the portable context contract. Begin with
deterministic fakes and persist proposed questions/alternatives/assumptions as explicit
entries. Acceptance: two different adapters continue the same Work without private
history; humans retain authority over formal commitments. No model integration yet.
Recommended next experiment: two deterministic adapters read the same schema 1 context,
one proposes a clarification and another continues from its persisted response. First
confirm the context contract is sufficient; then choose provider integrations.
## Milestone 3 — Adaptive OODA (design only)
Add observations, orientation, operational decisions and Attempts using the conceptual
contracts in protocol.md. Acceptance: experiment evidence changes the next action;
history prevents repeating an invalidated attempt without a stated new hypothesis.
No rigid Observe→Orient→Decide→Act pipeline or autonomous engine is implemented now.
## Milestone 4 — Coordinated repositories (design only)
Add per-Work permissions, independent worktrees, exact source revisions, coordinated
validation and separate commit outcomes. Acceptance: partial success in one repository
remains visible; destructive actions, push and merge require explicit authorization.
Never assume atomic commit across repositories.
## Milestone 5 — Knowledge integration and conformance (design only)
Check acceptance criteria against evidence; identify impacted canonical specs, tests,
code and documentation; expose conformance checks to agents and CI. Acceptance: a Work
can demonstrate its outcome and preserve actionable knowledge without a mandatory
Lesson model or redundant documentation trail.