Files
ai-engineering/docs/architecture.md
T
Nilton Constantino bd2c9133da feat: add agent coordination and automatic work completion
Add portable topics and surveys, decision approval history, execution plans, local Codex and Claude adapters, and general agent routing. Consolidate Plan and EPIC results through deterministic completion receipts, reviewed cancellation scope and recoverable reconciliation.

Track the completed WRK-0001 and WRK-0002 records, protocol documentation, source manifests and delivery evidence.

Validation: 117 tests including Modulith; GraalVM 25 native compilation, standalone smoke and ZIP/TAR verification on macOS ARM64; identical persisted context across JVM and native executions.
2026-11-14 01:12:21 +00:00

16 KiB

Architecture

Technology selection

The project deliberately stays on a compatible maintenance pair: Spring Boot 4.0.9 and Spring Modulith 2.0.9. The published Modulith 2.0.9 POM declares Boot autoconfigure 4.0.9. Java compilation and bytecode target are both 25, without preview features. Gradle 9.3.1 supports running on Java 25; its Wrapper distribution is pinned with SHA-256 verification.

The local development JDK is pinned through .sdkmanrc to SDKMAN candidate 25.5.5.0+3-graalce (GraalVM CE 25.5.5.0.3, OpenJDK 25.0.5.0.3). This GraalVM innovation release still uses Java 25. It provides both JVM and Native Image tooling without changing compilation target or application dependencies. Use sdk env for project-scoped activation; the Gradle Wrapper remains responsible for the Gradle version.

Primary compatibility references checked during implementation:

Picocli 4.7.7 handles argument validation, help and dispatch. Jackson 3 YAML/JSON comes from Boot's BOM. JUnit is explicitly pinned to 5.13.4 because the requested test baseline is JUnit 5; the Boot BOM otherwise selects a newer major generation. AssertJ and Modulith/ArchUnit verify behavior and dependencies. There is no Spring AI, Embabel, web starter, persistence server or workflow framework.

Module boundaries

All packages below are under net.intrepid.aiengineering. A module's base package is its explicit public API; implementation-only Git adapters live in project.internal. package-info.java declares allowed dependencies. ArchitectureTest calls ApplicationModules.verify() to enforce cycles, visibility and dependency rules.

Module Owns May depend on
filesystem Protocol codecs, safe paths, atomic file writes, locks, errors none
project Configuration, discovery, repository policy, read-only Git adapter filesystem
work Work identity/lifecycle, repository snapshots, references and versioned EPIC policy/graph project, filesystem
discussion Entries, topics, surveys, answer history and proposal dispositions work, project, artifact, filesystem
decision Draft, approval, revision and supersession records work, filesystem
planning Lightweight plans, dependencies, step results and revision history work, decision, artifact, filesystem
artifact Small evidence payloads and integrity metadata work, filesystem
agent Adapter contract, deterministic fixtures, routing and proposal application project, work, discussion, decision, planning, artifact, filesystem
verification Structural conformance, context, reviewed closure and deterministic EPIC coordination all protocol modules
cli Picocli commands, formats, exit codes all APIs above

Spring owns the process lifecycle and CLI bean, and supplies AOT processing. Protocol services are ordinary Java classes with explicit path arguments and no retained Work cache. This keeps direct tests and fresh instances simple. Infrastructure dependencies remain inside services, outside immutable domain records. The small GitAdapter port is internal to Project and can be replaced without changing Work contracts.

Context assembly, readiness and evidence-mapped closure belong to Verification because they combine independently persisted modules and refuse to export structurally inconsistent state. Work does not depend on Discussion, Decision or Artifact, avoiding cycles. It owns stable references only.

The engineering project is independent of repository count: a plain folder can use the protocol with no Git dependency, a single Git repository can register the root using ., and a larger workspace can register several child repositories. The same configuration schema and services support each case; no separate project mode or additional repository type is needed.

Work refinement uses an immutable update value that distinguishes omitted fields from explicitly emptied lists. The domain preserves identity, lifecycle and references; the filesystem service locks the project and replaces front matter while retaining trailing notes. Discussion file input belongs to the CLI adapter: Picocli enforces exclusive text sources and a bounded strict UTF-8 read supplies the existing domain entry. Neither feature changes canonical record schemas or introduces new modules.

Persistence and native execution

Writers take an OS file lock per project. Nested service calls on the same thread reuse the outer lock; other threads/processes cannot interleave the proposal application. Complete UTF-8 files are written to a sibling temporary file, forced to disk and published atomically. New files use a hard link to avoid overwriting an existing destination. Existing files use atomic replacement; unsupported filesystems fail instead of quietly falling back to partial writes. New Works, projects and artifact payload bundles are staged before directory rename. See the protocol document for crash limits and recovery.

Jackson parses explicit records, rejects unknown fields, duplicate YAML keys and unsupported schema versions. Native hints register those records; Picocli's annotation processor provides command reflection metadata. The Spring AOT tasks are wired by the native build plugin. No application arguments are passed through Spring's property parser. The entry point explicitly chooses non-web mode and closes its context.

Discussion document version 2 separates typed YAML metadata from an opaque Markdown body. The filesystem codec maps that body to the domain record's existing text field; it does not infer structure from prose headings. The reader accepts legacy YAML-only text as well. Explicit discussion migration backs up originals, preserves semantic state and skips converted files. New decision narrative (text) and plan description use the same Markdown-body codec; legacy draft DECs remain readable. Work front matter remains canonical, with additive operational fields. Markdown context renders discussion/decision prose without YAML escaping.

Operational replacement baseline

The user expanded the original bootstrap scope to replace the existing discussion workflow before introducing intelligent adapters or a legacy importer. Planning is a concrete ninth module, not a custom workflow engine. A plan records editable scope, dependencies and externally executed results. Starting/completing steps checks their dependencies and accepted decisions. No repository command is executed by Planning. Plans/steps can be reopened and revised when observations change.

DEC approval is an explicit human-attestation command, with actor, reason and time. Accepted records are immutable through the CLI; revision creates a new linked draft. Acceptance supersedes the prior accepted DEC. Decision and plan changes preserve the previous complete file under .history, linked by SHA-256. Hashes detect broken chains; they are not signatures or protection against someone rewriting every file. Cross-record supersession uses per-file atomic publication; validation diagnoses incomplete links. There is no distributed transaction or automatic history repair.

Verification separates structural validity, mechanical readiness and human judgment. PlanCoordinator consolidates valid Plan results without invoking an adapter, records SYSTEM_PLANS receipts and invokes existing EPIC propagation. Planning owns content-bound scope reviews of cancellations; Verification coordinates public mutations/reconciliation without a reverse dependency. Reads expose the same pending reasons without writes. Explicit reopening leaves time to change Plans. Legacy optional fields are omitted to preserve historical digests. See the contract.

For inspection-only Works without Plans, manual readiness requires explicit criteria, resolved questions, reviewed DECs and terminal plans. Completion records criterion-to-artifact mappings, a stable scope digest, human approval, and canonical integration file digests or a reason no update is needed. Canonical-file hashes are historical snapshots; later source evolution does not invalidate an old delivery. Current completed Work records must still match their frozen scope. Archive retains every file. Reopening preserves completion and lifecycle history and permits new discussion/evidence.

Project owns provider-neutral entry resources copied into .ai-engineering by init/onboard, including a machine-readable manifest. A managed root AGENTS.md block points to those resources while preserving surrounding rules. Changed instructions are backed up; repeated onboarding is idempotent. Context includes the guide. This supports agents that inspect the project; it cannot force discovery by an arbitrary agent. status and next-actions derive diagnostics from persisted state without executing work or inventing human approval.

Discussion refinement and adapter boundary

Discussion still owns refinement; DEC and Plan remain its downstream commitments. Topics and surveys are functional records in Discussion. Legacy threads are derived as implicit topics; reads never migrate files. Answers append explicit supersession links. Archive/restore dispositions retain proposals and require policy/evidence. There is no separate Idea entity. See record contracts.

Agent is the tenth module. AgentAdapter accepts immutable context/task/executor values and returns a typed AgentProposal; it receives no filesystem Path, persistence service or execution tools. Only two deterministic adapters are registered. General selects an activity when omitted and provides fallback for unspecialized activities; an explicit activity selects its configured executor. Classification in fixtures uses task markers, not natural-language model interpretation.

AgentService validates the entire proposal before applying it through domain APIs. It rechecks the source fingerprint under the project lock to reject concurrent stale application. A RUN preserves source digest, selected executor, requester, task, proposal, applied references and terminal/error state. Completed request IDs provide idempotent replay of identical input. An interrupted APPLYING run blocks new runs pending inspection. Publication is atomic per file, not a rollback transaction; FAILED records and validation expose partial application for deliberate repair.

Verification reads run provenance and computes staleness from canonical semantic sources, excluding run bookkeeping. State checks never invoke an adapter. A summary is a recorded result; fresh context does not implicitly rerun it. Adapter output cannot provide human survey answers, approve DECs, complete Works or execute commands. Typed records and nested proposal values have Native Image reflection hints. The initial slice did not add provider integration; the local CLI slice below extends it.

Local provider CLI transport

PLN-0003 implements the human-selected Codex/Claude CLI transport for Discussion and synthesis. The agent module retains the same public adapter boundary. Its internal LocalProcess uses fixed ProcessBuilder vectors, bounded streams, virtual threads, timeout/descendant termination and private temporary workspaces. It receives no engineering root. Provider tool/configuration controls constrain proposals, while OS/provider policy remains separate. The full contract and verified CLI references are in cli-adapters.md.

Versioned wire schemas normalize provider output into existing AgentProposal values; the framework assigns identity and performs ownership/policy preflight. Classification can request clarify surveys. An explicit activity skips that provider call. Immutable previous RUNs travel in context, but derived run bookkeeping stays out of semantic source fingerprints. Only validated results publish canonical RUNs. Earlier transport failures return safe diagnostics without mutating Work state or storing raw stderr.

Optional invocation provenance captures observed version, configured role/model, duration and output digest. Absent optional fields are omitted to preserve older fingerprints/digests. Setup collects/validates all answers before initialization and does not invoke providers. No Node SDK, HTTP client framework or external service is required by tests. Deterministic JDK subprocess fixtures use the production transport.

EPIC coordination

PLN-0002 implements DEC-0003 within the existing modules. Work owns the EPIC marker, definition, forest topology, revision history and explicit policy authorization. Verification coordinates lifecycle writes and checks affected ancestors, using Artifact for owned aggregate receipts. No extra module, broker, workflow engine or agent invocation is needed. Low-level storage callers must reconcile explicitly; public CLI/Verification lifecycle entry points propagate within the project lock.

Policy authority binds parent/important-child scopes and links, excluding execution status. Important child completions and local readiness control RESULTS completion; DECOMPOSITION remains a reviewed human delivery. System completions record their origin, policy delegation and receipt, with humanApproved=false. Historical direct human completions keep their existing digest/approval semantics.

Frozen EPIC scope digests include current important delivery records; optional execution can continue independently. Deterministic serialization uses ordered maps and canonical records across JVM/native processes. Automatic receipts exclude prior receipts from their input key, allowing reuse after interrupted publication. Reconciliation derives state from files; no mutable in-memory index is authoritative. Details and explicit reopening/reauthorization rules are in epics.md.

Earlier framework review and migration

Both implementations were inspected read-only:

  • The user-identified Python repository at /Users/niltonconstantino/personal/workspace.personal/intrepid/discussion-framework: README, model/locking/index persistence and structural validation.
  • The sibling Java discussion-framework: DiscussionOrchestrator, WorkflowValidator, WorkPacket, ArtifactDocumentParser and FilesystemDiscussionRepository concepts and implementations.

Reused concepts: deterministic context as the handoff surface, stable IDs, explicit references, file locks, atomic replacement, duplicate/reference checks and separating content from mechanical CLI state management. No Python code or dependency was copied. The old generic Markdown inference was replaced by typed, versioned front matter; semantic sections are explicit and cannot silently disappear from context parsing.

No legacy HTTP adapter, NDJSON global index, accepted-agenda gate, automatic housekeeping deletion or Lesson model was retained. Work is the project-scoped aggregate; Discussion is refinement within a Work. Source repositories were not changed.

There is no automated legacy importer. Migration is deferred until the operational replacement is established. A future importer should create one Work per coherent agenda, retain original discussion/decision text with origin references, and verify identifiers and links before publication. Legacy Lessons should be reviewed for integration into canonical docs, not recreated as a required entity.