diff --git a/README.md b/README.md index e0ca6bb4..35668f82 100644 --- a/README.md +++ b/README.md @@ -139,6 +139,9 @@ correctly returns `NO_FRAME_AVAILABLE`; it is not a runtime defect. Noninteracti clients must also provide an approval path for MCP work calls; the acceptance used Codex's approval review with a project-scoped workspace-write sandbox. +The completed baseline's architecture, acceptance evidence and post-implementation +follow-ups are consolidated in [LSN-0076](discussion/lessons/DSC-0068-studio-headless-runtime-ai-first/LSN-0076-ai-first-studio-through-shared-authority-and-headless-evidence.md). + ## Requirements - JDK 21 or newer recommended for local development diff --git a/discussion/lessons/DSC-0068-studio-headless-runtime-ai-first/LSN-0076-ai-first-studio-through-shared-authority-and-headless-evidence.md b/discussion/lessons/DSC-0068-studio-headless-runtime-ai-first/LSN-0076-ai-first-studio-through-shared-authority-and-headless-evidence.md new file mode 100644 index 00000000..3eb8510c --- /dev/null +++ b/discussion/lessons/DSC-0068-studio-headless-runtime-ai-first/LSN-0076-ai-first-studio-through-shared-authority-and-headless-evidence.md @@ -0,0 +1,303 @@ +--- +id: LSN-0076 +ticket: studio-headless-runtime-ai-first +title: AI-first Studio Through Shared Authority and Headless Evidence +created: 2026-09-29 +tags: [studio, runtime, headless, ai, mcp, automation] +--- + +## Context + +Owner: **studio**, with runtime preparation in `../runtime` and language-owned PBS +documentation/LSP integration. DSC-0068, accepted DEC-0060 and PLN-0139 delivered +28 ordered steps: runtime (01–07), DEV/shared services (08–14), MCP (15–21), and +real integration/acceptance (22–28). Those workflow identifiers are historical +references; maintained contracts live in the specs linked below. + +The original headless host was an in-process runtime API. Studio's human Debug +was useful, but neither a public automation protocol nor a safe authority boundary +for another editor. The goal was an external coding agent able to learn PBS, +edit a game, build/run it, see its pixels and iterate while the developer watches. +A complete gameplay library or embedded AI chat was not a prerequisite. + +## Key Decisions + +### Separate ownership, not parallel implementations + +The external coding client owns conversation, model/provider, native file edits +and budget. Studio owns project authority, build orchestration, process supervision +and the shared observation model. Runtime owns execution and framebuffer semantics. +The language frontend/SDK owns its references; the existing project LSP owns semantic +queries. DEV and MCP consume the same neutral services. + +```text +External coding agent -- local MCP HTTP --> Studio API/Core <-- DEV (JavaFX) + | + compiler/packer + existing LSP + | + supervised runtime JSONL/stdio + | + canonical PNG + frame identity +``` + +The configured installation provides `prometeu headless --stdio`. UTF-8 JSONL is +versioned, bounded and protocol-only on stdout; logs belong on stderr. A process +contains one session, with no implicit conflicting-work queue. Handshake establishes +runtime/protocol versions, capabilities and effective limits. Human Debug keeps its +own host/protocol. API/Core do not depend on JavaFX, Spring, HTTP/MCP or wire DTOs. + +This avoids a provider adapter inside Studio and avoids automating the human debugger +as the execution API. The deliberate cost is an explicit process boundary and +reconfiguration of the external client after Studio restarts. + +### Connection, authority and operation receipts are different identities + +Authentication to loopback `/mcp` permits discovery/context/reference reads, not +editing authority. A human grants a specific project in DEV; acquisition then +returns a task identity. Manual Debug and agent work exclude each other in services, +not only through disabled UI buttons. Waiting for acquisition already blocks +conflicting manual work. A disconnected client does not release the project. + +An operation's `clientRequestId` deduplicates a build or advance within the task. +The same ID and parameters return the same operation; changing parameters fails. +Lookup recovers a lost receipt without replaying frames. These records are in-memory, +not a source journal or restart-recovery mechanism. Runtime wire correlation has +its own `requestId`; neither identity is a substitute for valid task authority. + +### Return is a cleanup barrier + +The sequence is: invalidate task authority → stop managed work/processes → refresh +open documents from saved files → enable manual control. A late result cannot publish +a new capture or restore authority. Closing only DEV does not end the task; closing +the project/app does. Reopening starts manual, never with recovered agent authority. + +Compile errors, guest faults and runtime loss remain distinct outcomes. The first +two permit correction within the same task. Runtime loss requires explicit session +replacement, not replay. Forced cleanup reports infrastructure failure first; a +later explicit reclaim may acknowledge success only after the process tree is dead. + +The adversarial gate exposed a real app-teardown gap: UI closure alone did not own +every project session. The project-session factory now tracks and closes sessions +before dependent executors/LSP are destroyed, attempts other sessions after a failure, +and rejects new opens once closure begins. Cancellation also tolerates a concurrent +writer/timer shutdown so that it cannot bypass cleanup waiting. + +### Direct edits require truthful boundaries + +The agent edits actual project files. No worktree, merge/apply stage, code rollback, +Git cleanliness gate or automatic commit is imposed. Studio offers save-all-and-grant +for its dirty buffers and refreshes from disk on return. External unsaved buffers, +concurrent native writes and Git discipline remain the developer's responsibility. +Revocation cannot stop an external process from writing files directly; the developer +must also stop that client. SDK/external libraries are readable, not editable by +the game task. Read-only semantic refactoring scopes are enforced by the adapter. + +### Pixels are evidence; metadata explains which evidence + +MCP returns the lossless canonical PNG as native image content, plus session/build, +logical frame, render FrameId, dimensions, stride and pixel format. DEV projects +the same capture, with an explicit stale label after edits/rebuild/session changes. +Logical frames and render frames are independent counters. Neither a hash nor a +desktop screenshot replaces framebuffer pixels. Captures do not travel through +temporary image files, and there is no continuous video requirement. + +Default bounds are 600 frames/advance, 64 KiB/command, 32 MiB/response including +image/envelope, 5 s handshake, 60 s call, and 5 s cancel/shutdown grace. Oversized +requests fail rather than clamp. Deadlines supervise processes, not simulated time. + +## Final Result and Acceptance Evidence + +All 28 plan steps completed on the tested platform. Production exposes 17 MCP +tools, live DEV configuration-copy, shared task authority, existing LSP/build, +indexed offline language references and the supervised real Rust headless host. +There is no production fake, embedded agent/chat, generic MCP editor/shell or +provider-specific runtime behavior. + +### Requirement-to-evidence map + +| Requirement and steps | Evidence retained in code or acceptance record | +| --- | --- | +| Runtime contract, commands, limits, determinism; 01–07 | Runtime `automation_process.rs`, headless tests, published request/response schemas and packaged-installation smoke | +| Neutral architecture; 08, 14, 15, 27 | Studio specs 9–11; architecture/Modulith tests, 17 passing cases in final full rerun | +| Grant/revoke, symmetric locks, dirty-buffer handoff; 09–14 | `DefaultAutomationSessionTest`, Core execution/session tests, `DevDesktopTest`, `DevWorkspaceProjectionTest` | +| HTTP authentication/Origin/restart and real config; 16, 21 | `LocalMcpServerTest`, `McpContractTest`, `McpConnectionGuideTest`; packaged restart rejects prior credential | +| Manifest context, unknown languages/versions, indexed offline docs; 17–18 | `CompilerProjectKnowledgeTest`, `CompilerProjectLanguageCatalogTest`, `InstalledKnowledgeTest` | +| Saved-source LSP and dependency navigation; 19 | `AutomationLspQueriesTest`, including `externalDependencyNavigationIsReadableButRefactoringIsReadOnly`; SDK limitation remains explicit below | +| Task operations, lost receipts, deduplication; 20–21 | `McpTaskToolsTest`, Core operation tests; reconnect/duplicate advance in real desktop HTTP gate | +| Real supervision/build/corrective iterations; 22–23 | `HeadlessProcessSessionTest`, `HeadlessInstalledRuntimeTest`, `HeadlessTaskIntegrationTest` | +| Runtime/MCP/DEV pixel and identity parity; 24 | `HeadlessMcpDesktopTest` + independent `DirectHeadlessOracle`; compare every ImageView pixel, PNG bytes, metadata, reset/fresh-session/rebuild | +| Interruption/cleanup/Debug regression; 25 | Nine real HTTP/runtime/DEV tests, including crash, forced shutdown, app close, stale-result rejection and real manual Debug; controlled document participant makes refresh barrier observable | +| Real coding client and human confirmation; 26 | Codex CLI 0.158.0 run described below; developer confirmed DEV and manual Debug Play | +| Full regression and distribution; 27 | Commands/counts below; installed references and two extracted-ZIP application lifecycle cycles | +| Final propagation and handoff; 28 | This lesson, updated spec 11 delivery boundary, README troubleshooting, CLI validation and housekeeping | + +### Real external-client example + +The developer granted control in `../ai-first-smoke` and copied the descriptor. +Codex CLI 0.158.0 used a fresh ephemeral session, only the configured Studio MCP, +project-root workspace-write isolation and its approval-review mechanism. Default +noninteractive approval policy initially blocked acquisition before any edit; the +retry retained sandboxing and did not bypass the human grant. The configured default +model was used; this record does not assert an independently verified model identity. +Credentials stayed out of versioned files and reports. + +The agent consulted language/SDK references, edited only `src/main.pbs`, queried +LSP, built/packed and consumed native MCP image blocks. It changed the bundled +16x16 orange rectangle to one-pixel/frame directional movement in a 480x270 frame, +with no idle movement, opposing directions canceled before clamping, and unchanged +colors. Both builds succeeded; diagnostics were empty; final guest fault was null. +The 77 MCP calls included rejected/recovered calls and 13 native image blocks. + +| Scenario | Logical frame / FrameId | Client-reported observed position | +| --- | --- | --- | +| Idle for 10 frames | 10 / 9 | (0,32) | +| Right 20; down 20 | 30 / 29; 50 / 49 | (20,32); (20,52) | +| Left 10; up 10 | 60 / 59; 70 / 69 | (10,52); (10,42) | +| Horizontal then vertical opposites, 10 each | 80 / 79; 90 / 89 | (10,42), unchanged | +| Right + down, 500 | 590 / 589 | (464,254), intact at bottom-right | +| All directions, 10 | 600 / 599 | (464,254), unchanged at edge | +| Left + up, 500; idle 10 | 1100 / 1099; 1110 / 1109 | (0,0), intact at top-left | + +Build ID: `da441f65-479b-4099-8b86-fe48c7750e76`; runtime session: +`aa089209ec22c9a07286e5b01e967a57`. Frame identities/statuses were checked in tool +evidence; positions are the coding client's visual observations, not an invented +independent pixel measurement. Step 24 separately supplies exhaustive pixel parity. +Release returned MANUAL, no headless host remained, and game edits were preserved. +The developer confirmed the DEV result and successfully played the game through +normal Debug afterward. No game commit was made automatically. + +### Builds, commands and scope + +Studio executable code was `e6a83c94`; acceptance/troubleshooting documentation +was committed as `4c3597a2`. Runtime is merged at `01fab56`, distribution 0.1.0, +protocol 1.0. Native acceptance was **macOS ARM64 only**; no other coding client, +operating system or native-installer matrix is claimed. + +Studio final commands: + +```sh +./gradlew test build :prometeu-app:installDist --max-workers=2 --console=plain +./gradlew test :prometeu-app:installedKnowledgeTest :prometeu-app:headlessProcessTest --rerun-tasks -PstudioRuntimeInstall=/absolute/path/to/runtime --max-workers=2 --console=plain +``` + +The forced rerun executed all 91 tasks: normal suite **803 passed, 15 pre-existing +disabled tests, zero failures/errors**; installed knowledge **1 passed**; real +headless integration **4 passed**. The disabled compiler classes were +`IRVMIntrinsicRegistryParityTest`, `BackendGateIIntegrationTest` and +`MainProjectPipelineIntegrationTest`; none was disabled for this delivery. +The earlier step 25 targeted run had **254 passing tests**, including 9 real +HTTP/runtime/DEV and 4 DEV/Debug desktop tests. These overlap the full suite and +must not be added together as unique tests. + +The Studio ZIP was extracted outside the checkout. A test-only JavaFX probe loaded +only its distribution JARs, delegated to the real App init/start/stop, used isolated +preferences, and passed two shown-launcher/closed-listener cycles, credential +renewal and old-token rejection. It did not operate the developer's open window. +ZIP SHA-256: `256f80b25416c284dca58e5519fdde0be36e9a6a2a450cb58f9c453862188c8d`. + +Runtime final commands (cwd `../runtime`, separate temporary Cargo target): + +```sh +cargo test --workspace --offline --quiet +PROMETEU_PROTOCOL_TEST_INSTALL=/absolute/path/to/runtime cargo test -p prometeu-cli --offline --test automation_process -- --include-ignored +cargo test -p prometeu-host-desktop-winit --offline --lib -- --ignored +``` + +Workspace **678 passed**, with six opt-in tests ignored there; separate runs passed +all **13** process/protocol cases including the installed package, and all **5** +localhost Debug cases. Twelve process cases overlap the workspace result. The +runtime package was built during preparatory delivery; the final gate reused and +validated it rather than claiming a fresh distribution build. Installed headless +SHA-256: `24b74712397545ddd7f745daded859c12d19102f81c5b51bf46c202fb8d6b65d`. + +## Deferred Findings — Return Here After Implementation + +The user requested deferred investigation, not silent removal of these findings. + +| Finding | Owner and next investigation | +| --- | --- | +| `Input.pad()` SDK definition returns no location | Investigated on 2026-09-29: editorial lookup succeeds, but the physical-file policy filters the virtual SDK source. A separate chained-call editorial limitation was reproduced; see below. No runtime defect or fix is claimed. | +| Oversized reference page produces generic `-32602` | Studio MCP: improve parameter-limit discoverability/error specificity; default pagination already works. | +| Concurrent capture/fault/state may return BUSY | Studio MCP/Core: review concurrency guidance and clarity. Sequential calls work; do not weaken accepted exclusivity implicitly. | +| Framebuffer dimensions were discovered from capture | Language/runtime documentation: improve discovery; this does not prove dimensions are absent from every reference. | +| JDK 25/Kotlin target-24 fallback, JavaFX module/native-access/Unsafe warnings | Build/desktop tooling: review supported toolchain alignment and startup options separately. Passing tests do not erase warnings. | +| UI automation helper required newer Node than 22.13.1 | Environment tooling, not Studio: upgrade only as separate authorized work; human DEV confirmation supplied acceptance. | +| macOS ARM64 + Codex is the validated matrix | Future interoperability/platform testing; no uniform Grok/Windows/Linux promise. | + +`NO_FRAME_AVAILABLE` before the first completed frame and client permission prompts +are expected boundaries, not guest defects. The shutdown ownership/reclaim bugs +found during step 25 are fixed and tested, not outstanding issues. + +### Follow-up Investigation: SDK Navigation and Chained Calls + +Owner: `compiler/pbs`, with propagation to compiler tooling contracts and Studio/LSP +consumers. This diagnosis does not reopen the completed AI-first delivery or change +its accepted requirements. + +On 2026-09-29, a temporary Java probe exercised `CompilerLanguageServiceBridge` +with a request-local PBS overlay on `test-projects/main/src/main.pbs`, using the +application runtime classpath. It compared raw PBS editorial locations, exported +frontend locations, final LSP definitions, and hover. No production source was +changed. The comparison included imports from `@sdk:input` and `@sdk:gfx`, a local +`helper()`, an explicitly typed `pad: InputPad`, and `button: InputButton`. + +| Cursor target | Raw PBS locations | Frontend / LSP locations | Hover | +| --- | --- | --- | --- | +| Local `helper()` | 1 | 1 / 1 | Resolved | +| `pad` in `Input.pad()` | 1 | 0 / 0 | `pad() -> InputPad` | +| `left` in `pad.left()` | 1 | 0 / 0 | `left() -> InputButton` | +| `down` in `button.down()` | 1 | 0 / 0 | `down() -> bool` | +| `down` in `pad.left().down()` | 0 | 0 / 0 | No symbol information | +| `left` and `down` in `Input.pad().left().down()` | 0 each | 0 / 0 each | No symbol information | +| `clear` in `Gfx.clear(0)` | 1 | 0 / 0 | Resolved | + +There are two separate causes: + +1. **SDK destination policy, not failed lookup.** `InterfaceModuleLoader` registers + SDK declarations under `/virtual/stdlib/sdk/input/main.pbs` (and the corresponding + `gfx` path). The source handle can read the declaration text, but the path is not + a physical file. `PBSFrontendLanguageService.toFrontendLocation` explicitly + rejects it via `Files.isRegularFile`. The LSP bridge also reads destination text + from disk; removing the frontend check alone would not deliver a usable target. + This behavior follows Compiler Spec 23 §8.3 and the physical-file rule preserved + in `LSN-0068`, and already has an explicit `Gfx.clear()` regression assertion in + `CompilerLanguageServiceBridgeTest.definitionUsesEditorialLookupAndPhysicalStdlibFiles`. +2. **Chained-call editorial receiver resolution.** + `PbsEditorialSupportService.resolveReceiverTypeBeforeDot` uses + `resolveChainSegmentsEndingAt`, which walks identifier/dot chains, not a call + expression ending in `)`. Thus the receiver of `down` in `pad.left().down()` is + unknown before destination mapping even begins. An explicitly typed intermediate + `button` restores raw lookup and hover, while the virtual-source policy still + prevents navigation. This is an editorial limitation, not evidence that executing + the same expression fails. + +Recommended follow-up, not an accepted implementation decision: address chained +receiver resolution in the PBS editorial layer, and separately decide how to expose +the exact SDK sources as navigable read-only files. A version-scoped physical source +cache is compatible with the existing destination policy; virtual-document support +would require an explicit contract revision. Neither approach should duplicate SDK +semantics in MCP, permit SDK edits, or point to an unrelated developer checkout. + +## Pitfalls and Takeaways + +- A successful HTTP fixture is not an external-agent acceptance test. Prove both. +- Tab disposal is not task disposal; application teardown must own hidden sessions. +- Never unlock before process termination and document refresh acknowledge completion. +- Do not replay an advance because a connection or receipt was lost. +- A matching hash/blank canvas is weak visual evidence. Compare actual pixels, + metadata and an input timeline that changes a known object. +- An empty SDK definition is a language limitation, not permission to edit SDK code. +- Packaging language knowledge is part of making an unfamiliar language usable by + agents; adding another hidden language configuration is not. +- Keep provider permissions separate from Studio authority. Neither bypasses the other. +- Preserve explicit platform/client/test exclusions and deferred findings at closure. + +## References + +- Historical accepted decision DEC-0060 and completed plan PLN-0139, DSC-0068. +- [Studio architecture](../../../docs/specs/studio/9.%20Studio%20Core%20and%20Adapter%20Architecture%20Specification.md). +- [DEV and authority contract](../../../docs/specs/studio/10.%20DEV%20and%20Automation%20Specification.md). +- [MCP tools and knowledge contract](../../../docs/specs/studio/11.%20MCP%20Tools%20and%20Project%20Knowledge%20Contract.md). +- [Usage and troubleshooting](../../../README.md#external-coding-agents-mcp). +- Runtime-owned contract: `../runtime/docs/specs/runtime/11a-headless-automation-protocol.md`. +- Previous foundation: [Services must outlive UI](../DSC-0067-studio-core-javafx-decoupling/LSN-0075-studio-services-must-outlive-the-current-ui.md). diff --git a/docs/specs/studio/11. MCP Tools and Project Knowledge Contract.md b/docs/specs/studio/11. MCP Tools and Project Knowledge Contract.md index 02e8d663..c9adcc31 100644 --- a/docs/specs/studio/11. MCP Tools and Project Knowledge Contract.md +++ b/docs/specs/studio/11. MCP Tools and Project Knowledge Contract.md @@ -479,6 +479,17 @@ The normal desktop Debug process starts after release and blocks a new grant. The suspension/manual-Debug scenarios require a POSIX desktop and a free Debug port 7777; the gate never attaches to an existing user's Debug process. -The automated client is not an external coding agent. Real-agent compatibility -and final packaging acceptance remain later gates. API/Core and JavaFX MUST NOT -import MCP/HTTP implementations; architecture tests enforce it. +The automated client is not an external coding agent. Separate acceptance on +2026-09-29 used Codex CLI 0.158.0 against the packaged Studio and runtime 0.1.0 +on macOS ARM64. The agent consulted references/LSP, edited saved PBS source, +built/packed, consumed native MCP PNGs, tested 1,110 frames of directional input, +and explicitly released authority. The developer confirmed DEV and then ran the +edited game through manual Debug Play. This establishes compatibility for that +client/platform, not uniform compatibility across providers or operating systems. + +Full regression and distribution gates also passed: installed offline references, +real runtime protocol tests, architecture/Modulith, and two isolated startup/stop +cycles using the extracted Studio ZIP. The latter verified listener closure and +old-credential rejection after restart. API/Core and JavaFX MUST NOT import +MCP/HTTP implementations; architecture tests enforce it. Consolidated acceptance +evidence and deferred findings live in the DSC-0068 lesson, not in temporary logs.