Files
prometeu-runtime/docs/specs/runtime/11a-headless-automation-protocol.md
bquarkz c3589be1ad
Intrepid/Prometeu/Runtime/pipeline/head There was a failure building this commit
fix(headless): wait for memcard bootstrap IO progress
Coordinate bounded native publication progress during frame advance while preserving watchdogs, guest budgets and resource ownership. Add controlled and packaged reopen regressions, clarify the runtime specs, and retain qualification evidence.
2026-11-10 23:26:40 +00:00

24 KiB

Headless Automation Protocol v1

Domain: runtime host tooling. Function: normative.

This chapter defines the v1 automation wire contract owned by the headless host. It does not change guest ABI, firmware, VM, frame/input semantics or the desktop DevTools protocol.

The observation model is explained in Deterministic Observation Must Follow the Active Producer.

Process and transport

prometeu headless --stdio starts the distributed prometeu-headless tool from the same installation. One process owns one HeadlessHost. No display, native audio, network service or realtime pacing is required. A client must use its configured installation; missing binaries/capabilities are explicit failures.

Transport is UTF-8 JSON objects separated by LF (CRLF is accepted). stdout is protocol-only; diagnostics go to stderr. Fragmented reads must be accumulated; allocation is bounded by the command limit. An oversized line is rejected and drained through LF before reading the next request. EOF with an incomplete line reports invalid_request, then closes the session. EOF always requests shutdown.

All sizes exclude the LF delimiter and include the complete JSON envelope and base64 payload. Responses exceeding their bound become response_too_large, never truncated data. Clients must continuously consume stdout, including events. Backpressure may block output; the supervising client owns wall-clock deadlines and may terminate the process tree. There is no transport replay or durable queue.

Handshake and compatibility

The server first emits a hello object with sessionId, protocolVersion (major: 1, minor: 2), real runtimeVersion, capabilities and limits. Session identity is opaque and process-local, excluded from deterministic oracles.

Before work, the client sends initialize with protocolVersion and an array requiredCapabilities. An incompatible major returns incompatible_version; missing capabilities return unsupported_capability. The session remains uninitialized and cannot execute work. Compatible minor additions are permitted; the initialize result returns the lesser supported minor. Initialize is accepted once; repeated initialization returns already_initialized. Shutdown is permitted even before initialization. All other work requires successful initialization.

Envelopes

{"requestId":"a1","command":"advance","params":{"frames":60}}

requestId is a nonempty string of at most 128 UTF-8 bytes, selected by the client; concurrent requests must have distinct identities. command and params are required. Unknown fields and invalid parameter types are rejected. The request schema is distributed in devtools/headless-protocol/request.schema.json; response.schema.json defines hello, progress and every response/result shape. Valid and invalid request fixtures are shipped beside the schemas. Process tests validate actual server output against the response schema (including image results).

{"type":"response","sessionId":"opaque","requestId":"a1","result":{"requested":60,"completed":60,"stop":"completed"}}
{"type":"response","sessionId":"opaque","requestId":"a1","error":{"code":"busy","message":"...","details":{}}}

A response contains exactly one of result/error. Errors have stable code, human-readable message and structured details. Unparseable/unusable request identity is represented by null; valid identity is preserved on parameter errors. Malformed JSON, invalid schema and unknown commands must never disappear silently. Wire requestId is correlation, not deduplication: the client must not retry an advance blindly after a lost response. Studio task-level clientRequestId and operationId are a separate boundary, not part of this wire protocol.

Commands

Command Parameters Result/meaning
initialize protocolVersion, requiredCapabilities negotiated version and capabilities
boot path (nonempty directory-cartridge path) current state; existing boot preconditions apply
reset empty object reset state, clears input/fault/frame correlation
set_input complete snapshot fields listed below; omitted fields reset to defaults input replaced, not patched
clear_input empty object input restored to defaults
advance frames: unsigned integer, including zero requested, completed, stop, optional fault/error details
capture empty object PNG/base64 plus canonical frame metadata
get_state empty object pc, stackTop, logicalFrameIndex, tickIndex, appId, status
get_fault empty object fault object or null
get_logs required afterSequence: null/uint64, limit: 1..1024 native log page; requires negotiated 1.2 log_read
cancel targetRequestId acknowledgement that cancellation was requested for that active advance
shutdown empty object acknowledgement after the active operation stops and host is dropped

Input snapshot fields are up, down, left, right, a, b, x, y, l, r, start, select, touch (booleans), and touchX, touchY (signed i32). Omitted booleans are false and omitted coordinates are zero. The ordinary runtime begin-frame sampling derives press/release/hold edges. No new input semantics exist.

State statuses are running, paused, terminated, faulted; running does not mean background execution. Stack values preserve their type through tagged kind/value objects; i64 values are decimal strings and float values carry an exact round-trippable string (including non-finite values). Faults expose kind, pc, summary and structural trap code/opcode/message/source span or init-error code. No client must parse a summary to distinguish trap, panic or initialization failure.

Resource traps include a nullable trap.resource object with resource, reason, requested and limit; resource and reason are enumerated machine values. initError is discriminated by code: registry_load_failed includes load, runtime_resource_limit includes resource, entrypoint_arguments_invalid includes entry, and loader_patch_failed includes binding. Other initialization codes carry only their code and human message. load.code uses the stable LOAD_* tag and retains typed reason, section/offset, record/member, ID domain, referenced ID, function/PC and quota context, with null for unavailable fields. Binding context preserves canonical native identity, version, capabilities or declared/expected ABI counts as applicable. A load-relative PC remains nested inside its diagnostic; initialization failure does not invent a runtime PC. The published response schema rejects unknown fields and invalid contexts.

Ordering, cancellation and lifecycle

Only one host operation is admitted at a time. Conflicting work, including state inspection while the host is executing, returns busy; it is not implicitly queued. The command reader remains available for cancel/shutdown while an advance executes. Cancellation targets an active advance's requestId; no active matching advance returns not_cancelable. An acknowledgement is not a completed cancellation.

Advancement is repeated advance_frame(), checking cancellation between frames. The current frame keeps the existing deterministic host-tick watchdog. Zero frames is a no-op. Results preserve requested/completed counts and stop values: completed, faulted, terminated, canceled, host_error. Guest faults are outcomes, not malformed-request errors. Host errors include a stable code and known completed progress; they do not manufacture guest faults.

Each completed frame emits type: progress, requestId, sessionId, requested and completed counts before the terminal response. A client can retain known progress if the process dies. The final response is authoritative when delivered.

Shutdown/EOF requests cancellation, waits for the active work to reach its boundary, drops the host and exits; explicit shutdown receives its response after cleanup. Negotiated memcard shutdown may refuse with native state and keep the channel available, as specified in Memcard controls. Disconnect never starts/resumes execution. A failed process requires a new explicit session, not replay. Host session loss does not implicitly release Studio authority.

Capture

Capture returns mimeType: image/png, encoding: base64, data, width, height, stridePixels, pixelFormat: RGBA8888, logicalFrameIndex and frameId. The envelope carries sessionId; the Studio consumer adds build correlation. Frame identities/counters on this wire are unsigned JSON integers and must be decoded losslessly. logicalFrameIndex and frameId have no numeric equality rule.

PNG stores visible RGBA channels in row order, losslessly, without stride padding. stridePixels describes the original canonical buffer. Before a completed frame, including after reset, capture returns no_frame_available; it cannot reuse an old run. No desktop screenshot, hash-only substitute, temporary-image-file transport or continuous video is permitted.

Configurable limits and supervision

CLI option / handshake field Default
--max-frames / maxFrames 600
--max-command-bytes / maxCommandBytes 65536
--max-response-bytes / maxResponseBytes 33554432
--handshake-timeout-ms / handshakeTimeoutMs 5000
--call-timeout-ms / callTimeoutMs 60000
--shutdown-timeout-ms / shutdownTimeoutMs 5000

Limits must be positive; message bounds must be at least 4096 bytes to admit the handshake and bounded errors. Limits apply to serialized bytes, not characters. Clients must enforce the advertised handshake/call deadlines and 5-second default cancel/shutdown grace, then terminate only the process tree they own. Deadlines are host supervision, not simulated guest time. Runtime enforces frame/message bounds without clamping. Requests over limits fail explicitly. Clients configure startup timeouts before seeing the handshake, then verify the announced values.

Conformance

The runtime gate includes fragmented/invalid/oversized lines, negotiation failures, all commands, busy/cancel/shutdown races, EOF, real CLI distribution invocation, PNG pixel equality, independent logical/render identity, repeat/reset determinism, structured faults, progress on partial failure and retained desktop run/debug. Neither Studio UI nor an AI model is required. See tests/automation_process.rs in prometeu-cli and the in-process deterministic-control tests in the headless crate.

Native log observation

Protocol minor 1.2 adds capability log_read. Clients MUST negotiate minor >= 2 and include log_read in requiredCapabilities before using get_logs. Requiring it with an older minor returns incompatible_version; using it without negotiation returns unsupported_capability. Production clients negotiating 1.0 or 1.1 retain their existing command/result shapes and media behavior.

{"requestId":"init","command":"initialize","params":{"protocolVersion":{"major":1,"minor":2},"requiredCapabilities":["log_read"]}}
{"requestId":"logs","command":"get_logs","params":{"afterSequence":null,"limit":128}}

Send each request after the preceding response. afterSequence is a required nullable exclusive uint64 cursor. Null includes the first retained event, including sequence zero. limit is required and MUST be an integer in 1..1024. Unknown fields, missing parameters, strings/fractional/negative cursors and invalid limits return invalid_params. A cursor greater than newest, or any non-null cursor on an empty stream, returns invalid_cursor.

The result contains sessionId, logEpoch, events, oldestSequence, newestSequence, nextAfterSequence, gap and overflow. Events include complete sequence, frame, timestampMs, level, source, appId, tag and message. Levels are trace, debug, info, warn, error; sources are pos, hub, vm, fs, app. appId is uint32 for app and null for all other sources. tag is uint16. Sequences, frames, timestamps and epochs are unsigned JSON uint64 integers; clients MUST decode them losslessly. Native elapsed-time timestamps MUST NOT determine deterministic verdicts. Messages are native stored strings, without assertion parsing, filtering, formatting or additional truncation.

Events are returned in increasing native sequence order across all sources. Retained oldest/newest bounds are both null for an empty ring. The next cursor is the final returned sequence, or the input cursor for an empty page. Requerying an unchanged ring returns the same events; it never drains records. Reads MUST NOT tick the firmware, execute guest code or change lifecycle, input, media or faults. Existing single-operation admission applies: reading during an advance returns busy. Complete envelope byte bounds apply: an oversized page or single event returns response_too_large, without a partial result. Retry a smaller page if appropriate; a single event that cannot fit remains an explicit error.

Retention, cursors and reset

The ring retains at most 4096 native records. overflow reports whether retention has ever evicted records in this log stream. It is historical information, not proof of loss for this reader. gap reports missing history relative to the requested cursor: null with retained oldest > 0, or a non-null cursor whose next sequence is less than retained oldest. overflow: true, gap: false is a valid page for a reader that consumed the evicted records already. Readers MUST use cursor continuity and gap to detect unread retention loss, rather than reject historical overflow alone. For example, oldest=100 and afterSequence=150 has no gap; afterSequence=98 or null has a gap. No gap is hidden by source/tag filtering.

logEpoch is zero for the existing process-owned log stream. VM reset retains the ring, native sequences and epoch; frame counters may reset independently. It MUST NOT clear or renumber logs to simulate a new stream. A different process has a different sessionId and its own sequence zero. Clients MUST bind cursors and deduplication to sessionId/logEpoch; a numeric cursor alone is not a portable identity between processes. A native sequence already applied MUST NOT be applied again when inspecting overlapping pages.

Producer limits and consumer responsibility

The unchanged guest logging path accepts at most 10 messages per application per logical frame, with at most 256 UTF-8 bytes per message. Longer guest messages are truncated before insertion; excess calls are dropped and produce one native warning for that application/frame. Dropped calls receive no native sequence, and truncation does not create a native gap. Consequently gap: false does not establish that every attempted guest event was stored. Reading cannot recover those losses.

Structured-event consumers MUST bound their encoded messages and per-frame emission, accounting for other logs from the same application. They MUST validate their own producer event continuity and explicit completion (for example, ordinals and final event counts), rejecting missing, malformed or incomplete reports. The provider supplies generic native observations and does not interpret suites, assertions or success. Do not enlarge guest budgets to transport test reports.

Consumers advancing the guest MUST read all pending pages before the next frame block, including after boot and completion/failure when available. Choose conservative blocks within actual retention headroom; maxFrames is an admission ceiling, not a safe polling interval. At 10 app messages/frame, 600 frames can already exceed the 4096-record ring, independently of system messages. Bounded polling does not guarantee lossless delivery for arbitrary production rates or unbounded work. Response loss MUST NOT trigger automatic advance replay.

Memcard controls

Protocol minor 1.1 adds capability memcard. Initialize negotiates the lesser minor. These commands and parameters require minor >= 1 and memcard in requiredCapabilities; unsupported use fails before mutation. Clients negotiating 1.0 keep the previous command/result shapes. No executable-version inference, default image, alternate transport or headless media startup flags are defined.

Command Parameters Meaning
memcard_mount path, action: open/create, access: read_write/read_only prepare explicit media before boot; bounded result
memcard_unmount empty object release idle media outside a resident session
get_memcard_state empty object coherent native observation, without guest execution
boot path, optional memcardChoice: persistent/memory_fresh/memory_existing explicit player choice required for a memcard cartridge
memcard_reconcile empty object recover selected same-card identity while suspended
memcard_continue_in_memory empty object explicit safe SO detach, preserving RAM
reset / shutdown optional discardUnsaved (false), pendingWrites: wait/cancel (wait) controlled teardown; no autosave

Mount requires no resident game or outstanding IO. Create/read_only is invalid. Paths are resolved relative to process cwd, reported absolutely; parent directories must exist. Open does not create; create never replaces, converts or repairs media. Persistent requires writable coherent media; memory_existing requires a readable record for the cartridge appId. No error changes that choice automatically. After an admitted boot attempt, retry requires explicit reset; parsing and negotiation failures do not admit boot. Reset preserves selected media/lock.

State has closed objects media, session, io, recovery, a nullable request, dirty and dirtySlots. Media reports path/cardId/access/preparation/error/health; session reports identity/choice/mode/boot phase/residency/action requirement. Requests report original identity, operation, local slot, state, deadline reason, storage error and retention, independently of IO activity. cardId is 32 lowercase hex digits; request IDs and mount epochs are decimal strings. Unknown values are null, never invented outcomes. Schemas distributed with the installation specify all fields. Inspection never releases requests or exposes private save payloads.

Domain failures use memcard_error with details.operation, details.reason and details.state. Reasons include choice_required, unavailable, corrupt, unsupported, read_only, access_denied, already_exists, busy, no_space, out_of_memory, invalid_state, invalid_range, counter_exhausted, identity_mismatch, unsaved_changes, indeterminate and restart_required. Human messages are not codes. Original worker error, deadline cause and durable result are separate observations.

Shutdown stops an active advance at its existing frame boundary, then evaluates IO and dirty RAM. Wait/cancel only concern already admitted work. After IO ceases, remaining dirty RAM without discard consent returns unsaved_changes and leaves the session/channel usable. Terminal results retained by the guest alone do not block closure. Unconfirmed IO cessation never returns closed:true or permits resource/lock release. Successful negotiated reset adds memcard to the state result; shutdown returns closed:true and its final memcard snapshot, after checked cleanup. EOF/output loss attempts bounded cancellation without save/resumption; process death does not establish publication outcome.

An advance stopped for resident media action returns stop:action_required with known completed progress. Resident state is paused; a host dialog is not counted as a game frame. Idle IO polling leaves guest counters/input unchanged. Reconcile accepts no replacement path, validates expected identity before changes, preserves RAM/continuation/original retained results and does not rerun initialization.

When successful bootstrap remains blocked by its pending native publication, the owner coordinates lane progress within the same advance, outside machine ticks, with requested waits of at most 5 ms. This does not apply to an unrelated gameplay commit. Native deadlines remain 500 ms for cancellation and 1 s for stop/escalation, measured from the original request admission. One additional 2 s deadline from first wait entry in an advance_frame does not reset on wake/reentry or extend those deadlines. Existing native action-required/fault boundaries take precedence. If the dependency remains pending at the outer deadline without such a boundary, the advance returns stop:host_error, exact known completed progress, and error.code:memcard_error with details.operation:advance, details.reason:restart_required and details.state holding the native snapshot. The guest finishes paused with the host step request cleared. This preserves worker ownership until actual cessation, the 100,000-tick frame watchdog, transport responsiveness and cancellation between frames; it introduces no intraframe cancellation, guest fault, simulated progress, command replay or new protocol shape.

Runtime qualification uses neutral PBX/ABI fixtures, real distinct processes and an isolated extracted distribution. It proves SUCCESS, termination, exact readback without resave and stable appId across cartridge versions. Consumer source-language compilation and client adaptation are separate work; the provider has no frontend build dependency. No global installation or consumer configuration update is needed.

Integration sequence

Start prometeu headless --stdio from the selected installation and consume its hello before sending work. Require the capability explicitly:

{"requestId":"init","command":"initialize","params":{"protocolVersion":{"major":1,"minor":1},"requiredCapabilities":["memcard"]}}
{"requestId":"media","command":"memcard_mount","params":{"path":"/absolute/save.pmc","action":"open","access":"read_write"}}
{"requestId":"boot","command":"boot","params":{"path":"/absolute/cartridge","memcardChoice":"persistent"}}
{"requestId":"step","command":"advance","params":{"frames":1}}
{"requestId":"save-state","command":"get_memcard_state","params":{}}
{"requestId":"close","command":"shutdown","params":{"pendingWrites":"wait"}}

Each request above follows the successful response to the preceding one. Creating new media requires a deliberate action:create; do not retry an open failure with create. Memory choices are memory_fresh and memory_existing. No client may replace a failed persistent choice with either automatically.

If closure reports unsaved_changes, keep the process/session and expose that native state to the caller. A later explicit choice may send {"discardUnsaved":true,"pendingWrites":"cancel"}; do not infer that consent from a UI stop, timeout or process error. A successful commit is the guest/native request state success, independently of its reason; neither an advance response nor closed:true proves that a save occurred.

On reopening, launch a different process, open the same image and choose the appropriate mode explicitly. The neutral process oracle verifies exact prior bytes without resaving and repeats with a changed cartridge version and unchanged appId. Run cargo test -p prometeu-cli --test automation_process memcard_public_distinct_process_save_readback_and_version_identity; add --release to qualify that profile independently. For an extracted archive, set PROMETEU_PROTOCOL_TEST_INSTALL and run the ignored packaged_memcard_process_persistence test. It invokes the installed dispatcher from another cwd without modifying any consumer project or installation.

Program Registry integration

When exposing initialization failures, retain the structured load error code, typed reason and available context. Human-readable messages are not a parsing contract; a load error has no fabricated runtime PC. Failed loads must not publish executable sessions. Existing protocol request names and transport behavior are unchanged.

Heap reference inspection values use {"kind":"heap_ref","value":{"index":u32,"generation":u32}}. Both components are required to distinguish a live incarnation from a reused slot; the wire representation must not truncate handles to indices. Inspection does not grant cross-session transfer authority.