Files
prometeu-runtime/docs/specs/runtime/08-save-memory-and-memcard.md
T
bquarkz c3589be1ad
Intrepid/Prometeu/Runtime/pipeline/head There was a failure building this commit
fix(headless): wait for memcard bootstrap IO progress
Coordinate bounded native publication progress during frame advance while preserving watchdogs, guest budgets and resource ownership. Add controlled and packaged reopen regressions, clarify the runtime specs, and retain qualification evidence.
2026-11-10 23:26:40 +00:00

401 lines
23 KiB
Markdown

# Save Memory and MEMCARD
Domain: virtual hardware: save memory
Function: normative
The runtime implements image transactions, per-game RAM snapshots, a shared
asset/IO worker and an explicit guest bootstrap gate. Host latency and durability
qualification apply to the concrete host and storage device.
## 1. Ownership and capacity
A runtime has at most one installed card. A card contains 64 shared logical slots,
each holding 0..65,536 private bytes, and 68 physical blocks of 65,536 bytes.
There are four nominal rotating spares when all logical slots contain data; all
healthy unprotected blocks participate in the pool. Physical IDs are never game
slot IDs. Each of at most 64 game records owns at least one logical slot.
The cartridge declares a `memcard_slots` count in 1..64 together with the memcard
capability. Allocation and growth are all-or-nothing. A smaller declaration uses
the prefix of the prior allocation and retains excess slots/data. The authorized
`app_id` is the cartridge's uint32 identity. Guest indices are local 0..N-1.
Newly assigned slots expose no residual bytes. Game clear retains ownership;
external administration removes the entire game and all its metadata/slots.
There is no external single-slot deletion. Space management/card switching is
an explicit player action outside gameplay.
## 2. Image encoding
All integers are unsigned little-endian unless stated otherwise. There is no
filesystem inside the image. Every reserved byte is zero. Readers reject unknown
versions, malformed lengths, overlapping/out-of-range references and unknown
flags before following references. Creation allocates the entire 5,259,264-byte
image physically, including currently unused regions, and publishes the file's
namespace durably. Sparse length extension alone is insufficient.
| Region | Offset | Length |
| --- | ---: | ---: |
| Identity A | 0 | 4,096 |
| Identity B | 4,096 | 4,096 |
| Map A | 8,192 | 4,096 |
| Map B | 12,288 | 4,096 |
| Catalog A | 16,384 | 131,072 |
| Icons A | 147,456 | 262,144 |
| Catalog B | 409,600 | 131,072 |
| Icons B | 540,672 | 262,144 |
| Physical blocks 0..67 | 802,816 | 4,456,448 |
Each identity page contains: magic `PMCARD1\0` at0 (8 bytes), version1:u32
at8, image length:u32 at12, host-generated random cardId:[u8;16] at16,
zero padding32..4092 and CRC32C:u32 at4092. CRC covers bytes0..4092 (exclusive
of the CRC). Either valid identity is sufficient; two valid differing identities
are inconsistent. A degraded single identity must be reported.
CRC32C uses reflected Castagnoli polynomial0x82f63b78, initial0xffffffff,
final xor0xffffffff, reflected byte input/output. Empty input produces0;
ASCII `123456789` produces0xe3069283. Payload CRC covers exactly used bytes.
Catalog and icon CRCs cover exactly the declared used region. Administrative
page CRC covers all bytes preceding its last4-byte checksum, including padding.
Each map page contains:
| Offset | Field |
| --- | --- |
| 0 | magic `PMMAP1\0\0`, 8 bytes |
| 8 | publication sequence:u64, positive |
| 16 | public-copy selector:u32, 0 or1 |
| 20 | catalog used length:u32, <=131072 |
| 24 | catalog CRC32C:u32 |
| 28 | icons used length:u32, <=262144 |
| 32 | icons CRC32C:u32 |
| 36 | cardId:[u8;16], equal to identity |
| 52 | 12 reserved zero bytes |
| 64 | rotation order: permutation of physical IDs0..67, 68 bytes |
| 132 | quarantine flags: 68 bytes, each0 or1 |
| 200 | 56 reserved zero bytes |
| 256 | 64 slot descriptors, each40 bytes |
| 2816 | zero padding through4091 |
| 4092 | page CRC32C:u32 |
A slot descriptor has state:u8 at0 (0 unassigned,1 EMPTY,2 payload), physical
ID:u8 at1 (255 unless payload), zero:u16 at2, used bytes:u32 at4,
generation:u64 at8, update_at:u64 at16, payload CRC32C:u32 at24 and12 zero
bytes at28. Unassigned is canonical except physical ID255. EMPTY has no physical
block, zero used bytes/CRC, but can have a save generation/time. Payload requires
1..65536 used bytes, physical ID0..67 and positive generation. A physical block
has at most one owner per map and cannot also be quarantined. Assigned descriptor
IDs must equal the union of catalog assignments.
Generation0 means never saved, with canonical timestamp0; first persistent commit
is1, including empty commits. Later commits/clear increment it. With positive
generation, timestamp0 is a valid epoch date. Sequence and generation never wrap;
a required increment at uint64 maximum refuses publication. Equal sequence only
permits byte-identical map content; differing content is corruption.
## 3. Public catalog and icons
Each catalog is UTF-8 JSON, schema version1, maximum depth8, closed objects and no
duplicate keys. Integers are exact unsigned values in their declared range (no
floating-point conversion). Unknown fields, wrong types and invalid Unicode are
rejected. The root has exactly `metadataVersion` (1) and `games` (array, <=64).
Each game has exactly:
- `app_id`: uint32, unique and authorized by the runtime;
- `title`: nonblank UTF-8 string, <=128 decoded bytes, reject rather than truncate;
- `dataFormatVersion`: uint32, game-defined including0;
- `created_at`: uint64 UTC Unix seconds for first registration;
- `slots`: nonempty array of distinct logical IDs0..63, exclusive across games;
- `icon`: object with `offset`:uint32, `frames`:1..4, `duration`:uint16,
`crc32c`:uint32. Offset is relative to the selected public icon region.
An icon contains a48-byte palette in RGB order (16 opaque RGB888 colors), followed
by512 bytes per32x32 frame, row-major, first pixel in the high nibble. All16
indices are ordinary opaque colors. Its length is48+512*frames. Icon ranges must
not overlap and CRC covers palette and all frames. `duration` is1..65535 units of
1/30 second, shared by all animated frames. Static icons use canonical duration1.
A consumer cycles frames while visible using its SO/Hub clock; rendering or a
save-management application is not part of the game runtime.
The maximum icon payload is64*2096=134144 bytes, within262144. Even charging768
bytes per escaped128-byte title, maximal decimal integers, every logical-slot ID
and field punctuation, a canonical game record is bounded by2048 bytes; the
actual maximum catalog must fit131072 including its root. Producers use compact
serialization and validate final length; readers never allocate from an unchecked
length or require both public copies/icons in RAM.
Ordinary slot commits do not rewrite JSON. created_at survives updates, but a
removed/recreated record obtains a new date. Slot update_at/generation/CRC live
only in binary descriptors. The game's last-save date is derived as max update_at
over positive generations; no such slots means never saved. Public readers can
locate slot bytes and check CRC without interpreting the private payload.
## 4. Publication, retirement and recovery
Before overwriting any block/public region, protect the union of references from
both usable maps. Retire an older map by overwriting its full page with zero,
syncing and verifying it is no longer a usable map; only then may references held
exclusively by it be reused. The current coherent map remains untouched. A failed
retirement stops that operation; no referenced data is overwritten speculatively.
A nonempty commit freezes its RAM source, chooses a healthy unprotected block,
writes payload, syncs and verifies readback CRC. Then it writes the inactive map
with next sequence/descriptor, syncs and verifies that map before SUCCESS. Empty
commit publishes EMPTY without a data block. Former blocks join the end of the
rotation queue when eligible; they are not erased after commit.
Catalog changes prepare the inactive public copy, sync/readback-verify it, and
publish its reference together with allocation/descriptors in a new map. Retire
any old map referencing that destination before overwriting it. Registration,
growth, whole-game removal and presentation update are coherent administrative
publications. Payload is never saved automatically. dataFormatVersion changes do
not migrate data or promise atomic migration of multiple slots.
Mount validates candidate maps, catalog and all referenced payload CRCs; select
the newest coherent publication. A page checksum alone is insufficient. Report
recovery from an older map explicitly; never silently roll back confirmed saves.
No coherent map means corruption, not empty media. Persist quarantine exclusions;
known exclusions survive recovery of an older map. If exclusion cannot be made
durable safely, block writes until revalidation. Reintegration requires explicit
verification, never merely remount. Transient timeout does not prove block damage.
Mount state and health/write permission are separate: DEGRADED may write with a
safe candidate; READ_ONLY still permits valid reads.
Durable result codes are SUCCESS=1, FAILURE=2, INDETERMINATE=3. PENDING=0 and
MEMORY_ONLY=4 are request states/results without a durability claim. FAILURE
requires proof of no publication; write/sync errors may have partial effects.
The uncertain window begins when publication-capable metadata is attempted.
Retain attempt cardId/slot/target sequence/generation/CRC for same-card
reconciliation before more writes. Confirmed success survives cleanup failure.
## 5. RAM and guest operations
Before boot reserve N*65536 bytes of actual snapshot capacity, one 65536-byte
scratch and bounded metadata/backend overhead. Memory-only mode has the same
requirement. Insufficient RAM prevents admission with a diagnostic. Existing
saves are validated and loaded, never replaced with empty on integrity error.
Only the game's declared prefix is resident, not other games' payloads.
Read/write/clear are bounded RAM operations. Write zero-fills gaps and rejects
end offsets beyond65536. Clear makes used length0 and retains ownership; commit
persists that clear. During an in-flight commit the source slot rejects mutations
with BUSY, but remains readable; other slots/gameplay continue. No duplicate full
source snapshot is allowed. Protect its storage until the host ceases all use.
The mem module uses a session-bound Int64 context and local slot indices. Byte
transport is Array<Int32>, values 0..255, at most 4096 bytes per guest call. Validate
array type, element values, range/overflow, ownership and all output capacity
before copying or mutating. Charge array storage (four bytes per element) and
copy work. Host IO maximum remains 65536. A native array reference cannot escape
the bounded call into asynchronous work; async source is the native snapshot.
The [guest ABI](#guest-abi) fixes domain status and request result encodings.
Request identity is separate from status and durable result. All queries validate
session/mount/request identity. Only one request/result entry exists, retained
until explicit release; release never frees in-flight resources. No payload queue.
## 6. Bootstrap and producer contract
The Registry role 3 is MemcardBoot(context:Int64)->Int32, no captures, unique and
never FunctionId 0. Role number is not FunctionId. Capability requires that binding;
absence of capability skips snapshot and hook. Roles 0/1/2 are unchanged.
The producer emits an explicit memcard bootstrap boundary after its initialization
and before its first frame. Runtime recognizes a bootstrap phase, not FE concepts
such as Init/Frame. Snapshot/preload is ready before that boundary; the hook runs
once on the main lane, preserving continuation, with1,500,000 total cycles and
250,000-cycle slices outside frame accounting. READY=0 or INCONSISTENT=1 returns
with a bounded diagnostic (up to1024 UTF-8 bytes) set via the service. Invalid
return/trap/budget exhaustion blocks admission and reports through SO/Hub.
New records/slots are provisional while the hook provides presentation. A commit
may be submitted during the hook; it remains pending until READY and coherent
catalog publication. The hook must return rather than wait for that dependent
commit; normal frames may later query its result. This uses the one existing
request entry, not a second queue. INCONSISTENT invalidates provisional admission
and never publishes its payload. IO wait and guest cycle budgets are separate.
Retries/remount/resume never automatically rerun the hook or main initialization.
## 7. Executor, host and deadlines
Host implements complete-or-error read_at/write_at of up to 64 KiB, sync and
bounded cancellation/recovery or fault escalation. Partial transfers are normalized
inside host bounds; failure may have changed storage. Runtime owns all logical
mapping, checksums, publication and request policy. Deliver file backend and fault
backend; raw flash requires separate qualification.
Use one shared async_work lane with assets. Save has highest priority; an asset
finishes a bounded safe step, retains its progress and yields to the whole save,
then resumes from that point. Retained save result does not retain the worker.
No extra worker or waiting inside the only worker implements yielding.
Measure from admission with monotonic time: start within one nominal frame
(~16.7ms), healthy target 100ms, initiate cancellation at 500ms, stop or escalate
and notify by 1s. Deadline control runs outside potentially blocked IO. Coordinate
cancel with publication entry; no retries after deadline. Timeout is a reason,
not proof of rollback. If cessation is unconfirmed, isolate mount/resources,
block new work and retain source/scratch. Late completion belongs to original
reconciliation. Never kill threads unsafely, free buffers in use or create a
replacement worker. Report restart requirement when host recovery is impossible.
The first watchdog cancellation is recorded independently of the request result,
before signaling cancellation. Repeated polls preserve that cause. At 500ms the
request remains Pending with no terminal reason until the worker returns or the
1s escalation occurs. Removal or session termination that canceled first does
not become a watchdog timeout merely because a later poll crosses 500ms.
On completion before escalation, the worker's durable outcome determines
Success, Failure or Indeterminate. A recorded watchdog cancellation supplies
Timeout (guest reason 15) independently of that state; native diagnostics retain
the original worker storage error. For example:
- A save canceled while queued at 500ms, then completed at 501ms without entering
publication, reports Failure/15. The durable slot is unchanged.
- A save already inside publication at cancellation can finish successfully and
report Success/15. Its saved snapshot becomes clean; cancellation is not rollback.
- An uncertain publication reports Indeterminate/15 and requires reconciliation.
- At 1s without confirmed cessation, RestartRequired reports Failure/15 before
publication entry or Indeterminate/15 after entry. Late completion returns card
ownership for reconciliation but cannot rewrite this terminal guest result or
mark the snapshot clean. Releasing the result does not release in-use resources.
A deferred bootstrap save that has not reached the worker expires as Failure/15
at 500ms. Memory-only commits remain MemoryOnly with reason 0 and perform no IO.
The status/state/reason result slots and request identities do not change.
Fast-profile claims require measurements under asset load, including tail latency.
## 8. Player choice and lifecycle
The player chooses PERSISTENT or MEMORY_ONLY through SO/Hub. No required/optional
mode is imposed by the game. Memory-only commit performs no IO and makes no
persistence claim or hidden temporary save. With a readable existing save, offer
load-to-RAM or clean start explicitly. RAM survives pause/resume and removal;
warn before discarding unsaved changes when ending the session.
Removal uses existing OS pause and external choices to restore/switch card,
continue in RAM or exit. Same-card reinsertion reconciles with a new mount epoch
without overwriting RAM. Different card requires validation/allocation and explicit
keep-local/load-existing choice. No automatic upload, reload, save or reinit.
BeforePause can submit and return without waiting for disk; BeforeResume can query.
Session termination retains resources used by outstanding IO until safe cessation.
The legacy per-slot file format is unsupported by this image codec. Preserve
legacy files; do not migrate or delete them automatically. Internal fixtures are
regenerated; removal of the old runtime path follows validated integration.
## Guest ABI
All entries use module `mem`, version 1 and capability `memcard` (bit 9).
Context and request IDs are positive Int64 values. Other numeric domain results
use Int64; `dirty`, `new`, and health flags use Bool. Unsigned 64-bit generations
and timestamps use the Int64 bit pattern without changing their unsigned meaning.
Guest indices/ranges may use Int32 or Int64 and must be nonnegative.
Error results carry the leading status and Null for unused payload positions.
| ID | Operation | Results in declaration order |
| --- | --- | --- |
| 0x4200 | bootstrap() | hook result: Int32 (0 READY, 1 INCONSISTENT) |
| 0x4201 | slot_count(context) | status, count |
| 0x4202 | slot_stat(context, slot) | status, has_payload, RAM used_bytes, generation, update_at, CRC32C, dirty |
| 0x4203 | slot_read(context, slot, offset, array, array_offset, count) | status, bytes_read |
| 0x4204 | slot_write(context, slot, offset, array, array_offset, count) | status, bytes_written |
| 0x4205 | slot_commit(context, slot) | status, request |
| 0x4206 | slot_clear(context, slot) | status |
| 0x4207 | query(context, request) | status, state, reason |
| 0x4208 | release(context, request) | status |
| 0x4209 | info(context) | status, mode, new, recovered, degraded, write_blocked |
| 0x420A | metadata(context) | status, title, dataFormatVersion, created_at |
| 0x420B | presentation(context, title, dataFormatVersion, frames, duration, array, offset, count) | status |
| 0x420C | diagnostic(context, message) | status |
| 0x420D | publish_presentation(context) | status, request |
`slot_stat` reports the durable descriptor alongside RAM length/dirty state;
CRC32C does not pretend that uncommitted RAM is already durable. `presentation`
sets the game-owned public fields; initial boot publishes them after READY.
Subsequent explicit `publish_presentation` uses the same one-request slot.
Icon bytes include the 48-byte RGB888 palette followed by packed frames.
Diagnostics are UTF-8 strings bounded to 1024 bytes.
Status values: 0 OK, 3 NO_SPACE, 4 ACCESS_DENIED, 5 CORRUPT,
6 INDETERMINATE, 7 UNAVAILABLE, 8 INVALID_STATE/CANCELED, 9 BUSY,
10 READ_ONLY/COUNTER_EXHAUSTED, 11 INVALID_RANGE, 12 UNSUPPORTED,
13 IO_ERROR, 16 OUT_OF_MEMORY. Query states: 0 PENDING, 1 SUCCESS,
2 FAILURE, 3 INDETERMINATE, 4 MEMORY_ONLY. Query reason uses the status
catalog, plus 15 for deadline/restart escalation. Mode 0 is PERSISTENT and
1 is MEMORY_ONLY. Type/heap/range violations at the array carrier are VM traps;
ordinary storage failures are domain results.
## Host integration
### Public host controls
The installed headless interface is the negotiated memcard surface of
[Headless Automation](11a-headless-automation-protocol.md#memcard-controls).
Media selection belongs to the host instance, never the cartridge manifest.
Opening, creation and the three boot choices are explicit. Missing, corrupt,
inaccessible or read-only media never silently select fresh RAM or create a file.
File hosts acquire a nonblocking exclusive OS lock, including for read-only
access, before validating the opened image. The lock follows file identity and
therefore covers aliases across desktop/headless processes. Contention is BUSY;
unsupported locking is an explicit refusal. Creation locks the new file before
publishing its name and never replaces an existing destination. Locks and native
buffers remain owned until actual IO cessation, independently of result release,
discard consent and timeout. Inspection through a second mounted reader is not
supported while an owner holds the image.
Native observation distinguishes dirty RAM, retained terminal results and actual
IO ownership. Reset/shutdown wait for admitted IO, or request cancellation when
explicitly selected, then reevaluate dirty RAM. Remaining dirty RAM requires
explicit discard consent; refusal preserves the resident session. Neither wait,
cancel nor discard starts a save or proves rollback. Unconfirmed cessation blocks
reset/closure confirmation and requires restart supervision. EOF/transport loss
attempts bounded cancellation/cleanup without saving or resuming the guest.
An installed image with an indeterminate publication remains write-blocked after
reset; another persistent admission requires explicit revalidation first.
The headless owner polls IO/deadlines while idle without advancing VM instructions,
input, ticks or frames. A resident game awaiting media action is paused, not
terminated. Explicit reconciliation reopens only the selected card identity after
safe cessation, preserves RAM/continuation and the original retained request
result, and never reruns bootstrap. Different-card switching is not part of this
public host surface.
The desktop host accepts `--memcard PATH` for an existing image and
`--memcard PATH --create-memcard` for explicit creation. Creation never replaces
an existing file. File creation, opening, mount validation and save IO execute
on the shared worker. Unsupported hosts must reject physical-allocation claims.
The headless host accepts an injected `MemcardStorage` through `install_memcard`
and an explicit player policy through `boot_with_memcard_choice`. Its boot loop
waits for actual worker progress with a bounded watchdog timeout; frame execution
remains independent of real-time pacing.
During a headless frame advance, a successful bootstrap hook may still depend on
its native presentation publication. Only while that dependency blocks the active
VM and the corresponding IO remains queued or running, the host owner waits on
the shared lane's progress signal, outside firmware/VM ticks. Each requested wait
is at most 5 ms. Polling preserves the request's original 500 ms cancellation and
1 s stop/escalation deadlines. A single additional 2 s owner deadline starts on
first entry to this wait within an advance and never resets on wake or reentry.
It does not extend the native deadlines. Native media action stops the advance
with exact progress; if the dependency outlives the owner deadline without a
terminal boundary, it returns a native `memcard_error` for `advance` with reason
`restart_required` and the current media state. Neither outcome confirms IO
cessation or releases worker-owned buffers, media or locks. An ordinary gameplay
commit that does not block the VM does not activate this wait. Pure IO polling
and waiting do not advance guest counters or input and do not pace game frames.
Before guest boot, firmware presents persistent, fresh-memory-only and (when
available) existing-save-memory-only choices. Choosing no card does not bypass
snapshot reservation. Leaving a resident game with dirty RAM requires an explicit
discard decision. A blocked host operation produces a restart dialog by 1 second;
it does not free in-flight buffers or create another worker.
Native administrative operations run outside gameplay. Whole-game removal,
explicit card revalidation and explicit reintegration use the shared worker.
Revalidation preserves known quarantined blocks; reintegration separately tests
and durably clears one exclusion. Same-card replacement preserves RAM and advances
the mount epoch. Different-card replacement takes an explicit keep-RAM or
load-existing choice and never invokes initialization or bootstrap again.