Files
prometeu-studio/discussion/lessons/DSC-0105-studio-visual-regression-artifacts/LSN-0110-original-checkpoints-and-reviewed-visual-evidence.md
T
bquarkz 1471b427f1
Test / Tests and coverage occurred while executing withChecks step.
Intrepid/Prometeu/Studio/pipeline/pr-master There was a failure building this commit
feat(studio): deliver asset authoring and automation workflows
Add shared human/task asset batches with reviewed atomic publication, explicit
reference rebinds, ownership inspection and qualified glyph/scene budgets.
Expose the same authoring and capacity results through Assets and MCP.

Deliver isolated guest test targets, retained visual evidence and immutable
runtime lifecycle/control observations. Preserve provider qualification and
installed authoring examples alongside the completed discussion lessons.

Close AGD-106 coordination and the delivered discussion workflows through
housekeeping, updating lesson references and preserving qualification limits.
Sound remains outside the asset delivery wave.

Validation: 579 module tests and 31 additional desktop, process and installed
checks passed; native glyph/scene 16/17 boundaries qualified. Discussion
validation passed after housekeeping.
2026-11-11 01:21:25 +00:00

11 KiB
Raw Blame History

id, ticket, title, created, tags
id ticket title created tags
LSN-0110 studio-visual-regression-artifacts Original Checkpoints and Reviewed Visual Evidence 2026-11-10
studio
automation
evidence
mcp
javafx

Problem and consolidated decision

Owner: studio, visual automation evidence. This lesson consolidates accepted DEC-0093 and completed PLN-0171 from DSC-0105 / AGD-0107. It builds on the immutable lifecycle observation contract.

A canonical advance already retained its own PNG and frame identity. Consumers still needed scripts to transport, persist and compare that image, and a later capture could accidentally replace the checkpoint under investigation. The decision delivered the complete single-checkpoint flow through shared services: persist the original operation result, import an authorized reference, compare, inspect expected/actual/diff, and review a baseline explicitly as a human.

No sequence runner, extra server, automatic baseline promotion, resizing or perceptual oracle was added. All eleven decision rules were implemented. No runtime repository files were written and no commits were created.

Result and evidence

450 tests passed, with zero failures, errors or skips, across API, Core, JavaFX, application, architecture, desktop, installed runtime/media, MCP/desktop and installed knowledge. Qualification retains commands, suites/cases and source/report hashes. Native checkpoints and raw measurements preserve the actual results; runtime provenance identifies the executed installation. The inspector screenshot shows a deliberate one-pixel rejection, original coordinates and reviewed revision.

Independent hand-constructed RGBA fixtures cover exact pass, known failure, alpha, transparent RGB, tolerance edges, half-open ROI, overlapping masks, empty regions, dimension mismatch, bounds and diff. An installed native checkpoint additionally proves persistence from the original operation across later frames, rebuild, release and project reopen, with a deliberate one-pixel derived difference. The derived image is not labeled an independent oracle.

The Zelda pilot uses a disposable copy and explicit memory_fresh media. Saved port screen/HUD/scroll images retain their paths/hashes and external provenance. All three contextual comparisons rejected; that is a retained comparison result, not an operational failure or a claim of an independent NES oracle. Input checkpoint labels identify the reference context and do not assert that every game transition was reached. No comparison changed a baseline.

Preserve the original, then compare

DefaultEvidenceTools.persist resolves a terminated operation and copies exactly TrackedOperation.runtimeResult().capture(). There is no advance or native CAPTURE call. The service retains project, task, operation/client-request, build, session, logical frame and frameId provenance when available. Frame counters normalize to lossless uint64-compatible values after storage round trips.

Publication request identity is separate from the original advance request:

{
  "projectRoot": "/project",
  "taskId": "current-task",
  "clientRequestId": "persist-checkpoint-1",
  "sourceClientRequestId": "advance-checkpoint-1"
}

Repeating the same publication request and parameters returns the same immutable publication. Changed parameters conflict. The parameter fingerprint includes the authorized import selection and actual bytes without exposing grant tokens as plain metadata. Lost responses can be recovered by publication request lookup through list, including after a fresh task is acquired; guest work is not replayed.

An artifact has an opaque ID, SHA-256 of original bytes, byte size, MIME, dimensions and immutable provenance. Current staleness is a separate view computed against the live project capture. Rebuild or reset never rewrites historical identity.

Publication is one atomic directory move

The application FileEvidenceStore owns .studio/evidence/. Each immutable publication contains PNG objects and its record/report/revision. Staging writes and file flushes precede an atomic directory move; only complete publication directories are visible. Orphan staging is cleaned under the store lock and never advertised as evidence. Published data survives process/project closure.

The default quota is 512 MiB, configurable through prometeu.evidence.quotaBytes. Admission counts object and record bytes under a shared JVM monitor and filesystem lock, so concurrent publishers cannot both consume the same remaining space. Refusal reports used/required/limit and leaves previous publications untouched. Nothing is evicted. Explicit deletion checks report and baseline references before removing a publication.

Bounded reads validate ID, project namespace, range and integrity. A read is at most 1 MiB; a whole PNG image is bounded to 32 MiB and decoded images to 16,777,216 pixels. Neither ID knowledge nor a local path grants project authority.

Reference authorization is observed, not asserted by the caller

File import admits a canonical project path or an exact project-bound human selection grant. Symlinks cannot bypass that rule. The adapter returns the verified canonical path and whether human authorization actually matched; supplying an arbitrary grant string cannot manufacture human-selected provenance. Imported references have external origin and no invented runtime/frame identity.

Import does not review a baseline. The inspector asks for explicit confirmation and the service acquires the existing human manual lease. The revision records previous/new identity and reference origin, checks the expected previous revision, and remains immutable. Agent grants disable human mutation while inspection stays available. The MCP catalogue contains no baseline-approval or external-path-grant operation. Failed, invalid or cancelled comparisons cannot trigger promotion.

Deterministic pixels and explainable rejection

The PNG adapter expands original samples to neutral RGBA8 without gamma/profile or perceptual correction. Core performs comparison without AWT/ImageIO/JavaFX types. JavaFX uses the same neutral pixels for exact channel inspection; display zoom is presentation only and does not alter comparison coordinates.

ROI and masks use original top-left coordinates and half-open rectangles. Invalid bounds are rejected rather than clipped. Overlapping masks exclude their union once inside the ROI; a region with no compared pixels is invalid. Explicit alpha compare/ignore determines whether alpha participates, while RGB always does, including at alpha zero.

Exact defaults allow zero channel error and zero differing pixels. A tolerant comparison declares channelLimit 0..255 and allowedDifferentPixels. A pixel differs when a compared channel exceeds the channel limit; maximumChannelError retains the observed error before tolerance. BoundingBox is null without differing pixels. Dimension mismatch returns its own rejection verdict without resize. Invalid parameters, decoding, storage or authority failures are operational errors.

Reports retain normalized parameters, original expected/actual byte hashes, verdict, compared/different counts, maximum error, bounds and all three artifact IDs. Diff images preserve original dimensions: transparent black is excluded, opaque green is compared within tolerance, and opaque red/yellow is different.

Shared service, existing adapters

Task mutations use the existing project operation executor; human mutations use the existing manual lease. Worker termination, revoke/reclaim and document handoff remain authoritative. Already published evidence survives cancellation. Revoked tasks are denied; a fresh authorized task can read same-project history. The local bound human inspector remains read-only during an agent grant.

Eight studio_evidence tools add persist/import/compare/list/metadata/read/image/ delete to the existing authenticated MCP endpoint. Image results carry direct PNG content; bytes and metadata use IDs and bounded reads. HTTP tests verify missing credentials, cross-project denial and revoked-task denial. The real JavaFX test verifies exact pixel display, report parity with MCP, explicit baseline confirmation and disabled mutation during a grant.

Measure the actual tradeoff

The same three Zelda checkpoints were exercised through the former image-download and external-comparison path and the new publication/report path:

Measured adapter work Former path Evidence path
MCP calls 3 9
Serialized response bytes 187334 23766
PNG bytes transferred for metrics 106825 0
External comparison scripts executed 1 0

Three guest advance requests were shared by both paths. Build/advance/polling and optional later image inspection are outside these adapter counts. The former script compared full images; the new HUD case declared an ROI, so its pixel counts are not presented as identical workloads. These raw observations show fewer transported bytes and no external comparison script at the cost of more explicit service calls; they are not a universal performance percentage.

Acceptance coverage and pitfalls

D1–D3 are covered by original-checkpoint publication, request recovery and immutable identity; D4–D5 by atomic storage, reference checks and concurrent quota tests; D6–D7 by scoped HTTP/ID access and canonical reference admission; D8–D10 by the independent pixel fixtures and complete native reports; D11 by the desktop/MCP parity and human baseline-review gate. Installed guide/catalog examples ship in the packaged distribution and pass its knowledge probe.

  • Never recapture to reconstruct the outcome of an earlier advance.
  • Do not change historical frame identity when current state becomes stale.
  • Treat import and baseline approval as separate actions with different authority.
  • Preserve valid publications on quota refusal, cancellation and interrupted staging.
  • Do not hide dimension mismatch, empty regions or unknown authority as a pass.
  • Count mask overlaps once, compare transparent RGB, and keep diff coordinates.
  • Keep reference provenance and oracle independence explicit, especially for port captures compared with their own earlier renders.

The durable contracts are Studio 8, Studio 9, Studio 10 and Studio 11. Completed DEC-0093/PLN-0171 scaffolding is removed by CLI housekeeping. Isolated guest targets (DSC-0106) can reference these artifacts without adding another runner.