30 Commits
Author SHA1 Message Date
bquarkz f480b755ed fix(runtime): restore tooling coverage and allow atomic release ordering
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
2026-11-21 15:38:33 +00:00
bquarkz c2bf8d055c feat(stress): add a deterministic particle demo with sequenced music
Move Storm Eagle playback and source assets to Stress Console and keep Dummy Boy silent. Exercise VM math, seeded Random and persistent typed particles with a projected tunnel, bounded 512-sprite heavy workload, reproducible replays and measured costs.
2026-11-20 22:15:47 +00:00
bquarkz 8510c5b00d feat(audio): add 32-voice PCM mixing and native music sequencing
Add self-contained SoundBanks and MusicBanks with per-sample PCM16 and IMA ADPCM storage, materialized to resident PCM16. Integrate instrument envelopes, automation, shared DSP and host lifecycle controls while preserving staged real-time execution.
2026-11-20 22:15:24 +00:00
bquarkz a32f834b2b feat(graphics): migrate native target to 640x360
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
Centralize the fixed 640x360 RGBA8888/60 Hz contract in the existing HAL.
Preserve 8/16/32 tiles and composition, cover partial edges and mixed-size
camera movement, and adapt firmware, fixtures, hosts and native tests.

Share nearest-neighbor presentation geometry with pointer input. Measure
actual framebuffer publications and successful host presentations separately
from VM ticks, with bounded opt-in diagnostics and release harnesses.

Validated with the complete tooling verify gate, native rendering tests,
GPU readback, headless/process integration, formatting and strict clippy.
The existing synchronous session path is documented; no worker/session
architecture replacement or format compatibility layer is introduced.

Work: WRK-0016 / DEC-0001 / PLN-0001
Commit and push explicitly authorized by the user after completion.
2026-11-20 10:34:45 +00:00
bquarkz f1adf86dd5 feat(runtime): support Studio Play and separate integration gates
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
Add the windowless Studio host with authenticated binary control/frame channels,
managed input, pause/resume/stop, frame ownership and save-aware shutdown.
Preserve deterministic headless execution and add protocol/process fixtures.
Separate unit CI and coverage from opt-in integration/socket qualification,
retaining the prior combined coverage floors in the configuration history.

Validation: tooling make verify passed runtime ci-domains and integration.
2026-11-19 17:32:02 +00:00
bquarkz 2d5735cb13 chore: retire Discussion Framework workflow and retain lessons
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
2026-11-16 16:45:34 +00:00
bquarkz dccf4d0775 fix(ci): declare llvm-tools-preview in pinned toolchain
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
2026-11-16 10:00:49 +00:00
bquarkz c3589be1ad fix(headless): wait for memcard bootstrap IO progress
Intrepid/Prometeu/Runtime/pipeline/head There was a failure building this commit
Coordinate bounded native publication progress during frame advance while preserving watchdogs, guest budgets and resource ownership. Add controlled and packaged reopen regressions, clarify the runtime specs, and retain qualification evidence.
2026-11-10 23:26:40 +00:00
bquarkz 49d7fca0b7 chore(toolchain): pin Rust to 1.98.1
Intrepid/Prometeu/Runtime/pipeline/head There was a failure building this commit
2026-11-10 22:39:19 +00:00
bquarkz c649e02816 feat(headless): expose negotiated native log observation 2026-11-10 21:57:34 +00:00
bquarkz 09d8778f04 fix(runtime): support older stable Rust atomic updates
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
2026-11-05 09:00:19 +00:00
bquarkz 1aa939eca8 fix(runtime): replace deprecated atomic fetch_update calls
Intrepid/Prometeu/Runtime/pipeline/head There was a failure building this commit
2026-11-03 00:01:58 +00:00
bquarkz a4bec5bc45 feat(host): expose memcard controls and qualify process persistence
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
2026-10-30 10:10:07 +00:00
bquarkz dcf6561ea7 refactor(vm)!: remove publication and logical guard opcodes
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
Remove INDEX_GUARD, ARRAY_CHECK_LENGTH, STORAGE_PUBLISH and FIELDS_PUBLISH
from decoding, metadata, tooling, verification and execution. Keep ARRAY_LEN
and overlap-safe ARRAY_COPY with complete-group ownership and early charging
of calculable copy work, including refused attempts.

Update specifications, producer handoff, qualification evidence and lessons.
Share collection test support and remove redundant Copy clones for Clippy.

Validation: make ci-domains passed with 1,095 workspace tests, five separately
executed debugger socket tests and all 15 unchanged coverage domain gates.
VM release, resource diagnostics and layer/quality suites also passed.

BREAKING CHANGE: PBX using opcodes 0xA7, 0xA8, 0xAA or 0xAB is rejected,
including unreachable instructions. Consumers must adapt their emission and
collection algorithms or operational guarantees.
2026-10-23 23:37:56 +01:00
bquarkz 69ad1931de feat(vm): add reference identity and deterministic value hashing 2026-10-23 14:29:00 +01:00
Nilton Constantino 57992cf214 feat(vm): publish typed fields across multiple structs atomically
Add schema-driven FIELDS_PUBLISH without collection-specific runtime behavior. Validate every nullable destination and complete field group before allocation-free publication, with neutral ownership, malformed-input and quota coverage.
2026-10-23 09:53:40 +01:00
bquarkz d9b9eeb171 Add generic logical guards and atomic storage publication 2026-10-22 22:24:28 +01:00
bquarkz f7d7c16753 docs: include Random in registry handoff builtin list
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
2026-10-14 15:04:07 +01:00
bquarkz 07b69314b1 Merge pull request 'Dev/runtime improvements for games' (#42) from dev/runtime-improvements-for-games into master
Intrepid/Prometeu/Runtime/pipeline/head This commit looks good
Reviewed-on: #42
2026-10-14 13:58:56 +00:00
bquarkz 4669ced98e fix(ci): accept installed llvm-tools component name
Intrepid/Prometeu/Runtime/pipeline/pr-master This commit looks good
2026-10-14 14:43:09 +01:00
bquarkz a2275932a5 Keep PBX at v0 and simplify Jenkins quality gate
Intrepid/Prometeu/Runtime/pipeline/pr-master There was a failure building this commit
2026-10-14 14:35:44 +01:00
bquarkz 39468740c3 docs: close runtime quality review with consolidated lesson
Intrepid/Prometeu/Runtime/pipeline/pr-master There was a failure building this commit
2026-10-14 14:24:14 +01:00
bquarkz 3f9d235546 Index string provenance with metered image and session tables 2026-10-14 13:55:16 +01:00
bquarkz bb6f212779 Share asset decoding validation and reject truncated payloads 2026-10-14 13:08:34 +01:00
bquarkz 418e574ac8 ci: enforce a shared strict repository quality gate 2026-10-14 12:14:11 +01:00
bquarkz 461b554c1a perf(audio): reclaim callback owners through bounded return packets 2026-10-14 11:59:47 +01:00
bquarkz 46bbad19c2 perf(vm): eliminate recurring stack and return allocations 2026-10-14 10:54:11 +01:00
bquarkz e09f1197bc fix(audio): preserve full sample boundaries for one-shot playback 2026-10-14 10:46:03 +01:00
bquarkz ff73f8ad39 fix(coverage): restore specialized domain ownership and floors 2026-10-14 10:35:41 +01:00
bquarkz bad942fb25 fix(memcard): preserve watchdog timeout reasons across completion 2026-10-14 10:26:30 +01:00
307 changed files with 50448 additions and 1556 deletions
+19 -9
View File
@@ -13,17 +13,27 @@ jobs:
with: with:
submodules: recursive submodules: recursive
- uses: dtolnay/rust-toolchain@stable - name: Install repository Rust toolchain
with: run: |
components: rustfmt, clippy toolchain=$(python3 -c 'import tomllib; print(tomllib.load(open("rust-toolchain.toml", "rb"))["toolchain"]["channel"])')
rustup toolchain install "$toolchain" --profile minimal
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
- name: Format check - name: Discard reports from previous runs
run: cargo fmt -- --check run: rm -rf target/llvm-cov
- name: Clippy - name: Provision quality-gate tools and native dependencies
run: cargo clippy --workspace --all-features run: bash scripts/provision-quality-gate.sh
- name: Test - name: Repository quality gate
run: cargo test --workspace --all-targets --all-features --no-fail-fast run: make ci-domains
- name: Publish gate evidence (including failures)
if: always()
uses: actions/upload-artifact@v4
with:
name: quality-gate-${{ runner.os }}-${{ github.sha }}
path: |
target/llvm-cov/**
if-no-files-found: warn
-4
View File
@@ -66,7 +66,3 @@ AGENTS.md
.python-version .python-version
.venv .venv
discussion/*
!discussion/lessons/
!discussion/lessons/**
Generated
+4
View File
@@ -1883,6 +1883,7 @@ dependencies = [
"anyhow", "anyhow",
"prometeu-bytecode", "prometeu-bytecode",
"prometeu-hal", "prometeu-hal",
"prometeu-host-headless",
"serde_json", "serde_json",
] ]
@@ -1892,7 +1893,10 @@ version = "0.1.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"prometeu-bytecode", "prometeu-bytecode",
"prometeu-drivers",
"prometeu-hal", "prometeu-hal",
"prometeu-host-headless",
"prometeu-vm",
"serde_json", "serde_json",
] ]
+24 -11
View File
@@ -1,25 +1,32 @@
.PHONY: fmt fmt-check clippy test-local test-debugger-socket clean coverage coverage-xml coverage-json coverage-report-json coverage-domain-list coverage-domain-evidence coverage-domain-audit coverage-domain-gate test ci ci-domains cobertura .PHONY: fmt fmt-check clippy test-unit test-local test-integration test-debugger-socket clean coverage coverage-xml coverage-json coverage-report-json coverage-domain-list coverage-domain-evidence coverage-domain-audit coverage-domain-gate test ci ci-domains cobertura
fmt: fmt:
cargo fmt cargo fmt --all
fmt-check: fmt-check:
cargo fmt -- --check cargo fmt --all -- --check
clippy: clippy:
cargo clippy --workspace --all-features cargo clippy --workspace --all-targets --all-features -- -D warnings
test-local: test-unit:
cargo test --workspace --all-targets --all-features --no-fail-fast cargo test --workspace --lib --bins --all-features --no-fail-fast
# Compatibility entry point: repository-local verification is unit-only.
test-local: test-unit
# Explicit opt-in, orchestrated by ../tooling; test sources stay in this repository.
test-integration:
cargo test --workspace --tests --all-features --no-fail-fast
test-debugger-socket: test-debugger-socket:
cargo test -p prometeu-host-desktop-winit --lib -- --ignored cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored
clean: clean:
cargo llvm-cov clean --workspace cargo llvm-cov clean --workspace
coverage: coverage:
cargo llvm-cov --workspace --all-features --html --output-dir target/llvm-cov cargo llvm-cov --workspace --lib --bins --all-features --html --output-dir target/llvm-cov
coverage-report-json: coverage-report-json:
cargo llvm-cov report --json --output-path target/llvm-cov/report.json cargo llvm-cov report --json --output-path target/llvm-cov/report.json
@@ -46,7 +53,13 @@ coverage-domain-evidence:
coverage-domain-gate: coverage-domain-gate:
./scripts/coverage-domain-evidence.sh --check-all ./scripts/coverage-domain-evidence.sh --check-all
test: fmt-check clippy test-local test-debugger-socket test: fmt-check clippy test-unit
ci: clean fmt-check clippy coverage coverage-report-json # Local and GitHub entry point; Jenkins exposes the same targets as stages.
ci-domains: clean fmt-check clippy coverage coverage-report-json coverage-xml coverage-json coverage-domain-gate CI_DOMAIN_STEPS = clean fmt-check clippy test-unit coverage coverage-report-json coverage-xml coverage-json coverage-domain-audit coverage-domain-gate
ci: ci-domains
ci-domains:
rm -rf target/llvm-cov
+@set -e; for step in $(CI_DOMAIN_STEPS); do $(MAKE) --no-print-directory -j1 $$step; done
cobertura: coverage-xml coverage-json cobertura: coverage-xml coverage-json
+40 -25
View File
@@ -1,14 +1,14 @@
# PROMETEU Runtime # PROMETEU Runtime
PROMETEU is an educational and experimental fantasy handheld / fantasy console. PROMETEU is an educational and experimental fantasy handheld / fantasy console.
This repository contains the Rust runtime workspace for the machine: VM-facing components, virtual peripherals, firmware/system services, host integration, CLI entrypoints, development utilities, specs, and discussion artifacts. This repository contains the Rust runtime workspace for the machine: VM-facing components, virtual peripherals, firmware/system services, host integration, CLI entrypoints, development utilities, specs, and engineering lessons.
The VM is only one subsystem of the machine. The canonical docs in this repository intentionally distinguish: The VM is only one subsystem of the machine. The canonical docs in this repository intentionally distinguish:
- machine-level specs; - machine-level specs;
- VM/runtime internal architecture; - VM/runtime internal architecture;
- implementation crates; - implementation crates;
- discussion workflow and lessons learned. - engineering lessons learned.
## What This Repository Contains ## What This Repository Contains
@@ -19,7 +19,7 @@ The VM is only one subsystem of the machine. The canonical docs in this reposito
- `docs/specs/runtime/`: canonical PROMETEU machine specs - `docs/specs/runtime/`: canonical PROMETEU machine specs
- `docs/vm-arch/`: canonical VM/runtime architecture and ISA references - `docs/vm-arch/`: canonical VM/runtime architecture and ISA references
- `devtools/`: debugger protocol material - `devtools/`: debugger protocol material
- `discussion/`: agendas, decisions, plans, and lessons for architectural work - `discussion/lessons/`: published engineering lessons and supporting assets
- `test-cartridges/`: cartridge fixtures used for validation and manual runs - `test-cartridges/`: cartridge fixtures used for validation and manual runs
## Canonical Documentation ## Canonical Documentation
@@ -29,7 +29,11 @@ Use these entrypoints instead of inferring the model from isolated source files:
- [Machine specs](docs/specs/runtime/README.md): authoritative contract for the PROMETEU machine, peripherals, firmware, cartridge format, timing, and host ABI - [Machine specs](docs/specs/runtime/README.md): authoritative contract for the PROMETEU machine, peripherals, firmware, cartridge format, timing, and host ABI
- [VM architecture](docs/vm-arch/ARCHITECTURE.md): authoritative internal architecture for VM/runtime invariants - [VM architecture](docs/vm-arch/ARCHITECTURE.md): authoritative internal architecture for VM/runtime invariants
- [ISA reference](docs/vm-arch/ISA_CORE.md): bytecode-level instruction set authority - [ISA reference](docs/vm-arch/ISA_CORE.md): bytecode-level instruction set authority
- [Discussion workflow](discussion/index.ndjson): architectural agenda, decision, and execution traceability - [Engineering lessons](discussion/lessons/): published results and rationale
Engineering discussions, decisions, plans and execution evidence are maintained in
the enclosing [Prometeu Engineering workspace](../README.md) using
[AI Engineering](../.ai-engineering/README.md).
## Workspace Layout ## Workspace Layout
@@ -60,11 +64,21 @@ Other ecosystem commands, such as build, pack, and verification tooling, are not
## Requirements ## Requirements
- Rust toolchain from [rust-toolchain.toml](rust-toolchain.toml) - Rust **1.98.1**, pinned in [rust-toolchain.toml](rust-toolchain.toml)
- `rustup` for toolchain installation and management - `rustup` for toolchain installation and management
- `cargo-llvm-cov` for coverage targets such as `make coverage` and `make ci-domains` - `cargo-llvm-cov` for coverage targets such as `make coverage` and `make ci-domains`
- a local graphical environment for desktop runtime commands - a local graphical environment for desktop runtime commands
From the repository root, install and verify the pinned toolchain:
```bash
rustup toolchain install 1.98.1 --profile minimal --component rustfmt,clippy
rustc --version
```
Rustup selects the pinned version automatically for commands run in this repository.
CI installs the toolchain from the same file.
## Quick Start ## Quick Start
Build the workspace: Build the workspace:
@@ -107,19 +121,26 @@ Format, lint, and test the workspace:
make test make test
``` ```
Run the CI-oriented non-domain gate: Run the shared strict CI gate (`make ci` is an alias):
```bash ```bash
make ci make ci
``` ```
Run the CI-oriented domain coverage gate: The authoritative target used by both GitHub and Jenkins is:
```bash ```bash
make ci-domains make ci-domains
``` ```
The debugger socket tests bind localhost sockets and are marked ignored in Rust test metadata; `make test` includes them through the dedicated `test-debugger-socket` target. `make test-unit` selects library and binary unit-test harnesses with
`--lib --bins --all-features`; `test-local` remains an alias. Both `make test`
and `make ci-domains` run only this suite. Integration targets under `tests/`,
installed-process qualification and the five ignored debugger socket tests are
excluded from the pipeline. They remain available explicitly through
`make test-integration` and `make test-debugger-socket`, orchestrated by
`../tooling`. Tests and fixtures stay in this repository. Physical audio and manual probes remain opt-in. See
[quality-gate prerequisites and commands](docs/vm-arch/COVERAGE_DOMAINS.md#commands-and-ci).
## Coverage Governance ## Coverage Governance
@@ -128,20 +149,14 @@ The repository uses a two-layer runtime-edge coverage model:
- a mandatory CI gate based on canonical runtime-edge domains; - a mandatory CI gate based on canonical runtime-edge domains;
- domain-oriented review evidence for the canonical runtime-edge domains. - domain-oriented review evidence for the canonical runtime-edge domains.
Current canonical domains: The fifteen canonical domains and fixed floors are defined in
[the repository configuration](files/config/runtime-edge-coverage-domains.json).
- `system/runtime` Use `make coverage-domain-list` or the [ownership documentation](docs/vm-arch/COVERAGE_DOMAINS.md)
- `fs` to inspect the current map.
- `asset/bank`
- `firmware`
- `host-dependent`
- `host/headless`
Global coverage artifacts: Global coverage artifacts:
```bash ```bash
make ci
make cobertura
make ci-domains make ci-domains
``` ```
@@ -161,12 +176,12 @@ make coverage-domain-evidence DOMAIN=host-dependent
make coverage-domain-evidence DOMAIN=host/headless make coverage-domain-evidence DOMAIN=host/headless
``` ```
Per-domain baselines currently start around `60`. `make ci-domains` runs formatting, strict workspace/all-target/all-feature Clippy,
That does not waive test obligations. It means the project is adopting explicit domain evidence immediately while tightening quantitative expectations incrementally over time. library/binary unit tests, fresh unit-only coverage and every domain
ownership/evidence gate. Both CI providers archive available evidence on failure.
`make ci-domains` is the CI entrypoint used by Jenkins. A failing run never lowers a configured floor; local and CI checks use the same
It runs the standard coverage pipeline, produces the HTML/XML/JSON artifacts, and fails when any canonical domain falls below its configured `baseline_percent`. repository-owned commands. Reports identify their revision, source/configuration
The CI gate currently compares domain `lines` coverage against each domain baseline. and actual execution platform.
When a PR changes the observable contract of a canonical domain, it is expected to: When a PR changes the observable contract of a canonical domain, it is expected to:
@@ -179,7 +194,7 @@ Improving the global percentage alone is not sufficient review evidence when the
The project is still in active architectural and implementation convergence. The project is still in active architectural and implementation convergence.
- the machine contract is being clarified through the specs and discussion workflow; - the machine contract is being clarified through the specs and AI Engineering;
- the workspace already contains concrete runtime and host code; - the workspace already contains concrete runtime and host code;
- the CLI dispatcher currently exposes the runtime paths that are backed by binaries in this workspace. - the CLI dispatcher currently exposes the runtime paths that are backed by binaries in this workspace.
@@ -179,7 +179,7 @@ pub fn assemble(src: &str) -> Result<Vec<u8>, AsmError> {
emit_u16(CoreOpCode::Ret as u16, &mut out); emit_u16(CoreOpCode::Ret as u16, &mut out);
} }
"YIELD" | "SLEEP" | "SPAWN" => { "YIELD" | "SLEEP" | "SPAWN" => {
return Err(AsmError::InvalidOperand("retired coroutine instruction".into())) return Err(AsmError::InvalidOperand("retired coroutine instruction".into()));
} }
"FRAME_SYNC" => { "FRAME_SYNC" => {
emit_u16(CoreOpCode::FrameSync as u16, &mut out); emit_u16(CoreOpCode::FrameSync as u16, &mut out);
@@ -345,7 +345,7 @@ pub fn assemble(src: &str) -> Result<Vec<u8>, AsmError> {
emit_u32(ty, &mut out); emit_u32(ty, &mut out);
emit_u32(field, &mut out); emit_u32(field, &mut out);
} }
"STRUCT_NEW" | "ARRAY_NEW" | "ARRAY_LEN" | "ARRAY_GET" | "ARRAY_SET" => { "STRUCT_NEW" | "ARRAY_NEW" | "ARRAY_LEN" | "ARRAY_GET" | "ARRAY_SET" | "ARRAY_COPY" => {
if ops.is_empty() { if ops.is_empty() {
return Err(AsmError::MissingOperand(line.into())); return Err(AsmError::MissingOperand(line.into()));
} }
@@ -354,23 +354,32 @@ pub fn assemble(src: &str) -> Result<Vec<u8>, AsmError> {
"ARRAY_NEW" => CoreOpCode::ArrayNew, "ARRAY_NEW" => CoreOpCode::ArrayNew,
"ARRAY_LEN" => CoreOpCode::ArrayLen, "ARRAY_LEN" => CoreOpCode::ArrayLen,
"ARRAY_GET" => CoreOpCode::ArrayGet, "ARRAY_GET" => CoreOpCode::ArrayGet,
"ARRAY_COPY" => CoreOpCode::ArrayCopy,
_ => CoreOpCode::ArraySet, _ => CoreOpCode::ArraySet,
}; };
emit_u16(op as u16, &mut out); emit_u16(op as u16, &mut out);
emit_u32(parse_u32_any(ops)?, &mut out); emit_u32(parse_u32_any(ops)?, &mut out);
} }
"DYN_PACK" | "DYN_UNPACK" | "DYN_CAN_UNPACK" => { "DYN_PACK" | "DYN_UNPACK" | "DYN_CAN_UNPACK" | "REF_IDENTITY_HASH" => {
if ops.is_empty() { if ops.is_empty() {
return Err(AsmError::MissingOperand(line.into())); return Err(AsmError::MissingOperand(line.into()));
} }
let opcode = match mn { let opcode = match mn {
"DYN_PACK" => CoreOpCode::DynPack, "DYN_PACK" => CoreOpCode::DynPack,
"DYN_UNPACK" => CoreOpCode::DynUnpack, "DYN_UNPACK" => CoreOpCode::DynUnpack,
"REF_IDENTITY_HASH" => CoreOpCode::RefIdentityHash,
_ => CoreOpCode::DynCanUnpack, _ => CoreOpCode::DynCanUnpack,
}; };
emit_u16(opcode as u16, &mut out); emit_u16(opcode as u16, &mut out);
emit_u32(parse_u32_any(ops)?, &mut out); emit_u32(parse_u32_any(ops)?, &mut out);
} }
"REF_IDENTITY_EQ" => {
let (a, b) = parse_keyvals(ops)?;
let (left, right) = parse_pair(a, "left", b, "right")?;
emit_u16(CoreOpCode::RefIdentityEq as u16, &mut out);
emit_u32(left, &mut out);
emit_u32(right, &mut out);
}
"REF_IS_EXACT" | "REF_IMPLEMENTS" | "REF_TO_TRAIT" | "REF_CAST" => { "REF_IS_EXACT" | "REF_IMPLEMENTS" | "REF_TO_TRAIT" | "REF_CAST" => {
if ops.is_empty() { if ops.is_empty() {
return Err(AsmError::MissingOperand(line.into())); return Err(AsmError::MissingOperand(line.into()));
@@ -65,12 +65,14 @@ fn format_operand(op: CoreOpCode, imm: &[u8]) -> String {
| CoreOpCode::Hostcall | CoreOpCode::Hostcall
| CoreOpCode::DynPack | CoreOpCode::DynPack
| CoreOpCode::DynUnpack | CoreOpCode::DynUnpack
| CoreOpCode::RefIdentityHash
| CoreOpCode::DynCanUnpack | CoreOpCode::DynCanUnpack
| CoreOpCode::StructNew | CoreOpCode::StructNew
| CoreOpCode::ArrayNew | CoreOpCode::ArrayNew
| CoreOpCode::ArrayLen | CoreOpCode::ArrayLen
| CoreOpCode::ArrayGet | CoreOpCode::ArrayGet
| CoreOpCode::ArraySet => { | CoreOpCode::ArraySet
| CoreOpCode::ArrayCopy => {
let v = u32::from_le_bytes(imm.try_into().unwrap()); let v = u32::from_le_bytes(imm.try_into().unwrap());
format!("{}", v) format!("{}", v)
} }
@@ -79,6 +81,11 @@ fn format_operand(op: CoreOpCode, imm: &[u8]) -> String {
let field = u32::from_le_bytes(imm[4..].try_into().unwrap()); let field = u32::from_le_bytes(imm[4..].try_into().unwrap());
format!("type={}, field={}", ty, field) format!("type={}, field={}", ty, field)
} }
CoreOpCode::RefIdentityEq => {
let left = u32::from_le_bytes(imm[..4].try_into().unwrap());
let right = u32::from_le_bytes(imm[4..].try_into().unwrap());
format!("left={}, right={}", left, right)
}
CoreOpCode::RefIsExact CoreOpCode::RefIsExact
| CoreOpCode::RefImplements | CoreOpCode::RefImplements
| CoreOpCode::RefToTrait | CoreOpCode::RefToTrait
@@ -1,4 +1,4 @@
//! Shared PBX v1 resource limits and structured validation failures. //! Shared PBX v0 resource limits and structured validation failures.
//! Counters are deterministic; a failed reservation never changes usage. //! Counters are deterministic; a failed reservation never changes usage.
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::fmt; use std::fmt;
@@ -192,6 +192,10 @@ pub enum OpCode {
RefToTrait = 0x95, RefToTrait = 0x95,
/// REF_CAST: source:TypeId, target:TypeId; requires validated Registry metadata. /// REF_CAST: source:TypeId, target:TypeId; requires validated Registry metadata.
RefCast = 0x96, RefCast = 0x96,
/// Query a complete typed reference identity; source TypeId.
RefIdentityHash = 0x97,
/// Compare two complete typed references; left/right TypeIds.
RefIdentityEq = 0x98,
// Registry-backed collection operations. cycles() is their base component. // Registry-backed collection operations. cycles() is their base component.
StructNew = 0xA0, StructNew = 0xA0,
@@ -201,6 +205,8 @@ pub enum OpCode {
ArrayLen = 0xA4, ArrayLen = 0xA4,
ArrayGet = 0xA5, ArrayGet = 0xA5,
ArraySet = 0xA6, ArraySet = 0xA6,
/// Complete stored groups, overlap-safe; Array TypeId:u32.
ArrayCopy = 0xA9,
} }
impl TryFrom<u16> for OpCode { impl TryFrom<u16> for OpCode {
@@ -267,6 +273,8 @@ impl TryFrom<u16> for OpCode {
0x94 => Ok(OpCode::RefImplements), 0x94 => Ok(OpCode::RefImplements),
0x95 => Ok(OpCode::RefToTrait), 0x95 => Ok(OpCode::RefToTrait),
0x96 => Ok(OpCode::RefCast), 0x96 => Ok(OpCode::RefCast),
0x97 => Ok(OpCode::RefIdentityHash),
0x98 => Ok(OpCode::RefIdentityEq),
0xA0 => Ok(OpCode::StructNew), 0xA0 => Ok(OpCode::StructNew),
0xA1 => Ok(OpCode::StructGet), 0xA1 => Ok(OpCode::StructGet),
0xA2 => Ok(OpCode::StructSet), 0xA2 => Ok(OpCode::StructSet),
@@ -274,6 +282,7 @@ impl TryFrom<u16> for OpCode {
0xA4 => Ok(OpCode::ArrayLen), 0xA4 => Ok(OpCode::ArrayLen),
0xA5 => Ok(OpCode::ArrayGet), 0xA5 => Ok(OpCode::ArrayGet),
0xA6 => Ok(OpCode::ArraySet), 0xA6 => Ok(OpCode::ArraySet),
0xA9 => Ok(OpCode::ArrayCopy),
_ => Err(format!("Invalid OpCode: 0x{:04X}", value)), _ => Err(format!("Invalid OpCode: 0x{:04X}", value)),
} }
@@ -344,6 +353,8 @@ impl OpCode {
OpCode::RefImplements => 4, OpCode::RefImplements => 4,
OpCode::RefToTrait => 5, OpCode::RefToTrait => 5,
OpCode::RefCast => 4, OpCode::RefCast => 4,
OpCode::RefIdentityHash => 8,
OpCode::RefIdentityEq => 4,
OpCode::StructNew => 8, OpCode::StructNew => 8,
OpCode::StructGet => 3, OpCode::StructGet => 3,
OpCode::StructSet => 4, OpCode::StructSet => 4,
@@ -351,6 +362,7 @@ impl OpCode {
OpCode::ArrayLen => 3, OpCode::ArrayLen => 3,
OpCode::ArrayGet => 3, OpCode::ArrayGet => 3,
OpCode::ArraySet => 4, OpCode::ArraySet => 4,
OpCode::ArrayCopy => 4,
} }
} }
} }
@@ -596,6 +596,26 @@ impl OpCodeSpecExt for OpCode {
may_trap: true, may_trap: true,
is_safepoint: false, is_safepoint: false,
}, },
OpCode::RefIdentityHash => OpcodeSpec {
name: "REF_IDENTITY_HASH",
imm_bytes: 4,
pops: None,
pushes: None,
is_branch: false,
is_terminator: false,
may_trap: true,
is_safepoint: false,
},
OpCode::RefIdentityEq => OpcodeSpec {
name: "REF_IDENTITY_EQ",
imm_bytes: 8,
pops: None,
pushes: None,
is_branch: false,
is_terminator: false,
may_trap: true,
is_safepoint: false,
},
OpCode::StructNew => OpcodeSpec { OpCode::StructNew => OpcodeSpec {
name: "STRUCT_NEW", name: "STRUCT_NEW",
imm_bytes: 4, imm_bytes: 4,
@@ -666,6 +686,16 @@ impl OpCodeSpecExt for OpCode {
may_trap: true, may_trap: true,
is_safepoint: false, is_safepoint: false,
}, },
OpCode::ArrayCopy => OpcodeSpec {
name: "ARRAY_COPY",
imm_bytes: 4,
pops: None,
pushes: None,
is_branch: false,
is_terminator: false,
may_trap: true,
is_safepoint: false,
},
OpCode::FrameSync => OpcodeSpec { OpCode::FrameSync => OpcodeSpec {
name: "FRAME_SYNC", name: "FRAME_SYNC",
imm_bytes: 0, imm_bytes: 0,
@@ -52,6 +52,19 @@ pub enum TypeKind {
Trait(Vec<MethodDecl>), Trait(Vec<MethodDecl>),
Fn(Signature), Fn(Signature),
} }
impl TypeKind {
/// Categories supported by identity queries, not by legacy REF_* conversions.
pub fn is_identity_reference(&self) -> bool {
matches!(
self,
Self::Struct(_)
| Self::Array(_)
| Self::Fn(_)
| Self::Trait(_)
| Self::Builtin(BuiltinId::RANDOM)
)
}
}
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub struct TypeDecl { pub struct TypeDecl {
pub name: Option<String>, pub name: Option<String>,
@@ -1,4 +1,4 @@
//! Strict PBX v1 envelope and Registry sections. No legacy fallback. //! Strict PBX v0 envelope and Registry sections. No legacy fallback.
//! Envelope parsing is structural only: non-Registry sections and instructions //! Envelope parsing is structural only: non-Registry sections and instructions
//! must be validated by their consumers before an executable image is published. //! must be validated by their consumers before an executable image is published.
use crate::load_limits::{ use crate::load_limits::{
@@ -32,7 +32,7 @@ impl<'a> PbxEnvelope<'a> {
if &bytes[..4] != b"PBX\0" { if &bytes[..4] != b"PBX\0" {
return Err(E::new(C::InvalidFormat, R::Magic)); return Err(E::new(C::InvalidFormat, R::Magic));
} }
if bytes[4..6] != [1, 0] { if bytes[4..6] != [0, 0] {
return Err(E::new(C::UnsupportedFormat, R::Version)); return Err(E::new(C::UnsupportedFormat, R::Version));
} }
if bytes[6] != 0 { if bytes[6] != 0 {
@@ -314,14 +314,14 @@ impl<'a, 'b> Reader<'a, 'b> {
} }
} }
/// Raw section input for a v1 envelope writer. Non-Registry section bytes must /// Raw section input for a v0 envelope writer. Non-Registry section bytes must
/// come from their own validated writers; this API never invents defaults. /// come from their own validated writers; this API never invents defaults.
#[derive(Clone, Copy)] #[derive(Clone, Copy)]
pub struct SectionData<'a> { pub struct SectionData<'a> {
pub kind: u32, pub kind: u32,
pub bytes: &'a [u8], pub bytes: &'a [u8],
} }
/// Assemble strict v1 sections. The output buffer is charged to temporary load storage. /// Assemble strict v0 sections. The output buffer is charged to temporary load storage.
pub fn encode_envelope(sections: &[SectionData<'_>], b: &mut LoadBudget) -> Result<Vec<u8>, E> { pub fn encode_envelope(sections: &[SectionData<'_>], b: &mut LoadBudget) -> Result<Vec<u8>, E> {
b.transaction(|b| { b.transaction(|b| {
let mut by_kind = [None; 9]; let mut by_kind = [None; 9];
@@ -349,7 +349,7 @@ pub fn encode_envelope(sections: &[SectionData<'_>], b: &mut LoadBudget) -> Resu
b.work(total)?; b.work(total)?;
let mut out = b.vector(total, Q::TemporaryBytes)?; let mut out = b.vector(total, Q::TemporaryBytes)?;
out.extend_from_slice(b"PBX\0"); out.extend_from_slice(b"PBX\0");
out.extend_from_slice(&[1, 0, 0, 0]); out.extend_from_slice(&[0, 0, 0, 0]);
out.extend_from_slice(&(count as u32).to_le_bytes()); out.extend_from_slice(&(count as u32).to_le_bytes());
out.extend_from_slice(&[0; 20]); out.extend_from_slice(&[0; 20]);
let mut offset = directory_end; let mut offset = directory_end;
@@ -6,7 +6,7 @@ use crate::load_limits::IdDomain;
use crate::model::{ConstantPoolEntry, DebugInfo, Export, SyscallDecl}; use crate::model::{ConstantPoolEntry, DebugInfo, Export, SyscallDecl};
use std::cmp::Ordering; use std::cmp::Ordering;
/// Owned, bounded PBX v1 tables for the executable loader. This is not an /// Owned, bounded PBX v0 tables for the executable loader. This is not an
/// executable image: bindings and instruction/type verification are still due. /// executable image: bindings and instruction/type verification are still due.
#[derive(Debug)] #[derive(Debug)]
pub struct OwnedPbx { pub struct OwnedPbx {
@@ -1,5 +1,5 @@
//! Producer-side serialization of a complete typed module. This writer emits //! Producer-side serialization of a complete typed module. This writer emits
//! only v1; executable consumers still need bindings and code verification. //! only v0; executable consumers still need bindings and code verification.
use super::*; use super::*;
use crate::load_limits::IdDomain; use crate::load_limits::IdDomain;
use crate::model::{ConstantPoolEntry, DebugInfo, Export, SyscallDecl}; use crate::model::{ConstantPoolEntry, DebugInfo, Export, SyscallDecl};
@@ -86,7 +86,7 @@ impl DecodedInstr<'_> {
) )
} }
Op::GetCapture => { Op::GetCapture => {
return Err(LoadFailure::new(LoadCode::InvalidReference, LoadReason::CaptureIndex)) return Err(LoadFailure::new(LoadCode::InvalidReference, LoadReason::CaptureIndex));
} }
Op::CallTrait => { Op::CallTrait => {
let (ty, method) = self.imm_u32x2().map_err(|_| malformed())?; let (ty, method) = self.imm_u32x2().map_err(|_| malformed())?;
@@ -133,6 +133,29 @@ impl DecodedInstr<'_> {
}; };
(source_layout.storage.slots, pushes) (source_layout.storage.slots, pushes)
} }
Op::RefIdentityHash | Op::RefIdentityEq => {
let (left, right) = if self.opcode == Op::RefIdentityHash {
(self.imm_u32().map_err(|_| malformed())?, None)
} else {
let (a, b) = self.imm_u32x2().map_err(|_| malformed())?;
(a, Some(b))
};
let mut pops = 0;
for id in std::iter::once(left).chain(right) {
let (decl, layout) = type_info(id)?;
if !decl.kind.is_identity_reference() {
return Err(bad_category(id));
}
pops += layout.storage.slots;
}
(pops, 1)
}
Op::ArrayCopy => {
let array = self.imm_u32().map_err(|_| malformed())?;
let (_, layout) = type_info(array)?;
layout.array.ok_or_else(|| bad_category(array))?;
(5, 0)
}
Op::StructNew Op::StructNew
| Op::StructGet | Op::StructGet
| Op::StructSet | Op::StructSet
@@ -34,6 +34,8 @@ fn all_typed_instructions_match_independent_bytes_and_costs() {
("REF_IMPLEMENTS source=16909060, target=84281096", &[0x94, 0, 4, 3, 2, 1, 8, 7, 6, 5], 4), ("REF_IMPLEMENTS source=16909060, target=84281096", &[0x94, 0, 4, 3, 2, 1, 8, 7, 6, 5], 4),
("REF_TO_TRAIT source=16909060, target=84281096", &[0x95, 0, 4, 3, 2, 1, 8, 7, 6, 5], 5), ("REF_TO_TRAIT source=16909060, target=84281096", &[0x95, 0, 4, 3, 2, 1, 8, 7, 6, 5], 5),
("REF_CAST source=16909060, target=84281096", &[0x96, 0, 4, 3, 2, 1, 8, 7, 6, 5], 4), ("REF_CAST source=16909060, target=84281096", &[0x96, 0, 4, 3, 2, 1, 8, 7, 6, 5], 4),
("REF_IDENTITY_HASH 16909060", &[0x97, 0, 4, 3, 2, 1], 8),
("REF_IDENTITY_EQ left=16909060, right=84281096", &[0x98, 0, 4, 3, 2, 1, 8, 7, 6, 5], 4),
]; ];
for &(text, bytes, cycles) in cases { for &(text, bytes, cycles) in cases {
assert_eq!(assemble(text).unwrap(), bytes); assert_eq!(assemble(text).unwrap(), bytes);
@@ -49,8 +51,8 @@ fn all_typed_instructions_match_independent_bytes_and_costs() {
} }
} }
assert!(matches!( assert!(matches!(
decode_next(0, &[0x97, 0]), decode_next(0, &[0x99, 0]),
Err(DecodeError::UnknownOpcode { opcode: 0x97, .. }) Err(DecodeError::UnknownOpcode { opcode: 0x99, .. })
)); ));
} }
#[test] #[test]
@@ -68,6 +70,8 @@ fn slot_effects_resolve_semantic_widths_and_reference_categories() {
("REF_TO_TRAIT source=4, target=6", 1, 2), ("REF_TO_TRAIT source=4, target=6", 1, 2),
("REF_TO_TRAIT source=6, target=6", 2, 2), ("REF_TO_TRAIT source=6, target=6", 2, 2),
("REF_CAST source=6, target=4", 2, 1), ("REF_CAST source=6, target=4", 2, 1),
("REF_IDENTITY_HASH 6", 2, 1),
("REF_IDENTITY_EQ left=6, right=4", 3, 1),
] { ] {
let bytes = assemble(text).unwrap(); let bytes = assemble(text).unwrap();
assert_eq!( assert_eq!(
@@ -94,6 +98,9 @@ fn invalid_type_immediates_are_contextual_load_errors() {
"REF_IS_EXACT source=4, target=6", "REF_IS_EXACT source=4, target=6",
"REF_TO_TRAIT source=4, target=0", "REF_TO_TRAIT source=4, target=0",
"REF_IMPLEMENTS source=0, target=6", "REF_IMPLEMENTS source=0, target=6",
"REF_IDENTITY_HASH 0",
"REF_IDENTITY_HASH 3",
"REF_IDENTITY_EQ left=4, right=1",
] { ] {
let bytes = assemble(text).unwrap(); let bytes = assemble(text).unwrap();
let e = decode_next(0, &bytes).unwrap().registry_stack_effect(&registry).unwrap_err(); let e = decode_next(0, &bytes).unwrap().registry_stack_effect(&registry).unwrap_err();
@@ -114,6 +121,10 @@ fn assembler_rejects_missing_overflowing_or_ambiguous_type_ids() {
"REF_CAST source=4", "REF_CAST source=4",
"REF_CAST source=4, target=5, target=6", "REF_CAST source=4, target=5, target=6",
"REF_CAST source=4, trait=6", "REF_CAST source=4, trait=6",
"REF_IDENTITY_HASH",
"REF_IDENTITY_HASH -1",
"REF_IDENTITY_EQ left=4",
"REF_IDENTITY_EQ left=4, target=5",
] { ] {
assert!(assemble(text).is_err(), "{text}"); assert!(assemble(text).is_err(), "{text}");
} }
@@ -204,3 +215,62 @@ fn collection_effects_and_cached_initialization_width_are_exact() {
); );
} }
} }
#[test]
fn removed_storage_encodings_and_mnemonics_are_unsupported() {
for opcode in [0xa7u16, 0xa8, 0xaa, 0xab] {
assert!(Op::try_from(opcode).is_err());
let bytes = opcode.to_le_bytes();
assert!(
matches!(decode_next(0,&bytes),Err(DecodeError::UnknownOpcode{pc:0,opcode: actual}) if actual == opcode)
);
assert!(disassemble(&bytes).is_err());
}
for text in
["INDEX_GUARD", "ARRAY_CHECK_LENGTH 1", "STORAGE_PUBLISH 1,2,0,1", "FIELDS_PUBLISH 1"]
{
assert!(assemble(text).is_err(), "{text}");
}
}
#[test]
fn range_copy_encoding_effects_and_registry_categories_are_independent_oracles() {
let bytes = [0xa9, 0, 4, 3, 2, 1];
assert_eq!(assemble("ARRAY_COPY 16909060").unwrap(), bytes);
assert_eq!(disassemble(&bytes).unwrap(), "ARRAY_COPY 16909060");
let instruction = decode_next(0, &bytes).unwrap();
assert_eq!(instruction.opcode, Op::ArrayCopy);
assert_eq!(instruction.opcode.cycles(), 4);
let spec = instruction.opcode.spec();
assert_eq!((spec.imm_bytes, spec.pops, spec.pushes), (4, None, None));
assert!(instruction.opcode.requires_registry() && spec.may_trap);
assert!(!spec.is_safepoint && !spec.is_branch && !spec.is_terminator);
for end in 0..6 {
assert!(decode_next(0, &bytes[..end]).is_err());
}
let registry = registry();
let bytes = assemble("ARRAY_COPY 5").unwrap();
assert_eq!(
decode_next(0, &bytes).unwrap().registry_stack_effect(&registry).unwrap(),
Some(RegistryStackEffect { pops: 5, pushes: 0 })
);
for (id, reason) in [
(0, LoadReason::Category),
(4, LoadReason::Category),
(6, LoadReason::Category),
(99, LoadReason::TypeId),
] {
let bytes = assemble(&format!("ARRAY_COPY {id}")).unwrap();
let error = decode_next(0, &bytes).unwrap().registry_stack_effect(&registry).unwrap_err();
assert_eq!(
(error.code, error.reason, error.pc),
(LoadCode::InvalidReference, reason, Some(0))
);
}
for text in ["ARRAY_COPY", "ARRAY_COPY -1", "ARRAY_COPY 4294967296", "ARRAY_COPY 1 2"] {
assert!(assemble(text).is_err());
}
// These are legal immediate bytes, not instruction boundaries.
let bytes = assemble("PUSH_I64 18735214094965159\nHALT").unwrap();
assert!(disassemble(&bytes).is_ok());
}
@@ -172,7 +172,7 @@ fn hex(s: &str) -> Vec<u8> {
// syscalls=168/4, types=172/10, implementations=182/4, captures=186/8. Total=194. // syscalls=168/4, types=172/10, implementations=182/4, captures=186/8. Total=194.
fn minimal_bytes() -> Vec<u8> { fn minimal_bytes() -> Vec<u8> {
hex(" hex("
50 42 58 00 01 00 00 00 08 00 00 00 50 42 58 00 00 00 00 00 08 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 80 00 00 00 04 00 00 00 00 00 00 00 80 00 00 00 04 00 00 00
01 00 00 00 84 00 00 00 1e 00 00 00 01 00 00 00 84 00 00 00 1e 00 00 00
@@ -241,7 +241,7 @@ fn writer_matches_independent_minimal_envelope() {
} }
#[test] #[test]
fn owned_v1_decoder_accepts_independent_bytes_without_publishing_an_image() { fn owned_v0_decoder_accepts_independent_bytes_without_publishing_an_image() {
let mut b = budget(); let mut b = budget();
let owned = OwnedPbx::parse(&minimal_bytes(), &mut b).unwrap(); let owned = OwnedPbx::parse(&minimal_bytes(), &mut b).unwrap();
assert_eq!(owned.registry.declarations(), minimal_registry().declarations()); assert_eq!(owned.registry.declarations(), minimal_registry().declarations());
@@ -536,7 +536,7 @@ fn every_truncated_envelope_is_rejected_without_panic() {
fn envelope_mutations_have_typed_reasons() { fn envelope_mutations_have_typed_reasons() {
for (offset, value, code, reason) in [ for (offset, value, code, reason) in [
(0, 0, LoadCode::InvalidFormat, LoadReason::Magic), (0, 0, LoadCode::InvalidFormat, LoadReason::Magic),
(4, 0, LoadCode::UnsupportedFormat, LoadReason::Version), (4, 1, LoadCode::UnsupportedFormat, LoadReason::Version),
(6, 1, LoadCode::InvalidFormat, LoadReason::Endianness), (6, 1, LoadCode::InvalidFormat, LoadReason::Endianness),
(7, 1, LoadCode::InvalidFormat, LoadReason::Reserved), (7, 1, LoadCode::InvalidFormat, LoadReason::Reserved),
(31, 1, LoadCode::InvalidFormat, LoadReason::Reserved), (31, 1, LoadCode::InvalidFormat, LoadReason::Reserved),
@@ -0,0 +1,256 @@
//! Repeatable CPU/memory observations, not a handheld or physical-device guarantee.
use prometeu_drivers::{
Audio, MemoryBanks, SoundBankPoolInstaller,
bank_format::{decode_audio_bank, encode_ima_wav4},
engine::{AudioMixer, Packet},
};
use prometeu_hal::{
AudioBridge, LoopMode,
audio_bridge::AudioControl,
music_bank::{Instrument, MusicBank, MusicEvent},
sample::Sample,
sound_bank::SoundBank,
};
use serde_json::{Value, json};
use std::{
alloc::{GlobalAlloc, Layout, System},
sync::{
Arc,
atomic::{AtomicUsize, Ordering},
},
time::Instant,
};
struct Counter;
static LIVE: AtomicUsize = AtomicUsize::new(0);
static PEAK: AtomicUsize = AtomicUsize::new(0);
unsafe impl GlobalAlloc for Counter {
unsafe fn alloc(&self, l: Layout) -> *mut u8 {
let p = unsafe { System.alloc(l) };
if !p.is_null() {
let n = LIVE.fetch_add(l.size(), Ordering::Relaxed) + l.size();
PEAK.fetch_max(n, Ordering::Relaxed);
}
p
}
unsafe fn dealloc(&self, p: *mut u8, l: Layout) {
LIVE.fetch_sub(l.size(), Ordering::Relaxed);
unsafe { System.dealloc(p, l) }
}
}
#[global_allocator]
static ALLOCATOR: Counter = Counter;
fn stats(values: &mut [u64]) -> Value {
values.sort_unstable();
let n = values.len();
json!({"p50_ns":values[n/2],"p95_ns":values[n*95/100],"p99_ns":values[n*99/100],"max_ns":values[n-1]})
}
fn music(channels: u8) -> Arc<MusicBank> {
let pcm = (0..4096)
.flat_map(|i| {
std::iter::repeat_n(((i as f64 * 0.05).sin() * 800.0) as i16, channels as usize)
})
.collect();
let sample =
Arc::new(Sample::new(48000, pcm).with_channels(channels).unwrap().with_loop(0, 4096));
let events = (0..12)
.map(|id| MusicEvent { kind: 1, a: id, c: 60 + id as i32, d: 180, ..Default::default() })
.collect();
Arc::new(
MusicBank::compile(
"measure".into(),
vec![sample],
vec![Instrument {
looping: true,
attack_us: 1000,
decay_us: 10_000,
sustain: 50000,
release_us: 10_000,
send: 20000,
vibrato_cents: 10,
vibrato_millihz: 5000,
tremolo_depth: 1000,
tremolo_millihz: 3000,
..Default::default()
}],
events,
960,
500000,
3840,
None,
1,
)
.unwrap(),
)
}
fn scenario(channels: u8, dsp: bool, replacement: bool) -> Value {
let banks = Arc::new(MemoryBanks::new());
let mut audio = Audio::new(banks.clone());
banks.install_music_bank(0, music(channels));
banks.install_music_bank(1, music(channels));
audio.music_play(0, 0, true);
audio.music_play(1, 1, true);
let sample = Arc::new(
Sample::new(48000, vec![500; 4096 * channels as usize])
.with_channels(channels)
.unwrap()
.with_loop(0, 4096),
);
for voice in 0..8 {
audio.play_sample(sample.clone(), voice, 255, 128, 1.0, 0, LoopMode::On);
audio.control(AudioControl::SfxSend { voice, gain: 100 });
}
if dsp {
for group in 0..3 {
audio.control(AudioControl::Effect {
group,
cutoff_hz: 6000,
delay_frames: 4800,
feedback: 0.5,
wet: 0.3,
dry: 0.7,
});
}
}
audio.finish_frame();
let mut mixer = AudioMixer::new();
let mut packet = Packet::new();
mixer.apply_delivery_into(audio.transport().try_take().unwrap(), &mut packet);
packet.recycle(&audio.transport());
let mut pcm = [0.0; 512];
let mut timings = Vec::with_capacity(1000);
let mut overruns = 0;
let mut max_retired = 0;
let baseline = LIVE.load(Ordering::Relaxed);
PEAK.store(baseline, Ordering::Relaxed);
for iteration in 0..1000 {
if replacement && iteration % 10 == 0 {
banks.install_music_bank(0, music(channels));
audio.music_play(0, 0, true);
audio.control(AudioControl::MusicFade {
group: 1,
gain: 180,
frames: 2400,
smooth: true,
});
audio.finish_frame();
}
let start = Instant::now();
mixer.retire_inactive(&mut packet);
if let Some(delivery) = audio.transport().try_take() {
mixer.apply_delivery_into(delivery, &mut packet);
}
mixer.mix(&mut pcm);
mixer.retire_inactive(&mut packet);
let elapsed = start.elapsed().as_nanos() as u64;
if elapsed > 5_333_333 {
overruns += 1;
}
timings.push(elapsed);
max_retired = max_retired.max(packet.retained_pcm_bytes());
packet.recycle(&audio.transport());
std::hint::black_box(&pcm);
}
json!({"channels_per_voice":channels,"dsp":dsp,"replacement_every_10_buffers":replacement,"buffers":1000,"frames_per_buffer":256,"nominal_buffer_ns":5_333_333,"observed_budget_overruns":overruns,"timing":stats(&mut timings),"maximum_active_voices":mixer.diagnostics.maximum_voices,"engine_prepared_bytes":mixer.prepared_bytes(),"maximum_retired_pcm_bytes":max_retired,"baseline_allocated_bytes":baseline,"peak_allocated_bytes":PEAK.load(Ordering::Relaxed)})
}
fn codec_and_memory() -> Value {
let pcm: Vec<i16> = (0..480_000)
.flat_map(|i| {
[((i as f64 * 0.04).sin() * 12000.0) as i16, ((i as f64 * 0.06).cos() * 9000.0) as i16]
})
.collect();
let start = Instant::now();
let compressed = encode_ima_wav4(&pcm, 2, 512).unwrap();
let encode_ns = start.elapsed().as_nanos();
let mut bytes = vec![0u8; 88];
bytes[..4].copy_from_slice(b"SBNK");
bytes[4] = 1;
bytes[48] = 1;
bytes[49] = 2;
bytes[80] = 255;
bytes[81] = 128;
for (at, n) in [
(8, 1u32),
(20, 960),
(24, 500000),
(32, u32::MAX),
(52, 48000),
(56, 480000),
(68, 88),
(72, compressed.len() as u32),
(84, 65536),
] {
bytes[at..at + 4].copy_from_slice(&n.to_le_bytes());
}
bytes[76..78].copy_from_slice(&512u16.to_le_bytes());
bytes[78..80].copy_from_slice(&505u16.to_le_bytes());
bytes.extend(&compressed);
let mut decode = Vec::with_capacity(20);
let baseline = LIVE.load(Ordering::Relaxed);
PEAK.store(baseline, Ordering::Relaxed);
for _ in 0..20 {
let start = Instant::now();
let bank = decode_audio_bank(&bytes, false).unwrap();
decode.push(start.elapsed().as_nanos() as u64);
std::hint::black_box(bank);
}
let decode_peak = PEAK.load(Ordering::Relaxed);
let banks = MemoryBanks::new();
for slot in 0..16 {
banks.install_sound_bank(
slot,
Arc::new(SoundBank::new(vec![Arc::new(Sample::new(48000, vec![0; 1_048_576]))])),
);
}
let large_music = || {
let sample = Arc::new(Sample::new(48000, vec![0; 1_048_576]).with_loop(0, 1_048_576));
Arc::new(
MusicBank::compile(
"large".into(),
vec![sample],
vec![Instrument { looping: true, ..Default::default() }],
vec![MusicEvent { kind: 1, a: 1, c: 60, d: 255, ..Default::default() }],
960,
500000,
3840,
None,
1,
)
.unwrap(),
)
};
for slot in 0..16 {
banks.install_music_bank(slot, large_music());
}
let snapshot = banks.audio_memory();
let mut mixer = AudioMixer::new();
let mut packet = Packet::new();
use prometeu_drivers::SoundBankPoolAccess;
mixer.apply_delivery_into(
prometeu_drivers::AudioDelivery {
epoch: 0,
paused: false,
frame: Some(vec![prometeu_drivers::AudioCommand::MusicPlay {
group: 0,
bank: banks.music_bank_slot(0).unwrap(),
looping: true,
}]),
lifecycle: None,
},
&mut packet,
);
packet.recycle(&prometeu_drivers::AudioTransport::default());
mixer.mix(&mut [0.0; 2]);
let before = LIVE.load(Ordering::Relaxed);
PEAK.store(before, Ordering::Relaxed);
banks.install_music_bank(0, large_music());
mixer.retire_inactive(&mut packet);
let retired = packet.retained_pcm_bytes();
let replacement_peak = PEAK.load(Ordering::Relaxed);
packet.recycle(&prometeu_drivers::AudioTransport::default());
let after = LIVE.load(Ordering::Relaxed);
json!({"source_frames":480000,"channels":2,"pcm_bytes":pcm.len()*2,"cartridge_sample_bytes":compressed.len(),"compression_ratio":compressed.len()as f64/(pcm.len()*2)as f64,"encode_ns":encode_ns,"decode_timing":stats(&mut decode),"decode_baseline_allocated_bytes":baseline,"decode_peak_allocated_bytes":decode_peak,"large_bank_count":snapshot.banks.len(),"large_banks_unique_pcm_bytes":snapshot.unique_pcm_bytes,"replacement_baseline_allocated_bytes":before,"replacement_peak_allocated_bytes":replacement_peak,"replacement_after_reclamation_bytes":after,"retained_replaced_pcm_bytes":retired,"resident_accounted_metadata_bytes":snapshot.metadata_bytes,"resident_total_accounted_bytes":snapshot.total_accounted_bytes})
}
fn main() {
println!("{}",serde_json::to_string_pretty(&json!({"arch":std::env::consts::ARCH,"os":std::env::consts::OS,"observations":[scenario(1,false,false),scenario(2,false,false),scenario(2,true,false),scenario(2,true,true)],"codecs_and_memory":codec_and_memory()})).unwrap());
}
@@ -0,0 +1,185 @@
//! Controlled graphics measurement. This is not a cartridge stress-suite replacement.
use prometeu_drivers::{Gfx, GlyphBankPoolInstaller, MemoryBanks, hardware::Hardware};
use prometeu_hal::color::Color;
use prometeu_hal::glyph::Glyph;
use prometeu_hal::glyph_bank::{GlyphBank, TileSize};
use prometeu_hal::scene_bank::SceneBank;
use prometeu_hal::scene_layer::{ParallaxFactor, SceneLayer};
use prometeu_hal::scene_viewport_cache::SceneViewportCache;
use prometeu_hal::scene_viewport_resolver::SceneViewportResolver;
use prometeu_hal::sprite::Sprite;
use prometeu_hal::tile::Tile;
use prometeu_hal::tilemap::TileMap;
use prometeu_hal::{FrameId, OwnedRgba8888Frame, RenderOwnership, app_mode::AppMode};
use std::alloc::{GlobalAlloc, Layout, System};
use std::sync::{
Arc,
atomic::{AtomicUsize, Ordering},
};
use std::time::Instant;
struct CountingAllocator;
static LIVE: AtomicUsize = AtomicUsize::new(0);
static PEAK: AtomicUsize = AtomicUsize::new(0);
static ALLOCATIONS: AtomicUsize = AtomicUsize::new(0);
// Only the measurement executable pays the atomic allocation instrumentation cost.
unsafe impl GlobalAlloc for CountingAllocator {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
let ptr = unsafe { System.alloc(layout) };
if !ptr.is_null() {
let live = LIVE.fetch_add(layout.size(), Ordering::Relaxed) + layout.size();
PEAK.fetch_max(live, Ordering::Relaxed);
ALLOCATIONS.fetch_add(1, Ordering::Relaxed);
}
ptr
}
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
LIVE.fetch_sub(layout.size(), Ordering::Relaxed);
unsafe { System.dealloc(ptr, layout) };
}
}
#[global_allocator]
static ALLOCATOR: CountingAllocator = CountingAllocator;
fn summary(samples: &mut [u64]) -> serde_json::Value {
samples.sort_unstable();
let n = samples.len();
serde_json::json!({"mean_us": samples.iter().sum::<u64>() as f64 / n as f64 / 1000.0,
"p95_us": samples[(n * 95).div_ceil(100) - 1] as f64 / 1000.0,
"p99_us": samples[(n * 99).div_ceil(100) - 1] as f64 / 1000.0,
"max_us": samples[n-1] as f64 / 1000.0,
"over_60hz_budget": samples.iter().filter(|&&v| v > 1_000_000_000 / 60).count()})
}
fn main() {
let count: usize =
std::env::args().nth(1).map(|v| v.parse().expect("frame count")).unwrap_or(600);
assert!(count > 0);
let (w, h) = (Hardware::W, Hardware::H);
let banks = Arc::new(MemoryBanks::new());
let mixed = std::env::args().nth(2).as_deref() == Some("mixed");
let sizes = if mixed {
[TileSize::Size8, TileSize::Size16, TileSize::Size32, TileSize::Size8]
} else {
[TileSize::Size16; 4]
};
for (i, size) in sizes.iter().enumerate() {
let edge = *size as usize;
let mut bank = GlyphBank::new(*size, edge, edge);
bank.palettes[0][1] = Color::rgba((i * 47 + 30) as u8, 120, 210, 255);
// Transparent holes make all four layers participate in the visible output.
for (p, pixel) in bank.pixel_indices.iter_mut().enumerate() {
*pixel = u8::from(i == 0 || (p + i) % 3 != 0);
}
banks.install_glyph_bank(i, Arc::new(bank));
}
let scene = SceneBank {
layers: std::array::from_fn(|i| SceneLayer {
active: true,
glyph_asset_id: i as i32,
tile_size: sizes[i],
parallax_factor: ParallaxFactor { x: 1.0, y: 1.0 },
tilemap: TileMap {
width: 160,
height: 100,
tiles: vec![
Tile {
active: true,
glyph: Glyph { glyph_id: 0, palette_id: 0 },
flip_x: false,
flip_y: false,
};
160 * 100
],
},
}),
};
let min_tile = if mixed { 8 } else { 16 };
let (cw, ch) = (w.div_ceil(min_tile) + 5, h.div_ceil(min_tile) + 4);
let mut cache = SceneViewportCache::new(&scene, cw, ch);
let mut resolver = SceneViewportResolver::new(
w as i32,
h as i32,
cw,
ch,
min_tile as i32 - 4,
min_tile as i32 + 4,
);
let mut gfx = Gfx::new(w, h, banks);
let sprites: Vec<_> = (0..512)
.map(|i| Sprite {
glyph: Glyph { glyph_id: 0, palette_id: 0 },
x: ((i * 37) % w) as i32,
y: ((i * 53) % h) as i32,
layer: (i % 4) as u8,
bank_id: (i % 4) as u8,
active: true,
flip_x: i % 2 == 0,
flip_y: i % 3 == 0,
priority: (i % 4) as u8,
})
.collect();
gfx.load_frame_sprites(&sprites);
let mut compose = Vec::with_capacity(count);
let mut publish = Vec::with_capacity(count);
let mut upload = Vec::with_capacity(count);
let mut total = Vec::with_capacity(count);
let mut rgba = vec![0u8; w * h * 4];
let ownership = RenderOwnership::new(1, AppMode::Game, 1);
let mut checksum = 0u32;
for frame_id in 0..(count + 120) {
// Resolve/materialize cache outside the raster timing; total includes it.
let start = Instant::now();
let update = resolver.update(&scene, (frame_id % 71) as i32, (frame_id % 43) as i32);
for request in &update.copy_requests {
cache.move_layer_window_to(
request.layer_index,
request.cache_origin_tile_x,
request.cache_origin_tile_y,
);
}
cache.materialize_all_layers(&scene);
let t = Instant::now();
gfx.render_scene_from_cache(&cache, &update, &[0, 1, 2, 3]);
gfx.fill_rect(0, h as i32 - 8, w as i32, 8, Color::WHITE);
let raster = t.elapsed().as_nanos() as u64;
let t = Instant::now();
gfx.present();
let frame = OwnedRgba8888Frame::packed(
FrameId::new(frame_id as u64),
ownership,
w,
h,
gfx.front_buffer().to_vec(),
)
.unwrap();
let publication = t.elapsed().as_nanos() as u64;
let t = Instant::now();
for (pixel, bytes) in frame.pixels.iter().zip(rgba.as_chunks_mut::<4>().0.iter_mut()) {
bytes.copy_from_slice(&pixel.to_be_bytes());
}
std::hint::black_box(&rgba);
let staging = t.elapsed().as_nanos() as u64;
checksum ^= frame.pixels[(frame_id * 173) % (w * h)];
if frame_id >= 120 {
compose.push(raster);
publish.push(publication);
upload.push(staging);
total.push(start.elapsed().as_nanos() as u64);
}
}
let live = LIVE.load(Ordering::Relaxed);
let peak = PEAK.load(Ordering::Relaxed);
let allocations = ALLOCATIONS.load(Ordering::Relaxed);
println!(
"{}",
serde_json::json!({"schema":1,"width":w,"height":h,"frames":count,
"warmup_frames":120,"tiles":sizes.map(|s|s as u32),"sprites_accepted":512,"layers":4,
"framebuffer_bytes":w*h*4,"front_back_bytes":w*h*8,"host_staging_bytes":rgba.len(),
"compose":summary(&mut compose),"publish_owned":summary(&mut publish),
"rgba_staging":summary(&mut upload),"total":summary(&mut total),
"allocated_live_bytes":live,"allocated_peak_bytes":peak,"allocation_count":allocations,
"checksum":checksum,"vm_hz":null,"physical_display_hz":null,
"limitations":"unpaced controlled raster/cache/owned publication/staging; no VM, IPC or GPU present; allocator instrumented; RSS is measured by the process wrapper"})
);
}
+265 -231
View File
@@ -1,5 +1,7 @@
#![allow(clippy::collapsible_if)] #![allow(clippy::collapsible_if)]
mod decode;
mod resumable; mod resumable;
use crate::audio::bank_format::{DecodedAudioBank, decode_audio_bank};
use crate::memory_banks::{GlyphBankPoolInstaller, SceneBankPoolInstaller, SoundBankPoolInstaller}; use crate::memory_banks::{GlyphBankPoolInstaller, SceneBankPoolInstaller, SoundBankPoolInstaller};
use prometeu_hal::AssetBridge; use prometeu_hal::AssetBridge;
use prometeu_hal::asset::{ use prometeu_hal::asset::{
@@ -13,7 +15,7 @@ use prometeu_hal::cartridge::AssetsPayloadSource;
use prometeu_hal::color::Color; use prometeu_hal::color::Color;
use prometeu_hal::glyph::Glyph; use prometeu_hal::glyph::Glyph;
use prometeu_hal::glyph_bank::{GlyphBank, TileSize}; use prometeu_hal::glyph_bank::{GlyphBank, TileSize};
use prometeu_hal::sample::Sample; use prometeu_hal::music_bank::MusicBank;
use prometeu_hal::scene_bank::SceneBank; use prometeu_hal::scene_bank::SceneBank;
use prometeu_hal::scene_layer::{ParallaxFactor, SceneLayer}; use prometeu_hal::scene_layer::{ParallaxFactor, SceneLayer};
use prometeu_hal::sound_bank::SoundBank; use prometeu_hal::sound_bank::SoundBank;
@@ -60,6 +62,7 @@ pub struct AssetPipelineTelemetry {
pub active_progress: u16, pub active_progress: u16,
pub glyph_latency: AssetLatencyPercentiles, pub glyph_latency: AssetLatencyPercentiles,
pub sound_latency: AssetLatencyPercentiles, pub sound_latency: AssetLatencyPercentiles,
pub music_latency: AssetLatencyPercentiles,
pub scene_latency: AssetLatencyPercentiles, pub scene_latency: AssetLatencyPercentiles,
pub last_asset_id: Option<AssetId>, pub last_asset_id: Option<AssetId>,
pub last_asset_latency: AssetLatencyPercentiles, pub last_asset_latency: AssetLatencyPercentiles,
@@ -129,6 +132,7 @@ impl AssetPipelineTelemetryState {
); );
self.snapshot.glyph_latency = percentile_snapshot(self.bank_samples.get(&BankType::GLYPH)); self.snapshot.glyph_latency = percentile_snapshot(self.bank_samples.get(&BankType::GLYPH));
self.snapshot.sound_latency = percentile_snapshot(self.bank_samples.get(&BankType::SOUNDS)); self.snapshot.sound_latency = percentile_snapshot(self.bank_samples.get(&BankType::SOUNDS));
self.snapshot.music_latency = percentile_snapshot(self.bank_samples.get(&BankType::MUSIC));
self.snapshot.scene_latency = percentile_snapshot(self.bank_samples.get(&BankType::SCENE)); self.snapshot.scene_latency = percentile_snapshot(self.bank_samples.get(&BankType::SCENE));
self.snapshot.last_asset_id = Some(asset_id); self.snapshot.last_asset_id = Some(asset_id);
self.snapshot.last_asset_latency = percentile_snapshot(self.asset_samples.get(&asset_id)); self.snapshot.last_asset_latency = percentile_snapshot(self.asset_samples.get(&asset_id));
@@ -328,12 +332,14 @@ pub struct AssetManager {
gfx_slots: Arc<RwLock<[Option<AssetId>; 16]>>, gfx_slots: Arc<RwLock<[Option<AssetId>; 16]>>,
glyph_slot_index: GlyphAssetSlotIndex, glyph_slot_index: GlyphAssetSlotIndex,
sound_slots: Arc<RwLock<[Option<AssetId>; 16]>>, sound_slots: Arc<RwLock<[Option<AssetId>; 16]>>,
music_slots: Arc<RwLock<[Option<AssetId>; 16]>>,
scene_slots: Arc<RwLock<[Option<AssetId>; 16]>>, scene_slots: Arc<RwLock<[Option<AssetId>; 16]>>,
/// Residency policy for GFX glyph banks. /// Residency policy for GFX glyph banks.
gfx_policy: BankPolicy<GlyphBank>, gfx_policy: BankPolicy<GlyphBank>,
/// Residency policy for sound banks. /// Residency policy for sound banks.
sound_policy: BankPolicy<SoundBank>, sound_policy: BankPolicy<SoundBank>,
music_policy: BankPolicy<MusicBank>,
/// Residency policy for scene banks. /// Residency policy for scene banks.
scene_policy: BankPolicy<SceneBank>, scene_policy: BankPolicy<SceneBank>,
@@ -369,6 +375,7 @@ struct AssetLoadJobResources<'a> {
pipeline_telemetry: &'a Arc<Mutex<AssetPipelineTelemetryState>>, pipeline_telemetry: &'a Arc<Mutex<AssetPipelineTelemetryState>>,
gfx_policy: &'a BankPolicy<GlyphBank>, gfx_policy: &'a BankPolicy<GlyphBank>,
sound_policy: &'a BankPolicy<SoundBank>, sound_policy: &'a BankPolicy<SoundBank>,
music_policy: &'a BankPolicy<MusicBank>,
scene_policy: &'a BankPolicy<SceneBank>, scene_policy: &'a BankPolicy<SceneBank>,
} }
@@ -478,6 +485,7 @@ struct AssetLoadWorker {
} }
impl AssetLoadWorker { impl AssetLoadWorker {
#[allow(clippy::too_many_arguments)]
fn new( fn new(
handles: Arc<RwLock<HandleTable>>, handles: Arc<RwLock<HandleTable>>,
target_generations: Arc<RwLock<TargetGenerationTable>>, target_generations: Arc<RwLock<TargetGenerationTable>>,
@@ -485,6 +493,7 @@ impl AssetLoadWorker {
pipeline_telemetry: Arc<Mutex<AssetPipelineTelemetryState>>, pipeline_telemetry: Arc<Mutex<AssetPipelineTelemetryState>>,
gfx_policy: BankPolicy<GlyphBank>, gfx_policy: BankPolicy<GlyphBank>,
sound_policy: BankPolicy<SoundBank>, sound_policy: BankPolicy<SoundBank>,
music_policy: BankPolicy<MusicBank>,
scene_policy: BankPolicy<SceneBank>, scene_policy: BankPolicy<SceneBank>,
) -> Self { ) -> Self {
let queue = Arc::new(AssetLoadQueue::default()); let queue = Arc::new(AssetLoadQueue::default());
@@ -497,6 +506,7 @@ impl AssetLoadWorker {
pipeline_telemetry: &pipeline_telemetry, pipeline_telemetry: &pipeline_telemetry,
gfx_policy: &gfx_policy, gfx_policy: &gfx_policy,
sound_policy: &sound_policy, sound_policy: &sound_policy,
music_policy: &music_policy,
scene_policy: &scene_policy, scene_policy: &scene_policy,
}; };
while let Some(job) = worker_queue.next_job() { while let Some(job) = worker_queue.next_job() {
@@ -553,6 +563,9 @@ enum AssetOpMode {
} }
impl AssetBridge for AssetManager { impl AssetBridge for AssetManager {
fn unload_audio(&self, music: bool, slot: usize) -> AssetOpStatus {
self.unload_audio(music, slot)
}
fn bind_background_work( fn bind_background_work(
&mut self, &mut self,
lane: Arc<dyn prometeu_hal::background_work::BackgroundWork>, lane: Arc<dyn prometeu_hal::background_work::BackgroundWork>,
@@ -634,68 +647,21 @@ impl AssetManager {
fn decode_glyph_bank_layout( fn decode_glyph_bank_layout(
entry: &AssetEntry, entry: &AssetEntry,
) -> Result<(TileSize, usize, usize, usize, usize, usize), String> { ) -> Result<(TileSize, usize, usize, usize, usize, usize), String> {
let meta = entry.metadata_as_glyph_bank()?; decode::glyph_layout(entry)
let tile_size = match meta.tile_size {
8 => TileSize::Size8,
16 => TileSize::Size16,
32 => TileSize::Size32,
_ => return Err(format!("Invalid tile_size: {}", meta.tile_size)),
};
let palette_count = meta.palette_count as usize;
if !(1..=GLYPH_BANK_MAX_PALETTE_COUNT_V1).contains(&palette_count) {
return Err(format!("Invalid palette_count: {}", meta.palette_count));
}
let width = meta.width as usize;
let height = meta.height as usize;
let logical_pixels = width.checked_mul(height).ok_or("GlyphBank dimensions overflow")?;
let serialized_pixel_bytes = logical_pixels.div_ceil(2);
let palette_bytes = palette_count
.checked_mul(GLYPH_BANK_PALETTE_BYTES_PER_PALETTE)
.ok_or("GlyphBank palette size overflow")?;
let serialized_size = serialized_pixel_bytes
.checked_add(palette_bytes)
.ok_or("GlyphBank serialized size overflow")?;
let decoded_size =
logical_pixels.checked_add(palette_bytes).ok_or("GlyphBank decoded size overflow")?;
if entry.size != serialized_size as u64 {
return Err(format!(
"Invalid GLYPHBANK serialized size: expected {}, got {}",
serialized_size, entry.size
));
}
if entry.decoded_size != decoded_size as u64 {
return Err(format!(
"Invalid GLYPHBANK decoded_size: expected {}, got {}",
decoded_size, entry.decoded_size
));
}
Ok((tile_size, width, height, serialized_pixel_bytes, palette_count, palette_bytes))
} }
fn unpack_glyph_bank_pixels(packed_pixels: &[u8], logical_pixels: usize) -> Vec<u8> { fn unpack_glyph_bank_pixels(packed_pixels: &[u8], logical_pixels: usize) -> Vec<u8> {
let mut pixel_indices = Vec::with_capacity(logical_pixels); let mut pixels = Vec::with_capacity(logical_pixels);
for &packed in packed_pixels { decode::append_pixels(&mut pixels, packed_pixels, logical_pixels);
if pixel_indices.len() < logical_pixels { pixels
pixel_indices.push(packed >> 4);
}
if pixel_indices.len() < logical_pixels {
pixel_indices.push(packed & 0x0f);
}
}
pixel_indices
} }
fn op_mode_for(entry: &AssetEntry) -> Result<AssetOpMode, String> { fn op_mode_for(entry: &AssetEntry) -> Result<AssetOpMode, String> {
match (entry.bank_type, entry.codec) { match (entry.bank_type, entry.codec) {
(BankType::GLYPH, AssetCodec::None) => Ok(AssetOpMode::StageInMemory), (BankType::GLYPH, AssetCodec::None) => Ok(AssetOpMode::StageInMemory),
(BankType::SOUNDS, AssetCodec::None) => Ok(AssetOpMode::DirectFromSlice), (BankType::SOUNDS, AssetCodec::None) | (BankType::MUSIC, AssetCodec::None) => {
Ok(AssetOpMode::DirectFromSlice)
}
(BankType::SCENE, AssetCodec::None) => Ok(AssetOpMode::DirectFromSlice), (BankType::SCENE, AssetCodec::None) => Ok(AssetOpMode::DirectFromSlice),
} }
} }
@@ -718,6 +684,7 @@ impl AssetManager {
let pipeline_telemetry = Arc::new(Mutex::new(AssetPipelineTelemetryState::default())); let pipeline_telemetry = Arc::new(Mutex::new(AssetPipelineTelemetryState::default()));
let gfx_policy = BankPolicy::new(); let gfx_policy = BankPolicy::new();
let sound_policy = BankPolicy::new(); let sound_policy = BankPolicy::new();
let music_policy = BankPolicy::new();
let scene_policy = BankPolicy::new(); let scene_policy = BankPolicy::new();
let load_worker = AssetLoadWorker::new( let load_worker = AssetLoadWorker::new(
Arc::clone(&handles), Arc::clone(&handles),
@@ -726,6 +693,7 @@ impl AssetManager {
Arc::clone(&pipeline_telemetry), Arc::clone(&pipeline_telemetry),
gfx_policy.clone(), gfx_policy.clone(),
sound_policy.clone(), sound_policy.clone(),
music_policy.clone(),
scene_policy.clone(), scene_policy.clone(),
); );
@@ -737,9 +705,11 @@ impl AssetManager {
gfx_slots: Arc::new(RwLock::new(std::array::from_fn(|_| None))), gfx_slots: Arc::new(RwLock::new(std::array::from_fn(|_| None))),
glyph_slot_index: GlyphAssetSlotIndex::new(), glyph_slot_index: GlyphAssetSlotIndex::new(),
sound_slots: Arc::new(RwLock::new(std::array::from_fn(|_| None))), sound_slots: Arc::new(RwLock::new(std::array::from_fn(|_| None))),
music_slots: Arc::new(RwLock::new(std::array::from_fn(|_| None))),
scene_slots: Arc::new(RwLock::new(std::array::from_fn(|_| None))), scene_slots: Arc::new(RwLock::new(std::array::from_fn(|_| None))),
gfx_policy, gfx_policy,
sound_policy, sound_policy,
music_policy,
scene_policy, scene_policy,
handles, handles,
target_handles: Arc::new(RwLock::new(HashMap::new())), target_handles: Arc::new(RwLock::new(HashMap::new())),
@@ -824,6 +794,23 @@ impl AssetManager {
} }
} }
} }
BankType::MUSIC => {
if let Ok(bank) =
Self::perform_load_music_bank(&entry, self.assets_data.clone())
{
let bank_arc = Arc::new(bank);
self.music_policy.put_resident(
entry.asset_id,
Arc::clone(&bank_arc),
entry.decoded_size as usize,
);
self.sound_installer.install_music_bank(slot_index, bank_arc);
let mut slots = self.music_slots.write().unwrap();
if slot_index < slots.len() {
slots[slot_index] = Some(entry.asset_id);
}
}
}
BankType::SCENE => { BankType::SCENE => {
if let Ok(bank) = if let Ok(bank) =
Self::perform_load_scene_bank(&entry, self.assets_data.clone()) Self::perform_load_scene_bank(&entry, self.assets_data.clone())
@@ -888,6 +875,7 @@ impl AssetManager {
fn clear_staging_for_handle(&self, handle_id: HandleId) { fn clear_staging_for_handle(&self, handle_id: HandleId) {
self.gfx_policy.take_staging(handle_id); self.gfx_policy.take_staging(handle_id);
self.sound_policy.take_staging(handle_id); self.sound_policy.take_staging(handle_id);
self.music_policy.take_staging(handle_id);
self.scene_policy.take_staging(handle_id); self.scene_policy.take_staging(handle_id);
} }
@@ -903,6 +891,7 @@ impl AssetManager {
match slot.asset_type { match slot.asset_type {
BankType::GLYPH => self.gfx_slots.read().unwrap().get(slot.index).and_then(|s| *s), BankType::GLYPH => self.gfx_slots.read().unwrap().get(slot.index).and_then(|s| *s),
BankType::SOUNDS => self.sound_slots.read().unwrap().get(slot.index).and_then(|s| *s), BankType::SOUNDS => self.sound_slots.read().unwrap().get(slot.index).and_then(|s| *s),
BankType::MUSIC => self.music_slots.read().unwrap().get(slot.index).and_then(|s| *s),
BankType::SCENE => self.scene_slots.read().unwrap().get(slot.index).and_then(|s| *s), BankType::SCENE => self.scene_slots.read().unwrap().get(slot.index).and_then(|s| *s),
} }
} }
@@ -922,6 +911,7 @@ impl AssetManager {
let slot = match entry.bank_type { let slot = match entry.bank_type {
BankType::GLYPH => SlotRef::gfx(slot_index), BankType::GLYPH => SlotRef::gfx(slot_index),
BankType::SOUNDS => SlotRef::audio(slot_index), BankType::SOUNDS => SlotRef::audio(slot_index),
BankType::MUSIC => SlotRef::music(slot_index),
BankType::SCENE => SlotRef::scene(slot_index), BankType::SCENE => SlotRef::scene(slot_index),
}; };
@@ -947,6 +937,14 @@ impl AssetManager {
false false
} }
} }
BankType::MUSIC => {
if let Some(bank) = self.music_policy.get_resident(asset_id) {
self.music_policy.stage(handle_id, bank, entry.decoded_size as usize);
true
} else {
false
}
}
BankType::SCENE => { BankType::SCENE => {
if let Some(bank) = self.scene_policy.get_resident(asset_id) { if let Some(bank) = self.scene_policy.get_resident(asset_id) {
self.scene_policy.stage(handle_id, bank, entry.decoded_size as usize); self.scene_policy.stage(handle_id, bank, entry.decoded_size as usize);
@@ -1129,6 +1127,57 @@ impl AssetManager {
} }
} }
} }
BankType::MUSIC => {
let result =
Self::perform_load_music_bank(&job.entry, Arc::clone(resources.assets_data));
match result {
Ok(musicbank) => {
if !Self::is_current_target_generation(
resources.target_generations,
job.slot,
job.request_generation,
) {
resources.pipeline_telemetry.lock().unwrap().record_stale_discard();
return;
}
let bank_arc = Arc::new(musicbank);
let resident_arc = resources.music_policy.put_resident(
job.asset_id,
bank_arc,
job.entry.decoded_size as usize,
);
if !Self::is_current_target_generation(
resources.target_generations,
job.slot,
job.request_generation,
) {
resources.pipeline_telemetry.lock().unwrap().record_stale_discard();
return;
}
resources.music_policy.stage(
job.handle_id,
resident_arc,
job.entry.decoded_size as usize,
);
Self::complete_load_job(resources.handles, &job, LoadStatus::READY);
Self::record_load_job_finished(
resources.pipeline_telemetry,
&job,
started_at,
LoadStatus::READY,
);
}
Err(_) => {
Self::complete_load_job(resources.handles, &job, LoadStatus::ERROR);
Self::record_load_job_finished(
resources.pipeline_telemetry,
&job,
started_at,
LoadStatus::ERROR,
);
}
}
}
BankType::SCENE => { BankType::SCENE => {
let result = let result =
Self::perform_load_scene_bank(&job.entry, Arc::clone(resources.assets_data)); Self::perform_load_scene_bank(&job.entry, Arc::clone(resources.assets_data));
@@ -1260,17 +1309,7 @@ impl AssetManager {
let palette_data = &buffer[packed_pixel_bytes..packed_pixel_bytes + palette_bytes]; let palette_data = &buffer[packed_pixel_bytes..packed_pixel_bytes + palette_bytes];
let mut palettes = vec![[Color::BLACK; GLYPH_BANK_COLORS_PER_PALETTE]; palette_count]; let mut palettes = vec![[Color::BLACK; GLYPH_BANK_COLORS_PER_PALETTE]; palette_count];
for (p, pal) in palettes.iter_mut().enumerate() { decode::write_palettes(&mut palettes, 0, palette_data);
for (c, slot) in pal.iter_mut().enumerate() {
let offset = (p * 16 + c) * 4;
*slot = Color::rgba(
palette_data[offset],
palette_data[offset + 1],
palette_data[offset + 2],
palette_data[offset + 3],
);
}
}
Ok(GlyphBank { tile_size, width, height, pixel_indices, palettes }) Ok(GlyphBank { tile_size, width, height, pixel_indices, palettes })
} }
@@ -1295,17 +1334,7 @@ impl AssetManager {
.map_err(|_| "Buffer too small for GLYPHBANK".to_string())?; .map_err(|_| "Buffer too small for GLYPHBANK".to_string())?;
let mut palettes = vec![[Color::BLACK; GLYPH_BANK_COLORS_PER_PALETTE]; palette_count]; let mut palettes = vec![[Color::BLACK; GLYPH_BANK_COLORS_PER_PALETTE]; palette_count];
for (p, pal) in palettes.iter_mut().enumerate() { decode::write_palettes(&mut palettes, 0, &palette_data);
for (c, slot) in pal.iter_mut().enumerate() {
let offset = (p * 16 + c) * 4;
*slot = Color::rgba(
palette_data[offset],
palette_data[offset + 1],
palette_data[offset + 2],
palette_data[offset + 3],
);
}
}
Ok(GlyphBank { tile_size, width, height, pixel_indices, palettes }) Ok(GlyphBank { tile_size, width, height, pixel_indices, palettes })
} }
@@ -1336,11 +1365,37 @@ impl AssetManager {
} }
} }
fn perform_load_music_bank(
entry: &AssetEntry,
assets_data: Arc<RwLock<AssetsPayloadSource>>,
) -> Result<MusicBank, String> {
let op_mode = Self::op_mode_for(entry)?;
let slice = {
let assets_data = assets_data.read().unwrap();
assets_data
.open_slice(entry.offset, entry.size)
.map_err(|_| "Asset offset/size out of bounds".to_string())?
};
match op_mode {
AssetOpMode::DirectFromSlice => {
let mut reader =
slice.open_reader().map_err(|_| "Asset payload read failed".to_string())?;
Self::decode_music_bank_from_reader(entry, &mut reader)
}
AssetOpMode::StageInMemory => {
let buffer =
slice.read_all().map_err(|_| "Asset payload read failed".to_string())?;
Self::decode_music_bank_from_buffer(entry, &buffer)
}
}
}
fn perform_load_scene_bank( fn perform_load_scene_bank(
entry: &AssetEntry, entry: &AssetEntry,
assets_data: Arc<RwLock<AssetsPayloadSource>>, assets_data: Arc<RwLock<AssetsPayloadSource>>,
) -> Result<SceneBank, String> { ) -> Result<SceneBank, String> {
let _ = entry.metadata_as_scene_bank()?; decode::scene_metadata(entry)?;
let op_mode = Self::op_mode_for(entry)?; let op_mode = Self::op_mode_for(entry)?;
let slice = { let slice = {
let assets_data = assets_data.read().unwrap(); let assets_data = assets_data.read().unwrap();
@@ -1367,152 +1422,25 @@ impl AssetManager {
entry: &AssetEntry, entry: &AssetEntry,
buffer: &[u8], buffer: &[u8],
) -> Result<SceneBank, String> { ) -> Result<SceneBank, String> {
let _ = entry.metadata_as_scene_bank()?; decode::scene_metadata(entry)?;
decode::payload_size(entry, buffer.len())?;
if buffer.len() < SCENE_HEADER_BYTES_V1 { decode::scene_header(buffer)?;
return Err("Buffer too small for SCENE".to_string());
}
if buffer[0..4] != SCENE_PAYLOAD_MAGIC_V1 {
return Err("Invalid SCENE magic".to_string());
}
let version = u16::from_le_bytes([buffer[4], buffer[5]]);
if version != SCENE_PAYLOAD_VERSION_V1 {
return Err(format!("Unsupported SCENE version: {}", version));
}
let layer_count = u16::from_le_bytes([buffer[6], buffer[7]]) as usize;
if layer_count != SCENE_LAYER_COUNT_V1 {
return Err(format!("Invalid SCENE layer count: {}", layer_count));
}
let mut offset = SCENE_HEADER_BYTES_V1; let mut offset = SCENE_HEADER_BYTES_V1;
let mut decoded_size = 0_usize; let mut decoded = 0;
let layers = std::array::from_fn(|_| SceneLayer { let mut layers = decode::empty_layers();
active: false,
glyph_asset_id: 0,
tile_size: TileSize::Size8,
parallax_factor: ParallaxFactor { x: 1.0, y: 1.0 },
tilemap: TileMap { width: 0, height: 0, tiles: Vec::new() },
});
let mut layers = layers;
for layer in &mut layers { for layer in &mut layers {
let header_end = offset let end = offset
.checked_add(SCENE_LAYER_HEADER_BYTES_V1) .checked_add(SCENE_LAYER_HEADER_BYTES_V1)
.ok_or("SCENE layer header offset overflow")?; .ok_or("SCENE header offset overflow")?;
if header_end > buffer.len() { let header =
return Err("Buffer too small for SCENE layer header".to_string()); buffer.get(offset..end).ok_or("Buffer too small for SCENE layer header")?;
*layer = decode::scene_layer(header, end, entry.size, &mut decoded)?;
let tile_bytes =
layer.tilemap.width * layer.tilemap.height * SCENE_TILE_RECORD_BYTES_V1;
offset = end + tile_bytes; // checked by scene_layer before allocation
decode::append_tiles(&mut layer.tilemap.tiles, &buffer[end..offset]);
} }
decode::scene_finish(entry, offset, decoded)?;
let flags = buffer[offset];
let glyph_asset_id = i32::from_le_bytes([
buffer[offset + 1],
buffer[offset + 2],
buffer[offset + 3],
buffer[offset + 4],
]);
let tile_size_raw = buffer[offset + 5];
let tile_size = match tile_size_raw {
8 => TileSize::Size8,
16 => TileSize::Size16,
32 => TileSize::Size32,
other => return Err(format!("Invalid SCENE tile size: {}", other)),
};
let parallax_factor_x = f32::from_le_bytes([
buffer[offset + 8],
buffer[offset + 9],
buffer[offset + 10],
buffer[offset + 11],
]);
let parallax_factor_y = f32::from_le_bytes([
buffer[offset + 12],
buffer[offset + 13],
buffer[offset + 14],
buffer[offset + 15],
]);
if !parallax_factor_x.is_finite() || !parallax_factor_y.is_finite() {
return Err("Invalid SCENE parallax_factor".to_string());
}
let width = u32::from_le_bytes([
buffer[offset + 16],
buffer[offset + 17],
buffer[offset + 18],
buffer[offset + 19],
]) as usize;
let height = u32::from_le_bytes([
buffer[offset + 20],
buffer[offset + 21],
buffer[offset + 22],
buffer[offset + 23],
]) as usize;
let tile_count = u32::from_le_bytes([
buffer[offset + 24],
buffer[offset + 25],
buffer[offset + 26],
buffer[offset + 27],
]) as usize;
let expected_tile_count =
width.checked_mul(height).ok_or("SCENE tile count overflow")?;
if tile_count != expected_tile_count {
return Err(format!(
"Invalid SCENE tile count for layer: expected {}, got {}",
expected_tile_count, tile_count
));
}
offset = header_end;
let tile_bytes = tile_count
.checked_mul(SCENE_TILE_RECORD_BYTES_V1)
.ok_or("SCENE tile payload overflow")?;
let tiles_end = offset.checked_add(tile_bytes).ok_or("SCENE payload overflow")?;
if tiles_end > buffer.len() {
return Err("Buffer too small for SCENE tile data".to_string());
}
let mut tiles = Vec::with_capacity(tile_count);
for _ in 0..tile_count {
let tile_flags = buffer[offset];
let palette_id = buffer[offset + 1];
let glyph_id = u16::from_le_bytes([buffer[offset + 2], buffer[offset + 3]]);
tiles.push(Tile {
active: (tile_flags & 0b0000_0001) != 0,
glyph: Glyph { glyph_id, palette_id },
flip_x: (tile_flags & 0b0000_0010) != 0,
flip_y: (tile_flags & 0b0000_0100) != 0,
});
offset += SCENE_TILE_RECORD_BYTES_V1;
}
decoded_size = decoded_size
.checked_add(SCENE_DECODED_LAYER_OVERHEAD_BYTES_V1)
.and_then(|size| size.checked_add(tile_count * size_of::<Tile>()))
.ok_or("SCENE decoded_size overflow")?;
*layer = SceneLayer {
active: (flags & 0b0000_0001) != 0,
glyph_asset_id,
tile_size,
parallax_factor: ParallaxFactor { x: parallax_factor_x, y: parallax_factor_y },
tilemap: TileMap { width, height, tiles },
};
}
if offset != buffer.len() {
return Err("Trailing bytes in SCENE payload".to_string());
}
if entry.decoded_size != decoded_size as u64 {
return Err(format!(
"Invalid SCENE decoded_size: expected {}, got {}",
decoded_size, entry.decoded_size
));
}
Ok(SceneBank { layers }) Ok(SceneBank { layers })
} }
@@ -1529,18 +1457,32 @@ impl AssetManager {
entry: &AssetEntry, entry: &AssetEntry,
buffer: &[u8], buffer: &[u8],
) -> Result<SoundBank, String> { ) -> Result<SoundBank, String> {
let meta = entry.metadata_as_sound_bank()?; decode::payload_size(entry, buffer.len())?;
let sample_rate = meta.sample_rate; let bank = decode_audio_bank(buffer, false)?;
if bank.decoded_size() as u64 != entry.decoded_size {
let mut data = Vec::with_capacity(buffer.len() / 2); return Err("audio decoded_size mismatch".into());
for i in (0..buffer.len()).step_by(2) { }
if i + 1 < buffer.len() { if let DecodedAudioBank::Sound(bank) = bank {
data.push(i16::from_le_bytes([buffer[i], buffer[i + 1]])); Ok(bank)
} else {
Err("audio bank type mismatch".into())
} }
} }
let sample = Arc::new(Sample::new(sample_rate, data)); fn decode_music_bank_from_buffer(
Ok(SoundBank::new(vec![sample])) entry: &AssetEntry,
buffer: &[u8],
) -> Result<MusicBank, String> {
decode::payload_size(entry, buffer.len())?;
let bank = decode_audio_bank(buffer, true)?;
if bank.decoded_size() as u64 != entry.decoded_size {
return Err("audio decoded_size mismatch".into());
}
if let DecodedAudioBank::Music(bank) = bank {
Ok(bank)
} else {
Err("audio bank type mismatch".into())
}
} }
fn decode_sound_bank_from_reader( fn decode_sound_bank_from_reader(
@@ -1552,6 +1494,15 @@ impl AssetManager {
Self::decode_sound_bank_from_buffer(entry, &raw) Self::decode_sound_bank_from_buffer(entry, &raw)
} }
fn decode_music_bank_from_reader(
entry: &AssetEntry,
reader: &mut impl Read,
) -> Result<MusicBank, String> {
let mut raw = Vec::new();
reader.read_to_end(&mut raw).map_err(|_| "Asset payload read failed".to_string())?;
Self::decode_music_bank_from_buffer(entry, &raw)
}
pub fn status(&self, handle: HandleId) -> LoadStatus { pub fn status(&self, handle: HandleId) -> LoadStatus {
if let Some(status) = self.handles.read().unwrap().get(&handle).map(|h| h.status) { if let Some(status) = self.handles.read().unwrap().get(&handle).map(|h| h.status) {
return status; return status;
@@ -1680,6 +1631,7 @@ impl AssetManager {
let slot_ref = match bank_type { let slot_ref = match bank_type {
BankType::GLYPH => SlotRef::gfx(slot), BankType::GLYPH => SlotRef::gfx(slot),
BankType::SOUNDS => SlotRef::audio(slot), BankType::SOUNDS => SlotRef::audio(slot),
BankType::MUSIC => SlotRef::music(slot),
BankType::SCENE => SlotRef::scene(slot), BankType::SCENE => SlotRef::scene(slot),
}; };
let handle = self.handle_for_slot(slot_ref); let handle = self.handle_for_slot(slot_ref);
@@ -1795,6 +1747,16 @@ impl AssetManager {
h.status = LoadStatus::COMMITTED; h.status = LoadStatus::COMMITTED;
} }
} }
BankType::MUSIC => {
if let Some((bank, _)) = self.music_policy.take_staging(handle_id) {
self.sound_installer.install_music_bank(h.slot.index, bank);
let mut slots = self.music_slots.write().unwrap();
if h.slot.index < slots.len() {
slots[h.slot.index] = Some(h._asset_id);
}
h.status = LoadStatus::COMMITTED;
}
}
BankType::SCENE => { BankType::SCENE => {
if let Some((bank, _)) = self.scene_policy.take_staging(handle_id) { if let Some((bank, _)) = self.scene_policy.take_staging(handle_id) {
self.scene_installer.install_scene_bank(h.slot.index, bank); self.scene_installer.install_scene_bank(h.slot.index, bank);
@@ -1809,12 +1771,57 @@ impl AssetManager {
} }
} }
} }
self.reap_audio_cache();
}
fn reap_audio_cache(&self) {
fn reap<T>(policy: &BankPolicy<T>, slots: &[Option<AssetId>; 16]) {
let mut resident = policy.resident.write().unwrap();
resident.retain(|id, entry| {
let keep = slots.contains(&Some(*id)) || Arc::strong_count(&entry.value) > 1;
if !keep {
policy.used_bytes.fetch_sub(entry.bytes, Ordering::Relaxed);
}
keep
});
}
reap(&self.sound_policy, &self.sound_slots.read().unwrap());
reap(&self.music_policy, &self.music_slots.read().unwrap());
}
pub fn unload_audio(&self, music: bool, slot: usize) -> AssetOpStatus {
if slot >= 16 {
return AssetOpStatus::InvalidState;
}
let target = if music { SlotRef::music(slot) } else { SlotRef::audio(slot) };
self.bump_target_generation(target);
if let Some(handle) = self.target_handles.read().unwrap().get(&target).copied() {
self.clear_staging_for_handle(handle);
if let Some(h) = self.handles.write().unwrap().get_mut(&handle) {
h.status = LoadStatus::CANCELED;
}
}
let slots = if music { &self.music_slots } else { &self.sound_slots };
let mut ids = slots.write().unwrap();
let previous = ids[slot].take();
self.sound_installer.unload_audio_bank(music, slot);
if let Some(id) = previous.filter(|id| !ids.contains(&Some(*id))) {
if music {
if let Some(old) = self.music_policy.resident.write().unwrap().remove(&id) {
self.music_policy.used_bytes.fetch_sub(old.bytes, Ordering::Relaxed);
}
} else if let Some(old) = self.sound_policy.resident.write().unwrap().remove(&id) {
self.sound_policy.used_bytes.fetch_sub(old.bytes, Ordering::Relaxed);
}
}
AssetOpStatus::Ok
} }
pub fn bank_telemetry(&self) -> Vec<BankTelemetry> { pub fn bank_telemetry(&self) -> Vec<BankTelemetry> {
vec![ vec![
self.bank_telemetry_for(BankType::GLYPH), self.bank_telemetry_for(BankType::GLYPH),
self.bank_telemetry_for(BankType::SOUNDS), self.bank_telemetry_for(BankType::SOUNDS),
self.bank_telemetry_for(BankType::MUSIC),
self.bank_telemetry_for(BankType::SCENE), self.bank_telemetry_for(BankType::SCENE),
] ]
} }
@@ -1827,6 +1834,9 @@ impl AssetManager {
BankType::SOUNDS => { BankType::SOUNDS => {
self.sound_slots.read().unwrap().iter().filter(|slot| slot.is_some()).count() self.sound_slots.read().unwrap().iter().filter(|slot| slot.is_some()).count()
} }
BankType::MUSIC => {
self.music_slots.read().unwrap().iter().filter(|slot| slot.is_some()).count()
}
BankType::SCENE => { BankType::SCENE => {
self.scene_slots.read().unwrap().iter().filter(|slot| slot.is_some()).count() self.scene_slots.read().unwrap().iter().filter(|slot| slot.is_some()).count()
} }
@@ -1879,6 +1889,27 @@ impl AssetManager {
SlotStats { asset_id, asset_name, generation: 0, resident_bytes: bytes } SlotStats { asset_id, asset_name, generation: 0, resident_bytes: bytes }
} }
BankType::MUSIC => {
let slots = self.music_slots.read().unwrap();
let asset_id = slots.get(slot.index).and_then(|s| *s);
let (bytes, asset_name) = if let Some(id) = &asset_id {
let bytes = self
.music_policy
.resident
.read()
.unwrap()
.get(id)
.map(|entry| entry.bytes)
.unwrap_or(0);
let name = self.assets.read().unwrap().get(id).map(|e| e.asset_name.clone());
(bytes, name)
} else {
(0, None)
};
SlotStats { asset_id, asset_name, generation: 0, resident_bytes: bytes }
}
BankType::SCENE => { BankType::SCENE => {
let slots = self.scene_slots.read().unwrap(); let slots = self.scene_slots.read().unwrap();
let asset_id = slots.get(slot.index).and_then(|s| *s); let asset_id = slots.get(slot.index).and_then(|s| *s);
@@ -1911,6 +1942,7 @@ impl AssetManager {
} }
self.gfx_policy.clear(); self.gfx_policy.clear();
self.sound_policy.clear(); self.sound_policy.clear();
self.music_policy.clear();
self.scene_policy.clear(); self.scene_policy.clear();
self.handles.write().unwrap().clear(); self.handles.write().unwrap().clear();
self.target_handles.write().unwrap().clear(); self.target_handles.write().unwrap().clear();
@@ -1921,9 +1953,11 @@ impl AssetManager {
self.gfx_slots.write().unwrap().fill(None); self.gfx_slots.write().unwrap().fill(None);
self.glyph_slot_index.clear(); self.glyph_slot_index.clear();
self.sound_slots.write().unwrap().fill(None); self.sound_slots.write().unwrap().fill(None);
self.music_slots.write().unwrap().fill(None);
self.scene_slots.write().unwrap().fill(None); self.scene_slots.write().unwrap().fill(None);
self.gfx_installer.clear_glyph_banks(); self.gfx_installer.clear_glyph_banks();
self.sound_installer.clear_sound_banks(); self.sound_installer.clear_sound_banks();
self.sound_installer.clear_music_banks();
self.scene_installer.clear_scene_banks(); self.scene_installer.clear_scene_banks();
} }
@@ -2712,7 +2746,7 @@ mod tests {
let scene_installer = Arc::clone(&banks) as Arc<dyn SceneBankPoolInstaller>; let scene_installer = Arc::clone(&banks) as Arc<dyn SceneBankPoolInstaller>;
// 100 samples of 16-bit PCM (zeros) // 100 samples of 16-bit PCM (zeros)
let data = vec![0u8; 200]; let data = crate::bank_format::test_pcm_bank(&[0u8; 200], 44100);
let asset_entry = AssetEntry { let asset_entry = AssetEntry {
asset_id: 1, asset_id: 1,
@@ -2758,7 +2792,7 @@ mod tests {
let sound_installer = Arc::clone(&banks) as Arc<dyn SoundBankPoolInstaller>; let sound_installer = Arc::clone(&banks) as Arc<dyn SoundBankPoolInstaller>;
let scene_installer = Arc::clone(&banks) as Arc<dyn SceneBankPoolInstaller>; let scene_installer = Arc::clone(&banks) as Arc<dyn SceneBankPoolInstaller>;
let data = vec![0u8; 200]; let data = crate::bank_format::test_pcm_bank(&[0u8; 200], 44100);
let asset_entry = AssetEntry { let asset_entry = AssetEntry {
asset_id: 2, asset_id: 2,
@@ -0,0 +1,180 @@
//! Format authority shared by complete-buffer and bounded resumable decoding.
use super::*;
pub(super) fn glyph_layout(
entry: &AssetEntry,
) -> Result<(TileSize, usize, usize, usize, usize, usize), String> {
let meta = entry.metadata_as_glyph_bank()?;
let tile_size = match meta.tile_size {
8 => TileSize::Size8,
16 => TileSize::Size16,
32 => TileSize::Size32,
_ => return Err(format!("Invalid tile_size: {}", meta.tile_size)),
};
let palette_count = meta.palette_count as usize;
if !(1..=GLYPH_BANK_MAX_PALETTE_COUNT_V1).contains(&palette_count) {
return Err(format!("Invalid palette_count: {}", meta.palette_count));
}
let width = meta.width as usize;
let height = meta.height as usize;
let logical_pixels = width.checked_mul(height).ok_or("GlyphBank dimensions overflow")?;
let serialized_pixel_bytes = logical_pixels.div_ceil(2);
let palette_bytes = palette_count
.checked_mul(GLYPH_BANK_PALETTE_BYTES_PER_PALETTE)
.ok_or("GlyphBank palette size overflow")?;
let serialized_size = serialized_pixel_bytes
.checked_add(palette_bytes)
.ok_or("GlyphBank serialized size overflow")?;
let decoded_size =
logical_pixels.checked_add(palette_bytes).ok_or("GlyphBank decoded size overflow")?;
if entry.size != serialized_size as u64 {
return Err(format!(
"Invalid GLYPHBANK serialized size: expected {}, got {}",
serialized_size, entry.size
));
}
if entry.decoded_size != decoded_size as u64 {
return Err(format!(
"Invalid GLYPHBANK decoded_size: expected {}, got {}",
decoded_size, entry.decoded_size
));
}
Ok((tile_size, width, height, serialized_pixel_bytes, palette_count, palette_bytes))
}
pub(super) fn append_pixels(pixels: &mut Vec<u8>, bytes: &[u8], logical: usize) {
for &packed in bytes {
if pixels.len() < logical {
pixels.push(packed >> 4);
}
if pixels.len() < logical {
pixels.push(packed & 15);
}
}
}
pub(super) fn write_palettes(
palettes: &mut [[Color; GLYPH_BANK_COLORS_PER_PALETTE]],
position: usize,
bytes: &[u8],
) {
for (i, rgba) in bytes.as_chunks::<4>().0.iter().enumerate() {
let at = position / 4 + i;
palettes[at / 16][at % 16] = Color::rgba(rgba[0], rgba[1], rgba[2], rgba[3]);
}
}
pub(super) fn payload_size(entry: &AssetEntry, actual: usize) -> Result<(), String> {
if actual as u64 != entry.size {
return Err("Invalid asset payload size".into());
}
Ok(())
}
pub(super) fn scene_metadata(entry: &AssetEntry) -> Result<(), String> {
entry.metadata_as_scene_bank().map(|_| ())
}
pub(super) fn empty_layers() -> [SceneLayer; SCENE_LAYER_COUNT_V1] {
std::array::from_fn(|_| SceneLayer {
active: false,
glyph_asset_id: 0,
tile_size: TileSize::Size8,
parallax_factor: ParallaxFactor { x: 1.0, y: 1.0 },
tilemap: TileMap { width: 0, height: 0, tiles: Vec::new() },
})
}
pub(super) fn scene_header(bytes: &[u8]) -> Result<(), String> {
if bytes.len() < SCENE_HEADER_BYTES_V1 {
return Err("Buffer too small for SCENE".into());
}
if bytes[..4] != SCENE_PAYLOAD_MAGIC_V1 {
return Err("Invalid SCENE magic".into());
}
let version = u16::from_le_bytes([bytes[4], bytes[5]]);
if version != SCENE_PAYLOAD_VERSION_V1 {
return Err(format!("Unsupported SCENE version: {version}"));
}
let count = u16::from_le_bytes([bytes[6], bytes[7]]) as usize;
if count != SCENE_LAYER_COUNT_V1 {
return Err(format!("Invalid SCENE layer count: {count}"));
}
Ok(())
}
pub(super) fn scene_layer(
bytes: &[u8],
position: usize,
payload_len: u64,
decoded: &mut usize,
) -> Result<SceneLayer, String> {
if bytes.len() < SCENE_LAYER_HEADER_BYTES_V1 {
return Err("Buffer too small for SCENE layer header".into());
}
let u32at = |at| u32::from_le_bytes(bytes[at..at + 4].try_into().unwrap());
let tile_size = match bytes[5] {
8 => TileSize::Size8,
16 => TileSize::Size16,
32 => TileSize::Size32,
other => return Err(format!("Invalid SCENE tile size: {other}")),
};
let x = f32::from_le_bytes(bytes[8..12].try_into().unwrap());
let y = f32::from_le_bytes(bytes[12..16].try_into().unwrap());
if !x.is_finite() || !y.is_finite() {
return Err("Invalid SCENE parallax_factor".into());
}
let width = u32at(16) as usize;
let height = u32at(20) as usize;
let count = u32at(24) as usize;
let expected = width.checked_mul(height).ok_or("SCENE tile count overflow")?;
if count != expected {
return Err(format!(
"Invalid SCENE tile count for layer: expected {expected}, got {count}"
));
}
let end = count
.checked_mul(SCENE_TILE_RECORD_BYTES_V1)
.and_then(|n| position.checked_add(n))
.ok_or("SCENE payload overflow")?;
if end as u64 > payload_len {
return Err("Buffer too small for SCENE tile data".into());
}
*decoded = decoded
.checked_add(SCENE_DECODED_LAYER_OVERHEAD_BYTES_V1)
.and_then(|n| count.checked_mul(size_of::<Tile>()).and_then(|b| n.checked_add(b)))
.ok_or("SCENE decoded_size overflow")?;
Ok(SceneLayer {
active: bytes[0] & 1 != 0,
glyph_asset_id: i32::from_le_bytes(bytes[1..5].try_into().unwrap()),
tile_size,
parallax_factor: ParallaxFactor { x, y },
tilemap: TileMap { width, height, tiles: Vec::with_capacity(count) },
})
}
pub(super) fn append_tiles(tiles: &mut Vec<Tile>, bytes: &[u8]) {
tiles.extend(bytes.as_chunks::<SCENE_TILE_RECORD_BYTES_V1>().0.iter().map(|b| Tile {
active: b[0] & 1 != 0,
glyph: Glyph { glyph_id: u16::from_le_bytes([b[2], b[3]]), palette_id: b[1] },
flip_x: b[0] & 2 != 0,
flip_y: b[0] & 4 != 0,
}));
}
pub(super) fn scene_finish(
entry: &AssetEntry,
position: usize,
decoded: usize,
) -> Result<(), String> {
payload_size(entry, position)?;
if decoded as u64 != entry.decoded_size {
return Err(format!(
"Invalid SCENE decoded_size: expected {decoded}, got {}",
entry.decoded_size
));
}
Ok(())
}
#[cfg(test)]
pub(super) mod tests;
@@ -0,0 +1,461 @@
use super::*;
use serde_json::{Value, json};
struct FileFixture(std::path::PathBuf);
impl FileFixture {
fn new(name: &str, bytes: &[u8]) -> Self {
let path = std::env::temp_dir()
.join(format!("prometeu-conformance-{}-{name}", std::process::id()));
std::fs::write(&path, bytes).unwrap();
Self(path)
}
fn source(&self, declared_len: u64) -> Arc<RwLock<AssetsPayloadSource>> {
Arc::new(RwLock::new(AssetsPayloadSource::from_file(self.0.clone(), 0, declared_len)))
}
}
impl Drop for FileFixture {
fn drop(&mut self) {
std::fs::remove_file(&self.0).unwrap();
}
}
fn pcm_entry(size: u64) -> AssetEntry {
AssetEntry {
asset_id: 1,
asset_name: "pcm".into(),
bank_type: BankType::SOUNDS,
offset: 0,
size,
decoded_size: size,
codec: AssetCodec::None,
metadata: json!({"sample_rate": 44100, "channels": 1}),
}
}
#[test]
fn pcm_chunk_edges_validate_all_declared_frames_and_reject_odd_tails() {
for len in [0, 1, 2, 3, 4095, 4096, 4097, 8191, 8192, 8193] {
let raw: Vec<u8> = (0..len).map(|i| if i % 2 == 0 { 0x34 } else { 0x12 }).collect();
let bytes = crate::bank_format::test_pcm_bank(&raw, 44100);
let expected = if len % 2 == 0 {
Some(json!([[vec![0x1234_i16; len / 2], 44100, null, null]]))
} else {
None
};
check_case("pcm-chunk", &pcm_entry(bytes.len() as u64), bytes, expected);
}
}
#[test]
fn physically_truncated_pcm_file_is_rejected_by_all_modes() {
let bytes = crate::bank_format::test_pcm_bank(&[0x34, 0x12], 44100);
let entry = pcm_entry(bytes.len() as u64);
let fixture = FileFixture::new("truncated-pcm", &bytes[..bytes.len() - 1]);
let source = fixture.source(bytes.len() as u64);
assert!(AssetManager::perform_load_sound_bank(&entry, source.clone()).is_err());
assert!(resumable::conformance_result(&entry, &source).is_err());
std::fs::write(&fixture.0, &bytes).unwrap();
let expected = json!([[[0x1234], 44100, null, null]]);
assert_eq!(resumable::conformance_result(&entry, &source).unwrap(), expected);
assert_eq!(
sound_snapshot(&AssetManager::perform_load_sound_bank(&entry, source).unwrap()),
expected
);
}
#[test]
fn physically_short_scene_file_is_rejected_by_both_modes() {
// Minimal scene: four empty layers, 12-byte header + 4 * 32-byte headers.
let mut bytes = vec![0_u8; 140];
bytes[..8].copy_from_slice(&[b'S', b'C', b'N', b'E', 1, 0, 4, 0]);
for start in [12, 44, 76, 108] {
bytes[start + 5] = 8;
}
let fixture = FileFixture::new("short-scene", &bytes);
let mut entry = AssetEntry {
asset_id: 2,
asset_name: "scene".into(),
bank_type: BankType::SCENE,
offset: 0,
size: 141,
decoded_size: 64,
codec: AssetCodec::None,
metadata: json!({}),
};
let expected = json!(vec![json!([false, 0, 8, 0.0, 0.0, 0, 0, []]); 4]);
let source = fixture.source(141);
assert!(AssetManager::perform_load_scene_bank(&entry, source.clone()).is_err());
assert!(resumable::conformance_result(&entry, &source).is_err());
// Exact physical/declared size is accepted by both.
entry.size = 140;
assert_eq!(resumable::conformance_result(&entry, &source).unwrap(), expected);
assert_eq!(
scene_snapshot(&AssetManager::perform_load_scene_bank(&entry, source.clone()).unwrap()),
expected
);
// An actual trailing byte is rejected by both.
entry.size = 141;
bytes.push(0);
std::fs::write(&fixture.0, bytes).unwrap();
assert!(AssetManager::perform_load_scene_bank(&entry, source.clone()).is_err());
assert!(resumable::conformance_result(&entry, &source).is_err());
}
// Every mode sees the same declared slice, just as it does in production.
fn check_case(name: &str, entry: &AssetEntry, bytes: Vec<u8>, expected: Option<Value>) {
let source = Arc::new(RwLock::new(AssetsPayloadSource::from_bytes(bytes)));
for mode in 0..4 {
let result = (|| -> Result<Value, String> {
if mode == 3 {
return resumable::conformance_result(entry, &source);
}
if mode == 2 {
return match entry.bank_type {
BankType::GLYPH => AssetManager::perform_load_glyph_bank(entry, source.clone())
.map(|b| glyph_snapshot(&b)),
BankType::SOUNDS => {
AssetManager::perform_load_sound_bank(entry, source.clone())
.map(|b| sound_snapshot(&b))
}
BankType::MUSIC => AssetManager::perform_load_music_bank(entry, source.clone())
.map(|b| music_snapshot(&b)),
BankType::SCENE => AssetManager::perform_load_scene_bank(entry, source.clone())
.map(|b| scene_snapshot(&b)),
};
}
let slice = source
.read()
.unwrap()
.open_slice(entry.offset, entry.size)
.map_err(|e| e.to_string())?;
let bytes = slice.read_all().map_err(|e| e.to_string())?;
let mut reader = &bytes[..];
match (entry.bank_type, mode) {
(BankType::GLYPH, 0) => AssetManager::decode_glyph_bank_from_buffer(entry, &bytes)
.map(|b| glyph_snapshot(&b)),
(BankType::GLYPH, _) => {
AssetManager::decode_glyph_bank_from_reader(entry, &mut reader)
.map(|b| glyph_snapshot(&b))
}
(BankType::SOUNDS, 0) => AssetManager::decode_sound_bank_from_buffer(entry, &bytes)
.map(|b| sound_snapshot(&b)),
(BankType::SOUNDS, _) => {
AssetManager::decode_sound_bank_from_reader(entry, &mut reader)
.map(|b| sound_snapshot(&b))
}
(BankType::MUSIC, _) => AssetManager::decode_music_bank_from_buffer(entry, &bytes)
.map(|b| music_snapshot(&b)),
(BankType::SCENE, 0) => AssetManager::decode_scene_bank_from_buffer(entry, &bytes)
.map(|b| scene_snapshot(&b)),
(BankType::SCENE, _) => {
AssetManager::decode_scene_bank_from_reader(entry, &mut reader)
.map(|b| scene_snapshot(&b))
}
}
})();
assert_eq!(result.ok(), expected, "{name}, mode={mode}");
}
}
#[test]
fn glyph_layout_palette_nibbles_and_chunk_boundaries() {
for pixels in [0_usize, 1, 2, 8191, 8192, 8193, 16385] {
for palettes in [1, 3, 64] {
let mut bytes = vec![0x12; pixels.div_ceil(2)];
bytes.extend(std::iter::repeat_n([0x12, 0x34, 0x56, 0x78], palettes * 16).flatten());
let entry = AssetEntry {
asset_id: 1,
asset_name: "glyph".into(),
bank_type: BankType::GLYPH,
offset: 0,
size: bytes.len() as u64,
decoded_size: (pixels + palettes * 64) as u64,
codec: AssetCodec::None,
metadata: json!({"tile_size":16,"width":pixels,"height":1,"palette_count":palettes,"palette_authored":palettes}),
};
let expected = json!([
16,
pixels,
1,
(0..pixels).map(|i| 1 + i % 2).collect::<Vec<_>>(),
vec![vec![0x12345678_u32; 16]; palettes]
]);
check_case("glyph-valid", &entry, bytes.clone(), Some(expected));
for (key, value) in [
("tile_size", json!(7)),
("palette_count", json!(0)),
("palette_count", json!(65)),
("width", json!(-1)),
("height", json!("bad")),
] {
let mut invalid = entry.clone();
invalid.metadata[key] = value;
check_case(key, &invalid, bytes.clone(), None);
}
let mut invalid = entry.clone();
invalid.decoded_size += 1;
check_case("glyph-decoded-size", &invalid, bytes.clone(), None);
let mut invalid = entry.clone();
invalid.size += 1;
let mut extra = bytes.clone();
extra.push(0);
check_case("glyph-trailing", &invalid, extra, None);
bytes.pop();
check_case("glyph-truncated", &entry, bytes, None);
}
}
}
fn empty_scene() -> (AssetEntry, Vec<u8>, Value) {
let mut bytes = vec![0_u8; 140];
bytes[..8].copy_from_slice(&[b'S', b'C', b'N', b'E', 1, 0, 4, 0]);
for start in [12, 44, 76, 108] {
bytes[start + 5] = 8;
}
let entry = AssetEntry {
asset_id: 2,
asset_name: "scene".into(),
bank_type: BankType::SCENE,
offset: 0,
size: 140,
decoded_size: 64,
codec: AssetCodec::None,
metadata: json!({}),
};
(entry, bytes, json!(vec![json!([false, 0, 8, 0.0, 0.0, 0, 0, []]); 4]))
}
#[test]
fn scene_header_ranges_nonfinite_and_truncation_corpus() {
let (entry, bytes, expected) = empty_scene();
check_case("empty-scene", &entry, bytes.clone(), Some(expected));
for end in 0..bytes.len() {
let mut e = entry.clone();
e.size = end as u64;
check_case("scene-every-truncation", &e, bytes[..end].to_vec(), None);
}
for (offset, value) in [(0, 0xff), (4, 2), (6, 3), (17, 7), (36, 1)] {
let mut invalid = bytes.clone();
invalid[offset] = value;
check_case("scene-invalid-header-field", &entry, invalid, None);
}
for offset in [20, 24] {
for value in [f32::NAN, f32::INFINITY, f32::NEG_INFINITY] {
let mut invalid = bytes.clone();
invalid[offset..offset + 4].copy_from_slice(&value.to_le_bytes());
check_case("scene-nonfinite", &entry, invalid, None);
}
}
let mut invalid = bytes.clone();
for offset in [28, 32, 36] {
invalid[offset..offset + 4].copy_from_slice(&u32::MAX.to_le_bytes());
}
check_case("scene-dimensions-overflow-or-count-mismatch", &entry, invalid, None);
let mut invalid = bytes.clone();
for offset in [28, 32, 36] {
invalid[offset..offset + 4].copy_from_slice(&1_u32.to_le_bytes());
}
// Four layer headers no longer fit after the declared tile consumes bytes.
check_case("scene-tile-range", &entry, invalid, None);
let mut sequence_metadata = entry.clone();
sequence_metadata.metadata = json!([]);
check_case(
"scene-empty-sequence-metadata",
&sequence_metadata,
bytes.clone(),
Some(empty_scene().2),
);
let mut e = entry.clone();
e.decoded_size += 1;
check_case("scene-decoded-size", &e, bytes.clone(), None);
e = entry.clone();
e.size += 1;
let mut trailing = bytes.clone();
trailing.push(0);
check_case("scene-trailing", &e, trailing, None);
for metadata in [json!(null), json!("bad")] {
e = entry.clone();
e.metadata = metadata;
check_case("scene-metadata", &e, bytes.clone(), None);
}
}
#[test]
fn scene_tile_chunk_edges_flags_and_reserved_bytes() {
for count in [1_usize, 1023, 1024, 1025, 2049] {
let (mut entry, mut bytes, mut expected) = empty_scene();
bytes[12] = 0xff; // Only the active flag is meaningful.
bytes[13..17].copy_from_slice(&(-7_i32).to_le_bytes());
bytes[17] = 32;
bytes[18..20].copy_from_slice(&[0xff; 2]); // Reserved fields are ignored.
bytes[20..24].copy_from_slice(&(-0.5_f32).to_le_bytes());
bytes[24..28].copy_from_slice(&2.0_f32.to_le_bytes());
bytes[28..32].copy_from_slice(&(count as u32).to_le_bytes());
bytes[32..36].copy_from_slice(&1_u32.to_le_bytes());
bytes[36..40].copy_from_slice(&(count as u32).to_le_bytes());
bytes.splice(44..44, std::iter::repeat_n([0xff, 255, 0x34, 0x12], count).flatten());
entry.size = bytes.len() as u64;
entry.decoded_size += (count * size_of::<Tile>()) as u64;
expected[0] = json!([
true,
-7,
32,
-0.5,
2.0,
count,
1,
vec![(true, 0x1234, 255, true, true); count]
]);
check_case("scene-chunks", &entry, bytes, Some(expected));
}
}
#[test]
fn pcm_descriptor_and_source_range_corpus() {
let bytes = crate::bank_format::test_pcm_bank(&[0x34, 0x12], 44100);
let entry = pcm_entry(bytes.len() as u64);
// Audio metadata lives in the versioned payload, so it can vary per sample.
for (offset, value) in
[(48, 99u32), (49, 0), (52, 0), (56, 2), (60, 1), (68, 0), (72, 3), (84, 0)]
{
let mut invalid = bytes.clone();
if offset == 48 || offset == 49 {
invalid[offset] = value as u8;
} else {
invalid[offset..offset + 4].copy_from_slice(&value.to_le_bytes());
}
check_case("descriptor", &entry, invalid, None);
}
for offset in [1, u64::MAX] {
let mut invalid = entry.clone();
invalid.offset = offset;
check_case("source-range", &invalid, bytes.clone(), None);
}
let mut invalid = entry;
invalid.decoded_size += 1;
check_case("decoded-size", &invalid, bytes, None);
}
pub(in crate::asset) fn glyph_snapshot(bank: &GlyphBank) -> Value {
json!([
bank.tile_size as u8,
bank.width,
bank.height,
bank.pixel_indices,
bank.palettes
.iter()
.map(|p| p.iter().map(|c| c.raw()).collect::<Vec<_>>())
.collect::<Vec<_>>()
])
}
pub(in crate::asset) fn sound_snapshot(bank: &SoundBank) -> Value {
json!(
bank.samples
.iter()
.map(|s| (&s.data, s.sample_rate, s.loop_start, s.loop_end))
.collect::<Vec<_>>()
)
}
pub(in crate::asset) fn scene_snapshot(bank: &SceneBank) -> Value {
json!(
bank.layers
.iter()
.map(|l| json!([
l.active,
l.glyph_asset_id,
l.tile_size as u8,
l.parallax_factor.x,
l.parallax_factor.y,
l.tilemap.width,
l.tilemap.height,
l.tilemap
.tiles
.iter()
.map(|t| (t.active, t.glyph.glyph_id, t.glyph.palette_id, t.flip_x, t.flip_y))
.collect::<Vec<_>>()
]))
.collect::<Vec<_>>()
)
}
#[test]
fn shared_scene_arithmetic_rejects_overflow_before_allocating() {
let (_, bytes, _) = empty_scene();
let mut header = bytes[12..44].to_vec();
header[16..20].copy_from_slice(&1_u32.to_le_bytes());
header[20..24].copy_from_slice(&1_u32.to_le_bytes());
header[24..28].copy_from_slice(&1_u32.to_le_bytes());
assert!(scene_layer(&header, usize::MAX, u64::MAX, &mut 0).is_err());
let mut decoded = usize::MAX;
assert!(scene_layer(&header, 44, 48, &mut decoded).is_err());
assert!(scene_layer(&header, 44, 47, &mut 0).is_err());
assert!(scene_layer(&header[..31], 44, 48, &mut 0).is_err());
}
#[test]
fn physically_truncated_glyph_palette_is_rejected_by_both_modes() {
let fixture = FileFixture::new("truncated-glyph", &[0; 63]);
let entry = AssetEntry {
asset_id: 3,
asset_name: "empty-glyph".into(),
bank_type: BankType::GLYPH,
offset: 0,
size: 64,
decoded_size: 64,
codec: AssetCodec::None,
metadata: json!({"tile_size":8,"width":0,"height":0,"palette_count":1,"palette_authored":1}),
};
let source = fixture.source(64);
assert!(AssetManager::perform_load_glyph_bank(&entry, source.clone()).is_err());
assert!(resumable::conformance_result(&entry, &source).is_err());
std::fs::write(&fixture.0, [0; 64]).unwrap();
let expected = json!([8, 0, 0, [], vec![vec![0; 16]]]);
assert_eq!(
glyph_snapshot(&AssetManager::perform_load_glyph_bank(&entry, source.clone()).unwrap()),
expected
);
assert_eq!(resumable::conformance_result(&entry, &source).unwrap(), expected);
}
pub(in crate::asset) fn music_snapshot(bank: &prometeu_hal::music_bank::MusicBank) -> Value {
json!({"name": bank.name, "events": bank.events.len(), "pcm": bank.metrics.pcm_bytes})
}
#[test]
fn mixed_audio_fixtures_conform_across_all_asset_decode_modes() {
for (music, bytes) in [
(
false,
&include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../test-fixtures/audio/mixed-sound.sbnk"
))[..],
),
(
true,
&include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../test-fixtures/audio/mixed-music.mbnk"
))[..],
),
] {
let decoded = crate::bank_format::decode_audio_bank(bytes, music).unwrap();
let size = decoded.decoded_size();
let expected = match decoded {
DecodedAudioBank::Sound(b) => sound_snapshot(&b),
DecodedAudioBank::Music(b) => music_snapshot(&b),
};
let entry = AssetEntry {
asset_id: 7,
asset_name: "mixed".into(),
bank_type: if music { BankType::MUSIC } else { BankType::SOUNDS },
offset: 0,
size: bytes.len() as u64,
decoded_size: size as u64,
codec: AssetCodec::None,
metadata: json!({"version":1}),
};
check_case("mixed-audio", &entry, bytes.to_vec(), Some(expected));
let mut invalid = bytes.to_vec();
invalid[0] = 0;
check_case("mixed-bad-magic", &entry, invalid, None);
}
}
@@ -11,6 +11,7 @@ pub(super) struct Resources {
telemetry: Arc<Mutex<AssetPipelineTelemetryState>>, telemetry: Arc<Mutex<AssetPipelineTelemetryState>>,
gfx: BankPolicy<GlyphBank>, gfx: BankPolicy<GlyphBank>,
sound: BankPolicy<SoundBank>, sound: BankPolicy<SoundBank>,
music: BankPolicy<MusicBank>,
scene: BankPolicy<SceneBank>, scene: BankPolicy<SceneBank>,
} }
impl Resources { impl Resources {
@@ -22,6 +23,7 @@ impl Resources {
telemetry: m.pipeline_telemetry.clone(), telemetry: m.pipeline_telemetry.clone(),
gfx: m.gfx_policy.clone(), gfx: m.gfx_policy.clone(),
sound: m.sound_policy.clone(), sound: m.sound_policy.clone(),
music: m.music_policy.clone(),
scene: m.scene_policy.clone(), scene: m.scene_policy.clone(),
} }
} }
@@ -142,6 +144,18 @@ pub(super) fn schedule(
job.entry.decoded_size as usize, job.entry.decoded_size as usize,
); );
} }
Decoded::Music(bank) => {
let resident = resources.music.put_resident(
job.asset_id,
Arc::new(bank),
job.entry.decoded_size as usize,
);
resources.music.stage(
job.handle_id,
resident,
job.entry.decoded_size as usize,
);
}
Decoded::Scene(bank) => { Decoded::Scene(bank) => {
let resident = resources.scene.put_resident( let resident = resources.scene.put_resident(
job.asset_id, job.asset_id,
@@ -188,6 +202,7 @@ pub(super) fn schedule(
enum Decoded { enum Decoded {
Glyph(GlyphBank), Glyph(GlyphBank),
Sound(SoundBank), Sound(SoundBank),
Music(MusicBank),
Scene(SceneBank), Scene(SceneBank),
} }
enum State { enum State {
@@ -199,10 +214,11 @@ enum State {
palette_position: usize, palette_position: usize,
}, },
Sound { Sound {
sample_rate: u32, raw: Vec<u8>,
data: Vec<i16>,
remaining: usize, remaining: usize,
music: bool,
}, },
Audio(Box<crate::bank_format::AudioBankMaterializer<'static>>),
Scene { Scene {
layers: [SceneLayer; SCENE_LAYER_COUNT_V1], layers: [SceneLayer; SCENE_LAYER_COUNT_V1],
layer: usize, layer: usize,
@@ -245,24 +261,22 @@ impl Decoder {
palette_position: 0, palette_position: 0,
} }
} }
BankType::SOUNDS => { BankType::SOUNDS | BankType::MUSIC => {
let sample_rate = entry.metadata_as_sound_bank()?.sample_rate; let mut raw = Vec::new();
raw.try_reserve_exact(
usize::try_from(entry.size).map_err(|_| "audio size overflow")?,
)
.map_err(|_| "audio allocation failed")?;
State::Sound { State::Sound {
sample_rate, raw,
data: Vec::with_capacity(entry.size as usize / 2),
remaining: entry.size as usize, remaining: entry.size as usize,
music: entry.bank_type == BankType::MUSIC,
} }
} }
BankType::SCENE => { BankType::SCENE => {
entry.metadata_as_scene_bank()?; decode::scene_metadata(entry)?;
State::Scene { State::Scene {
layers: std::array::from_fn(|_| SceneLayer { layers: decode::empty_layers(),
active: false,
glyph_asset_id: 0,
tile_size: TileSize::Size8,
parallax_factor: ParallaxFactor { x: 1.0, y: 1.0 },
tilemap: TileMap { width: 0, height: 0, tiles: Vec::new() },
}),
layer: 0, layer: 0,
tiles_left: 0, tiles_left: 0,
header: false, header: false,
@@ -279,25 +293,18 @@ impl Decoder {
if *position < *packed { if *position < *packed {
let n = (*packed - *position).min(bytes.len()); let n = (*packed - *position).min(bytes.len());
self.reader.read_exact(&mut bytes[..n]).map_err(|e| e.to_string())?; self.reader.read_exact(&mut bytes[..n]).map_err(|e| e.to_string())?;
for &v in &bytes[..n] { decode::append_pixels(
if bank.pixel_indices.len() < bank.width * bank.height { &mut bank.pixel_indices,
bank.pixel_indices.push(v >> 4) &bytes[..n],
} bank.width * bank.height,
if bank.pixel_indices.len() < bank.width * bank.height { );
bank.pixel_indices.push(v & 15)
}
}
*position += n; *position += n;
return Ok(None); return Ok(None);
} }
if *palette_position < *palette_bytes { if *palette_position < *palette_bytes {
let n = (*palette_bytes - *palette_position).min(bytes.len()); let n = (*palette_bytes - *palette_position).min(bytes.len());
self.reader.read_exact(&mut bytes[..n]).map_err(|e| e.to_string())?; self.reader.read_exact(&mut bytes[..n]).map_err(|e| e.to_string())?;
for (i, rgba) in bytes[..n].as_chunks::<4>().0.iter().enumerate() { decode::write_palettes(&mut bank.palettes, *palette_position, &bytes[..n]);
let at = *palette_position / 4 + i;
bank.palettes[at / 16][at % 16] =
Color::rgba(rgba[0], rgba[1], rgba[2], rgba[3]);
}
*palette_position += n; *palette_position += n;
return Ok(None); return Ok(None);
} }
@@ -307,32 +314,41 @@ impl Decoder {
}; };
Ok(Some(Decoded::Glyph(bank))) Ok(Some(Decoded::Glyph(bank)))
} }
State::Sound { sample_rate, data, remaining } => { State::Sound { raw, remaining, music } => {
if *remaining > 0 { if *remaining > 0 {
let n = (*remaining).min(bytes.len()); let n = (*remaining).min(4096);
self.reader.read_exact(&mut bytes[..n]).map_err(|e| e.to_string())?; let mut buffer = [0; 4096];
for pair in bytes[..n].as_chunks::<2>().0 { self.reader.read_exact(&mut buffer[..n]).map_err(|e| e.to_string())?;
data.push(i16::from_le_bytes([pair[0], pair[1]])); raw.extend_from_slice(&buffer[..n]);
}
*remaining -= n; *remaining -= n;
return Ok(None); return Ok(None);
} }
let rate = *sample_rate; let decoder = crate::bank_format::AudioBankMaterializer::new(
let data = std::mem::take(data); std::borrow::Cow::Owned(std::mem::take(raw)),
*music,
)?;
self.state = State::Audio(Box::new(decoder));
Ok(None)
}
State::Audio(decoder) => {
let Some(bank) = decoder.step()? else {
return Ok(None);
};
if bank.decoded_size() as u64 != self.entry.decoded_size {
return Err("audio decoded_size mismatch".into());
}
self.state = State::Done; self.state = State::Done;
Ok(Some(Decoded::Sound(SoundBank::new(vec![Arc::new(Sample::new(rate, data))])))) Ok(Some(match bank {
DecodedAudioBank::Sound(b) => Decoded::Sound(b),
DecodedAudioBank::Music(b) => Decoded::Music(b),
}))
} }
State::Scene { layers, layer, tiles_left, header, decoded } => { State::Scene { layers, layer, tiles_left, header, decoded } => {
if !*header { if !*header {
self.reader self.reader
.read_exact(&mut bytes[..SCENE_HEADER_BYTES_V1]) .read_exact(&mut bytes[..SCENE_HEADER_BYTES_V1])
.map_err(|e| e.to_string())?; .map_err(|e| e.to_string())?;
if bytes[..4] != SCENE_PAYLOAD_MAGIC_V1 decode::scene_header(&bytes[..SCENE_HEADER_BYTES_V1])?;
|| u16::from_le_bytes([bytes[4], bytes[5]]) != SCENE_PAYLOAD_VERSION_V1
|| u16::from_le_bytes([bytes[6], bytes[7]]) as usize != SCENE_LAYER_COUNT_V1
{
return Err("Invalid scene header".into());
}
self.position += SCENE_HEADER_BYTES_V1; self.position += SCENE_HEADER_BYTES_V1;
*header = true; *header = true;
return Ok(None); return Ok(None);
@@ -341,17 +357,7 @@ impl Decoder {
let n = (*tiles_left).min(bytes.len() / SCENE_TILE_RECORD_BYTES_V1); let n = (*tiles_left).min(bytes.len() / SCENE_TILE_RECORD_BYTES_V1);
let count = n * SCENE_TILE_RECORD_BYTES_V1; let count = n * SCENE_TILE_RECORD_BYTES_V1;
self.reader.read_exact(&mut bytes[..count]).map_err(|e| e.to_string())?; self.reader.read_exact(&mut bytes[..count]).map_err(|e| e.to_string())?;
for b in bytes[..count].as_chunks::<SCENE_TILE_RECORD_BYTES_V1>().0 { decode::append_tiles(&mut layers[*layer - 1].tilemap.tiles, &bytes[..count]);
layers[*layer - 1].tilemap.tiles.push(Tile {
active: b[0] & 1 != 0,
glyph: Glyph {
glyph_id: u16::from_le_bytes([b[2], b[3]]),
palette_id: b[1],
},
flip_x: b[0] & 2 != 0,
flip_y: b[0] & 4 != 0,
});
}
*tiles_left -= n; *tiles_left -= n;
self.position += count; self.position += count;
return Ok(None); return Ok(None);
@@ -361,50 +367,18 @@ impl Decoder {
.read_exact(&mut bytes[..SCENE_LAYER_HEADER_BYTES_V1]) .read_exact(&mut bytes[..SCENE_LAYER_HEADER_BYTES_V1])
.map_err(|e| e.to_string())?; .map_err(|e| e.to_string())?;
self.position += SCENE_LAYER_HEADER_BYTES_V1; self.position += SCENE_LAYER_HEADER_BYTES_V1;
let u32at = |at| u32::from_le_bytes(bytes[at..at + 4].try_into().unwrap()); layers[*layer] = decode::scene_layer(
let width = u32at(16) as usize; &bytes[..SCENE_LAYER_HEADER_BYTES_V1],
let height = u32at(20) as usize; self.position,
let count = u32at(24) as usize; self.entry.size,
let tile_size = match bytes[5] { decoded,
8 => TileSize::Size8, )?;
16 => TileSize::Size16, let count = layers[*layer].tilemap.width * layers[*layer].tilemap.height;
32 => TileSize::Size32,
_ => return Err("Invalid scene tile size".into()),
};
let x = f32::from_le_bytes(bytes[8..12].try_into().unwrap());
let y = f32::from_le_bytes(bytes[12..16].try_into().unwrap());
if !x.is_finite()
|| !y.is_finite()
|| width.checked_mul(height) != Some(count)
|| count
.checked_mul(SCENE_TILE_RECORD_BYTES_V1)
.and_then(|n| n.checked_add(self.position))
.is_none_or(|n| n as u64 > self.entry.size)
{
return Err("Invalid scene layer range".into());
}
*decoded = decoded
.checked_add(SCENE_DECODED_LAYER_OVERHEAD_BYTES_V1)
.and_then(|n| {
count.checked_mul(size_of::<Tile>()).and_then(|b| n.checked_add(b))
})
.ok_or("Scene size overflow")?;
layers[*layer] = SceneLayer {
active: bytes[0] & 1 != 0,
glyph_asset_id: i32::from_le_bytes(bytes[1..5].try_into().unwrap()),
tile_size,
parallax_factor: ParallaxFactor { x, y },
tilemap: TileMap { width, height, tiles: Vec::with_capacity(count) },
};
*layer += 1; *layer += 1;
*tiles_left = count; *tiles_left = count;
return Ok(None); return Ok(None);
} }
if self.position as u64 != self.entry.size decode::scene_finish(&self.entry, self.position, *decoded)?;
|| *decoded as u64 != self.entry.decoded_size
{
return Err("Invalid scene payload size".into());
}
let State::Scene { layers, .. } = std::mem::replace(&mut self.state, State::Done) let State::Scene { layers, .. } = std::mem::replace(&mut self.state, State::Done)
else { else {
unreachable!() unreachable!()
@@ -416,6 +390,31 @@ impl Decoder {
} }
} }
// Test-only access to the production state machine; no alternate decode rules.
#[cfg(test)]
pub(super) fn conformance_result(
entry: &AssetEntry,
source: &Arc<RwLock<AssetsPayloadSource>>,
) -> Result<serde_json::Value, String> {
use std::io::Seek;
let mut decoder = Decoder::new(entry, source)?;
for _ in 0..100 {
let before = decoder.reader.stream_position().unwrap();
let result = decoder.step();
let after = decoder.reader.stream_position().unwrap();
assert!(after >= before && after - before <= 4096, "unbounded or rewound decode step");
if let Some(bank) = result? {
return Ok(match bank {
Decoded::Glyph(bank) => super::decode::tests::glyph_snapshot(&bank),
Decoded::Sound(bank) => super::decode::tests::sound_snapshot(&bank),
Decoded::Scene(bank) => super::decode::tests::scene_snapshot(&bank),
Decoded::Music(bank) => super::decode::tests::music_snapshot(&bank),
});
}
}
panic!("conformance fixture exceeded its step budget")
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -486,9 +485,12 @@ mod tests {
panic!("decoder did not finish") panic!("decoder did not finish")
} }
#[test] #[test]
fn resumable_sound_and_scene_preserve_existing_formats() { fn resumable_audio_bank_and_scene_use_canonical_formats() {
let pcm: Vec<i16> = (0..5000).map(|n| n as i16 - 2500).collect(); let pcm: Vec<i16> = (0..5000).map(|n| n as i16 - 2500).collect();
let bytes: Vec<u8> = pcm.iter().flat_map(|n| n.to_le_bytes()).collect(); let bytes = crate::bank_format::test_pcm_bank(
&pcm.iter().flat_map(|n| n.to_le_bytes()).collect::<Vec<_>>(),
44100,
);
let entry = AssetEntry { let entry = AssetEntry {
asset_id: 1, asset_id: 1,
asset_name: "pcm".into(), asset_name: "pcm".into(),
@@ -523,7 +525,7 @@ mod tests {
} }
#[test] #[test]
fn shared_preload_waits_for_worker_and_frame_publication() { fn shared_preload_waits_for_worker_and_frame_publication() {
let bytes = vec![0; 8192]; let bytes = crate::bank_format::test_pcm_bank(&[0; 8192], 44100);
let entry = AssetEntry { let entry = AssetEntry {
asset_id: 1, asset_id: 1,
asset_name: "pcm".into(), asset_name: "pcm".into(),
@@ -556,4 +558,55 @@ mod tests {
assert_eq!(manager.preloads_ready(), Ok(true)); assert_eq!(manager.preloads_ready(), Ok(true));
assert_eq!(manager.slot_info(SlotRef::audio(0)).asset_id, Some(1)); assert_eq!(manager.slot_info(SlotRef::audio(0)).asset_id, Some(1));
} }
#[test]
fn canceled_and_superseded_partial_decoders_never_publish() {
use crate::memory_banks::SoundBankPoolAccess;
let bytes = crate::bank_format::test_pcm_bank(&[0x12; 16384], 44100);
let entry = AssetEntry {
asset_id: 1,
asset_name: "pcm".into(),
bank_type: BankType::SOUNDS,
offset: 0,
size: bytes.len() as u64,
decoded_size: bytes.len() as u64,
codec: AssetCodec::None,
metadata: serde_json::json!({"sample_rate":44100,"channels":1}),
};
let mut newer = entry.clone();
newer.asset_id = 2;
let banks = Arc::new(crate::memory_banks::MemoryBanks::new());
let mut manager = AssetManager::new(
vec![entry, newer],
AssetsPayloadSource::from_bytes(bytes),
banks.clone(),
banks.clone(),
banks.clone(),
);
let lane = Arc::new(ManualLane::default());
assert!(manager.bind_background_work(lane.clone()));
let handle = manager.load(1, 0).unwrap();
assert!(lane.step());
assert!(lane.step());
assert_eq!(manager.status(handle), LoadStatus::LOADING);
assert_eq!(manager.cancel(handle), AssetOpStatus::Ok);
while lane.step() {}
assert_eq!(manager.status(handle), LoadStatus::CANCELED);
assert!(manager.sound_policy.staging.read().unwrap().is_empty());
manager.apply_commits();
assert!(banks.sound_bank_slot(0).is_none());
let old = manager.load(1, 0).unwrap();
assert!(lane.step());
assert!(lane.step());
let new = manager.load(2, 0).unwrap();
assert_eq!(old, new); // Stable slot handle, different request generation.
assert!(lane.step()); // Discard the in-flight old decoder.
assert!(manager.sound_policy.staging.read().unwrap().is_empty());
while lane.step() {}
assert_eq!(manager.status(new), LoadStatus::READY);
assert!(banks.sound_bank_slot(0).is_none());
assert_eq!(manager.commit(new), AssetOpStatus::Ok);
manager.apply_commits();
assert_eq!(manager.slot_info(SlotRef::audio(0)).asset_id, Some(2));
assert_eq!(banks.sound_bank_slot(0).unwrap().samples[0].data, vec![0x1212; 8192]);
}
} }
+103 -17
View File
@@ -1,8 +1,14 @@
use prometeu_hal::{AudioBridge, AudioOpStatus}; use prometeu_hal::music_bank::MusicBank;
use prometeu_hal::{AudioBridge, AudioOpStatus, audio_bridge::AudioControl};
use std::sync::Arc; use std::sync::Arc;
use std::sync::atomic::AtomicBool;
pub mod bank_format;
/// Maximum number of simultaneous audio voices supported by the hardware. /// Maximum number of simultaneous audio voices supported by the hardware.
pub const MAX_CHANNELS: usize = 16; pub const MAX_CHANNELS: usize = 32;
pub const SFX_CHANNELS: usize = 8;
pub const SFX_START: usize = 24;
pub mod engine;
/// Standard sample rate for the final audio output. /// Standard sample rate for the final audio output.
pub const OUTPUT_SAMPLE_RATE: u32 = 48000; pub const OUTPUT_SAMPLE_RATE: u32 = 48000;
@@ -19,13 +25,14 @@ pub struct Channel {
pub sample: Option<Arc<Sample>>, pub sample: Option<Arc<Sample>>,
/// Whether this channel is currently active. /// Whether this channel is currently active.
pub active: bool, pub active: bool,
pub alive: Option<Arc<AtomicBool>>,
/// Current playback position within the sample (fractional for pitch shifting). /// Current playback position within the sample (fractional for pitch shifting).
pub pos: f64, pub pos: f64,
/// Playback speed multiplier (1.0 = original speed). /// Playback speed multiplier (1.0 = original speed).
pub pitch: f64, pub pitch: f64,
/// Voice volume (0-255). /// Voice volume (0-255).
pub volume: u8, pub volume: u8,
/// Stereo panning (0=Full Left, 127=Center, 255=Full Right). /// Stereo panning (0=Full Left, 128=Center, 255=Full Right).
pub pan: u8, pub pan: u8,
/// Loop configuration for this voice. /// Loop configuration for this voice.
pub loop_mode: LoopMode, pub loop_mode: LoopMode,
@@ -38,10 +45,11 @@ impl Default for Channel {
Self { Self {
sample: None, sample: None,
active: false, active: false,
alive: None,
pos: 0.0, pos: 0.0,
pitch: 1.0, pitch: 1.0,
volume: 255, volume: 255,
pan: 127, pan: 128,
loop_mode: LoopMode::Off, loop_mode: LoopMode::Off,
priority: 0, priority: 0,
} }
@@ -53,9 +61,16 @@ impl Default for Channel {
/// Because the Core logic runs at 60Hz and Audio is generated at 48kHz, /// Because the Core logic runs at 60Hz and Audio is generated at 48kHz,
/// we use an asynchronous command queue to synchronize them. /// we use an asynchronous command queue to synchronize them.
pub enum AudioCommand { pub enum AudioCommand {
MusicPlay {
group: usize,
bank: Arc<MusicBank>,
looping: bool,
},
Control(AudioControl),
/// Start playing a sample on a specific voice. /// Start playing a sample on a specific voice.
Play { Play {
sample: Arc<Sample>, sample: Arc<Sample>,
alive: Option<Arc<AtomicBool>>,
voice_id: usize, voice_id: usize,
volume: u8, volume: u8,
pan: u8, pan: u8,
@@ -64,13 +79,24 @@ pub enum AudioCommand {
loop_mode: LoopMode, loop_mode: LoopMode,
}, },
/// Stop playback on a voice when this command is applied. /// Stop playback on a voice when this command is applied.
Stop { voice_id: usize }, Stop {
voice_id: usize,
},
/// Update volume of an ongoing playback. /// Update volume of an ongoing playback.
SetVolume { voice_id: usize, volume: u8 }, SetVolume {
voice_id: usize,
volume: u8,
},
/// Update panning of an ongoing playback. /// Update panning of an ongoing playback.
SetPan { voice_id: usize, pan: u8 }, SetPan {
voice_id: usize,
pan: u8,
},
/// Update pitch of an ongoing playback. /// Update pitch of an ongoing playback.
SetPitch { voice_id: usize, pitch: f64 }, SetPitch {
voice_id: usize,
pitch: f64,
},
} }
mod transport; mod transport;
@@ -109,7 +135,7 @@ impl Audio {
} }
fn enqueue(&mut self, voice: usize, command: AudioCommand) -> AudioOpStatus { fn enqueue(&mut self, voice: usize, command: AudioCommand) -> AudioOpStatus {
if voice >= MAX_CHANNELS { if voice >= SFX_CHANNELS {
return AudioOpStatus::VoiceInvalid; return AudioOpStatus::VoiceInvalid;
} }
let admitted = if self.in_lifecycle { let admitted = if self.in_lifecycle {
@@ -142,7 +168,7 @@ impl AudioBridge for Audio {
priority: u8, priority: u8,
loop_mode: LoopMode, loop_mode: LoopMode,
) -> AudioOpStatus { ) -> AudioOpStatus {
if voice_id >= MAX_CHANNELS { if voice_id >= SFX_CHANNELS {
return AudioOpStatus::VoiceInvalid; return AudioOpStatus::VoiceInvalid;
} }
if !pitch.is_finite() || pitch <= 0.0 { if !pitch.is_finite() || pitch <= 0.0 {
@@ -154,7 +180,19 @@ impl AudioBridge for Audio {
let Some(sample) = bank.samples.get(sample_id as usize).cloned() else { let Some(sample) = bank.samples.get(sample_id as usize).cloned() else {
return AudioOpStatus::SampleNotFound; return AudioOpStatus::SampleNotFound;
}; };
self.play_sample(sample, voice_id, volume, pan, pitch, priority, loop_mode) self.enqueue(
voice_id,
AudioCommand::Play {
sample,
alive: Some(bank.alive.clone()),
voice_id,
volume,
pan,
pitch,
priority,
loop_mode,
},
)
} }
fn play_sample( fn play_sample(
@@ -167,7 +205,7 @@ impl AudioBridge for Audio {
priority: u8, priority: u8,
loop_mode: LoopMode, loop_mode: LoopMode,
) -> AudioOpStatus { ) -> AudioOpStatus {
if voice_id >= MAX_CHANNELS { if voice_id >= SFX_CHANNELS {
return AudioOpStatus::VoiceInvalid; return AudioOpStatus::VoiceInvalid;
} }
if !pitch.is_finite() || pitch <= 0.0 { if !pitch.is_finite() || pitch <= 0.0 {
@@ -175,10 +213,57 @@ impl AudioBridge for Audio {
} }
self.enqueue( self.enqueue(
voice_id, voice_id,
AudioCommand::Play { sample, voice_id, volume, pan, pitch, priority, loop_mode }, AudioCommand::Play {
sample,
alive: None,
voice_id,
volume,
pan,
pitch,
priority,
loop_mode,
},
) )
} }
fn music_play(&mut self, group: usize, bank: usize, looping: bool) -> AudioOpStatus {
if group > 1 {
return AudioOpStatus::ArgRangeInvalid;
}
let Some(bank) = self.sound_banks.music_bank_slot(bank) else {
return AudioOpStatus::BankInvalid;
};
self.enqueue(0, AudioCommand::MusicPlay { group, bank, looping })
}
fn control(&mut self, control: AudioControl) -> AudioOpStatus {
let valid = match control {
AudioControl::MusicStop { group }
| AudioControl::MusicPause { group, .. }
| AudioControl::MusicVolume { group, .. }
| AudioControl::MusicFade { group, .. } => group < 2,
AudioControl::Crossfade { source, target, .. } => {
source < 2 && target < 2 && source != target
}
AudioControl::BusVolume { bus, .. } => bus < 3,
AudioControl::GroupVolume { group, .. } => group < 3,
AudioControl::SfxSend { voice, .. } => voice < SFX_CHANNELS,
AudioControl::Effect { group, cutoff_hz, delay_frames, feedback, wet, dry } => {
group < 3
&& cutoff_hz <= OUTPUT_SAMPLE_RATE / 2
&& delay_frames <= 96_000
&& feedback.is_finite()
&& feedback.abs() < 1.0
&& wet.is_finite()
&& dry.is_finite()
&& (0.0..=1.0).contains(&wet)
&& (0.0..=1.0).contains(&dry)
}
};
if !valid {
return AudioOpStatus::ArgRangeInvalid;
}
self.enqueue(0, AudioCommand::Control(control))
}
fn stop(&mut self, voice_id: usize) -> AudioOpStatus { fn stop(&mut self, voice_id: usize) -> AudioOpStatus {
self.enqueue(voice_id, AudioCommand::Stop { voice_id }) self.enqueue(voice_id, AudioCommand::Stop { voice_id })
} }
@@ -189,7 +274,7 @@ impl AudioBridge for Audio {
self.enqueue(voice_id, AudioCommand::SetPan { voice_id, pan }) self.enqueue(voice_id, AudioCommand::SetPan { voice_id, pan })
} }
fn set_pitch(&mut self, voice_id: usize, pitch: f64) -> AudioOpStatus { fn set_pitch(&mut self, voice_id: usize, pitch: f64) -> AudioOpStatus {
if voice_id >= MAX_CHANNELS { if voice_id >= SFX_CHANNELS {
return AudioOpStatus::VoiceInvalid; return AudioOpStatus::VoiceInvalid;
} }
if !pitch.is_finite() || pitch <= 0.0 { if !pitch.is_finite() || pitch <= 0.0 {
@@ -203,7 +288,8 @@ impl AudioBridge for Audio {
fn begin_frame(&mut self) { /* Staging may already contain debugger steps. */ fn begin_frame(&mut self) { /* Staging may already contain debugger steps. */
} }
fn finish_frame(&mut self) { fn finish_frame(&mut self) {
self.transport.publish(std::mem::take(&mut self.commands), false); let commands = std::mem::replace(&mut self.commands, self.transport.take_buffer());
self.transport.publish(commands, false);
self.frame_commands_admitted = 0; self.frame_commands_admitted = 0;
} }
fn abort_frame(&mut self) { fn abort_frame(&mut self) {
@@ -221,7 +307,7 @@ impl AudioBridge for Audio {
if success { if success {
// Publish the earlier frame prefix first so a callback stop cannot // Publish the earlier frame prefix first so a callback stop cannot
// be undone by an old play on resume. Do not replenish frame quota. // be undone by an old play on resume. Do not replenish frame quota.
let mut commands = std::mem::take(&mut self.commands); let mut commands = std::mem::replace(&mut self.commands, self.transport.take_buffer());
commands.append(&mut self.lifecycle_commands); commands.append(&mut self.lifecycle_commands);
self.transport.publish(commands, true); self.transport.publish(commands, true);
} else { } else {
@@ -310,7 +396,7 @@ mod control_tests {
for _ in 0..MAX_COMMANDS_PER_FRAME - 1 { for _ in 0..MAX_COMMANDS_PER_FRAME - 1 {
assert_eq!(audio.stop(0), AudioOpStatus::Ok); assert_eq!(audio.stop(0), AudioOpStatus::Ok);
} }
assert_eq!(audio.set_volume(15, 0), AudioOpStatus::Ok); assert_eq!(audio.set_volume(7, 0), AudioOpStatus::Ok);
assert_eq!(audio.set_pan(0, 255), AudioOpStatus::CommandLimit); assert_eq!(audio.set_pan(0, 255), AudioOpStatus::CommandLimit);
assert_eq!(audio.set_pitch(16, f64::NAN), AudioOpStatus::VoiceInvalid); assert_eq!(audio.set_pitch(16, f64::NAN), AudioOpStatus::VoiceInvalid);
assert_eq!(audio.set_pitch(0, f64::NAN), AudioOpStatus::ArgRangeInvalid); assert_eq!(audio.set_pitch(0, f64::NAN), AudioOpStatus::ArgRangeInvalid);
@@ -0,0 +1,592 @@
//! SBNK/MBNK v1. IMA WAVE4 algorithm implemented here from the published Intel
//! format, not copied from a third-party decoder. Project MIT license applies.
use prometeu_hal::{
music_bank::{Instrument, MusicBank, MusicCompiler, MusicEvent},
sample::Sample,
sound_bank::{AudioBankMetrics, SoundBank},
};
use std::sync::Arc;
pub const HEADER_BYTES: usize = 48;
pub const SAMPLE_BYTES: usize = 40;
pub const INSTRUMENT_BYTES: usize = 64;
pub const EVENT_BYTES: usize = 32;
pub enum DecodedAudioBank {
Sound(SoundBank),
Music(MusicBank),
}
impl DecodedAudioBank {
pub fn decoded_size(&self) -> usize {
match self {
Self::Sound(b) => b.metrics.canonical_decoded_bytes,
Self::Music(b) => b.metrics.canonical_decoded_bytes,
}
}
}
fn bytes(data: &[u8], offset: usize, len: usize) -> Result<&[u8], String> {
data.get(offset..offset.checked_add(len).ok_or("audio offset overflow")?)
.ok_or_else(|| "truncated audio bank".into())
}
fn u16le(b: &[u8], o: usize) -> Result<u16, String> {
Ok(u16::from_le_bytes(bytes(b, o, 2)?.try_into().unwrap()))
}
fn u32le(b: &[u8], o: usize) -> Result<u32, String> {
Ok(u32::from_le_bytes(bytes(b, o, 4)?.try_into().unwrap()))
}
fn zero(b: &[u8]) -> Result<(), String> {
if b.iter().any(|&x| x != 0) { Err("nonzero audio reserved bytes".into()) } else { Ok(()) }
}
fn vector<T>(n: usize) -> Result<Vec<T>, String> {
let mut v = Vec::new();
v.try_reserve_exact(n).map_err(|_| "audio allocation failed")?;
Ok(v)
}
/// One bank owns its serialized bytes while materializing. IO and CPU preparation
/// yield separately; no partially decoded bank escapes this object.
pub struct AudioBankMaterializer<'a> {
data: std::borrow::Cow<'a, [u8]>,
music: bool,
ns: usize,
ni: usize,
ne: usize,
ppq: u32,
tempo: u32,
duration: u32,
loop_tick: Option<u32>,
tracks: usize,
name: String,
samples_at: usize,
metadata: usize,
payload_end: usize,
samples: Vec<Arc<Sample>>,
instruments: Vec<Instrument>,
events: Vec<MusicEvent>,
current: Option<SampleMaterializer>,
compiler: Option<MusicCompiler>,
pcm_bytes: usize,
started: std::time::Instant,
}
struct SampleMaterializer {
sample: Sample,
codec: u8,
frames: usize,
offset: usize,
size: usize,
align: usize,
spb: usize,
consumed: usize,
}
impl<'a> AudioBankMaterializer<'a> {
pub fn new(data: std::borrow::Cow<'a, [u8]>, music: bool) -> Result<Self, String> {
let started = std::time::Instant::now();
bytes(&data, 0, HEADER_BYTES)?;
if &data[..4] != if music { b"MBNK" } else { b"SBNK" }
|| u16le(&data, 4)? != 1
|| u16le(&data, 6)? != 0
{
return Err("unsupported audio bank magic/version/flags".into());
}
let ns = u32le(&data, 8)? as usize;
let ni = u32le(&data, 12)? as usize;
let ne = u32le(&data, 16)? as usize;
let ppq = u32le(&data, 20)?;
let tempo = u32le(&data, 24)?;
let duration = u32le(&data, 28)?;
let lp = u32le(&data, 32)?;
let tracks = u32le(&data, 36)? as usize;
let name_len = u32le(&data, 40)? as usize;
zero(bytes(&data, 44, 4)?)?;
if ns > 65536
|| ppq != 960
|| tempo == 0
|| (!music
&& (ns == 0
|| ni != 0
|| ne != 0
|| duration != 0
|| tracks != 0
|| lp != u32::MAX))
{
return Err("invalid audio bank counts/clock".into());
}
if music
&& (duration == 0 || !(1..=256).contains(&tracks) || (lp != u32::MAX && lp >= duration))
{
return Err("invalid music clock/tracks/loop".into());
}
let mut offset = HEADER_BYTES.checked_add(name_len).ok_or("audio name overflow")?;
let samples_at = offset;
for (n, size) in [(ns, SAMPLE_BYTES), (ni, INSTRUMENT_BYTES), (ne, EVENT_BYTES)] {
offset = offset
.checked_add(n.checked_mul(size).ok_or("audio table overflow")?)
.ok_or("audio table overflow")?;
}
bytes(&data, 0, offset)?;
let text = std::str::from_utf8(bytes(&data, HEADER_BYTES, name_len)?)
.map_err(|_| "audio name UTF-8")?;
let mut name = String::new();
name.try_reserve_exact(text.len()).map_err(|_| "audio allocation failed")?;
name.push_str(text);
Ok(Self {
data,
music,
ns,
ni,
ne,
ppq,
tempo,
duration,
loop_tick: (lp != u32::MAX).then_some(lp),
tracks,
name,
samples_at,
metadata: offset,
payload_end: offset,
samples: vector(ns)?,
instruments: vector(ni)?,
events: vector(ne)?,
current: None,
compiler: None,
pcm_bytes: 0,
started,
})
}
fn start_sample(&mut self) -> Result<(), String> {
let d =
bytes(&self.data, self.samples_at + self.samples.len() * SAMPLE_BYTES, SAMPLE_BYTES)?;
let codec = d[0];
let channels = d[1];
let flags = u16le(d, 2)?;
let rate = u32le(d, 4)?;
let frames = u32le(d, 8)? as usize;
let start = u32le(d, 12)?;
let end = u32le(d, 16)?;
let offset = u32le(d, 20)? as usize;
let size = u32le(d, 24)? as usize;
let align = u16le(d, 28)? as usize;
let spb = u16le(d, 30)? as usize;
let pitch = u32le(d, 36)?;
zero(bytes(d, 34, 2)?)?;
if !(1..=2).contains(&channels)
|| rate == 0
|| pitch == 0
|| flags & !1 != 0
|| offset != self.payload_end
|| (flags == 0 && (start != 0 || end != 0))
|| (flags == 1 && (start >= end || end as usize > frames))
{
return Err("invalid audio sample descriptor/loop".into());
}
let scalar = frames.checked_mul(channels as usize).ok_or("PCM frame overflow")?;
let resident = scalar.checked_mul(2).ok_or("PCM byte overflow")?;
bytes(&self.data, offset, size)?;
match codec {
0 => {
if size != resident || align != 0 || spb != 0 {
return Err("PCM payload/frame count mismatch".into());
}
}
1 => {
let c = channels as usize;
if align < 8 * c
|| !align.is_multiple_of(4 * c)
|| spb != 1 + (align - 4 * c) * 2 / c
|| !size.is_multiple_of(align)
|| size / align != frames.div_ceil(spb)
{
return Err("IMA block layout/frame count".into());
}
}
_ => return Err("unsupported sample codec".into()),
}
let mut sample = Sample::new(rate, vector(scalar)?).with_channels(channels)?;
if flags == 1 {
sample = sample.with_loop(start, end);
}
sample.default_volume = d[32];
sample.default_pan = d[33];
sample.default_pitch = pitch as f64 / 65536.0;
self.pcm_bytes =
self.pcm_bytes.checked_add(resident).ok_or("resident audio size overflow")?;
self.current = Some(SampleMaterializer {
sample,
codec,
frames,
offset,
size,
align,
spb,
consumed: 0,
});
Ok(())
}
/// Decode at most 4096 PCM bytes or one independently bounded IMA block;
/// parse at most 64 metadata records; compile at most 64 music records.
pub fn step(&mut self) -> Result<Option<DecodedAudioBank>, String> {
if let Some(compiler) = &mut self.compiler {
return match compiler.step()? {
Some(mut bank) => {
bank.metrics = self.metrics()?;
Ok(Some(DecodedAudioBank::Music(bank)))
}
None => Ok(None),
};
}
if self.samples.len() < self.ns {
if self.current.is_none() {
self.start_sample()?;
}
let current = self.current.as_mut().unwrap();
if current.consumed < current.size {
if current.codec == 0 {
let count = (current.size - current.consumed).min(4096);
let part = bytes(&self.data, current.offset + current.consumed, count)?;
current
.sample
.data
.extend(part.as_chunks::<2>().0.iter().map(|p| i16::from_le_bytes(*p)));
current.consumed += count;
} else {
let frames = (current.frames - current.sample.frames_len()).min(current.spb);
let pcm = decode_ima_wav4(
bytes(&self.data, current.offset + current.consumed, current.align)?,
current.sample.channels,
frames,
current.align,
current.spb,
)?;
current.sample.data.extend(pcm);
current.consumed += current.align;
}
}
if current.consumed == current.size {
self.payload_end = current.offset + current.size;
let current = self.current.take().unwrap();
self.samples.push(Arc::new(current.sample));
}
return Ok(None);
}
if self.payload_end != self.data.len() {
return Err("audio trailing/unreferenced payload".into());
}
if !self.music {
let mut bank = SoundBank::new(std::mem::take(&mut self.samples));
bank.name = std::mem::take(&mut self.name);
bank.metrics = self.metrics()?;
return Ok(Some(DecodedAudioBank::Sound(bank)));
}
let inst_at = self.samples_at + self.ns * SAMPLE_BYTES;
if self.instruments.len() < self.ni {
for _ in 0..64 {
if self.instruments.len() == self.ni {
break;
}
let d = bytes(
&self.data,
inst_at + self.instruments.len() * INSTRUMENT_BYTES,
INSTRUMENT_BYTES,
)?;
if d[7] & !1 != 0 {
return Err("instrument flags".into());
}
zero(bytes(d, 22, 2)?)?;
zero(bytes(d, 43, 21)?)?;
self.instruments.push(Instrument {
id: u16le(d, 0)?,
sample: u16le(d, 2)?,
low: d[4],
high: d[5],
root: d[6],
looping: d[7] & 1 != 0,
tuning_cents: u16le(d, 8)? as i16,
pan: d[10],
gain: d[11],
attack_us: u32le(d, 12)?,
decay_us: u32le(d, 16)?,
sustain: u16le(d, 20)?,
release_us: u32le(d, 24)?,
vibrato_millihz: u32le(d, 28)?,
vibrato_cents: u16le(d, 32)?,
tremolo_depth: u16le(d, 34)?,
tremolo_millihz: u32le(d, 36)?,
send: u16le(d, 40)?,
curve: d[42],
});
}
return Ok(None);
}
if self.events.len() < self.ne {
let at = inst_at + self.ni * INSTRUMENT_BYTES;
for _ in 0..64 {
if self.events.len() == self.ne {
break;
}
let d = bytes(&self.data, at + self.events.len() * EVENT_BYTES, EVENT_BYTES)?;
self.events.push(MusicEvent {
tick: u32le(d, 0)?,
kind: d[4],
track: d[5],
flags: u16le(d, 6)?,
a: u32le(d, 8)?,
b: u32le(d, 12)?,
c: u32le(d, 16)? as i32,
d: u32le(d, 20)? as i32,
duration_frames: u32le(d, 24)?,
curve: u32le(d, 28)?,
});
}
return Ok(None);
}
self.compiler = Some(MusicCompiler::new(
std::mem::take(&mut self.name),
std::mem::take(&mut self.samples),
std::mem::take(&mut self.instruments),
std::mem::take(&mut self.events),
self.ppq,
self.tempo,
self.duration,
self.loop_tick,
self.tracks,
)?);
Ok(None)
}
fn metrics(&self) -> Result<AudioBankMetrics, String> {
Ok(AudioBankMetrics {
cartridge_bytes: self.data.len(),
pcm_bytes: self.pcm_bytes,
canonical_decoded_bytes: self
.metadata
.checked_add(self.pcm_bytes)
.ok_or("decoded audio size overflow")?,
materialization_ns: self.started.elapsed().as_nanos().min(u64::MAX as u128) as u64,
})
}
}
pub fn decode_audio_bank(data: &[u8], music: bool) -> Result<DecodedAudioBank, String> {
let mut decoder = AudioBankMaterializer::new(std::borrow::Cow::Borrowed(data), music)?;
loop {
if let Some(bank) = decoder.step()? {
return Ok(bank);
}
}
}
const STEP: [i32; 89] = [
7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 21, 23, 25, 28, 31, 34, 37, 41, 45, 50, 55, 60, 66,
73, 80, 88, 97, 107, 118, 130, 143, 157, 173, 190, 209, 230, 253, 279, 307, 337, 371, 408, 449,
494, 544, 598, 658, 724, 796, 876, 963, 1060, 1166, 1282, 1411, 1552, 1707, 1878, 2066, 2272,
2499, 2749, 3024, 3327, 3660, 4026, 4428, 4871, 5358, 5894, 6484, 7132, 7845, 8630, 9493,
10442, 11487, 12635, 13899, 15289, 16818, 18500, 20350, 22385, 24623, 27086, 29794, 32767,
];
const INDEX: [i32; 8] = [-1, -1, -1, -1, 2, 4, 6, 8];
fn ima_code(code: u8, predictor: &mut i32, index: &mut usize) -> i16 {
let step = STEP[*index];
let mut diff = step >> 3;
if code & 1 != 0 {
diff += step >> 2;
}
if code & 2 != 0 {
diff += step >> 1;
}
if code & 4 != 0 {
diff += step;
}
*predictor = (*predictor + if code & 8 != 0 { -diff } else { diff }).clamp(-32768, 32767);
*index = (*index as i32 + INDEX[(code & 7) as usize]).clamp(0, 88) as usize;
*predictor as i16
}
pub fn decode_ima_wav4(
data: &[u8],
channels: u8,
frames: usize,
align: usize,
spb: usize,
) -> Result<Vec<i16>, String> {
let c = channels as usize;
if !(1..=2).contains(&channels)
|| align > u16::MAX as usize
|| align < 8 * c
|| !align.is_multiple_of(4 * c)
|| spb != 1 + (align - 4 * c) * 2 / c
|| !data.len().is_multiple_of(align)
|| data.len() / align != frames.div_ceil(spb)
{
return Err("invalid IMA WAVE4 block/frame layout".into());
}
let mut out = vector(frames.checked_mul(c).ok_or("IMA frame overflow")?)?;
for (block_index, block) in data.chunks_exact(align).enumerate() {
let mut pred = [0i32; 2];
let mut index = [0usize; 2];
for ch in 0..c {
pred[ch] = i16::from_le_bytes([block[ch * 4], block[ch * 4 + 1]]) as i32;
index[ch] = block[ch * 4 + 2] as usize;
if index[ch] > 88 || block[ch * 4 + 3] != 0 {
return Err("IMA predictor index/reserved".into());
}
out.push(pred[ch] as i16);
}
let remaining = (frames - block_index * spb).min(spb);
for group in 0..(spb - 1) / 8 {
let mut decoded = [[0i16; 8]; 2];
for ch in 0..c {
for n in 0..8 {
let b = block[4 * c + group * 4 * c + ch * 4 + n / 2];
let code = (b >> ((n % 2) * 4)) & 15;
if 1 + group * 8 + n >= remaining && code != 0 {
return Err("nonzero IMA padding".into());
}
decoded[ch][n] = ima_code(code, &mut pred[ch], &mut index[ch]);
}
}
for n in 0..8 {
if 1 + group * 8 + n < remaining {
for channel in decoded.iter().take(c) {
out.push(channel[n]);
}
}
}
}
}
Ok(out)
}
/// Deterministic storage encoder used by conformance tools/fixtures (never RT).
pub fn encode_ima_wav4(pcm: &[i16], channels: u8, align: usize) -> Result<Vec<u8>, String> {
let c = channels as usize;
if !(1..=2).contains(&channels)
|| !pcm.len().is_multiple_of(c)
|| align > u16::MAX as usize
|| align < 8 * c
|| !align.is_multiple_of(4 * c)
{
return Err("IMA encoder shape".into());
}
let spb = 1 + (align - 4 * c) * 2 / c;
if spb > u16::MAX as usize {
return Err("IMA block exceeds descriptor range".into());
}
let frames = pcm.len() / c;
let nblocks = frames.div_ceil(spb);
let mut out = vector(nblocks.checked_mul(align).ok_or("IMA encoded size overflow")?)?;
for block in 0..nblocks {
let first = block * spb;
let count = (frames - first).min(spb);
let mut pred = [0i32; 2];
let mut index = [0usize; 2];
for ch in 0..c {
pred[ch] = pcm[first * c + ch] as i32;
let n = (count - 1).min(32);
let sum: i32 = (1..=n)
.map(|i| {
(pcm[(first + i) * c + ch] as i32 - pcm[(first + i - 1) * c + ch] as i32).abs()
})
.sum();
let mean = if n == 0 { 0 } else { sum / n as i32 };
index[ch] = STEP.iter().position(|&s| s >= mean).unwrap_or(88);
out.extend_from_slice(&(pred[ch] as i16).to_le_bytes());
out.push(index[ch] as u8);
out.push(0);
}
for group in 0..(spb - 1) / 8 {
for ch in 0..c {
for byte in 0..4 {
let mut packed = 0;
for nibble in 0..2 {
let f = 1 + group * 8 + byte * 2 + nibble;
let code = if f >= count {
0
} else {
let diff = pcm[(first + f) * c + ch] as i32 - pred[ch];
let mut magnitude = diff.abs();
let step = STEP[index[ch]];
let mut code = if diff < 0 { 8 } else { 0 };
if magnitude >= step {
code |= 4;
magnitude -= step;
}
if magnitude >= step >> 1 {
code |= 2;
magnitude -= step >> 1;
}
if magnitude >= step >> 2 {
code |= 1;
}
code
};
ima_code(code, &mut pred[ch], &mut index[ch]);
packed |= code << (nibble * 4);
}
out.push(packed);
}
}
}
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ima_known_nibbles_and_stereo_headers() {
let block = [0, 0, 0, 0, 0x11, 0, 0, 0];
assert_eq!(decode_ima_wav4(&block, 1, 3, 8, 9).unwrap(), [0, 1, 2]);
let mut stereo = vec![0, 0, 0, 0, 100, 0, 0, 0];
stereo.extend([0u8; 8]);
let decoded = decode_ima_wav4(&stereo, 2, 9, 16, 9).unwrap();
assert_eq!(&decoded[..6], [0, 100, 0, 100, 0, 100]);
stereo[2] = 89;
assert!(decode_ima_wav4(&stereo, 2, 9, 16, 9).is_err());
}
#[test]
fn ima_partial_block_padding_round_trip_is_deterministic() {
let pcm: Vec<i16> = (0..1111)
.flat_map(|i| {
[
((i as f64 * 0.03).sin() * 12000.0) as i16,
((i as f64 * 0.02).cos() * 9000.0) as i16,
]
})
.collect();
let encoded = encode_ima_wav4(&pcm, 2, 512).unwrap();
assert_eq!(encoded, encode_ima_wav4(&pcm, 2, 512).unwrap());
let decoded = decode_ima_wav4(&encoded, 2, 1111, 512, 505).unwrap();
assert_eq!(decoded.len(), pcm.len());
assert_eq!(decoded[1010], pcm[1010]);
assert!(decode_ima_wav4(&encoded[..encoded.len() - 1], 2, 1111, 512, 505).is_err());
let error: f64 =
pcm.iter().zip(decoded).map(|(&a, b)| (a as f64 - b as f64).powi(2)).sum::<f64>()
/ pcm.len() as f64;
assert!(error.sqrt() < 400.0, "RMS error {error}");
}
}
#[cfg(test)]
pub(crate) fn test_pcm_bank(raw: &[u8], rate: u32) -> Vec<u8> {
let mut bytes = vec![0u8; 88];
bytes[..4].copy_from_slice(b"SBNK");
bytes[4] = 1;
for (at, value) in [
(8, 1),
(20, 960),
(24, 500000),
(32, u32::MAX),
(52, rate),
(56, (raw.len() / 2) as u32),
(68, 88),
(72, raw.len() as u32),
(84, 65536),
] {
bytes[at..at + 4].copy_from_slice(&value.to_le_bytes());
}
bytes[49] = 1;
bytes[80] = 255;
bytes[81] = 128;
bytes.extend_from_slice(raw);
bytes
}
#[cfg(test)]
#[path = "bank_tests.rs"]
mod bank_tests;
@@ -0,0 +1,182 @@
use super::*;
use crate::{AssetManager, MemoryBanks, SoundBankPoolAccess};
use prometeu_hal::{
asset::{AssetCodec, AssetEntry, AssetOpStatus, BankType, LoadStatus, SlotRef},
cartridge::AssetsPayloadSource,
};
use std::time::{Duration, Instant};
fn fixtures() -> Vec<(&'static [u8], bool)> {
vec![
(
include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../test-fixtures/audio/pcm-only.sbnk"
)),
false,
),
(
include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../test-fixtures/audio/ima-only.sbnk"
)),
false,
),
(
include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../test-fixtures/audio/mixed-sound.sbnk"
)),
false,
),
(
include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../test-fixtures/audio/mixed-music.mbnk"
)),
true,
),
]
}
#[test]
fn packer_binary_fixtures_materialize_mixed_stereo_banks_and_separate_encoded_from_pcm_sizes() {
for (index, (bytes, music)) in fixtures().into_iter().enumerate() {
let decoded = decode_audio_bank(bytes, music).unwrap();
let (samples, metrics) = match &decoded {
DecodedAudioBank::Sound(b) => (&b.samples, b.metrics),
DecodedAudioBank::Music(b) => {
assert_eq!(b.events.len(), 4);
assert_eq!(b.instruments.len(), 2);
assert_eq!(b.tracks, 2);
(&b.samples, b.metrics)
}
};
assert_eq!(metrics.cartridge_bytes, bytes.len());
assert_eq!(metrics.pcm_bytes, if index < 2 { 2000 } else { 6000 });
assert_eq!(samples[0].frames_len(), 1000);
assert_eq!(samples[0].loop_end, Some(1000));
if index >= 2 {
assert_eq!(samples[1].channels, 2);
assert_eq!(samples[1].frames_len(), 1000);
}
if index != 0 {
assert!(decoded.decoded_size() > bytes.len());
}
}
}
#[test]
fn complete_validation_rejects_truncation_unknown_codec_bad_loops_and_reserved_bytes() {
for (bytes, music) in fixtures() {
for cut in [0, 4, 47, 48, bytes.len() - 1] {
assert!(decode_audio_bank(&bytes[..cut], music).is_err());
}
let name = u32le(bytes, 40).unwrap() as usize;
let sample = 48 + name;
for (at, value) in
[(4, 2), (44, 1), (sample, 99), (sample + 1, 3), (sample + 2, 2), (sample + 34, 1)]
{
let mut bad = bytes.to_vec();
bad[at] = value;
assert!(decode_audio_bank(&bad, music).is_err(), "{at}");
}
let mut bad = bytes.to_vec();
bad[sample + 12..sample + 16].copy_from_slice(&1001u32.to_le_bytes());
assert!(decode_audio_bank(&bad, music).is_err());
let mut bad = bytes.to_vec();
bad.push(0);
assert!(decode_audio_bank(&bad, music).is_err());
// Mutating every byte must produce either a validated bank or a clean error.
for i in 0..bytes.len() {
let mut changed = bytes.to_vec();
changed[i] ^= 0xff;
let _ = decode_audio_bank(&changed, music);
}
}
}
fn entry(id: i32, offset: usize, bytes: &[u8], music: bool) -> AssetEntry {
AssetEntry {
asset_id: id,
asset_name: format!("bank{id}"),
bank_type: if music { BankType::MUSIC } else { BankType::SOUNDS },
offset: offset as u64,
size: bytes.len() as u64,
decoded_size: decode_audio_bank(bytes, music).unwrap().decoded_size() as u64,
codec: AssetCodec::None,
metadata: serde_json::json!({"version":1}),
}
}
fn ready(manager: &AssetManager, handle: u32) {
let end = Instant::now() + Duration::from_secs(3);
while matches!(
manager.status(handle),
LoadStatus::QUEUED | LoadStatus::LOADING | LoadStatus::PENDING | LoadStatus::ACTIVE
) {
assert!(Instant::now() < end);
std::thread::yield_now();
}
assert_eq!(manager.status(handle), LoadStatus::READY);
}
#[test]
fn audio_loading_commit_replacement_and_unload_are_atomic_and_type_isolated() {
let fixtures = fixtures();
let sound = fixtures[2].0;
let music = fixtures[3].0;
let mut payload = sound.to_vec();
payload.extend_from_slice(music);
let bad_at = payload.len();
let mut corrupt = sound.to_vec();
corrupt[0] = 0;
payload.extend(&corrupt);
let mut bad = entry(3, bad_at, sound, false);
bad.decoded_size += 1;
let banks = Arc::new(MemoryBanks::new());
let manager = AssetManager::new(
vec![entry(1, 0, sound, false), entry(2, sound.len(), music, true), bad],
AssetsPayloadSource::from_bytes(payload),
banks.clone(),
banks.clone(),
banks.clone(),
);
let s = manager.load(1, 0).unwrap();
ready(&manager, s);
assert!(banks.sound_bank_slot(0).is_none());
assert_eq!(manager.commit(s), AssetOpStatus::Ok);
manager.apply_commits();
let old = banks.sound_bank_slot(0).unwrap();
let m = manager.load(2, 0).unwrap();
ready(&manager, m);
manager.commit(m);
manager.apply_commits();
assert!(old.alive.load(std::sync::atomic::Ordering::Acquire));
assert!(banks.music_bank_slot(0).is_some());
let failing = manager.load(3, 0).unwrap();
let end = Instant::now() + Duration::from_secs(3);
while manager.status(failing) != LoadStatus::ERROR {
assert!(Instant::now() < end);
std::thread::yield_now();
}
manager.apply_commits();
assert!(Arc::ptr_eq(&old, &banks.sound_bank_slot(0).unwrap()));
let replacement = manager.load(1, 0).unwrap();
ready(&manager, replacement);
assert!(old.alive.load(std::sync::atomic::Ordering::Acquire));
manager.commit(replacement);
manager.apply_commits();
assert!(!old.alive.load(std::sync::atomic::Ordering::Acquire));
assert_eq!(manager.unload_audio(true, 0), AssetOpStatus::Ok);
assert!(banks.music_bank_slot(0).is_none());
assert!(banks.sound_bank_slot(0).is_some());
assert_eq!(manager.slot_info(SlotRef::music(0)).asset_id, None);
assert_eq!(manager.unload_audio(false, 0), AssetOpStatus::Ok);
assert!(banks.sound_bank_slot(0).is_none());
assert_eq!(manager.unload_audio(false, 16), AssetOpStatus::InvalidState);
}
#[test]
fn codec_tooling_rejects_overflowing_block_parameters_without_panicking() {
for align in [0, 4, usize::MAX - 3, usize::MAX] {
assert!(encode_ima_wav4(&[1, 2], 1, align).is_err());
assert!(decode_ima_wav4(&[0; 8], 1, 9, align, 9).is_err());
}
assert!(encode_ima_wav4(&[1], 1, 65532).is_err());
assert!(decode_ima_wav4(&[0; 8], 1, usize::MAX, 8, 9).is_err());
assert!(encode_ima_wav4(&[1], 0, 256).is_err());
}
@@ -0,0 +1,924 @@
//! Host-independent bounded PCM engine. Construct resources off the render lane.
use super::{
AudioCommand, AudioDelivery, AudioTransport, Channel, LoopMode, MAX_CHANNELS,
MAX_COMMANDS_PER_FRAME, SFX_CHANNELS, SFX_START,
};
use prometeu_hal::{
audio_bridge::AudioControl,
music_bank::{Instrument, MAX_MUSIC_TRACKS, MusicBank, MusicEvent},
sample::Sample,
};
use std::{
sync::{
Arc,
atomic::{AtomicBool, Ordering},
},
time::{Duration, Instant},
};
pub const RETIRED_CAPACITY: usize = 2 * (3 * MAX_COMMANDS_PER_FRAME + MAX_CHANNELS + 2);
/// Owning resources leave the real-time lane in a preallocated packet.
#[allow(dead_code)]
enum Retired {
Sample(Arc<Sample>),
Bank(Arc<MusicBank>),
Token(Arc<AtomicBool>),
}
pub struct Packet {
retired: Vec<Retired>,
pub batches: [Option<Vec<AudioCommand>>; 2],
}
impl Default for Packet {
fn default() -> Self {
Self::new()
}
}
impl Packet {
pub fn new() -> Self {
Self { retired: Vec::with_capacity(RETIRED_CAPACITY), batches: [None, None] }
}
fn push(&mut self, owner: Retired) {
assert!(self.retired.len() < RETIRED_CAPACITY, "audio retirement bound");
self.retired.push(owner);
}
fn voice(&mut self, voice: &mut Channel) {
if let Some(s) = voice.sample.take() {
self.push(Retired::Sample(s));
}
if let Some(t) = voice.alive.take() {
self.push(Retired::Token(t));
}
voice.active = false;
}
pub fn recycle(&mut self, transport: &AudioTransport) {
self.retired.clear();
for batch in &mut self.batches {
if let Some(batch) = batch.take() {
transport.recycle_commands(batch);
}
}
}
/// Service thread only: count distinct retired PCM payloads, excluding Arc overhead.
pub fn retained_pcm_bytes(&self) -> usize {
let mut seen = std::collections::HashSet::new();
let mut bytes = 0;
let mut add = |sample: &Arc<Sample>| {
if seen.insert(Arc::as_ptr(sample)) {
bytes += sample.data.capacity() * 2;
}
};
for owner in &self.retired {
match owner {
Retired::Sample(sample) => add(sample),
Retired::Bank(bank) => {
for sample in bank.samples.iter() {
add(sample);
}
}
Retired::Token(_) => {}
}
}
for batch in self.batches.iter().flatten() {
for command in batch {
match command {
AudioCommand::Play { sample, .. } => add(sample),
AudioCommand::MusicPlay { bank, .. } => {
for sample in bank.samples.iter() {
add(sample);
}
}
_ => {}
}
}
}
bytes
}
pub fn is_empty(&self) -> bool {
self.retired.is_empty() && self.batches.iter().all(Option::is_none)
}
pub fn capacity(&self) -> usize {
self.retired.capacity()
}
}
#[derive(Clone, Copy)]
struct Ramp {
from: f64,
to: f64,
start: u64,
duration: u32,
smooth: bool,
}
impl Ramp {
fn constant(value: f64) -> Self {
Self { from: value, to: value, start: 0, duration: 0, smooth: false }
}
fn at(self, now: u64) -> f64 {
if self.duration == 0 {
return self.to;
}
let elapsed = now.saturating_sub(self.start);
if elapsed >= self.duration as u64 {
return self.to;
}
let x = elapsed as f64 / self.duration as f64;
let x = if self.smooth { x * x * (3.0 - 2.0 * x) } else { x };
self.from + (self.to - self.from) * x
}
fn set(&mut self, value: f64, now: u64, duration: u32, smooth: bool) {
*self = Self { from: self.at(now), to: value, start: now, duration, smooth };
}
fn done(self, now: u64) -> bool {
now.saturating_sub(self.start) >= self.duration as u64
}
}
#[derive(Clone, Copy)]
struct SendRamp {
base: Ramp,
scale: Ramp,
}
impl Default for SendRamp {
fn default() -> Self {
Self { base: Ramp::constant(0.0), scale: Ramp::constant(1.0) }
}
}
impl SendRamp {
fn at(self, now: u64, instrument_default: f64) -> f64 {
self.base.at(now) + self.scale.at(now) * instrument_default
}
fn set(&mut self, to: f64, now: u64, frames: u32, smooth: bool) {
self.base.set(to, now, frames, smooth);
self.scale.set(0.0, now, frames, smooth);
}
}
#[derive(Clone, Copy)]
struct Track {
gain: Ramp,
pan: Ramp,
pitch: Ramp,
send: SendRamp,
vibrato: Option<(u32, f64)>,
tremolo: Option<(u32, f64)>,
}
impl Default for Track {
fn default() -> Self {
Self {
gain: Ramp::constant(1.0),
pan: Ramp::constant(128.0),
pitch: Ramp::constant(0.0),
send: SendRamp::default(),
vibrato: None,
tremolo: None,
}
}
}
struct MusicGroup {
bank: Option<Arc<MusicBank>>,
active: bool,
paused: bool,
looping: bool,
time: u128,
cursor: usize,
frame: u64,
iteration: u64,
tracks: [Track; MAX_MUSIC_TRACKS],
gain: Ramp,
stop_after_fade: bool,
}
impl Default for MusicGroup {
fn default() -> Self {
Self {
bank: None,
active: false,
paused: false,
looping: false,
time: 0,
cursor: 0,
frame: 0,
iteration: 0,
tracks: [Track::default(); MAX_MUSIC_TRACKS],
gain: Ramp::constant(1.0),
stop_after_fade: false,
}
}
}
#[derive(Clone, Copy)]
struct VoiceState {
inst: Instrument,
track: usize,
note_id: u32,
iteration: u64,
born: u64,
age: u64,
release_age: Option<u64>,
release_level: f64,
release_order: Option<u64>,
level: f64,
cached_cents: f64,
pitch_factor: f64,
glide: Ramp,
send: f64,
zone: usize,
}
impl Default for VoiceState {
fn default() -> Self {
Self {
inst: Instrument::default(),
track: 0,
note_id: 0,
iteration: 0,
born: 0,
age: 0,
release_age: None,
release_level: 0.0,
release_order: None,
level: 1.0,
cached_cents: f64::INFINITY,
pitch_factor: 1.0,
glide: Ramp::constant(0.0),
send: 0.0,
zone: 0,
}
}
}
struct Effect {
delay: Vec<[f32; 2]>,
write: usize,
history: usize,
frames: usize,
feedback: f32,
wet: f32,
dry: f32,
coefficient: f32,
filter: [f32; 2],
}
impl Effect {
fn new() -> Self {
Self {
delay: vec![[0.0; 2]; 96_000],
write: 0,
history: 0,
frames: 0,
feedback: 0.0,
wet: 0.0,
dry: 1.0,
coefficient: 1.0,
filter: [0.0; 2],
}
}
fn reset(&mut self) {
self.write = 0;
self.history = 0;
self.filter = [0.0; 2];
}
fn process(&mut self, input: [f32; 2], send: [f32; 2]) -> [f32; 2] {
if self.frames == 0 && self.coefficient == 1.0 {
self.filter = input;
return [input[0] * self.dry, input[1] * self.dry];
}
let mut out = [0.0; 2];
for c in 0..2 {
self.filter[c] += self.coefficient * (input[c] - self.filter[c]);
// Flush subnormals without denormal-sensitive host settings.
if self.filter[c].abs() < 1e-20 {
self.filter[c] = 0.0;
}
let echo = if self.frames != 0 && self.history >= self.frames {
self.delay[(self.write + 96_000 - self.frames) % 96_000][c]
} else {
0.0
};
out[c] = self.filter[c] * self.dry + echo * self.wet;
self.delay[self.write][c] = send[c] + echo * self.feedback;
if self.delay[self.write][c].abs() < 1e-20 {
self.delay[self.write][c] = 0.0;
}
}
self.write = (self.write + 1) % 96_000;
self.history = (self.history + 1).min(96_000);
out
}
}
#[derive(Debug, Clone, Copy, Default)]
pub struct AudioDiagnostics {
pub frames: u64,
pub active_voices: usize,
pub maximum_voices: usize,
pub last_render_us: u64,
pub maximum_render_us: u64,
}
pub struct AudioMixer {
pub voices: [Channel; MAX_CHANNELS],
states: [VoiceState; MAX_CHANNELS],
groups: [MusicGroup; 2],
effects: [Effect; 3],
bus: [f32; 3],
sfx_gain: f32,
exponential: [f64; 1025],
sine: [f64; 4097],
pub last_processing_time: Duration,
paused: bool,
epoch: u64,
pub diagnostics: AudioDiagnostics,
note_serial: u64,
}
impl Default for AudioMixer {
fn default() -> Self {
Self::new()
}
}
impl AudioMixer {
pub fn new() -> Self {
Self {
voices: std::array::from_fn(|_| Channel::default()),
states: [VoiceState::default(); MAX_CHANNELS],
groups: std::array::from_fn(|_| MusicGroup::default()),
effects: std::array::from_fn(|_| Effect::new()),
bus: [1.0; 3],
sfx_gain: 1.0,
exponential: std::array::from_fn(|i| {
((-6.0 * i as f64 / 1024.0).exp() - (-6.0f64).exp()) / (1.0 - (-6.0f64).exp())
}),
sine: std::array::from_fn(|i| (std::f64::consts::TAU * i as f64 / 4096.0).sin()),
last_processing_time: Duration::ZERO,
paused: false,
epoch: 0,
diagnostics: AudioDiagnostics::default(),
note_serial: 0,
}
}
pub fn prepared_bytes(&self) -> usize {
std::mem::size_of::<Self>()
+ self
.effects
.iter()
.map(|e| e.delay.capacity() * std::mem::size_of::<[f32; 2]>())
.sum::<usize>()
}
fn stop_group(&mut self, group: usize, packet: &mut Packet) {
for v in &mut self.voices[group * 12..group * 12 + 12] {
packet.voice(v);
}
if let Some(bank) = self.groups[group].bank.take() {
packet.push(Retired::Bank(bank));
}
let gain = self.groups[group].gain.at(self.groups[group].frame);
self.groups[group] = MusicGroup::default();
self.groups[group].gain = Ramp::constant(gain);
self.effects[group].reset();
}
pub fn retire_inactive(&mut self, packet: &mut Packet) {
for g in 0..2 {
if self.groups[g].bank.as_ref().is_some_and(|b| !b.alive.load(Ordering::Acquire))
|| (self.groups[g].stop_after_fade && !self.groups[g].active)
{
self.stop_group(g, packet);
}
}
for v in &mut self.voices {
if !v.active || v.alive.as_ref().is_some_and(|a| !a.load(Ordering::Acquire)) {
packet.voice(v);
}
}
}
pub fn apply_delivery_into(&mut self, delivery: AudioDelivery, packet: &mut Packet) {
if delivery.epoch < self.epoch {
packet.batches = [delivery.frame, delivery.lifecycle];
return;
}
if delivery.epoch != self.epoch {
for group in 0..2 {
self.stop_group(group, packet);
}
for voice in &mut self.voices {
packet.voice(voice);
*voice = Channel::default();
}
for group in &mut self.groups {
group.gain = Ramp::constant(1.0);
}
for effect in &mut self.effects {
effect.reset();
effect.frames = 0;
effect.feedback = 0.0;
effect.wet = 0.0;
effect.dry = 1.0;
effect.coefficient = 1.0;
}
self.bus = [1.0; 3];
self.sfx_gain = 1.0;
self.epoch = delivery.epoch;
}
packet.batches = [delivery.frame, delivery.lifecycle];
for index in 0..2 {
if let Some(mut batch) = packet.batches[index].take() {
for command in &mut batch {
let command =
std::mem::replace(command, AudioCommand::Stop { voice_id: SFX_CHANNELS });
self.process_command(command, packet);
}
packet.batches[index] = Some(batch);
}
}
self.paused = delivery.paused;
}
/// Convenience for non-real-time signal tests/offline owners.
pub fn apply_delivery(&mut self, delivery: AudioDelivery) {
self.apply_delivery_into(delivery, &mut Packet::new());
}
pub fn fill_buffer(&mut self, buffer: &mut [f32]) {
self.mix(buffer);
self.retire_inactive(&mut Packet::new());
}
fn process_command(&mut self, command: AudioCommand, packet: &mut Packet) {
match command {
AudioCommand::Play {
sample,
alive,
voice_id,
volume,
pan,
pitch,
priority,
loop_mode,
} => {
if voice_id >= SFX_CHANNELS
|| alive.as_ref().is_some_and(|a| !a.load(Ordering::Acquire))
{
packet.push(Retired::Sample(sample));
if let Some(t) = alive {
packet.push(Retired::Token(t));
}
return;
}
let index = SFX_START + voice_id;
packet.voice(&mut self.voices[index]);
self.voices[index] = Channel {
sample: Some(sample),
alive,
active: true,
pos: 0.0,
pitch,
volume,
pan,
loop_mode,
priority,
};
self.states[index] = VoiceState::default();
}
AudioCommand::Stop { voice_id } => {
if voice_id < SFX_CHANNELS {
packet.voice(&mut self.voices[SFX_START + voice_id]);
}
}
AudioCommand::SetVolume { voice_id, volume } => {
if voice_id < SFX_CHANNELS {
self.voices[SFX_START + voice_id].volume = volume;
}
}
AudioCommand::SetPan { voice_id, pan } => {
if voice_id < SFX_CHANNELS {
self.voices[SFX_START + voice_id].pan = pan;
}
}
AudioCommand::SetPitch { voice_id, pitch } => {
if voice_id < SFX_CHANNELS {
self.voices[SFX_START + voice_id].pitch = pitch;
}
}
AudioCommand::MusicPlay { group, bank, looping } => {
if group >= 2 || !bank.alive.load(Ordering::Acquire) {
packet.push(Retired::Bank(bank));
return;
}
self.stop_group(group, packet);
self.groups[group].bank = Some(bank);
self.groups[group].active = true;
self.groups[group].looping = looping;
}
AudioCommand::Control(control) => match control {
AudioControl::GroupVolume { group, gain } => {
if group == 2 {
self.sfx_gain = gain as f32 / 255.0;
} else {
self.groups[group].gain = Ramp::constant(gain as f64 / 255.0);
self.groups[group].stop_after_fade = false;
}
}
AudioControl::MusicStop { group } => self.stop_group(group, packet),
AudioControl::MusicPause { group, paused } => self.groups[group].paused = paused,
AudioControl::MusicVolume { group, gain } => {
self.groups[group].gain = Ramp::constant(gain as f64 / 255.0);
self.groups[group].stop_after_fade = false;
}
AudioControl::MusicFade { group, gain, frames, smooth } => {
let g = &mut self.groups[group];
g.gain.set(gain as f64 / 255.0, g.frame, frames, smooth);
g.stop_after_fade = false;
}
AudioControl::Crossfade { source, target, frames, smooth } => {
if !self.groups[source].active || !self.groups[target].active {
return;
}
for (g, value) in [(source, 0.0), (target, 1.0)] {
let group = &mut self.groups[g];
group.gain.set(value, group.frame, frames, smooth);
group.stop_after_fade = g == source;
}
}
AudioControl::BusVolume { bus, gain } => self.bus[bus] = gain as f32 / 255.0,
AudioControl::SfxSend { voice, gain } => {
self.states[SFX_START + voice].send = gain as f64 / 255.0
}
AudioControl::Effect { group, cutoff_hz, delay_frames, feedback, wet, dry } => {
let effect = &mut self.effects[group];
effect.coefficient = if cutoff_hz == 0 {
1.0
} else {
(1.0 - (-std::f32::consts::TAU * cutoff_hz as f32 / 48_000.0).exp())
.clamp(0.0, 1.0)
};
if effect.frames != delay_frames as usize {
effect.reset();
}
effect.frames = delay_frames as usize;
effect.feedback = feedback;
effect.wet = wet;
effect.dry = dry;
}
},
}
}
fn begin_note_release(&mut self, index: usize) {
let state = &mut self.states[index];
if state.release_age.is_none() {
let level = envelope(state, &self.exponential);
state.release_age = Some(state.age);
state.release_order = Some(self.note_serial);
self.note_serial = self.note_serial.wrapping_add(1);
state.release_level = level;
}
}
fn event(&mut self, group: usize, event: MusicEvent, zone: usize) {
let frame = self.groups[group].frame;
let track = event.track as usize;
if event.kind == 1 {
let bank = self.groups[group].bank.as_ref().unwrap();
let inst = bank.instruments[zone];
let cents = (event.c - inst.root as i32) as f64 * 100.0 + inst.tuning_cents as f64;
let range = group * 12..group * 12 + 12;
self.note_serial = self.note_serial.wrapping_add(1);
let legato = if event.flags & 1 != 0 {
range
.clone()
.filter(|&i| {
self.voices[i].active
&& self.states[i].track == track
&& self.states[i].release_age.is_none()
})
.max_by_key(|&i| (self.states[i].born, i))
} else {
None
};
if let Some(index) = legato.filter(|&i| self.states[i].zone == zone) {
let state = &mut self.states[index];
state.glide.set(cents, frame, event.duration_frames, event.curve == 1);
state.note_id = event.a;
state.born = self.note_serial;
state.iteration = self.groups[group].iteration;
self.voices[index].volume = ((inst.gain as u16 * event.d as u16 + 127) / 255) as u8;
return;
}
let index = legato.unwrap_or_else(|| {
range.clone().find(|&i| !self.voices[i].active).unwrap_or_else(|| {
range
.min_by_key(|&i| {
(
self.states[i].release_age.is_none(),
self.states[i].release_order.unwrap_or(self.states[i].born),
i,
)
})
.unwrap()
})
});
// Music samples always have an owner in this group's retained bank.
self.voices[index] = Channel {
sample: Some(bank.samples[inst.sample as usize].clone()),
alive: None,
active: true,
pos: 0.0,
pitch: 1.0,
volume: ((inst.gain as u16 * event.d as u16 + 127) / 255) as u8,
pan: inst.pan,
loop_mode: if inst.looping { LoopMode::On } else { LoopMode::Off },
priority: 0,
};
self.states[index] = VoiceState {
inst,
track,
note_id: event.a,
iteration: self.groups[group].iteration,
born: self.note_serial,
glide: Ramp::constant(cents),
send: inst.send as f64 / 65535.0,
zone,
..VoiceState::default()
};
} else if event.kind == 2 {
for index in group * 12..group * 12 + 12 {
let state = &self.states[index];
if self.voices[index].active
&& state.track == track
&& state.note_id == event.a
&& state.iteration == self.groups[group].iteration
{
self.begin_note_release(index);
}
}
} else {
let t = &mut self.groups[group].tracks[track];
let smooth = event.curve == 1;
match event.kind {
4 => t.gain.set(event.c as f64 / 65535.0, frame, event.duration_frames, smooth),
5 => t.pan.set(event.c as f64, frame, event.duration_frames, smooth),
6 => t.pitch.set(event.c as f64, frame, event.duration_frames, smooth),
7 => t.vibrato = Some((event.a, event.c as f64)),
8 => t.tremolo = Some((event.a, event.c as f64 / 65535.0)),
9 => t.send.set(event.c as f64 / 65535.0, frame, event.duration_frames, smooth),
_ => {} // Tempo/instrument selection were resolved during compilation.
}
}
}
fn sequence(&mut self, group: usize) {
if !self.groups[group].active || self.groups[group].paused {
return;
}
loop {
let g = &self.groups[group];
let bank = g.bank.as_ref().unwrap();
if let Some(event) = bank.events.get(g.cursor).filter(|e| e.time <= g.time) {
let event = *event;
self.groups[group].cursor += 1;
self.event(group, event.event, event.zone);
continue;
}
if g.time < bank.end_time {
break;
}
let loop_start = bank.loop_time.unwrap_or(0);
let loop_event = if bank.loop_time.is_some() { bank.loop_event } else { 0 };
let loop_length = bank.end_time - loop_start;
for index in group * 12..group * 12 + 12 {
self.begin_note_release(index);
}
let g = &mut self.groups[group];
if !g.looping {
g.active = false;
break;
}
g.time -= loop_length;
g.cursor = loop_event;
g.iteration = g.iteration.wrapping_add(1);
}
}
pub fn mix(&mut self, buffer: &mut [f32]) {
let start = Instant::now();
buffer.fill(0.0);
if self.paused {
self.last_processing_time = start.elapsed();
return;
}
for frame in buffer.as_chunks_mut::<2>().0 {
// Invalidated resources stay owned until a reclamation packet is available.
for g in 0..2 {
if self.groups[g].bank.as_ref().is_some_and(|b| !b.alive.load(Ordering::Acquire)) {
self.groups[g].active = false;
for v in &mut self.voices[g * 12..g * 12 + 12] {
v.active = false;
}
self.effects[g].reset();
}
self.sequence(g);
}
let mut dry = [[0.0f32; 2]; 3];
let mut send = [[0.0f32; 2]; 3];
let mut active = 0;
for index in 0..MAX_CHANNELS {
let group = if index < 12 {
0
} else if index < 24 {
1
} else {
2
};
if group < 2 && self.groups[group].paused {
continue;
}
let voice = &mut self.voices[index];
if !voice.active {
continue;
}
if voice.alive.as_ref().is_some_and(|a| !a.load(Ordering::Acquire)) {
voice.active = false;
continue;
}
let Some(sample) = &voice.sample else {
continue;
};
let len = sample.frames_len();
if len == 0 {
voice.active = false;
continue;
}
active += 1;
let state = &mut self.states[index];
let mut volume = voice.volume as f64 / 255.0;
let mut pan = voice.pan as f64;
let mut pitch = voice.pitch;
let mut amount = state.send;
if group < 2 {
let g = &self.groups[group];
let track = g.tracks[state.track];
state.level = envelope(state, &self.exponential);
if state.release_age.is_some() && state.level <= 0.0 {
voice.active = false;
continue;
}
let vib = track
.vibrato
.unwrap_or((state.inst.vibrato_millihz, state.inst.vibrato_cents as f64));
let trem = track.tremolo.unwrap_or((
state.inst.tremolo_millihz,
state.inst.tremolo_depth as f64 / 65535.0,
));
let cents = state.glide.at(g.frame)
+ track.pitch.at(g.frame)
+ lfo(&self.sine, state.age, vib.0) * vib.1;
if cents != state.cached_cents {
state.pitch_factor = (cents / 1200.0).exp2();
state.cached_cents = cents;
}
pitch *= state.pitch_factor;
volume *= state.level
* track.gain.at(g.frame)
* (1.0 - trem.1 * (lfo(&self.sine, state.age, trem.0) + 1.0) * 0.5);
pan = (pan + track.pan.at(g.frame) - 128.0).clamp(0.0, 255.0);
amount = track.send.at(g.frame, state.send);
state.age = state.age.wrapping_add(1);
}
let channels = sample.channels as usize;
let (loop_start, end) = match (voice.loop_mode, sample.loop_start, sample.loop_end)
{
(LoopMode::On, Some(a), Some(b)) if a < b && b as usize <= len => {
(a as f64, b as f64)
}
_ => (0.0, len as f64),
};
let pos = voice.pos as usize;
if pos >= len {
voice.active = false;
continue;
}
let next = if voice.loop_mode == LoopMode::On && (pos + 1) as f64 >= end {
loop_start as usize
} else {
pos + 1
};
let frac = (voice.pos - pos as f64) as f32;
let value = |c: usize| {
let a = sample.data[pos * channels + c] as f32 / 32768.0;
let b =
sample.data.get(next * channels + c).copied().unwrap_or(0) as f32 / 32768.0;
a + (b - a) * frac
};
if volume != 0.0 {
let normalized_pan =
if pan <= 128.0 { pan / 256.0 } else { 0.5 + (pan - 128.0) / 254.0 };
let gains = if channels == 1 {
[(1.0 - normalized_pan).sqrt(), normalized_pan.sqrt()]
} else {
[
if pan <= 128.0 { 1.0 } else { (255.0 - pan) / 127.0 },
if pan >= 128.0 { 1.0 } else { pan / 128.0 },
]
};
for c in 0..2 {
let v =
value(if channels == 1 { 0 } else { c }) * (volume * gains[c]) as f32;
dry[group][c] += v;
send[group][c] += v * amount as f32;
}
}
let ratio = sample.sample_rate as f64 / 48_000.0;
if ratio > 0.0 {
if pitch >= (end - voice.pos) / ratio {
if voice.loop_mode == LoopMode::Off {
voice.active = false;
} else {
let length = end - loop_start;
let step = (pitch % (length / ratio)) * ratio;
voice.pos =
loop_start + (voice.pos - loop_start + step).rem_euclid(length);
if voice.pos >= end {
voice.pos = loop_start;
}
}
} else {
voice.pos += pitch * ratio;
}
}
}
self.diagnostics.active_voices = active;
self.diagnostics.maximum_voices = self.diagnostics.maximum_voices.max(active);
for group in 0..3 {
if group < 2 && self.groups[group].paused {
continue;
}
let gain = if group < 2 {
self.groups[group].gain.at(self.groups[group].frame) as f32 * self.bus[0]
} else {
self.bus[1] * self.sfx_gain
};
let values = self.effects[group].process(dry[group], send[group]);
for c in 0..2 {
frame[c] += values[c] * gain;
}
}
for value in frame {
*value = (*value * self.bus[2]).clamp(-1.0, 1.0);
}
for group in &mut self.groups {
if !group.paused
&& let Some(bank) = &group.bank
{
group.frame = group.frame.wrapping_add(1);
if group.active {
group.time += bank.denominator;
}
if group.stop_after_fade && group.gain.done(group.frame) {
group.active = false;
}
}
}
// Crossfade completion silences/relinquishes notes; bank reclamation is deferred.
for g in 0..2 {
if self.groups[g].stop_after_fade && !self.groups[g].active {
for voice in &mut self.voices[g * 12..g * 12 + 12] {
voice.active = false;
}
self.effects[g].reset();
}
}
self.diagnostics.frames += 1;
}
self.last_processing_time = start.elapsed();
self.diagnostics.last_render_us = self.last_processing_time.as_micros() as u64;
self.diagnostics.maximum_render_us =
self.diagnostics.maximum_render_us.max(self.diagnostics.last_render_us);
}
}
fn lfo(table: &[f64; 4097], age: u64, millihz: u32) -> f64 {
let phase = ((age % 48_000_000) * millihz as u64 % 48_000_000) as f64 / 48_000_000.0 * 4096.0;
let index = phase as usize;
table[index] + (table[index + 1] - table[index]) * (phase - index as f64)
}
fn micros_frames(value: u32) -> u64 {
(value as u64 * 48_000).div_ceil(1_000_000)
}
fn envelope(state: &VoiceState, table: &[f64; 1025]) -> f64 {
let curve = |x: f64| {
if state.inst.curve == 0 {
1.0 - x
} else {
let p = x.clamp(0.0, 1.0) * 1024.0;
let i = (p as usize).min(1023);
table[i] + (table[i + 1] - table[i]) * (p - i as f64)
}
};
if let Some(start) = state.release_age {
let duration = micros_frames(state.inst.release_us);
let elapsed = state.age - start;
return if duration == 0 || elapsed >= duration {
0.0
} else {
state.release_level * curve(elapsed as f64 / duration as f64)
};
}
let attack = micros_frames(state.inst.attack_us);
let decay = micros_frames(state.inst.decay_us);
if state.age < attack {
return state.age as f64 / attack as f64;
}
let sustain = state.inst.sustain as f64 / 65535.0;
if state.age - attack < decay {
sustain + (1.0 - sustain) * curve((state.age - attack) as f64 / decay as f64)
} else {
sustain
}
}
#[cfg(test)]
#[path = "engine_tests.rs"]
mod tests;
@@ -0,0 +1,508 @@
use super::*;
use crate::{Audio, MemoryBanks, SoundBankPoolAccess, SoundBankPoolInstaller};
use prometeu_hal::{AudioBridge, AudioOpStatus, music_bank::AUDIO_PPQ, sound_bank::SoundBank};
fn note(tick: u32, id: u32) -> MusicEvent {
MusicEvent { tick, kind: 1, a: id, c: 60, d: 255, ..Default::default() }
}
fn bank(events: Vec<MusicEvent>, inst: Instrument, tempo: u32, duration: u32) -> Arc<MusicBank> {
Arc::new(
MusicBank::compile(
"test".into(),
vec![Arc::new(
Sample::new(48_000, vec![1024, -2048, 1024, -2048])
.with_channels(2)
.unwrap()
.with_loop(0, 2),
)],
vec![inst],
events,
AUDIO_PPQ,
tempo,
duration,
None,
1,
)
.unwrap(),
)
}
fn pair() -> (Arc<MemoryBanks>, Audio, AudioMixer) {
let banks = Arc::new(MemoryBanks::new());
(banks.clone(), Audio::new(banks), AudioMixer::new())
}
fn publish(audio: &mut Audio, mixer: &mut AudioMixer) {
audio.finish_frame();
mixer.apply_delivery(audio.transport().try_take().unwrap());
}
fn install(banks: &MemoryBanks, slot: usize, events: Vec<MusicEvent>) {
banks.install_music_bank(
slot,
bank(events, Instrument { looping: true, ..Default::default() }, 20_000, 100_000),
);
}
#[test]
fn thirty_two_stereo_voices_have_fixed_group_ownership_and_safe_clipping() {
let (banks, mut audio, mut mixer) = pair();
install(&banks, 0, (0..12).map(|n| note(0, n)).collect());
assert_eq!(audio.music_play(0, 0, true), AudioOpStatus::Ok);
assert_eq!(audio.music_play(1, 0, true), AudioOpStatus::Ok);
let sample = Arc::new(Sample::new(48_000, vec![1024, -2048]));
// Stereo uses one cursor/slot; no channel consumes a second voice.
let sample = Arc::new(
Sample::new(48_000, sample.data.clone()).with_channels(2).unwrap().with_loop(0, 1),
);
for i in 0..8 {
assert_eq!(
audio.play_sample(sample.clone(), i, 255, 128, 1.0, 0, LoopMode::On),
AudioOpStatus::Ok
);
}
assert_eq!(audio.stop(8), AudioOpStatus::VoiceInvalid);
publish(&mut audio, &mut mixer);
let mut pcm = [0.0; 2];
mixer.mix(&mut pcm);
assert_eq!(mixer.voices.iter().filter(|v| v.active).count(), 32);
assert_eq!(mixer.diagnostics.maximum_voices, 32);
assert_eq!(pcm, [1.0, -1.0]);
audio.control(AudioControl::MusicStop { group: 0 });
publish(&mut audio, &mut mixer);
assert!(mixer.voices[..12].iter().all(|v| !v.active));
assert!(mixer.voices[12..].iter().all(|v| v.active));
for i in 0..8 {
audio.stop(i);
}
publish(&mut audio, &mut mixer);
assert!(mixer.voices[12..24].iter().all(|v| v.active));
}
#[test]
fn audio_time_does_not_wait_for_frame_sync_and_buffer_partition_is_invariant() {
let (banks, mut audio, mut a) = pair();
install(&banks, 0, vec![note(17, 1)]);
audio.music_play(0, 0, false);
publish(&mut audio, &mut a);
let mut b = AudioMixer::new();
let resident = banks.music_bank_slot(0).unwrap();
b.process_command(
AudioCommand::MusicPlay { group: 0, bank: resident, looping: false },
&mut Packet::new(),
);
let mut whole = vec![0.0; 4000];
a.mix(&mut whole);
let mut pieces = vec![0.0; 4000];
for piece in pieces.chunks_mut(14) {
b.mix(piece);
}
assert_eq!(whole, pieces);
assert!(whole[..34].iter().all(|&v| v == 0.0));
assert_ne!(whole[34], 0.0);
assert_eq!(a.groups[0].frame, 2000);
}
#[test]
fn fractional_tempo_changes_and_loop_remainder_have_no_cumulative_drift() {
let events = vec![MusicEvent { tick: 2, kind: 3, a: 30_001, ..Default::default() }, note(3, 1)];
let music = bank(events, Instrument::default(), 20_001, 5);
assert_eq!(music.events[1].time, 2 * 20_001u128 * 48_000 + 30_001u128 * 48_000);
let mut mixer = AudioMixer::new();
let end = music.end_time;
let den = music.denominator;
mixer.process_command(
AudioCommand::MusicPlay { group: 0, bank: music, looping: true },
&mut Packet::new(),
);
let n = 100_000;
mixer.mix(&mut vec![0.0; n * 2]);
// The first event of a loop is evaluated before the rendered frame.
assert_eq!(mixer.groups[0].iteration, ((n - 1) as u128 * den / end) as u64);
assert_eq!(mixer.groups[0].time, n as u128 * den - mixer.groups[0].iteration as u128 * end);
}
#[test]
fn music_allocator_uses_free_then_release_then_oldest_and_stale_off_does_not_stop_reuse() {
let (banks, mut audio, mut mixer) = pair();
let mut events: Vec<_> = (0..12).map(|n| note(0, n)).collect();
events.push(MusicEvent { tick: 1, kind: 2, a: 3, ..Default::default() });
events.push(note(2, 12));
events.push(MusicEvent { tick: 3, kind: 2, a: 3, ..Default::default() });
events.push(note(4, 13));
banks.install_music_bank(
0,
bank(
events,
Instrument { looping: true, release_us: 1_000_000, ..Default::default() },
20_000,
1000,
),
);
audio.music_play(0, 0, false);
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 10]);
assert_eq!(mixer.states[3].note_id, 12);
assert_eq!(mixer.states[0].note_id, 13);
assert!(mixer.states[3].release_age.is_none());
}
#[test]
fn envelopes_have_exact_endpoints_and_note_off_releases() {
let mut state = VoiceState {
inst: Instrument {
attack_us: 1000,
decay_us: 1000,
sustain: 32768,
release_us: 1000,
curve: 0,
..Default::default()
},
..Default::default()
};
let mut mixer = AudioMixer::new();
for (age, value) in [(0, 0.0), (24, 0.5), (48, 1.0), (96, 32768.0 / 65535.0)] {
state.age = age;
assert!((envelope(&state, &mixer.exponential) - value).abs() < 1e-12);
}
state.release_age = Some(96);
state.release_level = 0.5;
for (age, value) in [(96, 0.5), (120, 0.25), (144, 0.0)] {
state.age = age;
assert!((envelope(&state, &mixer.exponential) - value).abs() < 1e-12);
}
for i in 0..10_000 {
let x = i as f64 / 10_000.0;
state.inst.curve = 1;
state.inst.release_us = 1_000_000;
state.release_age = Some(0);
state.release_level = 1.0;
state.age = (x * 48_000.0) as u64;
let x = state.age as f64 / 48_000.0;
let expected = ((-6.0 * x).exp() - (-6.0f64).exp()) / (1.0 - (-6.0f64).exp());
assert!((envelope(&state, &mixer.exponential) - expected).abs() < 1e-4);
}
let music = bank(
vec![note(0, 1), MusicEvent { tick: 2, kind: 2, a: 1, ..Default::default() }],
Instrument { looping: true, release_us: 1000, ..Default::default() },
20_000,
1000,
);
mixer.process_command(
AudioCommand::MusicPlay { group: 0, bank: music, looping: false },
&mut Packet::new(),
);
mixer.mix(&mut [0.0; 8]);
assert!(mixer.voices[0].active);
mixer.mix(&mut [0.0; 100]);
assert!(!mixer.voices[0].active);
}
#[test]
fn legato_retains_cursor_and_envelope_only_for_same_zone() {
let (banks, mut audio, mut mixer) = pair();
let mut legato = note(1, 2);
legato.flags = 1;
legato.c = 72;
legato.duration_frames = 10;
legato.curve = 1;
install(&banks, 0, vec![note(0, 1), legato]);
audio.music_play(0, 0, false);
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 8]);
assert_eq!(mixer.voices[..12].iter().filter(|v| v.active).count(), 1);
assert_eq!(mixer.states[0].age, 4);
assert!((mixer.states[0].cached_cents - 124.8).abs() < 1e-10);
assert_eq!(mixer.states[0].note_id, 2);
assert_eq!(mixer.states[0].glide.to, 1200.0);
}
#[test]
fn pause_freezes_sequence_envelopes_fade_and_effect_timeline() {
let (banks, mut audio, mut mixer) = pair();
install(&banks, 0, vec![note(0, 1)]);
audio.music_play(0, 0, true);
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 6]);
audio.control(AudioControl::MusicPause { group: 0, paused: true });
publish(&mut audio, &mut mixer);
let before = (mixer.groups[0].time, mixer.states[0].age, mixer.effects[0].write);
let mut pcm = [1.0; 200];
mixer.mix(&mut pcm);
assert_eq!(pcm, [0.0; 200]);
assert_eq!(before, (mixer.groups[0].time, mixer.states[0].age, mixer.effects[0].write));
audio.control(AudioControl::MusicPause { group: 0, paused: false });
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 2]);
assert_eq!(mixer.states[0].age, before.1 + 1);
}
#[test]
fn crossfade_stops_outgoing_group_and_target_keeps_its_sequence() {
let (banks, mut audio, mut mixer) = pair();
install(&banks, 0, vec![note(0, 1)]);
audio.music_play(0, 0, true);
audio.music_play(1, 0, true);
audio.control(AudioControl::MusicVolume { group: 1, gain: 0 });
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 10]);
audio.control(AudioControl::Crossfade { source: 0, target: 1, frames: 10, smooth: true });
publish(&mut audio, &mut mixer);
mixer.fill_buffer(&mut [0.0; 20]);
assert!(mixer.groups[0].bank.is_none());
assert_eq!(mixer.groups[1].frame, 15);
assert_eq!(mixer.groups[1].gain.at(15), 1.0);
assert!(mixer.voices[12].active);
}
#[test]
fn shared_delay_has_selective_sends_and_sfx_tails_survive_unload() {
let (banks, mut audio, mut mixer) = pair();
banks.install_sound_bank(
0,
Arc::new(SoundBank::new(vec![Arc::new(Sample::new(48_000, vec![16384]))])),
);
audio.control(AudioControl::Effect {
group: 2,
cutoff_hz: 0,
delay_frames: 2,
feedback: 0.5,
wet: 1.0,
dry: 0.0,
});
audio.play(0, 0, 0, 255, 0, 1.0, 0, LoopMode::Off);
audio.control(AudioControl::SfxSend { voice: 0, gain: 255 });
publish(&mut audio, &mut mixer);
let mut first = [0.0; 2];
mixer.mix(&mut first);
assert_eq!(first, [0.0; 2]);
banks.unload_audio_bank(false, 0);
let mut tail = [0.0; 8];
mixer.fill_buffer(&mut tail);
assert_eq!(tail, [0.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.25, 0.0]);
}
#[test]
fn replacing_or_unloading_bank_invalidates_queued_play_and_only_its_resident_instance() {
let (banks, mut audio, mut mixer) = pair();
let source = Arc::new(SoundBank::new(vec![Arc::new(Sample::new(48_000, vec![16384; 4]))]));
banks.install_sound_bank(0, source.clone());
banks.install_sound_bank(1, source);
audio.play(0, 0, 0, 255, 0, 1.0, 0, LoopMode::On);
audio.play(1, 0, 1, 255, 0, 1.0, 0, LoopMode::On);
audio.finish_frame();
banks.unload_audio_bank(false, 0);
mixer.apply_delivery(audio.transport().try_take().unwrap());
assert!(!mixer.voices[24].active);
assert!(mixer.voices[25].active);
install(&banks, 0, vec![note(0, 1)]);
audio.music_play(0, 0, true);
audio.music_play(1, 0, true);
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 2]);
banks.unload_audio_bank(true, 0);
mixer.fill_buffer(&mut [0.0; 2]);
assert!(mixer.voices[..24].iter().all(|v| !v.active));
assert!(mixer.groups.iter().all(|g| g.bank.is_none()));
assert!(mixer.voices[25].active);
}
#[test]
fn stealing_respects_release_order_and_serialized_onsets_within_one_frame() {
let (banks, mut audio, mut mixer) = pair();
install(&banks, 0, (0..14).map(|n| note(0, n)).collect());
audio.music_play(0, 0, false);
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 2]);
assert_eq!(mixer.states[0].note_id, 12);
assert_eq!(mixer.states[1].note_id, 13);
let mut events: Vec<_> = (0..12).map(|n| note(0, n)).collect();
events.push(MusicEvent { tick: 1, kind: 2, a: 4, ..Default::default() });
events.push(MusicEvent { tick: 2, kind: 2, a: 2, ..Default::default() });
events.push(note(3, 20));
banks.install_music_bank(
0,
bank(
events,
Instrument { looping: true, release_us: 1_000_000, ..Default::default() },
20000,
1000,
),
);
audio.music_play(0, 0, false);
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 8]);
assert_eq!(mixer.states[4].note_id, 20);
assert!(mixer.states[2].release_age.is_some());
}
#[test]
fn incompatible_legato_retriggers_the_same_voice_with_no_hidden_polyphony() {
let sample = Arc::new(Sample::new(48000, vec![1000; 8]).with_loop(0, 8));
let mut legato = note(2, 2);
legato.flags = 1;
legato.c = 72;
let music = MusicBank::compile(
"zones".into(),
vec![sample.clone(), sample],
vec![
Instrument { low: 0, high: 63, looping: true, ..Default::default() },
Instrument { low: 64, high: 127, sample: 1, looping: true, ..Default::default() },
],
vec![note(0, 1), legato],
960,
20000,
1000,
None,
1,
)
.unwrap();
let mut mixer = AudioMixer::new();
mixer.process_command(
AudioCommand::MusicPlay { group: 0, bank: Arc::new(music), looping: false },
&mut Packet::new(),
);
mixer.mix(&mut [0.0; 6]);
assert_eq!(mixer.voices[..12].iter().filter(|v| v.active).count(), 1);
assert_eq!(mixer.states[0].age, 1);
assert_eq!(mixer.states[0].zone, 1);
assert_eq!(mixer.states[0].note_id, 2);
}
#[test]
fn parameter_ramps_lfos_and_send_override_use_the_current_audio_value() {
let mut send = SendRamp::default();
assert_eq!(send.at(0, 0.8), 0.8);
send.set(0.0, 0, 100, false);
assert_eq!(send.at(50, 0.8), 0.4);
send.set(1.0, 50, 100, false);
assert_eq!(send.at(50, 0.8), 0.4);
assert!((send.at(100, 0.8) - 0.7).abs() < 1e-12);
assert_eq!(send.at(150, 0.8), 1.0);
let events = vec![
note(0, 1),
MusicEvent { kind: 4, c: 0, duration_frames: 480, ..Default::default() },
MusicEvent { kind: 5, c: 255, duration_frames: 480, ..Default::default() },
MusicEvent { kind: 6, c: 1200, duration_frames: 480, ..Default::default() },
MusicEvent { kind: 7, a: 100000, c: 1200, ..Default::default() },
MusicEvent { kind: 8, a: 100000, c: 65535, ..Default::default() },
];
let mut mixer = AudioMixer::new();
mixer.process_command(
AudioCommand::MusicPlay {
group: 0,
bank: bank(events, Instrument { looping: true, ..Default::default() }, 20000, 1000),
looping: false,
},
&mut Packet::new(),
);
let mut pcm = [0.0; 962];
mixer.mix(&mut pcm);
assert_eq!(&pcm[..2], &[1.0 / 64.0, -1.0 / 32.0]);
assert!(pcm[240].abs() < 1e-8 && pcm[241].abs() < 1e-8);
assert_eq!(&pcm[960..], &[0.0, 0.0]);
assert!(mixer.voices[0].active);
assert_eq!(mixer.groups[0].tracks[0].gain.at(240), 0.5);
assert_eq!(mixer.groups[0].tracks[0].pan.at(240), 191.5);
assert_eq!(mixer.groups[0].tracks[0].pitch.at(240), 600.0);
let mut ramp = Ramp::constant(0.0);
ramp.set(1.0, 0, 100, true);
assert_eq!(ramp.at(0), 0.0);
assert_eq!(ramp.at(50), 0.5);
assert_eq!(ramp.at(100), 1.0);
let endpoint = 1.0 / 65535.0;
ramp.set(endpoint, 100, 100, false);
assert_eq!(ramp.at(200), endpoint);
assert_eq!(ramp.at(1000), endpoint);
}
#[test]
fn music_sfx_and_master_gains_remain_independent_of_group_gain() {
let (_, mut audio, mut mixer) = pair();
audio.play_sample(
Arc::new(Sample::new(48000, vec![16384; 4])),
0,
255,
0,
1.0,
0,
LoopMode::On,
);
audio.control(AudioControl::GroupVolume { group: 2, gain: 128 });
audio.control(AudioControl::BusVolume { bus: 1, gain: 128 });
audio.control(AudioControl::BusVolume { bus: 2, gain: 128 });
publish(&mut audio, &mut mixer);
let mut pcm = [0.0; 2];
mixer.mix(&mut pcm);
assert!((pcm[0] - 0.5 * (128.0f32 / 255.0).powi(3)).abs() < 1e-7);
assert_eq!(pcm[1], 0.0);
}
#[test]
fn invalidating_one_music_slot_clears_only_its_effect_history() {
let (banks, mut audio, mut mixer) = pair();
install(&banks, 0, vec![note(0, 1)]);
install(&banks, 1, vec![note(0, 1)]);
audio.music_play(0, 0, true);
audio.music_play(1, 1, true);
for group in 0..2 {
audio.control(AudioControl::Effect {
group,
cutoff_hz: 6000,
delay_frames: 48,
feedback: 0.5,
wet: 0.3,
dry: 0.7,
});
}
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 100]);
banks.unload_audio_bank(true, 0);
mixer.retire_inactive(&mut Packet::new());
assert_eq!(mixer.effects[0].history, 0);
assert_eq!(mixer.effects[1].history, 50);
assert!(mixer.voices[12].active);
}
#[test]
fn crossfade_with_absent_target_does_not_silence_the_active_source() {
let (banks, mut audio, mut mixer) = pair();
install(&banks, 0, vec![note(0, 1)]);
audio.music_play(0, 0, true);
audio.control(AudioControl::Crossfade { source: 0, target: 1, frames: 0, smooth: false });
publish(&mut audio, &mut mixer);
mixer.mix(&mut [0.0; 2]);
assert!(mixer.groups[0].active);
assert_eq!(mixer.groups[0].gain.at(1), 1.0);
}
#[test]
fn resident_memory_diagnostics_deduplicate_shared_pcm_across_all_slots() {
let banks = MemoryBanks::new();
let sample = Arc::new(Sample::new(48000, vec![1, 2, 3, 4]));
let source = Arc::new(SoundBank::new(vec![sample]));
for slot in 0..16 {
banks.install_sound_bank(slot, source.clone());
banks.install_music_bank(slot, bank(vec![note(0, 1)], Instrument::default(), 20000, 1000));
}
let snapshot = banks.audio_memory();
assert_eq!(snapshot.banks.len(), 32);
assert_eq!(snapshot.unique_pcm_bytes, 8 + 16 * 8);
assert!(snapshot.metadata_bytes > 0);
assert_eq!(
snapshot.total_accounted_bytes,
snapshot.allocated_pcm_bytes + snapshot.metadata_bytes
);
}
#[test]
fn group_configuration_survives_play_stop_and_replacement_but_not_session_reset() {
let (banks, mut audio, mut mixer) = pair();
install(&banks, 0, vec![note(0, 1)]);
audio.control(AudioControl::MusicVolume { group: 0, gain: 0 });
audio.music_play(0, 0, true);
publish(&mut audio, &mut mixer);
let mut pcm = [1.0; 2];
mixer.mix(&mut pcm);
assert_eq!(pcm, [0.0; 2]);
assert_eq!(mixer.groups[0].gain.at(0), 0.0);
audio.control(AudioControl::Effect {
group: 0,
cutoff_hz: 1000,
delay_frames: 48,
feedback: 0.5,
wet: 0.5,
dry: 0.5,
});
audio.control(AudioControl::MusicStop { group: 0 });
audio.music_play(0, 0, true);
publish(&mut audio, &mut mixer);
assert_eq!(mixer.groups[0].gain.at(0), 0.0);
assert_eq!(mixer.effects[0].frames, 48);
audio.reset();
mixer.apply_delivery(audio.transport().try_take().unwrap());
assert_eq!(mixer.groups[0].gain.at(0), 1.0);
assert_eq!(mixer.effects[0].frames, 0);
assert_eq!(mixer.effects[0].coefficient, 1.0);
assert_eq!(mixer.effects[0].wet, 0.0);
assert_eq!(mixer.effects[0].dry, 1.0);
}
@@ -14,6 +14,7 @@ struct Mailbox {
paused: bool, paused: bool,
frame: Option<Vec<AudioCommand>>, frame: Option<Vec<AudioCommand>>,
lifecycle: Option<Vec<AudioCommand>>, lifecycle: Option<Vec<AudioCommand>>,
recycled: [Option<Vec<AudioCommand>>; 4],
} }
pub struct AudioDelivery { pub struct AudioDelivery {
@@ -36,7 +37,31 @@ impl AudioTransport {
} }
pub fn consume_silently(&self) { pub fn consume_silently(&self) {
let _ = self.try_take(); if let Some(delivery) = self.try_take() {
for batch in [delivery.frame, delivery.lifecycle].into_iter().flatten() {
self.recycle_commands(batch);
}
}
}
/// Main/service lane only: drop command owners and retain bounded empty storage.
pub fn recycle_commands(&self, mut commands: Vec<AudioCommand>) {
commands.clear();
let mut state = self.0.lock().expect("audio mailbox");
if let Some(slot) = state.recycled.iter_mut().find(|slot| slot.is_none()) {
*slot = Some(commands);
}
// Excess empty buffers are destroyed here, never in the data callback.
}
pub(super) fn take_buffer(&self) -> Vec<AudioCommand> {
self.0
.lock()
.expect("audio mailbox")
.recycled
.iter_mut()
.find_map(Option::take)
.unwrap_or_default()
} }
pub fn ready(&self) -> bool { pub fn ready(&self) -> bool {
@@ -84,3 +109,37 @@ mod tests {
assert!(transport.try_take().is_some()); assert!(transport.try_take().is_some());
} }
} }
#[cfg(test)]
mod reuse_tests {
use super::*;
#[test]
fn service_recycles_storage_and_releases_sample_owners() {
use prometeu_hal::{LoopMode, sample::Sample};
let transport = AudioTransport::default();
let sample = Arc::new(Sample::new(48000, vec![1]));
let weak = Arc::downgrade(&sample);
let mut batch = Vec::with_capacity(1024);
let allocation = batch.as_ptr();
batch.push(AudioCommand::Play {
sample,
alive: None,
voice_id: 0,
volume: 255,
pan: 0,
pitch: 1.0,
priority: 0,
loop_mode: LoopMode::Off,
});
transport.recycle_commands(batch);
assert!(weak.upgrade().is_none());
let reused = transport.take_buffer();
assert_eq!(reused.as_ptr(), allocation);
assert_eq!(reused.capacity(), 1024);
assert!(reused.is_empty());
for _ in 0..8 {
transport.recycle_commands(Vec::with_capacity(16));
}
assert_eq!(transport.0.lock().unwrap().recycled.iter().flatten().count(), 4);
}
}
@@ -664,7 +664,7 @@ mod tests {
assert!(frame_composer.bind_scene(0)); assert!(frame_composer.bind_scene(0));
let cache = frame_composer.cache().expect("cache should exist for bound scene"); let cache = frame_composer.cache().expect("cache should exist for bound scene");
assert_eq!((cache.width(), cache.height()), (65, 38)); assert_eq!((cache.width(), cache.height()), (85, 49));
} }
#[test] #[test]
@@ -43,6 +43,8 @@ pub struct Gfx {
front: Vec<u32>, front: Vec<u32>,
/// Back buffer: the working buffer where canonical game frames are composed /// Back buffer: the working buffer where canonical game frames are composed
back: Vec<u32>, back: Vec<u32>,
/// Complete local framebuffer publications, including native Hub/Shell output.
publication_count: u64,
/// Shared access to graphical memory banks (tiles and palettes). /// Shared access to graphical memory banks (tiles and palettes).
pub glyph_banks: Arc<dyn GlyphBankPoolAccess>, pub glyph_banks: Arc<dyn GlyphBankPoolAccess>,
@@ -321,6 +323,7 @@ impl Gfx {
h, h,
front: vec![0; len], front: vec![0; len],
back: vec![0; len], back: vec![0; len],
publication_count: 0,
glyph_banks, glyph_banks,
sprites: [EMPTY_SPRITE; 512], sprites: [EMPTY_SPRITE; 512],
sprite_count: 0, sprite_count: 0,
@@ -570,6 +573,11 @@ impl Gfx {
/// Typically called by the Host when it's time to display the finished frame. /// Typically called by the Host when it's time to display the finished frame.
pub fn present(&mut self) { pub fn present(&mut self) {
std::mem::swap(&mut self.front, &mut self.back); std::mem::swap(&mut self.front, &mut self.back);
self.publication_count = self.publication_count.wrapping_add(1);
}
pub fn publication_count(&self) -> u64 {
self.publication_count
} }
pub fn load_frame_sprites(&mut self, sprites: &[Sprite]) { pub fn load_frame_sprites(&mut self, sprites: &[Sprite]) {
@@ -28,7 +28,7 @@ use std::sync::Arc;
/// for easier testing and different host implementations (Desktop, Web, etc.). /// for easier testing and different host implementations (Desktop, Web, etc.).
/// ///
/// ### Console Specifications: /// ### Console Specifications:
/// - **Resolution**: 480x270 (16:9 Aspect Ratio). /// - **Resolution**: 640x360 (16:9 Aspect Ratio).
/// - **Color Format**: RGBA8888. /// - **Color Format**: RGBA8888.
/// - **Audio**: Stereo, Command-based mixing. /// - **Audio**: Stereo, Command-based mixing.
/// - **Input**: 12-button Digital Gamepad + Absolute Touch/Mouse. /// - **Input**: 12-button Digital Gamepad + Absolute Touch/Mouse.
@@ -201,9 +201,9 @@ impl RuntimePlatform for Hardware {
impl Hardware { impl Hardware {
/// Internal hardware width in pixels. /// Internal hardware width in pixels.
pub const W: usize = 480; pub const W: usize = prometeu_hal::graphics_contract::WIDTH;
/// Internal hardware height in pixels. /// Internal hardware height in pixels.
pub const H: usize = 270; pub const H: usize = prometeu_hal::graphics_contract::HEIGHT;
/// Creates a fresh hardware instance with default settings. /// Creates a fresh hardware instance with default settings.
pub fn new(wall_clock: Arc<dyn WallClock>) -> Self { pub fn new(wall_clock: Arc<dyn WallClock>) -> Self {
@@ -5,12 +5,15 @@ mod gfx;
pub mod hardware; pub mod hardware;
mod local_render_backend; mod local_render_backend;
mod memory_banks; mod memory_banks;
#[cfg(test)]
mod native_graphics_tests;
mod pad; mod pad;
mod test_platform; mod test_platform;
mod touch; mod touch;
mod wall_clock; mod wall_clock;
pub use crate::asset::AssetManager; pub use crate::asset::AssetManager;
pub use crate::audio::bank_format;
pub use crate::audio::{ pub use crate::audio::{
Audio, AudioCommand, AudioDelivery, AudioTransport, Channel, MAX_CHANNELS, Audio, AudioCommand, AudioDelivery, AudioTransport, Channel, MAX_CHANNELS,
MAX_COMMANDS_PER_FRAME, OUTPUT_SAMPLE_RATE, MAX_COMMANDS_PER_FRAME, OUTPUT_SAMPLE_RATE,
@@ -27,3 +30,5 @@ pub use crate::test_platform::TestPlatform;
pub use crate::wall_clock::ControlledWallClock; pub use crate::wall_clock::ControlledWallClock;
pub mod memcard; pub mod memcard;
pub use crate::audio::{SFX_CHANNELS, SFX_START, engine};
+14 -3
View File
@@ -13,7 +13,8 @@ fn io_error(e: std::io::Error) -> Error {
use std::io::ErrorKind as K; use std::io::ErrorKind as K;
match e.kind() { match e.kind() {
K::NotFound => Error::Unavailable, K::NotFound => Error::Unavailable,
K::PermissionDenied => Error::ReadOnly, K::PermissionDenied => Error::AccessDenied,
K::AlreadyExists => Error::AlreadyExists,
K::StorageFull => Error::NoSpace, K::StorageFull => Error::NoSpace,
K::InvalidInput | K::UnexpectedEof => Error::InvalidRange, K::InvalidInput | K::UnexpectedEof => Error::InvalidRange,
K::Unsupported => Error::Unsupported, K::Unsupported => Error::Unsupported,
@@ -21,8 +22,15 @@ fn io_error(e: std::io::Error) -> Error {
} }
} }
impl FileMemcard { impl FileMemcard {
fn lock(file: &File) -> Result<(), Error> {
file.try_lock().map_err(|error| match error {
std::fs::TryLockError::WouldBlock => Error::Busy,
std::fs::TryLockError::Error(error) => io_error(error),
})
}
pub fn open(path: &Path, read_only: bool) -> Result<Self, Error> { pub fn open(path: &Path, read_only: bool) -> Result<Self, Error> {
let file = OpenOptions::new().read(true).write(!read_only).open(path).map_err(io_error)?; let file = OpenOptions::new().read(true).write(!read_only).open(path).map_err(io_error)?;
Self::lock(&file)?;
let length = file.metadata().map_err(io_error)?.len(); let length = file.metadata().map_err(io_error)?.len();
Ok(Self { file, length, read_only }) Ok(Self { file, length, read_only })
} }
@@ -47,7 +55,9 @@ impl FileMemcard {
.create_new(true) .create_new(true)
.open(&temporary) .open(&temporary)
.map_err(io_error)?; .map_err(io_error)?;
let mut published = false;
let result = (|| { let result = (|| {
Self::lock(&file)?;
let mut card = Self { file, length, read_only: false }; let mut card = Self { file, length, read_only: false };
let zeros = [0u8; 65_536]; let zeros = [0u8; 65_536];
let mut offset = 0; let mut offset = 0;
@@ -64,6 +74,7 @@ impl FileMemcard {
card.sync()?; card.sync()?;
// hard_link publishes without replacing a user file; parent sync makes the name durable. // hard_link publishes without replacing a user file; parent sync makes the name durable.
std::fs::hard_link(&temporary, path).map_err(io_error)?; std::fs::hard_link(&temporary, path).map_err(io_error)?;
published = true;
File::open(parent).and_then(|f| f.sync_all()).map_err(io_error)?; File::open(parent).and_then(|f| f.sync_all()).map_err(io_error)?;
Ok(card) Ok(card)
})(); })();
@@ -71,10 +82,10 @@ impl FileMemcard {
std::fs::remove_file(&temporary).and_then(|()| File::open(parent)?.sync_all()); std::fs::remove_file(&temporary).and_then(|()| File::open(parent)?.sync_all());
match result { match result {
Ok(card) => { Ok(card) => {
cleanup.map_err(io_error)?; cleanup.map_err(|_| Error::Indeterminate)?;
Ok(card) Ok(card)
} }
Err(e) => Err(e), Err(e) => Err(if published { Error::Indeterminate } else { e }),
} }
} }
#[cfg(not(unix))] #[cfg(not(unix))]
@@ -7,6 +7,7 @@
//! the render read surface. //! the render read surface.
use prometeu_hal::glyph_bank::GlyphBank; use prometeu_hal::glyph_bank::GlyphBank;
use prometeu_hal::music_bank::MusicBank;
use prometeu_hal::render_resources::RenderResourceAccess as HalRenderResourceAccess; use prometeu_hal::render_resources::RenderResourceAccess as HalRenderResourceAccess;
use prometeu_hal::scene_bank::SceneBank; use prometeu_hal::scene_bank::SceneBank;
use prometeu_hal::sound_bank::SoundBank; use prometeu_hal::sound_bank::SoundBank;
@@ -30,6 +31,7 @@ pub trait GlyphBankPoolInstaller: Send + Sync {
/// Non-generic interface for peripherals to access sound banks. /// Non-generic interface for peripherals to access sound banks.
pub trait SoundBankPoolAccess: Send + Sync { pub trait SoundBankPoolAccess: Send + Sync {
fn music_bank_slot(&self, slot: usize) -> Option<Arc<MusicBank>>;
/// Returns a reference to the resident SoundBank in the specified slot, if any. /// Returns a reference to the resident SoundBank in the specified slot, if any.
fn sound_bank_slot(&self, slot: usize) -> Option<Arc<SoundBank>>; fn sound_bank_slot(&self, slot: usize) -> Option<Arc<SoundBank>>;
/// Returns the total number of slots available in this bank. /// Returns the total number of slots available in this bank.
@@ -38,6 +40,9 @@ pub trait SoundBankPoolAccess: Send + Sync {
/// Non-generic interface for the AssetManager to install sound banks. /// Non-generic interface for the AssetManager to install sound banks.
pub trait SoundBankPoolInstaller: Send + Sync { pub trait SoundBankPoolInstaller: Send + Sync {
fn install_music_bank(&self, slot: usize, bank: Arc<MusicBank>);
fn clear_music_banks(&self);
fn unload_audio_bank(&self, music: bool, slot: usize);
/// Atomically swaps the resident SoundBank in the specified slot. /// Atomically swaps the resident SoundBank in the specified slot.
fn install_sound_bank(&self, slot: usize, bank: Arc<SoundBank>); fn install_sound_bank(&self, slot: usize, bank: Arc<SoundBank>);
/// Clears all resident SoundBank slots. /// Clears all resident SoundBank slots.
@@ -68,6 +73,7 @@ pub trait SceneBankPoolInstaller: Send + Sync {
pub struct MemoryBanks { pub struct MemoryBanks {
glyph_bank_pool: Arc<RwLock<[Option<Arc<GlyphBank>>; 16]>>, glyph_bank_pool: Arc<RwLock<[Option<Arc<GlyphBank>>; 16]>>,
sound_bank_pool: Arc<RwLock<[Option<Arc<SoundBank>>; 16]>>, sound_bank_pool: Arc<RwLock<[Option<Arc<SoundBank>>; 16]>>,
music_bank_pool: Arc<RwLock<[Option<Arc<MusicBank>>; 16]>>,
scene_bank_pool: Arc<RwLock<[Option<Arc<SceneBank>>; 16]>>, scene_bank_pool: Arc<RwLock<[Option<Arc<SceneBank>>; 16]>>,
} }
@@ -83,6 +89,7 @@ impl MemoryBanks {
Self { Self {
glyph_bank_pool: Arc::new(RwLock::new(std::array::from_fn(|_| None))), glyph_bank_pool: Arc::new(RwLock::new(std::array::from_fn(|_| None))),
sound_bank_pool: Arc::new(RwLock::new(std::array::from_fn(|_| None))), sound_bank_pool: Arc::new(RwLock::new(std::array::from_fn(|_| None))),
music_bank_pool: Arc::new(RwLock::new(std::array::from_fn(|_| None))),
scene_bank_pool: Arc::new(RwLock::new(std::array::from_fn(|_| None))), scene_bank_pool: Arc::new(RwLock::new(std::array::from_fn(|_| None))),
} }
} }
@@ -113,6 +120,9 @@ impl GlyphBankPoolInstaller for MemoryBanks {
} }
impl SoundBankPoolAccess for MemoryBanks { impl SoundBankPoolAccess for MemoryBanks {
fn music_bank_slot(&self, slot: usize) -> Option<Arc<MusicBank>> {
self.music_bank_pool.read().unwrap().get(slot).and_then(Clone::clone)
}
fn sound_bank_slot(&self, slot: usize) -> Option<Arc<SoundBank>> { fn sound_bank_slot(&self, slot: usize) -> Option<Arc<SoundBank>> {
let pool = self.sound_bank_pool.read().unwrap(); let pool = self.sound_bank_pool.read().unwrap();
pool.get(slot).and_then(|s| s.as_ref().map(Arc::clone)) pool.get(slot).and_then(|s| s.as_ref().map(Arc::clone))
@@ -124,15 +134,61 @@ impl SoundBankPoolAccess for MemoryBanks {
} }
impl SoundBankPoolInstaller for MemoryBanks { impl SoundBankPoolInstaller for MemoryBanks {
fn install_music_bank(&self, slot: usize, bank: Arc<MusicBank>) {
if slot >= 16 {
return;
}
let prepared = Arc::new(bank.resident_copy());
let mut pool = self.music_bank_pool.write().unwrap();
if let Some(target) = pool.get_mut(slot) {
if let Some(old) = target.take() {
old.invalidate();
}
*target = Some(prepared);
}
}
fn clear_music_banks(&self) {
let mut pool = self.music_bank_pool.write().unwrap();
for bank in pool.iter_mut() {
if let Some(old) = bank.take() {
old.invalidate();
}
}
}
fn unload_audio_bank(&self, music: bool, slot: usize) {
if music {
if let Some(bank) =
self.music_bank_pool.write().unwrap().get_mut(slot).and_then(Option::take)
{
bank.invalidate();
}
} else if let Some(bank) =
self.sound_bank_pool.write().unwrap().get_mut(slot).and_then(Option::take)
{
bank.invalidate();
}
}
fn install_sound_bank(&self, slot: usize, bank: Arc<SoundBank>) { fn install_sound_bank(&self, slot: usize, bank: Arc<SoundBank>) {
if slot >= 16 {
return;
}
let prepared = Arc::new(bank.resident_copy());
let mut pool = self.sound_bank_pool.write().unwrap(); let mut pool = self.sound_bank_pool.write().unwrap();
if slot < 16 { if slot < 16 {
pool[slot] = Some(bank); if let Some(old) = pool[slot].take() {
old.invalidate();
}
pool[slot] = Some(prepared);
} }
} }
fn clear_sound_banks(&self) { fn clear_sound_banks(&self) {
self.sound_bank_pool.write().unwrap().fill(None); let mut pool = self.sound_bank_pool.write().unwrap();
for bank in pool.iter_mut() {
if let Some(old) = bank.take() {
old.invalidate();
}
}
} }
} }
@@ -246,3 +302,84 @@ mod tests {
assert_eq!(HalRenderResourceAccess::scene_bank_count(&banks), 16); assert_eq!(HalRenderResourceAccess::scene_bank_count(&banks), 16);
} }
} }
/// Owner-thread diagnostics: canonical bank costs and unique PCM allocation costs
/// are separate, so loading the same asset in two slots does not double-count RAM.
#[derive(Debug, Clone)]
pub struct AudioSlotMemory {
pub music: bool,
pub slot: usize,
pub name: String,
pub metrics: prometeu_hal::sound_bank::AudioBankMetrics,
}
#[derive(Debug, Default)]
pub struct AudioMemorySnapshot {
pub banks: Vec<AudioSlotMemory>,
pub unique_pcm_bytes: usize,
pub allocated_pcm_bytes: usize,
/// Resident Rust structs/vector/string capacities; excludes allocator/Arc headers and host caches.
pub metadata_bytes: usize,
pub total_accounted_bytes: usize,
}
impl MemoryBanks {
pub fn audio_memory(&self) -> AudioMemorySnapshot {
let mut snapshot = AudioMemorySnapshot::default();
let mut seen = std::collections::HashSet::new();
let mut metadata = 0usize;
let mut vectors = std::collections::HashSet::new();
let mut add =
|music, slot, name: String, metrics, samples: &[Arc<prometeu_hal::sample::Sample>]| {
for sample in samples {
if seen.insert(Arc::as_ptr(sample)) {
metadata += std::mem::size_of::<prometeu_hal::sample::Sample>();
snapshot.unique_pcm_bytes += sample.data.len() * 2;
snapshot.allocated_pcm_bytes += sample.data.capacity() * 2;
}
}
snapshot.banks.push(AudioSlotMemory { music, slot, name, metrics });
};
for (slot, bank) in self.sound_bank_pool.read().unwrap().iter().enumerate() {
if let Some(b) = bank {
add(false, slot, b.name.clone(), b.metrics, &b.samples);
snapshot.metadata_bytes += std::mem::size_of::<SoundBank>()
+ b.name.capacity()
+ std::mem::size_of::<std::sync::atomic::AtomicBool>();
if vectors.insert(Arc::as_ptr(&b.samples) as usize) {
snapshot.metadata_bytes += std::mem::size_of::<
Vec<Arc<prometeu_hal::sample::Sample>>,
>() + b.samples.capacity()
* std::mem::size_of::<Arc<prometeu_hal::sample::Sample>>();
}
}
}
for (slot, bank) in self.music_bank_pool.read().unwrap().iter().enumerate() {
if let Some(b) = bank {
add(true, slot, b.name.clone(), b.metrics, &b.samples);
snapshot.metadata_bytes += std::mem::size_of::<MusicBank>()
+ b.name.capacity()
+ std::mem::size_of::<std::sync::atomic::AtomicBool>();
if vectors.insert(Arc::as_ptr(&b.samples) as usize) {
snapshot.metadata_bytes += std::mem::size_of::<
Vec<Arc<prometeu_hal::sample::Sample>>,
>() + b.samples.capacity()
* std::mem::size_of::<Arc<prometeu_hal::sample::Sample>>();
}
if vectors.insert(Arc::as_ptr(&b.instruments) as usize) {
snapshot.metadata_bytes += std::mem::size_of::<
Vec<prometeu_hal::music_bank::Instrument>,
>() + b.instruments.capacity()
* std::mem::size_of::<prometeu_hal::music_bank::Instrument>();
}
if vectors.insert(Arc::as_ptr(&b.events) as usize) {
snapshot.metadata_bytes += std::mem::size_of::<
Vec<prometeu_hal::music_bank::TimedMusicEvent>,
>() + b.events.capacity()
* std::mem::size_of::<prometeu_hal::music_bank::TimedMusicEvent>();
}
}
}
snapshot.metadata_bytes += metadata;
snapshot.total_accounted_bytes = snapshot.allocated_pcm_bytes + snapshot.metadata_bytes;
snapshot
}
}
@@ -0,0 +1,109 @@
use crate::{Gfx, GlyphBankPoolInstaller, MemoryBanks};
use prometeu_hal::color::Color;
use prometeu_hal::glyph::Glyph;
use prometeu_hal::glyph_bank::{GlyphBank, TileSize};
use prometeu_hal::graphics_contract::{FRAME_BYTES, HEIGHT, PIXELS, WIDTH};
use prometeu_hal::scene_bank::SceneBank;
use prometeu_hal::scene_layer::{ParallaxFactor, SceneLayer};
use prometeu_hal::scene_viewport_cache::SceneViewportCache;
use prometeu_hal::scene_viewport_resolver::SceneViewportResolver;
use prometeu_hal::tile::Tile;
use prometeu_hal::tilemap::TileMap;
use std::sync::Arc;
#[test]
fn native_allocation_stride_corners_and_external_coordinates() {
let mut gfx = Gfx::new(WIDTH, HEIGHT, Arc::new(MemoryBanks::new()));
assert_eq!(gfx.publication_count(), 0);
gfx.clear(Color::BLACK);
for (x, y) in [(0, 0), (639, 0), (0, 359), (639, 359)] {
gfx.draw_pixel(x, y, Color::RED);
}
for (x, y) in [(-1, 0), (640, 359), (639, 360), (0, -1)] {
gfx.draw_pixel(x, y, Color::GREEN);
}
assert_eq!(gfx.publication_count(), 0, "drawing is not a complete publication");
gfx.present();
assert_eq!(gfx.publication_count(), 1);
let front = gfx.front_buffer();
assert_eq!(gfx.size(), (640, 360));
assert_eq!(front.len(), PIXELS);
assert_eq!(std::mem::size_of_val(front), FRAME_BYTES);
for index in [0, 639, 359 * WIDTH, PIXELS - 1] {
assert_eq!(front[index], Color::RED.raw());
}
assert_eq!(front.iter().filter(|&&p| p == Color::RED.raw()).count(), 4);
gfx.present();
assert_eq!(gfx.publication_count(), 2, "each complete swap is a publication");
}
#[test]
fn native_tile_edges_and_mixed_camera_layers_cover_every_corner() {
for sizes in [
[TileSize::Size8; 4],
[TileSize::Size16; 4],
[TileSize::Size32; 4],
[TileSize::Size8, TileSize::Size16, TileSize::Size32, TileSize::Size8],
] {
let banks = Arc::new(MemoryBanks::new());
let colors = [Color::RED, Color::GREEN, Color::BLUE, Color::WHITE];
for i in 0..4 {
let n = sizes[i] as usize;
let mut bank = GlyphBank::new(sizes[i], n, n);
bank.palettes[0][1] = colors[i];
bank.pixel_indices.fill(1);
banks.install_glyph_bank(i, Arc::new(bank));
}
let scene = SceneBank {
layers: std::array::from_fn(|i| SceneLayer {
active: true,
glyph_asset_id: i as i32,
tile_size: sizes[i],
parallax_factor: ParallaxFactor { x: 1.0, y: 1.0 },
tilemap: TileMap {
width: 160,
height: 120,
tiles: vec![
Tile {
active: true,
glyph: Glyph { glyph_id: 0, palette_id: 0 },
flip_x: true,
flip_y: true,
};
160 * 120
],
},
}),
};
let min = sizes.iter().map(|s| *s as usize).min().unwrap();
let (cw, ch) = (WIDTH.div_ceil(min) + 5, HEIGHT.div_ceil(min) + 4);
let mut cache = SceneViewportCache::new(&scene, cw, ch);
let mut resolver = SceneViewportResolver::new(
WIDTH as i32,
HEIGHT as i32,
cw,
ch,
min as i32 - 4,
min as i32 + 4,
);
let mut gfx = Gfx::new(WIDTH, HEIGHT, banks);
for (x, y) in [(0, 0), (1, 1), (7, 7), (15, 15), (31, 31), (400, 300), (2, 3)] {
let update = resolver.update(&scene, x, y);
for request in &update.copy_requests {
cache.move_layer_window_to(
request.layer_index,
request.cache_origin_tile_x,
request.cache_origin_tile_y,
);
}
cache.materialize_all_layers(&scene);
gfx.render_scene_from_cache(&cache, &update, &[0, 1, 2, 3]);
gfx.present();
// Every pixel, including rows 352..359, must be supplied by the last layer.
assert!(
gfx.front_buffer().iter().all(|&p| p == Color::WHITE.raw()),
"sizes={sizes:?}, camera=({x},{y})"
);
}
}
}
@@ -18,6 +18,8 @@ impl ControlledWallClock {
} }
/// Advance civil time explicitly, saturating at the largest timestamp. /// Advance civil time explicitly, saturating at the largest timestamp.
// `try_update` requires Rust 1.95; keep older stable CI toolchains compatible.
#[allow(deprecated)]
pub fn advance(&self, seconds: u64) { pub fn advance(&self, seconds: u64) {
// Harness control may retry on a concurrent update; reads remain a single load. // Harness control may retry on a concurrent update; reads remain a single load.
let _ = self.seconds.fetch_update(Ordering::Relaxed, Ordering::Relaxed, |current| { let _ = self.seconds.fetch_update(Ordering::Relaxed, Ordering::Relaxed, |current| {
@@ -609,7 +609,7 @@ mod tests {
} }
fn front_buffer_pixel(platform: &TestPlatform, x: i32, y: i32) -> u32 { fn front_buffer_pixel(platform: &TestPlatform, x: i32, y: i32) -> u32 {
const FRAMEBUFFER_WIDTH: usize = 480; const FRAMEBUFFER_WIDTH: usize = prometeu_hal::graphics_contract::WIDTH;
let index = x as usize + y as usize * FRAMEBUFFER_WIDTH; let index = x as usize + y as usize * FRAMEBUFFER_WIDTH;
platform.local_hardware().gfx.front_buffer()[index] platform.local_hardware().gfx.front_buffer()[index]
} }
@@ -1024,6 +1024,10 @@ mod tests {
platform.assets().slot_info(SlotRef { asset_type: BankType::SCENE, index: 0 }).asset_id, platform.assets().slot_info(SlotRef { asset_type: BankType::SCENE, index: 0 }).asset_id,
Some(1) Some(1)
); );
assert_eq!(
platform.assets().slot_info(SlotRef { asset_type: BankType::MUSIC, index: 0 }).asset_id,
Some(201)
);
firmware.request_home_from_host(); firmware.request_home_from_host();
firmware.tick(&InputSignals::default(), &mut platform); firmware.tick(&InputSignals::default(), &mut platform);
@@ -1038,6 +1042,10 @@ mod tests {
platform.assets().slot_info(SlotRef { asset_type: BankType::SCENE, index: 0 }).asset_id, platform.assets().slot_info(SlotRef { asset_type: BankType::SCENE, index: 0 }).asset_id,
Some(1) Some(1)
); );
assert_eq!(
platform.assets().slot_info(SlotRef { asset_type: BankType::MUSIC, index: 0 }).asset_id,
Some(201)
);
firmware.tick(&InputSignals::default(), &mut platform); firmware.tick(&InputSignals::default(), &mut platform);
let dummy_click = let dummy_click =
@@ -1051,6 +1059,10 @@ mod tests {
platform.assets().slot_info(SlotRef { asset_type: BankType::SCENE, index: 0 }).asset_id, platform.assets().slot_info(SlotRef { asset_type: BankType::SCENE, index: 0 }).asset_id,
None None
); );
assert_eq!(
platform.assets().slot_info(SlotRef { asset_type: BankType::MUSIC, index: 0 }).asset_id,
None
);
assert!(platform.local_hardware().memory_banks.glyph_bank_slot(0).is_none()); assert!(platform.local_hardware().memory_banks.glyph_bank_slot(0).is_none());
assert!(platform.local_hardware().memory_banks.scene_bank_slot(0).is_none()); assert!(platform.local_hardware().memory_banks.scene_bank_slot(0).is_none());
@@ -1078,6 +1090,7 @@ mod tests {
); );
firmware.tick(&InputSignals::default(), &mut platform); firmware.tick(&InputSignals::default(), &mut platform);
wait_for_cartridge_preload(&mut firmware, &mut platform);
match &firmware.state { match &firmware.state {
FirmwareState::GameRunning(step) => assert_eq!(step.task_id, dummy_task), FirmwareState::GameRunning(step) => assert_eq!(step.task_id, dummy_task),
@@ -1094,7 +1107,9 @@ mod tests {
assert_eq!(dummy_ownership.app_id, 2); assert_eq!(dummy_ownership.app_id, 2);
let front = platform.local_hardware().gfx.front_buffer(); let front = platform.local_hardware().gfx.front_buffer();
let center = 240 + 135 * 480; let center = prometeu_hal::graphics_contract::WIDTH / 2
+ (prometeu_hal::graphics_contract::HEIGHT / 2)
* prometeu_hal::graphics_contract::WIDTH;
assert_eq!(front[center], Color::GREEN.raw()); assert_eq!(front[center], Color::GREEN.raw());
firmware.request_home_from_host(); firmware.request_home_from_host();
@@ -1126,6 +1141,10 @@ mod tests {
platform.assets().slot_info(SlotRef { asset_type: BankType::SCENE, index: 0 }).asset_id, platform.assets().slot_info(SlotRef { asset_type: BankType::SCENE, index: 0 }).asset_id,
Some(1) Some(1)
); );
assert_eq!(
platform.assets().slot_info(SlotRef { asset_type: BankType::MUSIC, index: 0 }).asset_id,
Some(201)
);
assert!(platform.local_hardware().memory_banks.glyph_bank_slot(0).is_some()); assert!(platform.local_hardware().memory_banks.glyph_bank_slot(0).is_some());
assert!(platform.local_hardware().memory_banks.scene_bank_slot(0).is_some()); assert!(platform.local_hardware().memory_banks.scene_bank_slot(0).is_some());
@@ -1512,8 +1531,12 @@ mod tests {
#[test] #[test]
fn shell_running_close_click_uses_lifecycle_and_returns_hub_home() { fn shell_running_close_click_uses_lifecycle_and_returns_hub_home() {
let (mut firmware, mut platform, _signals, task_id) = load_shell_running_firmware(); let (mut firmware, mut platform, _signals, task_id) = load_shell_running_firmware();
let close_signals = let close_signals = InputSignals {
InputSignals { f_signal: true, x_pos: 420, y_pos: 24, ..Default::default() }; f_signal: true,
x_pos: prometeu_hal::graphics_contract::WIDTH as i32 - 60,
y_pos: 24,
..Default::default()
};
firmware.tick(&close_signals, &mut platform); firmware.tick(&close_signals, &mut platform);
@@ -47,7 +47,12 @@ impl HubHomeStep {
let id = ctx.os.windows().add_window( let id = ctx.os.windows().add_window(
app.title().to_string(), app.title().to_string(),
WindowOwner::Task(task_id), WindowOwner::Task(task_id),
Rect { x: 40, y: 20, w: 400, h: 220 }, Rect {
x: 40,
y: 20,
w: prometeu_hal::graphics_contract::WIDTH as i32 - 80,
h: prometeu_hal::graphics_contract::HEIGHT as i32 - 50,
},
app.color(), app.color(),
); );
ctx.os.windows().set_focus(id); ctx.os.windows().set_focus(id);
@@ -48,6 +48,25 @@ impl LoadCartridgeStep {
pub fn needs_memcard_choice(&self) -> bool { pub fn needs_memcard_choice(&self) -> bool {
self.awaiting_memcard && !self.preparing_memcard self.awaiting_memcard && !self.preparing_memcard
} }
/// Typed host and UI admission share this single policy boundary.
pub fn choose_memcard(
&mut self,
os: &mut prometeu_system::SystemOS,
choice: prometeu_system::memcard::admission::Choice,
created_at: u64,
) -> prometeu_system::memcard::image::Result<()> {
if !self.needs_memcard_choice() {
return Err(prometeu_system::memcard::image::Error::InvalidState);
}
os.memcard().choose_for_game(
self.cartridge.app_id,
self.cartridge.memcard_slots.unwrap() as usize,
choice,
created_at,
)?;
self.preparing_memcard = true;
Ok(())
}
pub fn new_game_switch(cartridge: Cartridge) -> Self { pub fn new_game_switch(cartridge: Cartridge) -> Self {
Self { failure_recovery: LoadCartridgeFailureRecovery::ReturnToHub, ..Self::new(cartridge) } Self { failure_recovery: LoadCartridgeFailureRecovery::ReturnToHub, ..Self::new(cartridge) }
@@ -128,13 +147,8 @@ impl LoadCartridgeStep {
Choice::MemoryExisting Choice::MemoryExisting
}; };
let created_at = ctx.platform.wall_clock().utc_unix_seconds(); let created_at = ctx.platform.wall_clock().utc_unix_seconds();
match ctx.os.memcard().choose_for_game( match self.choose_memcard(ctx.os, choice, created_at) {
self.cartridge.app_id, Ok(()) => {}
self.cartridge.memcard_slots.unwrap() as usize,
choice,
created_at,
) {
Ok(()) => self.preparing_memcard = true,
Err(error) => self.memcard_message = Self::preparation_message(error), Err(error) => self.memcard_message = Self::preparation_message(error),
} }
} }
+15 -11
View File
@@ -8,6 +8,7 @@ pub type AssetId = i32;
pub enum BankType { pub enum BankType {
GLYPH, GLYPH,
SOUNDS, SOUNDS,
MUSIC,
SCENE, SCENE,
// TILEMAPS, // TILEMAPS,
// BLOBS, // BLOBS,
@@ -41,9 +42,8 @@ pub struct GlyphBankMetadata {
} }
#[derive(Debug, Clone, Deserialize, Serialize)] #[derive(Debug, Clone, Deserialize, Serialize)]
pub struct SoundBankMetadata { pub struct AudioBankMetadata {
pub sample_rate: u32, pub version: u16,
pub channels: u32,
} }
#[derive(Debug, Clone, Default, Deserialize, Serialize)] #[derive(Debug, Clone, Default, Deserialize, Serialize)]
@@ -73,15 +73,16 @@ impl AssetEntry {
.map_err(|e| format!("Invalid GLYPH metadata for asset {}: {}", self.asset_id, e)) .map_err(|e| format!("Invalid GLYPH metadata for asset {}: {}", self.asset_id, e))
} }
pub fn metadata_as_sound_bank(&self) -> Result<SoundBankMetadata, String> { pub fn metadata_as_audio_bank(&self) -> Result<AudioBankMetadata, String> {
if self.bank_type != BankType::SOUNDS { if !matches!(self.bank_type, BankType::SOUNDS | BankType::MUSIC) {
return Err(format!( return Err("asset is not an audio bank".into());
"Asset {} is not a SOUNDS bank (type: {:?})",
self.asset_id, self.bank_type
));
} }
serde_json::from_value(self.metadata.clone()) let metadata: AudioBankMetadata = serde_json::from_value(self.metadata.clone())
.map_err(|e| format!("Invalid SOUNDS metadata for asset {}: {}", self.asset_id, e)) .map_err(|e| format!("Invalid audio metadata: {e}"))?;
if metadata.version != 1 {
return Err("unsupported audio metadata version".into());
}
Ok(metadata)
} }
pub fn metadata_as_scene_bank(&self) -> Result<SceneBankMetadata, String> { pub fn metadata_as_scene_bank(&self) -> Result<SceneBankMetadata, String> {
@@ -189,6 +190,9 @@ pub struct SlotRef {
} }
impl SlotRef { impl SlotRef {
pub fn music(index: usize) -> Self {
Self { asset_type: BankType::MUSIC, index }
}
pub fn gfx(index: usize) -> Self { pub fn gfx(index: usize) -> Self {
Self { asset_type: BankType::GLYPH, index } Self { asset_type: BankType::GLYPH, index }
} }
@@ -6,6 +6,9 @@ use crate::asset::{
use crate::cartridge::AssetsPayloadSource; use crate::cartridge::AssetsPayloadSource;
pub trait AssetBridge { pub trait AssetBridge {
fn unload_audio(&self, _music: bool, _slot: usize) -> AssetOpStatus {
AssetOpStatus::InvalidState
}
/// Bind before cartridge preload/loading. Hosts supporting memcard must use /// Bind before cartridge preload/loading. Hosts supporting memcard must use
/// this same lane for assets; an unsupported bridge refuses the fast profile. /// this same lane for assets; an unsupported bridge refuses the fast profile.
fn bind_background_work( fn bind_background_work(
@@ -20,6 +20,20 @@ pub enum AudioOpStatus {
CommandLimit = 7, CommandLimit = 7,
} }
/// High-level commands are staged with the ordinary frame/lifecycle batch.
#[derive(Debug, Clone, Copy)]
pub enum AudioControl {
GroupVolume { group: usize, gain: u8 },
MusicStop { group: usize },
MusicPause { group: usize, paused: bool },
MusicVolume { group: usize, gain: u8 },
MusicFade { group: usize, gain: u8, frames: u32, smooth: bool },
Crossfade { source: usize, target: usize, frames: u32, smooth: bool },
BusVolume { bus: usize, gain: u8 },
Effect { group: usize, cutoff_hz: u32, delay_frames: u32, feedback: f32, wet: f32, dry: f32 },
SfxSend { voice: usize, gain: u8 },
}
pub trait AudioBridge { pub trait AudioBridge {
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
fn play( fn play(
@@ -44,6 +58,8 @@ pub trait AudioBridge {
priority: u8, priority: u8,
loop_mode: LoopMode, loop_mode: LoopMode,
) -> AudioOpStatus; ) -> AudioOpStatus;
fn music_play(&mut self, group: usize, bank: usize, looping: bool) -> AudioOpStatus;
fn control(&mut self, control: AudioControl) -> AudioOpStatus;
fn stop(&mut self, voice_id: usize) -> AudioOpStatus; fn stop(&mut self, voice_id: usize) -> AudioOpStatus;
fn set_volume(&mut self, voice_id: usize, volume: u8) -> AudioOpStatus; fn set_volume(&mut self, voice_id: usize, volume: u8) -> AudioOpStatus;
fn set_pan(&mut self, voice_id: usize, pan: u8) -> AudioOpStatus; fn set_pan(&mut self, voice_id: usize, pan: u8) -> AudioOpStatus;
@@ -0,0 +1,20 @@
//! Fixed machine graphics contract. Host surfaces never change these dimensions.
pub const WIDTH: usize = 640;
pub const HEIGHT: usize = 360;
pub const REFRESH_HZ: u32 = 60;
/// Canonical RGBA8888, with tightly packed native rows.
pub const BYTES_PER_PIXEL: usize = 4;
pub const PIXELS: usize = WIDTH * HEIGHT;
pub const STRIDE_PIXELS: usize = WIDTH;
pub const STRIDE_BYTES: usize = STRIDE_PIXELS * BYTES_PER_PIXEL;
pub const FRAME_BYTES: usize = STRIDE_BYTES * HEIGHT;
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn native_graphics_contract_is_fixed() {
assert_eq!((WIDTH, HEIGHT, REFRESH_HZ), (640, 360, 60));
assert_eq!((PIXELS, STRIDE_BYTES, FRAME_BYTES), (230_400, 2_560, 921_600));
}
}
+2
View File
@@ -12,11 +12,13 @@ pub mod debugger_protocol;
pub mod gfx_bridge; pub mod gfx_bridge;
pub mod glyph; pub mod glyph;
pub mod glyph_bank; pub mod glyph_bank;
pub mod graphics_contract;
pub mod host_context; pub mod host_context;
pub mod host_return; pub mod host_return;
pub mod input_signals; pub mod input_signals;
pub mod log; pub mod log;
pub mod memcard; pub mod memcard;
pub mod music_bank;
pub mod native_helpers; pub mod native_helpers;
pub mod native_interface; pub mod native_interface;
pub mod owned_frame; pub mod owned_frame;
@@ -57,6 +57,21 @@ impl LogService {
self.events.iter().filter(|e| e.seq > seq).cloned().collect() self.events.iter().filter(|e| e.seq > seq).cloned().collect()
} }
/// Non-destructive page in native sequence order. A null cursor includes zero.
pub fn get_page(&self, after: Option<u64>, limit: usize) -> Vec<LogEvent> {
self.events
.iter()
.filter(|event| after.is_none_or(|seq| event.seq > seq))
.take(limit)
.cloned()
.collect()
}
/// Retained bounds, independent of any reader's progress.
pub fn sequence_bounds(&self) -> (Option<u64>, Option<u64>) {
(self.events.front().map(|event| event.seq), self.last_seq())
}
pub fn last_seq(&self) -> Option<u64> { pub fn last_seq(&self) -> Option<u64> {
self.events.back().map(|e| e.seq) self.events.back().map(|e| e.seq)
} }
@@ -107,4 +122,35 @@ mod tests {
assert_eq!(after[0].msg, "Log 3"); assert_eq!(after[0].msg, "Log 3");
assert_eq!(after[1].msg, "Log 4"); assert_eq!(after[1].msg, "Log 4");
} }
#[test]
fn pages_preserve_zero_requery_and_retained_bounds() {
let mut service = LogService::new(3);
assert_eq!(service.sequence_bounds(), (None, None));
assert!(service.get_page(None, 2).is_empty());
for i in 0..3 {
service.log(i, i, LogLevel::Info, LogSource::Pos, 7, format!("{i}"));
}
assert_eq!(service.sequence_bounds(), (Some(0), Some(2)));
for _ in 0..2 {
let page = service.get_page(None, 2);
assert_eq!(page.iter().map(|event| event.seq).collect::<Vec<_>>(), vec![0, 1]);
}
assert_eq!(service.get_page(Some(1), 2)[0].seq, 2);
assert!(service.get_page(Some(2), 2).is_empty());
service.log(3, 3, LogLevel::Info, LogSource::Pos, 7, "next".into());
assert_eq!(service.sequence_bounds(), (Some(1), Some(3)));
assert_eq!(service.get_page(Some(2), 2)[0].seq, 3);
assert_eq!(service.get_page(None, 10).len(), 3);
}
#[test]
fn page_cursors_preserve_large_unsigned_sequences() {
let mut service = LogService::new(3);
service.next_seq = u64::MAX - 2;
service.log(0, 0, LogLevel::Info, LogSource::Pos, 0, "first".into());
service.log(0, 0, LogLevel::Info, LogSource::Pos, 0, "second".into());
assert_eq!(service.get_page(Some(u64::MAX - 2), 1)[0].seq, u64::MAX - 1);
assert!(service.get_page(Some(u64::MAX), 1).is_empty());
}
} }
@@ -6,6 +6,10 @@ pub enum MemcardIoError {
InvalidRange, InvalidRange,
NoSpace, NoSpace,
Unsupported, Unsupported,
Busy,
AccessDenied,
AlreadyExists,
Indeterminate,
IoError, IoError,
} }
@@ -0,0 +1,376 @@
//! Resident, self-contained music data. Construction/compilation is off the audio lane.
use crate::{sample::Sample, sound_bank::AudioBankMetrics};
use std::sync::{
Arc,
atomic::{AtomicBool, Ordering},
};
pub const AUDIO_PPQ: u32 = 960;
pub const AUDIO_RATE: u32 = 48_000;
pub const MAX_MUSIC_TRACKS: usize = 256;
pub const MAX_EVENTS_PER_FRAME: usize = 64;
#[derive(Debug, Clone, Copy)]
pub struct Instrument {
pub id: u16,
pub sample: u16,
pub low: u8,
pub high: u8,
pub root: u8,
pub looping: bool,
pub tuning_cents: i16,
pub pan: u8,
pub gain: u8,
pub attack_us: u32,
pub decay_us: u32,
pub sustain: u16,
pub release_us: u32,
pub vibrato_millihz: u32,
pub vibrato_cents: u16,
pub tremolo_depth: u16,
pub tremolo_millihz: u32,
pub send: u16,
pub curve: u8,
}
impl Default for Instrument {
fn default() -> Self {
Self {
id: 0,
sample: 0,
low: 0,
high: 127,
root: 60,
looping: false,
tuning_cents: 0,
pan: 128,
gain: 255,
attack_us: 0,
decay_us: 0,
sustain: u16::MAX,
release_us: 0,
vibrato_millihz: 0,
vibrato_cents: 0,
tremolo_depth: 0,
tremolo_millihz: 0,
send: 0,
curve: 1,
}
}
}
#[derive(Debug, Clone, Copy, Default)]
pub struct MusicEvent {
pub tick: u32,
pub kind: u8,
pub track: u8,
pub flags: u16,
pub a: u32,
pub b: u32,
pub c: i32,
pub d: i32,
pub duration_frames: u32,
pub curve: u32,
}
#[derive(Debug, Clone, Copy)]
pub struct TimedMusicEvent {
pub time: u128,
pub event: MusicEvent,
pub zone: usize,
}
#[derive(Debug)]
pub struct MusicBank {
pub name: String,
pub samples: Arc<Vec<Arc<Sample>>>,
pub instruments: Arc<Vec<Instrument>>,
pub events: Arc<Vec<TimedMusicEvent>>,
pub denominator: u128,
pub end_time: u128,
pub loop_time: Option<u128>,
pub loop_event: usize,
pub tracks: usize,
pub alive: Arc<AtomicBool>,
pub metrics: AudioBankMetrics,
}
impl MusicBank {
#[allow(clippy::too_many_arguments)]
pub fn compile(
name: String,
samples: Vec<Arc<Sample>>,
instruments: Vec<Instrument>,
events: Vec<MusicEvent>,
ppq: u32,
tempo: u32,
duration: u32,
loop_tick: Option<u32>,
tracks: usize,
) -> Result<Self, String> {
let mut compiler = MusicCompiler::new(
name,
samples,
instruments,
events,
ppq,
tempo,
duration,
loop_tick,
tracks,
)?;
loop {
if let Some(bank) = compiler.step()? {
return Ok(bank);
}
}
}
pub fn invalidate(&self) {
self.alive.store(false, Ordering::Release);
}
pub fn resident_copy(&self) -> Self {
Self {
name: self.name.clone(),
samples: self.samples.clone(),
instruments: self.instruments.clone(),
events: self.events.clone(),
denominator: self.denominator,
end_time: self.end_time,
loop_time: self.loop_time,
loop_event: self.loop_event,
tracks: self.tracks,
alive: Arc::new(AtomicBool::new(true)),
metrics: self.metrics,
}
}
}
/// Cooperative off-lane validation/clock compilation. At most 64 zones/events
/// per step; lookup is logarithmic and the loop-seam check is bounded by 65.
pub struct MusicCompiler {
name: Option<String>,
samples: Vec<Arc<Sample>>,
instruments: Vec<Instrument>,
source: std::vec::IntoIter<MusicEvent>,
compiled: Vec<TimedMusicEvent>,
zones: std::collections::BTreeMap<(u16, u8), usize>,
zone_cursor: usize,
ppq: u32,
tempo: u32,
duration: u32,
loop_tick: Option<u32>,
tracks: usize,
tick: u32,
time: u128,
loop_time: Option<u128>,
selected: [u32; MAX_MUSIC_TRACKS],
}
impl MusicCompiler {
#[allow(clippy::too_many_arguments)]
pub fn new(
name: String,
samples: Vec<Arc<Sample>>,
instruments: Vec<Instrument>,
events: Vec<MusicEvent>,
ppq: u32,
tempo: u32,
duration: u32,
loop_tick: Option<u32>,
tracks: usize,
) -> Result<Self, String> {
if ppq != AUDIO_PPQ
|| tempo == 0
|| duration == 0
|| tracks == 0
|| tracks > MAX_MUSIC_TRACKS
|| loop_tick.is_some_and(|t| t >= duration)
{
return Err("invalid music clock/tracks/loop".into());
}
let mut compiled = Vec::new();
compiled.try_reserve_exact(events.len()).map_err(|_| "music allocation failed")?;
Ok(Self {
name: Some(name),
samples,
instruments,
source: events.into_iter(),
compiled,
zones: std::collections::BTreeMap::new(),
zone_cursor: 0,
ppq,
tempo,
duration,
loop_tick,
tracks,
tick: 0,
time: 0,
loop_time: None,
selected: [u32::MAX; MAX_MUSIC_TRACKS],
})
}
pub fn step(&mut self) -> Result<Option<MusicBank>, String> {
let denominator = self.ppq as u128 * 1_000_000;
for _ in 0..64 {
if self.zone_cursor < self.instruments.len() {
let i = self.zone_cursor;
let inst = self.instruments[i];
if inst.low > inst.high
|| inst.high > 127
|| inst.root > 127
|| inst.curve > 1
|| inst.sample as usize >= self.samples.len()
|| inst.vibrato_millihz > 100_000
|| inst.tremolo_millihz > 100_000
|| inst.vibrato_cents > 2400
{
return Err("invalid instrument zone/modulation".into());
}
if inst.looping && self.samples[inst.sample as usize].loop_start.is_none() {
return Err("instrument requires a sample loop".into());
}
if self
.zones
.range((inst.id, 0)..=(inst.id, inst.high))
.next_back()
.is_some_and(|(_, j)| self.instruments[*j].high >= inst.low)
{
return Err("overlapping instrument zones".into());
}
self.zones.insert((inst.id, inst.low), i);
self.zone_cursor += 1;
continue;
}
let Some(event) = self.source.next() else {
return self.finish(denominator).map(Some);
};
if event.tick < self.tick
|| event.tick > self.duration
|| event.track as usize >= self.tracks
|| event.curve > 1
|| event.flags & !1 != 0
|| !(1..=10).contains(&event.kind)
{
return Err("invalid music event ordering/shape".into());
}
if let Some(t) = self.loop_tick.filter(|&t| t >= self.tick && t <= event.tick)
&& self.loop_time.is_none()
{
self.loop_time = Some(
self.time + (t - self.tick) as u128 * self.tempo as u128 * AUDIO_RATE as u128,
);
}
self.time += (event.tick - self.tick) as u128 * self.tempo as u128 * AUDIO_RATE as u128;
self.tick = event.tick;
let mut zone = 0;
match event.kind {
1 => {
if !(0..=127).contains(&event.c) || !(0..=255).contains(&event.d) {
return Err("invalid note/velocity".into());
}
let id = if event.b == u32::MAX {
self.selected[event.track as usize]
} else {
event.b
};
if id > u16::MAX as u32 {
return Err("note has no bank-local instrument".into());
}
zone = *self
.zones
.range((id as u16, 0)..=(id as u16, event.c as u8))
.next_back()
.map(|(_, i)| i)
.filter(|&&i| self.instruments[i].high as i32 >= event.c)
.ok_or("note has no bank-local instrument zone")?;
}
3 => {
if event.a == 0 {
return Err("zero tempo".into());
}
self.tempo = event.a;
}
4 | 9 => {
if !(0..=65535).contains(&event.c) {
return Err("invalid gain/send".into());
}
}
5 => {
if !(0..=255).contains(&event.c) {
return Err("invalid pan".into());
}
}
6 => {
if !(-9600..=9600).contains(&event.c) {
return Err("invalid pitch bend".into());
}
}
7 | 8 => {
if event.a > 100_000
|| event.c < 0
|| event.c > if event.kind == 7 { 2400 } else { 65535 }
{
return Err("invalid modulation".into());
}
}
10 => {
if event.a > u16::MAX as u32
|| self
.zones
.range((event.a as u16, 0)..=(event.a as u16, 127))
.next()
.is_none()
{
return Err("unknown instrument".into());
}
self.selected[event.track as usize] = event.a;
}
_ => {}
}
if self.compiled.len() >= MAX_EVENTS_PER_FRAME
&& self.time - self.compiled[self.compiled.len() - MAX_EVENTS_PER_FRAME].time
< denominator
{
return Err("music event density exceeds real-time bound".into());
}
self.compiled.push(TimedMusicEvent { time: self.time, event, zone });
}
Ok(None)
}
fn finish(&mut self, denominator: u128) -> Result<MusicBank, String> {
if let Some(t) = self.loop_tick.filter(|&t| t >= self.tick)
&& self.loop_time.is_none()
{
self.loop_time =
Some(self.time + (t - self.tick) as u128 * self.tempo as u128 * AUDIO_RATE as u128);
}
let end_time = self.time
+ (self.duration - self.tick) as u128 * self.tempo as u128 * AUDIO_RATE as u128;
if end_time < denominator || self.loop_time.is_some_and(|t| end_time - t < denominator) {
return Err("music/loop shorter than one output frame".into());
}
let loop_event =
self.loop_time.map_or(0, |t| self.compiled.partition_point(|e| e.time < t));
let length = end_time - self.loop_time.unwrap_or(0);
let slice = &self.compiled[loop_event..];
for i in 0..slice.len().min(MAX_EVENTS_PER_FRAME + 1) {
let tail =
slice.len().saturating_sub(MAX_EVENTS_PER_FRAME - i.min(MAX_EVENTS_PER_FRAME));
if tail < slice.len()
&& slice.len() - tail + i + 1 > MAX_EVENTS_PER_FRAME
&& slice[i].time + length - slice[tail].time < denominator
{
return Err("music loop seam event density".into());
}
}
let pcm_bytes = self.samples.iter().map(|s| s.data.len() * 2).sum();
Ok(MusicBank {
name: self.name.take().ok_or("compiler already complete")?,
samples: Arc::new(std::mem::take(&mut self.samples)),
instruments: Arc::new(std::mem::take(&mut self.instruments)),
events: Arc::new(std::mem::take(&mut self.compiled)),
denominator,
end_time,
loop_time: self.loop_time,
loop_event,
tracks: self.tracks,
alive: Arc::new(AtomicBool::new(true)),
metrics: AudioBankMetrics { pcm_bytes, ..Default::default() },
})
}
}
+1 -1
View File
@@ -64,7 +64,7 @@ pub trait RuntimePlatform {
fn wall_clock(&self) -> &dyn WallClock; fn wall_clock(&self) -> &dyn WallClock;
fn display_size(&self) -> (usize, usize) { fn display_size(&self) -> (usize, usize) {
(480, 270) (crate::graphics_contract::WIDTH, crate::graphics_contract::HEIGHT)
} }
fn render_submission_sink(&mut self) -> &mut dyn RenderSubmissionSink; fn render_submission_sink(&mut self) -> &mut dyn RenderSubmissionSink;
+24 -2
View File
@@ -1,13 +1,27 @@
#[derive(Debug)]
pub struct Sample { pub struct Sample {
pub sample_rate: u32, pub sample_rate: u32,
pub channels: u8,
pub data: Vec<i16>, pub data: Vec<i16>,
pub loop_start: Option<u32>, pub loop_start: Option<u32>,
pub loop_end: Option<u32>, pub loop_end: Option<u32>,
pub default_volume: u8,
pub default_pan: u8,
pub default_pitch: f64,
} }
impl Sample { impl Sample {
pub fn new(sample_rate: u32, data: Vec<i16>) -> Self { pub fn new(sample_rate: u32, data: Vec<i16>) -> Self {
Self { sample_rate, data, loop_start: None, loop_end: None } Self {
sample_rate,
channels: 1,
data,
loop_start: None,
loop_end: None,
default_volume: 255,
default_pan: 128,
default_pitch: 1.0,
}
} }
pub fn with_loop(mut self, start: u32, end: u32) -> Self { pub fn with_loop(mut self, start: u32, end: u32) -> Self {
@@ -17,6 +31,14 @@ impl Sample {
} }
pub fn frames_len(&self) -> usize { pub fn frames_len(&self) -> usize {
self.data.len() self.data.len() / self.channels.max(1) as usize
}
pub fn with_channels(mut self, channels: u8) -> Result<Self, String> {
if !(1..=2).contains(&channels) || !self.data.len().is_multiple_of(channels as usize) {
return Err("audio sample channels/frame count".into());
}
self.channels = channels;
Ok(self)
} }
} }
@@ -224,6 +224,8 @@ impl SceneViewportResolver {
scene_height_tiles: usize, scene_height_tiles: usize,
) -> TileAnchor { ) -> TileAnchor {
let mut next = current; let mut next = current;
// Mixed layers must not step past the center and immediately step back.
let trigger = self.hysteresis_trigger_px.max(tile_size_px / 2);
loop { loop {
let center_x_px = next.x * tile_size_px + tile_size_px / 2; let center_x_px = next.x * tile_size_px + tile_size_px / 2;
@@ -231,11 +233,11 @@ impl SceneViewportResolver {
if drift_x.abs() <= self.hysteresis_safe_px { if drift_x.abs() <= self.hysteresis_safe_px {
break; break;
} }
if drift_x > self.hysteresis_trigger_px { if drift_x > trigger {
next.x += 1; next.x += 1;
continue; continue;
} }
if drift_x < -self.hysteresis_trigger_px { if drift_x < -trigger {
next.x -= 1; next.x -= 1;
continue; continue;
} }
@@ -248,11 +250,11 @@ impl SceneViewportResolver {
if drift_y.abs() <= self.hysteresis_safe_px { if drift_y.abs() <= self.hysteresis_safe_px {
break; break;
} }
if drift_y > self.hysteresis_trigger_px { if drift_y > trigger {
next.y += 1; next.y += 1;
continue; continue;
} }
if drift_y < -self.hysteresis_trigger_px { if drift_y < -trigger {
next.y -= 1; next.y -= 1;
continue; continue;
} }
@@ -433,20 +435,34 @@ mod tests {
#[test] #[test]
fn first_update_initializes_master_and_layer_anchors() { fn first_update_initializes_master_and_layer_anchors() {
let scene = make_scene(); let scene = make_scene();
let mut resolver = SceneViewportResolver::new(480, 270, 35, 21, 12, 20); let mut resolver = SceneViewportResolver::new(
crate::graphics_contract::WIDTH as i32,
crate::graphics_contract::HEIGHT as i32,
45,
27,
12,
20,
);
let update = resolver.update(&scene, 0, 0); let update = resolver.update(&scene, 0, 0);
assert_eq!(update.master_anchor, TileAnchor { x: 17, y: 10 }); assert_eq!(update.master_anchor, TileAnchor { x: 22, y: 13 });
assert_eq!(update.layer_anchors[0], TileAnchor { x: 17, y: 10 }); assert_eq!(update.layer_anchors[0], TileAnchor { x: 22, y: 13 });
assert_eq!(update.layer_anchors[1], TileAnchor { x: 17, y: 10 }); assert_eq!(update.layer_anchors[1], TileAnchor { x: 22, y: 13 });
assert_eq!(update.refresh_requests.len(), 4); assert_eq!(update.refresh_requests.len(), 4);
} }
#[test] #[test]
fn per_layer_copy_requests_follow_parallax_factor() { fn per_layer_copy_requests_follow_parallax_factor() {
let scene = make_scene(); let scene = make_scene();
let mut resolver = SceneViewportResolver::new(480, 270, 35, 21, 12, 20); let mut resolver = SceneViewportResolver::new(
crate::graphics_contract::WIDTH as i32,
crate::graphics_contract::HEIGHT as i32,
45,
27,
12,
20,
);
let update = resolver.update(&scene, 32, 48); let update = resolver.update(&scene, 32, 48);
@@ -461,24 +477,38 @@ mod tests {
#[test] #[test]
fn hysteresis_prevents_small_back_and_forth_refresh_churn() { fn hysteresis_prevents_small_back_and_forth_refresh_churn() {
let scene = make_scene(); let scene = make_scene();
let mut resolver = SceneViewportResolver::new(480, 270, 35, 21, 12, 20); let mut resolver = SceneViewportResolver::new(
crate::graphics_contract::WIDTH as i32,
crate::graphics_contract::HEIGHT as i32,
45,
27,
12,
20,
);
let _ = resolver.update(&scene, 0, 0); let _ = resolver.update(&scene, 0, 0);
let update = resolver.update(&scene, 8, 0); let update = resolver.update(&scene, 8, 0);
assert!(update.refresh_requests.is_empty()); assert!(update.refresh_requests.is_empty());
assert_eq!(update.master_anchor, TileAnchor { x: 17, y: 10 }); assert_eq!(update.master_anchor, TileAnchor { x: 22, y: 13 });
} }
#[test] #[test]
fn repeated_high_speed_movement_advances_in_tile_steps() { fn repeated_high_speed_movement_advances_in_tile_steps() {
let scene = make_scene(); let scene = make_scene();
let mut resolver = SceneViewportResolver::new(480, 270, 35, 21, 12, 20); let mut resolver = SceneViewportResolver::new(
crate::graphics_contract::WIDTH as i32,
crate::graphics_contract::HEIGHT as i32,
45,
27,
12,
20,
);
let _ = resolver.update(&scene, 0, 0); let _ = resolver.update(&scene, 0, 0);
let update = resolver.update(&scene, 64, 0); let update = resolver.update(&scene, 64, 0);
assert!(update.master_anchor.x > 17); assert!(update.master_anchor.x > 22);
assert!(update.refresh_requests.iter().any(|request| matches!( assert!(update.refresh_requests.iter().any(|request| matches!(
request, request,
CacheRefreshRequest::RefreshColumn { layer_index: 0, .. } CacheRefreshRequest::RefreshColumn { layer_index: 0, .. }
@@ -489,18 +519,32 @@ mod tests {
#[test] #[test]
fn anchors_clamp_near_scene_edges() { fn anchors_clamp_near_scene_edges() {
let scene = make_scene(); let scene = make_scene();
let mut resolver = SceneViewportResolver::new(480, 270, 35, 21, 12, 20); let mut resolver = SceneViewportResolver::new(
crate::graphics_contract::WIDTH as i32,
crate::graphics_contract::HEIGHT as i32,
45,
27,
12,
20,
);
let update = resolver.update(&scene, 10_000, 10_000); let update = resolver.update(&scene, 10_000, 10_000);
assert_eq!(update.master_anchor, TileAnchor { x: 46, y: 53 }); assert_eq!(update.master_anchor, TileAnchor { x: 41, y: 50 });
assert_eq!(update.layer_anchors[0], TileAnchor { x: 46, y: 53 }); assert_eq!(update.layer_anchors[0], TileAnchor { x: 41, y: 50 });
} }
#[test] #[test]
fn corner_trigger_converts_to_non_overlapping_region_requests() { fn corner_trigger_converts_to_non_overlapping_region_requests() {
let scene = make_scene(); let scene = make_scene();
let mut resolver = SceneViewportResolver::new(480, 270, 35, 21, 12, 20); let mut resolver = SceneViewportResolver::new(
crate::graphics_contract::WIDTH as i32,
crate::graphics_contract::HEIGHT as i32,
45,
27,
12,
20,
);
let _ = resolver.update(&scene, 0, 0); let _ = resolver.update(&scene, 0, 0);
let update = resolver.update(&scene, 64, 80); let update = resolver.update(&scene, 64, 80);
@@ -515,15 +559,22 @@ mod tests {
.collect(); .collect();
assert_eq!(regions.len(), 2); assert_eq!(regions.len(), 2);
assert_eq!(regions[0].x + regions[0].width, 35); assert_eq!(regions[0].x + regions[0].width, 45);
assert_eq!(regions[0].height, 21); assert_eq!(regions[0].height, 27);
assert_eq!(regions[1].width, 34); assert_eq!(regions[1].width, 44);
assert_eq!(regions[1].height, 2); assert_eq!(regions[1].height, 2);
} }
#[test] #[test]
fn reset_scene_requests_full_invalidation_for_all_layers() { fn reset_scene_requests_full_invalidation_for_all_layers() {
let mut resolver = SceneViewportResolver::new(480, 270, 35, 21, 12, 20); let mut resolver = SceneViewportResolver::new(
crate::graphics_contract::WIDTH as i32,
crate::graphics_contract::HEIGHT as i32,
45,
27,
12,
20,
);
let requests = resolver.reset_scene(); let requests = resolver.reset_scene();
+33 -2
View File
@@ -1,16 +1,47 @@
use crate::sample::Sample; use crate::sample::Sample;
use std::sync::Arc; use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
#[derive(Debug, Clone, Copy, Default)]
pub struct AudioBankMetrics {
pub cartridge_bytes: usize,
pub pcm_bytes: usize,
pub canonical_decoded_bytes: usize,
pub materialization_ns: u64,
}
/// A container for audio assets. /// A container for audio assets.
/// ///
/// A SoundBank stores multiple audio samples that can be played by the /// A SoundBank stores multiple audio samples that can be played by the
/// audio subsystem. /// audio subsystem.
#[derive(Debug)]
pub struct SoundBank { pub struct SoundBank {
pub samples: Vec<Arc<Sample>>, pub samples: Arc<Vec<Arc<Sample>>>,
pub name: String,
pub alive: Arc<AtomicBool>,
pub metrics: AudioBankMetrics,
} }
impl SoundBank { impl SoundBank {
pub fn new(samples: Vec<Arc<Sample>>) -> Self { pub fn new(samples: Vec<Arc<Sample>>) -> Self {
Self { samples } let pcm_bytes = samples.iter().map(|s| s.data.len() * 2).sum();
Self {
samples: Arc::new(samples),
name: String::new(),
alive: Arc::new(AtomicBool::new(true)),
metrics: AudioBankMetrics { pcm_bytes, ..Default::default() },
}
}
pub fn invalidate(&self) {
self.alive.store(false, Ordering::Release);
}
pub fn resident_copy(&self) -> Self {
Self {
samples: self.samples.clone(),
name: self.name.clone(),
alive: Arc::new(AtomicBool::new(true)),
metrics: self.metrics,
}
} }
} }
@@ -59,6 +59,19 @@ pub enum Syscall {
AudioSetVolume = 0x3004, AudioSetVolume = 0x3004,
AudioSetPan = 0x3005, AudioSetPan = 0x3005,
AudioSetPitch = 0x3006, AudioSetPitch = 0x3006,
MusicPlay = 0x3010,
MusicStop = 0x3011,
MusicPause = 0x3012,
MusicResume = 0x3013,
MusicSetVolume = 0x3014,
MusicFadeTo = 0x3015,
MusicCrossfade = 0x3016,
AudioSetBusVolume = 0x3020,
AudioSetEffect = 0x3021,
AudioSetSend = 0x3022,
AudioUnloadBank = 0x3023,
AudioSetGroupVolume = 0x3024,
FsOpen = 0x4001, FsOpen = 0x4001,
FsRead = 0x4002, FsRead = 0x4002,
FsWrite = 0x4003, FsWrite = 0x4003,
@@ -31,4 +31,64 @@ pub(crate) const ENTRIES: &[SyscallRegistryEntry] = &[
.rets(1) .rets(1)
.caps(caps::AUDIO) .caps(caps::AUDIO)
.cost(5), .cost(5),
SyscallRegistryEntry::builder(Syscall::MusicPlay, "music", "play")
.args(3)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::MusicStop, "music", "stop")
.args(1)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::MusicPause, "music", "pause")
.args(1)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::MusicResume, "music", "resume")
.args(1)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::MusicSetVolume, "music", "set_volume")
.args(2)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::MusicFadeTo, "music", "fade_to")
.args(4)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::MusicCrossfade, "music", "crossfade")
.args(4)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::AudioSetBusVolume, "audio", "set_bus_volume")
.args(2)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::AudioSetEffect, "audio", "set_effect")
.args(6)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::AudioSetSend, "audio", "set_send")
.args(2)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::AudioUnloadBank, "audio", "unload_bank")
.args(2)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
SyscallRegistryEntry::builder(Syscall::AudioSetGroupVolume, "audio", "set_group_volume")
.args(2)
.rets(1)
.caps(caps::AUDIO)
.cost(5),
]; ];
@@ -37,6 +37,19 @@ impl Syscall {
0x3004 => Some(Self::AudioSetVolume), 0x3004 => Some(Self::AudioSetVolume),
0x3005 => Some(Self::AudioSetPan), 0x3005 => Some(Self::AudioSetPan),
0x3006 => Some(Self::AudioSetPitch), 0x3006 => Some(Self::AudioSetPitch),
0x3010 => Some(Self::MusicPlay),
0x3011 => Some(Self::MusicStop),
0x3012 => Some(Self::MusicPause),
0x3013 => Some(Self::MusicResume),
0x3014 => Some(Self::MusicSetVolume),
0x3015 => Some(Self::MusicFadeTo),
0x3016 => Some(Self::MusicCrossfade),
0x3020 => Some(Self::AudioSetBusVolume),
0x3021 => Some(Self::AudioSetEffect),
0x3022 => Some(Self::AudioSetSend),
0x3023 => Some(Self::AudioUnloadBank),
0x3024 => Some(Self::AudioSetGroupVolume),
0x4001 => Some(Self::FsOpen), 0x4001 => Some(Self::FsOpen),
0x4002 => Some(Self::FsRead), 0x4002 => Some(Self::FsRead),
0x4003 => Some(Self::FsWrite), 0x4003 => Some(Self::FsWrite),
@@ -110,6 +123,19 @@ impl Syscall {
Self::AudioSetVolume => "AudioSetVolume", Self::AudioSetVolume => "AudioSetVolume",
Self::AudioSetPan => "AudioSetPan", Self::AudioSetPan => "AudioSetPan",
Self::AudioSetPitch => "AudioSetPitch", Self::AudioSetPitch => "AudioSetPitch",
Self::MusicPlay => "MusicPlay",
Self::MusicStop => "MusicStop",
Self::MusicPause => "MusicPause",
Self::MusicResume => "MusicResume",
Self::MusicSetVolume => "MusicSetVolume",
Self::MusicFadeTo => "MusicFadeTo",
Self::MusicCrossfade => "MusicCrossfade",
Self::AudioSetBusVolume => "AudioSetBusVolume",
Self::AudioSetEffect => "AudioSetEffect",
Self::AudioSetSend => "AudioSetSend",
Self::AudioUnloadBank => "AudioUnloadBank",
Self::AudioSetGroupVolume => "AudioSetGroupVolume",
Self::FsOpen => "FsOpen", Self::FsOpen => "FsOpen",
Self::FsRead => "FsRead", Self::FsRead => "FsRead",
Self::FsWrite => "FsWrite", Self::FsWrite => "FsWrite",
@@ -385,6 +385,14 @@ impl<'a> VmFacade<'a> {
self.os.vm_runtime.record_repeated_render_worker_frame(frame_id); self.os.vm_runtime.record_repeated_render_worker_frame(frame_id);
} }
pub fn record_presented_render_worker_frame(
&self,
ownership: RenderOwnership,
frame_id: FrameId,
) {
self.os.vm_runtime.record_presented_render_worker_frame(ownership, frame_id);
}
pub fn active_render_ownership(&self) -> RenderOwnership { pub fn active_render_ownership(&self) -> RenderOwnership {
if let Some(task_id) = self.foreground_session_task_id() if let Some(task_id) = self.foreground_session_task_id()
&& let Some(session) = self.os.vm_sessions.for_task(task_id) && let Some(session) = self.os.vm_sessions.for_task(task_id)
@@ -64,6 +64,14 @@ impl SystemOS {
pub fn memcard(&mut self) -> &mut MemcardService { pub fn memcard(&mut self) -> &mut MemcardService {
&mut self.memcard &mut self.memcard
} }
/// A successful return confirms the shared worker has stopped and joined.
pub fn stop_background_io(&mut self) -> Result<(), crate::AsyncWorkLaneError> {
if let Some(controller) = &mut self.io_controller {
controller.stop()?;
}
self.io_controller = None;
Ok(())
}
/// Host/player operations are admitted only while gameplay is suspended. /// Host/player operations are admitted only while gameplay is suspended.
pub fn replace_memcard( pub fn replace_memcard(
@@ -162,6 +170,16 @@ impl SystemOS {
self.log_service.get_after(seq) self.log_service.get_after(seq)
} }
/// Coherent non-destructive page and retained bounds; does not execute the VM.
pub fn log_snapshot(
&self,
after: Option<u64>,
limit: usize,
) -> (Vec<LogEvent>, Option<u64>, Option<u64>) {
let (oldest, newest) = self.log_service.sequence_bounds();
(self.log_service.get_page(after, limit), oldest, newest)
}
pub fn frame_index(&mut self) -> u64 { pub fn frame_index(&mut self) -> u64 {
self.vm().logical_frame_index() self.vm().logical_frame_index()
} }
@@ -10,19 +10,41 @@ use prometeu_hal::{
const SHELL_A_BUTTON: Rect = Rect { x: 112, y: 172, w: 112, h: 32 }; const SHELL_A_BUTTON: Rect = Rect { x: 112, y: 172, w: 112, h: 32 };
const SHELL_B_BUTTON: Rect = Rect { x: 256, y: 172, w: 112, h: 32 }; const SHELL_B_BUTTON: Rect = Rect { x: 256, y: 172, w: 112, h: 32 };
const CLOSE_BUTTON: Rect = Rect { x: 420, y: 24, w: 16, h: 16 }; const CLOSE_BUTTON: Rect =
const VIEWPORT: Rect = Rect { x: 0, y: 0, w: 480, h: 270 }; Rect { x: prometeu_hal::graphics_contract::WIDTH as i32 - 60, y: 24, w: 16, h: 16 };
const HOME_PANEL: Rect = Rect { x: 48, y: 58, w: 384, h: 154 }; const VIEWPORT: Rect = Rect {
const TITLE_PANEL: Rect = Rect { x: 72, y: 34, w: 336, h: 28 }; x: 0,
y: 0,
w: prometeu_hal::graphics_contract::WIDTH as i32,
h: prometeu_hal::graphics_contract::HEIGHT as i32,
};
const HOME_PANEL: Rect = Rect {
x: 48,
y: 58,
w: prometeu_hal::graphics_contract::WIDTH as i32 - 96,
h: prometeu_hal::graphics_contract::HEIGHT as i32 - 116,
};
const TITLE_PANEL: Rect =
Rect { x: 72, y: 34, w: prometeu_hal::graphics_contract::WIDTH as i32 - 144, h: 28 };
const GAME_ROW_X: i32 = 72; const GAME_ROW_X: i32 = 72;
const GAME_ROW_Y: i32 = 90; const GAME_ROW_Y: i32 = 90;
const GAME_ROW_W: i32 = 336; const GAME_ROW_W: i32 = prometeu_hal::graphics_contract::WIDTH as i32 - 144;
const GAME_ROW_H: i32 = 22; const GAME_ROW_H: i32 = 22;
const GAME_ROW_GAP: i32 = 6; const GAME_ROW_GAP: i32 = 6;
const MAX_HOME_GAME_ROWS: usize = 3; const MAX_HOME_GAME_ROWS: usize = 3;
pub const HUB_SUSPENDED_GAME_KILL_CONTROL: Rect = Rect { x: 288, y: 230, w: 168, h: 24 }; pub const HUB_SUSPENDED_GAME_KILL_CONTROL: Rect = Rect {
x: prometeu_hal::graphics_contract::WIDTH as i32 - 192,
y: prometeu_hal::graphics_contract::HEIGHT as i32 - 40,
w: 168,
h: 24,
};
#[cfg(test)] #[cfg(test)]
const SHELL_FRAME: Rect = Rect { x: 40, y: 20, w: 400, h: 220 }; const SHELL_FRAME: Rect = Rect {
x: 40,
y: 20,
w: prometeu_hal::graphics_contract::WIDTH as i32 - 80,
h: prometeu_hal::graphics_contract::HEIGHT as i32 - 50,
};
const COLOR_BG: Color = Color::rgb(12, 14, 22); const COLOR_BG: Color = Color::rgb(12, 14, 22);
const COLOR_PANEL: Color = Color::rgb(25, 31, 48); const COLOR_PANEL: Color = Color::rgb(25, 31, 48);
@@ -2,6 +2,7 @@
//! shared IO worker owns storage during a publication. Legacy per-slot files are //! shared IO worker owns storage during a publication. Legacy per-slot files are
//! intentionally neither read nor migrated by this service. //! intentionally neither read nor migrated by this service.
pub mod admission; pub mod admission;
pub mod diagnostics;
pub mod guest; pub mod guest;
pub mod image; pub mod image;
pub mod session; pub mod session;
@@ -37,6 +37,7 @@ pub struct Admission {
pub state: PreparationState, pub state: PreparationState,
pub snapshot_budget: usize, pub snapshot_budget: usize,
faulted: bool, faulted: bool,
replacement_observation: Option<super::diagnostics::SessionDiagnostics>,
} }
impl Default for Admission { impl Default for Admission {
fn default() -> Self { fn default() -> Self {
@@ -48,10 +49,26 @@ impl Default for Admission {
state: PreparationState::Idle, state: PreparationState::Idle,
snapshot_budget: DEFAULT_SNAPSHOT_BUDGET, snapshot_budget: DEFAULT_SNAPSHOT_BUDGET,
faulted: false, faulted: false,
replacement_observation: None,
} }
} }
} }
impl MemcardService { impl MemcardService {
pub fn session_diagnostics(&self) -> super::diagnostics::SessionDiagnostics {
self.admission.replacement_observation.clone().unwrap_or_else(|| self.session.diagnostics())
}
pub fn preparation_active(&self) -> bool {
self.admission.receiver.is_some()
}
pub fn installed_card_id(&self) -> Option<[u8; 16]> {
self.admission.installed.as_ref().map(|c| c.current().card_id)
}
pub fn cancel_pending(&mut self) {
self.session.cancel_pending();
if let Some(control) = &self.admission.control {
control.cancel();
}
}
pub fn host_faulted(&self) -> bool { pub fn host_faulted(&self) -> bool {
self.admission.faulted || self.session.host_faulted() self.admission.faulted || self.session.host_faulted()
} }
@@ -60,7 +77,11 @@ impl MemcardService {
} }
pub fn installed_health(&self) -> Option<(bool, bool, bool)> { pub fn installed_health(&self) -> Option<(bool, bool, bool)> {
self.admission.installed.as_ref().map(|card| { self.admission.installed.as_ref().map(|card| {
(card.recovered, card.degraded, card.write_blocked || card.storage_read_only()) (
card.recovered,
card.degraded,
card.write_blocked || card.storage_read_only() || card.uncertain,
)
}) })
} }
pub fn card_has_game(&self, app_id: u32) -> bool { pub fn card_has_game(&self, app_id: u32) -> bool {
@@ -135,6 +156,9 @@ impl MemcardService {
let owned = let owned =
card.catalog.games.iter().find(|g| g.app_id == app_id).map_or(0, |g| g.slots.len()); card.catalog.games.iter().find(|g| g.app_id == app_id).map_or(0, |g| g.slots.len());
if choice == Choice::Persistent { if choice == Choice::Persistent {
if card.uncertain {
return Err(Error::Indeterminate);
}
if card.storage_read_only() || card.write_blocked { if card.storage_read_only() || card.write_blocked {
return Err(Error::ReadOnly); return Err(Error::ReadOnly);
} }
@@ -171,6 +195,10 @@ impl MemcardService {
} }
Choice::Persistent => { Choice::Persistent => {
let mut card = card.take().unwrap(); let mut card = card.take().unwrap();
if card.uncertain {
session.retain_idle_card(card);
return Err(Error::Indeterminate);
}
if card.storage_read_only() || card.write_blocked { if card.storage_read_only() || card.write_blocked {
session.retain_idle_card(card); session.retain_idle_card(card);
return Err(Error::ReadOnly); return Err(Error::ReadOnly);
@@ -256,9 +284,24 @@ impl MemcardService {
&mut self, &mut self,
storage: Box<dyn MemcardStorage>, storage: Box<dyn MemcardStorage>,
load_existing: bool, load_existing: bool,
) -> Result<()> {
self.replace_card_job(move |session, control| {
session.replace_storage(storage, load_existing, control)
})
}
pub fn reconcile_card_with(
&mut self,
open: impl FnOnce() -> Result<Box<dyn MemcardStorage>> + Send + 'static,
) -> Result<()> {
self.replace_card_job(move |session, control| session.reconcile_storage_with(open, control))
}
fn replace_card_job(
&mut self,
operation: impl FnOnce(&mut Session, &PublicationControl) -> Result<()> + Send + 'static,
) -> Result<()> { ) -> Result<()> {
if self.admission.receiver.is_some() if self.admission.receiver.is_some()
|| self.session.has_in_flight() || self.session.io_active()
|| self.session.has_deferred_write()
|| self.session.is_boot_active() || self.session.is_boot_active()
{ {
return Err(Error::Busy); return Err(Error::Busy);
@@ -266,6 +309,7 @@ impl MemcardService {
if self.admission.faulted { if self.admission.faulted {
return Err(Error::ReadOnly); return Err(Error::ReadOnly);
} }
self.admission.replacement_observation = Some(self.session.diagnostics());
let handoff = Arc::new(Mutex::new(Some(std::mem::take(&mut self.session)))); let handoff = Arc::new(Mutex::new(Some(std::mem::take(&mut self.session))));
let worker = handoff.clone(); let worker = handoff.clone();
let control = Arc::new(PublicationControl::default()); let control = Arc::new(PublicationControl::default());
@@ -275,13 +319,14 @@ impl MemcardService {
.lane .lane
.submit_memcard_commit_write(move |_| { .submit_memcard_commit_write(move |_| {
let mut session = worker.lock().unwrap().take().unwrap(); let mut session = worker.lock().unwrap().take().unwrap();
let result = session.replace_storage(storage, load_existing, &worker_control); let result = operation(&mut session, &worker_control);
let _ = tx.send(Prepared::Replacement(session, result)); let _ = tx.send(Prepared::Replacement(session, result));
crate::AsyncWorkJobOutcome::Completed crate::AsyncWorkJobOutcome::Completed
}) })
.is_err() .is_err()
{ {
self.session = handoff.lock().unwrap().take().unwrap(); self.session = handoff.lock().unwrap().take().unwrap();
self.admission.replacement_observation = None;
return Err(Error::InvalidState); return Err(Error::InvalidState);
} }
self.admission.control = Some(control); self.admission.control = Some(control);
@@ -331,6 +376,7 @@ impl MemcardService {
} }
self.admission.receiver = None; self.admission.receiver = None;
self.admission.control = None; self.admission.control = None;
self.admission.replacement_observation = None;
} }
Err(mpsc::TryRecvError::Disconnected) => { Err(mpsc::TryRecvError::Disconnected) => {
self.admission.state = PreparationState::Failed(Error::InvalidState); self.admission.state = PreparationState::Failed(Error::InvalidState);
@@ -361,3 +407,37 @@ impl MemcardService {
} }
} }
} }
#[cfg(all(test, unix))]
mod tests {
use super::*;
use prometeu_drivers::memcard::FileMemcard;
use prometeu_hal::memcard::MemcardIoError;
#[test]
fn idle_uncertain_media_remains_blocked_before_another_persistent_admission() {
let path = std::env::temp_dir().join(format!(
"memcard-uncertain-idle-{}-{}",
std::process::id(),
std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap().as_nanos()
));
let io = FileMemcard::create(&path, IMAGE_BYTES, |io, id| {
Card::format(io, id).map_err(|_| MemcardIoError::IoError)
})
.unwrap();
let mut card = Card::mount(Box::new(io)).unwrap();
card.uncertain = true;
let mut service = MemcardService::new();
// An ended session returns its backend to the same installed-card state.
service.admission.installed = Some(card);
assert_eq!(service.installed_health(), Some((false, false, true)));
assert_eq!(
service.choose_for_game(42, 1, Choice::Persistent, 0),
Err(Error::Indeterminate)
);
assert!(service.card_installed());
assert_eq!(service.lane.pending_depth(), 0);
service.uninstall_card().unwrap();
std::fs::remove_file(path).unwrap();
}
}
@@ -0,0 +1,31 @@
//! Read-only native observations. These do not consume the guest request entry.
use super::{
image::Error,
session::{Mode, RequestResult},
};
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RequestDiagnostics {
pub result: RequestResult,
pub storage_error: Option<Error>,
pub slot: Option<usize>,
pub operation: &'static str,
pub retained: bool,
pub io_active: bool,
pub deferred: bool,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SessionDiagnostics {
pub app_id: Option<u32>,
pub card_id: Option<[u8; 16]>,
pub epoch: u64,
pub mode: Option<Mode>,
pub health: (bool, bool, bool),
pub dirty_slots: Vec<usize>,
pub request: Option<RequestDiagnostics>,
pub io_active: bool,
pub removal_pending: bool,
pub reconciliation_required: bool,
pub restart_required: bool,
}
@@ -31,12 +31,14 @@ pub fn status(e: Error) -> i64 {
use prometeu_hal::memcard::MemcardIoError as I; use prometeu_hal::memcard::MemcardIoError as I;
match e { match e {
Error::NoSpace | Error::Io(I::NoSpace) => 3, Error::NoSpace | Error::Io(I::NoSpace) => 3,
Error::AccessDenied => 4, Error::AccessDenied
| Error::IdentityMismatch
| Error::Io(I::AccessDenied | I::AlreadyExists) => 4,
Error::Corrupt => 5, Error::Corrupt => 5,
Error::Indeterminate => 6, Error::Indeterminate | Error::Io(I::Indeterminate) => 6,
Error::Io(I::Unavailable) => 7, Error::Io(I::Unavailable) => 7,
Error::InvalidState | Error::Canceled => 8, Error::InvalidState | Error::Canceled => 8,
Error::Busy => 9, Error::Busy | Error::Io(I::Busy) => 9,
Error::ReadOnly | Error::CounterExhausted | Error::Io(I::ReadOnly) => 10, Error::ReadOnly | Error::CounterExhausted | Error::Io(I::ReadOnly) => 10,
Error::InvalidRange | Error::Io(I::InvalidRange) => 11, Error::InvalidRange | Error::Io(I::InvalidRange) => 11,
Error::Unsupported | Error::Io(I::Unsupported) => 12, Error::Unsupported | Error::Io(I::Unsupported) => 12,
@@ -29,6 +29,7 @@ pub enum Error {
OutOfMemory, OutOfMemory,
AccessDenied, AccessDenied,
InvalidState, InvalidState,
IdentityMismatch,
} }
impl From<MemcardIoError> for Error { impl From<MemcardIoError> for Error {
fn from(e: MemcardIoError) -> Self { fn from(e: MemcardIoError) -> Self {
@@ -7,6 +7,8 @@ use std::time::{Duration, Instant};
pub const GUEST_BYTES: usize = 4096; pub const GUEST_BYTES: usize = 4096;
static NEXT_CONTEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1); static NEXT_CONTEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);
// `try_update` requires Rust 1.95; keep older stable CI toolchains compatible.
#[allow(deprecated)]
fn next_context() -> Result<u64> { fn next_context() -> Result<u64> {
NEXT_CONTEXT NEXT_CONTEXT
.fetch_update( .fetch_update(
@@ -65,16 +67,20 @@ struct Request {
control: Arc<PublicationControl>, control: Arc<PublicationControl>,
receiver: mpsc::Receiver<Completion>, receiver: mpsc::Receiver<Completion>,
admitted: Instant, admitted: Instant,
watchdog_canceled_at: Option<Instant>,
worker_error: Option<Error>,
result: RequestResult, result: RequestResult,
job: AsyncWorkJobId, job: AsyncWorkJobId,
released: bool, released: bool,
saves_payload: bool, saves_payload: bool,
worker_returned: bool,
} }
/// Main-lane control; the Card moves into the single worker while its source is protected. /// Main-lane control; the Card moves into the single worker while its source is protected.
#[derive(Default)] #[derive(Default)]
pub struct Session { pub struct Session {
card_id: Option<[u8; 16]>, card_id: Option<[u8; 16]>,
uncertain_target: Option<Map>, uncertain_target: Option<Map>,
known_card_exclusions: Option<([u8; 16], [bool; BLOCK_COUNT])>,
health: (bool, bool, bool), health: (bool, bool, bool),
context: u64, context: u64,
epoch: u64, epoch: u64,
@@ -98,6 +104,82 @@ pub struct Session {
deferred: Option<(u64, usize, u64, Instant)>, deferred: Option<(u64, usize, u64, Instant)>,
} }
impl Session { impl Session {
/// IO ownership, independent of a completed result retained for guest release.
pub fn io_active(&self) -> bool {
self.request.as_ref().is_some_and(|r| !r.worker_returned)
}
pub fn has_deferred_write(&self) -> bool {
self.deferred.is_some()
}
pub fn cancel_pending(&mut self) {
if let Some(r) = &self.request {
r.control.cancel();
}
if let Some((id, local, ..)) = self.deferred.take() {
self.slots[local].protected = false;
self.memory_result = Some(RequestResult {
id,
state: RequestState::Failure,
reason: RequestReason::Storage(Error::Canceled),
});
}
}
pub fn diagnostics(&self) -> super::diagnostics::SessionDiagnostics {
use super::diagnostics::{RequestDiagnostics, SessionDiagnostics};
let request = if let Some((id, local, ..)) = self.deferred {
Some(RequestDiagnostics {
result: RequestResult {
id,
state: RequestState::Pending,
reason: RequestReason::None,
},
storage_error: None,
slot: Some(local),
operation: "commit",
retained: true,
io_active: false,
deferred: true,
})
} else if let Some(r) = &self.request {
Some(RequestDiagnostics {
result: r.result,
storage_error: r.worker_error,
slot: r.saves_payload.then_some(r.local),
operation: if r.saves_payload { "commit" } else { "presentation" },
retained: !r.released,
io_active: self.io_active(),
deferred: false,
})
} else {
self.memory_result.map(|result| RequestDiagnostics {
result,
storage_error: None,
slot: None,
operation: "commit",
retained: true,
io_active: false,
deferred: false,
})
};
SessionDiagnostics {
app_id: self.mode.map(|_| self.app_id),
card_id: self.card_id,
epoch: self.epoch,
mode: self.mode,
health: self.health(),
dirty_slots: self
.slots
.iter()
.enumerate()
.filter_map(|(i, s)| s.data.read().ok().filter(|s| s.dirty).map(|_| i))
.collect(),
request,
io_active: self.io_active(),
removal_pending: self.removal_pending,
reconciliation_required: self.reconcile_blocked,
restart_required: self.faulted,
}
}
pub fn context(&self) -> u64 { pub fn context(&self) -> u64 {
self.context self.context
} }
@@ -523,10 +605,13 @@ impl Session {
control, control,
receiver, receiver,
admitted, admitted,
watchdog_canceled_at: None,
worker_error: None,
result: RequestResult { id, state: RequestState::Pending, reason: RequestReason::None }, result: RequestResult { id, state: RequestState::Pending, reason: RequestReason::None },
job, job,
released: !saves_payload, released: !saves_payload,
saves_payload, saves_payload,
worker_returned: false,
}); });
Ok(()) Ok(())
} }
@@ -664,10 +749,13 @@ impl Session {
control, control,
receiver, receiver,
admitted: now, admitted: now,
watchdog_canceled_at: None,
worker_error: None,
result: RequestResult { id, state: RequestState::Pending, reason: RequestReason::None }, result: RequestResult { id, state: RequestState::Pending, reason: RequestReason::None },
job, job,
released: false, released: false,
saves_payload: true, saves_payload: true,
worker_returned: false,
}); });
Ok(id) Ok(id)
} }
@@ -686,27 +774,30 @@ impl Session {
let Some(r) = &mut self.request else { return }; let Some(r) = &mut self.request else { return };
match r.receiver.try_recv() { match r.receiver.try_recv() {
Ok(completion) => { Ok(completion) => {
r.worker_returned = true;
r.worker_error = completion.result.error;
let current = let current =
self.mode.is_some() && r.session == self.context && r.epoch == self.epoch; self.mode.is_some() && r.session == self.context && r.epoch == self.epoch;
if current { if current {
self.slots[r.local].protected = false; self.slots[r.local].protected = false;
if r.result.reason == RequestReason::None { if r.result.state == RequestState::Pending {
r.result.state = match completion.result.durable { r.result.state = match completion.result.durable {
DurableResult::Success => RequestState::Success, DurableResult::Success => RequestState::Success,
DurableResult::Failure => RequestState::Failure, DurableResult::Failure => RequestState::Failure,
DurableResult::Indeterminate => RequestState::Indeterminate, DurableResult::Indeterminate => RequestState::Indeterminate,
}; };
r.result.reason = completion r.result.reason = if r.watchdog_canceled_at.is_some() {
.result RequestReason::Timeout
.error } else {
.map_or(RequestReason::None, RequestReason::Storage); r.worker_error.map_or(RequestReason::None, RequestReason::Storage)
};
if completion.result.durable == DurableResult::Success && r.saves_payload { if completion.result.durable == DurableResult::Success && r.saves_payload {
let mut s = r.source.write().unwrap(); let mut s = r.source.write().unwrap();
s.descriptor = s.descriptor =
completion.card.current().slots[self.logical[r.local] as usize]; completion.card.current().slots[self.logical[r.local] as usize];
s.dirty = false; s.dirty = false;
} }
} // A timeout result is immutable; returned card requires explicit reconciliation. } // A restart escalation is immutable; returned card requires explicit reconciliation.
if self.provisional && completion.result.durable != DurableResult::Success { if self.provisional && completion.result.durable != DurableResult::Success {
self.boot_error = self.boot_error =
Some(completion.result.error.unwrap_or(Error::InvalidState)); Some(completion.result.error.unwrap_or(Error::InvalidState));
@@ -755,7 +846,10 @@ impl Session {
return; return;
} }
let elapsed = now.saturating_duration_since(r.admitted); let elapsed = now.saturating_duration_since(r.admitted);
if elapsed >= Duration::from_millis(500) { if elapsed >= Duration::from_millis(500) && !r.control.canceled() {
// Main-lane cancellation cause is independent of the worker's durable
// outcome. Do not relabel an earlier removal or termination cancel.
r.watchdog_canceled_at = Some(now);
r.control.cancel(); r.control.cancel();
} }
if elapsed >= Duration::from_secs(1) { if elapsed >= Duration::from_secs(1) {
@@ -788,6 +882,12 @@ impl Session {
.map(|r| r.result) .map(|r| r.result)
.ok_or(Error::InvalidRange) .ok_or(Error::InvalidRange)
} }
/// Original worker error for native diagnostics, independent of guest deadline reason.
/// Pending and RAM-only requests have no worker error. Retention follows query/release.
pub fn request_storage_error(&self, context: u64, id: u64) -> Result<Option<Error>> {
self.query(context, id)?;
Ok(self.request.as_ref().filter(|r| r.id == id).and_then(|r| r.worker_error))
}
pub fn release_request(&mut self, context: u64, id: u64) -> Result<()> { pub fn release_request(&mut self, context: u64, id: u64) -> Result<()> {
self.query(context, id)?; self.query(context, id)?;
if self.deferred.is_some() { if self.deferred.is_some() {
@@ -802,7 +902,7 @@ impl Session {
return Err(Error::Busy); return Err(Error::Busy);
} }
r.released = true; r.released = true;
if self.card.is_some() { if r.worker_returned {
self.request = None; self.request = None;
} }
Ok(()) Ok(())
@@ -816,7 +916,7 @@ impl Session {
r.control.cancel(); r.control.cancel();
r.released = true; r.released = true;
} }
if self.card.is_some() { if !self.io_active() {
self.request = None; self.request = None;
} }
self.removal_pending = false; self.removal_pending = false;
@@ -852,7 +952,23 @@ impl Session {
load_existing: bool, load_existing: bool,
control: &PublicationControl, control: &PublicationControl,
) -> Result<()> { ) -> Result<()> {
if self.has_in_flight() || self.boot_active || self.mode.is_none() { self.replace_storage_with(move || Ok(storage), load_existing, false, control)
}
pub(super) fn reconcile_storage_with(
&mut self,
open: impl FnOnce() -> Result<Box<dyn prometeu_hal::memcard::MemcardStorage>>,
control: &PublicationControl,
) -> Result<()> {
self.replace_storage_with(open, false, true, control)
}
fn replace_storage_with(
&mut self,
open: impl FnOnce() -> Result<Box<dyn prometeu_hal::memcard::MemcardStorage>>,
load_existing: bool,
same_only: bool,
control: &PublicationControl,
) -> Result<()> {
if self.io_active() || self.deferred.is_some() || self.boot_active || self.mode.is_none() {
return Err(Error::Busy); return Err(Error::Busy);
} }
if self.icon.is_empty() if self.icon.is_empty()
@@ -865,11 +981,18 @@ impl Session {
{ {
self.uncertain_target = card.attempted_publication.clone(); self.uncertain_target = card.attempted_publication.clone();
} }
let known = self.card.as_ref().map(|c| (c.current().card_id, c.known_exclusions())); if let Some(card) = &self.card {
self.known_card_exclusions = Some((card.current().card_id, card.known_exclusions()));
}
let known = self.known_card_exclusions;
self.card.take(); // Release the old scratch before mounting another image. self.card.take(); // Release the old scratch before mounting another image.
self.scratch = None; self.scratch = None;
self.reconcile_blocked = true; self.reconcile_blocked = true;
self.card = Some(Card::mount(storage)?); let candidate = Card::mount(open()?)?;
if same_only && self.card_id != Some(candidate.current().card_id) {
return Err(Error::IdentityMismatch);
}
self.card = Some(candidate);
let card = self.card.as_mut().unwrap(); let card = self.card.as_mut().unwrap();
let same = self.card_id == Some(card.current().card_id); let same = self.card_id == Some(card.current().card_id);
if let Some((id, known)) = known if let Some((id, known)) = known
@@ -956,14 +1079,15 @@ impl Session {
Ok(()) Ok(())
} }
pub fn detach(&mut self) -> Result<()> { pub fn detach(&mut self) -> Result<()> {
if self.deferred.is_some() if self.deferred.is_some() || self.io_active() {
|| self
.request
.as_ref()
.is_some_and(|r| r.result.state == RequestState::Pending || self.card.is_none())
{
return Err(Error::Busy); return Err(Error::Busy);
} }
if self.card.is_none() && self.scratch.is_none() {
let mut scratch = Vec::new();
scratch.try_reserve_exact(SLOT_BYTES).map_err(|_| Error::OutOfMemory)?;
scratch.resize(SLOT_BYTES, 0);
self.scratch = Some(scratch);
}
self.mode = Some(Mode::MemoryOnly); self.mode = Some(Mode::MemoryOnly);
self.removal_pending = false; self.removal_pending = false;
self.reconcile_blocked = false; self.reconcile_blocked = false;
@@ -16,6 +16,8 @@ impl PublicationControl {
pub fn publication_attempted(&self) -> bool { pub fn publication_attempted(&self) -> bool {
self.0.load(Ordering::SeqCst) & 1 != 0 self.0.load(Ordering::SeqCst) & 1 != 0
} }
// `try_update` requires Rust 1.95; keep older stable CI toolchains compatible.
#[allow(deprecated)]
fn enter(&self) -> Result<()> { fn enter(&self) -> Result<()> {
self.0 self.0
.fetch_update(Ordering::SeqCst, Ordering::SeqCst, |state| { .fetch_update(Ordering::SeqCst, Ordering::SeqCst, |state| {
@@ -280,8 +280,157 @@ fn async_commit_protects_only_source_and_retains_result_without_occupying_lane()
controller.stop().unwrap(); controller.stop().unwrap();
} }
// The sentinel executes after the save on the single lane. Wall time only bounds
// a broken fixture; all watchdog observations use explicit simulated instants.
fn finish_queued_save(lane: Arc<crate::AsyncWorkLane>) {
let (tx, rx) = std::sync::mpsc::channel();
lane.submit_asset_load(move |_| {
tx.send(()).unwrap();
crate::services::async_work::AsyncWorkJobOutcome::Completed
})
.unwrap();
let mut controller =
crate::AsyncWorkLaneController::start(crate::AsyncWorkLaneConfig::default(), lane);
rx.recv_timeout(std::time::Duration::from_secs(5)).unwrap();
controller.stop().unwrap();
}
fn assert_guest_query(
session: &mut super::super::session::Session,
id: u64,
state: i64,
reason: i64,
) {
use prometeu_bytecode::Value;
use prometeu_hal::{
HostReturn, host_return::GuestStringAllocator, syscalls::Syscall, vm_fault::VmFault,
};
struct NoStrings;
impl GuestStringAllocator for NoStrings {
fn byte_limit(&self) -> u64 {
0
}
fn build(
&mut self,
_: usize,
_: &mut dyn FnMut(&mut dyn std::fmt::Write) -> std::fmt::Result,
) -> std::result::Result<Arc<str>, VmFault> {
panic!("MemQuery must not allocate strings")
}
}
let mut service = super::super::MemcardService::new();
std::mem::swap(session, &mut service.session);
let args = [Value::Int64(service.session.context() as i64), Value::Int64(id as i64)];
let mut values = Vec::with_capacity(Syscall::MemQuery.results_count());
let mut strings = NoStrings;
let mut ret = HostReturn::new(&mut values, Syscall::MemQuery.results_count(), &mut strings);
service.guest(Syscall::MemQuery, &args, &mut ret, 4, 0, true).unwrap();
assert!(!ret.overflowed());
assert!(ret.failure().is_none());
assert_eq!(values, vec![Value::Int64(0), Value::Int64(state), Value::Int64(reason)]);
std::mem::swap(session, &mut service.session);
}
#[test] #[test]
fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() { fn queued_memcard_watchdog_preserves_timeout_and_guest_reason() {
use super::super::session::{RequestReason, RequestState, Session};
use std::time::{Duration, Instant};
for bootstrap in [false, true] {
let (mut card, disk) = formatted();
add(&mut card, 4, 1);
let before = card.current().slots[0];
let mut s = Session::default();
let c = s.load_card(card, 4, 1, Session::required_bytes(1).unwrap()).unwrap();
s.write(c, 0, 0, b"pending").unwrap();
let lane = Arc::new(crate::AsyncWorkLane::default());
let now = Instant::now();
if bootstrap {
s.begin_boot();
}
let id = s.commit(c, 0, 7, now, &lane).unwrap();
if bootstrap {
s.finish_boot(true, &lane, now).unwrap();
}
s.poll(now + Duration::from_millis(499));
assert_guest_query(&mut s, id, 0, 0);
for _ in 0..2 {
s.poll(now + Duration::from_millis(500));
}
assert_guest_query(&mut s, id, 0, 0);
assert_eq!(s.release_request(c, id), Err(Error::Busy));
assert_eq!(s.clear(c, 0), Err(Error::Busy));
finish_queued_save(lane);
s.poll(now + Duration::from_millis(501));
let result = s.query(c, id).unwrap();
assert_eq!(result.state, RequestState::Failure);
assert_eq!(result.reason, RequestReason::Timeout);
assert_eq!(s.request_storage_error(c, id), Ok(Some(Error::Canceled)));
assert_guest_query(&mut s, id, 2, 15);
assert!(s.unsaved());
assert_eq!(reboot(&disk).current().slots[0], before);
assert_eq!(s.query(c + 1, id), Err(Error::AccessDenied));
s.release_request(c, id).unwrap();
assert_eq!(s.query(c, id), Err(Error::InvalidRange));
s.clear(c, 0).unwrap();
}
}
#[test]
fn lifecycle_cancellation_is_not_a_watchdog_timeout() {
use super::super::session::{RequestReason, RequestState, Session};
use std::time::{Duration, Instant};
for end in [false, true] {
let (mut card, _) = formatted();
add(&mut card, 4, 1);
let mut s = Session::default();
let c = s.load_card(card, 4, 1, Session::required_bytes(1).unwrap()).unwrap();
let lane = Arc::new(crate::AsyncWorkLane::default());
let now = Instant::now();
let id = s.commit(c, 0, 7, now, &lane).unwrap();
if end {
s.end_session();
} else {
s.notify_removal();
}
s.poll(now + Duration::from_millis(500));
finish_queued_save(lane);
s.poll(now + Duration::from_millis(501));
if end {
assert_eq!(s.query(c, id), Err(Error::AccessDenied));
let card = s.take_idle_card().unwrap().unwrap();
let new = s.load_card(card, 4, 1, Session::required_bytes(1).unwrap()).unwrap();
assert_ne!(new, c);
assert_eq!(s.query(new, id), Err(Error::InvalidRange));
} else {
assert_eq!(s.query(c, id).unwrap().state, RequestState::Failure);
assert_eq!(s.query(c, id).unwrap().reason, RequestReason::Storage(Error::Canceled));
assert_guest_query(&mut s, id, 2, 8);
}
}
}
#[test]
fn deferred_boot_memcard_deadline_stays_timeout() {
use super::super::session::{RequestReason, Session};
use std::time::{Duration, Instant};
let (mut card, _) = formatted();
add(&mut card, 4, 1);
let mut s = Session::default();
let c = s.load_card(card, 4, 1, Session::required_bytes(1).unwrap()).unwrap();
s.begin_boot();
let lane = crate::AsyncWorkLane::default();
let now = Instant::now();
let id = s.commit(c, 0, 7, now, &lane).unwrap();
s.poll(now + Duration::from_millis(500));
assert_eq!(s.query(c, id).unwrap().reason, RequestReason::Timeout);
assert_guest_query(&mut s, id, 2, 15);
assert_eq!(lane.pending_depth(), 0);
s.clear(c, 0).unwrap();
s.release_request(c, id).unwrap();
}
#[test]
fn blocked_host_memcard_watchdog_preserves_outcome_and_escalation() {
use crate::services::async_work::{ use crate::services::async_work::{
AsyncWorkLane, AsyncWorkLaneConfig, AsyncWorkLaneController, AsyncWorkLane, AsyncWorkLaneConfig, AsyncWorkLaneController,
}; };
@@ -290,6 +439,7 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
struct Blocked { struct Blocked {
store: Box<dyn MemcardStorage>, store: Box<dyn MemcardStorage>,
publication: bool, publication: bool,
fail_publication: bool,
entered: std::sync::mpsc::Sender<()>, entered: std::sync::mpsc::Sender<()>,
resume: std::sync::mpsc::Receiver<()>, resume: std::sync::mpsc::Receiver<()>,
} }
@@ -310,12 +460,24 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
if if self.publication { b.starts_with(b"PMMAP1\0\0") } else { o >= DATA_OFFSET } { if if self.publication { b.starts_with(b"PMMAP1\0\0") } else { o >= DATA_OFFSET } {
self.entered.send(()).unwrap(); self.entered.send(()).unwrap();
self.resume.recv().unwrap(); self.resume.recv().unwrap();
if self.fail_publication {
return Err(MemcardIoError::IoError);
}
} }
self.store.write_at(o, b) self.store.write_at(o, b)
} }
} }
for publication in [false, true] { for (publication, fail_publication, escalate, release_early) in [
let (mut card, _) = formatted(); (false, false, false, false),
(true, false, false, false),
(true, true, false, false),
(false, false, true, false),
(true, false, true, false),
(true, true, true, false),
(false, false, true, true),
(true, false, true, true),
] {
let (mut card, disk) = formatted();
add(&mut card, 4, 1); add(&mut card, 4, 1);
let (et, er) = std::sync::mpsc::channel(); let (et, er) = std::sync::mpsc::channel();
let (rt, rr) = std::sync::mpsc::channel(); let (rt, rr) = std::sync::mpsc::channel();
@@ -324,6 +486,7 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
entered: et, entered: et,
resume: rr, resume: rr,
publication, publication,
fail_publication,
})) }))
.unwrap(); .unwrap();
let mut s = Session::default(); let mut s = Session::default();
@@ -336,6 +499,8 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
let id = s.commit(c, 0, 7, admitted, &lane).unwrap(); let id = s.commit(c, 0, 7, admitted, &lane).unwrap();
er.recv_timeout(Duration::from_secs(1)).unwrap(); er.recv_timeout(Duration::from_secs(1)).unwrap();
s.poll(admitted + Duration::from_millis(500)); s.poll(admitted + Duration::from_millis(500));
assert_guest_query(&mut s, id, 0, 0);
let terminal = if escalate {
s.poll(admitted + Duration::from_secs(1)); s.poll(admitted + Duration::from_secs(1));
let result = s.query(c, id).unwrap(); let result = s.query(c, id).unwrap();
assert_eq!( assert_eq!(
@@ -344,14 +509,74 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
); );
assert_eq!(result.reason, RequestReason::RestartRequired); assert_eq!(result.reason, RequestReason::RestartRequired);
assert!(s.host_faulted()); assert!(s.host_faulted());
if release_early {
s.release_request(c, id).unwrap(); s.release_request(c, id).unwrap();
}
Some(result)
} else {
None
};
assert_eq!(s.write(c, 0, 0, b"unsafe"), Err(Error::Busy)); assert_eq!(s.write(c, 0, 0, b"unsafe"), Err(Error::Busy));
assert_eq!(s.commit(c, 0, 7, Instant::now(), &lane), Err(Error::Busy)); assert_eq!(s.commit(c, 0, 7, admitted, &lane), Err(Error::Busy));
rt.send(()).unwrap(); rt.send(()).unwrap();
controller.stop().unwrap(); controller.stop().unwrap();
s.poll(Instant::now()); s.poll(admitted + Duration::from_millis(if escalate { 1001 } else { 501 }));
if let Some(terminal) = terminal {
if release_early {
assert_eq!(s.query(c, id), Err(Error::InvalidRange));
} else {
assert_eq!(s.query(c, id), Ok(terminal));
assert_guest_query(&mut s, id, if publication { 3 } else { 2 }, 15);
s.release_request(c, id).unwrap();
}
assert!(s.unsaved()); assert!(s.unsaved());
assert!(s.host_faulted()); assert!(s.host_faulted());
assert_eq!(s.commit(c, 0, 7, admitted, &lane), Err(Error::ReadOnly));
} else {
let expected = if fail_publication {
RequestState::Indeterminate
} else if publication {
RequestState::Success
} else {
RequestState::Failure
};
assert_eq!(s.query(c, id).unwrap().state, expected);
assert_eq!(s.query(c, id).unwrap().reason, RequestReason::Timeout);
assert_guest_query(
&mut s,
id,
if fail_publication {
3
} else if publication {
1
} else {
2
},
15,
);
assert_eq!(
s.request_storage_error(c, id),
Ok(if fail_publication {
Some(Error::Io(MemcardIoError::IoError))
} else if publication {
None
} else {
Some(Error::Canceled)
})
);
assert_eq!(s.unsaved(), expected != RequestState::Success);
assert_eq!(s.removal_pending(), fail_publication);
assert!(!s.host_faulted());
s.release_request(c, id).unwrap();
}
let mut durable = reboot(&disk);
let mut bytes = [0; 7];
let n = durable.read_slot(0, &mut bytes).unwrap();
if publication && !fail_publication {
assert_eq!(&bytes[..n], b"pending");
} else {
assert_eq!(n, 0);
}
} }
} }
@@ -660,3 +885,267 @@ fn ending_a_session_releases_a_completed_unconsumed_request() {
assert!(s.take_idle_card().unwrap().is_some()); assert!(s.take_idle_card().unwrap().is_some());
controller.stop().unwrap(); controller.stop().unwrap();
} }
#[test]
fn persistent_save_preempts_real_multichunk_asset_decoders() {
use crate::memcard::session::{RequestState, Session};
use crate::{AsyncWorkLane, AsyncWorkLaneConfig, AsyncWorkLaneController};
use prometeu_drivers::{
AssetManager, GlyphBankPoolAccess, MemoryBanks, SceneBankPoolAccess, SoundBankPoolAccess,
};
use prometeu_hal::{
AssetBridge,
asset::{AssetCodec, AssetEntry, AssetOpStatus, BankType, LoadStatus},
background_work::{AssetWork, BackgroundWork, WorkUnavailable},
cartridge::AssetsPayloadSource,
};
use std::{
sync::mpsc,
thread,
time::{Duration, Instant},
};
type Events = Arc<Mutex<Vec<(&'static str, thread::ThreadId)>>>;
struct ObservedLane {
lane: Arc<AsyncWorkLane>,
events: Events,
paused: mpsc::Sender<()>,
resume: Mutex<Option<mpsc::Receiver<()>>>,
}
impl BackgroundWork for ObservedLane {
fn submit_asset(&self, mut work: AssetWork) -> std::result::Result<(), WorkUnavailable> {
let events = self.events.clone();
let paused = self.paused.clone();
let resume = self.resume.lock().unwrap().take().unwrap();
let mut step = 0;
self.lane.submit_asset(Box::new(move |canceled| {
let result = work(canceled);
step += 1;
events.lock().unwrap().push(("asset", thread::current().id()));
if step == 4 {
paused.send(()).unwrap();
resume.recv_timeout(Duration::from_secs(10)).unwrap();
}
result
}))
}
}
struct ObservedStore {
inner: Store,
events: Events,
}
impl MemcardStorage for ObservedStore {
fn len(&self) -> std::result::Result<u64, MemcardIoError> {
self.inner.len()
}
fn read_only(&self) -> bool {
false
}
fn read_at(
&mut self,
offset: u64,
out: &mut [u8],
) -> std::result::Result<(), MemcardIoError> {
self.inner.read_at(offset, out)
}
fn write_at(
&mut self,
offset: u64,
bytes: &[u8],
) -> std::result::Result<(), MemcardIoError> {
self.events.lock().unwrap().push(("save write", thread::current().id()));
self.inner.write_at(offset, bytes)
}
fn sync(&mut self) -> std::result::Result<(), MemcardIoError> {
self.events.lock().unwrap().push(("save sync", thread::current().id()));
self.inner.sync()
}
}
for kind in [BankType::GLYPH, BankType::SOUNDS, BankType::MUSIC, BankType::SCENE] {
let (card, disk) = formatted();
drop(card);
let events: Events = Arc::default();
let mut card = Card::mount(Box::new(ObservedStore {
inner: Store(disk.clone()),
events: events.clone(),
}))
.unwrap();
add(&mut card, 1, 1);
let mut session = Session::default();
let context = session.load_card(card, 1, 1, 1024 * 1024).unwrap();
session.write(context, 0, 0, b"persistent under asset pressure").unwrap();
events.lock().unwrap().clear();
let (bytes, decoded_size, metadata) = match kind {
BankType::GLYPH => {
let mut bytes = vec![0x12; 16384];
bytes.extend([255; 64]);
(
bytes,
32768 + 64,
serde_json::json!({"tile_size":8,"width":256,"height":128,"palette_count":1,"palette_authored":1}),
)
}
BankType::SOUNDS => {
let mut bytes = vec![0u8; 88];
bytes[..4].copy_from_slice(b"SBNK");
bytes[4] = 1;
bytes[49] = 1;
bytes[80] = 255;
bytes[81] = 128;
for (at, value) in [
(8, 1u32),
(20, 960),
(24, 500000),
(32, u32::MAX),
(52, 44100),
(56, 8192),
(68, 88),
(72, 16384),
(84, 65536),
] {
bytes[at..at + 4].copy_from_slice(&value.to_le_bytes());
}
bytes.extend([0x12; 16384]);
(bytes, 16472, serde_json::json!({"version":1}))
}
BankType::MUSIC => (
include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../test-fixtures/audio/mixed-music.mbnk"
))
.to_vec(),
6395,
serde_json::json!({"version":1}),
),
BankType::SCENE => {
let mut bytes = vec![0; 140];
bytes[..8].copy_from_slice(&[b'S', b'C', b'N', b'E', 1, 0, 4, 0]);
for start in [12, 44, 76, 108] {
bytes[start + 5] = 8;
}
bytes[12] = 1;
bytes[28..32].copy_from_slice(&4097_u32.to_le_bytes());
bytes[32..36].copy_from_slice(&1_u32.to_le_bytes());
bytes[36..40].copy_from_slice(&4097_u32.to_le_bytes());
bytes.splice(44..44, std::iter::repeat_n([7, 3, 0x34, 0x12], 4097).flatten());
(
bytes,
64 + 4097 * size_of::<prometeu_hal::tile::Tile>() as u64,
serde_json::json!({}),
)
}
};
let entry = AssetEntry {
asset_id: 1,
asset_name: "pressure".into(),
bank_type: kind,
offset: 0,
size: bytes.len() as u64,
decoded_size,
codec: AssetCodec::None,
metadata,
};
let banks = Arc::new(MemoryBanks::new());
let mut manager = AssetManager::new(
vec![entry],
AssetsPayloadSource::from_bytes(bytes),
banks.clone(),
banks.clone(),
banks.clone(),
);
let lane = Arc::new(AsyncWorkLane::default());
let (paused_tx, paused_rx) = mpsc::channel();
let (resume_tx, resume_rx) = mpsc::channel();
let observed = Arc::new(ObservedLane {
lane: lane.clone(),
events: events.clone(),
paused: paused_tx,
resume: Mutex::new(Some(resume_rx)),
});
assert!(manager.bind_background_work(observed));
let mut controller =
AsyncWorkLaneController::start(AsyncWorkLaneConfig::default(), lane.clone());
let handle = manager.load(1, 0).unwrap();
paused_rx.recv_timeout(Duration::from_secs(10)).unwrap();
assert_eq!(manager.status(handle), LoadStatus::LOADING);
assert!(
banks.glyph_bank_slot(0).is_none()
&& banks.sound_bank_slot(0).is_none()
&& banks.scene_bank_slot(0).is_none()
);
let request = session.commit(context, 0, 7, Instant::now(), &lane).unwrap();
resume_tx.send(()).unwrap();
let deadline = Instant::now() + Duration::from_secs(10);
loop {
session.poll(Instant::now());
if session.query(context, request).unwrap().state != RequestState::Pending
&& manager.status(handle) == LoadStatus::READY
{
break;
}
assert!(Instant::now() < deadline, "save/asset did not complete for {kind:?}");
thread::yield_now();
}
assert_eq!(session.query(context, request).unwrap().state, RequestState::Success);
assert!(
banks.glyph_bank_slot(0).is_none()
&& banks.sound_bank_slot(0).is_none()
&& banks.scene_bank_slot(0).is_none()
);
assert_eq!(manager.commit(handle), AssetOpStatus::Ok);
manager.apply_commits();
match kind {
BankType::GLYPH => {
let bank = banks.glyph_bank_slot(0).unwrap();
assert_eq!((bank.width, bank.height), (256, 128));
assert_eq!(
bank.pixel_indices,
(0..32768).map(|i| 1 + (i % 2) as u8).collect::<Vec<_>>()
);
assert!(bank.palettes[0].iter().all(|c| c.raw() == 0xffffffff));
}
BankType::SOUNDS => {
let bank = banks.sound_bank_slot(0).unwrap();
assert_eq!(bank.samples[0].sample_rate, 44100);
assert_eq!(bank.samples[0].data, vec![0x1212; 8192]);
}
BankType::MUSIC => {
let bank = banks.music_bank_slot(0).unwrap();
assert_eq!(bank.events.len(), 4);
assert_eq!(bank.samples.len(), 2);
assert_eq!(bank.samples[1].channels, 2);
}
BankType::SCENE => {
let bank = banks.scene_bank_slot(0).unwrap();
assert_eq!(
(bank.layers[0].tilemap.width, bank.layers[0].tilemap.height),
(4097, 1)
);
assert!(bank.layers[0].active);
assert_eq!(bank.layers[0].tilemap.tiles.len(), 4097);
assert!(bank.layers[0].tilemap.tiles.iter().all(|t| t.active
&& t.flip_x
&& t.flip_y
&& t.glyph.glyph_id == 0x1234
&& t.glyph.palette_id == 3));
assert!(bank.layers[1..].iter().all(|l| l.tilemap.tiles.is_empty()));
}
}
controller.stop().unwrap();
let observed = events.lock().unwrap();
assert_eq!(observed[..4].iter().map(|e| e.0).collect::<Vec<_>>(), vec!["asset"; 4]);
assert_eq!(observed[4].0, "save write");
let resumed = observed[4..].iter().position(|e| e.0 == "asset").unwrap() + 4;
assert_eq!(observed[resumed - 1].0, "save sync");
assert!(observed[resumed..].iter().all(|e| e.0 == "asset"));
let worker = observed[0].1;
assert_ne!(worker, thread::current().id());
assert!(observed.iter().all(|e| e.1 == worker));
drop(observed);
drop(session);
let mut recovered = Session::default();
let c = recovered.load_card(reboot(&disk), 1, 1, 1024 * 1024).unwrap();
let mut saved = [0; 64];
let n = recovered.read(c, 0, 0, &mut saved).unwrap();
assert_eq!(&saved[..n], b"persistent under asset pressure");
}
}
@@ -4,6 +4,7 @@ use crate::services::memcard::MemcardService;
use prometeu_bytecode::{TRAP_INVALID_SYSCALL, TRAP_OOB, TRAP_TYPE, Value}; use prometeu_bytecode::{TRAP_INVALID_SYSCALL, TRAP_OOB, TRAP_TYPE, Value};
use prometeu_hal::app_mode::AppMode; use prometeu_hal::app_mode::AppMode;
use prometeu_hal::asset::{AssetId, AssetOpStatus, BankType}; use prometeu_hal::asset::{AssetId, AssetOpStatus, BankType};
use prometeu_hal::audio_bridge::AudioControl;
use prometeu_hal::color::Color; use prometeu_hal::color::Color;
use prometeu_hal::glyph::Glyph; use prometeu_hal::glyph::Glyph;
use prometeu_hal::log::{LogLevel, LogSource}; use prometeu_hal::log::{LogLevel, LogSource};
@@ -465,7 +466,7 @@ impl NativeInterface for VmRuntimeHost<'_> {
let pan_raw = expect_int(args, 3)?; let pan_raw = expect_int(args, 3)?;
let pitch = expect_number(args, 4, "pitch")?; let pitch = expect_number(args, 4, "pitch")?;
if !(0..16).contains(&voice_id_raw) { if !(0..8).contains(&voice_id_raw) {
ret.push_int(AudioOpStatus::VoiceInvalid as i64); ret.push_int(AudioOpStatus::VoiceInvalid as i64);
return Ok(()); return Ok(());
} }
@@ -508,7 +509,7 @@ impl NativeInterface for VmRuntimeHost<'_> {
prometeu_hal::LoopMode::On prometeu_hal::LoopMode::On
}; };
if !(0..16).contains(&voice_id_raw) { if !(0..8).contains(&voice_id_raw) {
ret.push_int(AudioOpStatus::VoiceInvalid as i64); ret.push_int(AudioOpStatus::VoiceInvalid as i64);
return Ok(()); return Ok(());
} }
@@ -558,7 +559,7 @@ impl NativeInterface for VmRuntimeHost<'_> {
} else { } else {
1.0 1.0
}; };
let status = if !(0..16).contains(&voice) { let status = if !(0..8).contains(&voice) {
AudioOpStatus::VoiceInvalid AudioOpStatus::VoiceInvalid
} else if !(0..=255).contains(&value) || !pitch.is_finite() || pitch <= 0.0 { } else if !(0..=255).contains(&value) || !pitch.is_finite() || pitch <= 0.0 {
AudioOpStatus::ArgRangeInvalid AudioOpStatus::ArgRangeInvalid
@@ -575,6 +576,113 @@ impl NativeInterface for VmRuntimeHost<'_> {
ret.push_int(status as i64); ret.push_int(status as i64);
Ok(()) Ok(())
} }
Syscall::MusicPlay
| Syscall::MusicStop
| Syscall::MusicPause
| Syscall::MusicResume
| Syscall::MusicSetVolume
| Syscall::MusicFadeTo
| Syscall::MusicCrossfade
| Syscall::AudioSetBusVolume
| Syscall::AudioSetEffect
| Syscall::AudioSetSend
| Syscall::AudioUnloadBank
| Syscall::AudioSetGroupVolume => {
macro_rules! uint {
($i:expr, $max:expr) => {{
let n = expect_int(args, $i)?;
if !(0..=$max).contains(&n) {
ret.push_int(AudioOpStatus::ArgRangeInvalid as i64);
return Ok(());
}
n as u32
}};
}
let status = match syscall {
Syscall::MusicPlay => {
let group = uint!(0, 1);
let bank = uint!(1, 15);
let looping = uint!(2, 1);
platform.audio_mut().music_play(group as usize, bank as usize, looping != 0)
}
Syscall::AudioUnloadBank => {
let music = uint!(0, 1);
let slot = uint!(1, 15);
let result = platform.assets().unload_audio(music != 0, slot as usize);
ret.push_int(if result == AssetOpStatus::Ok {
AudioOpStatus::Ok
} else {
AudioOpStatus::BankInvalid
} as i64);
return Ok(());
}
_ => {
let control = match syscall {
Syscall::MusicStop => {
AudioControl::MusicStop { group: uint!(0, 1) as usize }
}
Syscall::MusicPause | Syscall::MusicResume => {
AudioControl::MusicPause {
group: uint!(0, 1) as usize,
paused: syscall == Syscall::MusicPause,
}
}
Syscall::MusicSetVolume => AudioControl::MusicVolume {
group: uint!(0, 1) as usize,
gain: uint!(1, 255) as u8,
},
Syscall::MusicFadeTo => {
let group = uint!(0, 1) as usize;
let gain = uint!(1, 255) as u8;
let ms = uint!(2, (u32::MAX / 48) as i64);
let curve = uint!(3, 1);
AudioControl::MusicFade {
group,
gain,
frames: ms * 48,
smooth: curve != 0,
}
}
Syscall::MusicCrossfade => {
let source = uint!(0, 1) as usize;
let target = uint!(1, 1) as usize;
let ms = uint!(2, (u32::MAX / 48) as i64);
let curve = uint!(3, 1);
AudioControl::Crossfade {
source,
target,
frames: ms * 48,
smooth: curve != 0,
}
}
Syscall::AudioSetGroupVolume => AudioControl::GroupVolume {
group: uint!(0, 2) as usize,
gain: uint!(1, 255) as u8,
},
Syscall::AudioSetBusVolume => AudioControl::BusVolume {
bus: uint!(0, 2) as usize,
gain: uint!(1, 255) as u8,
},
Syscall::AudioSetSend => AudioControl::SfxSend {
voice: uint!(0, 7) as usize,
gain: uint!(1, 255) as u8,
},
Syscall::AudioSetEffect => AudioControl::Effect {
group: uint!(0, 2) as usize,
cutoff_hz: uint!(1, 24_000),
delay_frames: uint!(2, 2000) * 48,
feedback: expect_number(args, 3, "feedback")? as f32,
wet: expect_number(args, 4, "wet")? as f32,
dry: expect_number(args, 5, "dry")? as f32,
},
_ => unreachable!(),
};
platform.audio_mut().control(control)
}
};
ret.push_int(status as i64);
Ok(())
}
Syscall::FsOpen => { Syscall::FsOpen => {
let path = expect_string(args, 0, "path")?; let path = expect_string(args, 0, "path")?;
if *self.fs_state != FsState::Mounted { if *self.fs_state != FsState::Mounted {
@@ -749,6 +857,7 @@ impl NativeInterface for VmRuntimeHost<'_> {
0 => BankType::GLYPH, 0 => BankType::GLYPH,
1 => BankType::SOUNDS, 1 => BankType::SOUNDS,
2 => BankType::SCENE, 2 => BankType::SCENE,
3 => BankType::MUSIC,
other => { other => {
return Err(VmFault::Trap( return Err(VmFault::Trap(
TRAP_TYPE, TRAP_TYPE,
@@ -770,6 +879,8 @@ impl NativeInterface for VmRuntimeHost<'_> {
let asset_type = match expect_int(args, 0)? as u32 { let asset_type = match expect_int(args, 0)? as u32 {
0 => BankType::GLYPH, 0 => BankType::GLYPH,
1 => BankType::SOUNDS, 1 => BankType::SOUNDS,
2 => BankType::SCENE,
3 => BankType::MUSIC,
_ => return Err(VmFault::Trap(TRAP_TYPE, "Invalid asset type".to_string())), _ => return Err(VmFault::Trap(TRAP_TYPE, "Invalid asset type".to_string())),
}; };
let telemetry = platform let telemetry = platform
@@ -187,6 +187,16 @@ impl VirtualMachineRuntime {
self.render_worker_ownership.set(self.render_manager.active_ownership()); self.render_worker_ownership.set(self.render_manager.active_ownership());
} }
pub fn record_presented_render_worker_frame(
&self,
ownership: prometeu_hal::RenderOwnership,
frame_id: FrameId,
) {
if let Some(controller) = self.render_worker_controller.as_ref() {
controller.record_presented_frame(ownership, frame_id);
}
}
pub fn record_repeated_render_worker_frame(&self, frame_id: FrameId) { pub fn record_repeated_render_worker_frame(&self, frame_id: FrameId) {
if let Some(controller) = self.render_worker_controller.as_ref() { if let Some(controller) = self.render_worker_controller.as_ref() {
controller.record_repeated_frame(frame_id); controller.record_repeated_frame(frame_id);
@@ -25,6 +25,7 @@ impl Default for RenderWorkerConfig {
struct RenderWorkerState { struct RenderWorkerState {
telemetry: RenderWorkerTelemetry, telemetry: RenderWorkerTelemetry,
last_error: Option<RenderWorkerError>, last_error: Option<RenderWorkerError>,
last_presented: Option<(RenderOwnership, FrameId)>,
} }
#[derive(Clone, Debug, Default)] #[derive(Clone, Debug, Default)]
@@ -40,6 +41,19 @@ impl LatestRenderFrameStore {
pub fn latest_frame(&self) -> Option<OwnedRgba8888Frame> { pub fn latest_frame(&self) -> Option<OwnedRgba8888Frame> {
self.latest.lock().unwrap().clone() self.latest.lock().unwrap().clone()
} }
/// Copy only a different complete publication; paused observers need not copy pixels.
pub fn latest_frame_after(
&self,
previous: Option<(RenderOwnership, FrameId)>,
) -> Option<OwnedRgba8888Frame> {
self.latest
.lock()
.unwrap()
.as_ref()
.filter(|frame| previous != Some((frame.ownership, frame.frame_id)))
.cloned()
}
} }
impl RenderWorkerFrameSink for LatestRenderFrameStore { impl RenderWorkerFrameSink for LatestRenderFrameStore {
@@ -144,6 +158,7 @@ impl RenderWorkerController {
let mut telemetry = self.handoff.telemetry(); let mut telemetry = self.handoff.telemetry();
let state = self.state.lock().unwrap(); let state = self.state.lock().unwrap();
telemetry.published_frames = state.telemetry.published_frames; telemetry.published_frames = state.telemetry.published_frames;
telemetry.presented_frames = state.telemetry.presented_frames;
telemetry.render_failures = state.telemetry.render_failures; telemetry.render_failures = state.telemetry.render_failures;
telemetry.publish_failures = state.telemetry.publish_failures; telemetry.publish_failures = state.telemetry.publish_failures;
telemetry.present_failures = state.telemetry.present_failures; telemetry.present_failures = state.telemetry.present_failures;
@@ -166,6 +181,17 @@ impl RenderWorkerController {
self.ownership.snapshot() self.ownership.snapshot()
} }
/// Record only after a successful host presentation, never on publication.
pub fn record_presented_frame(&self, ownership: RenderOwnership, frame_id: FrameId) {
let mut state = self.state.lock().unwrap();
if state.last_presented == Some((ownership, frame_id)) {
state.telemetry.record_repeated_frame(frame_id.get());
} else {
state.telemetry.record_presented(frame_id.get());
}
state.last_presented = Some((ownership, frame_id));
}
pub fn record_repeated_frame(&self, frame_id: FrameId) { pub fn record_repeated_frame(&self, frame_id: FrameId) {
let mut state = self.state.lock().unwrap(); let mut state = self.state.lock().unwrap();
state.telemetry.record_repeated_frame(frame_id.get()); state.telemetry.record_repeated_frame(frame_id.get());
@@ -190,13 +216,13 @@ fn run_worker_loop<B, S>(
B: RenderWorkerBackend, B: RenderWorkerBackend,
S: RenderWorkerFrameSink, S: RenderWorkerFrameSink,
{ {
loop { let report_path = std::env::var_os("PROMETEU_GRAPHICS_METRICS").map(std::path::PathBuf::from);
let submission = match handoff.wait_take() { let mut measurements = Vec::new();
RenderWorkerHandoffWait::Submission(submission) => submission, let mut measured_frames = 0usize;
RenderWorkerHandoffWait::Shutdown => return, while let RenderWorkerHandoffWait::Submission(submission) = handoff.wait_take() {
};
let frame_id = submission.frame_id; let frame_id = submission.frame_id;
let render_begin = report_path.as_ref().map(|_| std::time::Instant::now());
let render_result = catch_unwind(AssertUnwindSafe(|| backend.render(&submission))) let render_result = catch_unwind(AssertUnwindSafe(|| backend.render(&submission)))
.map_err(|_| RenderWorkerError::WorkerPanic); .map_err(|_| RenderWorkerError::WorkerPanic);
let frame = match render_result { let frame = match render_result {
@@ -212,13 +238,45 @@ fn run_worker_loop<B, S>(
continue; continue;
} }
let render_us = render_begin.map(|start| start.elapsed().as_micros() as u64);
let publish_begin = report_path.as_ref().map(|_| std::time::Instant::now());
let publish_result = catch_unwind(AssertUnwindSafe(|| sink.publish(frame))) let publish_result = catch_unwind(AssertUnwindSafe(|| sink.publish(frame)))
.map_err(|_| RenderWorkerError::WorkerPanic); .map_err(|_| RenderWorkerError::WorkerPanic);
if let (Some(render_us), Some(begin)) = (render_us, publish_begin) {
let sample = (frame_id.get(), render_us, begin.elapsed().as_micros() as u64);
if measurements.len() < 16384 {
measurements.push(sample);
} else {
measurements[measured_frames % 16384] = sample;
}
measured_frames += 1;
}
match publish_result { match publish_result {
Ok(Ok(())) => record_published(&state, frame_id), Ok(Ok(())) => record_published(&state, frame_id),
Ok(Err(error)) | Err(error) => record_error(&state, error, frame_id), Ok(Err(error)) | Err(error) => record_error(&state, error, frame_id),
} }
} }
if let Some(path) = report_path {
use std::io::Write;
let worker = state.lock().unwrap();
let handoff = handoff.telemetry();
let report = serde_json::json!({"schema":1,"source":"async_render_worker",
"produced_submissions":handoff.produced_submissions,"consumed_submissions":handoff.consumed_submissions,
"replaced_before_consume":handoff.replaced_before_consume,
"published_frames":worker.telemetry.published_frames,"first_presentations":worker.telemetry.presented_frames,
"repeated_presentations":worker.telemetry.repeated_frames,
"render_failures":worker.telemetry.render_failures,"publish_failures":worker.telemetry.publish_failures,
"measured_frames":measured_frames,"samples_frame_render_publish_us":measurements,
"physical_display_hz":null,
"limitations":"last 16384 successful-render samples; render includes owned-frame copy; sink publication is not scanout; opt-in clock and storage overhead"});
if let Ok(mut file) = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(path.with_extension("renderer.jsonl"))
{
let _ = writeln!(file, "{report}");
}
}
} }
fn record_published(state: &Arc<Mutex<RenderWorkerState>>, frame_id: FrameId) { fn record_published(state: &Arc<Mutex<RenderWorkerState>>, frame_id: FrameId) {
@@ -333,6 +391,14 @@ mod tests {
SharedFakeBackend(backend), SharedFakeBackend(backend),
); );
let first_session = RenderOwnership::new(1, AppMode::Game, 7);
controller.record_presented_frame(first_session, FrameId::ZERO);
controller.record_presented_frame(first_session, FrameId::ZERO);
// Frame zero in a new epoch is a new image, not a repeat of the old session.
controller.record_presented_frame(RenderOwnership::new(2, AppMode::Game, 7), FrameId::ZERO);
assert_eq!(controller.telemetry().presented_frames, 2);
assert_eq!(controller.telemetry().repeated_frames, 1);
assert_eq!(controller.stop(), Ok(())); assert_eq!(controller.stop(), Ok(()));
assert_eq!(controller.last_error(), None); assert_eq!(controller.last_error(), None);
} }
@@ -14,13 +14,16 @@ struct Harness {
} }
impl Harness { impl Harness {
fn new(code: &str, name: &str, args: u16) -> Self { fn new(code: &str, name: &str, args: u16) -> Self {
Self::new_domain(code, "audio", name, args)
}
fn new_domain(code: &str, module: &str, name: &str, args: u16) -> Self {
let mut runtime = VirtualMachineRuntime::new(None); let mut runtime = VirtualMachineRuntime::new(None);
let mut vm = VirtualMachine::default(); let mut vm = VirtualMachine::default();
let mut log = LogService::new(4096); let mut log = LogService::new(4096);
let program = serialized_single_function_module( let program = serialized_single_function_module(
assemble(code).unwrap(), assemble(code).unwrap(),
vec![SyscallDecl { vec![SyscallDecl {
module: "audio".into(), module: module.into(),
name: name.into(), name: name.into(),
version: 1, version: 1,
arg_slots: args, arg_slots: args,
@@ -76,7 +79,7 @@ impl Harness {
fn guest_audio_status_matrix_and_type_traps() { fn guest_audio_status_matrix_and_type_traps() {
for (name, args, inputs, expected) in [ for (name, args, inputs, expected) in [
("stop", 1, "PUSH_I32 0", 0), ("stop", 1, "PUSH_I32 0", 0),
("stop", 1, "PUSH_I32 15", 0), ("stop", 1, "PUSH_I32 7", 0),
("stop", 1, "PUSH_I32 -1", 1), ("stop", 1, "PUSH_I32 -1", 1),
("stop", 1, "PUSH_I32 16", 1), ("stop", 1, "PUSH_I32 16", 1),
("set_volume", 2, "PUSH_I32 0\nPUSH_I32 0", 0), ("set_volume", 2, "PUSH_I32 0\nPUSH_I32 0", 0),
@@ -84,7 +87,7 @@ fn guest_audio_status_matrix_and_type_traps() {
("set_volume", 2, "PUSH_I32 0\nPUSH_I32 256", 3), ("set_volume", 2, "PUSH_I32 0\nPUSH_I32 256", 3),
("set_pan", 2, "PUSH_I32 0\nPUSH_I32 -1", 3), ("set_pan", 2, "PUSH_I32 0\nPUSH_I32 -1", 3),
("set_pan", 2, "PUSH_I32 16\nPUSH_I32 -1", 1), ("set_pan", 2, "PUSH_I32 16\nPUSH_I32 -1", 1),
("set_pan", 2, "PUSH_I32 15\nPUSH_I32 255", 0), ("set_pan", 2, "PUSH_I32 7\nPUSH_I32 255", 0),
("set_pitch", 2, "PUSH_I32 0\nPUSH_I32 1", 0), ("set_pitch", 2, "PUSH_I32 0\nPUSH_I32 1", 0),
("set_pitch", 2, "PUSH_I32 0\nPUSH_I32 0", 3), ("set_pitch", 2, "PUSH_I32 0\nPUSH_I32 0", 3),
("set_pitch", 2, "PUSH_I32 0\nPUSH_I32 -1", 3), ("set_pitch", 2, "PUSH_I32 0\nPUSH_I32 -1", 3),
@@ -216,3 +219,61 @@ fn normal_entry_return_is_a_successful_implicit_audio_frame() {
} }
assert_eq!(h.transport().try_take().unwrap().frame.unwrap().len(), 1); assert_eq!(h.transport().try_take().unwrap().frame.unwrap().len(), 1);
} }
#[test]
fn music_and_bus_controls_resolve_with_audio_capability_and_validate_without_partial_commands() {
use prometeu_drivers::SoundBankPoolInstaller;
use prometeu_drivers::bank_format::{DecodedAudioBank, decode_audio_bank};
for (module, name, args, inputs, expected) in [
("music", "play", 3, "PUSH_I32 0\nPUSH_I32 0\nPUSH_I32 1", 0),
("music", "play", 3, "PUSH_I32 0\nPUSH_I32 16\nPUSH_I32 1", 3),
("music", "stop", 1, "PUSH_I32 1", 0),
("music", "pause", 1, "PUSH_I32 0", 0),
("music", "resume", 1, "PUSH_I32 0", 0),
("music", "set_volume", 2, "PUSH_I32 0\nPUSH_I32 255", 0),
("music", "fade_to", 4, "PUSH_I32 0\nPUSH_I32 0\nPUSH_I32 2000\nPUSH_I32 1", 0),
("music", "crossfade", 4, "PUSH_I32 0\nPUSH_I32 1\nPUSH_I32 2000\nPUSH_I32 0", 0),
("music", "crossfade", 4, "PUSH_I32 0\nPUSH_I32 0\nPUSH_I32 2000\nPUSH_I32 0", 3),
("audio", "set_group_volume", 2, "PUSH_I32 2\nPUSH_I32 255", 0),
("audio", "set_bus_volume", 2, "PUSH_I32 2\nPUSH_I32 255", 0),
("audio", "set_send", 2, "PUSH_I32 7\nPUSH_I32 128", 0),
(
"audio",
"set_effect",
6,
"PUSH_I32 2\nPUSH_I32 4000\nPUSH_I32 2000\nPUSH_F64 f64:0x3fe0000000000000\nPUSH_F64 f64:0x3fd3333333333333\nPUSH_F64 f64:0x3fe6666666666666",
0,
),
(
"audio",
"set_effect",
6,
"PUSH_I32 2\nPUSH_I32 4000\nPUSH_I32 2001\nPUSH_F64 f64:0x3fe0000000000000\nPUSH_F64 f64:0x3fd3333333333333\nPUSH_F64 f64:0x3fe6666666666666",
3,
),
("audio", "unload_bank", 2, "PUSH_I32 1\nPUSH_I32 0", 0),
] {
let mut h = Harness::new_domain(
&format!("{inputs}\nHOSTCALL 0\nFRAME_SYNC\nHALT"),
module,
name,
args,
);
let bytes = include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../../test-fixtures/audio/mixed-music.mbnk"
));
let DecodedAudioBank::Music(bank) = decode_audio_bank(bytes, true).unwrap() else {
panic!()
};
h.platform.local_hardware().memory_banks.install_music_bank(0, Arc::new(bank));
assert!(h.tick().is_none(), "{module}.{name}");
assert_eq!(h.vm.operand_stack_top(1), vec![Value::Int64(expected)], "{module}.{name}");
let batch = h.transport().try_take().unwrap().frame;
assert_eq!(
batch.map_or(0, |v| v.len()),
usize::from(expected == 0 && name != "unload_bank"),
"{module}.{name}"
);
}
}
+4
View File
@@ -18,3 +18,7 @@ fpmath = { version = "=0.1.1", features = ["soft-float"] }
[dev-dependencies] [dev-dependencies]
prometeu-test-support = { path = "../../dev/prometeu-test-support" } prometeu-test-support = { path = "../../dev/prometeu-test-support" }
[[example]]
name = "string_provenance_benchmark"
required-features = ["resource-diagnostics"]
@@ -0,0 +1,83 @@
//! Run in release with --features resource-diagnostics; query setup is excluded.
#[allow(dead_code)]
#[path = "../tests/support/callables.rs"]
mod callables;
use prometeu_bytecode::model::ConstantPoolEntry;
use prometeu_vm::string_provenance_qualification::Fixture;
use std::{
alloc::{GlobalAlloc, Layout, System},
sync::atomic::{AtomicUsize, Ordering},
time::Instant,
};
struct Counting;
static ALLOC: AtomicUsize = AtomicUsize::new(0);
static REALLOC: AtomicUsize = AtomicUsize::new(0);
static FREE: AtomicUsize = AtomicUsize::new(0);
unsafe impl GlobalAlloc for Counting {
unsafe fn alloc(&self, l: Layout) -> *mut u8 {
ALLOC.fetch_add(1, Ordering::Relaxed);
unsafe { System.alloc(l) }
}
unsafe fn dealloc(&self, p: *mut u8, l: Layout) {
FREE.fetch_add(1, Ordering::Relaxed);
unsafe { System.dealloc(p, l) }
}
unsafe fn realloc(&self, p: *mut u8, l: Layout, n: usize) -> *mut u8 {
REALLOC.fetch_add(1, Ordering::Relaxed);
unsafe { System.realloc(p, l, n) }
}
}
#[global_allocator]
static GLOBAL: Counting = Counting;
fn counts() -> [usize; 3] {
[ALLOC.load(Ordering::Relaxed), REALLOC.load(Ordering::Relaxed), FREE.load(Ordering::Relaxed)]
}
fn main() {
for n in [1024, 16384, 65536] {
let constants = vec![ConstantPoolEntry::String("same".into()); n];
let start = Instant::now();
let image = callables::image_constants(
&[("RET", callables::sig(&[], &[]), vec![])],
vec![],
&constants,
)
.unwrap();
let image_ns = start.elapsed().as_nanos();
let payload = image.payload_bytes();
let before = counts();
let start = Instant::now();
let mut fixture = Fixture::new(image, n);
let creation_ns = start.elapsed().as_nanos();
let after = counts();
println!(
"setup n={n} image_load_ns={image_ns} image_payload_bytes={payload} session_ns={creation_ns} session_allocs={} auxiliary_bytes={} work={:?}",
after[0] - before[0],
fixture.auxiliary_bytes(),
fixture.work()
);
for name in [
"image_hit",
"image_miss",
"session_hit",
"session_miss",
"import_hit",
"snapshot_mixed",
"import_miss",
] {
let before = counts();
let start = Instant::now();
for _ in 0..1000 {
fixture.run(name);
}
let ns = start.elapsed().as_nanos();
let after = counts();
let delta = std::array::from_fn::<_, 3, _>(|i| after[i] - before[i]);
if name.ends_with("hit") || name == "image_miss" || name == "session_miss" {
assert_eq!(delta, [0; 3]);
}
println!(
"query n={n} name={name} iterations=1000 ns={ns} alloc_realloc_free={delta:?}"
);
}
}
}
+103 -14
View File
@@ -235,12 +235,13 @@ pub type IntrinsicHostImplementation =
pub enum IntrinsicImplementation { pub enum IntrinsicImplementation {
Host(IntrinsicHostImplementation), Host(IntrinsicHostImplementation),
VmOwned, VmOwned,
BorrowedInline(crate::IntrinsicInlineImplementation),
} }
impl IntrinsicImplementation { impl IntrinsicImplementation {
pub fn host_callback(self) -> Option<IntrinsicHostImplementation> { pub fn host_callback(self) -> Option<IntrinsicHostImplementation> {
match self { match self {
Self::Host(f) => Some(f), Self::Host(f) => Some(f),
Self::VmOwned => None, Self::VmOwned | Self::BorrowedInline(_) => None,
} }
} }
} }
@@ -249,12 +250,14 @@ impl IntrinsicImplementation {
pub enum IntrinsicCostModel { pub enum IntrinsicCostModel {
Fixed, Fixed,
RandomRejectionV1, RandomRejectionV1,
ArgumentUtf8BytesV1,
} }
impl IntrinsicCostModel { impl IntrinsicCostModel {
pub const fn name(self) -> &'static str { pub const fn name(self) -> &'static str {
match self { match self {
Self::Fixed => "fixed", Self::Fixed => "fixed",
Self::RandomRejectionV1 => "random_rejection_v1", Self::RandomRejectionV1 => "random_rejection_v1",
Self::ArgumentUtf8BytesV1 => "argument_utf8_bytes_v1",
} }
} }
} }
@@ -266,11 +269,14 @@ pub struct IntrinsicMeta {
pub name: &'static str, pub name: &'static str,
pub version: u16, pub version: u16,
pub arg_layout: &'static [AbiType], pub arg_layout: &'static [AbiType],
pub arg_nullable: &'static [bool],
pub ret_layout: &'static [AbiType], pub ret_layout: &'static [AbiType],
pub parameter_names: &'static [&'static str], pub parameter_names: &'static [&'static str],
/// Full fixed cost, or base cost for a variable model. /// Full fixed cost, or base cost for a variable model.
pub cost_cycles: u64, pub cost_cycles: u64,
pub cost_model: IntrinsicCostModel, pub cost_model: IntrinsicCostModel,
pub cost_arg_index: Option<usize>,
pub cost_per_byte: u64,
pub attempt_cost_cycles: u64, pub attempt_cost_cycles: u64,
pub max_attempts: u32, pub max_attempts: u32,
pub mutates_guest_state: bool, pub mutates_guest_state: bool,
@@ -300,7 +306,16 @@ impl IntrinsicMeta {
&self, &self,
values: &[Value], values: &[Value],
) -> Result<(), IntrinsicExecutionError> { ) -> Result<(), IntrinsicExecutionError> {
Self::validate_values(values, self.arg_layout) Self::validate_values(values, self.arg_layout)?;
for (index, (value, nullable)) in values.iter().zip(self.arg_nullable).enumerate() {
if matches!(value, Value::Null) && !nullable {
return Err(IntrinsicExecutionError::TypeMismatch {
index,
expected: self.arg_layout[index],
});
}
}
Ok(())
} }
fn validate_values( fn validate_values(
@@ -563,10 +578,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "dot", name: "dot",
version: 1, version: 1,
arg_layout: &VEC2_DOT_ARGS, arg_layout: &VEC2_DOT_ARGS,
arg_nullable: &[false; VEC2_DOT_ARGS.len()],
ret_layout: &SINGLE_FLOAT_RET, ret_layout: &SINGLE_FLOAT_RET,
parameter_names: &["ax", "ay", "bx", "by"], parameter_names: &["ax", "ay", "bx", "by"],
cost_cycles: 10, cost_cycles: 10,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -580,10 +598,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "length", name: "length",
version: 1, version: 1,
arg_layout: &VEC2_LENGTH_ARGS, arg_layout: &VEC2_LENGTH_ARGS,
arg_nullable: &[false; VEC2_LENGTH_ARGS.len()],
ret_layout: &SINGLE_FLOAT_RET, ret_layout: &SINGLE_FLOAT_RET,
parameter_names: &["x", "y"], parameter_names: &["x", "y"],
cost_cycles: 40, cost_cycles: 40,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -597,10 +618,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "from_raw", name: "from_raw",
version: 1, version: 1,
arg_layout: &INT_RET, arg_layout: &INT_RET,
arg_nullable: &[false; INT_RET.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &["raw"], parameter_names: &["raw"],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -614,10 +638,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "rgb", name: "rgb",
version: 1, version: 1,
arg_layout: &COLOR_RGB_ARGS, arg_layout: &COLOR_RGB_ARGS,
arg_nullable: &[false; COLOR_RGB_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &["r", "g", "b"], parameter_names: &["r", "g", "b"],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -631,10 +658,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "rgba", name: "rgba",
version: 1, version: 1,
arg_layout: &COLOR_RGBA_ARGS, arg_layout: &COLOR_RGBA_ARGS,
arg_nullable: &[false; COLOR_RGBA_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &["r", "g", "b", "a"], parameter_names: &["r", "g", "b", "a"],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -648,10 +678,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "html_rgba", name: "html_rgba",
version: 1, version: 1,
arg_layout: &COLOR_HTML_RGBA_ARGS, arg_layout: &COLOR_HTML_RGBA_ARGS,
arg_nullable: &[false; COLOR_HTML_RGBA_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &["value"], parameter_names: &["value"],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -665,10 +698,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "gray_scale", name: "gray_scale",
version: 1, version: 1,
arg_layout: &INT_RET, arg_layout: &INT_RET,
arg_nullable: &[false; INT_RET.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &["value"], parameter_names: &["value"],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -682,10 +718,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "hex", name: "hex",
version: 1, version: 1,
arg_layout: &COLOR_ARGS, arg_layout: &COLOR_ARGS,
arg_nullable: &[false; COLOR_ARGS.len()],
ret_layout: &INT_RET, ret_layout: &INT_RET,
parameter_names: &["color"], parameter_names: &["color"],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -699,10 +738,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "alpha", name: "alpha",
version: 1, version: 1,
arg_layout: &COLOR_ARGS, arg_layout: &COLOR_ARGS,
arg_nullable: &[false; COLOR_ARGS.len()],
ret_layout: &INT_RET, ret_layout: &INT_RET,
parameter_names: &["color"], parameter_names: &["color"],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -716,10 +758,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "black", name: "black",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -733,10 +778,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "white", name: "white",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -750,10 +798,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "red", name: "red",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -767,10 +818,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "green", name: "green",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -784,10 +838,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "blue", name: "blue",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -801,10 +858,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "yellow", name: "yellow",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -818,10 +878,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "orange", name: "orange",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -835,10 +898,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "indigo", name: "indigo",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -852,10 +918,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "gray", name: "gray",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -869,10 +938,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "cyan", name: "cyan",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -886,10 +958,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "magenta", name: "magenta",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -903,10 +978,13 @@ const INTRINSICS: [IntrinsicMeta; 21] = [
name: "transparent", name: "transparent",
version: 1, version: 1,
arg_layout: &NO_ARGS, arg_layout: &NO_ARGS,
arg_nullable: &[false; NO_ARGS.len()],
ret_layout: &COLOR_RET, ret_layout: &COLOR_RET,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -944,6 +1022,7 @@ pub fn lookup_intrinsic(owner: &str, name: &str, version: u16) -> Option<&'stati
.chain(crate::input_intrinsics::INPUT_INTRINSICS.iter()) .chain(crate::input_intrinsics::INPUT_INTRINSICS.iter())
.chain(crate::math_intrinsics::MATH_INTRINSICS.iter()) .chain(crate::math_intrinsics::MATH_INTRINSICS.iter())
.chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter()) .chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter())
.chain(crate::hash_intrinsics::HASH_INTRINSICS.iter())
.find(|meta| meta.owner == owner && meta.name == name && meta.version == version) .find(|meta| meta.owner == owner && meta.name == name && meta.version == version)
} }
@@ -953,6 +1032,7 @@ pub fn lookup_intrinsic_by_id(id: u32) -> Option<&'static IntrinsicMeta> {
.chain(crate::input_intrinsics::INPUT_INTRINSICS.iter()) .chain(crate::input_intrinsics::INPUT_INTRINSICS.iter())
.chain(crate::math_intrinsics::MATH_INTRINSICS.iter()) .chain(crate::math_intrinsics::MATH_INTRINSICS.iter())
.chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter()) .chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter())
.chain(crate::hash_intrinsics::HASH_INTRINSICS.iter())
.find(|meta| meta.id == id) .find(|meta| meta.id == id)
} }
@@ -1254,15 +1334,36 @@ pub(crate) fn metered_lookup_intrinsic(
.chain(crate::input_intrinsics::INPUT_INTRINSICS.iter()) .chain(crate::input_intrinsics::INPUT_INTRINSICS.iter())
.chain(crate::math_intrinsics::MATH_INTRINSICS.iter()) .chain(crate::math_intrinsics::MATH_INTRINSICS.iter())
.chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter()) .chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter())
.chain(crate::hash_intrinsics::HASH_INTRINSICS.iter())
{ {
budget.work(1)?; budget.work(1)?;
if meta.id == id { if meta.id == id {
budget.work(meta.arg_slots() + 1)?;
meta.validate_descriptor().map_err(|_| {
prometeu_bytecode::load_limits::LoadFailure::new(
prometeu_bytecode::load_limits::LoadCode::InvalidCode,
prometeu_bytecode::load_limits::LoadReason::NativeId,
)
})?;
return Ok(Some(meta)); return Ok(Some(meta));
} }
} }
Ok(None) Ok(None)
} }
/// Runtime-owned catalogue, including input and scalar math.
pub fn intrinsic_catalog() -> impl Iterator<Item = &'static IntrinsicMeta> {
INTRINSICS
.iter()
.chain(crate::input_intrinsics::INPUT_INTRINSICS.iter())
.chain(crate::math_intrinsics::MATH_INTRINSICS.iter())
.chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter())
.chain(crate::hash_intrinsics::HASH_INTRINSICS.iter())
}
pub fn builtin_constant_catalog() -> impl Iterator<Item = &'static BuiltinConstMeta> {
BUILTIN_CONSTS.iter()
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -1432,15 +1533,3 @@ mod tests {
assert_eq!(pixel.field("color").map(|field| field.start_slot), Some(2)); assert_eq!(pixel.field("color").map(|field| field.start_slot), Some(2));
} }
} }
/// Runtime-owned catalogue, including input and scalar math.
pub fn intrinsic_catalog() -> impl Iterator<Item = &'static IntrinsicMeta> {
INTRINSICS
.iter()
.chain(crate::input_intrinsics::INPUT_INTRINSICS.iter())
.chain(crate::math_intrinsics::MATH_INTRINSICS.iter())
.chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter())
}
pub fn builtin_constant_catalog() -> impl Iterator<Item = &'static BuiltinConstMeta> {
BUILTIN_CONSTS.iter()
}
+10 -3
View File
@@ -16,14 +16,15 @@ fn atom(t: AbiType) -> String {
} }
pub fn intrinsic_catalog_csv() -> String { pub fn intrinsic_catalog_csv() -> String {
let mut out = String::from( let mut out = String::from(
"final_id_hex,final_id_dec,canonical_name,canonical_version,owner,name,arg_slots,ret_slots,arg_layout,ret_layout,deterministic,may_allocate,parameter_names,cost_cycles,mutates_guest_state,cost_model,attempt_cost_cycles,max_attempts\n", "final_id_hex,final_id_dec,canonical_name,canonical_version,owner,name,arg_slots,ret_slots,arg_layout,ret_layout,deterministic,may_allocate,parameter_names,cost_cycles,mutates_guest_state,cost_model,attempt_cost_cycles,max_attempts,schema_version,arg_nullable,invocation_model,scratch_model,context_access,cost_arg_index,cost_per_byte\n",
); );
let mut entries: Vec<_> = intrinsic_catalog().collect(); let mut entries: Vec<_> = intrinsic_catalog().collect();
entries.sort_by_key(|m| m.id); entries.sort_by_key(|m| m.id);
for m in entries { for m in entries {
m.validate_descriptor().expect("valid canonical intrinsic descriptor");
writeln!( writeln!(
&mut out, &mut out,
"0x{:04X},{},{}.{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{}", "0x{:04X},{},{}.{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},2,{},{},{},{},{},{}",
m.id, m.id,
m.id, m.id,
m.owner, m.owner,
@@ -42,7 +43,13 @@ pub fn intrinsic_catalog_csv() -> String {
m.mutates_guest_state, m.mutates_guest_state,
m.cost_model.name(), m.cost_model.name(),
m.attempt_cost_cycles, m.attempt_cost_cycles,
m.max_attempts m.max_attempts,
m.arg_nullable.iter().map(bool::to_string).collect::<Vec<_>>().join("|"),
m.invocation_model(),
m.scratch_model(),
m.context_access(),
m.cost_arg_index.map_or_else(String::new, |i| i.to_string()),
m.cost_per_byte
) )
.unwrap(); .unwrap();
} }
@@ -0,0 +1,66 @@
use crate::{
AbiType, BuiltinScalarType as S, IntrinsicCostModel as C, IntrinsicExecutionError as E,
IntrinsicImplementation as I, IntrinsicMeta, IntrinsicResults,
};
use prometeu_bytecode::Value;
const fn entry(
id: u32,
name: &'static str,
abi: AbiType,
nullable: bool,
handler: crate::IntrinsicInlineImplementation,
) -> IntrinsicMeta {
IntrinsicMeta {
id,
owner: "hash",
name,
version: 1,
arg_layout: match abi {
AbiType::Scalar(S::Int64) => &[AbiType::Scalar(S::Int64)],
AbiType::Scalar(S::Float) => &[AbiType::Scalar(S::Float)],
AbiType::Scalar(S::String) => &[AbiType::Scalar(S::String)],
_ => panic!("unsupported hash input ABI"),
},
ret_layout: &[AbiType::Scalar(S::Int64)],
arg_nullable: if nullable { &[true] } else { &[false] },
parameter_names: &["value"],
cost_cycles: 8,
cost_model: if matches!(abi, AbiType::Scalar(S::String)) {
C::ArgumentUtf8BytesV1
} else {
C::Fixed
},
cost_arg_index: if matches!(abi, AbiType::Scalar(S::String)) { Some(0) } else { None },
cost_per_byte: if matches!(abi, AbiType::Scalar(S::String)) { 1 } else { 0 },
attempt_cost_cycles: 0,
max_attempts: 0,
mutates_guest_state: false,
deterministic: true,
may_allocate: false,
implementation: I::BorrowedInline(handler),
}
}
pub(crate) const HASH_INTRINSICS: [IntrinsicMeta; 3] = [
entry(0x4100, "i64", AbiType::Scalar(S::Int64), false, hash_i64),
entry(0x4101, "float", AbiType::Scalar(S::Float), false, hash_float),
entry(0x4102, "string", AbiType::Scalar(S::String), true, hash_string),
];
fn hash_i64(args: &[Value], out: &mut IntrinsicResults) -> Result<(), E> {
let Value::Int64(value) = args[0] else { unreachable!("exact validated Int64") };
out.push(Value::Int64(crate::hashing::xxh64(&value.to_le_bytes()) as i64))
}
fn hash_float(args: &[Value], out: &mut IntrinsicResults) -> Result<(), E> {
let Value::Float(value) = args[0] else { unreachable!("exact validated Float") };
crate::numeric::check_inputs(&[value])?;
let bits = if value == 0.0 { 0 } else { value.to_bits() };
out.push(Value::Int64(crate::hashing::xxh64(&bits.to_le_bytes()) as i64))
}
fn hash_string(args: &[Value], out: &mut IntrinsicResults) -> Result<(), E> {
let hash = match &args[0] {
Value::Null => 0,
Value::String(s) => crate::hashing::xxh64(s.as_bytes()) as i64,
_ => unreachable!("exact validated nullable String"),
};
out.push(Value::Int64(hash))
}
+131
View File
@@ -0,0 +1,131 @@
//! Hash-v1: portable XXH64, seed zero, over explicit machine byte encodings.
use prometeu_bytecode::HeapRef;
const P1: u64 = 0x9E3779B185EBCA87;
const P2: u64 = 0xC2B2AE3D27D4EB4F;
const P3: u64 = 0x165667B19E3779F9;
const P4: u64 = 0x85EBCA77C2B2AE63;
const P5: u64 = 0x27D4EB2F165667C5;
fn word(bytes: &[u8]) -> u64 {
u64::from_le_bytes(bytes[..8].try_into().expect("complete XXH64 lane"))
}
fn round(acc: u64, lane: u64) -> u64 {
acc.wrapping_add(lane.wrapping_mul(P2)).rotate_left(31).wrapping_mul(P1)
}
fn merge(acc: u64, lane: u64) -> u64 {
(acc ^ round(0, lane)).wrapping_mul(P1).wrapping_add(P4)
}
pub(crate) fn xxh64(mut bytes: &[u8]) -> u64 {
let length = bytes.len() as u64;
let mut hash = if bytes.len() >= 32 {
let mut acc = [P1.wrapping_add(P2), P2, 0, 0u64.wrapping_sub(P1)];
while bytes.len() >= 32 {
for (index, value) in acc.iter_mut().enumerate() {
*value = round(*value, word(&bytes[index * 8..]));
}
bytes = &bytes[32..];
}
let mut hash = acc[0]
.rotate_left(1)
.wrapping_add(acc[1].rotate_left(7))
.wrapping_add(acc[2].rotate_left(12))
.wrapping_add(acc[3].rotate_left(18));
for value in acc {
hash = merge(hash, value);
}
hash
} else {
P5
};
hash = hash.wrapping_add(length);
while bytes.len() >= 8 {
hash ^= round(0, word(bytes));
hash = hash.rotate_left(27).wrapping_mul(P1).wrapping_add(P4);
bytes = &bytes[8..];
}
if bytes.len() >= 4 {
hash ^= u64::from(u32::from_le_bytes(bytes[..4].try_into().unwrap())).wrapping_mul(P1);
hash = hash.rotate_left(23).wrapping_mul(P2).wrapping_add(P3);
bytes = &bytes[4..];
}
for byte in bytes {
hash ^= u64::from(*byte).wrapping_mul(P5);
hash = hash.rotate_left(11).wrapping_mul(P1);
}
hash ^= hash >> 33;
hash = hash.wrapping_mul(P2);
hash ^= hash >> 29;
hash = hash.wrapping_mul(P3);
hash ^ (hash >> 32)
}
pub(crate) fn reference(value: Option<HeapRef>) -> i64 {
value.map_or(0, |reference| {
let mut bytes = [0; 8];
bytes[..4].copy_from_slice(&reference.index.to_le_bytes());
bytes[4..].copy_from_slice(&reference.generation.to_le_bytes());
xxh64(&bytes) as i64
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn matches_all_independent_c_reference_vectors() {
let mut count = 0;
for row in include_str!("../tests/fixtures/hash-conformance.tsv").lines().skip(1) {
let f: Vec<_> = row.split('\t').collect();
let expected = u64::from_str_radix(f[2], 16).unwrap();
let actual = match f[0] {
"bytes" | "string" => {
let bytes: Vec<_> = (0..f[1].len())
.step_by(2)
.map(|i| u8::from_str_radix(&f[1][i..i + 2], 16).unwrap())
.collect();
xxh64(&bytes)
}
"i64" => xxh64(&f[1].parse::<i64>().unwrap().to_le_bytes()),
"float" => {
let bits = u64::from_str_radix(f[1], 16).unwrap();
let bits = if f64::from_bits(bits) == 0.0 { 0 } else { bits };
xxh64(&bits.to_le_bytes())
}
"reference" => {
let (index, generation) = f[1].split_once(',').unwrap();
reference(Some(HeapRef {
index: index.parse().unwrap(),
generation: generation.parse().unwrap(),
})) as u64
}
"null" => reference(None) as u64,
kind => panic!("unknown vector {kind}"),
};
assert_eq!(actual, expected, "{row}");
count += 1;
}
assert_eq!(count, 118);
}
#[test]
fn sequential_handles_spread_and_bucket_collisions_do_not_imply_identity() {
let mut buckets = [0; 256];
let mut first = [None; 256];
let mut collisions = 0;
for index in 1..=16384 {
let handle = HeapRef { index, generation: 1 };
let bucket = reference(Some(handle)) as u64 as usize & 255;
buckets[bucket] += 1;
if let Some(previous) = first[bucket] {
assert_ne!(previous, handle);
collisions += 1;
} else {
first[bucket] = Some(handle);
}
}
assert!(buckets.iter().all(|&count| (32..=100).contains(&count)), "{buckets:?}");
assert_eq!(collisions, 16384 - 256);
}
}
+40
View File
@@ -428,6 +428,34 @@ impl Heap {
self.allocation_count.load(Ordering::Relaxed) self.allocation_count.load(Ordering::Relaxed)
} }
/// Host-side observations only, outside guest execution and atomic commits.
#[cfg(feature = "resource-diagnostics")]
pub fn diagnostic_typed_storage(&self) -> Vec<crate::resource_diagnostics::StorageObject> {
use crate::resource_diagnostics::{StorageObject, StorageValue};
self.objects
.iter()
.enumerate()
.filter_map(|(index, object)| {
let storage = object.as_ref()?.typed.as_ref()?;
let groups = (0..storage.len())
.map(|i| {
storage
.read(i)
.expect("initialized typed storage")
.as_slice()
.iter()
.map(StorageValue::observe)
.collect()
})
.collect();
Some(StorageObject {
reference: HeapRef { index: index as u32, generation: self.generations[index] },
type_id: storage.type_id(),
groups,
})
})
.collect()
}
#[cfg(feature = "resource-diagnostics")] #[cfg(feature = "resource-diagnostics")]
pub(crate) fn diagnostic_handles(&self) -> (usize, usize) { pub(crate) fn diagnostic_handles(&self) -> (usize, usize) {
(self.free_list.len(), self.objects.len() - self.free_list.len() - self.len()) (self.free_list.len(), self.objects.len() - self.free_list.len() - self.len())
@@ -520,6 +548,18 @@ impl Heap {
value_contract::validate(registry, self, storage.slot_type(index)?, values.as_slice())?; value_contract::validate(registry, self, storage.slot_type(index)?, values.as_slice())?;
Ok(values) Ok(values)
} }
/// Caller has checked all receivers, bounds and groups for the entire attempt.
/// This commit path performs no allocation, provenance lookup or guest callback.
pub(crate) fn commit_typed(&mut self, r: HeapRef, index: usize, values: &[Value]) {
self.objects[r.index as usize]
.as_mut()
.unwrap()
.typed
.as_mut()
.unwrap()
.write_validated(index, values)
.expect("prevalidated storage publication");
}
pub(crate) fn store_typed( pub(crate) fn store_typed(
&mut self, &mut self,
registry: &Registry, registry: &Registry,
@@ -20,10 +20,13 @@ const fn query(
name, name,
version: 1, version: 1,
arg_layout: &[], arg_layout: &[],
arg_nullable: &[],
ret_layout, ret_layout,
parameter_names: &[], parameter_names: &[],
cost_cycles: 1, cost_cycles: 1,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -0,0 +1,143 @@
//! The native handler variant and descriptor jointly define the invocation contract.
use crate::{
AbiType, BuiltinScalarType, IntrinsicCostModel, IntrinsicExecutionError as E,
IntrinsicImplementation as I, IntrinsicMeta,
};
use prometeu_bytecode::Value;
pub type IntrinsicInlineImplementation = fn(&[Value], &mut IntrinsicResults) -> Result<(), E>;
const INLINE_SLOTS: usize = 3;
/// Fixed native staging only; no heap buffer or mutable execution context.
#[derive(Debug)]
pub struct IntrinsicResults {
values: [Value; INLINE_SLOTS],
len: usize,
limit: usize,
overflowed: bool,
}
impl IntrinsicResults {
pub(crate) fn new(limit: usize) -> Self {
assert!(limit <= INLINE_SLOTS);
Self { values: [const { Value::Null }; INLINE_SLOTS], len: 0, limit, overflowed: false }
}
pub fn push(&mut self, value: Value) -> Result<(), E> {
if self.len >= self.limit {
self.overflowed = true;
return Err(E::ArityMismatch { expected: self.limit, got: self.len + 1 });
}
self.values[self.len] = value;
self.len += 1;
Ok(())
}
pub(crate) fn validate(&self, meta: &IntrinsicMeta) -> Result<(), E> {
if self.overflowed {
return Err(E::ArityMismatch { expected: self.limit, got: self.limit + 1 });
}
meta.validate_result_values(&self.values[..self.len])
}
pub(crate) fn as_slice(&self) -> &[Value] {
&self.values[..self.len]
}
pub(crate) fn into_values(self) -> impl Iterator<Item = Value> {
self.values.into_iter().take(self.len)
}
}
impl IntrinsicMeta {
pub fn invocation_model(&self) -> &'static str {
match self.implementation {
I::Host(_) => "host_buffered_v1",
I::VmOwned => "vm_owned_v1",
I::BorrowedInline(_) => "borrowed_inline_v1",
}
}
pub fn scratch_model(&self) -> &'static str {
match self.implementation {
I::BorrowedInline(_) => "none",
_ => "existing_contract_v1",
}
}
pub fn context_access(&self) -> &'static str {
match self.implementation {
I::Host(_) => "host",
I::VmOwned => "vm_state",
I::BorrowedInline(_) => "none",
}
}
/// Derivation from the handler variant makes incompatible profile claims unrepresentable.
pub fn validate_descriptor(&self) -> Result<(), &'static str> {
if self.arg_nullable.len() != self.arg_slots()
|| self.parameter_names.len() != self.arg_slots()
{
return Err("argument metadata widths disagree");
}
for abi in self.arg_layout.iter().chain(self.ret_layout) {
if !matches!(abi, AbiType::Scalar(_))
&& !matches!(abi, AbiType::Builtin(key) if (key.name == "Random" || key.name == "color") && key.version == 1)
{
return Err("unsupported intrinsic ABI layout");
}
}
for (abi, nullable) in self.arg_layout.iter().zip(self.arg_nullable) {
let permits_null = matches!(abi, AbiType::Scalar(BuiltinScalarType::String))
|| matches!(abi, AbiType::Builtin(key) if key.name == "Random" && key.version == 1);
if *nullable && !permits_null {
return Err("non-nullable ABI carrier declared nullable");
}
}
match self.cost_model {
IntrinsicCostModel::Fixed => {
if self.cost_arg_index.is_some()
|| self.cost_per_byte != 0
|| self.attempt_cost_cycles != 0
|| self.max_attempts != 0
{
return Err("fixed cost has variable parameters");
}
}
IntrinsicCostModel::RandomRejectionV1 => {
if !matches!(self.implementation, I::VmOwned)
|| self.cost_arg_index.is_some()
|| self.cost_per_byte != 0
|| self.attempt_cost_cycles == 0
|| self.max_attempts == 0
{
return Err("invalid random rejection cost parameters");
}
}
IntrinsicCostModel::ArgumentUtf8BytesV1 => {
if !matches!(self.implementation, I::BorrowedInline(_))
|| self.cost_arg_index.and_then(|i| self.arg_layout.get(i))
!= Some(&AbiType::Scalar(BuiltinScalarType::String))
|| self.cost_per_byte == 0
|| self.attempt_cost_cycles != 0
|| self.max_attempts != 0
{
return Err("invalid String byte-cost parameters");
}
}
}
if matches!(self.implementation, I::BorrowedInline(_))
&& (self.ret_slots() > INLINE_SLOTS
|| self.ret_slots() > self.arg_slots()
|| self.may_allocate
|| self.mutates_guest_state
|| !self.deterministic)
{
return Err("handler violates bounded invocation capabilities");
}
Ok(())
}
/// None is checked cost overflow. Wrong/missing carriers have no inferred length.
pub(crate) fn attempted_cost(&self, args: &[Value]) -> Option<u64> {
if self.cost_model != IntrinsicCostModel::ArgumentUtf8BytesV1 {
return Some(self.cost_cycles);
}
let bytes = match self.cost_arg_index.and_then(|i| args.get(i)) {
Some(Value::String(value)) => value.len() as u64,
_ => 0,
};
bytes.checked_mul(self.cost_per_byte)?.checked_add(self.cost_cycles)
}
}
+8
View File
@@ -2,7 +2,11 @@
extern crate self as prometeu_vm; extern crate self as prometeu_vm;
mod builtins; mod builtins;
mod catalog; mod catalog;
mod hash_intrinsics;
mod hashing;
mod intrinsic_contract;
pub use catalog::{builtin_constant_catalog_csv, intrinsic_catalog_csv}; pub use catalog::{builtin_constant_catalog_csv, intrinsic_catalog_csv};
pub use intrinsic_contract::{IntrinsicInlineImplementation, IntrinsicResults};
mod call_frame; mod call_frame;
mod input_intrinsics; mod input_intrinsics;
mod math_intrinsics; mod math_intrinsics;
@@ -25,6 +29,7 @@ mod program_image;
mod registry_verifier; mod registry_verifier;
pub use program_image::{ImageStringLease, ValidatedProgramImage}; pub use program_image::{ImageStringLease, ValidatedProgramImage};
// Image/session creation and native provenance consume this in step 19. // Image/session creation and native provenance consume this in step 19.
mod string_identity_index;
#[allow(dead_code)] #[allow(dead_code)]
mod string_ledger; mod string_ledger;
// Registry-backed consumers are integrated with executable images in PLN-0176. // Registry-backed consumers are integrated with executable images in PLN-0176.
@@ -52,3 +57,6 @@ pub use virtual_machine::{BudgetReport, LogicalFrameEndingReason, VirtualMachine
pub use vm_init_error::{LoaderPatchError, VmInitError}; pub use vm_init_error::{LoaderPatchError, VmInitError};
pub use verifier::VerifierError; pub use verifier::VerifierError;
#[cfg(feature = "resource-diagnostics")]
pub use virtual_machine::string_provenance_qualification;
@@ -20,10 +20,17 @@ const fn entry<const ID: u32>(
name, name,
version: 1, version: 1,
arg_layout, arg_layout,
arg_nullable: match arg_layout.len() {
1 => &[false],
2 => &[false, false],
_ => &[false, false, false],
},
ret_layout, ret_layout,
parameter_names, parameter_names,
cost_cycles, cost_cycles,
cost_model: crate::IntrinsicCostModel::Fixed, cost_model: crate::IntrinsicCostModel::Fixed,
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: 0, attempt_cost_cycles: 0,
max_attempts: 0, max_attempts: 0,
mutates_guest_state: false, mutates_guest_state: false,
@@ -1,5 +1,6 @@
//! PBX v1 publication: immutable metadata and shared payload, after bindings //! PBX v0 publication: immutable metadata and shared payload, after bindings
//! and bounded whole-program verification. No public unchecked constructor. //! and bounded whole-program verification. No public unchecked constructor.
use crate::string_identity_index::Identity;
use crate::{LoaderPatchError, VmInitError}; use crate::{LoaderPatchError, VmInitError};
use prometeu_bytecode::isa::core::CoreOpCode as Op; use prometeu_bytecode::isa::core::CoreOpCode as Op;
use prometeu_bytecode::load_limits::{ use prometeu_bytecode::load_limits::{
@@ -107,8 +108,30 @@ impl ValidatedProgramImage {
Some(ImageStringLease { content: content.clone(), domain: self.strings.clone() }) Some(ImageStringLease { content: content.clone(), domain: self.strings.clone() })
} }
pub fn retains_string(&self, content: &Arc<str>) -> bool { pub fn retains_string(&self, content: &Arc<str>) -> bool {
self.lookup_string(content, || {})
}
fn lookup_string(&self, content: &Arc<str>, mut compare: impl FnMut()) -> bool {
let strings = self.strings.lock().unwrap_or_else(std::sync::PoisonError::into_inner); let strings = self.strings.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
strings.records.iter().any(|(weak, _)| std::ptr::eq(weak.as_ptr(), Arc::as_ptr(content))) let target = Identity::of(content);
let mut low = 0;
let mut high = strings.records.len();
while low < high {
let mid = low + (high - low) / 2;
compare();
match record_identity(&strings.records[mid]).cmp(&target) {
std::cmp::Ordering::Less => low = mid + 1,
std::cmp::Ordering::Greater => high = mid,
std::cmp::Ordering::Equal => return true,
}
}
false
}
#[cfg(any(test, feature = "resource-diagnostics"))]
pub(crate) fn lookup_work(&self, content: &Arc<str>) -> (bool, usize) {
let mut work = 0;
let found = self.lookup_string(content, || work += 1);
(found, work)
} }
pub fn constant_content(&self) -> (u64, usize) { pub fn constant_content(&self) -> (u64, usize) {
let mut strings = self.strings.lock().unwrap_or_else(std::sync::PoisonError::into_inner); let mut strings = self.strings.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
@@ -205,6 +228,7 @@ impl ValidatedProgramImage {
}; };
values.push(value); values.push(value);
} }
sort_string_records(&mut records, b)?;
b.refund(Q::TemporaryBytes, constants_capacity); b.refund(Q::TemporaryBytes, constants_capacity);
// Binding no longer needs source declarations. They were validated, // Binding no longer needs source declarations. They were validated,
// used exactly, and can be released before final payload publication. // used exactly, and can be released before final payload publication.
@@ -256,6 +280,48 @@ impl ValidatedProgramImage {
} }
} }
fn record_identity(record: &(Weak<str>, usize)) -> Identity {
Identity { address: record.0.as_ptr() as *const u8 as usize, length: record.1 }
}
// In-place heapsort: every record comparison (four field reads + comparison)
// and swap (two record reads + writes) is precharged. No scratch allocation.
fn sort_string_records(records: &mut [(Weak<str>, usize)], b: &mut LoadBudget) -> Result<(), E> {
fn sift(
records: &mut [(Weak<str>, usize)],
mut root: usize,
end: usize,
b: &mut LoadBudget,
) -> Result<(), E> {
while root < end / 2 {
let mut child = root * 2 + 1;
if child + 1 < end {
b.work(5)?;
if record_identity(&records[child]) < record_identity(&records[child + 1]) {
child += 1;
}
}
b.work(5)?;
if record_identity(&records[root]) >= record_identity(&records[child]) {
break;
}
b.work(4)?;
records.swap(root, child);
root = child;
}
Ok(())
}
for root in (0..records.len() / 2).rev() {
sift(records, root, records.len(), b)?;
}
for end in (1..records.len()).rev() {
b.work(4)?;
records.swap(0, end);
sift(records, 0, end, b)?;
}
Ok(())
}
pub(crate) fn scratch<T>(count: usize, b: &mut LoadBudget) -> Result<Vec<T>, E> { pub(crate) fn scratch<T>(count: usize, b: &mut LoadBudget) -> Result<Vec<T>, E> {
let bytes = count let bytes = count
.checked_mul(size_of::<T>()) .checked_mul(size_of::<T>())
@@ -309,3 +309,65 @@ fn sealed_image_load_probe() {
elapsed elapsed
); );
} }
#[test]
fn string_identity_binary_search_preserves_constant_order_and_lease_indices() {
let constants = (0..1024).map(|i| K::String(format!("owner-{i}"))).collect::<Vec<_>>();
let bytes = fixture(&[1, 0], 0, &constants, None, &[]);
let image = ValidatedProgramImage::load(&bytes, &mut budget(), caps::NONE).unwrap();
for (i, value) in image.constants().iter().enumerate() {
let Value::String(text) = value else { panic!() };
assert_eq!(&**text, format!("owner-{i}"));
let (found, work) = image.lookup_work(text);
assert!(found && work <= 11, "binary work={work}");
assert!(Arc::ptr_eq(text, image.lease_constant(i).unwrap().content()));
assert!(!image.lookup_work(&Arc::from(text.as_ref())).0);
}
let lease = image.lease_constant(1023).unwrap();
drop(image);
assert_eq!(lease.retained_content(), (10, 1));
let domain = lease.domain.lock().unwrap();
assert_eq!(record_identity(&domain.records[0]), Identity::of(lease.content()));
}
#[test]
fn string_heap_sort_precharges_each_comparison_and_swap_without_scratch() {
let owners = (0..16).map(|_| Arc::<str>::from("same")).collect::<Vec<_>>();
let source = owners.iter().rev().map(|v| (Arc::downgrade(v), v.len())).collect::<Vec<_>>();
let mut full = budget();
let mut records = source.clone();
sort_string_records(&mut records, &mut full).unwrap();
assert!(records.windows(2).all(|v| record_identity(&v[0]) < record_identity(&v[1])));
assert_eq!(full.used(Q::TemporaryBytes), 0);
for limit in 0..full.used(Q::Work) {
let mut b = LoadBudget::new(LoadLimits { work: limit, ..Default::default() });
let mut records = source.clone();
let error = sort_string_records(&mut records, &mut b).unwrap_err();
assert_eq!(error.resource, Some(Q::Work));
assert!(b.used(Q::Work) <= limit);
assert_eq!(b.used(Q::TemporaryBytes), 0);
// Failed sort still owns exactly the original Weak identities.
records.sort_unstable_by_key(record_identity);
let mut expected = source.clone();
expected.sort_unstable_by_key(record_identity);
assert!(records.iter().zip(&expected).all(|(a, b)| Weak::ptr_eq(&a.0, &b.0)));
}
}
#[test]
fn string_sort_exhaustion_rolls_back_sealed_load_domains() {
let constants = vec![K::String("same".into()); 2];
let bytes = fixture(&[1, 0], 0, &constants, None, &[]);
let mut full = budget();
ValidatedProgramImage::load(&bytes, &mut full, caps::NONE).unwrap();
// Post-sort sharing costs 2 + 2 + 1 work units. Sorting two records
// costs 9 or 13 units, depending on their addresses. Removing another
// five leaves at most eight: failure is inside sorting for either order.
let mut b =
LoadBudget::new(LoadLimits { work: full.used(Q::Work) - 5 - 5, ..Default::default() });
let error = ValidatedProgramImage::load(&bytes, &mut b, caps::NONE).unwrap_err();
assert!(matches!(error,VmInitError::RegistryLoadFailed(e) if e.resource==Some(Q::Work)));
for resource in DOMAINS {
assert_eq!(b.used(resource), 0, "{resource:?}");
}
}
@@ -21,6 +21,13 @@ const fn entry(
name, name,
version: 1, version: 1,
arg_layout: args, arg_layout: args,
arg_nullable: if id == 0x4000 {
&[false]
} else if rejection {
&[true, false, false]
} else {
&[true]
},
ret_layout: ret, ret_layout: ret,
parameter_names: names, parameter_names: names,
cost_cycles: cost, cost_cycles: cost,
@@ -29,6 +36,8 @@ const fn entry(
} else { } else {
IntrinsicCostModel::Fixed IntrinsicCostModel::Fixed
}, },
cost_arg_index: None,
cost_per_byte: 0,
attempt_cost_cycles: if rejection { 8 } else { 0 }, attempt_cost_cycles: if rejection { 8 } else { 0 },
max_attempts: if rejection { crate::random::MAX_ATTEMPTS } else { 0 }, max_attempts: if rejection { crate::random::MAX_ATTEMPTS } else { 0 },
deterministic: true, deterministic: true,
@@ -485,6 +485,10 @@ fn transfer(
group(&mut working[base..], registry, ty, (instruction.pc as u64) << 32, budget)?; group(&mut working[base..], registry, ty, (instruction.pc as u64) << 32, budget)?;
} }
} }
Op::RefIdentityHash => {
budget.work(1)?;
working[base] = Fact::Carrier(I64);
}
Op::RefCast | Op::RefToTrait => { Op::RefCast | Op::RefToTrait => {
group( group(
&mut working[base..], &mut working[base..],
@@ -503,6 +507,7 @@ fn transfer(
| Op::And | Op::And
| Op::Or | Op::Or
| Op::Not | Op::Not
| Op::RefIdentityEq
| Op::RefIsExact | Op::RefIsExact
| Op::RefImplements | Op::RefImplements
| Op::DynCanUnpack => { | Op::DynCanUnpack => {
@@ -705,6 +710,15 @@ fn check(
} }
} }
Op::ArrayNew => require(working[end - 1], I64, 0, registry, budget)?, Op::ArrayNew => require(working[end - 1], I64, 0, registry, budget)?,
Op::ArrayCopy => {
let array = TypeId(instruction.imm_u32().unwrap());
let base = end - 5;
check_group(&working[base..base + 1], array, registry, budget)?;
check_group(&working[base + 2..base + 3], array, registry, budget)?;
for i in [1, 3, 4] {
require(working[base + i], I64, i, registry, budget)?;
}
}
Op::StructGet | Op::StructSet | Op::ArrayLen | Op::ArrayGet | Op::ArraySet => { Op::StructGet | Op::StructSet | Op::ArrayLen | Op::ArrayGet | Op::ArraySet => {
budget.work(3)?; budget.work(3)?;
let (id, field) = if matches!(instruction.opcode, Op::StructGet | Op::StructSet) { let (id, field) = if matches!(instruction.opcode, Op::StructGet | Op::StructSet) {
@@ -760,6 +774,21 @@ fn check(
return Err(mismatch(1)); return Err(mismatch(1));
} }
} }
Op::RefIdentityHash | Op::RefIdentityEq => {
let (left, right) = if instruction.opcode == Op::RefIdentityHash {
(instruction.imm_u32().unwrap(), None)
} else {
let (a, b) = instruction.imm_u32x2().unwrap();
(a, Some(b))
};
let lw = registry.type_layout(TypeId(left)).unwrap().storage.slots as usize;
let rw = right
.map_or(0, |id| registry.type_layout(TypeId(id)).unwrap().storage.slots as usize);
check_group(&working[end - lw - rw..end - rw], TypeId(left), registry, budget)?;
if let Some(id) = right {
check_group(&working[end - rw..end], TypeId(id), registry, budget)?;
}
}
Op::RefCast | Op::RefToTrait | Op::RefIsExact | Op::RefImplements => { Op::RefCast | Op::RefToTrait | Op::RefIsExact | Op::RefImplements => {
budget.work(1)?; budget.work(1)?;
let ty = TypeId(instruction.imm_u32x2().unwrap().0); let ty = TypeId(instruction.imm_u32x2().unwrap().0);
@@ -772,7 +801,12 @@ fn check(
.unwrap(); .unwrap();
let base = end - native.arg_slots(); let base = end - native.arg_slots();
for (part, abi) in native.arg_layout.iter().enumerate() { for (part, abi) in native.arg_layout.iter().enumerate() {
require(working[base + part], abi_mask(*abi), part, registry, budget)?; let accepted = if native.arg_nullable[part] {
abi_mask(*abi) | NULL
} else {
abi_mask(*abi) & !NULL
};
require(working[base + part], accepted, part, registry, budget)?;
} }
} }
Op::JmpIfTrue | Op::JmpIfFalse | Op::Not => { Op::JmpIfTrue | Op::JmpIfFalse | Op::Not => {
@@ -385,6 +385,10 @@ impl<T> BudgetVec<T> {
let bytes = b.bytes::<T>(capacity, q)?; let bytes = b.bytes::<T>(capacity, q)?;
let mut extra = b.reserve(q, bytes - self.charge.amount)?; let mut extra = b.reserve(q, bytes - self.charge.amount)?;
let temporary = b.reserve(RuntimeResource::TemporaryBytes, self.charge.amount)?; let temporary = b.reserve(RuntimeResource::TemporaryBytes, self.charge.amount)?;
#[cfg(test)]
if FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.replace(false)) {
return Err(b.failure(q, ResourceReason::AllocationFailed, bytes));
}
let mut candidate = Vec::new(); let mut candidate = Vec::new();
candidate candidate
.try_reserve_exact(capacity) .try_reserve_exact(capacity)
@@ -405,6 +409,21 @@ impl<T> BudgetVec<T> {
self.charge.absorb(growth.extra); self.charge.absorb(growth.extra);
} }
} }
/// Rebuild a table with all fallible capacity/coexistence admission completed
/// before moving any old entry. The relocation callback must not allocate.
pub(crate) fn rebuild_with(
&mut self,
len: usize,
vacant: impl FnMut() -> T,
relocate: impl FnOnce(&mut [T], &mut [T]),
) -> Result<(), ResourceFailure> {
let mut growth = self.prepare_capacity(len.max(self.data.capacity()))?;
growth.candidate.resize_with(len, vacant);
relocate(&mut self.data, &mut growth.candidate);
self.data = growth.candidate;
self.charge.absorb(growth.extra);
Ok(())
}
pub fn reserve(&mut self, additional: usize) -> Result<(), ResourceFailure> { pub fn reserve(&mut self, additional: usize) -> Result<(), ResourceFailure> {
let growth = self.prepare_growth(additional)?; let growth = self.prepare_growth(additional)?;
self.commit_growth(growth); self.commit_growth(growth);
@@ -445,9 +464,6 @@ impl<T> BudgetVec<T> {
pub fn clear(&mut self) { pub fn clear(&mut self) {
self.data.clear(); self.data.clear();
} }
pub(crate) fn retain(&mut self, keep: impl FnMut(&T) -> bool) {
self.data.retain(keep);
}
pub(crate) fn extend_from_slice(&mut self, values: &[T]) -> Result<(), ResourceFailure> pub(crate) fn extend_from_slice(&mut self, values: &[T]) -> Result<(), ResourceFailure>
where where
T: Clone, T: Clone,
@@ -509,3 +525,8 @@ pub(crate) struct VecGrowth<T> {
_temporary: Reservation, _temporary: Reservation,
old_capacity: usize, old_capacity: usize,
} }
#[cfg(test)]
thread_local! {
pub(crate) static FAIL_NEXT_VEC_ALLOCATION: std::cell::Cell<bool> = const { std::cell::Cell::new(false) };
}
@@ -87,3 +87,39 @@ mod tests {
assert_eq!(h.max_ns, 99); assert_eq!(h.max_ns, 99);
} }
} }
/// Non-owning storage observations for engineering qualification. These values
/// never retain a guest object or a native string owner and cannot write storage.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum StorageValue {
Null,
Int32(i32),
Int64(i64),
FloatBits(u64),
Boolean(bool),
Reference(prometeu_bytecode::HeapRef),
StringIdentity { address: usize, bytes: usize },
}
impl StorageValue {
pub(crate) fn observe(value: &prometeu_bytecode::Value) -> Self {
use prometeu_bytecode::Value;
match value {
Value::Null => Self::Null,
Value::Int32(n) => Self::Int32(*n),
Value::Int64(n) => Self::Int64(*n),
Value::Float(n) => Self::FloatBits(n.to_bits()),
Value::Boolean(n) => Self::Boolean(*n),
Value::HeapRef(r) => Self::Reference(*r),
Value::String(s) => Self::StringIdentity {
address: std::sync::Arc::as_ptr(s) as *const () as usize,
bytes: s.len(),
},
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct StorageObject {
pub reference: prometeu_bytecode::HeapRef,
pub type_id: prometeu_bytecode::registry::TypeId,
pub groups: Vec<Vec<StorageValue>>,
}
@@ -0,0 +1,123 @@
//! Private allocation identity index. Values retain Weak ownership, never content.
use crate::resource_budget::{
BudgetVec, ResourceFailure, ResourceReason, RuntimeBudget, RuntimeResource as Q,
};
use std::sync::Arc;
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub(crate) struct Identity {
pub(crate) address: usize,
pub(crate) length: usize,
}
impl Identity {
pub(crate) fn of(content: &Arc<str>) -> Self {
Self { address: Arc::as_ptr(content) as *const u8 as usize, length: content.len() }
}
fn hash(self) -> usize {
// Mix aligned addresses and lengths; collisions still compare both fields.
let mut n = (self.address as u64) ^ (self.length as u64).wrapping_mul(0x9e3779b97f4a7c15);
n = (n ^ (n >> 30)).wrapping_mul(0xbf58476d1ce4e5b9);
n = (n ^ (n >> 27)).wrapping_mul(0x94d049bb133111eb);
(n ^ (n >> 31)) as usize
}
}
#[derive(Debug)]
enum Bucket<T> {
Empty,
Tombstone,
Occupied(Identity, T),
}
#[derive(Debug)]
pub(crate) struct IdentityIndex<T> {
buckets: BudgetVec<Bucket<T>>,
live: usize,
tombstones: usize,
}
impl<T> IdentityIndex<T> {
pub(crate) fn new(budget: &RuntimeBudget) -> Self {
Self { buckets: BudgetVec::new(budget, Q::AuxiliaryBytes), live: 0, tombstones: 0 }
}
pub(crate) fn len(&self) -> usize {
self.live
}
pub(crate) fn contains(&self, key: Identity, mut probe: impl FnMut()) -> bool {
let size = self.buckets.len();
if size == 0 {
return false;
}
let mut at = key.hash() & (size - 1);
loop {
probe();
match &self.buckets[at] {
Bucket::Empty => return false,
Bucket::Occupied(found, _) if *found == key => return true,
_ => at = (at + 1) & (size - 1),
}
}
}
pub(crate) fn prepare(&mut self, budget: &RuntimeBudget) -> Result<(), ResourceFailure> {
let size = self.buckets.len();
let next = self.live + 1;
let capacity = if size == 0 {
8
} else if next > size / 2 {
size.checked_mul(2).ok_or_else(|| {
budget.failure(Q::AuxiliaryBytes, ResourceReason::SizeOverflow, u64::MAX)
})?
} else if next + self.tombstones > size / 2 {
size
} else {
return Ok(());
};
self.buckets.rebuild_with(
capacity,
|| Bucket::Empty,
|old, new| {
for slot in old {
if let Bucket::Occupied(key, value) = std::mem::replace(slot, Bucket::Empty) {
Self::place(new, key, value);
}
}
},
)?;
self.tombstones = 0;
Ok(())
}
fn place(buckets: &mut [Bucket<T>], key: Identity, value: T) -> bool {
let mut at = key.hash() & (buckets.len() - 1);
loop {
match &buckets[at] {
Bucket::Empty | Bucket::Tombstone => {
let reused = matches!(buckets[at], Bucket::Tombstone);
buckets[at] = Bucket::Occupied(key, value);
return reused;
}
Bucket::Occupied(found, _) => {
assert_ne!(*found, key, "duplicate allocation identity")
}
}
at = (at + 1) & (buckets.len() - 1);
}
}
pub(crate) fn insert(&mut self, key: Identity, value: T) {
assert!(self.live < self.buckets.len() / 2, "prepare must precede publication");
if Self::place(&mut self.buckets, key, value) {
self.tombstones -= 1;
}
self.live += 1;
}
pub(crate) fn retain(&mut self, mut keep: impl FnMut(&T) -> bool) {
for slot in &mut *self.buckets {
if matches!(slot, Bucket::Occupied(_, value) if !keep(value)) {
// Erase the key before releasing the Weak/reservations. Its
// allocation address cannot be recycled while Weak survives.
let removed = std::mem::replace(slot, Bucket::Tombstone);
self.live -= 1;
self.tombstones += 1;
drop(removed);
}
}
}
}
#[cfg(test)]
mod tests;
@@ -0,0 +1,72 @@
use super::*;
#[test]
fn collision_chains_distinguish_full_identity_and_survive_tombstones() {
let b = RuntimeBudget::default();
let mut table = IdentityIndex::new(&b);
let keys: Vec<_> = (0..1000)
.map(|address| Identity { address, length: 4 })
.filter(|k| k.hash() & 7 == 0)
.take(4)
.collect();
for (i, key) in keys.iter().enumerate() {
table.prepare(&b).unwrap();
table.insert(*key, i);
}
for key in &keys {
assert!(table.contains(*key, || {}));
}
table.retain(|i| *i != 1);
assert!(!table.contains(keys[1], || {}));
assert!(table.contains(keys[3], || {}));
assert!(!table.contains(Identity { length: 5, ..keys[3] }, || {}));
table.prepare(&b).unwrap(); // Rebuild removes tombstones at half occupancy.
table.insert(keys[1], 1);
assert_eq!(table.len(), 4);
assert_eq!(table.tombstones, 0);
}
#[test]
fn tombstone_reuse_keeps_collision_successors_and_length_is_part_of_key() {
let b = RuntimeBudget::default();
let mut t = IdentityIndex::new(&b);
let key = Identity { address: 16, length: 0 };
t.prepare(&b).unwrap();
t.insert(key, 1);
t.prepare(&b).unwrap();
t.insert(Identity { length: 1, ..key }, 2);
t.retain(|v| *v != 1);
assert_eq!(t.tombstones, 1);
t.prepare(&b).unwrap();
t.insert(key, 3);
assert_eq!(t.tombstones, 0);
assert!(t.contains(key, || {}));
assert!(t.contains(Identity { length: 1, ..key }, || {}));
}
#[test]
fn controlled_probe_counts_and_half_load_survive_growth_and_rebuild() {
let b = RuntimeBudget::default();
let mut table = IdentityIndex::new(&b);
for n in 0..1024 {
table.prepare(&b).unwrap();
table.insert(Identity { address: n * 16, length: n % 7 }, n);
assert!(table.live * 2 <= table.buckets.len());
}
let mut max_probes = 0;
for n in 0..2048 {
let mut probes = 0;
assert_eq!(
table.contains(Identity { address: n * 16, length: n % 7 }, || probes += 1),
n < 1024
);
max_probes = max_probes.max(probes);
}
// A reproducible synthetic-key bound for this fixture, not a hash-table
// worst-case claim or a constraint on allocator-selected real addresses.
assert!(max_probes <= 32, "fixture probes={max_probes}");
table.retain(|n| n % 2 == 0);
table.prepare(&b).unwrap();
for n in 0..1024 {
assert_eq!(table.contains(Identity { address: n * 16, length: n % 7 }, || {}), n % 2 == 0);
}
}
+23 -11
View File
@@ -4,6 +4,7 @@
use crate::resource_budget::{ use crate::resource_budget::{
BudgetVec, Reservation, ResourceFailure, ResourceReason, RuntimeBudget, RuntimeResource as Q, BudgetVec, Reservation, ResourceFailure, ResourceReason, RuntimeBudget, RuntimeResource as Q,
}; };
use crate::string_identity_index::{Identity, IdentityIndex};
use prometeu_bytecode::Value; use prometeu_bytecode::Value;
use std::sync::{Arc, Mutex, MutexGuard, Weak}; use std::sync::{Arc, Mutex, MutexGuard, Weak};
@@ -17,7 +18,7 @@ struct Entry {
#[derive(Debug)] #[derive(Debug)]
struct Domain { struct Domain {
budget: RuntimeBudget, budget: RuntimeBudget,
entries: BudgetVec<Entry>, entries: IdentityIndex<Entry>,
_state: Reservation, _state: Reservation,
} }
@@ -49,7 +50,7 @@ impl StringLedger {
Q::AuxiliaryBytes, Q::AuxiliaryBytes,
budget.bytes::<Mutex<Domain>>(1, Q::AuxiliaryBytes)? + 2 * size_of::<usize>() as u64, budget.bytes::<Mutex<Domain>>(1, Q::AuxiliaryBytes)? + 2 * size_of::<usize>() as u64,
)?; )?;
let entries = BudgetVec::new(&budget, Q::AuxiliaryBytes); let entries = IdentityIndex::new(&budget);
Ok(Self(Arc::new(Mutex::new(Domain { budget, entries, _state: state })))) Ok(Self(Arc::new(Mutex::new(Domain { budget, entries, _state: state }))))
} }
fn domain(&self) -> MutexGuard<'_, Domain> { fn domain(&self) -> MutexGuard<'_, Domain> {
@@ -57,7 +58,7 @@ impl StringLedger {
} }
/// Safepoint reconciliation is bounded by the accepted allocation count. /// Safepoint reconciliation is bounded by the accepted allocation count.
/// Retain compacts in place; unused capacity remains charged until drop. /// Dead buckets become tombstones; retained capacity stays charged until drop.
pub(crate) fn reconcile(&self) -> usize { pub(crate) fn reconcile(&self) -> usize {
self.domain().reconcile() self.domain().reconcile()
} }
@@ -65,9 +66,19 @@ impl StringLedger {
self.domain().entries.len() self.domain().entries.len()
} }
pub(crate) fn retains(&self, content: &Arc<str>) -> bool { pub(crate) fn retains(&self, content: &Arc<str>) -> bool {
let address = Arc::as_ptr(content); self.lookup(content, || {})
self.domain().entries.iter().any(|entry| std::ptr::eq(entry.content.as_ptr(), address))
} }
fn lookup(&self, content: &Arc<str>, mut probe: impl FnMut()) -> bool {
self.domain().entries.contains(Identity::of(content), &mut probe)
}
#[cfg(any(test, feature = "resource-diagnostics"))]
pub(crate) fn lookup_work(&self, content: &Arc<str>) -> (bool, usize) {
let mut work = 0;
let found = self.lookup(content, || work += 1);
(found, work)
}
pub(crate) fn lease(&self, content: &Arc<str>) -> Option<StringLease> { pub(crate) fn lease(&self, content: &Arc<str>) -> Option<StringLease> {
self.retains(content) self.retains(content)
.then(|| StringLease { content: content.clone(), domain: self.clone() }) .then(|| StringLease { content: content.clone(), domain: self.clone() })
@@ -239,23 +250,24 @@ impl Domain {
} }
Err(failure) => return Err(failure), Err(failure) => return Err(failure),
}; };
if let Err(failure) = self.entries.reserve(1) { if let Err(failure) = self.entries.prepare(&self.budget) {
if failure.reason != ResourceReason::Limit { if failure.reason != ResourceReason::Limit {
return Err(failure); return Err(failure);
} }
self.reconcile(); self.reconcile();
self.entries.reserve(1)?; self.entries.prepare(&self.budget)?;
} }
Ok(charges) Ok(charges)
} }
fn publish(&mut self, content: &Arc<str>, charges: Charges) { fn publish(&mut self, content: &Arc<str>, charges: Charges) {
self.entries self.entries.insert(
.push(Entry { Identity::of(content),
Entry {
content: Arc::downgrade(content), content: Arc::downgrade(content),
_bytes: charges.bytes, _bytes: charges.bytes,
_allocation: charges.allocation, _allocation: charges.allocation,
}) },
.expect("prepare reserved one ledger record before materialization"); );
} }
} }
@@ -259,3 +259,95 @@ fn formatting_refuses_result_and_temporary_limits_with_no_content_publication()
assert_eq!(budget.used(Q::StringAllocations), 0); assert_eq!(budget.used(Q::StringAllocations), 0);
assert_eq!(ledger.tracked_allocations(), 0); assert_eq!(ledger.tracked_allocations(), 0);
} }
#[test]
fn index_growth_refuses_auxiliary_temporary_and_allocator_before_materialization() {
for failure in [0, 1, 2] {
let (budget, ledger) = fixture(&[]);
let kept = (0..4).map(|_| ledger.import("kept").unwrap()).collect::<Vec<_>>();
let auxiliary = budget.used(Q::AuxiliaryBytes);
match failure {
0 => budget.test_lower_limit(Q::AuxiliaryBytes, auxiliary),
1 => budget.test_lower_limit(Q::TemporaryBytes, 0),
_ => crate::resource_budget::FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.set(true)),
}
let mut called = false;
let result = ledger.build(4, &mut |w| {
called = true;
w.write_str("next")
});
let Err(MaterializeError::Resource(error)) = result else {
panic!("expected prepublication refusal")
};
assert!(!called);
assert_eq!(
error.resource,
if failure == 1 { Q::TemporaryBytes } else { Q::AuxiliaryBytes }
);
assert_eq!(
error.reason,
if failure == 2 { ResourceReason::AllocationFailed } else { ResourceReason::Limit }
);
assert_eq!(ledger.tracked_allocations(), 4);
assert_eq!(budget.used(Q::StringBytes), 16);
assert_eq!(budget.used(Q::StringAllocations), 4);
assert_eq!(budget.used(Q::AuxiliaryBytes), auxiliary);
assert_eq!(budget.used(Q::TemporaryBytes), 0);
for value in &kept {
assert!(ledger.retains(value));
}
}
}
#[test]
fn index_initial_allocation_failure_and_rebuild_failure_refund_reservations() {
let (budget, ledger) = fixture(&[]);
let initial = budget.used(Q::AuxiliaryBytes);
crate::resource_budget::FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.set(true));
assert_eq!(ledger.import("x").unwrap_err().reason, ResourceReason::AllocationFailed);
assert_eq!(ledger.tracked_allocations(), 0);
assert_eq!(budget.used(Q::AuxiliaryBytes), initial);
assert_eq!(budget.used(Q::StringBytes), 0);
assert_eq!(budget.used(Q::StringAllocations), 0);
let mut kept = (0..4).map(|_| ledger.import("same").unwrap()).collect::<Vec<_>>();
let old = Arc::downgrade(&kept.pop().unwrap());
assert_eq!(ledger.reconcile(), 1);
let auxiliary = budget.used(Q::AuxiliaryBytes);
budget.test_lower_limit(Q::TemporaryBytes, 0);
assert_eq!(ledger.import("new").unwrap_err().resource, Q::TemporaryBytes);
assert_eq!(ledger.tracked_allocations(), 3);
assert_eq!(budget.used(Q::AuxiliaryBytes), auxiliary);
assert_eq!(budget.used(Q::TemporaryBytes), 0);
assert_eq!(budget.used(Q::StringBytes), 12);
for value in &kept {
assert_eq!(Arc::strong_count(value), 1);
assert!(ledger.retains(value));
}
assert!(old.upgrade().is_none());
// Rebuild under a restored test budget; no production ceiling changes.
budget.test_lower_limit(Q::TemporaryBytes, 64 * 1024 * 1024);
let newer = ledger.import("same").unwrap();
assert!(ledger.retains(&newer));
assert!(!ledger.retains(&Arc::from("same")));
assert!(ledger.lookup_work(&newer).0);
}
#[test]
fn tombstone_rebuild_allocator_refusal_preserves_live_keys_and_charges() {
let (budget, ledger) = fixture(&[]);
let mut owners = (0..4).map(|_| ledger.import("same").unwrap()).collect::<Vec<_>>();
owners.pop();
assert_eq!(ledger.reconcile(), 1);
let auxiliary = budget.used(Q::AuxiliaryBytes);
crate::resource_budget::FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.set(true));
assert_eq!(ledger.import("new").unwrap_err().reason, ResourceReason::AllocationFailed);
assert_eq!(ledger.tracked_allocations(), 3);
assert_eq!(budget.used(Q::StringBytes), 12);
assert_eq!(budget.used(Q::StringAllocations), 3);
assert_eq!(budget.used(Q::AuxiliaryBytes), auxiliary);
assert_eq!(budget.used(Q::TemporaryBytes), 0);
assert!(owners.iter().all(|v| ledger.retains(v)));
let new = ledger.import("new").unwrap();
assert!(ledger.retains(&new));
assert_eq!(budget.used(Q::AuxiliaryBytes), auxiliary);
}
@@ -221,6 +221,24 @@ pub(crate) fn unpack_dynamic(
// All reference operations validate complete source values before querying the // All reference operations validate complete source values before querying the
// underlying concrete object. Invalid handles/pairs never become false or null. // underlying concrete object. Invalid handles/pairs never become false or null.
pub(crate) fn identity_reference(
registry: &Registry,
heap: &Heap,
source: TypeId,
values: &[Value],
) -> Result<Option<HeapRef>, ValueError> {
let kind = registry.declarations().types.get(source.0 as usize).ok_or(ValueError::Type)?;
if !kind.kind.is_identity_reference() {
return Err(ValueError::Type);
}
validate(registry, heap, source, values)?;
match values.first() {
Some(Value::Null) => Ok(None),
Some(Value::HeapRef(reference)) => Ok(Some(*reference)),
_ => Err(ValueError::Type),
}
}
fn reference( fn reference(
registry: &Registry, registry: &Registry,
heap: &Heap, heap: &Heap,
@@ -1,12 +1,20 @@
use crate::numeric; use crate::numeric;
use crate::resource_budget::BudgetVec; use crate::resource_budget::BudgetVec;
mod array_copy;
#[cfg(test)]
mod array_copy_tests;
mod callable; mod callable;
#[cfg(test)]
#[path = "../tests/support/collections.rs"]
mod collection_test_support;
mod collections; mod collections;
#[cfg(test)] #[cfg(test)]
mod collections_tests; mod collections_tests;
mod control; mod control;
mod dynamic; mod dynamic;
mod gc; mod gc;
mod identity;
mod intrinsic;
mod lifecycle; mod lifecycle;
mod loader; mod loader;
mod native_arrays; mod native_arrays;
@@ -327,8 +335,10 @@ impl VirtualMachine {
} }
let charged_math = if opcode == OpCode::Intrinsic { let charged_math = if opcode == OpCode::Intrinsic {
lookup_intrinsic_by_id(instr.imm_u32().unwrap()) lookup_intrinsic_by_id(instr.imm_u32().unwrap()).filter(|m| {
.filter(|m| m.owner == "vec2" || m.owner == "mathf" || m.owner.starts_with("math.")) matches!(m.implementation, crate::IntrinsicImplementation::Host(_))
&& (m.owner == "vec2" || m.owner == "mathf" || m.owner.starts_with("math."))
})
} else { } else {
None None
}; };
@@ -952,6 +962,12 @@ impl VirtualMachine {
) )
})?; })?;
if matches!(
intrinsic.implementation,
crate::IntrinsicImplementation::BorrowedInline(_)
) {
return self.execute_inline_intrinsic(intrinsic, pc_at_intrinsic);
}
if matches!(intrinsic.implementation, crate::IntrinsicImplementation::VmOwned) { if matches!(intrinsic.implementation, crate::IntrinsicImplementation::VmOwned) {
return self.execute_random(intrinsic, pc_at_intrinsic); return self.execute_random(intrinsic, pc_at_intrinsic);
} }
@@ -1083,6 +1099,18 @@ impl VirtualMachine {
let (source, target) = instr.imm_u32x2().unwrap(); let (source, target) = instr.imm_u32x2().unwrap();
return self.execute_reference(opcode, source, target, start_pc as u32); return self.execute_reference(opcode, source, target, start_pc as u32);
} }
OpCode::RefIdentityHash | OpCode::RefIdentityEq => {
let (left, right) = if opcode == OpCode::RefIdentityHash {
(instr.imm_u32().unwrap(), None)
} else {
let (left, right) = instr.imm_u32x2().unwrap();
(left, Some(right))
};
return self.execute_identity(opcode, left, right, start_pc as u32);
}
OpCode::ArrayCopy => {
return self.execute_array_copy(instr.imm_u32().unwrap(), start_pc as u32);
}
OpCode::StructNew OpCode::StructNew
| OpCode::StructGet | OpCode::StructGet
| OpCode::StructSet | OpCode::StructSet
@@ -2485,7 +2513,7 @@ mod tests {
let mut vm = VirtualMachine::default(); let mut vm = VirtualMachine::default();
let mut header = vec![0u8; 32]; let mut header = vec![0u8; 32];
header[0..4].copy_from_slice(&prometeu_bytecode::PBX_MAGIC); header[0..4].copy_from_slice(&prometeu_bytecode::PBX_MAGIC);
header[4..6].copy_from_slice(&0u16.to_le_bytes()); // retired version header[4..6].copy_from_slice(&1u16.to_le_bytes()); // unsupported version
let res = vm.initialize(header); let res = vm.initialize(header);
assert!( assert!(
@@ -2494,11 +2522,11 @@ mod tests {
} }
#[test] #[test]
fn test_loader_hardening_malformed_pbx_v1() { fn test_loader_hardening_malformed_pbx_v0() {
let mut vm = VirtualMachine::default(); let mut vm = VirtualMachine::default();
let mut header = vec![0u8; 32]; let mut header = vec![0u8; 32];
header[0..4].copy_from_slice(&prometeu_bytecode::PBX_MAGIC); header[0..4].copy_from_slice(&prometeu_bytecode::PBX_MAGIC);
header[4..6].copy_from_slice(&1u16.to_le_bytes()); header[4..6].copy_from_slice(&0u16.to_le_bytes());
header[8..12].copy_from_slice(&1u32.to_le_bytes()); // missing directory header[8..12].copy_from_slice(&1u32.to_le_bytes()); // missing directory
let res = vm.initialize(header); let res = vm.initialize(header);
@@ -3890,3 +3918,10 @@ mod tests {
#[cfg(test)] #[cfg(test)]
#[path = "virtual_machine/callable_tests.rs"] #[path = "virtual_machine/callable_tests.rs"]
mod callable_tests; mod callable_tests;
#[cfg(feature = "resource-diagnostics")]
pub mod string_provenance_qualification;
#[cfg(test)]
#[path = "virtual_machine/identity_tests.rs"]
mod identity_tests;
@@ -0,0 +1,110 @@
//! Complete-group snapshot copy: charge, validate, prepare, then commit.
use super::*;
use crate::value_contract::{ValueError, ValueGroup};
use prometeu_bytecode::registry::TypeId;
fn planned_cost(count: u32, width: u64) -> Option<u64> {
u64::from(count).checked_mul(2)?.checked_mul(width)?.checked_add(4)
}
impl VirtualMachine {
pub(super) fn execute_array_copy(
&mut self,
array_id: u32,
pc: u32,
) -> Result<(), LogicalFrameEndingReason> {
let opcode = OpCode::ArrayCopy;
self.cycles += opcode.cycles();
let image = self.validated_image.clone().expect("validated Array TypeId");
let registry = image.registry();
let array_ty = TypeId(array_id);
let element = registry.type_layout(array_ty).unwrap().array.unwrap().element_type_id;
let width = registry.type_layout(element).unwrap().storage.slots;
if self.temporary_slots() < 5 {
return Err(self.trap(
TRAP_STACK_UNDERFLOW,
opcode as u16,
"Incomplete array copy operands".into(),
pc,
));
}
let base = self.operand_stack.len() - 5;
// Inspect count only to select cost, without changing ordinary fault order.
if let Value::Int64(n) = self.operand_stack[base + 4]
&& let Ok(n) = u32::try_from(n)
{
let cost = planned_cost(n, u64::from(width))
.ok_or_else(|| self.collection_size_trap(opcode, pc, u64::MAX))?;
self.cycles += cost - opcode.cycles();
}
let receiver = |offset: usize| {
let r = match self.operand_stack[base + offset] {
Value::HeapRef(r) => r,
Value::Null => return Err(self.value_trap(ValueError::NullReference, opcode, pc)),
_ => return Err(self.value_trap(ValueError::Type, opcode, pc)),
};
if self.heap.concrete_type(r, registry).map_err(|e| self.value_trap(e, opcode, pc))?
!= array_ty
{
return Err(self.value_trap(ValueError::Type, opcode, pc));
}
Ok(r)
};
let source = receiver(0)?;
let target = receiver(2)?;
let integer = |offset: usize| match self.operand_stack[base + offset] {
Value::Int64(n) => Ok(n),
_ => Err(self.value_trap(ValueError::Type, opcode, pc)),
};
let source_start = integer(1)?;
let target_start = integer(3)?;
let count = integer(4)?;
let range = |r, start: i64| {
let len = self
.heap
.typed_array_len(r, registry)
.map_err(|e| self.value_trap(e, opcode, pc))?;
if start < 0 || count < 0 || start > len as i64 || count > len as i64 - start {
return Err(self.value_trap(ValueError::Bounds, opcode, pc));
}
Ok(start as usize)
};
let source_start = range(source, source_start)?;
let target_start = range(target, target_start)?;
let count = count as usize;
// Operands remain roots; every fallible operation precedes shared writes.
let mut prepared =
BudgetVec::<ValueGroup>::new(&self.budget, RuntimeResource::TemporaryBytes);
prepared.reserve(count).map_err(|e| self.resource_trap(e, opcode, pc))?;
for i in 0..count {
let group = self
.heap
.load_typed(registry, source, source_start + i)
.map_err(|e| self.value_trap(e, opcode, pc))?;
prepared.push(group).expect("complete scratch reservation");
}
// Snapshot captured. No allocation, validation, GC, callback or scheduler.
for (i, group) in prepared.iter().enumerate() {
self.heap.commit_typed(target, target_start + i, group.as_slice());
}
self.operand_stack.truncate(base);
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::planned_cost;
#[test]
fn checked_cost_covers_the_full_sum_without_wrap_or_saturation() {
assert_eq!(planned_cost(10, 1), Some(24));
assert_eq!(planned_cost(10, 3), Some(64));
assert_eq!(planned_cost(0, u64::MAX), Some(4));
assert_eq!(planned_cost(u32::MAX, 3), Some(25_769_803_774));
// Synthetic widths exercise defensive overflow, not valid guest layouts.
assert_eq!(planned_cost(1, u64::MAX), None);
assert_eq!(planned_cost(1, u64::MAX / 2), None);
assert_eq!(planned_cost(u32::MAX, u64::MAX), None);
}
}
@@ -0,0 +1,487 @@
use super::collection_test_support as support;
use super::*;
use crate::ValidatedProgramImage;
use prometeu_bytecode::{assemble, registry::TypeId};
use support::Native;
fn machine(instruction: &str, count: usize) -> VirtualMachine {
let mut text = String::new();
for i in 0..count {
text += &format!("GET_GLOBAL {i}\n");
}
text += instruction;
text += "\nHALT";
let mut vm = VirtualMachine::from_image(
support::load(&assemble(&text).unwrap()).unwrap(),
RuntimeLimits::default(),
);
vm.prepare_boot_call().unwrap();
vm
}
fn operands(vm: &mut VirtualMachine, values: &[Value]) {
vm.globals.extend_from_slice(values).unwrap();
for _ in values {
vm.step(&mut Native, &mut HostContext::new(None)).unwrap();
}
}
fn step(vm: &mut VirtualMachine) -> Result<(), LogicalFrameEndingReason> {
vm.step(&mut Native, &mut HostContext::new(None))
}
#[test]
fn raw_copy_preserves_absence_and_box_identity_without_heap_allocations() {
let mut vm = machine("ARRAY_COPY 15", 5);
let registry = vm.validated_image.as_ref().unwrap().registry().clone();
let a = vm.heap.allocate_typed_array(registry.clone(), TypeId(15), 4).unwrap();
let boxed = vm
.heap
.allocate_dynamic_box(registry.clone(), TypeId(5), &[Value::Float(1.), Value::Float(2.)])
.unwrap();
let present = [Value::Int64(5), Value::HeapRef(boxed)];
vm.heap.store_typed(&registry, a, 0, &present).unwrap();
operands(
&mut vm,
&[Value::HeapRef(a), Value::Int64(0), Value::HeapRef(a), Value::Int64(1), Value::Int64(3)],
);
let before = vm.heap.allocation_count();
vm.budget
.test_lower_limit(RuntimeResource::GuestBytes, vm.budget.used(RuntimeResource::GuestBytes));
step(&mut vm).unwrap();
assert_eq!(vm.heap.allocation_count(), before);
assert_eq!(vm.heap.load_typed(&registry, a, 1).unwrap().as_slice(), present);
for i in [2, 3] {
assert_eq!(
vm.heap.load_typed(&registry, a, i).unwrap().as_slice(),
[Value::Null, Value::Null]
);
}
}
fn pair(ty: u32, len: u32) -> (VirtualMachine, HeapRef, HeapRef) {
let mut vm = machine(&format!("ARRAY_COPY {ty}"), 5);
let registry = vm.validated_image.as_ref().unwrap().registry().clone();
let source = vm.heap.allocate_typed_array(registry.clone(), TypeId(ty), len).unwrap();
let target = vm.heap.allocate_typed_array(registry, TypeId(ty), len).unwrap();
(vm, source, target)
}
fn copy_inputs(source: HeapRef, target: HeapRef, count: i64) -> [Value; 5] {
[
Value::HeapRef(source),
Value::Int64(0),
Value::HeapRef(target),
Value::Int64(0),
Value::Int64(count),
]
}
fn refused(vm: &mut VirtualMachine, code: u32, cost: u64) {
let inputs = vm.operand_stack.as_slice().to_vec();
let (pc, cycles, temporary, objects) =
(vm.pc, vm.cycles, vm.budget.used(RuntimeResource::TemporaryBytes), vm.heap.len());
let LogicalFrameEndingReason::Trap(info) = step(vm).unwrap_err() else { panic!() };
assert_eq!((info.code, info.opcode, info.pc), (code, 0xa9, pc as u32));
assert_eq!(vm.cycles - cycles, cost);
assert_eq!(vm.operand_stack.as_slice(), inputs);
assert_eq!(vm.budget.used(RuntimeResource::TemporaryBytes), temporary);
assert_eq!(vm.heap.len(), objects);
}
#[test]
fn calculable_work_is_precharged_on_receivers_carriers_and_ranges_in_fault_order() {
for case in 0..20 {
let (mut vm, source, target) = pair(11, 10);
let registry = vm.validated_image.as_ref().unwrap().registry().clone();
let wrong = vm.heap.allocate_typed_struct(registry.clone(), TypeId(19)).unwrap();
let foreign = support::load(&assemble("HALT").unwrap()).unwrap();
let foreign_ref =
vm.heap.allocate_typed_array(foreign.registry().clone(), TypeId(11), 10).unwrap();
let stale = HeapRef { index: source.index, generation: source.generation + 1 };
let mut inputs = copy_inputs(source, target, 10);
let (offset, value, code) = match case {
0 => (0, Value::Null, TRAP_NULL_REFERENCE),
1 => (2, Value::Null, TRAP_NULL_REFERENCE),
2 => (0, Value::HeapRef(stale), TRAP_INVALID_HANDLE),
3 => (2, Value::HeapRef(stale), TRAP_INVALID_HANDLE),
4 => (0, Value::HeapRef(wrong), TRAP_TYPE),
5 => (2, Value::HeapRef(wrong), TRAP_TYPE),
6 => (0, Value::HeapRef(foreign_ref), TRAP_TYPE),
7 => (2, Value::Int32(0), TRAP_TYPE),
8 => (1, Value::Int32(0), TRAP_TYPE),
9 => (3, Value::Null, TRAP_TYPE),
10 => (1, Value::Int64(-1), TRAP_OOB),
11 => (3, Value::Int64(-1), TRAP_OOB),
12 => (1, Value::Int64(1), TRAP_OOB),
13 => (3, Value::Int64(1), TRAP_OOB),
14 => {
inputs[1] = Value::Int32(0); // Receiver fault precedes offset carrier.
(2, Value::Null, TRAP_NULL_REFERENCE)
}
15 => {
inputs[1] = Value::Int64(-1); // All carriers precede source range.
(3, Value::Int32(0), TRAP_TYPE)
}
16 => (0, Value::Int32(0), TRAP_TYPE),
17 => {
let wrong_array =
vm.heap.allocate_typed_array(registry.clone(), TypeId(22), 10).unwrap();
(2, Value::HeapRef(wrong_array), TRAP_TYPE)
}
18 => (1, Value::Int64(i64::MAX), TRAP_OOB),
_ => (3, Value::Int64(i64::MAX), TRAP_OOB),
};
inputs[offset] = value;
operands(&mut vm, &inputs);
// Quota refusal must not mask earlier faults, even with full cost selected.
vm.budget.test_lower_limit(
RuntimeResource::TemporaryBytes,
vm.budget.used(RuntimeResource::TemporaryBytes),
);
refused(&mut vm, code, 24);
for i in 0..10 {
assert_eq!(
vm.heap.load_typed(&registry, target, i).unwrap().as_slice(),
[Value::Int32(0)]
);
}
}
}
#[test]
fn invalid_count_selects_only_base_without_reordering_ordinary_faults() {
for count in [
Value::Int32(10),
Value::Null,
Value::Int64(-1),
Value::Int64(u32::MAX as i64 + 1),
Value::Int64(i64::MAX),
] {
for bad_receiver in [false, true] {
let (mut vm, source, target) = pair(11, 10);
let mut inputs = copy_inputs(source, target, 0);
inputs[4] = count.clone();
if bad_receiver {
inputs[0] = Value::Null;
}
operands(&mut vm, &inputs);
let code = if bad_receiver {
TRAP_NULL_REFERENCE
} else if matches!(count, Value::Int64(_)) {
TRAP_OOB
} else {
TRAP_TYPE
};
refused(&mut vm, code, 4);
}
}
}
#[test]
fn incomplete_temporary_operands_never_read_frame_locals_for_cost() {
let code = assemble(
"GET_GLOBAL 0\nGET_GLOBAL 1\nGET_GLOBAL 2\nGET_GLOBAL 3\nGET_GLOBAL 4\nARRAY_COPY 11\nHALT",
)
.unwrap();
for available in 0..5 {
let mut declarations = support::declarations(code.len());
declarations.functions[0].local_slots = 5;
let image = ValidatedProgramImage::load(
&support::bytes_with(&code, declarations),
&mut prometeu_bytecode::load_limits::LoadBudget::new(Default::default()),
prometeu_hal::syscalls::caps::NONE,
)
.unwrap();
let mut vm = VirtualMachine::from_image(image, RuntimeLimits::default());
vm.prepare_boot_call().unwrap();
// Internal defense: skip the verified producers, leaving only locals and
// an incomplete temporary group. This is not an admitted bad program.
vm.operand_stack[4] = Value::Int64(10);
vm.operand_stack.extend_from_slice(&vec![Value::Int64(10); available]).unwrap();
vm.pc = 30;
refused(&mut vm, TRAP_STACK_UNDERFLOW, 4);
}
}
#[test]
fn overlap_snapshots_complete_multislot_groups_and_has_no_identity_discount() {
for (source_start, target_start, count) in [(0, 1, 3), (1, 0, 3), (0, 0, 4)] {
for refuse in [false, true] {
let mut vm = machine("ARRAY_COPY 12", 5);
let registry = vm.validated_image.as_ref().unwrap().registry().clone();
let array = vm.heap.allocate_typed_array(registry.clone(), TypeId(12), 4).unwrap();
for i in 0..4 {
vm.heap
.store_typed(
&registry,
array,
i,
&[Value::Float(i as f64), Value::Float(i as f64 + 10.)],
)
.unwrap();
}
let before: Vec<_> = (0..4)
.map(|i| vm.heap.load_typed(&registry, array, i).unwrap().as_slice().to_vec())
.collect();
operands(
&mut vm,
&[
Value::HeapRef(array),
Value::Int64(source_start),
Value::HeapRef(array),
Value::Int64(target_start),
Value::Int64(count),
],
);
let temporary = vm.budget.used(RuntimeResource::TemporaryBytes);
if refuse {
vm.budget.test_lower_limit(RuntimeResource::TemporaryBytes, temporary);
refused(&mut vm, TRAP_RESOURCE_LIMIT, if count == 3 { 16 } else { 20 });
} else {
let cycles = vm.cycles;
step(&mut vm).unwrap();
assert_eq!(vm.cycles - cycles, if count == 3 { 16 } else { 20 });
assert!(vm.operand_stack.is_empty());
}
for i in 0..4 {
let expected =
if !refuse && i >= target_start as usize && i < (target_start + count) as usize
{
&before[(i as i64 - target_start + source_start) as usize]
} else {
&before[i]
};
assert_eq!(vm.heap.load_typed(&registry, array, i).unwrap().as_slice(), expected);
}
assert_eq!(vm.budget.used(RuntimeResource::TemporaryBytes), temporary);
}
}
}
#[test]
fn scratch_quota_and_allocator_refusals_preserve_all_destinations_and_work() {
for allocation_failure in [false, true] {
let (mut vm, source, target) = pair(11, 10);
let registry = vm.validated_image.as_ref().unwrap().registry().clone();
for i in 0..10 {
vm.heap.store_typed(&registry, source, i, &[Value::Int32(i as i32 + 1)]).unwrap();
}
operands(&mut vm, &copy_inputs(source, target, 10));
if allocation_failure {
crate::resource_budget::FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.set(true));
} else {
vm.budget.test_lower_limit(
RuntimeResource::TemporaryBytes,
vm.budget.used(RuntimeResource::TemporaryBytes),
);
}
let (cycles, temporary) = (vm.cycles, vm.budget.used(RuntimeResource::TemporaryBytes));
let LogicalFrameEndingReason::Trap(info) = step(&mut vm).unwrap_err() else { panic!() };
assert_eq!((info.code, info.opcode, info.pc), (TRAP_RESOURCE_LIMIT, 0xa9, 30));
let resource = info.resource.unwrap();
assert_eq!(resource.resource, RuntimeResource::TemporaryBytes);
assert_eq!(
resource.reason,
if allocation_failure {
crate::resource_budget::ResourceReason::AllocationFailed
} else {
crate::resource_budget::ResourceReason::Limit
}
);
assert_eq!(vm.cycles - cycles, 24);
assert_eq!(vm.budget.used(RuntimeResource::TemporaryBytes), temporary);
assert_eq!(vm.operand_stack.as_slice(), copy_inputs(source, target, 10));
for i in 0..10 {
assert_eq!(
vm.heap.load_typed(&registry, target, i).unwrap().as_slice(),
[Value::Int32(0)]
);
}
}
}
#[test]
fn late_invalid_source_group_releases_partial_preparation_without_any_write() {
let (mut vm, source, target) = pair(17, 10);
let registry = vm.validated_image.as_ref().unwrap().registry().clone();
let node = vm.heap.allocate_typed_struct(registry.clone(), TypeId(9)).unwrap();
vm.heap.store_typed(&registry, source, 0, &[Value::HeapRef(node)]).unwrap();
// Internal corruption of the last stored handle exercises a late read fault.
vm.heap.commit_typed(
source,
9,
&[Value::HeapRef(HeapRef { index: node.index, generation: node.generation + 1 })],
);
operands(&mut vm, &copy_inputs(source, target, 10));
refused(&mut vm, TRAP_INVALID_HANDLE, 24);
for i in 0..10 {
assert_eq!(vm.heap.load_typed(&registry, target, i).unwrap().as_slice(), [Value::Null]);
}
}
#[test]
fn empty_copy_validates_both_receivers_and_strict_ranges_without_scratch() {
for case in 0..5 {
let (mut vm, source, target) = pair(11, 2);
let mut inputs = copy_inputs(source, target, 0);
inputs[1] = Value::Int64(2);
inputs[3] = Value::Int64(2);
let code = match case {
0 => None,
1 => {
inputs[0] = Value::Null;
Some(TRAP_NULL_REFERENCE)
}
2 => {
inputs[2] = Value::Null;
Some(TRAP_NULL_REFERENCE)
}
3 => {
inputs[1] = Value::Int64(3);
Some(TRAP_OOB)
}
_ => {
inputs[3] = Value::Int64(-1);
Some(TRAP_OOB)
}
};
operands(&mut vm, &inputs);
vm.budget.test_lower_limit(
RuntimeResource::TemporaryBytes,
vm.budget.used(RuntimeResource::TemporaryBytes),
);
if let Some(code) = code {
refused(&mut vm, code, 4);
} else {
let cycles = vm.cycles;
step(&mut vm).unwrap();
assert_eq!(vm.cycles - cycles, 4);
assert!(vm.operand_stack.is_empty());
}
}
}
#[test]
fn zero_budget_and_positive_overshoot_preserve_trap_priority_and_step_count() {
for fail in [false, true] {
let (mut vm, source, target) = pair(11, 10);
operands(&mut vm, &copy_inputs(source, target, 10));
if fail {
vm.budget.test_lower_limit(
RuntimeResource::TemporaryBytes,
vm.budget.used(RuntimeResource::TemporaryBytes),
);
}
let (pc, cycles, inputs) = (vm.pc, vm.cycles, vm.operand_stack.as_slice().to_vec());
let report = vm.run_budget(0, &mut Native, &mut HostContext::new(None)).unwrap();
assert_eq!((report.cycles_used, report.steps_executed), (0, 0));
assert_eq!((vm.pc, vm.cycles), (pc, cycles));
assert_eq!(vm.operand_stack.as_slice(), inputs);
let report = vm.run_budget(1, &mut Native, &mut HostContext::new(None)).unwrap();
assert_eq!((report.cycles_used, report.steps_executed), (24, if fail { 0 } else { 1 }));
if fail {
let LogicalFrameEndingReason::Trap(info) = report.reason else { panic!() };
assert_eq!((info.code, info.opcode, info.pc), (TRAP_RESOURCE_LIMIT, 0xa9, pc as u32));
assert_eq!(vm.operand_stack.as_slice(), inputs);
} else {
assert_eq!(report.reason, LogicalFrameEndingReason::BudgetExhausted);
}
}
}
#[test]
fn copied_groups_preserve_owners_and_references_through_collection_without_guest_growth() {
for ty in [11, 12, 13, 14, 15, 16, 17] {
let (mut vm, source, target) = pair(ty, 1);
let registry = vm.validated_image.as_ref().unwrap().registry().clone();
let node = vm.heap.allocate_typed_struct(registry.clone(), TypeId(9)).unwrap();
vm.push(Value::String(std::sync::Arc::from("copy owner"))).unwrap();
let string = vm.pop().unwrap();
let group = match ty {
11 => vec![Value::Int32(7)],
12 => vec![Value::Float(1.), Value::Float(2.)],
13 => vec![Value::Int64(1), Value::Int64(2), Value::Int64(3)],
14 => vec![string.clone()],
15 => vec![Value::Int64(4), string.clone()],
16 => vec![Value::HeapRef(node), Value::Int64(0)],
_ => vec![Value::HeapRef(node)],
};
vm.heap.store_typed(&registry, source, 0, &group).unwrap();
operands(&mut vm, &copy_inputs(source, target, 1));
let objects = vm.heap.allocation_count();
let guest = vm.budget.used(RuntimeResource::GuestBytes);
for q in [
RuntimeResource::GuestBytes,
RuntimeResource::HandleSlots,
RuntimeResource::AuxiliaryBytes,
] {
vm.budget.test_lower_limit(q, vm.budget.used(q));
}
let cycles = vm.cycles;
step(&mut vm).unwrap();
assert_eq!(
vm.cycles - cycles,
match ty {
12 | 15 | 16 => 8,
13 => 10,
_ => 6,
}
);
assert_eq!(vm.heap.allocation_count(), objects);
assert_eq!(vm.budget.used(RuntimeResource::GuestBytes), guest);
assert_eq!(vm.heap.load_typed(&registry, target, 0).unwrap().as_slice(), group);
drop(group);
drop(string);
vm.globals.clear();
vm.globals.push(Value::HeapRef(target)).unwrap();
vm.gc_alloc_threshold = 1;
vm.handle_safepoint().unwrap();
assert!(!vm.heap.is_valid(source));
assert!(vm.heap.is_valid(target));
if [16, 17].contains(&ty) {
assert!(vm.heap.is_valid(node));
}
if [14, 15].contains(&ty) {
assert_eq!(vm.budget.used(RuntimeResource::StringAllocations), 1);
}
// Remove all roots and collect using the actual session root traversal.
vm.globals.clear();
vm.last_gc_live_count = 0;
vm.handle_safepoint().unwrap();
assert_eq!(vm.heap.len(), 0);
assert_eq!(vm.budget.used(RuntimeResource::StringAllocations), 0);
}
}
#[test]
fn refused_copy_keeps_source_and_previous_target_native_owners_rooted() {
let (mut vm, source, target) = pair(14, 1);
let registry = vm.validated_image.as_ref().unwrap().registry().clone();
for (array, text) in [(source, "candidate owner"), (target, "previous owner")] {
vm.push(Value::String(std::sync::Arc::from(text))).unwrap();
let value = vm.pop().unwrap();
vm.heap.store_typed(&registry, array, 0, &[value]).unwrap();
}
operands(&mut vm, &copy_inputs(source, target, 1));
let temporary = vm.budget.used(RuntimeResource::TemporaryBytes);
vm.budget.test_lower_limit(RuntimeResource::TemporaryBytes, temporary);
refused(&mut vm, TRAP_RESOURCE_LIMIT, 6);
assert_eq!(
vm.heap.load_typed(&registry, source, 0).unwrap().as_slice(),
[Value::String(std::sync::Arc::from("candidate owner"))]
);
assert_eq!(
vm.heap.load_typed(&registry, target, 0).unwrap().as_slice(),
[Value::String(std::sync::Arc::from("previous owner"))]
);
vm.budget.test_lower_limit(
RuntimeResource::TemporaryBytes,
RuntimeLimits::default().get(RuntimeResource::TemporaryBytes),
);
vm.globals.clear(); // Failed operands are now the only session roots.
vm.gc_alloc_threshold = 1;
vm.handle_safepoint().unwrap();
assert!(vm.heap.is_valid(source) && vm.heap.is_valid(target));
assert_eq!(vm.budget.used(RuntimeResource::StringAllocations), 2);
vm.operand_stack.clear();
vm.last_gc_live_count = 0;
vm.handle_safepoint().unwrap();
assert_eq!(vm.heap.len(), 0);
assert_eq!(vm.budget.used(RuntimeResource::StringAllocations), 0);
}
@@ -331,3 +331,45 @@ fn recursive_fn_calls_use_bounded_ordinary_frames() {
); );
assert_eq!(v.call_stack.len(), 4); assert_eq!(v.call_stack.len(), 4);
} }
#[test]
fn root_return_scratch_refusal_keeps_transferred_roots_without_partial_sweep() {
let mut v = vm(&[
("MAKE_CLOSURE fn=1,type=4\nPOP\nMAKE_CLOSURE fn=1,type=4\nRET", sig(&[], &[4]), vec![]),
("PUSH_I32 55\nRET", sig(&[], &[0]), vec![]),
]);
v.gc_alloc_threshold = 1;
for _ in 0..3 {
v.step(&mut Native, &mut HostContext::new(None)).unwrap();
}
let Value::HeapRef(owner) = *v.operand_stack.last().unwrap() else { panic!() };
let live = v.heap.len();
v.budget.test_lower_limit(RuntimeResource::TemporaryBytes, 0);
assert!(matches!(v.step(&mut Native, &mut HostContext::new(None)),
Err(LogicalFrameEndingReason::Trap(t)) if t.code == TRAP_RESOURCE_LIMIT));
assert!(v.call_stack.is_empty());
assert_eq!(v.operand_stack.as_slice(), &[Value::HeapRef(owner)]);
assert!(v.heap.is_valid(owner));
assert_eq!(v.heap.len(), live);
v.budget.test_lower_limit(RuntimeResource::TemporaryBytes, 1024 * 1024);
v.handle_safepoint().unwrap();
assert_eq!(v.heap.len(), 1);
assert!(v.heap.is_valid(owner));
}
#[test]
fn bad_return_height_preserves_frame_and_owners() {
let mut v = vm(&[
("MAKE_CLOSURE fn=1,type=4\nRET", sig(&[], &[4]), vec![]),
("PUSH_I32 55\nRET", sig(&[], &[0]), vec![]),
]);
v.step(&mut Native, &mut HostContext::new(None)).unwrap();
v.operand_stack.push(Value::Int32(9)).unwrap();
let before = v.operand_stack.as_slice().to_vec();
assert!(matches!(v.step(&mut Native, &mut HostContext::new(None)),
Err(LogicalFrameEndingReason::Trap(t)) if t.code == prometeu_bytecode::TRAP_BAD_RET_SLOTS));
assert_eq!(v.call_stack.len(), 1);
assert_eq!(v.operand_stack.as_slice(), before);
let Value::HeapRef(owner) = before[before.len() - 2] else { panic!() };
assert!(v.heap.is_valid(owner));
}
@@ -175,7 +175,7 @@ impl VirtualMachine {
.expect("result capacity reserved before publication"); .expect("result capacity reserved before publication");
Ok(()) Ok(())
} }
fn collection_size_trap( pub(super) fn collection_size_trap(
&self, &self,
opcode: OpCode, opcode: OpCode,
pc: u32, pc: u32,
@@ -1,6 +1,5 @@
use super::collection_test_support as support;
use super::*; use super::*;
#[path = "../../tests/support/collections.rs"]
mod support;
use crate::value_contract::ValueError; use crate::value_contract::ValueError;
use prometeu_bytecode::assemble; use prometeu_bytecode::assemble;
use support::Native; use support::Native;
@@ -46,21 +46,13 @@ impl VirtualMachine {
) )
.map_err(|(_, error)| self.value_trap(error, opcode, pc))?; .map_err(|(_, error)| self.value_trap(error, opcode, pc))?;
// Copy return values (preserving order: pop return_slots values, then reverse to push back) // Validation is complete. Move the ordered result suffix ahead of this
let mut return_vals = BudgetVec::new(&self.budget, RuntimeResource::TemporaryBytes); // frame's parameters/locals without cloning owners or reserving scratch.
return_vals.reserve(return_slots).map_err(|e| self.resource_trap(e, opcode, pc))?; if return_slots != 0 {
for _ in 0..return_slots { self.operand_stack[frame.stack_base..].rotate_right(return_slots);
return_vals
.push(self.pop().map_err(LogicalFrameEndingReason::Panic)?)
.map_err(|e| self.resource_trap(e, opcode, pc))?;
} }
return_vals.reverse(); self.operand_stack.truncate(frame.stack_base + return_slots);
self.call_stack.pop(); self.call_stack.pop();
self.operand_stack.truncate(frame.stack_base);
for val in return_vals.drain(..) {
self.push_trap(val, opcode, pc)?;
}
self.pc = frame.return_pc as usize; self.pc = frame.return_pc as usize;
if self.call_stack.is_empty() { if self.call_stack.is_empty() {
self.handle_safepoint().map_err(|e| self.resource_trap(e, opcode, pc))?; self.handle_safepoint().map_err(|e| self.resource_trap(e, opcode, pc))?;
@@ -0,0 +1,59 @@
use super::*;
use crate::value_contract;
use prometeu_bytecode::registry::TypeId;
impl VirtualMachine {
pub(super) fn execute_identity(
&mut self,
opcode: OpCode,
left: u32,
right: Option<u32>,
pc: u32,
) -> Result<(), LogicalFrameEndingReason> {
let image = self.validated_image.clone().ok_or_else(|| {
self.trap(
prometeu_bytecode::TRAP_ILLEGAL_INSTRUCTION,
opcode as u16,
"Identity instruction requires a validated Registry".into(),
pc,
)
})?;
let registry = image.registry();
let left = TypeId(left);
let right = right.map(TypeId);
let lw = registry.type_layout(left).unwrap().storage.slots as usize;
let rw = right.map_or(0, |ty| registry.type_layout(ty).unwrap().storage.slots as usize);
self.cycles += opcode.cycles() + (lw + rw) as u64;
if self.temporary_slots() < lw + rw {
return Err(self.trap(
TRAP_STACK_UNDERFLOW,
opcode as u16,
"Incomplete identity operands".into(),
pc,
));
}
let base = self.operand_stack.len() - lw - rw;
let first = value_contract::identity_reference(
registry,
&self.heap,
left,
&self.operand_stack[base..base + lw],
)
.map_err(|e| self.value_trap(e, opcode, pc))?;
let result = if let Some(ty) = right {
let second = value_contract::identity_reference(
registry,
&self.heap,
ty,
&self.operand_stack[base + lw..],
)
.map_err(|e| self.value_trap(e, opcode, pc))?;
Value::Boolean(first == second)
} else {
Value::Int64(crate::hashing::reference(first))
};
self.operand_stack.truncate(base);
self.operand_stack.push(result).expect("identity result replaces existing inputs");
Ok(())
}
}

Some files were not shown because too many files have changed in this diff Show More