Compare commits
9
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
22769aa438
|
||
|
|
3f9d235546
|
||
|
|
bb6f212779
|
||
|
|
418e574ac8
|
||
|
|
461b554c1a
|
||
|
|
46bbad19c2
|
||
|
|
e09f1197bc
|
||
|
|
ff73f8ad39
|
||
|
|
bad942fb25
|
@@ -15,15 +15,25 @@ jobs:
|
||||
|
||||
- uses: dtolnay/rust-toolchain@stable
|
||||
with:
|
||||
components: rustfmt, clippy
|
||||
components: rustfmt, clippy, llvm-tools-preview
|
||||
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
|
||||
- name: Format check
|
||||
run: cargo fmt -- --check
|
||||
- name: Discard reports from previous runs
|
||||
run: rm -rf target/quality-gate target/llvm-cov
|
||||
|
||||
- name: Clippy
|
||||
run: cargo clippy --workspace --all-features
|
||||
- name: Provision quality-gate tools and native dependencies
|
||||
run: bash scripts/provision-quality-gate.sh
|
||||
|
||||
- name: Test
|
||||
run: cargo test --workspace --all-targets --all-features --no-fail-fast
|
||||
- name: Repository quality gate
|
||||
run: make ci-domains
|
||||
|
||||
- name: Publish gate evidence (including failures)
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: quality-gate-${{ runner.os }}-${{ github.sha }}
|
||||
path: |
|
||||
target/quality-gate/**
|
||||
target/llvm-cov/**
|
||||
if-no-files-found: warn
|
||||
|
||||
@@ -1,19 +1,19 @@
|
||||
.PHONY: fmt fmt-check clippy test-local test-debugger-socket clean coverage coverage-xml coverage-json coverage-report-json coverage-domain-list coverage-domain-evidence coverage-domain-audit coverage-domain-gate test ci ci-domains cobertura
|
||||
.PHONY: fmt fmt-check clippy test-local test-debugger-socket clean coverage coverage-xml coverage-json coverage-report-json coverage-domain-list coverage-domain-evidence coverage-domain-audit coverage-domain-gate test ci ci-domains test-quality-gate cobertura
|
||||
|
||||
fmt:
|
||||
cargo fmt
|
||||
cargo fmt --all
|
||||
|
||||
fmt-check:
|
||||
cargo fmt -- --check
|
||||
cargo fmt --all -- --check
|
||||
|
||||
clippy:
|
||||
cargo clippy --workspace --all-features
|
||||
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||
|
||||
test-local:
|
||||
cargo test --workspace --all-targets --all-features --no-fail-fast
|
||||
|
||||
test-debugger-socket:
|
||||
cargo test -p prometeu-host-desktop-winit --lib -- --ignored
|
||||
cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored
|
||||
|
||||
clean:
|
||||
cargo llvm-cov clean --workspace
|
||||
@@ -47,6 +47,14 @@ coverage-domain-gate:
|
||||
./scripts/coverage-domain-evidence.sh --check-all
|
||||
|
||||
test: fmt-check clippy test-local test-debugger-socket
|
||||
ci: clean fmt-check clippy coverage coverage-report-json
|
||||
ci-domains: clean fmt-check clippy coverage coverage-report-json coverage-xml coverage-json coverage-domain-gate
|
||||
# Keep the step list here: CI providers consume the repository contract.
|
||||
# The runner executes each step serially even if the caller supplies make -j.
|
||||
CI_DOMAIN_STEPS = clean fmt-check clippy test-quality-gate test-local test-debugger-socket coverage coverage-report-json coverage-xml coverage-json coverage-domain-audit coverage-domain-gate
|
||||
|
||||
ci: ci-domains
|
||||
ci-domains:
|
||||
python3 scripts/run-quality-gate.py $(CI_DOMAIN_STEPS)
|
||||
|
||||
test-quality-gate:
|
||||
python3 -B -m unittest discover -s scripts/tests -p 'test_quality_gate.py'
|
||||
cobertura: coverage-xml coverage-json
|
||||
|
||||
@@ -107,19 +107,19 @@ Format, lint, and test the workspace:
|
||||
make test
|
||||
```
|
||||
|
||||
Run the CI-oriented non-domain gate:
|
||||
Run the shared strict CI gate (`make ci` is an alias):
|
||||
|
||||
```bash
|
||||
make ci
|
||||
```
|
||||
|
||||
Run the CI-oriented domain coverage gate:
|
||||
The authoritative target used by both GitHub and Jenkins is:
|
||||
|
||||
```bash
|
||||
make ci-domains
|
||||
```
|
||||
|
||||
The debugger socket tests bind localhost sockets and are marked ignored in Rust test metadata; `make test` includes them through the dedicated `test-debugger-socket` target.
|
||||
The five debugger socket tests bind localhost sockets and are marked ignored in Rust test metadata; both `make test` and `make ci-domains` select only `runner::tests` through `test-debugger-socket`. Physical audio and other manual probes remain opt-in. See [quality-gate prerequisites and commands](docs/vm-arch/COVERAGE_DOMAINS.md#commands-and-ci).
|
||||
|
||||
## Coverage Governance
|
||||
|
||||
@@ -128,20 +128,14 @@ The repository uses a two-layer runtime-edge coverage model:
|
||||
- a mandatory CI gate based on canonical runtime-edge domains;
|
||||
- domain-oriented review evidence for the canonical runtime-edge domains.
|
||||
|
||||
Current canonical domains:
|
||||
|
||||
- `system/runtime`
|
||||
- `fs`
|
||||
- `asset/bank`
|
||||
- `firmware`
|
||||
- `host-dependent`
|
||||
- `host/headless`
|
||||
The fifteen canonical domains and fixed floors are defined in
|
||||
[the repository configuration](files/config/runtime-edge-coverage-domains.json).
|
||||
Use `make coverage-domain-list` or the [ownership documentation](docs/vm-arch/COVERAGE_DOMAINS.md)
|
||||
to inspect the current map.
|
||||
|
||||
Global coverage artifacts:
|
||||
|
||||
```bash
|
||||
make ci
|
||||
make cobertura
|
||||
make ci-domains
|
||||
```
|
||||
|
||||
@@ -151,6 +145,7 @@ This produces:
|
||||
- `target/llvm-cov/report.json`
|
||||
- `target/llvm-cov/summary.json`
|
||||
- `target/llvm-cov/cobertura.xml`
|
||||
- `target/quality-gate/metadata.json` and per-step logs
|
||||
|
||||
Domain-oriented evidence helpers:
|
||||
|
||||
@@ -161,12 +156,12 @@ make coverage-domain-evidence DOMAIN=host-dependent
|
||||
make coverage-domain-evidence DOMAIN=host/headless
|
||||
```
|
||||
|
||||
Per-domain baselines currently start around `60`.
|
||||
That does not waive test obligations. It means the project is adopting explicit domain evidence immediately while tightening quantitative expectations incrementally over time.
|
||||
|
||||
`make ci-domains` is the CI entrypoint used by Jenkins.
|
||||
It runs the standard coverage pipeline, produces the HTML/XML/JSON artifacts, and fails when any canonical domain falls below its configured `baseline_percent`.
|
||||
The CI gate currently compares domain `lines` coverage against each domain baseline.
|
||||
`make ci-domains` runs formatting, strict workspace/all-target/all-feature Clippy,
|
||||
all-target tests, the five debugger socket tests, fresh coverage and every domain
|
||||
ownership/evidence gate. Both CI providers archive available evidence on failure.
|
||||
A failing run never lowers a configured floor; local and CI checks use the same
|
||||
repository-owned commands. Reports identify their revision, source/configuration
|
||||
and actual execution platform.
|
||||
|
||||
When a PR changes the observable contract of a canonical domain, it is expected to:
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
#![allow(clippy::collapsible_if)]
|
||||
mod decode;
|
||||
mod resumable;
|
||||
use crate::memory_banks::{GlyphBankPoolInstaller, SceneBankPoolInstaller, SoundBankPoolInstaller};
|
||||
use prometeu_hal::AssetBridge;
|
||||
@@ -634,62 +635,13 @@ impl AssetManager {
|
||||
fn decode_glyph_bank_layout(
|
||||
entry: &AssetEntry,
|
||||
) -> Result<(TileSize, usize, usize, usize, usize, usize), String> {
|
||||
let meta = entry.metadata_as_glyph_bank()?;
|
||||
|
||||
let tile_size = match meta.tile_size {
|
||||
8 => TileSize::Size8,
|
||||
16 => TileSize::Size16,
|
||||
32 => TileSize::Size32,
|
||||
_ => return Err(format!("Invalid tile_size: {}", meta.tile_size)),
|
||||
};
|
||||
|
||||
let palette_count = meta.palette_count as usize;
|
||||
if !(1..=GLYPH_BANK_MAX_PALETTE_COUNT_V1).contains(&palette_count) {
|
||||
return Err(format!("Invalid palette_count: {}", meta.palette_count));
|
||||
}
|
||||
|
||||
let width = meta.width as usize;
|
||||
let height = meta.height as usize;
|
||||
|
||||
let logical_pixels = width.checked_mul(height).ok_or("GlyphBank dimensions overflow")?;
|
||||
let serialized_pixel_bytes = logical_pixels.div_ceil(2);
|
||||
let palette_bytes = palette_count
|
||||
.checked_mul(GLYPH_BANK_PALETTE_BYTES_PER_PALETTE)
|
||||
.ok_or("GlyphBank palette size overflow")?;
|
||||
let serialized_size = serialized_pixel_bytes
|
||||
.checked_add(palette_bytes)
|
||||
.ok_or("GlyphBank serialized size overflow")?;
|
||||
let decoded_size =
|
||||
logical_pixels.checked_add(palette_bytes).ok_or("GlyphBank decoded size overflow")?;
|
||||
|
||||
if entry.size != serialized_size as u64 {
|
||||
return Err(format!(
|
||||
"Invalid GLYPHBANK serialized size: expected {}, got {}",
|
||||
serialized_size, entry.size
|
||||
));
|
||||
}
|
||||
|
||||
if entry.decoded_size != decoded_size as u64 {
|
||||
return Err(format!(
|
||||
"Invalid GLYPHBANK decoded_size: expected {}, got {}",
|
||||
decoded_size, entry.decoded_size
|
||||
));
|
||||
}
|
||||
|
||||
Ok((tile_size, width, height, serialized_pixel_bytes, palette_count, palette_bytes))
|
||||
decode::glyph_layout(entry)
|
||||
}
|
||||
|
||||
fn unpack_glyph_bank_pixels(packed_pixels: &[u8], logical_pixels: usize) -> Vec<u8> {
|
||||
let mut pixel_indices = Vec::with_capacity(logical_pixels);
|
||||
for &packed in packed_pixels {
|
||||
if pixel_indices.len() < logical_pixels {
|
||||
pixel_indices.push(packed >> 4);
|
||||
}
|
||||
if pixel_indices.len() < logical_pixels {
|
||||
pixel_indices.push(packed & 0x0f);
|
||||
}
|
||||
}
|
||||
pixel_indices
|
||||
let mut pixels = Vec::with_capacity(logical_pixels);
|
||||
decode::append_pixels(&mut pixels, packed_pixels, logical_pixels);
|
||||
pixels
|
||||
}
|
||||
|
||||
fn op_mode_for(entry: &AssetEntry) -> Result<AssetOpMode, String> {
|
||||
@@ -1260,17 +1212,7 @@ impl AssetManager {
|
||||
let palette_data = &buffer[packed_pixel_bytes..packed_pixel_bytes + palette_bytes];
|
||||
|
||||
let mut palettes = vec![[Color::BLACK; GLYPH_BANK_COLORS_PER_PALETTE]; palette_count];
|
||||
for (p, pal) in palettes.iter_mut().enumerate() {
|
||||
for (c, slot) in pal.iter_mut().enumerate() {
|
||||
let offset = (p * 16 + c) * 4;
|
||||
*slot = Color::rgba(
|
||||
palette_data[offset],
|
||||
palette_data[offset + 1],
|
||||
palette_data[offset + 2],
|
||||
palette_data[offset + 3],
|
||||
);
|
||||
}
|
||||
}
|
||||
decode::write_palettes(&mut palettes, 0, palette_data);
|
||||
|
||||
Ok(GlyphBank { tile_size, width, height, pixel_indices, palettes })
|
||||
}
|
||||
@@ -1295,17 +1237,7 @@ impl AssetManager {
|
||||
.map_err(|_| "Buffer too small for GLYPHBANK".to_string())?;
|
||||
|
||||
let mut palettes = vec![[Color::BLACK; GLYPH_BANK_COLORS_PER_PALETTE]; palette_count];
|
||||
for (p, pal) in palettes.iter_mut().enumerate() {
|
||||
for (c, slot) in pal.iter_mut().enumerate() {
|
||||
let offset = (p * 16 + c) * 4;
|
||||
*slot = Color::rgba(
|
||||
palette_data[offset],
|
||||
palette_data[offset + 1],
|
||||
palette_data[offset + 2],
|
||||
palette_data[offset + 3],
|
||||
);
|
||||
}
|
||||
}
|
||||
decode::write_palettes(&mut palettes, 0, &palette_data);
|
||||
|
||||
Ok(GlyphBank { tile_size, width, height, pixel_indices, palettes })
|
||||
}
|
||||
@@ -1340,7 +1272,7 @@ impl AssetManager {
|
||||
entry: &AssetEntry,
|
||||
assets_data: Arc<RwLock<AssetsPayloadSource>>,
|
||||
) -> Result<SceneBank, String> {
|
||||
let _ = entry.metadata_as_scene_bank()?;
|
||||
decode::scene_metadata(entry)?;
|
||||
let op_mode = Self::op_mode_for(entry)?;
|
||||
let slice = {
|
||||
let assets_data = assets_data.read().unwrap();
|
||||
@@ -1367,152 +1299,25 @@ impl AssetManager {
|
||||
entry: &AssetEntry,
|
||||
buffer: &[u8],
|
||||
) -> Result<SceneBank, String> {
|
||||
let _ = entry.metadata_as_scene_bank()?;
|
||||
|
||||
if buffer.len() < SCENE_HEADER_BYTES_V1 {
|
||||
return Err("Buffer too small for SCENE".to_string());
|
||||
}
|
||||
|
||||
if buffer[0..4] != SCENE_PAYLOAD_MAGIC_V1 {
|
||||
return Err("Invalid SCENE magic".to_string());
|
||||
}
|
||||
|
||||
let version = u16::from_le_bytes([buffer[4], buffer[5]]);
|
||||
if version != SCENE_PAYLOAD_VERSION_V1 {
|
||||
return Err(format!("Unsupported SCENE version: {}", version));
|
||||
}
|
||||
|
||||
let layer_count = u16::from_le_bytes([buffer[6], buffer[7]]) as usize;
|
||||
if layer_count != SCENE_LAYER_COUNT_V1 {
|
||||
return Err(format!("Invalid SCENE layer count: {}", layer_count));
|
||||
}
|
||||
|
||||
decode::scene_metadata(entry)?;
|
||||
decode::payload_size(entry, buffer.len())?;
|
||||
decode::scene_header(buffer)?;
|
||||
let mut offset = SCENE_HEADER_BYTES_V1;
|
||||
let mut decoded_size = 0_usize;
|
||||
let layers = std::array::from_fn(|_| SceneLayer {
|
||||
active: false,
|
||||
glyph_asset_id: 0,
|
||||
tile_size: TileSize::Size8,
|
||||
parallax_factor: ParallaxFactor { x: 1.0, y: 1.0 },
|
||||
tilemap: TileMap { width: 0, height: 0, tiles: Vec::new() },
|
||||
});
|
||||
let mut layers = layers;
|
||||
|
||||
let mut decoded = 0;
|
||||
let mut layers = decode::empty_layers();
|
||||
for layer in &mut layers {
|
||||
let header_end = offset
|
||||
let end = offset
|
||||
.checked_add(SCENE_LAYER_HEADER_BYTES_V1)
|
||||
.ok_or("SCENE layer header offset overflow")?;
|
||||
if header_end > buffer.len() {
|
||||
return Err("Buffer too small for SCENE layer header".to_string());
|
||||
}
|
||||
|
||||
let flags = buffer[offset];
|
||||
let glyph_asset_id = i32::from_le_bytes([
|
||||
buffer[offset + 1],
|
||||
buffer[offset + 2],
|
||||
buffer[offset + 3],
|
||||
buffer[offset + 4],
|
||||
]);
|
||||
let tile_size_raw = buffer[offset + 5];
|
||||
let tile_size = match tile_size_raw {
|
||||
8 => TileSize::Size8,
|
||||
16 => TileSize::Size16,
|
||||
32 => TileSize::Size32,
|
||||
other => return Err(format!("Invalid SCENE tile size: {}", other)),
|
||||
};
|
||||
let parallax_factor_x = f32::from_le_bytes([
|
||||
buffer[offset + 8],
|
||||
buffer[offset + 9],
|
||||
buffer[offset + 10],
|
||||
buffer[offset + 11],
|
||||
]);
|
||||
let parallax_factor_y = f32::from_le_bytes([
|
||||
buffer[offset + 12],
|
||||
buffer[offset + 13],
|
||||
buffer[offset + 14],
|
||||
buffer[offset + 15],
|
||||
]);
|
||||
if !parallax_factor_x.is_finite() || !parallax_factor_y.is_finite() {
|
||||
return Err("Invalid SCENE parallax_factor".to_string());
|
||||
}
|
||||
|
||||
let width = u32::from_le_bytes([
|
||||
buffer[offset + 16],
|
||||
buffer[offset + 17],
|
||||
buffer[offset + 18],
|
||||
buffer[offset + 19],
|
||||
]) as usize;
|
||||
let height = u32::from_le_bytes([
|
||||
buffer[offset + 20],
|
||||
buffer[offset + 21],
|
||||
buffer[offset + 22],
|
||||
buffer[offset + 23],
|
||||
]) as usize;
|
||||
let tile_count = u32::from_le_bytes([
|
||||
buffer[offset + 24],
|
||||
buffer[offset + 25],
|
||||
buffer[offset + 26],
|
||||
buffer[offset + 27],
|
||||
]) as usize;
|
||||
|
||||
let expected_tile_count =
|
||||
width.checked_mul(height).ok_or("SCENE tile count overflow")?;
|
||||
if tile_count != expected_tile_count {
|
||||
return Err(format!(
|
||||
"Invalid SCENE tile count for layer: expected {}, got {}",
|
||||
expected_tile_count, tile_count
|
||||
));
|
||||
}
|
||||
|
||||
offset = header_end;
|
||||
|
||||
let tile_bytes = tile_count
|
||||
.checked_mul(SCENE_TILE_RECORD_BYTES_V1)
|
||||
.ok_or("SCENE tile payload overflow")?;
|
||||
let tiles_end = offset.checked_add(tile_bytes).ok_or("SCENE payload overflow")?;
|
||||
if tiles_end > buffer.len() {
|
||||
return Err("Buffer too small for SCENE tile data".to_string());
|
||||
}
|
||||
|
||||
let mut tiles = Vec::with_capacity(tile_count);
|
||||
for _ in 0..tile_count {
|
||||
let tile_flags = buffer[offset];
|
||||
let palette_id = buffer[offset + 1];
|
||||
let glyph_id = u16::from_le_bytes([buffer[offset + 2], buffer[offset + 3]]);
|
||||
tiles.push(Tile {
|
||||
active: (tile_flags & 0b0000_0001) != 0,
|
||||
glyph: Glyph { glyph_id, palette_id },
|
||||
flip_x: (tile_flags & 0b0000_0010) != 0,
|
||||
flip_y: (tile_flags & 0b0000_0100) != 0,
|
||||
});
|
||||
offset += SCENE_TILE_RECORD_BYTES_V1;
|
||||
}
|
||||
|
||||
decoded_size = decoded_size
|
||||
.checked_add(SCENE_DECODED_LAYER_OVERHEAD_BYTES_V1)
|
||||
.and_then(|size| size.checked_add(tile_count * size_of::<Tile>()))
|
||||
.ok_or("SCENE decoded_size overflow")?;
|
||||
|
||||
*layer = SceneLayer {
|
||||
active: (flags & 0b0000_0001) != 0,
|
||||
glyph_asset_id,
|
||||
tile_size,
|
||||
parallax_factor: ParallaxFactor { x: parallax_factor_x, y: parallax_factor_y },
|
||||
tilemap: TileMap { width, height, tiles },
|
||||
};
|
||||
.ok_or("SCENE header offset overflow")?;
|
||||
let header =
|
||||
buffer.get(offset..end).ok_or("Buffer too small for SCENE layer header")?;
|
||||
*layer = decode::scene_layer(header, end, entry.size, &mut decoded)?;
|
||||
let tile_bytes =
|
||||
layer.tilemap.width * layer.tilemap.height * SCENE_TILE_RECORD_BYTES_V1;
|
||||
offset = end + tile_bytes; // checked by scene_layer before allocation
|
||||
decode::append_tiles(&mut layer.tilemap.tiles, &buffer[end..offset]);
|
||||
}
|
||||
|
||||
if offset != buffer.len() {
|
||||
return Err("Trailing bytes in SCENE payload".to_string());
|
||||
}
|
||||
|
||||
if entry.decoded_size != decoded_size as u64 {
|
||||
return Err(format!(
|
||||
"Invalid SCENE decoded_size: expected {}, got {}",
|
||||
decoded_size, entry.decoded_size
|
||||
));
|
||||
}
|
||||
|
||||
decode::scene_finish(entry, offset, decoded)?;
|
||||
Ok(SceneBank { layers })
|
||||
}
|
||||
|
||||
@@ -1529,15 +1334,10 @@ impl AssetManager {
|
||||
entry: &AssetEntry,
|
||||
buffer: &[u8],
|
||||
) -> Result<SoundBank, String> {
|
||||
let meta = entry.metadata_as_sound_bank()?;
|
||||
let sample_rate = meta.sample_rate;
|
||||
|
||||
let sample_rate = decode::sound_rate(entry)?;
|
||||
decode::payload_size(entry, buffer.len())?;
|
||||
let mut data = Vec::with_capacity(buffer.len() / 2);
|
||||
for i in (0..buffer.len()).step_by(2) {
|
||||
if i + 1 < buffer.len() {
|
||||
data.push(i16::from_le_bytes([buffer[i], buffer[i + 1]]));
|
||||
}
|
||||
}
|
||||
decode::append_pcm(&mut data, buffer);
|
||||
|
||||
let sample = Arc::new(Sample::new(sample_rate, data));
|
||||
Ok(SoundBank::new(vec![sample]))
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
//! Format authority shared by complete-buffer and bounded resumable decoding.
|
||||
use super::*;
|
||||
|
||||
pub(super) fn glyph_layout(
|
||||
entry: &AssetEntry,
|
||||
) -> Result<(TileSize, usize, usize, usize, usize, usize), String> {
|
||||
let meta = entry.metadata_as_glyph_bank()?;
|
||||
|
||||
let tile_size = match meta.tile_size {
|
||||
8 => TileSize::Size8,
|
||||
16 => TileSize::Size16,
|
||||
32 => TileSize::Size32,
|
||||
_ => return Err(format!("Invalid tile_size: {}", meta.tile_size)),
|
||||
};
|
||||
|
||||
let palette_count = meta.palette_count as usize;
|
||||
if !(1..=GLYPH_BANK_MAX_PALETTE_COUNT_V1).contains(&palette_count) {
|
||||
return Err(format!("Invalid palette_count: {}", meta.palette_count));
|
||||
}
|
||||
|
||||
let width = meta.width as usize;
|
||||
let height = meta.height as usize;
|
||||
|
||||
let logical_pixels = width.checked_mul(height).ok_or("GlyphBank dimensions overflow")?;
|
||||
let serialized_pixel_bytes = logical_pixels.div_ceil(2);
|
||||
let palette_bytes = palette_count
|
||||
.checked_mul(GLYPH_BANK_PALETTE_BYTES_PER_PALETTE)
|
||||
.ok_or("GlyphBank palette size overflow")?;
|
||||
let serialized_size = serialized_pixel_bytes
|
||||
.checked_add(palette_bytes)
|
||||
.ok_or("GlyphBank serialized size overflow")?;
|
||||
let decoded_size =
|
||||
logical_pixels.checked_add(palette_bytes).ok_or("GlyphBank decoded size overflow")?;
|
||||
|
||||
if entry.size != serialized_size as u64 {
|
||||
return Err(format!(
|
||||
"Invalid GLYPHBANK serialized size: expected {}, got {}",
|
||||
serialized_size, entry.size
|
||||
));
|
||||
}
|
||||
|
||||
if entry.decoded_size != decoded_size as u64 {
|
||||
return Err(format!(
|
||||
"Invalid GLYPHBANK decoded_size: expected {}, got {}",
|
||||
decoded_size, entry.decoded_size
|
||||
));
|
||||
}
|
||||
|
||||
Ok((tile_size, width, height, serialized_pixel_bytes, palette_count, palette_bytes))
|
||||
}
|
||||
|
||||
pub(super) fn append_pixels(pixels: &mut Vec<u8>, bytes: &[u8], logical: usize) {
|
||||
for &packed in bytes {
|
||||
if pixels.len() < logical {
|
||||
pixels.push(packed >> 4);
|
||||
}
|
||||
if pixels.len() < logical {
|
||||
pixels.push(packed & 15);
|
||||
}
|
||||
}
|
||||
}
|
||||
pub(super) fn write_palettes(
|
||||
palettes: &mut [[Color; GLYPH_BANK_COLORS_PER_PALETTE]],
|
||||
position: usize,
|
||||
bytes: &[u8],
|
||||
) {
|
||||
for (i, rgba) in bytes.as_chunks::<4>().0.iter().enumerate() {
|
||||
let at = position / 4 + i;
|
||||
palettes[at / 16][at % 16] = Color::rgba(rgba[0], rgba[1], rgba[2], rgba[3]);
|
||||
}
|
||||
}
|
||||
pub(super) fn sound_rate(entry: &AssetEntry) -> Result<u32, String> {
|
||||
Ok(entry.metadata_as_sound_bank()?.sample_rate)
|
||||
}
|
||||
// Callers split on even chunk boundaries. Only the final odd byte is ignored.
|
||||
pub(super) fn append_pcm(data: &mut Vec<i16>, bytes: &[u8]) {
|
||||
data.extend(bytes.as_chunks::<2>().0.iter().map(|p| i16::from_le_bytes(*p)));
|
||||
}
|
||||
pub(super) fn payload_size(entry: &AssetEntry, actual: usize) -> Result<(), String> {
|
||||
if actual as u64 != entry.size {
|
||||
return Err("Invalid asset payload size".into());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
pub(super) fn scene_metadata(entry: &AssetEntry) -> Result<(), String> {
|
||||
entry.metadata_as_scene_bank().map(|_| ())
|
||||
}
|
||||
pub(super) fn empty_layers() -> [SceneLayer; SCENE_LAYER_COUNT_V1] {
|
||||
std::array::from_fn(|_| SceneLayer {
|
||||
active: false,
|
||||
glyph_asset_id: 0,
|
||||
tile_size: TileSize::Size8,
|
||||
parallax_factor: ParallaxFactor { x: 1.0, y: 1.0 },
|
||||
tilemap: TileMap { width: 0, height: 0, tiles: Vec::new() },
|
||||
})
|
||||
}
|
||||
pub(super) fn scene_header(bytes: &[u8]) -> Result<(), String> {
|
||||
if bytes.len() < SCENE_HEADER_BYTES_V1 {
|
||||
return Err("Buffer too small for SCENE".into());
|
||||
}
|
||||
if bytes[..4] != SCENE_PAYLOAD_MAGIC_V1 {
|
||||
return Err("Invalid SCENE magic".into());
|
||||
}
|
||||
let version = u16::from_le_bytes([bytes[4], bytes[5]]);
|
||||
if version != SCENE_PAYLOAD_VERSION_V1 {
|
||||
return Err(format!("Unsupported SCENE version: {version}"));
|
||||
}
|
||||
let count = u16::from_le_bytes([bytes[6], bytes[7]]) as usize;
|
||||
if count != SCENE_LAYER_COUNT_V1 {
|
||||
return Err(format!("Invalid SCENE layer count: {count}"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
pub(super) fn scene_layer(
|
||||
bytes: &[u8],
|
||||
position: usize,
|
||||
payload_len: u64,
|
||||
decoded: &mut usize,
|
||||
) -> Result<SceneLayer, String> {
|
||||
if bytes.len() < SCENE_LAYER_HEADER_BYTES_V1 {
|
||||
return Err("Buffer too small for SCENE layer header".into());
|
||||
}
|
||||
let u32at = |at| u32::from_le_bytes(bytes[at..at + 4].try_into().unwrap());
|
||||
let tile_size = match bytes[5] {
|
||||
8 => TileSize::Size8,
|
||||
16 => TileSize::Size16,
|
||||
32 => TileSize::Size32,
|
||||
other => return Err(format!("Invalid SCENE tile size: {other}")),
|
||||
};
|
||||
let x = f32::from_le_bytes(bytes[8..12].try_into().unwrap());
|
||||
let y = f32::from_le_bytes(bytes[12..16].try_into().unwrap());
|
||||
if !x.is_finite() || !y.is_finite() {
|
||||
return Err("Invalid SCENE parallax_factor".into());
|
||||
}
|
||||
let width = u32at(16) as usize;
|
||||
let height = u32at(20) as usize;
|
||||
let count = u32at(24) as usize;
|
||||
let expected = width.checked_mul(height).ok_or("SCENE tile count overflow")?;
|
||||
if count != expected {
|
||||
return Err(format!(
|
||||
"Invalid SCENE tile count for layer: expected {expected}, got {count}"
|
||||
));
|
||||
}
|
||||
let end = count
|
||||
.checked_mul(SCENE_TILE_RECORD_BYTES_V1)
|
||||
.and_then(|n| position.checked_add(n))
|
||||
.ok_or("SCENE payload overflow")?;
|
||||
if end as u64 > payload_len {
|
||||
return Err("Buffer too small for SCENE tile data".into());
|
||||
}
|
||||
*decoded = decoded
|
||||
.checked_add(SCENE_DECODED_LAYER_OVERHEAD_BYTES_V1)
|
||||
.and_then(|n| count.checked_mul(size_of::<Tile>()).and_then(|b| n.checked_add(b)))
|
||||
.ok_or("SCENE decoded_size overflow")?;
|
||||
Ok(SceneLayer {
|
||||
active: bytes[0] & 1 != 0,
|
||||
glyph_asset_id: i32::from_le_bytes(bytes[1..5].try_into().unwrap()),
|
||||
tile_size,
|
||||
parallax_factor: ParallaxFactor { x, y },
|
||||
tilemap: TileMap { width, height, tiles: Vec::with_capacity(count) },
|
||||
})
|
||||
}
|
||||
pub(super) fn append_tiles(tiles: &mut Vec<Tile>, bytes: &[u8]) {
|
||||
tiles.extend(bytes.as_chunks::<SCENE_TILE_RECORD_BYTES_V1>().0.iter().map(|b| Tile {
|
||||
active: b[0] & 1 != 0,
|
||||
glyph: Glyph { glyph_id: u16::from_le_bytes([b[2], b[3]]), palette_id: b[1] },
|
||||
flip_x: b[0] & 2 != 0,
|
||||
flip_y: b[0] & 4 != 0,
|
||||
}));
|
||||
}
|
||||
pub(super) fn scene_finish(
|
||||
entry: &AssetEntry,
|
||||
position: usize,
|
||||
decoded: usize,
|
||||
) -> Result<(), String> {
|
||||
payload_size(entry, position)?;
|
||||
if decoded as u64 != entry.decoded_size {
|
||||
return Err(format!(
|
||||
"Invalid SCENE decoded_size: expected {decoded}, got {}",
|
||||
entry.decoded_size
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(super) mod tests;
|
||||
@@ -0,0 +1,432 @@
|
||||
use super::*;
|
||||
use serde_json::{Value, json};
|
||||
|
||||
struct FileFixture(std::path::PathBuf);
|
||||
impl FileFixture {
|
||||
fn new(name: &str, bytes: &[u8]) -> Self {
|
||||
let path = std::env::temp_dir()
|
||||
.join(format!("prometeu-conformance-{}-{name}", std::process::id()));
|
||||
std::fs::write(&path, bytes).unwrap();
|
||||
Self(path)
|
||||
}
|
||||
fn source(&self, declared_len: u64) -> Arc<RwLock<AssetsPayloadSource>> {
|
||||
Arc::new(RwLock::new(AssetsPayloadSource::from_file(self.0.clone(), 0, declared_len)))
|
||||
}
|
||||
}
|
||||
impl Drop for FileFixture {
|
||||
fn drop(&mut self) {
|
||||
std::fs::remove_file(&self.0).unwrap();
|
||||
}
|
||||
}
|
||||
fn pcm_entry(size: u64) -> AssetEntry {
|
||||
AssetEntry {
|
||||
asset_id: 1,
|
||||
asset_name: "pcm".into(),
|
||||
bank_type: BankType::SOUNDS,
|
||||
offset: 0,
|
||||
size,
|
||||
decoded_size: size,
|
||||
codec: AssetCodec::None,
|
||||
metadata: json!({"sample_rate": 44100, "channels": 1}),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pcm_chunk_edges_and_odd_tails_have_explicit_baseline_outputs() {
|
||||
for len in [0, 1, 2, 3, 4095, 4096, 4097, 8191, 8192, 8193] {
|
||||
let bytes: Vec<u8> = (0..len).map(|i| if i % 2 == 0 { 0x34 } else { 0x12 }).collect();
|
||||
let entry = pcm_entry(len as u64);
|
||||
let expected = json!([[vec![0x1234_i16; len / 2], 44100, null, null]]);
|
||||
let buffer = AssetManager::decode_sound_bank_from_buffer(&entry, &bytes).unwrap();
|
||||
let reader = AssetManager::decode_sound_bank_from_reader(&entry, &mut &bytes[..]).unwrap();
|
||||
let source = Arc::new(RwLock::new(AssetsPayloadSource::from_bytes(bytes)));
|
||||
let production = AssetManager::perform_load_sound_bank(&entry, source.clone()).unwrap();
|
||||
assert_eq!(sound_snapshot(&buffer), expected, "buffer len={len}");
|
||||
assert_eq!(sound_snapshot(&reader), expected, "reader len={len}");
|
||||
assert_eq!(sound_snapshot(&production), expected, "legacy load len={len}");
|
||||
assert_eq!(
|
||||
resumable::conformance_result(&entry, &source).unwrap(),
|
||||
expected,
|
||||
"resumable len={len}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn physically_truncated_pcm_file_is_rejected_by_both_modes() {
|
||||
// Two actual bytes, three declared bytes: the missing byte is an odd tail.
|
||||
// The approved rule rejects missing declared bytes, including an odd tail.
|
||||
let fixture = FileFixture::new("truncated-pcm", &[0x34, 0x12]);
|
||||
let entry = pcm_entry(3);
|
||||
let source = fixture.source(3);
|
||||
assert!(AssetManager::perform_load_sound_bank(&entry, source.clone()).is_err());
|
||||
assert!(resumable::conformance_result(&entry, &source).is_err());
|
||||
// With the declared byte physically present, both accept and ignore it.
|
||||
std::fs::write(&fixture.0, [0x34, 0x12, 0xff]).unwrap();
|
||||
let expected = json!([[[0x1234], 44100, null, null]]);
|
||||
assert_eq!(resumable::conformance_result(&entry, &source).unwrap(), expected);
|
||||
assert_eq!(
|
||||
sound_snapshot(&AssetManager::perform_load_sound_bank(&entry, source.clone()).unwrap()),
|
||||
expected
|
||||
);
|
||||
// Memory carriers reject the range before either decoder can accept it.
|
||||
let memory = Arc::new(RwLock::new(AssetsPayloadSource::from_bytes(vec![0x34, 0x12])));
|
||||
assert!(AssetManager::perform_load_sound_bank(&entry, memory.clone()).is_err());
|
||||
assert!(resumable::conformance_result(&entry, &memory).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn physically_short_scene_file_is_rejected_by_both_modes() {
|
||||
// Minimal scene: four empty layers, 12-byte header + 4 * 32-byte headers.
|
||||
let mut bytes = vec![0_u8; 140];
|
||||
bytes[..8].copy_from_slice(&[b'S', b'C', b'N', b'E', 1, 0, 4, 0]);
|
||||
for start in [12, 44, 76, 108] {
|
||||
bytes[start + 5] = 8;
|
||||
}
|
||||
let fixture = FileFixture::new("short-scene", &bytes);
|
||||
let mut entry = AssetEntry {
|
||||
asset_id: 2,
|
||||
asset_name: "scene".into(),
|
||||
bank_type: BankType::SCENE,
|
||||
offset: 0,
|
||||
size: 141,
|
||||
decoded_size: 64,
|
||||
codec: AssetCodec::None,
|
||||
metadata: json!({}),
|
||||
};
|
||||
let expected = json!(vec![json!([false, 0, 8, 0.0, 0.0, 0, 0, []]); 4]);
|
||||
let source = fixture.source(141);
|
||||
assert!(AssetManager::perform_load_scene_bank(&entry, source.clone()).is_err());
|
||||
assert!(resumable::conformance_result(&entry, &source).is_err());
|
||||
// Exact physical/declared size is accepted by both.
|
||||
entry.size = 140;
|
||||
assert_eq!(resumable::conformance_result(&entry, &source).unwrap(), expected);
|
||||
assert_eq!(
|
||||
scene_snapshot(&AssetManager::perform_load_scene_bank(&entry, source.clone()).unwrap()),
|
||||
expected
|
||||
);
|
||||
// An actual trailing byte is rejected by both.
|
||||
entry.size = 141;
|
||||
bytes.push(0);
|
||||
std::fs::write(&fixture.0, bytes).unwrap();
|
||||
assert!(AssetManager::perform_load_scene_bank(&entry, source.clone()).is_err());
|
||||
assert!(resumable::conformance_result(&entry, &source).is_err());
|
||||
}
|
||||
|
||||
// Every mode sees the same declared slice, just as it does in production.
|
||||
fn check_case(name: &str, entry: &AssetEntry, bytes: Vec<u8>, expected: Option<Value>) {
|
||||
let source = Arc::new(RwLock::new(AssetsPayloadSource::from_bytes(bytes)));
|
||||
for mode in 0..4 {
|
||||
let result = (|| -> Result<Value, String> {
|
||||
if mode == 3 {
|
||||
return resumable::conformance_result(entry, &source);
|
||||
}
|
||||
if mode == 2 {
|
||||
return match entry.bank_type {
|
||||
BankType::GLYPH => AssetManager::perform_load_glyph_bank(entry, source.clone())
|
||||
.map(|b| glyph_snapshot(&b)),
|
||||
BankType::SOUNDS => {
|
||||
AssetManager::perform_load_sound_bank(entry, source.clone())
|
||||
.map(|b| sound_snapshot(&b))
|
||||
}
|
||||
BankType::SCENE => AssetManager::perform_load_scene_bank(entry, source.clone())
|
||||
.map(|b| scene_snapshot(&b)),
|
||||
};
|
||||
}
|
||||
let slice = source
|
||||
.read()
|
||||
.unwrap()
|
||||
.open_slice(entry.offset, entry.size)
|
||||
.map_err(|e| e.to_string())?;
|
||||
let bytes = slice.read_all().map_err(|e| e.to_string())?;
|
||||
let mut reader = &bytes[..];
|
||||
match (entry.bank_type, mode) {
|
||||
(BankType::GLYPH, 0) => AssetManager::decode_glyph_bank_from_buffer(entry, &bytes)
|
||||
.map(|b| glyph_snapshot(&b)),
|
||||
(BankType::GLYPH, _) => {
|
||||
AssetManager::decode_glyph_bank_from_reader(entry, &mut reader)
|
||||
.map(|b| glyph_snapshot(&b))
|
||||
}
|
||||
(BankType::SOUNDS, 0) => AssetManager::decode_sound_bank_from_buffer(entry, &bytes)
|
||||
.map(|b| sound_snapshot(&b)),
|
||||
(BankType::SOUNDS, _) => {
|
||||
AssetManager::decode_sound_bank_from_reader(entry, &mut reader)
|
||||
.map(|b| sound_snapshot(&b))
|
||||
}
|
||||
(BankType::SCENE, 0) => AssetManager::decode_scene_bank_from_buffer(entry, &bytes)
|
||||
.map(|b| scene_snapshot(&b)),
|
||||
(BankType::SCENE, _) => {
|
||||
AssetManager::decode_scene_bank_from_reader(entry, &mut reader)
|
||||
.map(|b| scene_snapshot(&b))
|
||||
}
|
||||
}
|
||||
})();
|
||||
assert_eq!(result.ok(), expected, "{name}, mode={mode}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn glyph_layout_palette_nibbles_and_chunk_boundaries() {
|
||||
for pixels in [0_usize, 1, 2, 8191, 8192, 8193, 16385] {
|
||||
for palettes in [1, 3, 64] {
|
||||
let mut bytes = vec![0x12; pixels.div_ceil(2)];
|
||||
bytes.extend(std::iter::repeat_n([0x12, 0x34, 0x56, 0x78], palettes * 16).flatten());
|
||||
let entry = AssetEntry {
|
||||
asset_id: 1,
|
||||
asset_name: "glyph".into(),
|
||||
bank_type: BankType::GLYPH,
|
||||
offset: 0,
|
||||
size: bytes.len() as u64,
|
||||
decoded_size: (pixels + palettes * 64) as u64,
|
||||
codec: AssetCodec::None,
|
||||
metadata: json!({"tile_size":16,"width":pixels,"height":1,"palette_count":palettes,"palette_authored":palettes}),
|
||||
};
|
||||
let expected = json!([
|
||||
16,
|
||||
pixels,
|
||||
1,
|
||||
(0..pixels).map(|i| 1 + i % 2).collect::<Vec<_>>(),
|
||||
vec![vec![0x12345678_u32; 16]; palettes]
|
||||
]);
|
||||
check_case("glyph-valid", &entry, bytes.clone(), Some(expected));
|
||||
for (key, value) in [
|
||||
("tile_size", json!(7)),
|
||||
("palette_count", json!(0)),
|
||||
("palette_count", json!(65)),
|
||||
("width", json!(-1)),
|
||||
("height", json!("bad")),
|
||||
] {
|
||||
let mut invalid = entry.clone();
|
||||
invalid.metadata[key] = value;
|
||||
check_case(key, &invalid, bytes.clone(), None);
|
||||
}
|
||||
let mut invalid = entry.clone();
|
||||
invalid.decoded_size += 1;
|
||||
check_case("glyph-decoded-size", &invalid, bytes.clone(), None);
|
||||
let mut invalid = entry.clone();
|
||||
invalid.size += 1;
|
||||
let mut extra = bytes.clone();
|
||||
extra.push(0);
|
||||
check_case("glyph-trailing", &invalid, extra, None);
|
||||
bytes.pop();
|
||||
check_case("glyph-truncated", &entry, bytes, None);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn empty_scene() -> (AssetEntry, Vec<u8>, Value) {
|
||||
let mut bytes = vec![0_u8; 140];
|
||||
bytes[..8].copy_from_slice(&[b'S', b'C', b'N', b'E', 1, 0, 4, 0]);
|
||||
for start in [12, 44, 76, 108] {
|
||||
bytes[start + 5] = 8;
|
||||
}
|
||||
let entry = AssetEntry {
|
||||
asset_id: 2,
|
||||
asset_name: "scene".into(),
|
||||
bank_type: BankType::SCENE,
|
||||
offset: 0,
|
||||
size: 140,
|
||||
decoded_size: 64,
|
||||
codec: AssetCodec::None,
|
||||
metadata: json!({}),
|
||||
};
|
||||
(entry, bytes, json!(vec![json!([false, 0, 8, 0.0, 0.0, 0, 0, []]); 4]))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scene_header_ranges_nonfinite_and_truncation_corpus() {
|
||||
let (entry, bytes, expected) = empty_scene();
|
||||
check_case("empty-scene", &entry, bytes.clone(), Some(expected));
|
||||
for end in 0..bytes.len() {
|
||||
let mut e = entry.clone();
|
||||
e.size = end as u64;
|
||||
check_case("scene-every-truncation", &e, bytes[..end].to_vec(), None);
|
||||
}
|
||||
for (offset, value) in [(0, 0xff), (4, 2), (6, 3), (17, 7), (36, 1)] {
|
||||
let mut invalid = bytes.clone();
|
||||
invalid[offset] = value;
|
||||
check_case("scene-invalid-header-field", &entry, invalid, None);
|
||||
}
|
||||
for offset in [20, 24] {
|
||||
for value in [f32::NAN, f32::INFINITY, f32::NEG_INFINITY] {
|
||||
let mut invalid = bytes.clone();
|
||||
invalid[offset..offset + 4].copy_from_slice(&value.to_le_bytes());
|
||||
check_case("scene-nonfinite", &entry, invalid, None);
|
||||
}
|
||||
}
|
||||
let mut invalid = bytes.clone();
|
||||
for offset in [28, 32, 36] {
|
||||
invalid[offset..offset + 4].copy_from_slice(&u32::MAX.to_le_bytes());
|
||||
}
|
||||
check_case("scene-dimensions-overflow-or-count-mismatch", &entry, invalid, None);
|
||||
let mut invalid = bytes.clone();
|
||||
for offset in [28, 32, 36] {
|
||||
invalid[offset..offset + 4].copy_from_slice(&1_u32.to_le_bytes());
|
||||
}
|
||||
// Four layer headers no longer fit after the declared tile consumes bytes.
|
||||
check_case("scene-tile-range", &entry, invalid, None);
|
||||
let mut sequence_metadata = entry.clone();
|
||||
sequence_metadata.metadata = json!([]);
|
||||
check_case(
|
||||
"scene-empty-sequence-metadata",
|
||||
&sequence_metadata,
|
||||
bytes.clone(),
|
||||
Some(empty_scene().2),
|
||||
);
|
||||
let mut e = entry.clone();
|
||||
e.decoded_size += 1;
|
||||
check_case("scene-decoded-size", &e, bytes.clone(), None);
|
||||
e = entry.clone();
|
||||
e.size += 1;
|
||||
let mut trailing = bytes.clone();
|
||||
trailing.push(0);
|
||||
check_case("scene-trailing", &e, trailing, None);
|
||||
for metadata in [json!(null), json!("bad")] {
|
||||
e = entry.clone();
|
||||
e.metadata = metadata;
|
||||
check_case("scene-metadata", &e, bytes.clone(), None);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scene_tile_chunk_edges_flags_and_reserved_bytes() {
|
||||
for count in [1_usize, 1023, 1024, 1025, 2049] {
|
||||
let (mut entry, mut bytes, mut expected) = empty_scene();
|
||||
bytes[12] = 0xff; // Only the active flag is meaningful.
|
||||
bytes[13..17].copy_from_slice(&(-7_i32).to_le_bytes());
|
||||
bytes[17] = 32;
|
||||
bytes[18..20].copy_from_slice(&[0xff; 2]); // Reserved fields are ignored.
|
||||
bytes[20..24].copy_from_slice(&(-0.5_f32).to_le_bytes());
|
||||
bytes[24..28].copy_from_slice(&2.0_f32.to_le_bytes());
|
||||
bytes[28..32].copy_from_slice(&(count as u32).to_le_bytes());
|
||||
bytes[32..36].copy_from_slice(&1_u32.to_le_bytes());
|
||||
bytes[36..40].copy_from_slice(&(count as u32).to_le_bytes());
|
||||
bytes.splice(44..44, std::iter::repeat_n([0xff, 255, 0x34, 0x12], count).flatten());
|
||||
entry.size = bytes.len() as u64;
|
||||
entry.decoded_size += (count * size_of::<Tile>()) as u64;
|
||||
expected[0] = json!([
|
||||
true,
|
||||
-7,
|
||||
32,
|
||||
-0.5,
|
||||
2.0,
|
||||
count,
|
||||
1,
|
||||
vec![(true, 0x1234, 255, true, true); count]
|
||||
]);
|
||||
check_case("scene-chunks", &entry, bytes, Some(expected));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pcm_metadata_and_source_range_corpus() {
|
||||
for metadata in [
|
||||
json!(null),
|
||||
json!({}),
|
||||
json!({"sample_rate":-1,"channels":1}),
|
||||
json!({"sample_rate":44100,"channels":"bad"}),
|
||||
] {
|
||||
let mut e = pcm_entry(2);
|
||||
e.metadata = metadata;
|
||||
check_case("pcm-metadata", &e, vec![0x34, 0x12], None);
|
||||
}
|
||||
// These fields are currently descriptive, not extra PCM layout constraints.
|
||||
let mut e = pcm_entry(2);
|
||||
e.metadata = json!({"sample_rate":0,"channels":0});
|
||||
e.decoded_size = 999;
|
||||
check_case(
|
||||
"pcm-permissive-metadata",
|
||||
&e,
|
||||
vec![0x34, 0x12],
|
||||
Some(json!([[[0x1234], 0, null, null]])),
|
||||
);
|
||||
for offset in [1, u64::MAX] {
|
||||
let mut e = pcm_entry(2);
|
||||
e.offset = offset;
|
||||
check_case("source-range", &e, vec![0x34, 0x12], None);
|
||||
}
|
||||
let mut e = pcm_entry(2);
|
||||
e.size = u64::MAX;
|
||||
check_case("source-size-overflow", &e, vec![0x34, 0x12], None);
|
||||
}
|
||||
|
||||
pub(in crate::asset) fn glyph_snapshot(bank: &GlyphBank) -> Value {
|
||||
json!([
|
||||
bank.tile_size as u8,
|
||||
bank.width,
|
||||
bank.height,
|
||||
bank.pixel_indices,
|
||||
bank.palettes
|
||||
.iter()
|
||||
.map(|p| p.iter().map(|c| c.raw()).collect::<Vec<_>>())
|
||||
.collect::<Vec<_>>()
|
||||
])
|
||||
}
|
||||
pub(in crate::asset) fn sound_snapshot(bank: &SoundBank) -> Value {
|
||||
json!(
|
||||
bank.samples
|
||||
.iter()
|
||||
.map(|s| (&s.data, s.sample_rate, s.loop_start, s.loop_end))
|
||||
.collect::<Vec<_>>()
|
||||
)
|
||||
}
|
||||
pub(in crate::asset) fn scene_snapshot(bank: &SceneBank) -> Value {
|
||||
json!(
|
||||
bank.layers
|
||||
.iter()
|
||||
.map(|l| json!([
|
||||
l.active,
|
||||
l.glyph_asset_id,
|
||||
l.tile_size as u8,
|
||||
l.parallax_factor.x,
|
||||
l.parallax_factor.y,
|
||||
l.tilemap.width,
|
||||
l.tilemap.height,
|
||||
l.tilemap
|
||||
.tiles
|
||||
.iter()
|
||||
.map(|t| (t.active, t.glyph.glyph_id, t.glyph.palette_id, t.flip_x, t.flip_y))
|
||||
.collect::<Vec<_>>()
|
||||
]))
|
||||
.collect::<Vec<_>>()
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shared_scene_arithmetic_rejects_overflow_before_allocating() {
|
||||
let (_, bytes, _) = empty_scene();
|
||||
let mut header = bytes[12..44].to_vec();
|
||||
header[16..20].copy_from_slice(&1_u32.to_le_bytes());
|
||||
header[20..24].copy_from_slice(&1_u32.to_le_bytes());
|
||||
header[24..28].copy_from_slice(&1_u32.to_le_bytes());
|
||||
assert!(scene_layer(&header, usize::MAX, u64::MAX, &mut 0).is_err());
|
||||
let mut decoded = usize::MAX;
|
||||
assert!(scene_layer(&header, 44, 48, &mut decoded).is_err());
|
||||
assert!(scene_layer(&header, 44, 47, &mut 0).is_err());
|
||||
assert!(scene_layer(&header[..31], 44, 48, &mut 0).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn physically_truncated_glyph_palette_is_rejected_by_both_modes() {
|
||||
let fixture = FileFixture::new("truncated-glyph", &[0; 63]);
|
||||
let entry = AssetEntry {
|
||||
asset_id: 3,
|
||||
asset_name: "empty-glyph".into(),
|
||||
bank_type: BankType::GLYPH,
|
||||
offset: 0,
|
||||
size: 64,
|
||||
decoded_size: 64,
|
||||
codec: AssetCodec::None,
|
||||
metadata: json!({"tile_size":8,"width":0,"height":0,"palette_count":1,"palette_authored":1}),
|
||||
};
|
||||
let source = fixture.source(64);
|
||||
assert!(AssetManager::perform_load_glyph_bank(&entry, source.clone()).is_err());
|
||||
assert!(resumable::conformance_result(&entry, &source).is_err());
|
||||
std::fs::write(&fixture.0, [0; 64]).unwrap();
|
||||
let expected = json!([8, 0, 0, [], vec![vec![0; 16]]]);
|
||||
assert_eq!(
|
||||
glyph_snapshot(&AssetManager::perform_load_glyph_bank(&entry, source.clone()).unwrap()),
|
||||
expected
|
||||
);
|
||||
assert_eq!(resumable::conformance_result(&entry, &source).unwrap(), expected);
|
||||
}
|
||||
@@ -246,7 +246,7 @@ impl Decoder {
|
||||
}
|
||||
}
|
||||
BankType::SOUNDS => {
|
||||
let sample_rate = entry.metadata_as_sound_bank()?.sample_rate;
|
||||
let sample_rate = decode::sound_rate(entry)?;
|
||||
State::Sound {
|
||||
sample_rate,
|
||||
data: Vec::with_capacity(entry.size as usize / 2),
|
||||
@@ -254,15 +254,9 @@ impl Decoder {
|
||||
}
|
||||
}
|
||||
BankType::SCENE => {
|
||||
entry.metadata_as_scene_bank()?;
|
||||
decode::scene_metadata(entry)?;
|
||||
State::Scene {
|
||||
layers: std::array::from_fn(|_| SceneLayer {
|
||||
active: false,
|
||||
glyph_asset_id: 0,
|
||||
tile_size: TileSize::Size8,
|
||||
parallax_factor: ParallaxFactor { x: 1.0, y: 1.0 },
|
||||
tilemap: TileMap { width: 0, height: 0, tiles: Vec::new() },
|
||||
}),
|
||||
layers: decode::empty_layers(),
|
||||
layer: 0,
|
||||
tiles_left: 0,
|
||||
header: false,
|
||||
@@ -279,25 +273,18 @@ impl Decoder {
|
||||
if *position < *packed {
|
||||
let n = (*packed - *position).min(bytes.len());
|
||||
self.reader.read_exact(&mut bytes[..n]).map_err(|e| e.to_string())?;
|
||||
for &v in &bytes[..n] {
|
||||
if bank.pixel_indices.len() < bank.width * bank.height {
|
||||
bank.pixel_indices.push(v >> 4)
|
||||
}
|
||||
if bank.pixel_indices.len() < bank.width * bank.height {
|
||||
bank.pixel_indices.push(v & 15)
|
||||
}
|
||||
}
|
||||
decode::append_pixels(
|
||||
&mut bank.pixel_indices,
|
||||
&bytes[..n],
|
||||
bank.width * bank.height,
|
||||
);
|
||||
*position += n;
|
||||
return Ok(None);
|
||||
}
|
||||
if *palette_position < *palette_bytes {
|
||||
let n = (*palette_bytes - *palette_position).min(bytes.len());
|
||||
self.reader.read_exact(&mut bytes[..n]).map_err(|e| e.to_string())?;
|
||||
for (i, rgba) in bytes[..n].as_chunks::<4>().0.iter().enumerate() {
|
||||
let at = *palette_position / 4 + i;
|
||||
bank.palettes[at / 16][at % 16] =
|
||||
Color::rgba(rgba[0], rgba[1], rgba[2], rgba[3]);
|
||||
}
|
||||
decode::write_palettes(&mut bank.palettes, *palette_position, &bytes[..n]);
|
||||
*palette_position += n;
|
||||
return Ok(None);
|
||||
}
|
||||
@@ -311,9 +298,7 @@ impl Decoder {
|
||||
if *remaining > 0 {
|
||||
let n = (*remaining).min(bytes.len());
|
||||
self.reader.read_exact(&mut bytes[..n]).map_err(|e| e.to_string())?;
|
||||
for pair in bytes[..n].as_chunks::<2>().0 {
|
||||
data.push(i16::from_le_bytes([pair[0], pair[1]]));
|
||||
}
|
||||
decode::append_pcm(data, &bytes[..n]);
|
||||
*remaining -= n;
|
||||
return Ok(None);
|
||||
}
|
||||
@@ -327,12 +312,7 @@ impl Decoder {
|
||||
self.reader
|
||||
.read_exact(&mut bytes[..SCENE_HEADER_BYTES_V1])
|
||||
.map_err(|e| e.to_string())?;
|
||||
if bytes[..4] != SCENE_PAYLOAD_MAGIC_V1
|
||||
|| u16::from_le_bytes([bytes[4], bytes[5]]) != SCENE_PAYLOAD_VERSION_V1
|
||||
|| u16::from_le_bytes([bytes[6], bytes[7]]) as usize != SCENE_LAYER_COUNT_V1
|
||||
{
|
||||
return Err("Invalid scene header".into());
|
||||
}
|
||||
decode::scene_header(&bytes[..SCENE_HEADER_BYTES_V1])?;
|
||||
self.position += SCENE_HEADER_BYTES_V1;
|
||||
*header = true;
|
||||
return Ok(None);
|
||||
@@ -341,17 +321,7 @@ impl Decoder {
|
||||
let n = (*tiles_left).min(bytes.len() / SCENE_TILE_RECORD_BYTES_V1);
|
||||
let count = n * SCENE_TILE_RECORD_BYTES_V1;
|
||||
self.reader.read_exact(&mut bytes[..count]).map_err(|e| e.to_string())?;
|
||||
for b in bytes[..count].as_chunks::<SCENE_TILE_RECORD_BYTES_V1>().0 {
|
||||
layers[*layer - 1].tilemap.tiles.push(Tile {
|
||||
active: b[0] & 1 != 0,
|
||||
glyph: Glyph {
|
||||
glyph_id: u16::from_le_bytes([b[2], b[3]]),
|
||||
palette_id: b[1],
|
||||
},
|
||||
flip_x: b[0] & 2 != 0,
|
||||
flip_y: b[0] & 4 != 0,
|
||||
});
|
||||
}
|
||||
decode::append_tiles(&mut layers[*layer - 1].tilemap.tiles, &bytes[..count]);
|
||||
*tiles_left -= n;
|
||||
self.position += count;
|
||||
return Ok(None);
|
||||
@@ -361,50 +331,18 @@ impl Decoder {
|
||||
.read_exact(&mut bytes[..SCENE_LAYER_HEADER_BYTES_V1])
|
||||
.map_err(|e| e.to_string())?;
|
||||
self.position += SCENE_LAYER_HEADER_BYTES_V1;
|
||||
let u32at = |at| u32::from_le_bytes(bytes[at..at + 4].try_into().unwrap());
|
||||
let width = u32at(16) as usize;
|
||||
let height = u32at(20) as usize;
|
||||
let count = u32at(24) as usize;
|
||||
let tile_size = match bytes[5] {
|
||||
8 => TileSize::Size8,
|
||||
16 => TileSize::Size16,
|
||||
32 => TileSize::Size32,
|
||||
_ => return Err("Invalid scene tile size".into()),
|
||||
};
|
||||
let x = f32::from_le_bytes(bytes[8..12].try_into().unwrap());
|
||||
let y = f32::from_le_bytes(bytes[12..16].try_into().unwrap());
|
||||
if !x.is_finite()
|
||||
|| !y.is_finite()
|
||||
|| width.checked_mul(height) != Some(count)
|
||||
|| count
|
||||
.checked_mul(SCENE_TILE_RECORD_BYTES_V1)
|
||||
.and_then(|n| n.checked_add(self.position))
|
||||
.is_none_or(|n| n as u64 > self.entry.size)
|
||||
{
|
||||
return Err("Invalid scene layer range".into());
|
||||
}
|
||||
*decoded = decoded
|
||||
.checked_add(SCENE_DECODED_LAYER_OVERHEAD_BYTES_V1)
|
||||
.and_then(|n| {
|
||||
count.checked_mul(size_of::<Tile>()).and_then(|b| n.checked_add(b))
|
||||
})
|
||||
.ok_or("Scene size overflow")?;
|
||||
layers[*layer] = SceneLayer {
|
||||
active: bytes[0] & 1 != 0,
|
||||
glyph_asset_id: i32::from_le_bytes(bytes[1..5].try_into().unwrap()),
|
||||
tile_size,
|
||||
parallax_factor: ParallaxFactor { x, y },
|
||||
tilemap: TileMap { width, height, tiles: Vec::with_capacity(count) },
|
||||
};
|
||||
layers[*layer] = decode::scene_layer(
|
||||
&bytes[..SCENE_LAYER_HEADER_BYTES_V1],
|
||||
self.position,
|
||||
self.entry.size,
|
||||
decoded,
|
||||
)?;
|
||||
let count = layers[*layer].tilemap.width * layers[*layer].tilemap.height;
|
||||
*layer += 1;
|
||||
*tiles_left = count;
|
||||
return Ok(None);
|
||||
}
|
||||
if self.position as u64 != self.entry.size
|
||||
|| *decoded as u64 != self.entry.decoded_size
|
||||
{
|
||||
return Err("Invalid scene payload size".into());
|
||||
}
|
||||
decode::scene_finish(&self.entry, self.position, *decoded)?;
|
||||
let State::Scene { layers, .. } = std::mem::replace(&mut self.state, State::Done)
|
||||
else {
|
||||
unreachable!()
|
||||
@@ -416,6 +354,30 @@ impl Decoder {
|
||||
}
|
||||
}
|
||||
|
||||
// Test-only access to the production state machine; no alternate decode rules.
|
||||
#[cfg(test)]
|
||||
pub(super) fn conformance_result(
|
||||
entry: &AssetEntry,
|
||||
source: &Arc<RwLock<AssetsPayloadSource>>,
|
||||
) -> Result<serde_json::Value, String> {
|
||||
use std::io::Seek;
|
||||
let mut decoder = Decoder::new(entry, source)?;
|
||||
for _ in 0..100 {
|
||||
let before = decoder.reader.stream_position().unwrap();
|
||||
let result = decoder.step();
|
||||
let after = decoder.reader.stream_position().unwrap();
|
||||
assert!(after >= before && after - before <= 4096, "unbounded or rewound decode step");
|
||||
if let Some(bank) = result? {
|
||||
return Ok(match bank {
|
||||
Decoded::Glyph(bank) => super::decode::tests::glyph_snapshot(&bank),
|
||||
Decoded::Sound(bank) => super::decode::tests::sound_snapshot(&bank),
|
||||
Decoded::Scene(bank) => super::decode::tests::scene_snapshot(&bank),
|
||||
});
|
||||
}
|
||||
}
|
||||
panic!("conformance fixture exceeded its step budget")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -556,4 +518,55 @@ mod tests {
|
||||
assert_eq!(manager.preloads_ready(), Ok(true));
|
||||
assert_eq!(manager.slot_info(SlotRef::audio(0)).asset_id, Some(1));
|
||||
}
|
||||
#[test]
|
||||
fn canceled_and_superseded_partial_decoders_never_publish() {
|
||||
use crate::memory_banks::SoundBankPoolAccess;
|
||||
let bytes = vec![0x12; 16384];
|
||||
let entry = AssetEntry {
|
||||
asset_id: 1,
|
||||
asset_name: "pcm".into(),
|
||||
bank_type: BankType::SOUNDS,
|
||||
offset: 0,
|
||||
size: bytes.len() as u64,
|
||||
decoded_size: bytes.len() as u64,
|
||||
codec: AssetCodec::None,
|
||||
metadata: serde_json::json!({"sample_rate":44100,"channels":1}),
|
||||
};
|
||||
let mut newer = entry.clone();
|
||||
newer.asset_id = 2;
|
||||
let banks = Arc::new(crate::memory_banks::MemoryBanks::new());
|
||||
let mut manager = AssetManager::new(
|
||||
vec![entry, newer],
|
||||
AssetsPayloadSource::from_bytes(bytes),
|
||||
banks.clone(),
|
||||
banks.clone(),
|
||||
banks.clone(),
|
||||
);
|
||||
let lane = Arc::new(ManualLane::default());
|
||||
assert!(manager.bind_background_work(lane.clone()));
|
||||
let handle = manager.load(1, 0).unwrap();
|
||||
assert!(lane.step());
|
||||
assert!(lane.step());
|
||||
assert_eq!(manager.status(handle), LoadStatus::LOADING);
|
||||
assert_eq!(manager.cancel(handle), AssetOpStatus::Ok);
|
||||
while lane.step() {}
|
||||
assert_eq!(manager.status(handle), LoadStatus::CANCELED);
|
||||
assert!(manager.sound_policy.staging.read().unwrap().is_empty());
|
||||
manager.apply_commits();
|
||||
assert!(banks.sound_bank_slot(0).is_none());
|
||||
let old = manager.load(1, 0).unwrap();
|
||||
assert!(lane.step());
|
||||
assert!(lane.step());
|
||||
let new = manager.load(2, 0).unwrap();
|
||||
assert_eq!(old, new); // Stable slot handle, different request generation.
|
||||
assert!(lane.step()); // Discard the in-flight old decoder.
|
||||
assert!(manager.sound_policy.staging.read().unwrap().is_empty());
|
||||
while lane.step() {}
|
||||
assert_eq!(manager.status(new), LoadStatus::READY);
|
||||
assert!(banks.sound_bank_slot(0).is_none());
|
||||
assert_eq!(manager.commit(new), AssetOpStatus::Ok);
|
||||
manager.apply_commits();
|
||||
assert_eq!(manager.slot_info(SlotRef::audio(0)).asset_id, Some(2));
|
||||
assert_eq!(banks.sound_bank_slot(0).unwrap().samples[0].data, vec![0x1212; 8192]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -203,7 +203,8 @@ impl AudioBridge for Audio {
|
||||
fn begin_frame(&mut self) { /* Staging may already contain debugger steps. */
|
||||
}
|
||||
fn finish_frame(&mut self) {
|
||||
self.transport.publish(std::mem::take(&mut self.commands), false);
|
||||
let commands = std::mem::replace(&mut self.commands, self.transport.take_buffer());
|
||||
self.transport.publish(commands, false);
|
||||
self.frame_commands_admitted = 0;
|
||||
}
|
||||
fn abort_frame(&mut self) {
|
||||
@@ -221,7 +222,7 @@ impl AudioBridge for Audio {
|
||||
if success {
|
||||
// Publish the earlier frame prefix first so a callback stop cannot
|
||||
// be undone by an old play on resume. Do not replenish frame quota.
|
||||
let mut commands = std::mem::take(&mut self.commands);
|
||||
let mut commands = std::mem::replace(&mut self.commands, self.transport.take_buffer());
|
||||
commands.append(&mut self.lifecycle_commands);
|
||||
self.transport.publish(commands, true);
|
||||
} else {
|
||||
|
||||
@@ -14,6 +14,7 @@ struct Mailbox {
|
||||
paused: bool,
|
||||
frame: Option<Vec<AudioCommand>>,
|
||||
lifecycle: Option<Vec<AudioCommand>>,
|
||||
recycled: [Option<Vec<AudioCommand>>; 4],
|
||||
}
|
||||
|
||||
pub struct AudioDelivery {
|
||||
@@ -36,7 +37,31 @@ impl AudioTransport {
|
||||
}
|
||||
|
||||
pub fn consume_silently(&self) {
|
||||
let _ = self.try_take();
|
||||
if let Some(delivery) = self.try_take() {
|
||||
for batch in [delivery.frame, delivery.lifecycle].into_iter().flatten() {
|
||||
self.recycle_commands(batch);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Main/service lane only: drop command owners and retain bounded empty storage.
|
||||
pub fn recycle_commands(&self, mut commands: Vec<AudioCommand>) {
|
||||
commands.clear();
|
||||
let mut state = self.0.lock().expect("audio mailbox");
|
||||
if let Some(slot) = state.recycled.iter_mut().find(|slot| slot.is_none()) {
|
||||
*slot = Some(commands);
|
||||
}
|
||||
// Excess empty buffers are destroyed here, never in the data callback.
|
||||
}
|
||||
|
||||
pub(super) fn take_buffer(&self) -> Vec<AudioCommand> {
|
||||
self.0
|
||||
.lock()
|
||||
.expect("audio mailbox")
|
||||
.recycled
|
||||
.iter_mut()
|
||||
.find_map(Option::take)
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
pub fn ready(&self) -> bool {
|
||||
@@ -84,3 +109,36 @@ mod tests {
|
||||
assert!(transport.try_take().is_some());
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod reuse_tests {
|
||||
use super::*;
|
||||
#[test]
|
||||
fn service_recycles_storage_and_releases_sample_owners() {
|
||||
use prometeu_hal::{LoopMode, sample::Sample};
|
||||
let transport = AudioTransport::default();
|
||||
let sample = Arc::new(Sample::new(48000, vec![1]));
|
||||
let weak = Arc::downgrade(&sample);
|
||||
let mut batch = Vec::with_capacity(1024);
|
||||
let allocation = batch.as_ptr();
|
||||
batch.push(AudioCommand::Play {
|
||||
sample,
|
||||
voice_id: 0,
|
||||
volume: 255,
|
||||
pan: 0,
|
||||
pitch: 1.0,
|
||||
priority: 0,
|
||||
loop_mode: LoopMode::Off,
|
||||
});
|
||||
transport.recycle_commands(batch);
|
||||
assert!(weak.upgrade().is_none());
|
||||
let reused = transport.take_buffer();
|
||||
assert_eq!(reused.as_ptr(), allocation);
|
||||
assert_eq!(reused.capacity(), 1024);
|
||||
assert!(reused.is_empty());
|
||||
for _ in 0..8 {
|
||||
transport.recycle_commands(Vec::with_capacity(16));
|
||||
}
|
||||
assert_eq!(transport.0.lock().unwrap().recycled.iter().flatten().count(), 4);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -65,6 +65,8 @@ struct Request {
|
||||
control: Arc<PublicationControl>,
|
||||
receiver: mpsc::Receiver<Completion>,
|
||||
admitted: Instant,
|
||||
watchdog_canceled_at: Option<Instant>,
|
||||
worker_error: Option<Error>,
|
||||
result: RequestResult,
|
||||
job: AsyncWorkJobId,
|
||||
released: bool,
|
||||
@@ -523,6 +525,8 @@ impl Session {
|
||||
control,
|
||||
receiver,
|
||||
admitted,
|
||||
watchdog_canceled_at: None,
|
||||
worker_error: None,
|
||||
result: RequestResult { id, state: RequestState::Pending, reason: RequestReason::None },
|
||||
job,
|
||||
released: !saves_payload,
|
||||
@@ -664,6 +668,8 @@ impl Session {
|
||||
control,
|
||||
receiver,
|
||||
admitted: now,
|
||||
watchdog_canceled_at: None,
|
||||
worker_error: None,
|
||||
result: RequestResult { id, state: RequestState::Pending, reason: RequestReason::None },
|
||||
job,
|
||||
released: false,
|
||||
@@ -686,27 +692,29 @@ impl Session {
|
||||
let Some(r) = &mut self.request else { return };
|
||||
match r.receiver.try_recv() {
|
||||
Ok(completion) => {
|
||||
r.worker_error = completion.result.error;
|
||||
let current =
|
||||
self.mode.is_some() && r.session == self.context && r.epoch == self.epoch;
|
||||
if current {
|
||||
self.slots[r.local].protected = false;
|
||||
if r.result.reason == RequestReason::None {
|
||||
if r.result.state == RequestState::Pending {
|
||||
r.result.state = match completion.result.durable {
|
||||
DurableResult::Success => RequestState::Success,
|
||||
DurableResult::Failure => RequestState::Failure,
|
||||
DurableResult::Indeterminate => RequestState::Indeterminate,
|
||||
};
|
||||
r.result.reason = completion
|
||||
.result
|
||||
.error
|
||||
.map_or(RequestReason::None, RequestReason::Storage);
|
||||
r.result.reason = if r.watchdog_canceled_at.is_some() {
|
||||
RequestReason::Timeout
|
||||
} else {
|
||||
r.worker_error.map_or(RequestReason::None, RequestReason::Storage)
|
||||
};
|
||||
if completion.result.durable == DurableResult::Success && r.saves_payload {
|
||||
let mut s = r.source.write().unwrap();
|
||||
s.descriptor =
|
||||
completion.card.current().slots[self.logical[r.local] as usize];
|
||||
s.dirty = false;
|
||||
}
|
||||
} // A timeout result is immutable; returned card requires explicit reconciliation.
|
||||
} // A restart escalation is immutable; returned card requires explicit reconciliation.
|
||||
if self.provisional && completion.result.durable != DurableResult::Success {
|
||||
self.boot_error =
|
||||
Some(completion.result.error.unwrap_or(Error::InvalidState));
|
||||
@@ -755,7 +763,10 @@ impl Session {
|
||||
return;
|
||||
}
|
||||
let elapsed = now.saturating_duration_since(r.admitted);
|
||||
if elapsed >= Duration::from_millis(500) {
|
||||
if elapsed >= Duration::from_millis(500) && !r.control.canceled() {
|
||||
// Main-lane cancellation cause is independent of the worker's durable
|
||||
// outcome. Do not relabel an earlier removal or termination cancel.
|
||||
r.watchdog_canceled_at = Some(now);
|
||||
r.control.cancel();
|
||||
}
|
||||
if elapsed >= Duration::from_secs(1) {
|
||||
@@ -788,6 +799,12 @@ impl Session {
|
||||
.map(|r| r.result)
|
||||
.ok_or(Error::InvalidRange)
|
||||
}
|
||||
/// Original worker error for native diagnostics, independent of guest deadline reason.
|
||||
/// Pending and RAM-only requests have no worker error. Retention follows query/release.
|
||||
pub fn request_storage_error(&self, context: u64, id: u64) -> Result<Option<Error>> {
|
||||
self.query(context, id)?;
|
||||
Ok(self.request.as_ref().filter(|r| r.id == id).and_then(|r| r.worker_error))
|
||||
}
|
||||
pub fn release_request(&mut self, context: u64, id: u64) -> Result<()> {
|
||||
self.query(context, id)?;
|
||||
if self.deferred.is_some() {
|
||||
|
||||
@@ -280,8 +280,157 @@ fn async_commit_protects_only_source_and_retains_result_without_occupying_lane()
|
||||
controller.stop().unwrap();
|
||||
}
|
||||
|
||||
// The sentinel executes after the save on the single lane. Wall time only bounds
|
||||
// a broken fixture; all watchdog observations use explicit simulated instants.
|
||||
fn finish_queued_save(lane: Arc<crate::AsyncWorkLane>) {
|
||||
let (tx, rx) = std::sync::mpsc::channel();
|
||||
lane.submit_asset_load(move |_| {
|
||||
tx.send(()).unwrap();
|
||||
crate::services::async_work::AsyncWorkJobOutcome::Completed
|
||||
})
|
||||
.unwrap();
|
||||
let mut controller =
|
||||
crate::AsyncWorkLaneController::start(crate::AsyncWorkLaneConfig::default(), lane);
|
||||
rx.recv_timeout(std::time::Duration::from_secs(5)).unwrap();
|
||||
controller.stop().unwrap();
|
||||
}
|
||||
|
||||
fn assert_guest_query(
|
||||
session: &mut super::super::session::Session,
|
||||
id: u64,
|
||||
state: i64,
|
||||
reason: i64,
|
||||
) {
|
||||
use prometeu_bytecode::Value;
|
||||
use prometeu_hal::{
|
||||
HostReturn, host_return::GuestStringAllocator, syscalls::Syscall, vm_fault::VmFault,
|
||||
};
|
||||
struct NoStrings;
|
||||
impl GuestStringAllocator for NoStrings {
|
||||
fn byte_limit(&self) -> u64 {
|
||||
0
|
||||
}
|
||||
fn build(
|
||||
&mut self,
|
||||
_: usize,
|
||||
_: &mut dyn FnMut(&mut dyn std::fmt::Write) -> std::fmt::Result,
|
||||
) -> std::result::Result<Arc<str>, VmFault> {
|
||||
panic!("MemQuery must not allocate strings")
|
||||
}
|
||||
}
|
||||
let mut service = super::super::MemcardService::new();
|
||||
std::mem::swap(session, &mut service.session);
|
||||
let args = [Value::Int64(service.session.context() as i64), Value::Int64(id as i64)];
|
||||
let mut values = Vec::with_capacity(Syscall::MemQuery.results_count());
|
||||
let mut strings = NoStrings;
|
||||
let mut ret = HostReturn::new(&mut values, Syscall::MemQuery.results_count(), &mut strings);
|
||||
service.guest(Syscall::MemQuery, &args, &mut ret, 4, 0, true).unwrap();
|
||||
assert!(!ret.overflowed());
|
||||
assert!(ret.failure().is_none());
|
||||
assert_eq!(values, vec![Value::Int64(0), Value::Int64(state), Value::Int64(reason)]);
|
||||
std::mem::swap(session, &mut service.session);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
|
||||
fn queued_memcard_watchdog_preserves_timeout_and_guest_reason() {
|
||||
use super::super::session::{RequestReason, RequestState, Session};
|
||||
use std::time::{Duration, Instant};
|
||||
for bootstrap in [false, true] {
|
||||
let (mut card, disk) = formatted();
|
||||
add(&mut card, 4, 1);
|
||||
let before = card.current().slots[0];
|
||||
let mut s = Session::default();
|
||||
let c = s.load_card(card, 4, 1, Session::required_bytes(1).unwrap()).unwrap();
|
||||
s.write(c, 0, 0, b"pending").unwrap();
|
||||
let lane = Arc::new(crate::AsyncWorkLane::default());
|
||||
let now = Instant::now();
|
||||
if bootstrap {
|
||||
s.begin_boot();
|
||||
}
|
||||
let id = s.commit(c, 0, 7, now, &lane).unwrap();
|
||||
if bootstrap {
|
||||
s.finish_boot(true, &lane, now).unwrap();
|
||||
}
|
||||
s.poll(now + Duration::from_millis(499));
|
||||
assert_guest_query(&mut s, id, 0, 0);
|
||||
for _ in 0..2 {
|
||||
s.poll(now + Duration::from_millis(500));
|
||||
}
|
||||
assert_guest_query(&mut s, id, 0, 0);
|
||||
assert_eq!(s.release_request(c, id), Err(Error::Busy));
|
||||
assert_eq!(s.clear(c, 0), Err(Error::Busy));
|
||||
finish_queued_save(lane);
|
||||
s.poll(now + Duration::from_millis(501));
|
||||
let result = s.query(c, id).unwrap();
|
||||
assert_eq!(result.state, RequestState::Failure);
|
||||
assert_eq!(result.reason, RequestReason::Timeout);
|
||||
assert_eq!(s.request_storage_error(c, id), Ok(Some(Error::Canceled)));
|
||||
assert_guest_query(&mut s, id, 2, 15);
|
||||
assert!(s.unsaved());
|
||||
assert_eq!(reboot(&disk).current().slots[0], before);
|
||||
assert_eq!(s.query(c + 1, id), Err(Error::AccessDenied));
|
||||
s.release_request(c, id).unwrap();
|
||||
assert_eq!(s.query(c, id), Err(Error::InvalidRange));
|
||||
s.clear(c, 0).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lifecycle_cancellation_is_not_a_watchdog_timeout() {
|
||||
use super::super::session::{RequestReason, RequestState, Session};
|
||||
use std::time::{Duration, Instant};
|
||||
for end in [false, true] {
|
||||
let (mut card, _) = formatted();
|
||||
add(&mut card, 4, 1);
|
||||
let mut s = Session::default();
|
||||
let c = s.load_card(card, 4, 1, Session::required_bytes(1).unwrap()).unwrap();
|
||||
let lane = Arc::new(crate::AsyncWorkLane::default());
|
||||
let now = Instant::now();
|
||||
let id = s.commit(c, 0, 7, now, &lane).unwrap();
|
||||
if end {
|
||||
s.end_session();
|
||||
} else {
|
||||
s.notify_removal();
|
||||
}
|
||||
s.poll(now + Duration::from_millis(500));
|
||||
finish_queued_save(lane);
|
||||
s.poll(now + Duration::from_millis(501));
|
||||
if end {
|
||||
assert_eq!(s.query(c, id), Err(Error::AccessDenied));
|
||||
let card = s.take_idle_card().unwrap().unwrap();
|
||||
let new = s.load_card(card, 4, 1, Session::required_bytes(1).unwrap()).unwrap();
|
||||
assert_ne!(new, c);
|
||||
assert_eq!(s.query(new, id), Err(Error::InvalidRange));
|
||||
} else {
|
||||
assert_eq!(s.query(c, id).unwrap().state, RequestState::Failure);
|
||||
assert_eq!(s.query(c, id).unwrap().reason, RequestReason::Storage(Error::Canceled));
|
||||
assert_guest_query(&mut s, id, 2, 8);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deferred_boot_memcard_deadline_stays_timeout() {
|
||||
use super::super::session::{RequestReason, Session};
|
||||
use std::time::{Duration, Instant};
|
||||
let (mut card, _) = formatted();
|
||||
add(&mut card, 4, 1);
|
||||
let mut s = Session::default();
|
||||
let c = s.load_card(card, 4, 1, Session::required_bytes(1).unwrap()).unwrap();
|
||||
s.begin_boot();
|
||||
let lane = crate::AsyncWorkLane::default();
|
||||
let now = Instant::now();
|
||||
let id = s.commit(c, 0, 7, now, &lane).unwrap();
|
||||
s.poll(now + Duration::from_millis(500));
|
||||
assert_eq!(s.query(c, id).unwrap().reason, RequestReason::Timeout);
|
||||
assert_guest_query(&mut s, id, 2, 15);
|
||||
assert_eq!(lane.pending_depth(), 0);
|
||||
s.clear(c, 0).unwrap();
|
||||
s.release_request(c, id).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn blocked_host_memcard_watchdog_preserves_outcome_and_escalation() {
|
||||
use crate::services::async_work::{
|
||||
AsyncWorkLane, AsyncWorkLaneConfig, AsyncWorkLaneController,
|
||||
};
|
||||
@@ -290,6 +439,7 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
|
||||
struct Blocked {
|
||||
store: Box<dyn MemcardStorage>,
|
||||
publication: bool,
|
||||
fail_publication: bool,
|
||||
entered: std::sync::mpsc::Sender<()>,
|
||||
resume: std::sync::mpsc::Receiver<()>,
|
||||
}
|
||||
@@ -310,12 +460,24 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
|
||||
if if self.publication { b.starts_with(b"PMMAP1\0\0") } else { o >= DATA_OFFSET } {
|
||||
self.entered.send(()).unwrap();
|
||||
self.resume.recv().unwrap();
|
||||
if self.fail_publication {
|
||||
return Err(MemcardIoError::IoError);
|
||||
}
|
||||
}
|
||||
self.store.write_at(o, b)
|
||||
}
|
||||
}
|
||||
for publication in [false, true] {
|
||||
let (mut card, _) = formatted();
|
||||
for (publication, fail_publication, escalate, release_early) in [
|
||||
(false, false, false, false),
|
||||
(true, false, false, false),
|
||||
(true, true, false, false),
|
||||
(false, false, true, false),
|
||||
(true, false, true, false),
|
||||
(true, true, true, false),
|
||||
(false, false, true, true),
|
||||
(true, false, true, true),
|
||||
] {
|
||||
let (mut card, disk) = formatted();
|
||||
add(&mut card, 4, 1);
|
||||
let (et, er) = std::sync::mpsc::channel();
|
||||
let (rt, rr) = std::sync::mpsc::channel();
|
||||
@@ -324,6 +486,7 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
|
||||
entered: et,
|
||||
resume: rr,
|
||||
publication,
|
||||
fail_publication,
|
||||
}))
|
||||
.unwrap();
|
||||
let mut s = Session::default();
|
||||
@@ -336,22 +499,84 @@ fn blocked_host_timeout_release_keeps_source_and_prevents_replacement_work() {
|
||||
let id = s.commit(c, 0, 7, admitted, &lane).unwrap();
|
||||
er.recv_timeout(Duration::from_secs(1)).unwrap();
|
||||
s.poll(admitted + Duration::from_millis(500));
|
||||
s.poll(admitted + Duration::from_secs(1));
|
||||
let result = s.query(c, id).unwrap();
|
||||
assert_eq!(
|
||||
result.state,
|
||||
if publication { RequestState::Indeterminate } else { RequestState::Failure }
|
||||
);
|
||||
assert_eq!(result.reason, RequestReason::RestartRequired);
|
||||
assert!(s.host_faulted());
|
||||
s.release_request(c, id).unwrap();
|
||||
assert_guest_query(&mut s, id, 0, 0);
|
||||
let terminal = if escalate {
|
||||
s.poll(admitted + Duration::from_secs(1));
|
||||
let result = s.query(c, id).unwrap();
|
||||
assert_eq!(
|
||||
result.state,
|
||||
if publication { RequestState::Indeterminate } else { RequestState::Failure }
|
||||
);
|
||||
assert_eq!(result.reason, RequestReason::RestartRequired);
|
||||
assert!(s.host_faulted());
|
||||
if release_early {
|
||||
s.release_request(c, id).unwrap();
|
||||
}
|
||||
Some(result)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
assert_eq!(s.write(c, 0, 0, b"unsafe"), Err(Error::Busy));
|
||||
assert_eq!(s.commit(c, 0, 7, Instant::now(), &lane), Err(Error::Busy));
|
||||
assert_eq!(s.commit(c, 0, 7, admitted, &lane), Err(Error::Busy));
|
||||
rt.send(()).unwrap();
|
||||
controller.stop().unwrap();
|
||||
s.poll(Instant::now());
|
||||
assert!(s.unsaved());
|
||||
assert!(s.host_faulted());
|
||||
s.poll(admitted + Duration::from_millis(if escalate { 1001 } else { 501 }));
|
||||
if let Some(terminal) = terminal {
|
||||
if release_early {
|
||||
assert_eq!(s.query(c, id), Err(Error::InvalidRange));
|
||||
} else {
|
||||
assert_eq!(s.query(c, id), Ok(terminal));
|
||||
assert_guest_query(&mut s, id, if publication { 3 } else { 2 }, 15);
|
||||
s.release_request(c, id).unwrap();
|
||||
}
|
||||
assert!(s.unsaved());
|
||||
assert!(s.host_faulted());
|
||||
assert_eq!(s.commit(c, 0, 7, admitted, &lane), Err(Error::ReadOnly));
|
||||
} else {
|
||||
let expected = if fail_publication {
|
||||
RequestState::Indeterminate
|
||||
} else if publication {
|
||||
RequestState::Success
|
||||
} else {
|
||||
RequestState::Failure
|
||||
};
|
||||
assert_eq!(s.query(c, id).unwrap().state, expected);
|
||||
assert_eq!(s.query(c, id).unwrap().reason, RequestReason::Timeout);
|
||||
assert_guest_query(
|
||||
&mut s,
|
||||
id,
|
||||
if fail_publication {
|
||||
3
|
||||
} else if publication {
|
||||
1
|
||||
} else {
|
||||
2
|
||||
},
|
||||
15,
|
||||
);
|
||||
assert_eq!(
|
||||
s.request_storage_error(c, id),
|
||||
Ok(if fail_publication {
|
||||
Some(Error::Io(MemcardIoError::IoError))
|
||||
} else if publication {
|
||||
None
|
||||
} else {
|
||||
Some(Error::Canceled)
|
||||
})
|
||||
);
|
||||
assert_eq!(s.unsaved(), expected != RequestState::Success);
|
||||
assert_eq!(s.removal_pending(), fail_publication);
|
||||
assert!(!s.host_faulted());
|
||||
s.release_request(c, id).unwrap();
|
||||
}
|
||||
let mut durable = reboot(&disk);
|
||||
let mut bytes = [0; 7];
|
||||
let n = durable.read_slot(0, &mut bytes).unwrap();
|
||||
if publication && !fail_publication {
|
||||
assert_eq!(&bytes[..n], b"pending");
|
||||
} else {
|
||||
assert_eq!(n, 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -660,3 +885,232 @@ fn ending_a_session_releases_a_completed_unconsumed_request() {
|
||||
assert!(s.take_idle_card().unwrap().is_some());
|
||||
controller.stop().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn persistent_save_preempts_real_multichunk_asset_decoders() {
|
||||
use crate::memcard::session::{RequestState, Session};
|
||||
use crate::{AsyncWorkLane, AsyncWorkLaneConfig, AsyncWorkLaneController};
|
||||
use prometeu_drivers::{
|
||||
AssetManager, GlyphBankPoolAccess, MemoryBanks, SceneBankPoolAccess, SoundBankPoolAccess,
|
||||
};
|
||||
use prometeu_hal::{
|
||||
AssetBridge,
|
||||
asset::{AssetCodec, AssetEntry, AssetOpStatus, BankType, LoadStatus},
|
||||
background_work::{AssetWork, BackgroundWork, WorkUnavailable},
|
||||
cartridge::AssetsPayloadSource,
|
||||
};
|
||||
use std::{
|
||||
sync::mpsc,
|
||||
thread,
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
type Events = Arc<Mutex<Vec<(&'static str, thread::ThreadId)>>>;
|
||||
struct ObservedLane {
|
||||
lane: Arc<AsyncWorkLane>,
|
||||
events: Events,
|
||||
paused: mpsc::Sender<()>,
|
||||
resume: Mutex<Option<mpsc::Receiver<()>>>,
|
||||
}
|
||||
impl BackgroundWork for ObservedLane {
|
||||
fn submit_asset(&self, mut work: AssetWork) -> std::result::Result<(), WorkUnavailable> {
|
||||
let events = self.events.clone();
|
||||
let paused = self.paused.clone();
|
||||
let resume = self.resume.lock().unwrap().take().unwrap();
|
||||
let mut step = 0;
|
||||
self.lane.submit_asset(Box::new(move |canceled| {
|
||||
let result = work(canceled);
|
||||
step += 1;
|
||||
events.lock().unwrap().push(("asset", thread::current().id()));
|
||||
if step == 4 {
|
||||
paused.send(()).unwrap();
|
||||
resume.recv_timeout(Duration::from_secs(10)).unwrap();
|
||||
}
|
||||
result
|
||||
}))
|
||||
}
|
||||
}
|
||||
struct ObservedStore {
|
||||
inner: Store,
|
||||
events: Events,
|
||||
}
|
||||
impl MemcardStorage for ObservedStore {
|
||||
fn len(&self) -> std::result::Result<u64, MemcardIoError> {
|
||||
self.inner.len()
|
||||
}
|
||||
fn read_only(&self) -> bool {
|
||||
false
|
||||
}
|
||||
fn read_at(
|
||||
&mut self,
|
||||
offset: u64,
|
||||
out: &mut [u8],
|
||||
) -> std::result::Result<(), MemcardIoError> {
|
||||
self.inner.read_at(offset, out)
|
||||
}
|
||||
fn write_at(
|
||||
&mut self,
|
||||
offset: u64,
|
||||
bytes: &[u8],
|
||||
) -> std::result::Result<(), MemcardIoError> {
|
||||
self.events.lock().unwrap().push(("save write", thread::current().id()));
|
||||
self.inner.write_at(offset, bytes)
|
||||
}
|
||||
fn sync(&mut self) -> std::result::Result<(), MemcardIoError> {
|
||||
self.events.lock().unwrap().push(("save sync", thread::current().id()));
|
||||
self.inner.sync()
|
||||
}
|
||||
}
|
||||
for kind in [BankType::GLYPH, BankType::SOUNDS, BankType::SCENE] {
|
||||
let (card, disk) = formatted();
|
||||
drop(card);
|
||||
let events: Events = Arc::default();
|
||||
let mut card = Card::mount(Box::new(ObservedStore {
|
||||
inner: Store(disk.clone()),
|
||||
events: events.clone(),
|
||||
}))
|
||||
.unwrap();
|
||||
add(&mut card, 1, 1);
|
||||
let mut session = Session::default();
|
||||
let context = session.load_card(card, 1, 1, 1024 * 1024).unwrap();
|
||||
session.write(context, 0, 0, b"persistent under asset pressure").unwrap();
|
||||
events.lock().unwrap().clear();
|
||||
let (bytes, decoded_size, metadata) = match kind {
|
||||
BankType::GLYPH => {
|
||||
let mut bytes = vec![0x12; 16384];
|
||||
bytes.extend([255; 64]);
|
||||
(
|
||||
bytes,
|
||||
32768 + 64,
|
||||
serde_json::json!({"tile_size":8,"width":256,"height":128,"palette_count":1,"palette_authored":1}),
|
||||
)
|
||||
}
|
||||
BankType::SOUNDS => {
|
||||
(vec![0x12; 16385], 16385, serde_json::json!({"sample_rate":44100,"channels":1}))
|
||||
}
|
||||
BankType::SCENE => {
|
||||
let mut bytes = vec![0; 140];
|
||||
bytes[..8].copy_from_slice(&[b'S', b'C', b'N', b'E', 1, 0, 4, 0]);
|
||||
for start in [12, 44, 76, 108] {
|
||||
bytes[start + 5] = 8;
|
||||
}
|
||||
bytes[12] = 1;
|
||||
bytes[28..32].copy_from_slice(&4097_u32.to_le_bytes());
|
||||
bytes[32..36].copy_from_slice(&1_u32.to_le_bytes());
|
||||
bytes[36..40].copy_from_slice(&4097_u32.to_le_bytes());
|
||||
bytes.splice(44..44, std::iter::repeat_n([7, 3, 0x34, 0x12], 4097).flatten());
|
||||
(
|
||||
bytes,
|
||||
64 + 4097 * size_of::<prometeu_hal::tile::Tile>() as u64,
|
||||
serde_json::json!({}),
|
||||
)
|
||||
}
|
||||
};
|
||||
let entry = AssetEntry {
|
||||
asset_id: 1,
|
||||
asset_name: "pressure".into(),
|
||||
bank_type: kind,
|
||||
offset: 0,
|
||||
size: bytes.len() as u64,
|
||||
decoded_size,
|
||||
codec: AssetCodec::None,
|
||||
metadata,
|
||||
};
|
||||
let banks = Arc::new(MemoryBanks::new());
|
||||
let mut manager = AssetManager::new(
|
||||
vec![entry],
|
||||
AssetsPayloadSource::from_bytes(bytes),
|
||||
banks.clone(),
|
||||
banks.clone(),
|
||||
banks.clone(),
|
||||
);
|
||||
let lane = Arc::new(AsyncWorkLane::default());
|
||||
let (paused_tx, paused_rx) = mpsc::channel();
|
||||
let (resume_tx, resume_rx) = mpsc::channel();
|
||||
let observed = Arc::new(ObservedLane {
|
||||
lane: lane.clone(),
|
||||
events: events.clone(),
|
||||
paused: paused_tx,
|
||||
resume: Mutex::new(Some(resume_rx)),
|
||||
});
|
||||
assert!(manager.bind_background_work(observed));
|
||||
let mut controller =
|
||||
AsyncWorkLaneController::start(AsyncWorkLaneConfig::default(), lane.clone());
|
||||
let handle = manager.load(1, 0).unwrap();
|
||||
paused_rx.recv_timeout(Duration::from_secs(10)).unwrap();
|
||||
assert_eq!(manager.status(handle), LoadStatus::LOADING);
|
||||
assert!(
|
||||
banks.glyph_bank_slot(0).is_none()
|
||||
&& banks.sound_bank_slot(0).is_none()
|
||||
&& banks.scene_bank_slot(0).is_none()
|
||||
);
|
||||
let request = session.commit(context, 0, 7, Instant::now(), &lane).unwrap();
|
||||
resume_tx.send(()).unwrap();
|
||||
let deadline = Instant::now() + Duration::from_secs(10);
|
||||
loop {
|
||||
session.poll(Instant::now());
|
||||
if session.query(context, request).unwrap().state != RequestState::Pending
|
||||
&& manager.status(handle) == LoadStatus::READY
|
||||
{
|
||||
break;
|
||||
}
|
||||
assert!(Instant::now() < deadline, "save/asset did not complete for {kind:?}");
|
||||
thread::yield_now();
|
||||
}
|
||||
assert_eq!(session.query(context, request).unwrap().state, RequestState::Success);
|
||||
assert!(
|
||||
banks.glyph_bank_slot(0).is_none()
|
||||
&& banks.sound_bank_slot(0).is_none()
|
||||
&& banks.scene_bank_slot(0).is_none()
|
||||
);
|
||||
assert_eq!(manager.commit(handle), AssetOpStatus::Ok);
|
||||
manager.apply_commits();
|
||||
match kind {
|
||||
BankType::GLYPH => {
|
||||
let bank = banks.glyph_bank_slot(0).unwrap();
|
||||
assert_eq!((bank.width, bank.height), (256, 128));
|
||||
assert_eq!(
|
||||
bank.pixel_indices,
|
||||
(0..32768).map(|i| 1 + (i % 2) as u8).collect::<Vec<_>>()
|
||||
);
|
||||
assert!(bank.palettes[0].iter().all(|c| c.raw() == 0xffffffff));
|
||||
}
|
||||
BankType::SOUNDS => {
|
||||
let bank = banks.sound_bank_slot(0).unwrap();
|
||||
assert_eq!(bank.samples[0].sample_rate, 44100);
|
||||
assert_eq!(bank.samples[0].data, vec![0x1212; 8192]);
|
||||
}
|
||||
BankType::SCENE => {
|
||||
let bank = banks.scene_bank_slot(0).unwrap();
|
||||
assert_eq!(
|
||||
(bank.layers[0].tilemap.width, bank.layers[0].tilemap.height),
|
||||
(4097, 1)
|
||||
);
|
||||
assert!(bank.layers[0].active);
|
||||
assert_eq!(bank.layers[0].tilemap.tiles.len(), 4097);
|
||||
assert!(bank.layers[0].tilemap.tiles.iter().all(|t| t.active
|
||||
&& t.flip_x
|
||||
&& t.flip_y
|
||||
&& t.glyph.glyph_id == 0x1234
|
||||
&& t.glyph.palette_id == 3));
|
||||
assert!(bank.layers[1..].iter().all(|l| l.tilemap.tiles.is_empty()));
|
||||
}
|
||||
}
|
||||
controller.stop().unwrap();
|
||||
let observed = events.lock().unwrap();
|
||||
assert_eq!(observed[..4].iter().map(|e| e.0).collect::<Vec<_>>(), vec!["asset"; 4]);
|
||||
assert_eq!(observed[4].0, "save write");
|
||||
let resumed = observed[4..].iter().position(|e| e.0 == "asset").unwrap() + 4;
|
||||
assert_eq!(observed[resumed - 1].0, "save sync");
|
||||
assert!(observed[resumed..].iter().all(|e| e.0 == "asset"));
|
||||
let worker = observed[0].1;
|
||||
assert_ne!(worker, thread::current().id());
|
||||
assert!(observed.iter().all(|e| e.1 == worker));
|
||||
drop(observed);
|
||||
drop(session);
|
||||
let mut recovered = Session::default();
|
||||
let c = recovered.load_card(reboot(&disk), 1, 1, 1024 * 1024).unwrap();
|
||||
let mut saved = [0; 64];
|
||||
let n = recovered.read(c, 0, 0, &mut saved).unwrap();
|
||||
assert_eq!(&saved[..n], b"persistent under asset pressure");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -18,3 +18,7 @@ fpmath = { version = "=0.1.1", features = ["soft-float"] }
|
||||
|
||||
[dev-dependencies]
|
||||
prometeu-test-support = { path = "../../dev/prometeu-test-support" }
|
||||
|
||||
[[example]]
|
||||
name = "string_provenance_benchmark"
|
||||
required-features = ["resource-diagnostics"]
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
//! Run in release with --features resource-diagnostics; query setup is excluded.
|
||||
#[allow(dead_code)]
|
||||
#[path = "../tests/support/callables.rs"]
|
||||
mod callables;
|
||||
use prometeu_bytecode::model::ConstantPoolEntry;
|
||||
use prometeu_vm::string_provenance_qualification::Fixture;
|
||||
use std::{
|
||||
alloc::{GlobalAlloc, Layout, System},
|
||||
sync::atomic::{AtomicUsize, Ordering},
|
||||
time::Instant,
|
||||
};
|
||||
struct Counting;
|
||||
static ALLOC: AtomicUsize = AtomicUsize::new(0);
|
||||
static REALLOC: AtomicUsize = AtomicUsize::new(0);
|
||||
static FREE: AtomicUsize = AtomicUsize::new(0);
|
||||
unsafe impl GlobalAlloc for Counting {
|
||||
unsafe fn alloc(&self, l: Layout) -> *mut u8 {
|
||||
ALLOC.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.alloc(l) }
|
||||
}
|
||||
unsafe fn dealloc(&self, p: *mut u8, l: Layout) {
|
||||
FREE.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.dealloc(p, l) }
|
||||
}
|
||||
unsafe fn realloc(&self, p: *mut u8, l: Layout, n: usize) -> *mut u8 {
|
||||
REALLOC.fetch_add(1, Ordering::Relaxed);
|
||||
unsafe { System.realloc(p, l, n) }
|
||||
}
|
||||
}
|
||||
#[global_allocator]
|
||||
static GLOBAL: Counting = Counting;
|
||||
fn counts() -> [usize; 3] {
|
||||
[ALLOC.load(Ordering::Relaxed), REALLOC.load(Ordering::Relaxed), FREE.load(Ordering::Relaxed)]
|
||||
}
|
||||
fn main() {
|
||||
for n in [1024, 16384, 65536] {
|
||||
let constants = vec![ConstantPoolEntry::String("same".into()); n];
|
||||
let start = Instant::now();
|
||||
let image = callables::image_constants(
|
||||
&[("RET", callables::sig(&[], &[]), vec![])],
|
||||
vec![],
|
||||
&constants,
|
||||
)
|
||||
.unwrap();
|
||||
let image_ns = start.elapsed().as_nanos();
|
||||
let payload = image.payload_bytes();
|
||||
let before = counts();
|
||||
let start = Instant::now();
|
||||
let mut fixture = Fixture::new(image, n);
|
||||
let creation_ns = start.elapsed().as_nanos();
|
||||
let after = counts();
|
||||
println!(
|
||||
"setup n={n} image_load_ns={image_ns} image_payload_bytes={payload} session_ns={creation_ns} session_allocs={} auxiliary_bytes={} work={:?}",
|
||||
after[0] - before[0],
|
||||
fixture.auxiliary_bytes(),
|
||||
fixture.work()
|
||||
);
|
||||
for name in [
|
||||
"image_hit",
|
||||
"image_miss",
|
||||
"session_hit",
|
||||
"session_miss",
|
||||
"import_hit",
|
||||
"snapshot_mixed",
|
||||
"import_miss",
|
||||
] {
|
||||
let before = counts();
|
||||
let start = Instant::now();
|
||||
for _ in 0..1000 {
|
||||
fixture.run(name);
|
||||
}
|
||||
let ns = start.elapsed().as_nanos();
|
||||
let after = counts();
|
||||
let delta = std::array::from_fn::<_, 3, _>(|i| after[i] - before[i]);
|
||||
if name.ends_with("hit") || name == "image_miss" || name == "session_miss" {
|
||||
assert_eq!(delta, [0; 3]);
|
||||
}
|
||||
println!(
|
||||
"query n={n} name={name} iterations=1000 ns={ns} alloc_realloc_free={delta:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1263,6 +1263,18 @@ pub(crate) fn metered_lookup_intrinsic(
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
/// Runtime-owned catalogue, including input and scalar math.
|
||||
pub fn intrinsic_catalog() -> impl Iterator<Item = &'static IntrinsicMeta> {
|
||||
INTRINSICS
|
||||
.iter()
|
||||
.chain(crate::input_intrinsics::INPUT_INTRINSICS.iter())
|
||||
.chain(crate::math_intrinsics::MATH_INTRINSICS.iter())
|
||||
.chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter())
|
||||
}
|
||||
pub fn builtin_constant_catalog() -> impl Iterator<Item = &'static BuiltinConstMeta> {
|
||||
BUILTIN_CONSTS.iter()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -1432,15 +1444,3 @@ mod tests {
|
||||
assert_eq!(pixel.field("color").map(|field| field.start_slot), Some(2));
|
||||
}
|
||||
}
|
||||
|
||||
/// Runtime-owned catalogue, including input and scalar math.
|
||||
pub fn intrinsic_catalog() -> impl Iterator<Item = &'static IntrinsicMeta> {
|
||||
INTRINSICS
|
||||
.iter()
|
||||
.chain(crate::input_intrinsics::INPUT_INTRINSICS.iter())
|
||||
.chain(crate::math_intrinsics::MATH_INTRINSICS.iter())
|
||||
.chain(crate::random_intrinsics::RANDOM_INTRINSICS.iter())
|
||||
}
|
||||
pub fn builtin_constant_catalog() -> impl Iterator<Item = &'static BuiltinConstMeta> {
|
||||
BUILTIN_CONSTS.iter()
|
||||
}
|
||||
|
||||
@@ -25,6 +25,7 @@ mod program_image;
|
||||
mod registry_verifier;
|
||||
pub use program_image::{ImageStringLease, ValidatedProgramImage};
|
||||
// Image/session creation and native provenance consume this in step 19.
|
||||
mod string_identity_index;
|
||||
#[allow(dead_code)]
|
||||
mod string_ledger;
|
||||
// Registry-backed consumers are integrated with executable images in PLN-0176.
|
||||
@@ -52,3 +53,6 @@ pub use virtual_machine::{BudgetReport, LogicalFrameEndingReason, VirtualMachine
|
||||
pub use vm_init_error::{LoaderPatchError, VmInitError};
|
||||
|
||||
pub use verifier::VerifierError;
|
||||
|
||||
#[cfg(feature = "resource-diagnostics")]
|
||||
pub use virtual_machine::string_provenance_qualification;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
//! PBX v1 publication: immutable metadata and shared payload, after bindings
|
||||
//! and bounded whole-program verification. No public unchecked constructor.
|
||||
use crate::string_identity_index::Identity;
|
||||
use crate::{LoaderPatchError, VmInitError};
|
||||
use prometeu_bytecode::isa::core::CoreOpCode as Op;
|
||||
use prometeu_bytecode::load_limits::{
|
||||
@@ -107,8 +108,30 @@ impl ValidatedProgramImage {
|
||||
Some(ImageStringLease { content: content.clone(), domain: self.strings.clone() })
|
||||
}
|
||||
pub fn retains_string(&self, content: &Arc<str>) -> bool {
|
||||
self.lookup_string(content, || {})
|
||||
}
|
||||
fn lookup_string(&self, content: &Arc<str>, mut compare: impl FnMut()) -> bool {
|
||||
let strings = self.strings.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
strings.records.iter().any(|(weak, _)| std::ptr::eq(weak.as_ptr(), Arc::as_ptr(content)))
|
||||
let target = Identity::of(content);
|
||||
let mut low = 0;
|
||||
let mut high = strings.records.len();
|
||||
while low < high {
|
||||
let mid = low + (high - low) / 2;
|
||||
compare();
|
||||
match record_identity(&strings.records[mid]).cmp(&target) {
|
||||
std::cmp::Ordering::Less => low = mid + 1,
|
||||
std::cmp::Ordering::Greater => high = mid,
|
||||
std::cmp::Ordering::Equal => return true,
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
#[cfg(any(test, feature = "resource-diagnostics"))]
|
||||
pub(crate) fn lookup_work(&self, content: &Arc<str>) -> (bool, usize) {
|
||||
let mut work = 0;
|
||||
let found = self.lookup_string(content, || work += 1);
|
||||
(found, work)
|
||||
}
|
||||
pub fn constant_content(&self) -> (u64, usize) {
|
||||
let mut strings = self.strings.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
@@ -205,6 +228,7 @@ impl ValidatedProgramImage {
|
||||
};
|
||||
values.push(value);
|
||||
}
|
||||
sort_string_records(&mut records, b)?;
|
||||
b.refund(Q::TemporaryBytes, constants_capacity);
|
||||
// Binding no longer needs source declarations. They were validated,
|
||||
// used exactly, and can be released before final payload publication.
|
||||
@@ -256,6 +280,48 @@ impl ValidatedProgramImage {
|
||||
}
|
||||
}
|
||||
|
||||
fn record_identity(record: &(Weak<str>, usize)) -> Identity {
|
||||
Identity { address: record.0.as_ptr() as *const u8 as usize, length: record.1 }
|
||||
}
|
||||
|
||||
// In-place heapsort: every record comparison (four field reads + comparison)
|
||||
// and swap (two record reads + writes) is precharged. No scratch allocation.
|
||||
fn sort_string_records(records: &mut [(Weak<str>, usize)], b: &mut LoadBudget) -> Result<(), E> {
|
||||
fn sift(
|
||||
records: &mut [(Weak<str>, usize)],
|
||||
mut root: usize,
|
||||
end: usize,
|
||||
b: &mut LoadBudget,
|
||||
) -> Result<(), E> {
|
||||
while root < end / 2 {
|
||||
let mut child = root * 2 + 1;
|
||||
if child + 1 < end {
|
||||
b.work(5)?;
|
||||
if record_identity(&records[child]) < record_identity(&records[child + 1]) {
|
||||
child += 1;
|
||||
}
|
||||
}
|
||||
b.work(5)?;
|
||||
if record_identity(&records[root]) >= record_identity(&records[child]) {
|
||||
break;
|
||||
}
|
||||
b.work(4)?;
|
||||
records.swap(root, child);
|
||||
root = child;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
for root in (0..records.len() / 2).rev() {
|
||||
sift(records, root, records.len(), b)?;
|
||||
}
|
||||
for end in (1..records.len()).rev() {
|
||||
b.work(4)?;
|
||||
records.swap(0, end);
|
||||
sift(records, 0, end, b)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn scratch<T>(count: usize, b: &mut LoadBudget) -> Result<Vec<T>, E> {
|
||||
let bytes = count
|
||||
.checked_mul(size_of::<T>())
|
||||
|
||||
@@ -309,3 +309,65 @@ fn sealed_image_load_probe() {
|
||||
elapsed
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn string_identity_binary_search_preserves_constant_order_and_lease_indices() {
|
||||
let constants = (0..1024).map(|i| K::String(format!("owner-{i}"))).collect::<Vec<_>>();
|
||||
let bytes = fixture(&[1, 0], 0, &constants, None, &[]);
|
||||
let image = ValidatedProgramImage::load(&bytes, &mut budget(), caps::NONE).unwrap();
|
||||
for (i, value) in image.constants().iter().enumerate() {
|
||||
let Value::String(text) = value else { panic!() };
|
||||
assert_eq!(&**text, format!("owner-{i}"));
|
||||
let (found, work) = image.lookup_work(text);
|
||||
assert!(found && work <= 11, "binary work={work}");
|
||||
assert!(Arc::ptr_eq(text, image.lease_constant(i).unwrap().content()));
|
||||
assert!(!image.lookup_work(&Arc::from(text.as_ref())).0);
|
||||
}
|
||||
let lease = image.lease_constant(1023).unwrap();
|
||||
drop(image);
|
||||
assert_eq!(lease.retained_content(), (10, 1));
|
||||
let domain = lease.domain.lock().unwrap();
|
||||
assert_eq!(record_identity(&domain.records[0]), Identity::of(lease.content()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn string_heap_sort_precharges_each_comparison_and_swap_without_scratch() {
|
||||
let owners = (0..16).map(|_| Arc::<str>::from("same")).collect::<Vec<_>>();
|
||||
let source = owners.iter().rev().map(|v| (Arc::downgrade(v), v.len())).collect::<Vec<_>>();
|
||||
let mut full = budget();
|
||||
let mut records = source.clone();
|
||||
sort_string_records(&mut records, &mut full).unwrap();
|
||||
assert!(records.windows(2).all(|v| record_identity(&v[0]) < record_identity(&v[1])));
|
||||
assert_eq!(full.used(Q::TemporaryBytes), 0);
|
||||
for limit in 0..full.used(Q::Work) {
|
||||
let mut b = LoadBudget::new(LoadLimits { work: limit, ..Default::default() });
|
||||
let mut records = source.clone();
|
||||
let error = sort_string_records(&mut records, &mut b).unwrap_err();
|
||||
assert_eq!(error.resource, Some(Q::Work));
|
||||
assert!(b.used(Q::Work) <= limit);
|
||||
assert_eq!(b.used(Q::TemporaryBytes), 0);
|
||||
// Failed sort still owns exactly the original Weak identities.
|
||||
records.sort_unstable_by_key(record_identity);
|
||||
let mut expected = source.clone();
|
||||
expected.sort_unstable_by_key(record_identity);
|
||||
assert!(records.iter().zip(&expected).all(|(a, b)| Weak::ptr_eq(&a.0, &b.0)));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn string_sort_exhaustion_rolls_back_sealed_load_domains() {
|
||||
let constants = vec![K::String("same".into()); 2];
|
||||
let bytes = fixture(&[1, 0], 0, &constants, None, &[]);
|
||||
let mut full = budget();
|
||||
ValidatedProgramImage::load(&bytes, &mut full, caps::NONE).unwrap();
|
||||
// Post-sort sharing costs 2 + 2 + 1 work units. Sorting two records
|
||||
// costs 9 or 13 units, depending on their addresses. Removing another
|
||||
// five leaves at most eight: failure is inside sorting for either order.
|
||||
let mut b =
|
||||
LoadBudget::new(LoadLimits { work: full.used(Q::Work) - 5 - 5, ..Default::default() });
|
||||
let error = ValidatedProgramImage::load(&bytes, &mut b, caps::NONE).unwrap_err();
|
||||
assert!(matches!(error,VmInitError::RegistryLoadFailed(e) if e.resource==Some(Q::Work)));
|
||||
for resource in DOMAINS {
|
||||
assert_eq!(b.used(resource), 0, "{resource:?}");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -385,6 +385,10 @@ impl<T> BudgetVec<T> {
|
||||
let bytes = b.bytes::<T>(capacity, q)?;
|
||||
let mut extra = b.reserve(q, bytes - self.charge.amount)?;
|
||||
let temporary = b.reserve(RuntimeResource::TemporaryBytes, self.charge.amount)?;
|
||||
#[cfg(test)]
|
||||
if FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.replace(false)) {
|
||||
return Err(b.failure(q, ResourceReason::AllocationFailed, bytes));
|
||||
}
|
||||
let mut candidate = Vec::new();
|
||||
candidate
|
||||
.try_reserve_exact(capacity)
|
||||
@@ -405,6 +409,21 @@ impl<T> BudgetVec<T> {
|
||||
self.charge.absorb(growth.extra);
|
||||
}
|
||||
}
|
||||
/// Rebuild a table with all fallible capacity/coexistence admission completed
|
||||
/// before moving any old entry. The relocation callback must not allocate.
|
||||
pub(crate) fn rebuild_with(
|
||||
&mut self,
|
||||
len: usize,
|
||||
vacant: impl FnMut() -> T,
|
||||
relocate: impl FnOnce(&mut [T], &mut [T]),
|
||||
) -> Result<(), ResourceFailure> {
|
||||
let mut growth = self.prepare_capacity(len.max(self.data.capacity()))?;
|
||||
growth.candidate.resize_with(len, vacant);
|
||||
relocate(&mut self.data, &mut growth.candidate);
|
||||
self.data = growth.candidate;
|
||||
self.charge.absorb(growth.extra);
|
||||
Ok(())
|
||||
}
|
||||
pub fn reserve(&mut self, additional: usize) -> Result<(), ResourceFailure> {
|
||||
let growth = self.prepare_growth(additional)?;
|
||||
self.commit_growth(growth);
|
||||
@@ -445,9 +464,6 @@ impl<T> BudgetVec<T> {
|
||||
pub fn clear(&mut self) {
|
||||
self.data.clear();
|
||||
}
|
||||
pub(crate) fn retain(&mut self, keep: impl FnMut(&T) -> bool) {
|
||||
self.data.retain(keep);
|
||||
}
|
||||
pub(crate) fn extend_from_slice(&mut self, values: &[T]) -> Result<(), ResourceFailure>
|
||||
where
|
||||
T: Clone,
|
||||
@@ -509,3 +525,8 @@ pub(crate) struct VecGrowth<T> {
|
||||
_temporary: Reservation,
|
||||
old_capacity: usize,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
thread_local! {
|
||||
pub(crate) static FAIL_NEXT_VEC_ALLOCATION: std::cell::Cell<bool> = const { std::cell::Cell::new(false) };
|
||||
}
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
//! Private allocation identity index. Values retain Weak ownership, never content.
|
||||
use crate::resource_budget::{
|
||||
BudgetVec, ResourceFailure, ResourceReason, RuntimeBudget, RuntimeResource as Q,
|
||||
};
|
||||
use std::sync::Arc;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub(crate) struct Identity {
|
||||
pub(crate) address: usize,
|
||||
pub(crate) length: usize,
|
||||
}
|
||||
impl Identity {
|
||||
pub(crate) fn of(content: &Arc<str>) -> Self {
|
||||
Self { address: Arc::as_ptr(content) as *const u8 as usize, length: content.len() }
|
||||
}
|
||||
fn hash(self) -> usize {
|
||||
// Mix aligned addresses and lengths; collisions still compare both fields.
|
||||
let mut n = (self.address as u64) ^ (self.length as u64).wrapping_mul(0x9e3779b97f4a7c15);
|
||||
n = (n ^ (n >> 30)).wrapping_mul(0xbf58476d1ce4e5b9);
|
||||
n = (n ^ (n >> 27)).wrapping_mul(0x94d049bb133111eb);
|
||||
(n ^ (n >> 31)) as usize
|
||||
}
|
||||
}
|
||||
#[derive(Debug)]
|
||||
enum Bucket<T> {
|
||||
Empty,
|
||||
Tombstone,
|
||||
Occupied(Identity, T),
|
||||
}
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct IdentityIndex<T> {
|
||||
buckets: BudgetVec<Bucket<T>>,
|
||||
live: usize,
|
||||
tombstones: usize,
|
||||
}
|
||||
impl<T> IdentityIndex<T> {
|
||||
pub(crate) fn new(budget: &RuntimeBudget) -> Self {
|
||||
Self { buckets: BudgetVec::new(budget, Q::AuxiliaryBytes), live: 0, tombstones: 0 }
|
||||
}
|
||||
pub(crate) fn len(&self) -> usize {
|
||||
self.live
|
||||
}
|
||||
pub(crate) fn contains(&self, key: Identity, mut probe: impl FnMut()) -> bool {
|
||||
let size = self.buckets.len();
|
||||
if size == 0 {
|
||||
return false;
|
||||
}
|
||||
let mut at = key.hash() & (size - 1);
|
||||
loop {
|
||||
probe();
|
||||
match &self.buckets[at] {
|
||||
Bucket::Empty => return false,
|
||||
Bucket::Occupied(found, _) if *found == key => return true,
|
||||
_ => at = (at + 1) & (size - 1),
|
||||
}
|
||||
}
|
||||
}
|
||||
pub(crate) fn prepare(&mut self, budget: &RuntimeBudget) -> Result<(), ResourceFailure> {
|
||||
let size = self.buckets.len();
|
||||
let next = self.live + 1;
|
||||
let capacity = if size == 0 {
|
||||
8
|
||||
} else if next > size / 2 {
|
||||
size.checked_mul(2).ok_or_else(|| {
|
||||
budget.failure(Q::AuxiliaryBytes, ResourceReason::SizeOverflow, u64::MAX)
|
||||
})?
|
||||
} else if next + self.tombstones > size / 2 {
|
||||
size
|
||||
} else {
|
||||
return Ok(());
|
||||
};
|
||||
self.buckets.rebuild_with(
|
||||
capacity,
|
||||
|| Bucket::Empty,
|
||||
|old, new| {
|
||||
for slot in old {
|
||||
if let Bucket::Occupied(key, value) = std::mem::replace(slot, Bucket::Empty) {
|
||||
Self::place(new, key, value);
|
||||
}
|
||||
}
|
||||
},
|
||||
)?;
|
||||
self.tombstones = 0;
|
||||
Ok(())
|
||||
}
|
||||
fn place(buckets: &mut [Bucket<T>], key: Identity, value: T) -> bool {
|
||||
let mut at = key.hash() & (buckets.len() - 1);
|
||||
loop {
|
||||
match &buckets[at] {
|
||||
Bucket::Empty | Bucket::Tombstone => {
|
||||
let reused = matches!(buckets[at], Bucket::Tombstone);
|
||||
buckets[at] = Bucket::Occupied(key, value);
|
||||
return reused;
|
||||
}
|
||||
Bucket::Occupied(found, _) => {
|
||||
assert_ne!(*found, key, "duplicate allocation identity")
|
||||
}
|
||||
}
|
||||
at = (at + 1) & (buckets.len() - 1);
|
||||
}
|
||||
}
|
||||
pub(crate) fn insert(&mut self, key: Identity, value: T) {
|
||||
assert!(self.live < self.buckets.len() / 2, "prepare must precede publication");
|
||||
if Self::place(&mut self.buckets, key, value) {
|
||||
self.tombstones -= 1;
|
||||
}
|
||||
self.live += 1;
|
||||
}
|
||||
pub(crate) fn retain(&mut self, mut keep: impl FnMut(&T) -> bool) {
|
||||
for slot in &mut *self.buckets {
|
||||
if matches!(slot, Bucket::Occupied(_, value) if !keep(value)) {
|
||||
// Erase the key before releasing the Weak/reservations. Its
|
||||
// allocation address cannot be recycled while Weak survives.
|
||||
let removed = std::mem::replace(slot, Bucket::Tombstone);
|
||||
self.live -= 1;
|
||||
self.tombstones += 1;
|
||||
drop(removed);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
@@ -0,0 +1,72 @@
|
||||
use super::*;
|
||||
#[test]
|
||||
fn collision_chains_distinguish_full_identity_and_survive_tombstones() {
|
||||
let b = RuntimeBudget::default();
|
||||
let mut table = IdentityIndex::new(&b);
|
||||
let keys: Vec<_> = (0..1000)
|
||||
.map(|address| Identity { address, length: 4 })
|
||||
.filter(|k| k.hash() & 7 == 0)
|
||||
.take(4)
|
||||
.collect();
|
||||
for (i, key) in keys.iter().enumerate() {
|
||||
table.prepare(&b).unwrap();
|
||||
table.insert(*key, i);
|
||||
}
|
||||
for key in &keys {
|
||||
assert!(table.contains(*key, || {}));
|
||||
}
|
||||
table.retain(|i| *i != 1);
|
||||
assert!(!table.contains(keys[1], || {}));
|
||||
assert!(table.contains(keys[3], || {}));
|
||||
assert!(!table.contains(Identity { length: 5, ..keys[3] }, || {}));
|
||||
table.prepare(&b).unwrap(); // Rebuild removes tombstones at half occupancy.
|
||||
table.insert(keys[1], 1);
|
||||
assert_eq!(table.len(), 4);
|
||||
assert_eq!(table.tombstones, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tombstone_reuse_keeps_collision_successors_and_length_is_part_of_key() {
|
||||
let b = RuntimeBudget::default();
|
||||
let mut t = IdentityIndex::new(&b);
|
||||
let key = Identity { address: 16, length: 0 };
|
||||
t.prepare(&b).unwrap();
|
||||
t.insert(key, 1);
|
||||
t.prepare(&b).unwrap();
|
||||
t.insert(Identity { length: 1, ..key }, 2);
|
||||
t.retain(|v| *v != 1);
|
||||
assert_eq!(t.tombstones, 1);
|
||||
t.prepare(&b).unwrap();
|
||||
t.insert(key, 3);
|
||||
assert_eq!(t.tombstones, 0);
|
||||
assert!(t.contains(key, || {}));
|
||||
assert!(t.contains(Identity { length: 1, ..key }, || {}));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn controlled_probe_counts_and_half_load_survive_growth_and_rebuild() {
|
||||
let b = RuntimeBudget::default();
|
||||
let mut table = IdentityIndex::new(&b);
|
||||
for n in 0..1024 {
|
||||
table.prepare(&b).unwrap();
|
||||
table.insert(Identity { address: n * 16, length: n % 7 }, n);
|
||||
assert!(table.live * 2 <= table.buckets.len());
|
||||
}
|
||||
let mut max_probes = 0;
|
||||
for n in 0..2048 {
|
||||
let mut probes = 0;
|
||||
assert_eq!(
|
||||
table.contains(Identity { address: n * 16, length: n % 7 }, || probes += 1),
|
||||
n < 1024
|
||||
);
|
||||
max_probes = max_probes.max(probes);
|
||||
}
|
||||
// A reproducible synthetic-key bound for this fixture, not a hash-table
|
||||
// worst-case claim or a constraint on allocator-selected real addresses.
|
||||
assert!(max_probes <= 32, "fixture probes={max_probes}");
|
||||
table.retain(|n| n % 2 == 0);
|
||||
table.prepare(&b).unwrap();
|
||||
for n in 0..1024 {
|
||||
assert_eq!(table.contains(Identity { address: n * 16, length: n % 7 }, || {}), n % 2 == 0);
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@
|
||||
use crate::resource_budget::{
|
||||
BudgetVec, Reservation, ResourceFailure, ResourceReason, RuntimeBudget, RuntimeResource as Q,
|
||||
};
|
||||
use crate::string_identity_index::{Identity, IdentityIndex};
|
||||
use prometeu_bytecode::Value;
|
||||
use std::sync::{Arc, Mutex, MutexGuard, Weak};
|
||||
|
||||
@@ -17,7 +18,7 @@ struct Entry {
|
||||
#[derive(Debug)]
|
||||
struct Domain {
|
||||
budget: RuntimeBudget,
|
||||
entries: BudgetVec<Entry>,
|
||||
entries: IdentityIndex<Entry>,
|
||||
_state: Reservation,
|
||||
}
|
||||
|
||||
@@ -49,7 +50,7 @@ impl StringLedger {
|
||||
Q::AuxiliaryBytes,
|
||||
budget.bytes::<Mutex<Domain>>(1, Q::AuxiliaryBytes)? + 2 * size_of::<usize>() as u64,
|
||||
)?;
|
||||
let entries = BudgetVec::new(&budget, Q::AuxiliaryBytes);
|
||||
let entries = IdentityIndex::new(&budget);
|
||||
Ok(Self(Arc::new(Mutex::new(Domain { budget, entries, _state: state }))))
|
||||
}
|
||||
fn domain(&self) -> MutexGuard<'_, Domain> {
|
||||
@@ -57,7 +58,7 @@ impl StringLedger {
|
||||
}
|
||||
|
||||
/// Safepoint reconciliation is bounded by the accepted allocation count.
|
||||
/// Retain compacts in place; unused capacity remains charged until drop.
|
||||
/// Dead buckets become tombstones; retained capacity stays charged until drop.
|
||||
pub(crate) fn reconcile(&self) -> usize {
|
||||
self.domain().reconcile()
|
||||
}
|
||||
@@ -65,9 +66,19 @@ impl StringLedger {
|
||||
self.domain().entries.len()
|
||||
}
|
||||
pub(crate) fn retains(&self, content: &Arc<str>) -> bool {
|
||||
let address = Arc::as_ptr(content);
|
||||
self.domain().entries.iter().any(|entry| std::ptr::eq(entry.content.as_ptr(), address))
|
||||
self.lookup(content, || {})
|
||||
}
|
||||
fn lookup(&self, content: &Arc<str>, mut probe: impl FnMut()) -> bool {
|
||||
self.domain().entries.contains(Identity::of(content), &mut probe)
|
||||
}
|
||||
|
||||
#[cfg(any(test, feature = "resource-diagnostics"))]
|
||||
pub(crate) fn lookup_work(&self, content: &Arc<str>) -> (bool, usize) {
|
||||
let mut work = 0;
|
||||
let found = self.lookup(content, || work += 1);
|
||||
(found, work)
|
||||
}
|
||||
|
||||
pub(crate) fn lease(&self, content: &Arc<str>) -> Option<StringLease> {
|
||||
self.retains(content)
|
||||
.then(|| StringLease { content: content.clone(), domain: self.clone() })
|
||||
@@ -239,23 +250,24 @@ impl Domain {
|
||||
}
|
||||
Err(failure) => return Err(failure),
|
||||
};
|
||||
if let Err(failure) = self.entries.reserve(1) {
|
||||
if let Err(failure) = self.entries.prepare(&self.budget) {
|
||||
if failure.reason != ResourceReason::Limit {
|
||||
return Err(failure);
|
||||
}
|
||||
self.reconcile();
|
||||
self.entries.reserve(1)?;
|
||||
self.entries.prepare(&self.budget)?;
|
||||
}
|
||||
Ok(charges)
|
||||
}
|
||||
fn publish(&mut self, content: &Arc<str>, charges: Charges) {
|
||||
self.entries
|
||||
.push(Entry {
|
||||
self.entries.insert(
|
||||
Identity::of(content),
|
||||
Entry {
|
||||
content: Arc::downgrade(content),
|
||||
_bytes: charges.bytes,
|
||||
_allocation: charges.allocation,
|
||||
})
|
||||
.expect("prepare reserved one ledger record before materialization");
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -259,3 +259,95 @@ fn formatting_refuses_result_and_temporary_limits_with_no_content_publication()
|
||||
assert_eq!(budget.used(Q::StringAllocations), 0);
|
||||
assert_eq!(ledger.tracked_allocations(), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn index_growth_refuses_auxiliary_temporary_and_allocator_before_materialization() {
|
||||
for failure in [0, 1, 2] {
|
||||
let (budget, ledger) = fixture(&[]);
|
||||
let kept = (0..4).map(|_| ledger.import("kept").unwrap()).collect::<Vec<_>>();
|
||||
let auxiliary = budget.used(Q::AuxiliaryBytes);
|
||||
match failure {
|
||||
0 => budget.test_lower_limit(Q::AuxiliaryBytes, auxiliary),
|
||||
1 => budget.test_lower_limit(Q::TemporaryBytes, 0),
|
||||
_ => crate::resource_budget::FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.set(true)),
|
||||
}
|
||||
let mut called = false;
|
||||
let result = ledger.build(4, &mut |w| {
|
||||
called = true;
|
||||
w.write_str("next")
|
||||
});
|
||||
let Err(MaterializeError::Resource(error)) = result else {
|
||||
panic!("expected prepublication refusal")
|
||||
};
|
||||
assert!(!called);
|
||||
assert_eq!(
|
||||
error.resource,
|
||||
if failure == 1 { Q::TemporaryBytes } else { Q::AuxiliaryBytes }
|
||||
);
|
||||
assert_eq!(
|
||||
error.reason,
|
||||
if failure == 2 { ResourceReason::AllocationFailed } else { ResourceReason::Limit }
|
||||
);
|
||||
assert_eq!(ledger.tracked_allocations(), 4);
|
||||
assert_eq!(budget.used(Q::StringBytes), 16);
|
||||
assert_eq!(budget.used(Q::StringAllocations), 4);
|
||||
assert_eq!(budget.used(Q::AuxiliaryBytes), auxiliary);
|
||||
assert_eq!(budget.used(Q::TemporaryBytes), 0);
|
||||
for value in &kept {
|
||||
assert!(ledger.retains(value));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn index_initial_allocation_failure_and_rebuild_failure_refund_reservations() {
|
||||
let (budget, ledger) = fixture(&[]);
|
||||
let initial = budget.used(Q::AuxiliaryBytes);
|
||||
crate::resource_budget::FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.set(true));
|
||||
assert_eq!(ledger.import("x").unwrap_err().reason, ResourceReason::AllocationFailed);
|
||||
assert_eq!(ledger.tracked_allocations(), 0);
|
||||
assert_eq!(budget.used(Q::AuxiliaryBytes), initial);
|
||||
assert_eq!(budget.used(Q::StringBytes), 0);
|
||||
assert_eq!(budget.used(Q::StringAllocations), 0);
|
||||
let mut kept = (0..4).map(|_| ledger.import("same").unwrap()).collect::<Vec<_>>();
|
||||
let old = Arc::downgrade(&kept.pop().unwrap());
|
||||
assert_eq!(ledger.reconcile(), 1);
|
||||
let auxiliary = budget.used(Q::AuxiliaryBytes);
|
||||
budget.test_lower_limit(Q::TemporaryBytes, 0);
|
||||
assert_eq!(ledger.import("new").unwrap_err().resource, Q::TemporaryBytes);
|
||||
assert_eq!(ledger.tracked_allocations(), 3);
|
||||
assert_eq!(budget.used(Q::AuxiliaryBytes), auxiliary);
|
||||
assert_eq!(budget.used(Q::TemporaryBytes), 0);
|
||||
assert_eq!(budget.used(Q::StringBytes), 12);
|
||||
for value in &kept {
|
||||
assert_eq!(Arc::strong_count(value), 1);
|
||||
assert!(ledger.retains(value));
|
||||
}
|
||||
assert!(old.upgrade().is_none());
|
||||
// Rebuild under a restored test budget; no production ceiling changes.
|
||||
budget.test_lower_limit(Q::TemporaryBytes, 64 * 1024 * 1024);
|
||||
let newer = ledger.import("same").unwrap();
|
||||
assert!(ledger.retains(&newer));
|
||||
assert!(!ledger.retains(&Arc::from("same")));
|
||||
assert!(ledger.lookup_work(&newer).0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tombstone_rebuild_allocator_refusal_preserves_live_keys_and_charges() {
|
||||
let (budget, ledger) = fixture(&[]);
|
||||
let mut owners = (0..4).map(|_| ledger.import("same").unwrap()).collect::<Vec<_>>();
|
||||
owners.pop();
|
||||
assert_eq!(ledger.reconcile(), 1);
|
||||
let auxiliary = budget.used(Q::AuxiliaryBytes);
|
||||
crate::resource_budget::FAIL_NEXT_VEC_ALLOCATION.with(|fail| fail.set(true));
|
||||
assert_eq!(ledger.import("new").unwrap_err().reason, ResourceReason::AllocationFailed);
|
||||
assert_eq!(ledger.tracked_allocations(), 3);
|
||||
assert_eq!(budget.used(Q::StringBytes), 12);
|
||||
assert_eq!(budget.used(Q::StringAllocations), 3);
|
||||
assert_eq!(budget.used(Q::AuxiliaryBytes), auxiliary);
|
||||
assert_eq!(budget.used(Q::TemporaryBytes), 0);
|
||||
assert!(owners.iter().all(|v| ledger.retains(v)));
|
||||
let new = ledger.import("new").unwrap();
|
||||
assert!(ledger.retains(&new));
|
||||
assert_eq!(budget.used(Q::AuxiliaryBytes), auxiliary);
|
||||
}
|
||||
|
||||
@@ -3890,3 +3890,6 @@ mod tests {
|
||||
#[cfg(test)]
|
||||
#[path = "virtual_machine/callable_tests.rs"]
|
||||
mod callable_tests;
|
||||
|
||||
#[cfg(feature = "resource-diagnostics")]
|
||||
pub mod string_provenance_qualification;
|
||||
|
||||
@@ -331,3 +331,45 @@ fn recursive_fn_calls_use_bounded_ordinary_frames() {
|
||||
);
|
||||
assert_eq!(v.call_stack.len(), 4);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn root_return_scratch_refusal_keeps_transferred_roots_without_partial_sweep() {
|
||||
let mut v = vm(&[
|
||||
("MAKE_CLOSURE fn=1,type=4\nPOP\nMAKE_CLOSURE fn=1,type=4\nRET", sig(&[], &[4]), vec![]),
|
||||
("PUSH_I32 55\nRET", sig(&[], &[0]), vec![]),
|
||||
]);
|
||||
v.gc_alloc_threshold = 1;
|
||||
for _ in 0..3 {
|
||||
v.step(&mut Native, &mut HostContext::new(None)).unwrap();
|
||||
}
|
||||
let Value::HeapRef(owner) = *v.operand_stack.last().unwrap() else { panic!() };
|
||||
let live = v.heap.len();
|
||||
v.budget.test_lower_limit(RuntimeResource::TemporaryBytes, 0);
|
||||
assert!(matches!(v.step(&mut Native, &mut HostContext::new(None)),
|
||||
Err(LogicalFrameEndingReason::Trap(t)) if t.code == TRAP_RESOURCE_LIMIT));
|
||||
assert!(v.call_stack.is_empty());
|
||||
assert_eq!(v.operand_stack.as_slice(), &[Value::HeapRef(owner)]);
|
||||
assert!(v.heap.is_valid(owner));
|
||||
assert_eq!(v.heap.len(), live);
|
||||
v.budget.test_lower_limit(RuntimeResource::TemporaryBytes, 1024 * 1024);
|
||||
v.handle_safepoint().unwrap();
|
||||
assert_eq!(v.heap.len(), 1);
|
||||
assert!(v.heap.is_valid(owner));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bad_return_height_preserves_frame_and_owners() {
|
||||
let mut v = vm(&[
|
||||
("MAKE_CLOSURE fn=1,type=4\nRET", sig(&[], &[4]), vec![]),
|
||||
("PUSH_I32 55\nRET", sig(&[], &[0]), vec![]),
|
||||
]);
|
||||
v.step(&mut Native, &mut HostContext::new(None)).unwrap();
|
||||
v.operand_stack.push(Value::Int32(9)).unwrap();
|
||||
let before = v.operand_stack.as_slice().to_vec();
|
||||
assert!(matches!(v.step(&mut Native, &mut HostContext::new(None)),
|
||||
Err(LogicalFrameEndingReason::Trap(t)) if t.code == prometeu_bytecode::TRAP_BAD_RET_SLOTS));
|
||||
assert_eq!(v.call_stack.len(), 1);
|
||||
assert_eq!(v.operand_stack.as_slice(), before);
|
||||
let Value::HeapRef(owner) = before[before.len() - 2] else { panic!() };
|
||||
assert!(v.heap.is_valid(owner));
|
||||
}
|
||||
|
||||
@@ -46,21 +46,13 @@ impl VirtualMachine {
|
||||
)
|
||||
.map_err(|(_, error)| self.value_trap(error, opcode, pc))?;
|
||||
|
||||
// Copy return values (preserving order: pop return_slots values, then reverse to push back)
|
||||
let mut return_vals = BudgetVec::new(&self.budget, RuntimeResource::TemporaryBytes);
|
||||
return_vals.reserve(return_slots).map_err(|e| self.resource_trap(e, opcode, pc))?;
|
||||
for _ in 0..return_slots {
|
||||
return_vals
|
||||
.push(self.pop().map_err(LogicalFrameEndingReason::Panic)?)
|
||||
.map_err(|e| self.resource_trap(e, opcode, pc))?;
|
||||
// Validation is complete. Move the ordered result suffix ahead of this
|
||||
// frame's parameters/locals without cloning owners or reserving scratch.
|
||||
if return_slots != 0 {
|
||||
self.operand_stack[frame.stack_base..].rotate_right(return_slots);
|
||||
}
|
||||
return_vals.reverse();
|
||||
|
||||
self.operand_stack.truncate(frame.stack_base + return_slots);
|
||||
self.call_stack.pop();
|
||||
self.operand_stack.truncate(frame.stack_base);
|
||||
for val in return_vals.drain(..) {
|
||||
self.push_trap(val, opcode, pc)?;
|
||||
}
|
||||
self.pc = frame.return_pc as usize;
|
||||
if self.call_stack.is_empty() {
|
||||
self.handle_safepoint().map_err(|e| self.resource_trap(e, opcode, pc))?;
|
||||
|
||||
@@ -62,7 +62,7 @@ impl VirtualMachine {
|
||||
if self.temporary_slots() == 0 {
|
||||
return Err("Stack underflow".into());
|
||||
}
|
||||
self.operand_stack.pop().ok_or("Stack underflow".into())
|
||||
self.operand_stack.pop().ok_or_else(|| "Stack underflow".into())
|
||||
}
|
||||
|
||||
pub fn pop_number(&mut self) -> Result<f64, String> {
|
||||
@@ -82,7 +82,7 @@ impl VirtualMachine {
|
||||
if self.temporary_slots() == 0 {
|
||||
return Err("Stack underflow".into());
|
||||
}
|
||||
self.operand_stack.last().ok_or("Stack underflow".into())
|
||||
self.operand_stack.last().ok_or_else(|| "Stack underflow".into())
|
||||
}
|
||||
|
||||
pub(super) fn pop_trap<S: Into<String>>(
|
||||
@@ -112,8 +112,22 @@ impl VirtualMachine {
|
||||
where
|
||||
F: FnOnce(Value, Value) -> Result<Value, OpError>,
|
||||
{
|
||||
let b = self.pop_trap(opcode, start_pc, format!("{:?} requires two operands", opcode))?;
|
||||
let a = self.pop_trap(opcode, start_pc, format!("{:?} requires two operands", opcode))?;
|
||||
let b = self.pop().map_err(|_| {
|
||||
self.trap(
|
||||
TRAP_STACK_UNDERFLOW,
|
||||
opcode as u16,
|
||||
format!("{:?} requires two operands", opcode),
|
||||
start_pc,
|
||||
)
|
||||
})?;
|
||||
let a = self.pop().map_err(|_| {
|
||||
self.trap(
|
||||
TRAP_STACK_UNDERFLOW,
|
||||
opcode as u16,
|
||||
format!("{:?} requires two operands", opcode),
|
||||
start_pc,
|
||||
)
|
||||
})?;
|
||||
match f(a, b) {
|
||||
Ok(res) => {
|
||||
self.push_trap(res, opcode, start_pc)?;
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
//! Opt-in diagnostics for the actual native provenance boundaries.
|
||||
use super::*;
|
||||
use crate::ValidatedProgramImage;
|
||||
use std::{hint::black_box, sync::Arc};
|
||||
|
||||
pub struct Fixture {
|
||||
vm: VirtualMachine,
|
||||
owners: Vec<Arc<str>>,
|
||||
image_hit: Arc<str>,
|
||||
miss: Arc<str>,
|
||||
}
|
||||
impl Fixture {
|
||||
pub fn new(image: Arc<ValidatedProgramImage>, count: usize) -> Self {
|
||||
let Value::String(image_hit) = image.constants().last().unwrap() else { panic!() };
|
||||
let image_hit = image_hit.clone();
|
||||
let mut vm = VirtualMachine::from_image(image, Default::default());
|
||||
let ledger = crate::string_ledger::StringLedger::new(vm.budget.clone()).unwrap();
|
||||
let owners = (0..count).map(|_| ledger.import("same").unwrap()).collect::<Vec<_>>();
|
||||
vm.strings = Some(ledger);
|
||||
vm.operand_stack.push(Value::String(image_hit.clone())).unwrap();
|
||||
vm.operand_stack.push(Value::String(owners.last().unwrap().clone())).unwrap();
|
||||
Self { vm, owners, image_hit, miss: Arc::from("same") }
|
||||
}
|
||||
pub fn auxiliary_bytes(&self) -> u64 {
|
||||
self.vm.budget.used(RuntimeResource::AuxiliaryBytes)
|
||||
}
|
||||
pub fn work(&self) -> [usize; 4] {
|
||||
let image = self.vm.validated_image.as_ref().unwrap();
|
||||
let session = self.vm.strings.as_ref().unwrap();
|
||||
let (ih, iw) = image.lookup_work(&self.image_hit);
|
||||
let (im, mw) = image.lookup_work(&self.miss);
|
||||
let (sh, sw) = session.lookup_work(self.owners.last().unwrap());
|
||||
let (sm, tw) = session.lookup_work(&self.miss);
|
||||
assert!(ih && sh && !im && !sm);
|
||||
[iw, mw, sw, tw]
|
||||
}
|
||||
pub fn run(&mut self, workload: &str) {
|
||||
match workload {
|
||||
"image_hit" => {
|
||||
black_box(
|
||||
self.vm
|
||||
.validated_image
|
||||
.as_ref()
|
||||
.unwrap()
|
||||
.retains_string(black_box(&self.image_hit)),
|
||||
);
|
||||
}
|
||||
"image_miss" => {
|
||||
black_box(
|
||||
self.vm.validated_image.as_ref().unwrap().retains_string(black_box(&self.miss)),
|
||||
);
|
||||
}
|
||||
"session_hit" => {
|
||||
black_box(
|
||||
self.vm
|
||||
.strings
|
||||
.as_ref()
|
||||
.unwrap()
|
||||
.retains(black_box(self.owners.last().unwrap())),
|
||||
);
|
||||
}
|
||||
"session_miss" => {
|
||||
black_box(self.vm.strings.as_ref().unwrap().retains(black_box(&self.miss)));
|
||||
}
|
||||
"import_hit" => {
|
||||
black_box(
|
||||
self.vm
|
||||
.import_string_value(&Value::String(self.owners.last().unwrap().clone()))
|
||||
.unwrap(),
|
||||
);
|
||||
}
|
||||
"import_miss" => {
|
||||
black_box(self.vm.import_string_value(&Value::String(self.miss.clone())).unwrap());
|
||||
}
|
||||
"snapshot_mixed" => {
|
||||
black_box(self.vm.operand_stack_top(2));
|
||||
}
|
||||
_ => panic!("unknown workload"),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -699,28 +699,28 @@ fn ordered_multi_group_return_preserves_native_owners_and_receiver_identity() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unknown_return_and_late_return_resource_failures_keep_current_frame_and_operands() {
|
||||
fn unknown_return_refusal_keeps_frame_but_valid_return_needs_no_scratch() {
|
||||
let mut code = vec![];
|
||||
arg(&mut code, OpCode::GetGlobal, 0);
|
||||
op(&mut code, OpCode::Ret);
|
||||
let image = image(&code, vec![function(0, code.len(), &[], &[0], 0, 1)]);
|
||||
for (value, limits, expected) in [
|
||||
(Value::Boolean(true), RuntimeLimits::default(), TRAP_TYPE),
|
||||
(
|
||||
Value::Int32(3),
|
||||
RuntimeLimits::default().lowered(RuntimeResource::TemporaryBytes, 0).unwrap(),
|
||||
TRAP_RESOURCE_LIMIT,
|
||||
),
|
||||
] {
|
||||
for value in [Value::Boolean(true), Value::Int32(3)] {
|
||||
let limits = RuntimeLimits::default().lowered(RuntimeResource::TemporaryBytes, 0).unwrap();
|
||||
let mut vm = VirtualMachine::from_image(image.clone(), limits);
|
||||
vm.prepare_boot_call().unwrap();
|
||||
vm.globals.push(value.clone()).unwrap();
|
||||
let result = run(&mut vm);
|
||||
assert!(
|
||||
matches!(result,LogicalFrameEndingReason::Trap(info) if info.code==expected && info.pc==6)
|
||||
);
|
||||
assert_eq!(vm.call_stack.len(), 1);
|
||||
if matches!(value, Value::Boolean(_)) {
|
||||
assert!(
|
||||
matches!(result, LogicalFrameEndingReason::Trap(info) if info.code == TRAP_TYPE && info.pc == 6)
|
||||
);
|
||||
assert_eq!(vm.call_stack.len(), 1);
|
||||
} else {
|
||||
assert_eq!(result, LogicalFrameEndingReason::EndOfRom);
|
||||
assert!(vm.call_stack.is_empty());
|
||||
}
|
||||
assert_eq!(vm.operand_stack.as_slice(), &[value]);
|
||||
assert_eq!(vm.budget.used(RuntimeResource::TemporaryBytes), 0);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
//! Count only this thread's warmed instruction region, including reallocations.
|
||||
use prometeu_bytecode::{assemble, load_limits::LoadBudget, registry::*};
|
||||
use prometeu_hal::{HostContext, syscalls::caps};
|
||||
use prometeu_test_support::program::{TestProgram, int64_signature};
|
||||
use prometeu_vm::{ValidatedProgramImage, VirtualMachine};
|
||||
use std::alloc::{GlobalAlloc, Layout, System};
|
||||
use std::cell::Cell;
|
||||
struct Native;
|
||||
impl prometeu_hal::NativeInterface for Native {
|
||||
fn syscall(
|
||||
&mut self,
|
||||
_: u32,
|
||||
_: &[prometeu_bytecode::Value],
|
||||
_: &mut prometeu_hal::HostReturn,
|
||||
_: &mut HostContext,
|
||||
) -> Result<(), prometeu_hal::vm_fault::VmFault> {
|
||||
panic!("no syscalls in allocation fixtures")
|
||||
}
|
||||
}
|
||||
|
||||
thread_local! {
|
||||
static COUNTS: Cell<Option<(usize, usize)>> = const { Cell::new(None) };
|
||||
}
|
||||
struct Counter;
|
||||
fn count(alloc: usize, free: usize) {
|
||||
let _ = COUNTS.try_with(|c| {
|
||||
if let Some((a, d)) = c.get() {
|
||||
c.set(Some((a + alloc, d + free)));
|
||||
}
|
||||
});
|
||||
}
|
||||
unsafe impl GlobalAlloc for Counter {
|
||||
unsafe fn alloc(&self, l: Layout) -> *mut u8 {
|
||||
count(1, 0);
|
||||
unsafe { System.alloc(l) }
|
||||
}
|
||||
unsafe fn alloc_zeroed(&self, l: Layout) -> *mut u8 {
|
||||
count(1, 0);
|
||||
unsafe { System.alloc_zeroed(l) }
|
||||
}
|
||||
unsafe fn dealloc(&self, p: *mut u8, l: Layout) {
|
||||
count(0, 1);
|
||||
unsafe { System.dealloc(p, l) }
|
||||
}
|
||||
unsafe fn realloc(&self, p: *mut u8, l: Layout, n: usize) -> *mut u8 {
|
||||
count(1, 1);
|
||||
unsafe { System.realloc(p, l, n) }
|
||||
}
|
||||
}
|
||||
#[global_allocator]
|
||||
static ALLOCATOR: Counter = Counter;
|
||||
|
||||
#[test]
|
||||
fn warmed_instruction_regions_do_not_allocate() {
|
||||
let mut observations = vec![];
|
||||
for (name, caller, callee, results, steps) in [
|
||||
("scalar", "CALL 1\nPOP\nJMP 0", "PUSH_I64 42\nRET", 1, 5),
|
||||
("void", "CALL 1\nJMP 0", "RET", 0, 3),
|
||||
("multi", "CALL 1\nPOP\nPOP\nJMP 0", "PUSH_I64 11\nPUSH_I64 22\nRET", 2, 7),
|
||||
("arithmetic_peek", "PUSH_I64 3\nDUP\nADD\nPOP\nJMP 0", "RET", 0, 5),
|
||||
] {
|
||||
let mut code = assemble(caller).unwrap();
|
||||
let offset = code.len();
|
||||
let body = assemble(callee).unwrap();
|
||||
let mut program = TestProgram::single(code.clone(), int64_signature(0, 0), 4, 8);
|
||||
program.functions.push(FunctionDecl {
|
||||
signature: int64_signature(0, results),
|
||||
code_offset: offset as u32,
|
||||
code_len: body.len() as u32,
|
||||
local_slots: 4,
|
||||
max_stack_slots: 8,
|
||||
..Default::default()
|
||||
});
|
||||
code.extend(body);
|
||||
program.code = code;
|
||||
let image = ValidatedProgramImage::load(
|
||||
&program.serialize(),
|
||||
&mut LoadBudget::new(Default::default()),
|
||||
caps::NONE,
|
||||
)
|
||||
.unwrap();
|
||||
let mut vm = VirtualMachine::from_image(image, Default::default());
|
||||
vm.prepare_boot_call().unwrap();
|
||||
let mut host = HostContext::new(None);
|
||||
for _ in 0..steps * 10 {
|
||||
vm.step(&mut Native, &mut host).unwrap();
|
||||
}
|
||||
COUNTS.with(|c| c.set(Some((0, 0))));
|
||||
for _ in 0..steps * 1000 {
|
||||
vm.step(&mut Native, &mut host).unwrap();
|
||||
}
|
||||
let counts = COUNTS.with(|c| c.replace(None).unwrap());
|
||||
println!("{name}: 1000 iterations, {counts:?} allocations/deallocations");
|
||||
observations.push((name, counts));
|
||||
}
|
||||
for (name, counts) in observations {
|
||||
assert_eq!(counts, (0, 0), "{name}");
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "resource-diagnostics")]
|
||||
#[allow(dead_code)]
|
||||
#[path = "support/callables.rs"]
|
||||
mod provenance_callables;
|
||||
|
||||
#[cfg(feature = "resource-diagnostics")]
|
||||
#[test]
|
||||
fn provenance_queries_allocate_nothing_at_all_qualification_sizes() {
|
||||
use prometeu_bytecode::model::ConstantPoolEntry;
|
||||
use prometeu_vm::string_provenance_qualification::Fixture;
|
||||
for count in [1024_usize, 16384, 65536] {
|
||||
let constants = vec![ConstantPoolEntry::String("same".into()); count];
|
||||
let image = provenance_callables::image_constants(
|
||||
&[("RET", provenance_callables::sig(&[], &[]), vec![])],
|
||||
vec![],
|
||||
&constants,
|
||||
)
|
||||
.unwrap();
|
||||
let mut fixture = Fixture::new(image, count);
|
||||
let work = fixture.work();
|
||||
let bound = count.ilog2() as usize + 1;
|
||||
assert!(work[0] <= bound && work[1] <= bound);
|
||||
for workload in ["image_hit", "image_miss", "session_hit", "session_miss", "import_hit"] {
|
||||
COUNTS.with(|c| c.set(Some((0, 0))));
|
||||
for _ in 0..100 {
|
||||
fixture.run(workload);
|
||||
}
|
||||
let counts = COUNTS.with(|c| c.replace(None)).unwrap();
|
||||
assert_eq!(counts, (0, 0), "{workload}, owners={count}");
|
||||
}
|
||||
COUNTS.with(|c| c.set(Some((0, 0))));
|
||||
fixture.run("snapshot_mixed");
|
||||
let counts = COUNTS.with(|c| c.replace(None)).unwrap();
|
||||
assert_eq!(counts, (1, 1), "only snapshot Value storage allocates");
|
||||
}
|
||||
}
|
||||
@@ -216,3 +216,47 @@ fn integer_division_and_remainder_overflow_wrap_without_host_panic() {
|
||||
assert_eq!(machine.operand_stack_top(1), vec![Value::Int64(expected)]);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn maximum_width_mixed_return_preserves_order_and_caller_locals() {
|
||||
// TestProgram catalog: Int64, Vec2, Float = 0, 6, 2.
|
||||
let mut caller = String::from("PUSH_I64 73\nSET_LOCAL 0\nCALL 1\n");
|
||||
caller.push_str(&"POP\n".repeat(1024));
|
||||
caller.push_str("GET_LOCAL 0\nRET");
|
||||
let mut callee = String::new();
|
||||
let mut results = vec![];
|
||||
let mut expected = vec![];
|
||||
for i in 0..256 {
|
||||
callee.push_str(&format!("PUSH_I64 {i}\nPUSH_F64 f64:0x3ff0000000000000\nPUSH_F64 f64:0x4000000000000000\nPUSH_F64 f64:0x4008000000000000\n"));
|
||||
results.extend([TypeId(0), TypeId(6), TypeId(2)]);
|
||||
expected.extend([Value::Int64(i), Value::Float(1.), Value::Float(2.), Value::Float(3.)]);
|
||||
}
|
||||
callee.push_str("RET");
|
||||
let caller_code = assemble(&caller).unwrap();
|
||||
let body = assemble(&callee).unwrap();
|
||||
let mut program = TestProgram::single(caller_code.clone(), int64_signature(0, 1), 1, 1024);
|
||||
program.functions.push(FunctionDecl {
|
||||
signature: Signature { parameters: vec![], results },
|
||||
code_offset: caller_code.len() as u32,
|
||||
code_len: body.len() as u32,
|
||||
local_slots: 3,
|
||||
max_stack_slots: 1024,
|
||||
..Default::default()
|
||||
});
|
||||
program.code.extend(body);
|
||||
let image = ValidatedProgramImage::load(
|
||||
&program.serialize(),
|
||||
&mut LoadBudget::new(Default::default()),
|
||||
caps::NONE,
|
||||
)
|
||||
.unwrap();
|
||||
let mut machine = VirtualMachine::from_image(image, Default::default());
|
||||
machine.prepare_boot_call().unwrap();
|
||||
for _ in 0..3 + 1024 + 1 {
|
||||
machine.step(&mut Native, &mut HostContext::new(None)).unwrap();
|
||||
}
|
||||
expected.reverse();
|
||||
assert_eq!(machine.operand_stack_top(1024), expected);
|
||||
assert_eq!(run(&mut machine, 10000).reason, LogicalFrameEndingReason::EndOfRom);
|
||||
assert_eq!(machine.operand_stack_top(1), vec![Value::Int64(73)]);
|
||||
}
|
||||
|
||||
@@ -61,7 +61,9 @@ fn integer_limits_are_exact_and_abs_does_not_wrap() {
|
||||
] {
|
||||
assert_eq!(call(base, &[b.clone(), a.clone()]).unwrap(), a);
|
||||
assert_eq!(call(base + 1, &[a.clone(), b.clone()]).unwrap(), b);
|
||||
assert!(matches!(call(base+3,&[a.clone()]),Err(E::Numeric(e)) if e.code==RANGE));
|
||||
assert!(
|
||||
matches!(call(base+3,std::slice::from_ref(&a)),Err(E::Numeric(e)) if e.code==RANGE)
|
||||
);
|
||||
assert!(matches!(call(base+2,&[b.clone(),b,a]),Err(E::Numeric(e)) if e.code==DOMAIN));
|
||||
}
|
||||
assert_eq!(
|
||||
|
||||
@@ -45,7 +45,7 @@ fn independent_words_signed_and_float_bits_for_all_seed_patterns() {
|
||||
});
|
||||
}
|
||||
code.push_str("RET");
|
||||
let mut v = vm(&[(&code, sig(&[], &vec![if floating { 2 } else { 0 }; 8]), vec![], 1)]);
|
||||
let mut v = vm(&[(&code, sig(&[], &[if floating { 2 } else { 0 }; 8]), vec![], 1)]);
|
||||
let r = run(&mut v, 10000);
|
||||
assert_eq!(r.reason, End::EndOfRom);
|
||||
let expected: Vec<_> = vectors(seed)
|
||||
|
||||
@@ -114,6 +114,111 @@ fn repository_sources_have_exactly_one_coverage_owner() {
|
||||
);
|
||||
}
|
||||
|
||||
fn repository_policy(script: &str) -> Output {
|
||||
Command::new("python3")
|
||||
.arg("-B")
|
||||
.arg("-c")
|
||||
.arg(format!(
|
||||
r#"
|
||||
import importlib.util
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
root = Path(sys.argv[1])
|
||||
spec = importlib.util.spec_from_file_location("coverage_auditor", root / "scripts/audit-coverage-domains.py")
|
||||
auditor = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(auditor)
|
||||
config = json.loads((root / "files/config/runtime-edge-coverage-domains.json").read_text())
|
||||
domains = config["domains"]
|
||||
trees = [
|
||||
("crates/console/prometeu-system/src/services/memcard", "system/fs", "system/core"),
|
||||
("crates/console/prometeu-drivers/src/asset", "asset/bank", "drivers/core"),
|
||||
]
|
||||
{script}
|
||||
"#
|
||||
))
|
||||
.arg(repository())
|
||||
.output()
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn specialized_entry_files_and_all_descendants_have_the_expected_owner() {
|
||||
success(repository_policy(
|
||||
r#"
|
||||
for stem, expected, residual in trees:
|
||||
names = [stem + ".rs"]
|
||||
descendants = sorted((root / stem).rglob("*.rs"))
|
||||
assert descendants, stem
|
||||
names += [path.relative_to(root).as_posix() for path in descendants]
|
||||
names += [stem + "/future.rs", stem + "/future/nested.rs"]
|
||||
for name in names:
|
||||
actual = auditor.owners(name, domains)
|
||||
assert actual == [expected], (name, expected, actual)
|
||||
# Exclusive residual fallback would pass the generic audit but is wrong.
|
||||
import copy
|
||||
fallback = copy.deepcopy(domains)
|
||||
fallback[expected]["paths"].remove(stem + "/")
|
||||
fallback[residual]["exclude_paths"].remove(stem + "/")
|
||||
name = names[-1]
|
||||
assert auditor.owners(name, fallback) == [residual]
|
||||
assert auditor.owners(name, fallback) != [expected]
|
||||
overlap = copy.deepcopy(domains)
|
||||
overlap[residual]["exclude_paths"].remove(stem + "/")
|
||||
assert set(auditor.owners(name, overlap)) == {expected, residual}
|
||||
"#,
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn specialized_ownership_survives_absent_and_zero_instrumentation() {
|
||||
success(repository_policy(
|
||||
r#"
|
||||
import tempfile
|
||||
with tempfile.TemporaryDirectory(prefix="prometeu-domain-trees-") as directory:
|
||||
fixture = Path(directory)
|
||||
records = []
|
||||
# Use the real map and source layout, with independent synthetic evidence.
|
||||
for source in (root / "crates").glob("**/src/**/*.rs"):
|
||||
name = source.relative_to(root).as_posix()
|
||||
path = fixture / name
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text("// ownership fixture\n")
|
||||
records.append({"filename": name, "summary": {"lines": {"count": 1, "covered": 1}}})
|
||||
for stem, expected, residual in trees:
|
||||
for suffix in ["future/absent.rs", "future/zero.rs"]:
|
||||
path = fixture / stem / suffix
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text("// no executable lines\n")
|
||||
records.append({"filename": stem + "/future/zero.rs", "summary": {"lines": {"count": 0, "covered": 0}}})
|
||||
config_path = fixture / "config.json"
|
||||
config_path.write_text(json.dumps(config))
|
||||
report_path = fixture / "report.json"
|
||||
report_path.write_text(json.dumps({"data": [{"files": records}]}))
|
||||
before = auditor.audit(root, root / "files/config/runtime-edge-coverage-domains.json")
|
||||
result = auditor.audit(fixture, config_path, report_path)
|
||||
assert len(result["domains"]) == 15
|
||||
assert all(domain["gate_passed"] for domain in result["domains"].values())
|
||||
for stem, expected, residual in trees:
|
||||
domain = result["domains"][expected]
|
||||
assert domain["source_files"] == before["domains"][expected]["source_files"] + 2
|
||||
assert stem + "/future/zero.rs" in domain["files"]
|
||||
assert stem + "/future/absent.rs" not in domain["files"]
|
||||
assert result["domains"][residual]["source_files"] == before["domains"][residual]["source_files"]
|
||||
# Missing residual exclusions must fail even for sources absent from reports.
|
||||
for stem, expected, residual in trees:
|
||||
domains[residual]["exclude_paths"].remove(stem + "/")
|
||||
config_path.write_text(json.dumps(config))
|
||||
try:
|
||||
auditor.audit(fixture, config_path, report_path)
|
||||
except auditor.AuditError as error:
|
||||
assert "expected exactly one domain" in str(error), error
|
||||
else:
|
||||
raise AssertionError("overlapping module trees passed ownership audit")
|
||||
"#,
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn exclusions_and_absolute_relative_report_names_have_exclusive_ownership() {
|
||||
let fixture = Fixture::new();
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
mod reclamation;
|
||||
pub use reclamation::AudioReclaimer;
|
||||
use reclamation::{CallbackLane, Packet};
|
||||
|
||||
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
|
||||
use prometeu_drivers::{AudioCommand, Channel, MAX_CHANNELS, OUTPUT_SAMPLE_RATE};
|
||||
use prometeu_drivers::{AudioDelivery, AudioTransport};
|
||||
@@ -10,9 +14,15 @@ use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
trait AudioStreamHandle {}
|
||||
trait AudioStreamHandle {
|
||||
fn stop(&self) {}
|
||||
}
|
||||
|
||||
impl AudioStreamHandle for cpal::Stream {}
|
||||
impl AudioStreamHandle for cpal::Stream {
|
||||
fn stop(&self) {
|
||||
let _ = self.pause();
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum HostAudioError {
|
||||
@@ -97,6 +107,7 @@ pub struct HostAudio {
|
||||
last_progress: Instant,
|
||||
pub perf_consumer: Option<ringbuf::wrap::CachingCons<Arc<HeapRb<u64>>>>,
|
||||
_stream: Option<Box<dyn AudioStreamHandle>>,
|
||||
reclaimer: Option<AudioReclaimer>,
|
||||
}
|
||||
|
||||
impl Default for HostAudio {
|
||||
@@ -115,6 +126,7 @@ impl HostAudio {
|
||||
last_progress: Instant::now(),
|
||||
perf_consumer: None,
|
||||
_stream: None,
|
||||
reclaimer: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -129,7 +141,7 @@ impl HostAudio {
|
||||
self.last_heartbeat = 0;
|
||||
self.last_progress = Instant::now();
|
||||
self.perf_consumer = None;
|
||||
self._stream = None;
|
||||
self.stop_stream();
|
||||
|
||||
let device = runtime.default_output_device().ok_or(HostAudioError::NoOutputDevice)?;
|
||||
|
||||
@@ -143,7 +155,7 @@ impl HostAudio {
|
||||
let failed = self.failed.clone();
|
||||
let heartbeat = self.heartbeat.clone();
|
||||
|
||||
let mut mixer = AudioMixer::new();
|
||||
let (mut callback, reclaimer) = AudioCallback::new();
|
||||
|
||||
// To pass performance data from the audio thread to the main thread
|
||||
let audio_perf_rb = HeapRb::<u64>::new(64);
|
||||
@@ -154,13 +166,9 @@ impl HostAudio {
|
||||
&config,
|
||||
move |data: &mut [f32]| {
|
||||
heartbeat.fetch_add(1, Ordering::Relaxed);
|
||||
if let Some(delivery) = transport.try_take() {
|
||||
mixer.apply_delivery(delivery);
|
||||
}
|
||||
// Mixes audio
|
||||
mixer.fill_buffer(data);
|
||||
callback.render(&transport, data);
|
||||
// Sends processing time in microseconds
|
||||
let _ = perf_prod.try_push(mixer.last_processing_time.as_micros() as u64);
|
||||
let _ = perf_prod.try_push(callback.mixer.last_processing_time.as_micros() as u64);
|
||||
},
|
||||
move |err| {
|
||||
failed.store(true, Ordering::Relaxed);
|
||||
@@ -169,6 +177,7 @@ impl HostAudio {
|
||||
)?;
|
||||
|
||||
runtime.play_stream(&stream)?;
|
||||
self.reclaimer = Some(reclaimer);
|
||||
self.perf_consumer = Some(perf_cons);
|
||||
self._stream = Some(Box::new(stream));
|
||||
Ok(())
|
||||
@@ -177,6 +186,9 @@ impl HostAudio {
|
||||
/// Called even while guest execution is backpressured. A failed or stalled
|
||||
/// device cannot retain the only publication slot indefinitely.
|
||||
pub fn service(&mut self) {
|
||||
if let Some(reclaimer) = &mut self.reclaimer {
|
||||
reclaimer.service(&self.transport);
|
||||
}
|
||||
let heartbeat = self.heartbeat.load(Ordering::Relaxed);
|
||||
if heartbeat != self.last_heartbeat {
|
||||
self.last_heartbeat = heartbeat;
|
||||
@@ -187,7 +199,7 @@ impl HostAudio {
|
||||
|| self.last_progress.elapsed() >= Duration::from_secs(2))
|
||||
{
|
||||
eprintln!("[HostAudio] Output unavailable/stalled; consuming audio silently.");
|
||||
self._stream = None;
|
||||
self.stop_stream();
|
||||
self.perf_consumer = None;
|
||||
}
|
||||
if self._stream.is_none() {
|
||||
@@ -195,6 +207,16 @@ impl HostAudio {
|
||||
}
|
||||
}
|
||||
|
||||
fn stop_stream(&mut self) {
|
||||
if let Some(stream) = self._stream.take() {
|
||||
stream.stop();
|
||||
drop(stream); // Stop/drop the callback before any silent consumer takes over.
|
||||
}
|
||||
if let Some(mut reclaimer) = self.reclaimer.take() {
|
||||
reclaimer.service(&self.transport);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn update_stats(&mut self, stats: &mut crate::stats::HostStats) {
|
||||
if let Some(cons) = &mut self.perf_consumer {
|
||||
while let Some(us) = cons.try_pop() {
|
||||
@@ -204,6 +226,40 @@ impl HostAudio {
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for HostAudio {
|
||||
fn drop(&mut self) {
|
||||
self.stop_stream();
|
||||
self.transport.consume_silently();
|
||||
}
|
||||
}
|
||||
|
||||
/// The same bounded runtime-owned data path used by CPAL and allocator tests.
|
||||
pub struct AudioCallback {
|
||||
mixer: AudioMixer,
|
||||
lane: CallbackLane,
|
||||
}
|
||||
impl AudioCallback {
|
||||
pub fn new() -> (Self, AudioReclaimer) {
|
||||
let (lane, reclaimer) = reclamation::pool();
|
||||
(Self { mixer: AudioMixer::new(), lane }, reclaimer)
|
||||
}
|
||||
pub fn render(&mut self, transport: &AudioTransport, data: &mut [f32]) {
|
||||
let mut packet = self.lane.take();
|
||||
if let Some(packet) = &mut packet {
|
||||
// Reserve reclamation before acquiring any publication ownership.
|
||||
self.mixer.retire_inactive(packet);
|
||||
if let Some(delivery) = transport.try_take() {
|
||||
self.mixer.apply_delivery_into(delivery, packet);
|
||||
}
|
||||
}
|
||||
self.mixer.mix(data);
|
||||
if let Some(mut packet) = packet {
|
||||
self.mixer.retire_inactive(&mut packet);
|
||||
self.lane.give_back(packet);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
#[allow(clippy::items_after_test_module)]
|
||||
mod tests {
|
||||
@@ -347,27 +403,59 @@ impl AudioMixer {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn apply_delivery(&mut self, delivery: AudioDelivery) {
|
||||
fn apply_delivery_into(&mut self, delivery: AudioDelivery, packet: &mut Packet) {
|
||||
if delivery.epoch < self.epoch {
|
||||
packet.batches = [delivery.frame, delivery.lifecycle];
|
||||
return;
|
||||
}
|
||||
if delivery.epoch != self.epoch {
|
||||
self.voices = Default::default();
|
||||
for voice in &mut self.voices {
|
||||
packet.retire(voice.sample.take());
|
||||
*voice = Channel::default();
|
||||
}
|
||||
self.epoch = delivery.epoch;
|
||||
}
|
||||
// A frame pending before a lifecycle hook is always applied first.
|
||||
for batch in [delivery.frame, delivery.lifecycle].into_iter().flatten() {
|
||||
for command in batch {
|
||||
self.process_command(command);
|
||||
for (slot, batch) in packet.batches.iter_mut().zip([delivery.frame, delivery.lifecycle]) {
|
||||
*slot = batch;
|
||||
}
|
||||
// Move commands out of slots, keeping each vector allocation in its packet.
|
||||
for index in 0..2 {
|
||||
if let Some(mut batch) = packet.batches[index].take() {
|
||||
for command in &mut batch {
|
||||
let command =
|
||||
std::mem::replace(command, AudioCommand::Stop { voice_id: MAX_CHANNELS });
|
||||
self.process_command(command, packet);
|
||||
}
|
||||
packet.batches[index] = Some(batch);
|
||||
}
|
||||
}
|
||||
self.paused = delivery.paused;
|
||||
}
|
||||
|
||||
pub fn process_command(&mut self, cmd: AudioCommand) {
|
||||
fn retire_inactive(&mut self, packet: &mut Packet) {
|
||||
for voice in &mut self.voices {
|
||||
if !voice.active {
|
||||
packet.retire(voice.sample.take());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Signal-only tests use synchronous reclamation outside a data callback.
|
||||
#[cfg(test)]
|
||||
fn apply_delivery(&mut self, delivery: AudioDelivery) {
|
||||
self.apply_delivery_into(delivery, &mut Packet::new());
|
||||
}
|
||||
#[cfg(test)]
|
||||
fn fill_buffer(&mut self, buffer: &mut [f32]) {
|
||||
self.mix(buffer);
|
||||
self.retire_inactive(&mut Packet::new());
|
||||
}
|
||||
|
||||
fn process_command(&mut self, cmd: AudioCommand, packet: &mut Packet) {
|
||||
match cmd {
|
||||
AudioCommand::Play { sample, voice_id, volume, pan, pitch, priority, loop_mode } => {
|
||||
if voice_id < MAX_CHANNELS {
|
||||
packet.retire(self.voices[voice_id].sample.take());
|
||||
self.voices[voice_id] = Channel {
|
||||
sample: Some(sample),
|
||||
active: true,
|
||||
@@ -378,12 +466,14 @@ impl AudioMixer {
|
||||
loop_mode,
|
||||
priority,
|
||||
};
|
||||
} else {
|
||||
packet.retire(Some(sample));
|
||||
}
|
||||
}
|
||||
AudioCommand::Stop { voice_id } => {
|
||||
if voice_id < MAX_CHANNELS {
|
||||
self.voices[voice_id].active = false;
|
||||
self.voices[voice_id].sample = None;
|
||||
packet.retire(self.voices[voice_id].sample.take());
|
||||
}
|
||||
}
|
||||
AudioCommand::SetVolume { voice_id, volume } => {
|
||||
@@ -404,7 +494,7 @@ impl AudioMixer {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn fill_buffer(&mut self, buffer: &mut [f32]) {
|
||||
fn mix(&mut self, buffer: &mut [f32]) {
|
||||
let start = std::time::Instant::now();
|
||||
// Zeroes the buffer (stereo)
|
||||
for sample in buffer.iter_mut() {
|
||||
@@ -417,17 +507,19 @@ impl AudioMixer {
|
||||
}
|
||||
|
||||
for voice in self.voices.iter_mut() {
|
||||
if !voice.active {
|
||||
continue;
|
||||
}
|
||||
let Some(sample) = voice.sample.as_ref() else {
|
||||
continue;
|
||||
};
|
||||
let len = sample.data.len();
|
||||
if len == 0 {
|
||||
voice.active = false;
|
||||
voice.sample = None;
|
||||
continue;
|
||||
}
|
||||
let (loop_start, end) = match (sample.loop_start, sample.loop_end) {
|
||||
(Some(start), Some(end)) if start < end && end as usize <= len => {
|
||||
let (loop_start, end) = match (voice.loop_mode, sample.loop_start, sample.loop_end) {
|
||||
(LoopMode::On, Some(start), Some(end)) if start < end && end as usize <= len => {
|
||||
(start as f64, end as f64)
|
||||
}
|
||||
_ => (0.0, len as f64),
|
||||
@@ -441,7 +533,6 @@ impl AudioMixer {
|
||||
let pos = voice.pos as usize;
|
||||
if pos >= len {
|
||||
voice.active = false;
|
||||
voice.sample = None;
|
||||
break;
|
||||
}
|
||||
let s1 = sample.data[pos] as f32 / 32768.0;
|
||||
@@ -460,7 +551,6 @@ impl AudioMixer {
|
||||
if voice.pitch >= (end - voice.pos) / ratio {
|
||||
if voice.loop_mode == LoopMode::Off {
|
||||
voice.active = false;
|
||||
voice.sample = None;
|
||||
break;
|
||||
}
|
||||
let length = end - loop_start;
|
||||
@@ -512,6 +602,168 @@ mod control_tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn one_shot_ignores_markers_and_loop_preserves_intro() {
|
||||
for looping in [false, true] {
|
||||
let (mut audio, mut mixer) = pair();
|
||||
let sample =
|
||||
Sample::new(48_000, vec![1024, 2048, 4096, 8192, 12288, 16384, 20480, 24576])
|
||||
.with_loop(2, 4);
|
||||
play(&mut audio, Arc::new(sample), 1.0, looping);
|
||||
audio.finish_frame();
|
||||
apply(&audio, &mut mixer);
|
||||
let mut pcm = [0.0; 20];
|
||||
mixer.fill_buffer(&mut pcm);
|
||||
let expected = if looping {
|
||||
[1024, 2048, 4096, 8192, 4096, 8192, 4096, 8192, 4096, 8192]
|
||||
} else {
|
||||
[1024, 2048, 4096, 8192, 12288, 16384, 20480, 24576, 0, 0]
|
||||
};
|
||||
for (frame, value) in pcm.as_chunks::<2>().0.iter().zip(expected) {
|
||||
assert_eq!(*frame, [value as f32 / 32768.0, 0.0]);
|
||||
}
|
||||
assert_eq!(mixer.voices[0].active, looping);
|
||||
assert_eq!(mixer.voices[0].sample.is_some(), looping);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn absent_partial_and_invalid_markers_fall_back_to_full_sample() {
|
||||
for (start, end) in [
|
||||
(None, None),
|
||||
(Some(1), None),
|
||||
(None, Some(2)),
|
||||
(Some(2), Some(2)),
|
||||
(Some(3), Some(1)),
|
||||
(Some(0), Some(5)),
|
||||
(Some(u32::MAX), Some(u32::MAX)),
|
||||
(Some(0), Some(0)),
|
||||
] {
|
||||
for looping in [false, true] {
|
||||
let (mut audio, mut mixer) = pair();
|
||||
let mut sample = Sample::new(48_000, vec![4096, 8192, 12288, 16384]);
|
||||
sample.loop_start = start;
|
||||
sample.loop_end = end;
|
||||
play(&mut audio, Arc::new(sample), 1.0, looping);
|
||||
audio.finish_frame();
|
||||
apply(&audio, &mut mixer);
|
||||
let mut pcm = [0.0; 16];
|
||||
mixer.fill_buffer(&mut pcm);
|
||||
for (i, frame) in pcm.as_chunks::<2>().0.iter().enumerate() {
|
||||
let value = if looping || i < 4 { (i % 4 + 1) as f32 / 8.0 } else { 0.0 };
|
||||
assert_eq!(*frame, [value, 0.0], "markers {start:?}/{end:?}, loop {looping}");
|
||||
}
|
||||
assert_eq!(mixer.voices[0].active, looping);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fractional_interpolation_uses_loop_seam_or_actual_tail() {
|
||||
// Both rate conversion and fractional pitch produce a half-frame step.
|
||||
for (rate, pitch) in [(48_000, 0.5), (24_000, 1.0), (96_000, 0.25)] {
|
||||
for looping in [false, true] {
|
||||
let (mut audio, mut mixer) = pair();
|
||||
play(
|
||||
&mut audio,
|
||||
Arc::new(Sample::new(rate, vec![4096, 8192, 16384, 24576]).with_loop(1, 3)),
|
||||
pitch,
|
||||
looping,
|
||||
);
|
||||
audio.finish_frame();
|
||||
apply(&audio, &mut mixer);
|
||||
let mut pcm = [0.0; 18];
|
||||
mixer.fill_buffer(&mut pcm);
|
||||
let expected = if looping {
|
||||
[0.125, 0.1875, 0.25, 0.375, 0.5, 0.375, 0.25, 0.375, 0.5]
|
||||
} else {
|
||||
[0.125, 0.1875, 0.25, 0.375, 0.5, 0.625, 0.75, 0.375, 0.0]
|
||||
};
|
||||
for (frame, value) in pcm.as_chunks::<2>().0.iter().zip(expected) {
|
||||
assert!((frame[0] - value).abs() <= 1e-6);
|
||||
assert_eq!(frame[1], 0.0);
|
||||
}
|
||||
assert_eq!(mixer.voices[0].active, looping);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_samples_terminate_without_indexing() {
|
||||
for looping in [false, true] {
|
||||
for marked in [false, true] {
|
||||
let (mut audio, mut mixer) = pair();
|
||||
let sample = Sample::new(48_000, vec![]);
|
||||
let sample = if marked { sample.with_loop(0, 1) } else { sample };
|
||||
play(&mut audio, Arc::new(sample), f64::MAX, looping);
|
||||
audio.finish_frame();
|
||||
apply(&audio, &mut mixer);
|
||||
let mut pcm = [1.0; 8];
|
||||
mixer.fill_buffer(&mut pcm);
|
||||
assert_eq!(pcm, [0.0; 8]);
|
||||
assert!(!mixer.voices[0].active);
|
||||
assert!(mixer.voices[0].sample.is_none());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn multiple_wrap_pitch_preserves_selected_interval_and_signal() {
|
||||
for (rate, pitch) in [(48_000, 5.0), (96_000, 2.5)] {
|
||||
let (mut audio, mut mixer) = pair();
|
||||
play(
|
||||
&mut audio,
|
||||
Arc::new(Sample::new(rate, vec![4096, 8192, 16384, 24576]).with_loop(1, 3)),
|
||||
pitch,
|
||||
true,
|
||||
);
|
||||
audio.finish_frame();
|
||||
apply(&audio, &mut mixer);
|
||||
let mut pcm = [0.0; 10];
|
||||
mixer.fill_buffer(&mut pcm);
|
||||
assert_eq!(pcm, [0.125, 0.0, 0.25, 0.0, 0.5, 0.0, 0.25, 0.0, 0.5, 0.0]);
|
||||
assert_eq!(mixer.voices[0].pos, 1.0);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extreme_pitch_and_source_rates_keep_positions_finite() {
|
||||
for rate in [0, 1, 44_100, 48_000, 96_000, u32::MAX] {
|
||||
for looping in [false, true] {
|
||||
let (mut audio, mut mixer) = pair();
|
||||
play(
|
||||
&mut audio,
|
||||
Arc::new(Sample::new(rate, vec![4096, 8192, 16384, 24576]).with_loop(1, 3)),
|
||||
f64::MAX,
|
||||
looping,
|
||||
);
|
||||
audio.finish_frame();
|
||||
apply(&audio, &mut mixer);
|
||||
let mut pcm = [0.0; 128];
|
||||
mixer.fill_buffer(&mut pcm);
|
||||
assert!(pcm.iter().all(|value| value.is_finite()));
|
||||
assert!(mixer.voices[0].pos.is_finite());
|
||||
if rate == 0 {
|
||||
assert_eq!(mixer.voices[0].pos, 0.0);
|
||||
assert!(pcm.as_chunks::<2>().0.iter().all(|frame| *frame == [0.125, 0.0]));
|
||||
} else if looping {
|
||||
assert!((1.0..3.0).contains(&mixer.voices[0].pos));
|
||||
assert!(
|
||||
pcm.as_chunks::<2>()
|
||||
.0
|
||||
.iter()
|
||||
.skip(1)
|
||||
.all(|frame| (0.25..=0.5).contains(&frame[0]))
|
||||
);
|
||||
} else {
|
||||
assert_eq!(&pcm[..2], &[0.125, 0.0]);
|
||||
assert!(pcm[2..].iter().all(|value| *value == 0.0));
|
||||
}
|
||||
assert_eq!(mixer.voices[0].active, looping || rate == 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pcm_controls_preserve_position_and_stop_is_idempotent() {
|
||||
let (mut audio, mut mixer) = pair();
|
||||
@@ -632,7 +884,7 @@ mod control_tests {
|
||||
let mut pcm = [0.0; 128];
|
||||
mixer.fill_buffer(&mut pcm);
|
||||
assert!(pcm.iter().all(|v| v.is_finite()));
|
||||
assert!(pcm.chunks_exact(2).all(|frame| frame == [0.5, 0.0]));
|
||||
assert!(pcm.as_chunks::<2>().0.iter().all(|frame| *frame == [0.5, 0.0]));
|
||||
assert!(mixer.voices[0].active);
|
||||
assert!(mixer.voices[0].pos >= 1.0 && mixer.voices[0].pos < 3.0);
|
||||
}
|
||||
@@ -705,6 +957,7 @@ mod control_tests {
|
||||
audio.finish_frame();
|
||||
let deadline = Instant::now() + Duration::from_secs(2);
|
||||
while !audio.ready() && Instant::now() < deadline {
|
||||
host.service();
|
||||
std::thread::sleep(Duration::from_millis(10));
|
||||
}
|
||||
assert!(audio.ready());
|
||||
@@ -713,12 +966,119 @@ mod control_tests {
|
||||
audio.set_pan(0, 255);
|
||||
audio.set_pitch(0, 1.5);
|
||||
audio.finish_frame();
|
||||
std::thread::sleep(Duration::from_millis(100));
|
||||
for _ in 0..10 {
|
||||
host.service();
|
||||
std::thread::sleep(Duration::from_millis(10));
|
||||
}
|
||||
assert!(audio.ready());
|
||||
audio.stop(0);
|
||||
audio.finish_frame();
|
||||
std::thread::sleep(Duration::from_millis(100));
|
||||
for _ in 0..10 {
|
||||
host.service();
|
||||
std::thread::sleep(Duration::from_millis(10));
|
||||
}
|
||||
assert!(audio.ready());
|
||||
assert!(!host.failed.load(Ordering::Relaxed));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod reclamation_tests {
|
||||
use super::*;
|
||||
use prometeu_drivers::{Audio, MemoryBanks};
|
||||
use prometeu_hal::{AudioBridge, sample::Sample};
|
||||
|
||||
#[test]
|
||||
fn stale_delivery_moves_whole_vectors_to_service_without_changing_playback() {
|
||||
let mut audio = Audio::new(Arc::new(MemoryBanks::new()));
|
||||
let sample = Arc::new(Sample::new(48000, vec![1000; 16]));
|
||||
let weak = Arc::downgrade(&sample);
|
||||
audio.play_sample(sample, 0, 255, 0, 1.0, 0, LoopMode::On);
|
||||
audio.finish_frame();
|
||||
let stale = audio.transport().try_take().unwrap();
|
||||
let allocation = stale.frame.as_ref().unwrap().as_ptr();
|
||||
audio.reset();
|
||||
let mut mixer = AudioMixer::new();
|
||||
mixer.apply_delivery(audio.transport().try_take().unwrap());
|
||||
let mut packet = Packet::new();
|
||||
mixer.apply_delivery_into(stale, &mut packet);
|
||||
assert_eq!(packet.batches[0].as_ref().unwrap().as_ptr(), allocation);
|
||||
assert!(weak.upgrade().is_some());
|
||||
mixer.mix(&mut [0.0; 32]);
|
||||
assert!(mixer.voices.iter().all(|voice| !voice.active));
|
||||
drop(packet); // Main/service thread, including stale command owners.
|
||||
assert!(weak.upgrade().is_none());
|
||||
}
|
||||
|
||||
type Callback = Box<dyn FnMut(&mut [f32]) + Send>;
|
||||
struct ManualStream(Arc<std::sync::Mutex<Option<Callback>>>);
|
||||
impl AudioStreamHandle for ManualStream {}
|
||||
impl Drop for ManualStream {
|
||||
fn drop(&mut self) {
|
||||
self.0.lock().unwrap().take();
|
||||
}
|
||||
}
|
||||
struct ManualRuntime(Arc<std::sync::Mutex<Option<Callback>>>);
|
||||
impl AudioRuntime for ManualRuntime {
|
||||
type Device = ();
|
||||
type Stream = ManualStream;
|
||||
fn default_output_device(&self) -> Option<()> {
|
||||
Some(())
|
||||
}
|
||||
fn build_output_stream<D, E>(
|
||||
&self,
|
||||
_: &(),
|
||||
_: &cpal::StreamConfig,
|
||||
data: D,
|
||||
_: E,
|
||||
) -> Result<ManualStream, HostAudioError>
|
||||
where
|
||||
D: FnMut(&mut [f32]) + Send + 'static,
|
||||
E: FnMut(String) + Send + 'static,
|
||||
{
|
||||
*self.0.lock().unwrap() = Some(Box::new(data));
|
||||
Ok(ManualStream(self.0.clone()))
|
||||
}
|
||||
fn play_stream(&self, _: &ManualStream) -> Result<(), HostAudioError> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
#[test]
|
||||
fn failure_stall_and_teardown_release_callback_and_pending_owners() {
|
||||
for mode in 0..3 {
|
||||
let callbacks = Arc::new(std::sync::Mutex::new(None));
|
||||
let mut audio = Audio::new(Arc::new(MemoryBanks::new()));
|
||||
let mut host = HostAudio::new();
|
||||
host.transport = audio.transport();
|
||||
host.init_with_runtime(&ManualRuntime(callbacks.clone())).unwrap();
|
||||
let mut owners = vec![];
|
||||
for _ in 0..2 {
|
||||
let sample = Arc::new(Sample::new(48000, vec![1000; 16]));
|
||||
owners.push(Arc::downgrade(&sample));
|
||||
audio.play_sample(sample, 0, 255, 0, 1.0, 0, LoopMode::On);
|
||||
audio.finish_frame();
|
||||
callbacks.lock().unwrap().as_mut().unwrap()(&mut []);
|
||||
}
|
||||
let sample = Arc::new(Sample::new(48000, vec![1000; 16]));
|
||||
owners.push(Arc::downgrade(&sample));
|
||||
audio.play_sample(sample, 0, 255, 0, 1.0, 0, LoopMode::On);
|
||||
audio.finish_frame();
|
||||
assert!(!audio.ready());
|
||||
if mode == 0 {
|
||||
host.failed.store(true, Ordering::Relaxed);
|
||||
}
|
||||
if mode == 1 {
|
||||
host.last_progress = Instant::now() - Duration::from_secs(3);
|
||||
host.last_heartbeat = host.heartbeat.load(Ordering::Relaxed);
|
||||
}
|
||||
if mode < 2 {
|
||||
host.service();
|
||||
assert!(host._stream.is_none());
|
||||
}
|
||||
drop(host);
|
||||
assert!(audio.ready());
|
||||
assert!(callbacks.lock().unwrap().is_none());
|
||||
assert!(owners.iter().all(|owner| owner.upgrade().is_none()));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
//! Two packets circulate free -> callback -> returned -> service -> free.
|
||||
//! While the callback owns a packet, at most one packet can occupy the return
|
||||
//! lane, so its capacity-two enqueue cannot fail. Neither lane grows.
|
||||
use prometeu_drivers::{AudioCommand, AudioTransport, MAX_CHANNELS, MAX_COMMANDS_PER_FRAME};
|
||||
use prometeu_hal::sample::Sample;
|
||||
use ringbuf::{
|
||||
HeapRb,
|
||||
traits::{Consumer, Producer, Split},
|
||||
wrap::{CachingCons, CachingProd},
|
||||
};
|
||||
use std::sync::Arc;
|
||||
|
||||
pub(super) const RETIRED_CAPACITY: usize = 3 * MAX_COMMANDS_PER_FRAME + MAX_CHANNELS;
|
||||
pub(super) struct Packet {
|
||||
retired: Vec<Arc<Sample>>,
|
||||
pub batches: [Option<Vec<AudioCommand>>; 2],
|
||||
}
|
||||
impl Packet {
|
||||
pub(super) fn new() -> Self {
|
||||
Self { retired: Vec::with_capacity(RETIRED_CAPACITY), batches: [None, None] }
|
||||
}
|
||||
pub(super) fn retire(&mut self, sample: Option<Arc<Sample>>) {
|
||||
if let Some(sample) = sample {
|
||||
// Every owner comes from at most 3072 Play commands or 16 slots.
|
||||
assert!(self.retired.len() < RETIRED_CAPACITY, "audio retirement bound");
|
||||
self.retired.push(sample);
|
||||
}
|
||||
}
|
||||
fn recycle(&mut self, transport: &AudioTransport) {
|
||||
self.retired.clear();
|
||||
for batch in &mut self.batches {
|
||||
if let Some(batch) = batch.take() {
|
||||
transport.recycle_commands(batch);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
type Prod = CachingProd<Arc<HeapRb<Packet>>>;
|
||||
type Cons = CachingCons<Arc<HeapRb<Packet>>>;
|
||||
pub(super) struct CallbackLane {
|
||||
free: Cons,
|
||||
returned: Prod,
|
||||
}
|
||||
/// Main-thread endpoint. Service independently of guest progress or pause.
|
||||
pub struct AudioReclaimer {
|
||||
free: Prod,
|
||||
returned: Cons,
|
||||
}
|
||||
pub(super) fn pool() -> (CallbackLane, AudioReclaimer) {
|
||||
let (mut free, free_cons) = HeapRb::new(2).split();
|
||||
let (returned, returned_cons) = HeapRb::new(2).split();
|
||||
for _ in 0..2 {
|
||||
assert!(free.try_push(Packet::new()).is_ok());
|
||||
}
|
||||
(CallbackLane { free: free_cons, returned }, AudioReclaimer { free, returned: returned_cons })
|
||||
}
|
||||
impl CallbackLane {
|
||||
pub(super) fn take(&mut self) -> Option<Packet> {
|
||||
self.free.try_pop()
|
||||
}
|
||||
pub(super) fn give_back(&mut self, packet: Packet) {
|
||||
// Conservation proves capacity; there is no overflow/drop fallback.
|
||||
assert!(self.returned.try_push(packet).is_ok(), "audio packet conservation");
|
||||
}
|
||||
}
|
||||
impl AudioReclaimer {
|
||||
pub fn service(&mut self, transport: &AudioTransport) {
|
||||
while let Some(mut packet) = self.returned.try_pop() {
|
||||
packet.recycle(transport);
|
||||
assert!(self.free.try_push(packet).is_ok(), "audio packet conservation");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
#[test]
|
||||
fn two_packets_conserve_capacity_across_service_starvation() {
|
||||
let (mut callback, mut service) = pool();
|
||||
let first = callback.take().unwrap();
|
||||
let second = callback.take().unwrap();
|
||||
assert!(callback.take().is_none());
|
||||
callback.give_back(first);
|
||||
callback.give_back(second);
|
||||
assert!(callback.take().is_none());
|
||||
let transport = AudioTransport::default();
|
||||
for _ in 0..1000 {
|
||||
service.service(&transport);
|
||||
for _ in 0..2 {
|
||||
let packet = callback.take().unwrap();
|
||||
assert_eq!(packet.retired.capacity(), RETIRED_CAPACITY);
|
||||
assert!(packet.retired.is_empty());
|
||||
assert!(packet.batches.iter().all(Option::is_none));
|
||||
callback.give_back(packet);
|
||||
}
|
||||
assert!(callback.take().is_none());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -433,6 +433,9 @@ impl ApplicationHandler for HostRunner {
|
||||
|
||||
// Consume time in exact 60Hz slices.
|
||||
while self.accumulator >= self.frame_target_dt {
|
||||
// Reclaim returned audio owners before another guest attempt, even
|
||||
// when the previous attempt paused or stopped at backpressure.
|
||||
self.audio.service();
|
||||
// Unless the debugger is waiting for a 'start' command, advance the system.
|
||||
if !self.debugger.waiting_for_start {
|
||||
self.firmware.tick(&self.input.signals, &mut self.hardware);
|
||||
@@ -834,7 +837,7 @@ mod tests {
|
||||
// real socket bind/connect coordination. Deterministic debugger state changes
|
||||
// are covered below the host layer in `debugger.rs`.
|
||||
#[test]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `cargo test -p prometeu-host-desktop-winit --lib -- --ignored`"]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `make test-debugger-socket`"]
|
||||
fn test_debug_port_opens() {
|
||||
let mut runner = HostRunner::new(None, None);
|
||||
let port = 9999;
|
||||
@@ -894,7 +897,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `cargo test -p prometeu-host-desktop-winit --lib -- --ignored`"]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `make test-debugger-socket`"]
|
||||
fn test_debug_reconnection() {
|
||||
let mut runner = HostRunner::new(None, None);
|
||||
let port = 9998;
|
||||
@@ -937,7 +940,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `cargo test -p prometeu-host-desktop-winit --lib -- --ignored`"]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `make test-debugger-socket`"]
|
||||
fn test_debug_refuse_second_connection() {
|
||||
let mut runner = HostRunner::new(None, None);
|
||||
let port = 9997;
|
||||
@@ -973,7 +976,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `cargo test -p prometeu-host-desktop-winit --lib -- --ignored`"]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `make test-debugger-socket`"]
|
||||
fn test_get_state_returns_response() {
|
||||
let mut runner = HostRunner::new(None, None);
|
||||
let port = 9996;
|
||||
@@ -1018,7 +1021,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `cargo test -p prometeu-host-desktop-winit --lib -- --ignored`"]
|
||||
#[ignore = "requires localhost TCP bind/connect; run via `make test-debugger-socket`"]
|
||||
fn test_debug_resume_on_disconnect() {
|
||||
let mut runner = HostRunner::new(None, None);
|
||||
let port = 9995;
|
||||
|
||||
@@ -0,0 +1,347 @@
|
||||
use prometeu_drivers::{Audio, MemoryBanks};
|
||||
use prometeu_hal::{AudioBridge, LoopMode, sample::Sample};
|
||||
use prometeu_host_desktop_winit::audio::AudioCallback;
|
||||
use std::alloc::{GlobalAlloc, Layout, System};
|
||||
use std::cell::Cell;
|
||||
use std::sync::Arc;
|
||||
|
||||
thread_local! {
|
||||
static COUNTS: Cell<Option<(usize, usize)>> = const { Cell::new(None) };
|
||||
}
|
||||
struct Counter;
|
||||
fn count(alloc: usize, free: usize) {
|
||||
let _ = COUNTS.try_with(|c| {
|
||||
if let Some((a, d)) = c.get() {
|
||||
c.set(Some((a + alloc, d + free)));
|
||||
}
|
||||
});
|
||||
}
|
||||
unsafe impl GlobalAlloc for Counter {
|
||||
unsafe fn alloc(&self, l: Layout) -> *mut u8 {
|
||||
count(1, 0);
|
||||
unsafe { System.alloc(l) }
|
||||
}
|
||||
unsafe fn alloc_zeroed(&self, l: Layout) -> *mut u8 {
|
||||
count(1, 0);
|
||||
unsafe { System.alloc_zeroed(l) }
|
||||
}
|
||||
unsafe fn dealloc(&self, p: *mut u8, l: Layout) {
|
||||
count(0, 1);
|
||||
unsafe { System.dealloc(p, l) }
|
||||
}
|
||||
unsafe fn realloc(&self, p: *mut u8, l: Layout, n: usize) -> *mut u8 {
|
||||
count(1, 1);
|
||||
unsafe { System.realloc(p, l, n) }
|
||||
}
|
||||
}
|
||||
#[global_allocator]
|
||||
static ALLOCATOR: Counter = Counter;
|
||||
|
||||
fn measured(f: impl FnOnce()) -> (usize, usize) {
|
||||
COUNTS.with(|c| c.set(Some((0, 0))));
|
||||
f();
|
||||
COUNTS.with(|c| c.replace(None).unwrap())
|
||||
}
|
||||
|
||||
fn play(audio: &mut Audio, voice: usize, looping: bool) -> std::sync::Weak<Sample> {
|
||||
let sample = Arc::new(Sample::new(48000, vec![1000; 4]));
|
||||
let weak = Arc::downgrade(&sample);
|
||||
assert_eq!(
|
||||
audio.play_sample(
|
||||
sample,
|
||||
voice,
|
||||
255,
|
||||
0,
|
||||
1.0,
|
||||
0,
|
||||
if looping { LoopMode::On } else { LoopMode::Off }
|
||||
),
|
||||
prometeu_hal::AudioOpStatus::Ok
|
||||
);
|
||||
weak
|
||||
}
|
||||
fn render(callback: &mut AudioCallback, audio: &Audio, pcm: &mut [f32], label: &str) {
|
||||
let counts = measured(|| callback.render(&audio.transport(), pcm));
|
||||
println!("{label}: {counts:?}");
|
||||
assert_eq!(counts, (0, 0), "{label}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn callback_owners_do_not_use_allocator() {
|
||||
let mut audio = Audio::new(Arc::new(MemoryBanks::new()));
|
||||
let (mut callback, mut reclaimer) = AudioCallback::new();
|
||||
let mut pcm = [0.0; 32];
|
||||
let weak = play(&mut audio, 0, false);
|
||||
audio.finish_frame();
|
||||
render(&mut callback, &audio, &mut pcm, "natural_completion_and_batch");
|
||||
assert!(weak.upgrade().is_some(), "owner must await service");
|
||||
reclaimer.service(&audio.transport());
|
||||
assert!(weak.upgrade().is_none());
|
||||
for _ in 0..8 {
|
||||
play(&mut audio, 0, true);
|
||||
audio.stop(0);
|
||||
let replacement = play(&mut audio, 0, true);
|
||||
audio.finish_frame();
|
||||
render(&mut callback, &audio, &mut pcm, "replacement_stop_fifo");
|
||||
assert!(pcm.iter().step_by(2).all(|&v| v == 1000.0 / 32768.0));
|
||||
reclaimer.service(&audio.transport());
|
||||
audio.set_paused(true);
|
||||
render(&mut callback, &audio, &mut pcm, "pause");
|
||||
assert!(pcm.iter().all(|&v| v == 0.0));
|
||||
reclaimer.service(&audio.transport());
|
||||
audio.reset();
|
||||
render(&mut callback, &audio, &mut pcm, "epoch_reset");
|
||||
assert!(replacement.upgrade().is_some());
|
||||
reclaimer.service(&audio.transport());
|
||||
assert!(replacement.upgrade().is_none());
|
||||
}
|
||||
audio.stop(0);
|
||||
audio.finish_frame();
|
||||
render(&mut callback, &audio, &mut pcm, "control_batch");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stalled_service_bounds_retention_and_recovers_pending_delivery() {
|
||||
let mut audio = Audio::new(Arc::new(MemoryBanks::new()));
|
||||
let (mut callback, mut reclaimer) = AudioCallback::new();
|
||||
let owners: Vec<_> = (0..16).map(|voice| play(&mut audio, voice, false)).collect();
|
||||
audio.finish_frame();
|
||||
// Keep all voices playing until both packets have been returned.
|
||||
render(&mut callback, &audio, &mut [], "start_16");
|
||||
render(&mut callback, &audio, &mut [], "exhaust_pool");
|
||||
let mut pcm = [0.0; 32];
|
||||
render(&mut callback, &audio, &mut pcm, "finish_16_without_packet");
|
||||
assert!(owners.iter().all(|owner| owner.upgrade().is_some()));
|
||||
let next = play(&mut audio, 0, true);
|
||||
audio.finish_frame();
|
||||
for _ in 0..10 {
|
||||
render(&mut callback, &audio, &mut pcm, "starved_silent_pending");
|
||||
assert!(!audio.ready());
|
||||
assert!(pcm.iter().all(|&v| v == 0.0));
|
||||
}
|
||||
reclaimer.service(&audio.transport());
|
||||
render(&mut callback, &audio, &mut pcm, "recover");
|
||||
assert!(audio.ready());
|
||||
reclaimer.service(&audio.transport());
|
||||
assert!(owners.iter().all(|owner| owner.upgrade().is_none()));
|
||||
assert!(next.upgrade().is_some());
|
||||
drop(callback); // Teardown belongs to the service/main thread.
|
||||
drop(reclaimer);
|
||||
assert!(next.upgrade().is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn full_frame_and_combined_lifecycle_fit_one_packet() {
|
||||
let mut audio = Audio::new(Arc::new(MemoryBanks::new()));
|
||||
let (mut callback, mut reclaimer) = AudioCallback::new();
|
||||
let mut owners = Vec::new();
|
||||
for voice in 0..16 {
|
||||
owners.push(play(&mut audio, voice, false));
|
||||
}
|
||||
audio.finish_frame();
|
||||
render(&mut callback, &audio, &mut [], "initial_16");
|
||||
reclaimer.service(&audio.transport());
|
||||
// The published frame, interrupted next-frame prefix and lifecycle each
|
||||
// admit 1024 commands. Every command replaces an owner on the same voice.
|
||||
for _ in 0..1024 {
|
||||
owners.push(play(&mut audio, 0, false));
|
||||
}
|
||||
audio.finish_frame();
|
||||
for _ in 0..1024 {
|
||||
owners.push(play(&mut audio, 0, false));
|
||||
}
|
||||
audio.begin_lifecycle();
|
||||
for _ in 0..1024 {
|
||||
owners.push(play(&mut audio, 0, false));
|
||||
}
|
||||
audio.finish_lifecycle(true);
|
||||
let mut pcm = [0.0; 32];
|
||||
render(&mut callback, &audio, &mut pcm, "3072_commands_plus_completion");
|
||||
reclaimer.service(&audio.transport());
|
||||
assert!(owners.iter().skip(16).all(|owner| owner.upgrade().is_none()));
|
||||
audio.reset();
|
||||
render(&mut callback, &audio, &mut pcm, "reset_remaining_voices");
|
||||
reclaimer.service(&audio.transport());
|
||||
assert!(owners.iter().all(|owner| owner.upgrade().is_none()));
|
||||
}
|
||||
|
||||
/// Physical observations, deliberately excluded from ordinary CI. The pressure
|
||||
/// fixture reads the checked-in asset pack and writes a temporary host save;
|
||||
/// it does not certify asset decoding or guest memcard scheduling.
|
||||
#[test]
|
||||
#[ignore = "requires physical audio output; 16 quiet voices and temporary file I/O"]
|
||||
fn physical_callback_duration_16_voices_with_io_pressure() {
|
||||
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
|
||||
use prometeu_host_desktop_winit::fs_backend::HostDirBackend;
|
||||
use prometeu_system::fs::FsBackend;
|
||||
use ringbuf::{
|
||||
HeapRb,
|
||||
traits::{Consumer, Producer, Split},
|
||||
};
|
||||
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
|
||||
use std::time::{Duration, Instant};
|
||||
let device = cpal::default_host().default_output_device().expect("physical output");
|
||||
let device_name = device.name().unwrap();
|
||||
let mut audio = Audio::new(Arc::new(MemoryBanks::new()));
|
||||
let (mut callback, mut reclaimer) = AudioCallback::new();
|
||||
let mut owners = vec![];
|
||||
for voice in 0..16 {
|
||||
let sample = Arc::new(Sample::new(48000, vec![32; 4800]));
|
||||
owners.push(Arc::downgrade(&sample));
|
||||
audio.play_sample(sample, voice, 8, 127, 1.0, 0, LoopMode::On);
|
||||
}
|
||||
audio.finish_frame();
|
||||
let config = cpal::StreamConfig {
|
||||
channels: 2,
|
||||
sample_rate: cpal::SampleRate(48000),
|
||||
buffer_size: cpal::BufferSize::Fixed(256),
|
||||
};
|
||||
let transport = audio.transport();
|
||||
let (mut timings, mut observations) = HeapRb::<(u64, usize)>::new(4096).split();
|
||||
let allocator_calls = Arc::new(AtomicUsize::new(0));
|
||||
let failed = Arc::new(AtomicBool::new(false));
|
||||
let counter = allocator_calls.clone();
|
||||
let error = failed.clone();
|
||||
let stream = device
|
||||
.build_output_stream(
|
||||
&config,
|
||||
move |data: &mut [f32], _: &cpal::OutputCallbackInfo| {
|
||||
let start = Instant::now();
|
||||
let (alloc, free) = measured(|| callback.render(&transport, data));
|
||||
counter.fetch_add(alloc + free, Ordering::Relaxed);
|
||||
let _ = timings.try_push((start.elapsed().as_nanos() as u64, data.len() / 2));
|
||||
},
|
||||
move |_| {
|
||||
error.store(true, Ordering::Relaxed);
|
||||
},
|
||||
None,
|
||||
)
|
||||
.unwrap();
|
||||
let root =
|
||||
std::env::temp_dir().join(format!("prometeu-audio-qualification-{}", std::process::id()));
|
||||
std::fs::create_dir_all(&root).unwrap();
|
||||
let mut filesystem = HostDirBackend::new(root.clone());
|
||||
let asset = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.join("../../../test-cartridges/stress-console/assets.pa");
|
||||
let save = vec![0x5a; 1024 * 1024];
|
||||
stream.play().unwrap();
|
||||
let deadline = Instant::now() + Duration::from_secs(2);
|
||||
let mut io_cycles = 0;
|
||||
while Instant::now() < deadline {
|
||||
let payload = std::fs::read(&asset).unwrap();
|
||||
assert!(!payload.is_empty());
|
||||
filesystem.write_file("/user/audio-pressure.bin", &save).unwrap();
|
||||
reclaimer.service(&audio.transport());
|
||||
io_cycles += 1;
|
||||
std::thread::sleep(Duration::from_millis(2));
|
||||
}
|
||||
stream.pause().unwrap();
|
||||
let live_before = owners.iter().filter(|owner| owner.upgrade().is_some()).count();
|
||||
drop(stream);
|
||||
reclaimer.service(&audio.transport());
|
||||
let live_after = owners.iter().filter(|owner| owner.upgrade().is_some()).count();
|
||||
let mut samples = vec![];
|
||||
let mut frames = vec![];
|
||||
while let Some((time, count)) = observations.try_pop() {
|
||||
samples.push(time);
|
||||
frames.push(count);
|
||||
}
|
||||
filesystem.delete("/user/audio-pressure.bin").unwrap();
|
||||
std::fs::remove_dir(root.join("user")).unwrap();
|
||||
std::fs::remove_dir(root).unwrap();
|
||||
samples.sort_unstable();
|
||||
assert!(!samples.is_empty(), "physical stream delivered no callbacks");
|
||||
println!(
|
||||
"device={device_name:?}, callbacks={}, io_cycles={io_cycles}, frames_min={}, frames_max={}, p50_ns={}, p95_ns={}, p99_ns={}, max_ns={}, allocator_calls={}, owners_before={live_before}, owners_after={live_after}",
|
||||
samples.len(),
|
||||
frames.iter().min().unwrap(),
|
||||
frames.iter().max().unwrap(),
|
||||
samples[samples.len() / 2],
|
||||
samples[samples.len() * 95 / 100],
|
||||
samples[samples.len() * 99 / 100],
|
||||
samples.last().unwrap(),
|
||||
allocator_calls.load(Ordering::Relaxed)
|
||||
);
|
||||
assert!(!failed.load(Ordering::Relaxed));
|
||||
assert_eq!(allocator_calls.load(Ordering::Relaxed), 0);
|
||||
assert_eq!((live_before, live_after), (16, 0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "release timing observation with temporary asset-read/save-write pressure"]
|
||||
fn callback_duration_16_voices_with_io_pressure() {
|
||||
use prometeu_host_desktop_winit::fs_backend::HostDirBackend;
|
||||
use prometeu_system::fs::FsBackend;
|
||||
use std::time::Instant;
|
||||
let mut audio = Audio::new(Arc::new(MemoryBanks::new()));
|
||||
let (mut callback, mut reclaimer) = AudioCallback::new();
|
||||
let owners: Vec<_> = (0..16).map(|voice| play(&mut audio, voice, true)).collect();
|
||||
audio.finish_frame();
|
||||
let mut pcm = [0.0; 512]; // 256 stereo frames: nominal 5.333 ms at 48 kHz.
|
||||
callback.render(&audio.transport(), &mut pcm);
|
||||
reclaimer.service(&audio.transport());
|
||||
let root = std::env::temp_dir().join(format!("prometeu-audio-timing-{}", std::process::id()));
|
||||
std::fs::create_dir_all(&root).unwrap();
|
||||
let mut filesystem = HostDirBackend::new(root.clone());
|
||||
let asset = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.join("../../../test-cartridges/stress-console/assets.pa");
|
||||
let save = vec![0x5a; 1024 * 1024];
|
||||
let mut samples = Vec::with_capacity(1000);
|
||||
let mut counts = (0, 0);
|
||||
for _ in 0..1000 {
|
||||
assert!(!std::fs::read(&asset).unwrap().is_empty());
|
||||
filesystem.write_file("/user/audio-pressure.bin", &save).unwrap();
|
||||
let start = Instant::now();
|
||||
let measured = measured(|| callback.render(&audio.transport(), &mut pcm));
|
||||
samples.push(start.elapsed().as_nanos());
|
||||
counts.0 += measured.0;
|
||||
counts.1 += measured.1;
|
||||
reclaimer.service(&audio.transport());
|
||||
}
|
||||
samples.sort_unstable();
|
||||
let before = owners.iter().filter(|owner| owner.upgrade().is_some()).count();
|
||||
drop(callback);
|
||||
drop(reclaimer);
|
||||
let after = owners.iter().filter(|owner| owner.upgrade().is_some()).count();
|
||||
filesystem.delete("/user/audio-pressure.bin").unwrap();
|
||||
std::fs::remove_dir(root.join("user")).unwrap();
|
||||
std::fs::remove_dir(root).unwrap();
|
||||
println!(
|
||||
"synthetic callbacks=1000, frames=256, nominal_period_ns=5333333, io_cycles=1000, p50_ns={}, p95_ns={}, p99_ns={}, max_ns={}, alloc_free={counts:?}, owners_before={before}, owners_after={after}",
|
||||
samples[500], samples[950], samples[990], samples[999]
|
||||
);
|
||||
assert_eq!(counts, (0, 0));
|
||||
assert_eq!((before, after), (16, 0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unloaded_banks_empty_samples_and_controls_reclaim_on_service() {
|
||||
use prometeu_drivers::SoundBankPoolInstaller;
|
||||
use prometeu_hal::sound_bank::SoundBank;
|
||||
let banks = Arc::new(MemoryBanks::new());
|
||||
let sample = Arc::new(Sample::new(48000, vec![1000; 16]));
|
||||
let weak = Arc::downgrade(&sample);
|
||||
banks.install_sound_bank(0, Arc::new(SoundBank::new(vec![sample])));
|
||||
let mut audio = Audio::new(banks.clone());
|
||||
let (mut callback, mut reclaimer) = AudioCallback::new();
|
||||
audio.play(0, 0, 0, 255, 0, 1.0, 0, LoopMode::On);
|
||||
audio.finish_frame();
|
||||
banks.clear_sound_banks();
|
||||
let mut pcm = [0.0; 16];
|
||||
render(&mut callback, &audio, &mut pcm, "unloaded_bank");
|
||||
assert!(weak.upgrade().is_some());
|
||||
reclaimer.service(&audio.transport());
|
||||
audio.set_volume(0, 127);
|
||||
audio.set_pan(0, 255);
|
||||
audio.set_pitch(0, 0.5);
|
||||
audio.stop(0);
|
||||
let empty = Arc::new(Sample::new(48000, vec![]));
|
||||
let weak_empty = Arc::downgrade(&empty);
|
||||
audio.play_sample(empty, 1, 255, 0, 1.0, 0, LoopMode::Off);
|
||||
audio.finish_frame();
|
||||
render(&mut callback, &audio, &mut pcm, "controls_stop_empty");
|
||||
assert!(pcm.iter().all(|&sample| sample == 0.0));
|
||||
assert!(weak.upgrade().is_some() && weak_empty.upgrade().is_some());
|
||||
reclaimer.service(&audio.transport());
|
||||
assert!(weak.upgrade().is_none() && weak_empty.upgrade().is_none());
|
||||
}
|
||||
+78
-2
@@ -58,8 +58,33 @@ or symlink aliases while producing plausible percentages.
|
||||
**Trade-offs:**
|
||||
New crates require explicit ownership. Reports with missing domains, duplicate
|
||||
sources or zero instrumented lines are rejected before threshold comparison.
|
||||
The current source map assigns 232 files to 15 exclusive owners without
|
||||
changing the original eight domain baselines.
|
||||
The initial ownership extension assigned 232 files to 15 exclusive owners
|
||||
without changing the original eight domain baselines. Source counts grow with
|
||||
the repository; the current evidence records the measured layout.
|
||||
|
||||
### Exclusive ownership must also be the intended ownership
|
||||
|
||||
**What:**
|
||||
Module entry files and their sibling directories belong to the same specialized
|
||||
domain. Memcard descendants belong to system/fs; asset-driver descendants
|
||||
belong to asset/bank. Both residual owners exclude the entry file and the whole
|
||||
module tree. Regression tests call the production auditor's ownership function
|
||||
for current and future paths, and audit independent reports containing absent
|
||||
and zero-line descendants.
|
||||
|
||||
**Why:**
|
||||
A prefix ending in `.rs` covers the entry file but not the directory created
|
||||
when a module is split. A residual owner can absorb every new descendant and
|
||||
still pass a generic exactly-one-owner audit. The resulting percentages look
|
||||
valid but measure the wrong responsibility. Expected-owner assertions detect
|
||||
this failure before instrumentation or percentage comparison.
|
||||
|
||||
**Trade-offs:**
|
||||
Correcting ownership moves the denominator and may reduce a specialized
|
||||
domain's margin. Preserve reviewed floors and investigate fresh failures with
|
||||
behavioral evidence. Historical measurements remain useful only when their
|
||||
original source map and fingerprints remain intact; they should not be
|
||||
silently rewritten to match the repaired map.
|
||||
|
||||
## Patterns and Algorithms
|
||||
|
||||
@@ -80,6 +105,8 @@ changing the original eight domain baselines.
|
||||
- Allowing an absent or zero-line report to pass a zero baseline.
|
||||
- Assigning the whole VM to the system integration domain, or counting one
|
||||
source in both a specialized owner and its enclosing residual crate owner.
|
||||
- Treating exactly one owner as proof that a module retained its intended
|
||||
owner after its implementation moved into sibling source files.
|
||||
- Treating local host measurements as evidence for unexecuted platforms.
|
||||
|
||||
## Published Extension and Evidence
|
||||
@@ -97,6 +124,18 @@ Mathematics and its portable MPFR-generated fixture now exercise vm/core;
|
||||
bytecode has its own gate. The standalone MPFR qualification remains separate.
|
||||
This evidence does not claim Linux or Intel Mac coverage execution.
|
||||
|
||||
The module-tree ownership correction derives from accepted DEC-0059. Its
|
||||
[current measured evidence](../../../docs/vm-arch/COVERAGE_DOMAINS_CURRENT.json)
|
||||
preserves the earlier evidence records and restores system/core to 87.71%
|
||||
without changing the other 14 floors. The initial repaired map had 257 source files
|
||||
and 15 exclusive domains. Its domain gates passed: system/fs measured 78.73%,
|
||||
system/core 93.69%, asset/bank 90.52% and drivers/core 89.17%.
|
||||
Ten policy tests cover both generic validity and intended specialized owners.
|
||||
That initial repository target passed 1,021 tests with 11 ignored while strict
|
||||
workspace Clippy still reported VM diagnostics. The final strict gate subsequently
|
||||
passed on the completed source, as recorded in the [runtime quality lesson](../DSC-0064-runtime-branch-quality-review/LSN-0070-runtime-quality-needs-independent-oracles-and-owned-evidence.md).
|
||||
The current evidence retains the initial ownership comparison separately.
|
||||
|
||||
## Takeaways
|
||||
|
||||
- Runtime-edge coverage is safer when CI enforces domain ownership instead of trusting a workspace aggregate.
|
||||
@@ -104,3 +143,40 @@ This evidence does not claim Linux or Intel Mac coverage execution.
|
||||
- Coverage gates that live in repository helpers are easier to evolve, audit, and keep consistent than CI-local logic.
|
||||
- Audit source ownership and evidence validity before comparing percentages.
|
||||
A passing percentage is useful only when its denominator has a clear owner.
|
||||
|
||||
## A Shared Gate Must Share Failure Semantics Too
|
||||
|
||||
The strict-gate correction derives from DEC-0063. A common coverage helper did
|
||||
not establish common CI policy while one pipeline only ran permissive lint/tests
|
||||
and another produced domain reports. Both CI consumers now provision the declared
|
||||
tools and invoke the same Makefile-owned `ci-domains` target. Strict Clippy covers
|
||||
workspace targets and features with warnings denied; concrete lint fixes replace
|
||||
warning suppression. The existing coverage map and floors remain unchanged.
|
||||
|
||||
The Makefile owns the ordered steps. A serial runner performs format, strict lint,
|
||||
all-target tests, the five named debugger socket cases, fresh coverage/export and
|
||||
all domain audits/gates. Serial child Make invocations avoid races between clean,
|
||||
Cargo execution and report generation. A second independent gate in the same
|
||||
checkout still requires operator coordination.
|
||||
|
||||
Failure publication is part of the contract. Before a run, old reports are
|
||||
removed; every executed step receives a log, exit status, revision and source/config
|
||||
identity. A later failure retains current partial reports. An early failure cannot
|
||||
archive old HTML as current success. Both CI providers publish available evidence
|
||||
on failure and preserve the gate's nonzero status. Regression fixtures exercise
|
||||
failed execution, success and failure after partial report generation.
|
||||
|
||||
Ignored tests are not one qualification category. Five localhost socket tests
|
||||
belong in automation, but physical audio, endurance, packaged installation and
|
||||
manual timing/layout probes need their own named execution. Restricting the socket
|
||||
filter prevents accidentally turning a device requirement into a CI prerequisite.
|
||||
|
||||
The local macOS/aarch64 strict gate passed 1,039 ordinary tests, five socket tests,
|
||||
three runner regressions and all fifteen coverage gates; fresh coverage separately
|
||||
replayed 1,039 tests. The ordinary phases reported thirteen ignored tests; five
|
||||
of these ran in the separate socket step, while eight remained unselected. The initial sandbox
|
||||
socket refusal is distinguished from the successful authorized run. Pipeline
|
||||
configuration was inspected, but no remote or Linux gate was executed. Platform
|
||||
equivalence concerns required commands, not identical measured coverage. See the
|
||||
[published operational contract and evidence](../../../docs/vm-arch/COVERAGE_DOMAINS.md)
|
||||
for prerequisites, manual commands and practical limits.
|
||||
|
||||
+29
@@ -86,6 +86,35 @@ surfaces unless the caller violated the structural ABI.
|
||||
Do not reopen FS public API scope while wiring internal async-lane consumers.
|
||||
The lane can support FS-style work without deciding request/poll semantics.
|
||||
|
||||
## Share format rules without sharing execution strategy
|
||||
|
||||
The glyph, sound and scene decoders now consume one private driver validation
|
||||
module. Complete-buffer decoding can materialize a whole payload, while the
|
||||
shared worker retains state and reads at most 4096 bytes per decode step. Sharing
|
||||
validation does not require replacing cooperative IO with a full-buffer read.
|
||||
|
||||
A differential test alone is insufficient: both paths can agree on a mistake.
|
||||
Use independent expected banks and rejection fixtures, then compare each entry
|
||||
point against those expectations. Include physical file carriers: an in-memory
|
||||
slice rejects an out-of-range request before a decoder runs, which can conceal
|
||||
a short-file discrepancy in a reader that accepts EOF. The initial corpus found
|
||||
that legacy PCM and scene loads accepted missing declared bytes. After an
|
||||
explicit compatibility decision, both modes reject those files while still
|
||||
ignoring a physically present odd PCM tail.
|
||||
|
||||
The production pressure regression pauses the real decoder after a safe step,
|
||||
queues a persistent memcard save, and observes save writes and synchronization
|
||||
before the next asset step. It checks all three multi-chunk asset kinds, the
|
||||
single worker identity, complete output, deferred frame publication and saved
|
||||
data after remount. This establishes priority and retained progress through
|
||||
observable ordering, rather than a host-specific latency threshold. Separate
|
||||
regressions cover cancellation, superseding and preload publication.
|
||||
|
||||
This refinement follows DEC-0064, summarized in the [runtime quality lesson](../DSC-0064-runtime-branch-quality-review/LSN-0070-runtime-quality-needs-independent-oracles-and-owned-evidence.md).
|
||||
The published contract is [Asset Management](../../../docs/specs/runtime/15-asset-management.md#43-decoding-validation-and-physical-payload-completeness).
|
||||
The ordering fixture uses the persistent transaction protocol with a deterministic
|
||||
storage backend; file-host latency measurements remain separate evidence.
|
||||
|
||||
## Takeaways
|
||||
|
||||
- A runtime async lane is an ownership boundary, not just an implementation
|
||||
|
||||
+25
@@ -67,3 +67,28 @@ String concatenation still creates a new string when the language semantics requ
|
||||
- The right performance question is often "where is the repeated cost?" rather than "where is any cost at all?"
|
||||
- Immutable payload sharing is a strong default when public semantics must remain stable and hot-path copies are the real issue.
|
||||
- Internal telemetry should help engineering decisions without silently becoming a product contract.
|
||||
|
||||
## Successful execution can allocate without materializing guest values
|
||||
|
||||
The later stack/return correction (DEC-0061) demonstrated another source of
|
||||
recurring native allocations: eager diagnostic Strings and disposable return
|
||||
storage. A warmed scalar call/return loop allocated and freed 3,000 times per
|
||||
1,000 iterations; lazy diagnostics and in-place return transfer reduced both
|
||||
counts to zero in debug and release. The original string materialization evidence
|
||||
above remains a separate observation.
|
||||
|
||||
Validate before moving owners. Rotating only the returning frame's suffix and
|
||||
then truncating locals preserves ordered logical groups, Arc identity and heap
|
||||
aliases without cloning or scratch storage. Remove the frame after transfer so
|
||||
the existing last-root safepoint sees the results. GC scratch refusal is a later
|
||||
failure boundary: transferred roots must remain intact and sweeping must not
|
||||
start after failed marking admission.
|
||||
|
||||
Account for storage that actually exists. Eliminating temporary result storage
|
||||
can make a return succeed under a zero temporary budget without changing any
|
||||
quota ceiling. A triggered collector still needs its own admitted scratch.
|
||||
|
||||
The [execution allocation qualification](../../../docs/vm-arch/execution-allocation-qualification.md)
|
||||
records instruction regions, before/after counts and limits. The
|
||||
[memory contract](../../../docs/specs/runtime/03-memory-stack-heap-and-allocation.md)
|
||||
remains normative. These measurements say nothing about gameplay throughput.
|
||||
|
||||
+46
@@ -46,6 +46,18 @@ The mixer also reduces large pitch steps before multiplication when wrapping loo
|
||||
|
||||
## Evidence and Limits
|
||||
|
||||
### Sample Metadata Does Not Select Playback Mode
|
||||
|
||||
Loop markers describe an optional interval within a sample; the playback command decides whether to use it. Applying a loop end to one-shot playback silently discards the tail, even though the data remains available. The mixer now selects the full sample boundary when looping is off. When looping is on, a complete interval satisfying `start < end <= length` preserves the intro and wraps at the selected end; absent, partial or invalid markers fall back to the full sample. Empty samples terminate before indexing.
|
||||
|
||||
Interpolation follows the same selected boundary. A one-shot interpolates into the actual tail and finally toward zero, while a loop seam interpolates toward the loop-start frame. Distinct intro, loop and tail values make these behaviors observable: an eight-frame sample with markers `(2, 4)` emits all eight frames once with looping off, and emits frame indices `0, 1, 2, 3, 2, 3, 2, 3` with looping on. Constant PCM alone cannot detect this defect.
|
||||
|
||||
The correction derives from DEC-0060 in DSC-0064. Synthetic regressions cover exact PCM, fractional seams and tails, incomplete and invalid markers, empty samples, rate conversion, multiple wraps and extreme finite pitch. These tests exercise the existing delivery and control paths without establishing new device timing or reclamation guarantees. The [sample contract](../../../docs/specs/runtime/05-audio-peripheral.md#6-samples-interpolation-and-ownership) remains normative.
|
||||
|
||||
Local macOS/aarch64 qualification on 2026-10-14 passed 19 host audio tests (one physical-device test ignored), nine driver audio tests and ten runtime audio tests. The existing `make ci-domains` target passed with 1,027 tests passed, zero failed and 11 ignored, including all 15 domain gates. Strict all-target/all-feature Clippy passed for the desktop host. That initial run preceded repair of existing VM lint errors. The later strict shared gate passed on the completed source; see the [runtime quality lesson](../DSC-0064-runtime-branch-quality-review/LSN-0070-runtime-quality-needs-independent-oracles-and-owned-evidence.md) for final acceptance and complementary production decode/save pressure evidence.
|
||||
|
||||
### Original Voice-Control Qualification
|
||||
|
||||
Validation was run locally on macOS/aarch64 on 2026-10-14:
|
||||
|
||||
- `cargo test --workspace --all-targets --all-features --no-fail-fast`: **1,016 passed**, zero failed, eleven ignored across 55 reported suites.
|
||||
@@ -65,3 +77,37 @@ The API is a foreground PCM control surface. It does not establish musical seque
|
||||
- Keep shutdown, pause and session invalidation independent of guest saturation.
|
||||
- Place completion-sensitive state with the component that actually observes completion.
|
||||
- Test both explicit and implicit runtime boundaries, including debugger transitions.
|
||||
|
||||
## Completion and Reclamation Have Different Owners
|
||||
|
||||
The callback reclamation correction derives from DEC-0062. A nonblocking mailbox
|
||||
alone did not make the callback allocator-free: dropping a consumed Vec freed its
|
||||
storage, and finishing a voice could free the final Sample and PCM allocation.
|
||||
The useful boundary is ownership destruction, not just explicit allocation calls.
|
||||
|
||||
Two preallocated packets now move discarded references and vector allocations to
|
||||
main-thread service. The bound follows from admitted input: 1,024 frame commands,
|
||||
2,048 combined lifecycle commands and sixteen initial voices imply at most 3,088
|
||||
retired owners and two vectors per packet. Conservation, rather than an overflow
|
||||
fallback, guarantees room to return a packet. When both await service, publications
|
||||
remain pending and naturally finished voices become silent while retaining at most
|
||||
sixteen references in their existing slots. A stalled reclaimer cannot create an
|
||||
unbounded queue or resurrect a finished voice.
|
||||
|
||||
The original signal observations above remain valid, but physical release now
|
||||
happens during service. Service runs before guest attempts even during pause or
|
||||
backpressure. It clears retired owners and reuses bounded empty command storage.
|
||||
Stream shutdown precedes silent consumption, preserving a single transport consumer.
|
||||
Tests distinguish logical silence, delayed retention and eventual destruction with
|
||||
weak references; owner release is no longer inferred merely from silent PCM.
|
||||
|
||||
The [callback qualification](../../../docs/vm-arch/audio-callback-qualification.md)
|
||||
records zero runtime allocator calls in debug/release paths, the capacity-saturation
|
||||
case and local device timing. A physical release run observed 375 callbacks on
|
||||
MacBook Pro Speakers with 256-frame buffers and a 30.458 µs maximum runtime duration;
|
||||
all sixteen owners released at teardown. Its asset-pack reads and temporary 1 MiB
|
||||
host-file writes are an explicitly limited pressure workload, not guest save or
|
||||
asset-pipeline certification. Earlier device-open/stall failures are retained in
|
||||
the evidence rather than erased by a successful retry. The
|
||||
[audio specification](../../../docs/specs/runtime/05-audio-peripheral.md) defines
|
||||
the bounded reclamation contract; timing measurements do not define device deadlines.
|
||||
|
||||
+30
@@ -109,6 +109,36 @@ AGD-0055's future plan. It does not approve that agenda's proposal, weaken the
|
||||
final PBX contract or label transferred code as executed. Other agendas retain
|
||||
their own collection/string/stress/game obligations.
|
||||
|
||||
## Index ownership identity without owning content
|
||||
|
||||
Provenance is an allocation-identity question, not an equality-of-text question.
|
||||
The image index sorts the existing Weak records by address and length during
|
||||
sealed loading. The session index uses a half-full open-addressed table. Both
|
||||
preserve Arc values and explicit native leases, so ordinary string reads remain
|
||||
direct and accounting does not keep content alive strongly.
|
||||
|
||||
The allocation lifetime matters as much as the key. A Weak prevents the allocator
|
||||
from recycling an address while its record remains indexed. Remove the key before
|
||||
dropping that Weak. Tombstones preserve collision chains, and a metered rebuild
|
||||
removes them without losing live identities. Admit the new capacity and its
|
||||
coexistence with old storage before moving entries or constructing new strings;
|
||||
failed admission must leave live owners recognizable and unpublished charges
|
||||
refunded.
|
||||
|
||||
The [release qualification](../../../docs/vm-arch/string-provenance-qualification.md)
|
||||
measured the existing linear boundary queries before changing them. At 65,536
|
||||
owners the indexed image took 16–17 comparisons and the session fixtures took
|
||||
1–2 probes, with no lookup allocation. The tradeoff is explicit: session metadata
|
||||
rose from about 4 MiB to 10 MiB and setup became more expensive under the same
|
||||
quotas. This is boundary-work evidence, not a claim of faster game frames.
|
||||
Synthetic collision keys and work-limit tests complement real-address timings;
|
||||
allocator-selected address order is not a stable test oracle across loads.
|
||||
|
||||
This implementation follows DEC-0065, summarized in the [runtime quality lesson](../DSC-0064-runtime-branch-quality-review/LSN-0070-runtime-quality-needs-independent-oracles-and-owned-evidence.md).
|
||||
The [published ownership contract](../../../docs/specs/runtime/02c-program-registry-and-pbx-v1.md#memory-ownership-and-quotas)
|
||||
remains normative; native import, snapshot retention and typed storage tests
|
||||
verify the same owner lifetimes through the new indices.
|
||||
|
||||
## Takeaways
|
||||
|
||||
- Keep each plan atomic to its originating agenda and identify integration ownership before execution.
|
||||
|
||||
+23
-2
@@ -40,12 +40,33 @@ already completed result, which exposed an ownership leak during integration.
|
||||
|
||||
Cancellation and entry into publication share an atomic control state. At the
|
||||
deadline, a canceled attempt that never entered publication can report failure.
|
||||
Once publication may have started, the result remains indeterminate until
|
||||
reconciliation. A blocked host requires restart escalation even when publication
|
||||
Once publication may have started, cancellation alone cannot determine the outcome.
|
||||
A worker that returns a known success before escalation still reports success;
|
||||
an uncertain publication requires reconciliation. A blocked host requires restart escalation even when publication
|
||||
was provably prevented: storage and source ownership still have to remain valid.
|
||||
The runtime neither kills the thread nor starts a replacement worker over its
|
||||
resources. The watchdog runs outside that worker.
|
||||
|
||||
The watchdog cause is a third observation, independent of durable outcome and
|
||||
resource lifetime. Keeping it only in the terminal result loses information:
|
||||
a worker's generic cancellation error can overwrite the reason the cancellation
|
||||
was requested. Conversely, setting a terminal timeout before the worker returns
|
||||
can accidentally suppress its eventual success. The request now records the
|
||||
first watchdog cancellation separately, leaves the result Pending, and preserves
|
||||
the original worker error for native diagnostics. Completion before escalation
|
||||
combines the actual durable state with guest reason 15, including Success/15.
|
||||
Removal or termination cancellation that happened first retains its own reason.
|
||||
|
||||
This correction follows accepted decision DEC-0058. Deterministic regression
|
||||
fixtures hold queued work until after the simulated 500ms poll and deliver it
|
||||
at 501ms. Channel barriers hold IO before or inside publication, exercising
|
||||
Failure/15, Success/15, Indeterminate/15 and immutable restart results after 1s.
|
||||
The tests invoke the actual guest query handler and assert all three ABI slots;
|
||||
wall-clock receive limits only detect a hung fixture. They also verify deferred
|
||||
and composite bootstrap saves, dirty snapshots, durable bytes after reopening,
|
||||
result release and teardown isolation. These checks establish behavior under
|
||||
controlled scheduling, not real-device latency.
|
||||
|
||||
## A checksum does not make an old reference safe
|
||||
|
||||
Two valid map pages can reference different generations of payload and public
|
||||
|
||||
+122
@@ -0,0 +1,122 @@
|
||||
---
|
||||
id: LSN-0070
|
||||
ticket: runtime-branch-quality-review
|
||||
title: Runtime Quality Needs Independent Oracles and Owned Evidence
|
||||
created: 2026-10-14
|
||||
tags: [quality, runtime, ownership, validation]
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
A branch review found eight gaps spanning deadline reasons, coverage ownership,
|
||||
one-shot audio, recurring VM allocations, callback reclamation, CI enforcement,
|
||||
asset decoder parity and string provenance lookup. Accepted decisions DEC-0058
|
||||
through DEC-0065 were executed by PLN-0189 through PLN-0196. This lesson preserves
|
||||
the resulting reasoning after removal of those workflow artifacts; the linked
|
||||
published contracts remain normative.
|
||||
|
||||
The common failure mode was evidence that proved a nearby property while leaving
|
||||
the intended property unchecked. A callback can avoid locks while still freeing
|
||||
memory. Two decoders can agree on output while accepting the same malformed file.
|
||||
A coverage percentage can rise because ownership moved, without any new test.
|
||||
A fast lookup can hide expensive construction and uncharged metadata.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
### Observe the boundary where the guarantee becomes true
|
||||
|
||||
The [memcard contract](../../../docs/specs/runtime/08-save-memory-and-memcard.md)
|
||||
keeps the first deadline escalation reason stable while distinguishing durable
|
||||
completion from pending or failed work. Fake-clock boundary tests and explicit
|
||||
publication ordering make those distinctions observable without sleeping.
|
||||
|
||||
The [audio contract](../../../docs/specs/runtime/05-audio-peripheral.md) distinguishes
|
||||
one-shot completion from looping and logical inactivity from physical ownership.
|
||||
A finished voice may become silent before its sample can be reclaimed. Two bounded
|
||||
return packets move sample and vector owners to host service; pool exhaustion
|
||||
retains bounded ownership and defers delivery. Count both allocations and frees
|
||||
inside the callback, then independently prove eventual owner release outside it.
|
||||
|
||||
### Shared mechanics still need independent expected results
|
||||
|
||||
The [asset contract](../../../docs/specs/runtime/15-asset-management.md) uses common
|
||||
metadata validation and decoding mechanics for synchronous and incremental paths.
|
||||
Conformance checks include independently specified pixels, palette bytes and PCM,
|
||||
plus malformed metadata and physically truncated payloads. Agreement between two
|
||||
entry points is useful, but cannot by itself establish correctness.
|
||||
|
||||
The real production decoder/save ordering fixture pauses at a safe decode step,
|
||||
queues persistent work and checks writes and synchronization before decode resumes.
|
||||
That deterministic ordering oracle is stronger evidence of priority than a latency
|
||||
threshold on one host. Host latency remains a separate observation.
|
||||
|
||||
### Move cost deliberately and account for the destination
|
||||
|
||||
Recurring VM stack/return work reuses capacity after warmup; release allocator
|
||||
checks measure the execution region separately from setup. String provenance uses
|
||||
image and session indices under existing ownership quotas. Index metadata and
|
||||
construction cost are charged and measured, including collision and refusal paths.
|
||||
See the [Registry contract](../../../docs/specs/runtime/02c-program-registry-and-pbx-v1.md)
|
||||
and [provenance qualification](../../../docs/vm-arch/string-provenance-qualification.md).
|
||||
Lookup improvements do not establish faster game frames or cheaper construction.
|
||||
|
||||
### The repository owns acceptance and measurement identity
|
||||
|
||||
The [coverage policy](../../../docs/vm-arch/COVERAGE_DOMAINS.md) assigns each source
|
||||
to exactly one domain, includes module descendants, and keeps acceptance floors
|
||||
independent of observed percentages. Ownership repair and behavioral improvement
|
||||
must be evaluated separately against the same report before comparing fresh runs.
|
||||
|
||||
The shared strict gate records source/config identity, preserves command failures,
|
||||
removes stale reports and publishes available evidence even when a stage fails.
|
||||
Local execution and CI call the same target. A successful focused test does not
|
||||
replace the full gate; a successful local gate does not certify an unexecuted OS.
|
||||
|
||||
## Published Evidence and Its Limits
|
||||
|
||||
The final source commit is `3f9d2355462d3b5675f5edc6e488ab325eb024d8`, with Rust
|
||||
fingerprint `309a069f8b0560d1e9d1f9f4e011b709c89729383aeacd100c8c2ff5c3cff28d`.
|
||||
On macOS/aarch64 with rustc 1.98.1, all twelve `make ci-domains` stages passed:
|
||||
1,060 ordinary tests, five socket tests, three runner tests and all fifteen fixed
|
||||
coverage floors. Thirteen ordinary tests were ignored. The fresh coverage replay
|
||||
also passed 1,060 tests; those are repeated tests, not additional unique cases.
|
||||
[Current coverage evidence](../../../docs/vm-arch/COVERAGE_DOMAINS_CURRENT.json)
|
||||
retains the earlier ownership-repair comparison and identifies the final corpus.
|
||||
The ten focused coverage tests and two release/all-feature VM allocation tests
|
||||
passed again during closure without source changes.
|
||||
|
||||
[Audio qualification](../../../docs/vm-arch/audio-callback-qualification.md) retains
|
||||
the named physical-device smoke and the initial device failures. Complementary
|
||||
release measurement ran 1,000 sixteen-voice callbacks while a separate process
|
||||
repeated the real production decoder/FileMemcard fixture fourteen times. Callback
|
||||
p50/p95/p99/max were 14.542/97.667/399.041/1,070.708 microseconds against a nominal
|
||||
5,333.333-microsecond period. Callback allocation/free counts were both zero;
|
||||
sample owners went from sixteen to zero after teardown. The
|
||||
[measurement record](../../../docs/vm-arch/audio-production-pressure-qualification.json)
|
||||
records overlapping process intervals, environment and individual pressure rounds.
|
||||
|
||||
This is host-concurrent pressure evidence, not same-process console integration
|
||||
or a worst-case latency guarantee. Each pressure round alternates twenty production
|
||||
loads and durable 64-KiB commits; the separate deterministic fixture establishes
|
||||
save preemption during decoding. Physical audio, synthetic timing, allocator
|
||||
counts and save ordering answer different questions and retain separate scopes.
|
||||
No Linux/remote CI, human listening assessment, long-duration soak or gameplay
|
||||
speedup is claimed by this closure.
|
||||
|
||||
## Related Domain Lessons
|
||||
|
||||
- [Save publication and resource ownership](../DSC-0061-memcard-fixed-slots-v1/LSN-0067-save-publication-and-resource-ownership-are-separate.md).
|
||||
- [Repository-owned domain gates](../DSC-0002-runtime-edge-test-plan/LSN-0037-domain-gates-must-be-owned-by-the-repository.md).
|
||||
- [Audio delivery and lifecycle](../DSC-0055-guest-audio-voice-control/LSN-0069-audio-admission-needs-a-delivery-boundary-and-independent-lifecycle-control.md).
|
||||
- [Materialization and recurring VM cost](../DSC-0014-perf-vm-allocation-and-copy-pressure/LSN-0035-first-materialization-is-not-the-same-as-hot-path-copy-pressure.md).
|
||||
- [Serial async lanes](../DSC-0009-perf-async-background-work-lanes-for-assets-and-fs/LSN-0050-serial-async-lanes-bound-backlog-complexity.md).
|
||||
- [Registry boundaries and owned values](../DSC-0058-program-registry-type-and-function-metadata/LSN-0059-registry-boundaries-and-owned-values.md).
|
||||
|
||||
## Pitfalls and Takeaways
|
||||
|
||||
- Test the promised boundary with an independent oracle, including failure paths.
|
||||
- Separate admission, execution, publication and reclamation when reasoning about cost.
|
||||
- Preserve owner counts and quota accounting when replacing scans or moving work.
|
||||
- Bind acceptance evidence to source/config fingerprints, not merely a nearby commit.
|
||||
- Close deferred gates on the final source, while keeping environmental limits explicit.
|
||||
- Preserve reusable reasoning and durable measurements; discard completed workflow scaffolding.
|
||||
@@ -234,6 +234,23 @@ Image constants and session-created strings are separate accounting domains; cha
|
||||
|
||||
Each domain tracks UTF-8 bytes and allocation count using a bounded Weak<str> ledger without a strong accounting owner. Reads remain direct Arc accesses, not handle-table lookups. Reconcile dead entries at safepoints and before quota refusal, release Weak plus charge then; Weak may retain native allocation after last strong drop, so no immediate-release promise. Keep accounting domains alive while tracked allocations survive, including host/tool retention. Import untracked native strings as charged new allocations; explicit session transfer copies into the destination domain, while shared-image constants remain shared.
|
||||
|
||||
Provenance keys include both allocation data address and UTF-8 length. Image
|
||||
Weak records are sorted once during sealed loading by a bounded in-place heap
|
||||
sort, with record comparisons/reads/writes charged before work through LoadBudget;
|
||||
binary search serves native boundary identity queries. This does not reorder
|
||||
constants, exports or lease indices. Record capacities remain part of the
|
||||
existing image payload accounting.
|
||||
|
||||
Session provenance uses a precharged open-addressed BudgetVec table with at most
|
||||
50% live occupancy, geometric growth and explicit tombstones. Charge retained
|
||||
capacity through AuxiliaryBytes and old/new capacity coexistence through
|
||||
TemporaryBytes before rehashing. Reserve content and bucket growth before
|
||||
materialization; refusal must not publish content or an entry and must refund
|
||||
provisional reservations. Remove stale keys before releasing their Weak ownership;
|
||||
reuse tombstones and rebuild under the same capacity accounting. Queries allocate
|
||||
nothing, ordinary reads do not query the index, and resource ceilings and
|
||||
reconciliation safepoints are unchanged.
|
||||
|
||||
Check result sizes/overflow and reserve before materializing; roll back reservations and initialized state on failure. Charge transient buffer/result coexistence. No hidden unmetered concatenation/native result paths. Do not introduce reentrant GC or scheduling at arbitrary allocation points to satisfy quotas; dead guest objects stay charged until safe sweep. Layout headers, metadata, payload capacities, native ownership, auxiliary buffers and temporary storage must have explicit nonduplicated domains. Limits are not RSS or upfront reservations.
|
||||
|
||||
LoadLimits uses the same defaults on hosts; cartridges cannot raise them. Test configurations may lower limits. MiB=1,048,576 bytes.
|
||||
|
||||
@@ -84,12 +84,56 @@ Reset, termination, crash or cartridge replacement ends the audio session, stops
|
||||
|
||||
## 6 Samples, interpolation and ownership
|
||||
|
||||
Samples are immutable PCM16 mono, with a source sample rate and optional loop bounds. Linear interpolation uses fractional playback position. Playback rate combines source/output sample-rate ratio and pitch. Looping uses the sample's valid loop interval, or the full sample when no valid interval is present. Loop steps may cross multiple repetitions; finite positive pitch must not accidentally terminate a short loop or produce an infinite position.
|
||||
Samples are immutable PCM16 mono, with a source sample rate and optional loop bounds. With looping off, playback ignores all loop markers and ends at the full sample length. With looping on, both markers must be present and satisfy `start < end <= sample length`; otherwise the loop interval is the full sample `[0, length)`. Playback starts at position zero, preserving the intro before the selected loop start, and wraps at the exclusive loop end. Empty samples terminate without reading PCM data.
|
||||
|
||||
Unloading or replacing a bank does not interrupt a sample already retained by an admitted command or mixer voice. New plays resolve the current bank. Memory can remain retained until the command is cancelled or the playback ends, stops, is replaced, or its session ends. The driver must not retain a second sample reference indefinitely after the mixer observes natural completion.
|
||||
Linear interpolation uses fractional playback position. With looping off, the next value is the next actual sample frame, or zero beyond the full sample length. With looping on, interpolation across the selected loop end uses the loop-start frame. Playback rate combines source/output sample-rate ratio and pitch. Loop steps may cross multiple repetitions; finite positive pitch, including `f64::MAX`, must not accidentally terminate a short loop or produce an infinite position.
|
||||
|
||||
Unloading or replacing a bank does not interrupt a sample already retained by an admitted command or mixer voice. New plays resolve the current bank. Memory may remain retained after the command is cancelled or playback ends, stops, is replaced, or its session ends, until the host service reclaims the retired owner. Logical inactivity and physical reclamation are separate. The driver must not maintain a second playback mirror that cannot observe completion.
|
||||
|
||||
## 7 Host responsibilities and evidence
|
||||
|
||||
The desktop host consumes batches, mixes PCM and schedules the physical output. If no output device is available, it consumes committed batches silently. A failed or stalled device is diagnosed and detached so pending publication cannot block the guest indefinitely. The desktop implementation treats two seconds without callback progress as a stalled output.
|
||||
|
||||
Headless execution consumes committed batches silently and preserves incomplete frame/callback staging. It validates admission and runtime behavior, without proving audible output. Synthetic mixer tests establish signal behavior; a real device smoke establishes stream initialization and callback consumption, while listening remains a separate observation. Bit-identical PCM across physical devices is not required.
|
||||
|
||||
### Bounded desktop reclamation
|
||||
|
||||
The successful desktop data callback MUST perform no allocation or deallocation
|
||||
of runtime-owned memory, including command-vector storage and the last Sample/PCM
|
||||
owner. This excludes CPAL/device internals and asynchronous driver error reporting.
|
||||
It is not an absolute device-latency guarantee.
|
||||
|
||||
Initialization allocates exactly two retirement packets outside the callback.
|
||||
Each has room for two returned command-vector owners and 3,088 retired Sample
|
||||
references: one 1,024-command frame, one combined 2,048-command lifecycle delivery,
|
||||
and sixteen existing voices. Fixed-capacity SPSC free and return lanes each hold
|
||||
two packets and MUST NOT grow. A checked-out packet implies a free return slot;
|
||||
returning it never waits or frees overflow ownership.
|
||||
|
||||
The callback MUST obtain a packet before taking a publication. It moves commands
|
||||
out of vector slots, retaining the vector allocation in the packet, and moves
|
||||
replaced/stopped/reset voice owners into retirement storage. Stale deliveries are
|
||||
returned whole. Natural completion marks a voice inactive. Without an available
|
||||
packet, no publication is taken, accepted commands remain pending under existing
|
||||
backpressure, and at most sixteen inactive voice slots retain their Sample owners.
|
||||
Inactive slots produce silence and do not advance. They are existing mixer slots,
|
||||
not another playback mirror. Pending pause/session transitions apply when the
|
||||
mixer next obtains a packet and processes control state; already emitted device
|
||||
buffers cannot be recalled.
|
||||
|
||||
`HostAudio::service` MUST drain and recycle packets independently of guest progress,
|
||||
including paused/backpressured execution, before the next guest attempt. It frees
|
||||
retired references and clears returned commands outside the callback. Empty vector
|
||||
storage can be reused through a bounded four-buffer transport cache; excess empty
|
||||
storage is released on the service lane. These buffers retain no Sample owners.
|
||||
There is no change to guest quotas, FIFO/prefix order, session epochs, sixteen-voice
|
||||
capacity, or pause position.
|
||||
|
||||
Failed/stalled output is stopped and dropped before silent consumption takes over,
|
||||
so only one consumer drains the publication transport. Teardown releases callback,
|
||||
packet and pending-publication ownership outside the data callback. Headless/test
|
||||
hosts continue consuming silently through the same transport, without constructing
|
||||
an audio callback or requiring a device.
|
||||
|
||||
See [callback qualification](../../vm-arch/audio-callback-qualification.md) for
|
||||
executed allocation, signal, ownership and device observations.
|
||||
|
||||
@@ -236,6 +236,32 @@ not proof of rollback. If cessation is unconfirmed, isolate mount/resources,
|
||||
block new work and retain source/scratch. Late completion belongs to original
|
||||
reconciliation. Never kill threads unsafely, free buffers in use or create a
|
||||
replacement worker. Report restart requirement when host recovery is impossible.
|
||||
|
||||
The first watchdog cancellation is recorded independently of the request result,
|
||||
before signaling cancellation. Repeated polls preserve that cause. At 500ms the
|
||||
request remains Pending with no terminal reason until the worker returns or the
|
||||
1s escalation occurs. Removal or session termination that canceled first does
|
||||
not become a watchdog timeout merely because a later poll crosses 500ms.
|
||||
|
||||
On completion before escalation, the worker's durable outcome determines
|
||||
Success, Failure or Indeterminate. A recorded watchdog cancellation supplies
|
||||
Timeout (guest reason 15) independently of that state; native diagnostics retain
|
||||
the original worker storage error. For example:
|
||||
|
||||
- A save canceled while queued at 500ms, then completed at 501ms without entering
|
||||
publication, reports Failure/15. The durable slot is unchanged.
|
||||
- A save already inside publication at cancellation can finish successfully and
|
||||
report Success/15. Its saved snapshot becomes clean; cancellation is not rollback.
|
||||
- An uncertain publication reports Indeterminate/15 and requires reconciliation.
|
||||
- At 1s without confirmed cessation, RestartRequired reports Failure/15 before
|
||||
publication entry or Indeterminate/15 after entry. Late completion returns card
|
||||
ownership for reconciliation but cannot rewrite this terminal guest result or
|
||||
mark the snapshot clean. Releasing the result does not release in-use resources.
|
||||
|
||||
A deferred bootstrap save that has not reached the worker expires as Failure/15
|
||||
at 500ms. Memory-only commits remain MemoryOnly with reason 0 and perform no IO.
|
||||
The status/state/reason result slots and request identities do not change.
|
||||
|
||||
Fast-profile claims require measurements under asset load, including tail latency.
|
||||
|
||||
## 8. Player choice and lifecycle
|
||||
|
||||
@@ -242,6 +242,32 @@ If a scene glyph dependency cannot be resolved:
|
||||
- draw/composition MUST fail as a fatal machine error with clear logging;
|
||||
- runtime MUST NOT continue scene composition after the missing dependency is detected.
|
||||
|
||||
### 4.3 Decoding validation and physical payload completeness
|
||||
|
||||
Complete-buffer/reader decoding and resumable decoding MUST use the same driver
|
||||
format authority for glyph layout, sound metadata and PCM conversion, and scene
|
||||
headers, layers, tile ranges and decoded-size validation. Execution mode MUST NOT
|
||||
change acceptance or semantic bank output for the same declared asset slice.
|
||||
|
||||
Every byte of the declared slice MUST be physically available. A short file read
|
||||
MUST fail; it must not publish a partial sound or accept a physically complete
|
||||
scene whose byte length is shorter than the declared asset size. This applies
|
||||
even when the missing sound byte would have been an ignored odd PCM tail.
|
||||
A physically present final unpaired PCM byte remains accepted and ignored.
|
||||
|
||||
The shared worker retains resumable state and reads at most 4096 payload bytes
|
||||
per decode step. Scene header and layer steps read their fixed headers; tile
|
||||
steps use complete tile records. PCM steps end on sample boundaries except for
|
||||
the final odd tail. Save work takes priority at the next safe step. Cancellation
|
||||
and target generation are checked before continuing and before staging a
|
||||
complete bank; frame-boundary publication remains separate from decoding.
|
||||
|
||||
The conformance corpus checks explicit outputs and refusal outcomes across
|
||||
buffer, reader, legacy load and resumable entry points. It covers empty supported
|
||||
forms, chunk edges, palettes, truncated and trailing data, metadata, declared
|
||||
sizes, scene ranges and nonfinite parallax. Ignored reserved fields remain
|
||||
ignored; this validation sharing does not introduce new codecs or payload versions.
|
||||
|
||||
## 5 Banks and Slots
|
||||
|
||||
The current runtime exposes bank types:
|
||||
|
||||
@@ -28,10 +28,10 @@ LLVM lines still need an owner.
|
||||
| --- | --- |
|
||||
| system/os | OS facades, process and task services |
|
||||
| system/runtime | System-side VM lifecycle, scheduling and render integration |
|
||||
| system/fs | Runtime filesystem and memcard service |
|
||||
| system/fs | Runtime filesystem, memcard.rs and the complete memcard/ module tree |
|
||||
| system/programs | Resident system programs |
|
||||
| system/core | Remaining system sources, excluding the four specialized owners |
|
||||
| asset/bank | Existing asset, cartridge and asset-driver prefixes |
|
||||
| asset/bank | HAL asset/cartridge prefixes, driver asset.rs and the complete driver asset/ module tree |
|
||||
| firmware | Firmware state and transitions |
|
||||
| host-dependent | Desktop host |
|
||||
| host/headless | Headless host and automation implementation |
|
||||
@@ -43,6 +43,10 @@ LLVM lines still need an owner.
|
||||
| dev/test-support | Shared test-program construction and development infrastructure source stubs |
|
||||
|
||||
Existing specialized paths retain their original ownership.
|
||||
Both module entry files and sibling module directories need explicit prefixes.
|
||||
Memcard and asset-driver descendants, including future nested modules, retain
|
||||
their specialized owner even when absent from the coverage report or reporting
|
||||
zero executable lines. The corresponding residual exclusions cover both forms.
|
||||
Residual domains use broad crate source prefixes with explicit exclusions,
|
||||
so a new module inherits the appropriate owner. A new crate or conflicting
|
||||
prefix must be assigned before the ownership audit can pass.
|
||||
@@ -60,6 +64,10 @@ Quality tests remain in prometeu-quality-checks/tests. They enforce the map
|
||||
and drive the auditor/helper with independent source/report fixtures,
|
||||
including omitted/overlapping owners, exclusions, absolute/relative filenames,
|
||||
absent/empty evidence, invalid statistics and percentage regression.
|
||||
Expected-owner tests use the auditor's ownership function for entry files,
|
||||
every existing descendant and future nested paths. Synthetic reports also
|
||||
check absent and zero-line descendants. Exclusive residual fallback is not
|
||||
sufficient: it can silently transfer specialized code into the wrong gate.
|
||||
The empty quality/layer library stubs inherit infrastructure ownership;
|
||||
they do not create separate gates with zero executable lines.
|
||||
|
||||
@@ -93,13 +101,16 @@ this percentage is at least baseline_percent. Counts for functions and
|
||||
regions remain supporting evidence. Scenario/contract assertions continue
|
||||
to govern behavior; a percentage does not replace them.
|
||||
|
||||
Measured coverage and the acceptance floor are separate values. This explicitly
|
||||
reviewed revision gives every domain at least 5.00 percentage points of headroom
|
||||
from the recorded local measurement. Its rule is
|
||||
Measured coverage and the acceptance floor are separate values. The historical
|
||||
margin revision gave every domain at least 5.00 percentage points of headroom
|
||||
from that recorded local measurement. Its rule was
|
||||
`new_floor = min(original_floor, measured_percent - 5.00)`, using the same
|
||||
downward truncation. Floors already providing that margin remain unchanged.
|
||||
Five percentage points means an 88.56% measurement permits an 83.56% floor;
|
||||
it is not a relative five-percent reduction.
|
||||
That historical margin is not a guarantee about later measurements or corrected
|
||||
ownership. The current map restores the reviewed system/core floor of 87.71%
|
||||
and preserves every other floor.
|
||||
|
||||
Floors remain fixed during CI. A failing run does not automatically lower them.
|
||||
An intentional scope or floor revision requires explicit review with fresh
|
||||
@@ -108,30 +119,107 @@ fresh measurement.
|
||||
|
||||
## Commands and CI
|
||||
|
||||
List domains, inspect source ownership and validate coverage evidence:
|
||||
`make ci-domains` is the authoritative repository quality gate. Both
|
||||
[GitHub CI](../../.github/workflows/ci.yml) and
|
||||
[Jenkins](../../files/config/Jenkinsfile) provision dependencies and invoke that
|
||||
same target. `make ci` is an alias. Invoke one gate per checkout at a time.
|
||||
|
||||
make coverage-domain-list
|
||||
python3 scripts/audit-coverage-domains.py
|
||||
make coverage-domain-audit
|
||||
python3 scripts/audit-coverage-domains.py --report target/llvm-cov/report.json --json
|
||||
The Makefile supplies this ordered sequence to the serial evidence runner:
|
||||
|
||||
Produce a fresh report and run all gates:
|
||||
1. Clean LLVM coverage instrumentation from the previous run.
|
||||
2. Check formatting with `cargo fmt --all -- --check`.
|
||||
3. Run `cargo clippy --workspace --all-targets --all-features -- -D warnings`.
|
||||
4. Test gate failure/report handling with the Python regression suite.
|
||||
5. Run `cargo test --workspace --all-targets --all-features --no-fail-fast`.
|
||||
6. Run the five ignored debugger socket tests through `make test-debugger-socket`.
|
||||
7. Generate fresh workspace/all-feature LLVM coverage, HTML, full JSON,
|
||||
Cobertura XML and summary JSON.
|
||||
8. Audit source ownership/evidence and check every fixed coverage domain floor.
|
||||
|
||||
make ci-domains
|
||||
The socket command is deliberately restricted:
|
||||
|
||||
The existing Jenkins configuration invokes this repository target. Its
|
||||
coverage-domain-gate step also performs the ownership/evidence audit.
|
||||
GitHub's workspace-test step discovers the quality regression tests; it
|
||||
does not separately produce a domain coverage report.
|
||||
```sh
|
||||
cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored
|
||||
```
|
||||
|
||||
Inspect a domain:
|
||||
It selects `test_debug_port_opens`, `test_debug_reconnection`,
|
||||
`test_debug_refuse_second_connection`, `test_get_state_returns_response` and
|
||||
`test_debug_resume_on_disconnect`. It does not select the audio-device test.
|
||||
Local sandbox TCP permissions must permit bind/connect; a permission denial is
|
||||
not evidence of a debugger behavior regression.
|
||||
|
||||
make coverage-domain-evidence DOMAIN=vm/core
|
||||
scripts/coverage-domain-evidence.sh --check --verbose hal/core
|
||||
scripts/coverage-domain-evidence.sh --check-all
|
||||
The runner executes child Make targets with `-j1` even if the outer invocation
|
||||
uses `-j`. CI invokes `make ci-domains` serially. Do not run a separate Cargo,
|
||||
coverage-clean or report process concurrently in the same checkout: serialized
|
||||
steps inside one gate cannot protect against an independent process.
|
||||
|
||||
Dependencies are the existing Rust/LLVM coverage tooling, Bash, Git, jq and rg,
|
||||
plus Python 3 for the standard-library ownership auditor.
|
||||
### Prerequisites and reports
|
||||
|
||||
Both pipelines use `bash scripts/provision-quality-gate.sh`. A Debian/Ubuntu
|
||||
agent needs root or sudo for apt provisioning, and an existing Rust/rustup
|
||||
installation. GitHub installs stable Rust first; Jenkins must supply a supported
|
||||
Rust toolchain and rustup in CARGO_HOME/bin. The script installs rustfmt, Clippy,
|
||||
llvm-tools-preview and pinned cargo-llvm-cov 0.8.5 (`--locked`).
|
||||
|
||||
Linux packages include build-essential, pkg-config, libasound2-dev, libudev-dev,
|
||||
libxkbcommon-dev, libwayland-dev, libx11-dev, libxrandr-dev, libxi-dev, Python 3,
|
||||
jq, ripgrep, Git, curl and CA certificates. ALSA development files satisfy CPAL's
|
||||
Linux build dependency; a running audio device is not needed. Non-Linux local
|
||||
hosts must already provide native compiler/SDK tools, Bash, Make, pkg-config,
|
||||
Python 3, jq, rg and Git. The provisioning script verifies these commands.
|
||||
|
||||
`target/quality-gate/metadata.json` records revision, dirty working tree, host,
|
||||
Rust/coverage-tool versions, configuration/source fingerprints, ordered step
|
||||
exit codes and report hashes. Each step has a separate combined-output log.
|
||||
The runner stops on the first failed Make target and returns its nonzero exit
|
||||
status; the log preserves underlying Cargo diagnostics. Three regression cases
|
||||
check failure logs/identity, fresh success evidence and partial-report retention
|
||||
when a later gate fails.
|
||||
|
||||
Both pipelines discard old generated report directories before provisioning and
|
||||
publish current artifacts even after gate failure. An early failed gate has
|
||||
metadata/logs but no old coverage report. Failure after generation preserves the
|
||||
partial current reports. Provisioning failure can leave no report at all and
|
||||
still fails CI. Reports are archived from `target/quality-gate/**` and
|
||||
`target/llvm-cov/**`; Jenkins also publishes current HTML when present. Artifact
|
||||
publication does not turn a failed gate into success.
|
||||
|
||||
List domains or inspect ownership and existing evidence:
|
||||
|
||||
```sh
|
||||
make coverage-domain-list
|
||||
python3 scripts/audit-coverage-domains.py
|
||||
make coverage-domain-audit
|
||||
make coverage-domain-evidence DOMAIN=vm/core
|
||||
scripts/coverage-domain-evidence.sh --check --verbose hal/core
|
||||
```
|
||||
|
||||
### Explicit manual qualification
|
||||
|
||||
Ordinary all-target tests and coverage leave ignored device/measurement tests
|
||||
ignored. The shared target only opts in to the five socket tests above. It does
|
||||
not certify the following manual workloads:
|
||||
|
||||
```sh
|
||||
# Physical desktop audio, exact named test.
|
||||
cargo test -p prometeu-host-desktop-winit --lib audio::control_tests::desktop_audio_device_smoke -- --ignored --exact --nocapture
|
||||
# Physical callback duration under host I/O pressure.
|
||||
cargo test --release -p prometeu-host-desktop-winit --test audio_callback_allocations physical_callback_duration_16_voices_with_io_pressure -- --ignored --exact --nocapture
|
||||
# Eight-hour active release soak.
|
||||
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --seconds 28800
|
||||
# Packaged installation; set PROMETEU_PROTOCOL_TEST_INSTALL to an extracted distribution first.
|
||||
cargo test -p prometeu-cli --test automation_process packaged_installation_smoke -- --ignored --exact --nocapture
|
||||
# File-host commit latency with the production asset decoder.
|
||||
cargo test --release -p prometeu-system --all-features qualify_file_commits_with_production_asset_decoder -- --ignored --nocapture
|
||||
# Explicit layout observation.
|
||||
cargo test --release -p prometeu-bytecode --test registry_v1 reference_map_probe -- --ignored --exact --nocapture
|
||||
```
|
||||
|
||||
Other ignored timing probes remain opt-in by their individual test names.
|
||||
All-target execution can print ordinary benchmark observations; this does not
|
||||
establish completion of ignored manual qualification. For scope and prior measured
|
||||
results see [resource stability](resource-stability-qualification.md) and
|
||||
[audio callback qualification](audio-callback-qualification.md).
|
||||
|
||||
## Historical local baseline
|
||||
|
||||
@@ -153,14 +241,14 @@ on macOS/aarch64:
|
||||
|
||||
These are historical starting floors; current acceptance floors appear below.
|
||||
|
||||
## Current local margins
|
||||
## Historical margin revision
|
||||
|
||||
The [margin evidence](COVERAGE_DOMAINS_MARGIN.json) records fresh measurements,
|
||||
original and revised floors, toolchain, fingerprints and executed validation.
|
||||
Only domains with less than 5.00 percentage points of headroom were adjusted.
|
||||
The historical evidence above remains unchanged.
|
||||
|
||||
| Domain | Measured lines | Original floor | Current floor | Margin (points) |
|
||||
| Domain | Measured lines | Original floor | Revised floor | Margin (points) |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| system/os | 82.24% | 77.24% | 77.24% | 5.00 |
|
||||
| system/runtime | 81.19% | 76.50% | 76.19% | 5.00 |
|
||||
@@ -183,6 +271,70 @@ The full local `make ci-domains` validation passed with 939 tests passed and
|
||||
in verification; its final floor uses the lower measurement. All gates were
|
||||
rechecked against the verification report after that final adjustment.
|
||||
|
||||
## Current coverage measurement
|
||||
|
||||
The [current evidence](COVERAGE_DOMAINS_CURRENT.json) identifies the committed
|
||||
source by HEAD and content fingerprints, records the strict shared gate and
|
||||
retains the earlier ownership-repair measurement as historical evidence.
|
||||
Both original historical JSON records above remain unchanged. The latest
|
||||
measurement uses the repaired module-tree map with all 15 fixed floors.
|
||||
|
||||
| Domain | Covered / total lines | Measured lines | Fixed floor | Margin (points) |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| system/os | 1410 / 1711 | 82.40% | 77.24% | 5.16 |
|
||||
| system/runtime | 2757 / 3309 | 83.31% | 76.19% | 7.12 |
|
||||
| system/fs | 1946 / 2469 | 78.81% | 76.08% | 2.73 |
|
||||
| system/programs | 556 / 567 | 98.05% | 93.05% | 5.00 |
|
||||
| asset/bank | 3176 / 3434 | 92.48% | 86.76% | 5.72 |
|
||||
| firmware | 1990 / 2314 | 85.99% | 83.20% | 2.79 |
|
||||
| host-dependent | 1531 / 2393 | 63.97% | 40.05% | 23.92 |
|
||||
| host/headless | 739 / 855 | 86.43% | 83.39% | 3.04 |
|
||||
| vm/core | 8522 / 9526 | 89.46% | 84.29% | 5.17 |
|
||||
| bytecode | 3085 / 3303 | 93.39% | 88.26% | 5.13 |
|
||||
| hal/core | 2093 / 2369 | 88.34% | 83.56% | 4.78 |
|
||||
| drivers/core | 2775 / 3107 | 89.31% | 79.51% | 9.80 |
|
||||
| system/core | 1353 / 1444 | 93.69% | 87.71% | 5.98 |
|
||||
| tooling | 396 / 612 | 64.70% | 59.75% | 4.95 |
|
||||
| dev/test-support | 99 / 115 | 86.08% | 80.96% | 5.12 |
|
||||
|
||||
The fresh macOS/aarch64 `make ci-domains` run passed all 15 domain gates,
|
||||
1,060 ordinary tests (13 ignored), five explicit socket tests and three
|
||||
runner regression tests. The fresh coverage replay also passed 1,060 tests;
|
||||
it is not an additional set of unique behavioral tests. All ten focused
|
||||
ownership/evidence tests passed again on the committed source. No floor
|
||||
was lowered. The retained ownership-repair measurement used the earlier
|
||||
non-strict gate; the current measurement passes strict workspace Clippy.
|
||||
|
||||
Platform-specific compilation and host scenarios can change report totals.
|
||||
Local evidence does not certify an unexecuted Linux or Intel Mac run; evaluate
|
||||
those runs against the configured floors and investigate differences.
|
||||
|
||||
## Strict shared-gate qualification
|
||||
|
||||
On 2026-10-14, macOS/aarch64 with rustc 1.98.1, the shared target passed strict
|
||||
workspace/all-target/all-feature Clippy, all 1,060 ordinary tests (13 ignored),
|
||||
the five explicitly selected socket tests, three gate-runner regression tests,
|
||||
and all 15 fixed coverage domain gates. The separate fresh coverage execution
|
||||
also passed 1,060 tests with 13 ignored. Those repeated runs are not distinct
|
||||
behavioral tests and must not be added into a unique-test count.
|
||||
|
||||
The baseline strict invocation failed on the concrete VM lint sources above;
|
||||
the repaired sources pass without new lint allowances. Socket tests initially
|
||||
failed because the sandbox denied localhost TCP operations; the authorized
|
||||
outside-sandbox run passed all five. Gate failure fixtures preserve nonzero
|
||||
Make status, logs and commit/config identity, discard stale coverage, and retain
|
||||
fresh partial reports when a later gate fails. Interrupted runners record failure
|
||||
rather than success.
|
||||
|
||||
GitHub's YAML was parsed locally, and both pipeline definitions were inspected
|
||||
for shared provisioning, `make ci-domains` and unconditional failure-artifact
|
||||
publication. No remote GitHub or Jenkins build was launched. No Linux gate ran:
|
||||
the local Docker API was denied by the sandbox and the unrestricted daemon query
|
||||
did not return a usable result before interruption. Local macOS coverage does not
|
||||
certify Linux compilation or domain percentages. Such failures must be investigated
|
||||
on that platform without lowering the existing floors.
|
||||
|
||||
No physical audio, eight-hour soak, packaged-installation or ignored manual
|
||||
latency/layout qualification was selected by this gate. Prior manual observations
|
||||
remain separate evidence. Current executable metadata and reports are emitted
|
||||
under `target/quality-gate/` and `target/llvm-cov/` by every invocation.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,121 @@
|
||||
# Desktop audio callback qualification
|
||||
|
||||
The desktop callback moves discarded Sample references and command-vector owners
|
||||
into two preallocated return packets. Host service reclaims them outside the data
|
||||
callback. Logical playback completion does not imply immediate physical release.
|
||||
The [audio peripheral contract](../specs/runtime/05-audio-peripheral.md) is normative.
|
||||
|
||||
## Capacity and ownership proof
|
||||
|
||||
One publication can contain 1,024 frame commands and a combined 2,048-command
|
||||
lifecycle batch. Each command introduces at most one Sample reference, and the
|
||||
mixer starts with at most sixteen. Therefore at most 3,088 Sample owners can retire
|
||||
in one callback, even if every command replaces a voice and every resulting voice
|
||||
finishes immediately. Each packet reserves that reference capacity plus two vector
|
||||
slots before starting the stream. Stale command references remain in the returned
|
||||
vectors instead of moving into the retired-reference vector.
|
||||
|
||||
Exactly two packets circulate through capacity-two SPSC lanes. While the callback
|
||||
owns a packet, no more than one other packet can occupy the return lane. Enqueue
|
||||
therefore needs no waiting, resizing or overflow disposal. Symmetrically, service
|
||||
can always return a reclaimed packet to the free lane. If both packets await service,
|
||||
the callback leaves publications untouched. Up to sixteen naturally completed
|
||||
voices retain owners in their existing slots, becoming silent until reclamation
|
||||
capacity returns. PCM content can be large; this is a bound on references and fixed
|
||||
metadata, not an additional PCM byte quota.
|
||||
|
||||
Service clears retired references, clears commands and caches at most four empty
|
||||
command vectors for producer reuse. It runs before guest attempts and during pause
|
||||
or backpressure. Stream stop/drop precedes silent-consumer takeover. No additional
|
||||
reclamation worker or playback mirror is introduced.
|
||||
|
||||
## Allocation and semantic evidence
|
||||
|
||||
`tests/audio_callback_allocations.rs` installs a thread-local allocator counter.
|
||||
Only runtime callback execution is counted; fixture construction, I/O, reporting,
|
||||
service reclamation and teardown run outside that region. It counts alloc,
|
||||
alloc_zeroed, realloc and dealloc (realloc contributes to both counters).
|
||||
|
||||
The original last-owner natural completion made zero allocations and two
|
||||
deallocations; applying a control batch freed one vector. After the change,
|
||||
debug and release counters report zero allocations/deallocations for natural
|
||||
completion, all controls, repeated play/stop/replacement, pause, epoch reset,
|
||||
empty samples, bank unload, 16-voice completion without a packet, ten starved
|
||||
callbacks and recovery. The saturation case combines 3,072 Play commands with
|
||||
sixteen existing voices and completion, exercising the full 3,088-owner bound.
|
||||
Weak-reference assertions prove retention until service and final release.
|
||||
|
||||
Unit tests additionally check stale-vector identity and deferred release, 1,000
|
||||
pool recycling cycles, vector-allocation reuse, bounded empty-buffer caching,
|
||||
failed/stalled stream takeover and teardown. Existing exact PCM tests retain
|
||||
one-shot tails, loop seams, fractional interpolation, FIFO/prefix order and pause
|
||||
position. Runtime audio controls and headless tests retain their logical admission
|
||||
behavior; the Rust silent-consumer interface remains unchanged.
|
||||
|
||||
Reproduce allocation qualification with:
|
||||
|
||||
```sh
|
||||
cargo test -p prometeu-host-desktop-winit --test audio_callback_allocations -- --nocapture
|
||||
cargo test --release -p prometeu-host-desktop-winit --test audio_callback_allocations -- --nocapture
|
||||
```
|
||||
|
||||
## Timing observations
|
||||
|
||||
Executed on 2026-10-14, macOS/aarch64, rustc 1.98.1, release profile. Both workloads
|
||||
used sixteen voices and 256 stereo frames at 48 kHz (nominal period 5.333 ms).
|
||||
The explicitly named pressure workload reads the checked-in stress-console asset
|
||||
pack and writes a temporary 1 MiB file through HostDirBackend. It does not execute
|
||||
guest asset decoding or the memcard save scheduler; it is not a substitute for a
|
||||
separate full integration workload.
|
||||
|
||||
| Workload | Callbacks | I/O cycles | p50 | p95 | p99 | Maximum |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| Synthetic callback invocation, interleaved I/O | 1,000 | 1,000 | 13.791 µs | 30.458 µs | 54.292 µs | 72.166 µs |
|
||||
| MacBook Pro Speakers, concurrent main-thread I/O | 375 | 273 | 11.458 µs | 12.875 µs | 15.916 µs | 30.458 µs |
|
||||
|
||||
Both observed zero runtime allocator calls and sixteen live Sample owners before
|
||||
teardown, zero afterward. Physical buffers were observed at exactly 256 frames.
|
||||
Timing includes the runtime render call and counter overhead, excludes I/O and
|
||||
CPAL/device internals, and does not measure device presentation latency. No timing
|
||||
threshold is asserted. These are local observations, not cross-device certification.
|
||||
|
||||
Explicit commands (both timing tests are ignored by ordinary CI):
|
||||
|
||||
```sh
|
||||
cargo test --release -p prometeu-host-desktop-winit --test audio_callback_allocations callback_duration_16_voices_with_io_pressure -- --ignored --exact --nocapture
|
||||
cargo test --release -p prometeu-host-desktop-winit --test audio_callback_allocations physical_callback_duration_16_voices_with_io_pressure -- --ignored --exact --nocapture
|
||||
cargo test -p prometeu-host-desktop-winit --lib audio::control_tests::desktop_audio_device_smoke -- --ignored --exact --nocapture
|
||||
```
|
||||
|
||||
The ordinary physical smoke passed on retry. The first sandboxed attempt could
|
||||
not open the device; the first unrestricted attempt received no callbacks and
|
||||
triggered the existing two-second watchdog. Those failures remain part of the
|
||||
qualification evidence and demonstrate why successful local timing is not an
|
||||
availability guarantee. No human listening assessment is claimed.
|
||||
|
||||
Strict host Clippy passes. Repository-wide strict-gate status remains separate
|
||||
from these behavioral and device observations.
|
||||
|
||||
## Production decode and save pressure
|
||||
|
||||
A complementary release observation on 2026-10-14 used the same 1,000-callback,
|
||||
16-voice synthetic harness while a separate process repeatedly ran
|
||||
`qualify_file_commits_with_production_asset_decoder` from `prometeu-system`.
|
||||
Each pressure round performs twenty production glyph loads and durable 64-KiB
|
||||
FileMemcard commits. Fourteen rounds completed successfully during the audio
|
||||
process interval. The two test binaries were built before measurement; a worker
|
||||
started the first pressure process before audio and repeated it until audio
|
||||
exited, then joined the final invocation. Monotonic process intervals confirm
|
||||
concurrency in the [measurement record](audio-production-pressure-qualification.json).
|
||||
|
||||
Callback p50/p95/p99/max were 14.542/97.667/399.041/1,070.708 microseconds for the
|
||||
nominal 5,333.333-microsecond period. Allocation and deallocation counts remained
|
||||
zero; sixteen live sample owners became zero after teardown. All test processes
|
||||
exited successfully. The record identifies the committed source, toolchain,
|
||||
per-round save observations and the additional policy tests running on the host.
|
||||
|
||||
This measures host contention between processes, not an integrated console
|
||||
session or a physical callback under this exact workload. The previously named
|
||||
physical-device observation remains separate. The pressure fixture alternates
|
||||
loads and saves; deterministic production-decoder tests establish save preemption
|
||||
during decoding. None of these host-specific maxima is a real-time guarantee.
|
||||
@@ -0,0 +1,255 @@
|
||||
{
|
||||
"scope": "Concurrent host processes: real production decode/FileMemcard fixture and 16-voice callback; not a same-process console integration",
|
||||
"audio": {
|
||||
"start_ns": 1213231353849333,
|
||||
"end_ns": 1213235637563291,
|
||||
"exit_code": 0,
|
||||
"callbacks": 1000,
|
||||
"voices": 16,
|
||||
"frames": 256,
|
||||
"nominal_period_ns": 5333333,
|
||||
"p50_ns": 14542,
|
||||
"p95_ns": 97667,
|
||||
"p99_ns": 399041,
|
||||
"max_ns": 1070708,
|
||||
"allocations": 0,
|
||||
"deallocations": 0,
|
||||
"owners_before": 16,
|
||||
"owners_after": 0
|
||||
},
|
||||
"pressure": [
|
||||
{
|
||||
"start_ns": 1213231351766125,
|
||||
"end_ns": 1213232142082875,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 8,
|
||||
"p95": 43,
|
||||
"max": 220
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 14187,
|
||||
"p95": 29711,
|
||||
"max": 30902
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213232142089250,
|
||||
"end_ns": 1213232647182500,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 6,
|
||||
"p95": 15,
|
||||
"max": 29
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 16130,
|
||||
"p95": 19710,
|
||||
"max": 23276
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213232647184250,
|
||||
"end_ns": 1213232911448208,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 8,
|
||||
"p95": 38,
|
||||
"max": 59
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 8550,
|
||||
"p95": 10179,
|
||||
"max": 10965
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213232911451958,
|
||||
"end_ns": 1213233156142708,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 9,
|
||||
"p95": 15,
|
||||
"max": 26
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 8981,
|
||||
"p95": 10341,
|
||||
"max": 11438
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213233156146791,
|
||||
"end_ns": 1213233422857166,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 8,
|
||||
"p95": 18,
|
||||
"max": 19
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 10074,
|
||||
"p95": 12108,
|
||||
"max": 12709
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213233422860875,
|
||||
"end_ns": 1213233691177833,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 7,
|
||||
"p95": 23,
|
||||
"max": 89
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 10453,
|
||||
"p95": 12621,
|
||||
"max": 12941
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213233691180500,
|
||||
"end_ns": 1213233907892666,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 6,
|
||||
"p95": 17,
|
||||
"max": 48
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 7962,
|
||||
"p95": 10914,
|
||||
"max": 11195
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213233907895666,
|
||||
"end_ns": 1213234115777250,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 15,
|
||||
"p95": 111,
|
||||
"max": 245
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 8040,
|
||||
"p95": 12131,
|
||||
"max": 14538
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213234115783333,
|
||||
"end_ns": 1213234378236041,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 18,
|
||||
"p95": 77,
|
||||
"max": 143
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 9393,
|
||||
"p95": 11115,
|
||||
"max": 11919
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213234378243041,
|
||||
"end_ns": 1213234604680916,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 10,
|
||||
"p95": 94,
|
||||
"max": 1120
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 8773,
|
||||
"p95": 10268,
|
||||
"max": 11906
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213234604683875,
|
||||
"end_ns": 1213234822563625,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 9,
|
||||
"p95": 104,
|
||||
"max": 188
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 8485,
|
||||
"p95": 10250,
|
||||
"max": 11311
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213234822567208,
|
||||
"end_ns": 1213235110141666,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 17,
|
||||
"p95": 55,
|
||||
"max": 122
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 8975,
|
||||
"p95": 16481,
|
||||
"max": 16947
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213235110144333,
|
||||
"end_ns": 1213235379186208,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 13,
|
||||
"p95": 26,
|
||||
"max": 49
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 9699,
|
||||
"p95": 13929,
|
||||
"max": 15228
|
||||
},
|
||||
"samples": 20
|
||||
},
|
||||
{
|
||||
"start_ns": 1213235379189083,
|
||||
"end_ns": 1213235676903791,
|
||||
"exit_code": 0,
|
||||
"first_io_us": {
|
||||
"p50": 23,
|
||||
"p95": 1085,
|
||||
"max": 1339
|
||||
},
|
||||
"commit_us": {
|
||||
"p50": 9024,
|
||||
"p95": 20041,
|
||||
"max": 21674
|
||||
},
|
||||
"samples": 20
|
||||
}
|
||||
],
|
||||
"date": "2026-10-14",
|
||||
"git_head": "3f9d2355462d3b5675f5edc6e488ab325eb024d8",
|
||||
"source_sha256": "309a069f8b0560d1e9d1f9f4e011b709c89729383aeacd100c8c2ff5c3cff28d",
|
||||
"host": "macOS-27.0.1-arm64-arm-64bit-Mach-O",
|
||||
"rustc": "1.98.1",
|
||||
"profile": "release",
|
||||
"pressure_test": "prometeu-system: qualify_file_commits_with_production_asset_decoder",
|
||||
"audio_test": "prometeu-host-desktop-winit: callback_duration_16_voices_with_io_pressure",
|
||||
"orchestration": "Build both release test binaries first. Start a worker that repeatedly invokes the production pressure test with --ignored --nocapture. After the first pressure process starts, run the exact audio test with --ignored --exact --nocapture. Stop repetition when audio exits and join the final pressure invocation. Record monotonic start/end times and require all exits zero. Focused coverage-policy tests also ran during part of this host observation."
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
# VM execution allocation qualification
|
||||
|
||||
Successful stack reads construct diagnostics only on failure. Return completion
|
||||
validates the exact frame height and ordered signature groups before rotating the
|
||||
current frame suffix right by its result width and truncating parameters and
|
||||
locals. Void returns truncate directly. This preserves result order and owners
|
||||
without a temporary result vector or Value clones.
|
||||
|
||||
Only the returning frame is removed. Caller locals and continuation remain intact.
|
||||
Direct RET and branches to function end share completion while retaining their
|
||||
existing instruction charges. The last-frame safepoint still runs after results
|
||||
are rooted in the operand stack. If GC scratch admission fails, those roots remain
|
||||
published and no sweep occurs.
|
||||
|
||||
## Measurement
|
||||
|
||||
On 2026-10-14, aarch64 macOS with rustc 1.98.1, the dedicated
|
||||
`crates/console/prometeu-vm/tests/execution_allocations.rs` integration binary
|
||||
measured the following native allocation/deallocation calls. Both debug and
|
||||
release produced the same counts:
|
||||
|
||||
| Instruction loop | Iterations | Before alloc/free | After alloc/free |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| CALL / PUSH_I64 / RET / POP / JMP | 1,000 | 3,000 / 3,000 | 0 / 0 |
|
||||
| CALL / RET / JMP (void) | 1,000 | 0 / 0 | 0 / 0 |
|
||||
| CALL / two PUSH_I64 / RET / two POP / JMP | 1,000 | 5,000 / 5,000 | 0 / 0 |
|
||||
| PUSH_I64 / DUP / ADD / POP / JMP | 1,000 | 6,000 / 6,000 | 0 / 0 |
|
||||
|
||||
Run `cargo test -p prometeu-vm --test execution_allocations -- --nocapture`
|
||||
and the same command with `--release` before `-p`.
|
||||
|
||||
The allocator wrapper enables counting only on the executing test thread after
|
||||
ten warm-up iterations. It includes alloc, alloc_zeroed, dealloc and realloc
|
||||
(realloc contributes to both counters). Fixture construction, image validation,
|
||||
reporting and destruction occur outside the measured region. The fixtures do not
|
||||
create GC pressure. Error diagnostics, deliberate guest allocations, collection
|
||||
and dead-string reclamation are outside this zero-allocation claim. This is an
|
||||
instruction-region qualification, not a gameplay throughput measurement.
|
||||
|
||||
## Semantic and resource evidence
|
||||
|
||||
VM regressions exercise direct, closure and trait calls, taken and untaken
|
||||
function-end branches, captured strings, ordered mixed result groups, Arc pointer
|
||||
identity and heap receiver identity. A 1,024-slot mixed return checks the published
|
||||
result-width ceiling and caller-local preservation. Internal tests check height
|
||||
and type refusal before mutation, last-root closure survival, GC scratch refusal
|
||||
without partial sweep, and successful collection after scratch admission returns.
|
||||
Lifecycle tests continue to check saved continuation and completion boundaries.
|
||||
|
||||
Actual storage remains quota-accounted. Removing the disposable return vector
|
||||
removes its TemporaryBytes reservation; a valid scalar return can therefore
|
||||
succeed with zero temporary capacity when no collection is needed. This does not
|
||||
raise a ceiling, introduce an unmetered cache, or bypass the temporary memory
|
||||
required by a triggered collector. Operand and call stack capacity accounting,
|
||||
GC thresholds, verifier contracts and syscall ABI are unchanged.
|
||||
|
||||
The focused VM and transitive-root layer tests qualify these behaviors. Workspace
|
||||
quality-gate status is tracked separately; these measurements do not establish
|
||||
that all repository diagnostics or platform checks pass.
|
||||
|
||||
See the [memory and allocation contract](../specs/runtime/03-memory-stack-heap-and-allocation.md)
|
||||
and the [materialization and copy-pressure lesson](../../discussion/lessons/DSC-0014-perf-vm-allocation-and-copy-pressure/LSN-0035-first-materialization-is-not-the-same-as-hot-path-copy-pressure.md).
|
||||
@@ -29,11 +29,17 @@ cargo test --workspace --features prometeu-host-headless/resource-diagnostics
|
||||
cargo test --release --workspace --features prometeu-host-headless/resource-diagnostics
|
||||
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --frames 216000
|
||||
cargo run --release -p prometeu-vm --features resource-diagnostics --example resource_qualification -- --seconds 28800
|
||||
cargo test --release -p prometeu-vm --features resource-diagnostics --lib -- --ignored --nocapture
|
||||
cargo test --release -p prometeu-vm --features resource-diagnostics --lib sealed_image_load_probe -- --ignored --nocapture
|
||||
cargo test --release -p prometeu-vm --features resource-diagnostics --lib storage_and_gc_probe -- --ignored --nocapture
|
||||
cargo test --release -p prometeu-bytecode reference_map_probe -- --ignored --nocapture
|
||||
cargo test -p prometeu-host-desktop-winit --lib -- --ignored
|
||||
cargo test -p prometeu-host-desktop-winit --lib runner::tests -- --ignored
|
||||
cargo test -p prometeu-host-desktop-winit --lib audio::control_tests::desktop_audio_device_smoke -- --ignored --exact --nocapture
|
||||
```
|
||||
|
||||
The reproduction commands name manual probes individually. The original recorded
|
||||
desktop qualification grouped ignored tests; current automation selects only the
|
||||
five debugger socket cases and leaves device audio explicit.
|
||||
|
||||
The `resource-diagnostics` feature adds engineering observations through Rust
|
||||
only. It adds no guest syscall. The eight-hour example is a manual command;
|
||||
the repository CI, release workflow and Jenkins pipeline do not invoke it. Normal builds omit timing histograms, additional
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
# String provenance qualification
|
||||
|
||||
Measured on 2026-10-14, macOS arm64, Rust 1.98.1, release profile. The workload
|
||||
uses 1,024, 16,384 and 65,536 independently allocated equal-text owners in each
|
||||
image/session domain. Setup is outside query timing. Each query row executes
|
||||
1,000 operations. The native import and mixed snapshot cases call the actual VM
|
||||
boundary methods; a separate diagnostics feature exposes only the fixture.
|
||||
|
||||
Reproduce with:
|
||||
|
||||
```sh
|
||||
cargo run --release -p prometeu-vm --features resource-diagnostics --example string_provenance_benchmark
|
||||
cargo test -p prometeu-vm --test execution_allocations --all-features
|
||||
```
|
||||
|
||||
## Structure and ownership
|
||||
|
||||
Image Weak records are sorted once by allocation data address and UTF-8 length.
|
||||
An in-place heap sort precharges LoadBudget work before each record comparison
|
||||
(four field reads plus comparison) and swap (two reads plus two writes). The
|
||||
existing scratch/retained payload accounting includes the record capacity;
|
||||
sorting adds no auxiliary allocation. Constant-pool and export order, including
|
||||
lease indices, are unchanged. Binary search performs at most floor(log2(N))+1
|
||||
record comparisons for a nonempty table. In-place dead-record retention preserves
|
||||
sorted order.
|
||||
|
||||
Session records use an open-addressed BudgetVec table with no strong content
|
||||
owner, at most 50% live occupancy, geometric growth and tombstones. Full identity
|
||||
is checked after hashing; equal text in another allocation is a miss. Growth and
|
||||
same-capacity rebuild reserve the retained capacity delta in AuxiliaryBytes and
|
||||
the old allocation in TemporaryBytes before allocating or moving records.
|
||||
Publication reserves content and table space before constructing an Arc. Lookup
|
||||
neither reserves nor resizes. Reconciliation removes a key before releasing its
|
||||
Weak and content charges, preventing recycled addresses from matching stale keys.
|
||||
Dead content is still reclaimed at the existing safepoints/pressure pass.
|
||||
|
||||
## Query work and allocator calls
|
||||
|
||||
| Owners per domain | Image hit comparisons before → after | Image miss comparisons before → after | Session hit probes before → after | Session miss probes before → after |
|
||||
| ---: | ---: | ---: | ---: | ---: |
|
||||
| 1,024 | 1024 → 10 | 1024 → 10 | 1024 → 1 | 1024 → 1 |
|
||||
| 16,384 | 16384 → 14 | 16384 → 15 | 16384 → 1 | 16384 → 1 |
|
||||
| 65,536 | 65536 → 16 | 65536 → 17 | 65536 → 2 | 65536 → 1 |
|
||||
|
||||
The hit is the last-created owner, which was last in the original linear ledger.
|
||||
The miss has equal text in a distinct allocation. Hash probe observations depend
|
||||
on allocator-selected addresses and are not worst-case constant-time guarantees.
|
||||
A separate fixed synthetic-key collision fixture verifies chains, full keys and
|
||||
tombstones. Integration tests bound image comparisons and assert zero allocations,
|
||||
reallocations and deallocations for image/session hits/misses and tracked imports
|
||||
at all three sizes. Ordinary string reads still access Arc directly.
|
||||
|
||||
All direct queries and tracked imports reported `[0, 0, 0]` allocator calls
|
||||
before and after. Mixed two-value snapshots reported `[1000, 0, 1000]`, for their
|
||||
Value-vector storage. Untracked imports reported `[1001, 0, 1]`: 1,000 new Arcs
|
||||
plus one table growth; dead content remains Weak-tracked until reconciliation.
|
||||
The benchmark asserts zero calls in the lookup regions; setup, reporting and
|
||||
fixture teardown are excluded. The CI allocator oracle is thread-local.
|
||||
|
||||
## Supporting timings
|
||||
|
||||
Elapsed milliseconds for 1,000 operations; a single local run per implementation,
|
||||
not a percentile estimate or a CI threshold. Warmup, allocator layout, scheduling
|
||||
and CPU frequency can affect these values.
|
||||
|
||||
| Owners | Workload | Linear (ms) | Indexed (ms) |
|
||||
| ---: | --- | ---: | ---: |
|
||||
| 1,024 | `image_hit` | 1.602 | 0.058 |
|
||||
| 1,024 | `image_miss` | 1.511 | 0.046 |
|
||||
| 1,024 | `session_hit` | 1.160 | 0.030 |
|
||||
| 1,024 | `session_miss` | 0.684 | 0.025 |
|
||||
| 1,024 | `import_hit` | 1.513 | 0.101 |
|
||||
| 1,024 | `snapshot_mixed` | 2.031 | 0.216 |
|
||||
| 1,024 | `import_miss` | 2.233 | 0.376 |
|
||||
| 16,384 | `image_hit` | 10.658 | 0.026 |
|
||||
| 16,384 | `image_miss` | 11.847 | 0.023 |
|
||||
| 16,384 | `session_hit` | 10.890 | 0.009 |
|
||||
| 16,384 | `session_miss` | 11.412 | 0.009 |
|
||||
| 16,384 | `import_hit` | 21.726 | 0.035 |
|
||||
| 16,384 | `snapshot_mixed` | 33.012 | 0.083 |
|
||||
| 16,384 | `import_miss` | 22.832 | 0.887 |
|
||||
| 65,536 | `image_hit` | 44.191 | 0.029 |
|
||||
| 65,536 | `image_miss` | 43.959 | 0.025 |
|
||||
| 65,536 | `session_hit` | 47.441 | 0.010 |
|
||||
| 65,536 | `session_miss` | 45.249 | 0.009 |
|
||||
| 65,536 | `import_hit` | 88.518 | 0.038 |
|
||||
| 65,536 | `snapshot_mixed` | 132.199 | 0.094 |
|
||||
| 65,536 | `import_miss` | 90.935 | 3.137 |
|
||||
|
||||
## Creation and memory cost
|
||||
|
||||
The `image_load_ns` diagnostic includes fixture assembly/serialization and sealed
|
||||
loading, not just the sort. Session setup includes the VM, ledger and retained
|
||||
owners. Auxiliary measurements include that fixture's small VM/domain state;
|
||||
image payload measurements include the complete immutable payload, not only its
|
||||
index. These are charged native capacities, not process RSS.
|
||||
|
||||
| Owners | Image fixture before → after (ms) | Session setup before → after (ms) | Session AuxiliaryBytes before → after | Image payload bytes (unchanged) | Session setup allocations before → after |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 1,024 | 0.600 → 1.128 | 0.236 → 0.348 | 65,696 → 164,016 | 74,072 | 1046 → 1044 |
|
||||
| 16,384 | 1.702 → 4.067 | 1.298 → 2.889 | 1,048,736 → 2,621,616 | 1,179,992 | 16410 → 16408 |
|
||||
| 65,536 | 6.248 → 14.130 | 5.230 → 9.885 | 4,194,464 → 10,485,936 | 4,718,936 | 65564 → 65562 |
|
||||
|
||||
At 65,536 session owners the charged auxiliary footprint rises from roughly 4 MiB
|
||||
to 10 MiB. The 50% occupancy policy costs capacity and increases setup work, while
|
||||
removing whole-ledger traversals at native boundaries. Existing quotas are
|
||||
unchanged; this metadata can reduce available headroom for other session owners.
|
||||
No extra strong Arc owner, guest handle, content interning or new GC schedule was
|
||||
introduced.
|
||||
|
||||
## Refusal and scope of evidence
|
||||
|
||||
Tests force initial allocation, growth and tombstone rebuild failure, auxiliary
|
||||
and coexistence limits, and image sort work exhaustion. They verify no partial
|
||||
entry/content publication, retained live identities and refunded provisional
|
||||
charges. Allocator failures are injected at the fallible allocation seam; stable
|
||||
Arc allocation can still abort on physical allocator exhaustion, as before.
|
||||
Cross-session copies, native lease/snapshot survival and typed storage/root tests
|
||||
remain part of the regression suite. Address ordering can change between loads;
|
||||
the sort refusal fixture is bounded for either order, not tied to a prior address
|
||||
layout.
|
||||
|
||||
These synthetic boundary measurements establish lookup work and ownership
|
||||
behavior. They do not establish gameplay/frame speedups, long-run memory stability,
|
||||
other-platform timings or hardware-independent latency limits. The normative
|
||||
ownership contract is in [Registry and PBX](../specs/runtime/02c-program-registry-and-pbx-v1.md#memory-ownership-and-quotas).
|
||||
Vendored
+34
-40
@@ -4,55 +4,49 @@ pipeline {
|
||||
environment {
|
||||
CARGO_HOME = '/var/jenkins_home/.cargo'
|
||||
CARGO_TARGET_DIR = 'target'
|
||||
|
||||
CARGO_TERM_COLOR = 'always'
|
||||
}
|
||||
|
||||
stages {
|
||||
stage('Build') {
|
||||
stage('Prepare reports') {
|
||||
steps {
|
||||
dir('target/quality-gate') { deleteDir() }
|
||||
dir('target/llvm-cov') { deleteDir() }
|
||||
}
|
||||
}
|
||||
stage('Provision') {
|
||||
steps {
|
||||
sh '''
|
||||
set -eux
|
||||
set -eu
|
||||
export PATH="$CARGO_HOME/bin:$PATH"
|
||||
bash scripts/provision-quality-gate.sh
|
||||
'''
|
||||
}
|
||||
}
|
||||
stage('Quality gate') {
|
||||
steps {
|
||||
sh '''
|
||||
set -eu
|
||||
export PATH="$CARGO_HOME/bin:$PATH"
|
||||
make ci-domains
|
||||
'''
|
||||
// withChecks(name: 'Test', includeStage: true) {
|
||||
// sh '''
|
||||
// set -eux
|
||||
// make ci-domains
|
||||
// '''
|
||||
// recordCoverage(
|
||||
// tools: [[parser: 'COBERTURA', pattern: 'target/llvm-cov/cobertura.xml']],
|
||||
// sourceCodeRetention: 'LAST_BUILD',
|
||||
// enabledForFailure: true,
|
||||
// failOnError: false,
|
||||
// checksAnnotationScope: 'MODIFIED_LINES',
|
||||
// id: 'rust-coverage',
|
||||
// name: 'Rust Coverage',
|
||||
// checksName: 'Rust Coverage',
|
||||
// qualityGates: [
|
||||
// [metric: 'LINE', baseline: 'MODIFIED_LINES', threshold: 0.0],
|
||||
// [metric: 'LINE', baseline: 'PROJECT', threshold: 0.0],
|
||||
// [metric: 'BRANCH', baseline: 'PROJECT', threshold: 0.0]
|
||||
// ]
|
||||
// )
|
||||
// }
|
||||
}
|
||||
} // Test
|
||||
}
|
||||
}
|
||||
|
||||
stage('Reports') {
|
||||
steps {
|
||||
publishHTML(target: [
|
||||
allowMissing: false,
|
||||
alwaysLinkToLastBuild: true,
|
||||
keepAll: true,
|
||||
reportDir: 'target/llvm-cov/html',
|
||||
reportFiles: 'index.html',
|
||||
reportName: 'Rust Coverage HTML',
|
||||
reportTitles: 'Coverage Report'
|
||||
])
|
||||
|
||||
archiveArtifacts artifacts: 'target/llvm-cov/**', fingerprint: true
|
||||
}
|
||||
} // Reports
|
||||
post {
|
||||
always {
|
||||
archiveArtifacts artifacts: 'target/quality-gate/**,target/llvm-cov/**',
|
||||
fingerprint: true, allowEmptyArchive: true
|
||||
publishHTML(target: [
|
||||
allowMissing: true,
|
||||
alwaysLinkToLastBuild: true,
|
||||
keepAll: true,
|
||||
reportDir: 'target/llvm-cov/html',
|
||||
reportFiles: 'index.html',
|
||||
reportName: 'Rust Coverage HTML',
|
||||
reportTitles: 'Coverage Report'
|
||||
])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,7 +19,8 @@
|
||||
"baseline_percent": 76.08,
|
||||
"paths": [
|
||||
"crates/console/prometeu-system/src/services/fs/",
|
||||
"crates/console/prometeu-system/src/services/memcard.rs"
|
||||
"crates/console/prometeu-system/src/services/memcard.rs",
|
||||
"crates/console/prometeu-system/src/services/memcard/"
|
||||
]
|
||||
},
|
||||
"system/programs": {
|
||||
@@ -36,7 +37,8 @@
|
||||
"crates/console/prometeu-hal/src/asset_bridge.rs",
|
||||
"crates/console/prometeu-hal/src/cartridge.rs",
|
||||
"crates/console/prometeu-hal/src/cartridge_loader.rs",
|
||||
"crates/console/prometeu-drivers/src/asset.rs"
|
||||
"crates/console/prometeu-drivers/src/asset.rs",
|
||||
"crates/console/prometeu-drivers/src/asset/"
|
||||
]
|
||||
},
|
||||
"firmware": {
|
||||
@@ -88,11 +90,12 @@
|
||||
"crates/console/prometeu-drivers/src/"
|
||||
],
|
||||
"exclude_paths": [
|
||||
"crates/console/prometeu-drivers/src/asset.rs"
|
||||
"crates/console/prometeu-drivers/src/asset.rs",
|
||||
"crates/console/prometeu-drivers/src/asset/"
|
||||
]
|
||||
},
|
||||
"system/core": {
|
||||
"baseline_percent": 80.0,
|
||||
"baseline_percent": 87.71,
|
||||
"paths": [
|
||||
"crates/console/prometeu-system/src/"
|
||||
],
|
||||
@@ -103,6 +106,7 @@
|
||||
"crates/console/prometeu-system/src/services/vm_runtime",
|
||||
"crates/console/prometeu-system/src/services/fs/",
|
||||
"crates/console/prometeu-system/src/services/memcard.rs",
|
||||
"crates/console/prometeu-system/src/services/memcard/",
|
||||
"crates/console/prometeu-system/src/programs/"
|
||||
]
|
||||
},
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
#!/usr/bin/env bash
|
||||
# Shared CI prerequisites. Run on a disposable Debian/Ubuntu agent with sudo
|
||||
# (or root), or on an already provisioned macOS development machine.
|
||||
set -euo pipefail
|
||||
if [[ "$(uname -s)" == Linux ]]; then
|
||||
admin=()
|
||||
if [[ "$EUID" -ne 0 ]]; then admin=(sudo); fi
|
||||
"${admin[@]}" apt-get update
|
||||
"${admin[@]}" apt-get install -y --no-install-recommends \
|
||||
build-essential pkg-config libasound2-dev libudev-dev libxkbcommon-dev \
|
||||
libwayland-dev libx11-dev libxrandr-dev libxi-dev \
|
||||
python3 jq ripgrep git curl ca-certificates
|
||||
fi
|
||||
for command in rustup cargo python3 jq rg git make pkg-config; do
|
||||
command -v "$command" >/dev/null || { echo "Missing quality-gate prerequisite: $command" >&2; exit 1; }
|
||||
done
|
||||
rustup component add rustfmt clippy llvm-tools-preview
|
||||
if [[ "$(cargo llvm-cov --version 2>/dev/null || true)" != 'cargo-llvm-cov 0.8.5' ]]; then
|
||||
cargo install cargo-llvm-cov --version 0.8.5 --locked
|
||||
fi
|
||||
if [[ "$(uname -s)" == Linux ]]; then pkg-config --exists alsa; fi
|
||||
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run Makefile-owned gate steps serially and retain evidence on failure."""
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
def capture(*command):
|
||||
try:
|
||||
result = subprocess.run(command, text=True, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT, check=False)
|
||||
return {"exit_code": result.returncode, "output": result.stdout.strip()}
|
||||
except OSError as error:
|
||||
return {"exit_code": 127, "output": str(error)}
|
||||
|
||||
|
||||
def sha(path):
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def main():
|
||||
root = Path(__file__).resolve().parent.parent
|
||||
os.chdir(root)
|
||||
output = root / "target/quality-gate"
|
||||
# These two directories contain generated reports only. Never archive a
|
||||
# previous run's coverage as evidence for an early lint/test failure.
|
||||
for directory in (output, root / "target/llvm-cov"):
|
||||
if directory.exists():
|
||||
shutil.rmtree(directory)
|
||||
output.mkdir(parents=True)
|
||||
inputs = [Path("Makefile"), Path("Cargo.lock"), Path(".github/workflows/ci.yml"),
|
||||
Path("files/config/Jenkinsfile"), Path("files/config/runtime-edge-coverage-domains.json")]
|
||||
inputs.extend(sorted(Path("scripts").rglob("*.py")))
|
||||
inputs.extend(sorted(Path("scripts").glob("*.sh")))
|
||||
fingerprints = {str(path): sha(path) for path in inputs if path.is_file()}
|
||||
sources = "".join(f"{path}\0{sha(path)}\n" for path in sorted(Path("crates").rglob("*.rs")))
|
||||
evidence = {
|
||||
"started_utc": datetime.now(timezone.utc).isoformat(),
|
||||
"host": platform.platform(), "revision": capture("git", "rev-parse", "HEAD"),
|
||||
"working_tree": capture("git", "status", "--porcelain"),
|
||||
"rustc": capture("rustc", "-vV"), "cargo_llvm_cov": capture("cargo", "llvm-cov", "--version"),
|
||||
"config_sha256": fingerprints,
|
||||
"rust_source_sha256": hashlib.sha256(sources.encode()).hexdigest(),
|
||||
"source_hash_algorithm": "SHA-256 of sorted path + NUL + file SHA-256 hex + newline records",
|
||||
"steps": [], "status": "running",
|
||||
}
|
||||
metadata = output / "metadata.json"
|
||||
|
||||
def persist():
|
||||
metadata.write_text(json.dumps(evidence, indent=2) + "\n")
|
||||
|
||||
persist()
|
||||
exit_code = 0
|
||||
try:
|
||||
if not sys.argv[1:]:
|
||||
raise ValueError("Makefile must supply the required gate steps")
|
||||
for target in sys.argv[1:]:
|
||||
if not target.replace("-", "").isalnum():
|
||||
raise ValueError(f"invalid make target: {target!r}")
|
||||
command = ["make", "--no-print-directory", "-j1", target]
|
||||
log = output / f"{target}.log"
|
||||
print(f"Quality gate: {target}", flush=True)
|
||||
with log.open("w") as stream:
|
||||
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
|
||||
text=True, errors="replace")
|
||||
for line in process.stdout:
|
||||
stream.write(line)
|
||||
print(line, end="", flush=True)
|
||||
result = process.wait()
|
||||
exit_code = result if result >= 0 else 128 - result
|
||||
evidence["steps"].append({"target": target, "command": command,
|
||||
"exit_code": exit_code, "log_sha256": sha(log)})
|
||||
persist()
|
||||
if exit_code:
|
||||
break
|
||||
except KeyboardInterrupt:
|
||||
evidence["error"] = "Interrupted"
|
||||
exit_code = 130
|
||||
except Exception as error:
|
||||
evidence["error"] = str(error)
|
||||
exit_code = 1
|
||||
finally:
|
||||
evidence["status"] = "passed" if exit_code == 0 else "failed"
|
||||
evidence["exit_code"] = exit_code
|
||||
evidence["finished_utc"] = datetime.now(timezone.utc).isoformat()
|
||||
evidence["reports_sha256"] = {
|
||||
str(path.relative_to(root)): sha(path)
|
||||
for path in sorted((root / "target/llvm-cov").rglob("*")) if path.is_file()
|
||||
}
|
||||
persist()
|
||||
return exit_code
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,55 @@
|
||||
"""Exercise real make failures without running Cargo or modifying the repository."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
class QualityGateTests(unittest.TestCase):
|
||||
def run_gate(self, makefile, targets):
|
||||
temporary = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(temporary.cleanup)
|
||||
root = Path(temporary.name)
|
||||
(root / "scripts").mkdir()
|
||||
shutil.copy(Path(__file__).resolve().parents[1] / "run-quality-gate.py", root / "scripts")
|
||||
(root / "Makefile").write_text(makefile)
|
||||
(root / "target/llvm-cov").mkdir(parents=True)
|
||||
(root / "target/llvm-cov/stale.json").write_text("old evidence")
|
||||
result = subprocess.run([sys.executable, str(root / "scripts/run-quality-gate.py"), *targets],
|
||||
capture_output=True, text=True)
|
||||
evidence = json.loads((root / "target/quality-gate/metadata.json").read_text())
|
||||
self.assertFalse((root / "target/llvm-cov/stale.json").exists())
|
||||
return root, result, evidence
|
||||
|
||||
def test_failure_preserves_logs_identity_exit_and_stops_later_steps(self):
|
||||
root, result, evidence = self.run_gate(
|
||||
'fail:\n\t@echo diagnostic\n\t@exit 7\nlater:\n\t@touch should-not-exist\n',
|
||||
['fail', 'later'])
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertEqual(evidence['exit_code'], result.returncode)
|
||||
self.assertEqual(evidence['status'], 'failed')
|
||||
self.assertEqual([step['target'] for step in evidence['steps']], ['fail'])
|
||||
self.assertIn('diagnostic', (root / 'target/quality-gate/fail.log').read_text())
|
||||
self.assertIn('Makefile', evidence['config_sha256'])
|
||||
self.assertIn('revision', evidence)
|
||||
self.assertEqual(evidence['reports_sha256'], {})
|
||||
self.assertFalse((root / 'should-not-exist').exists())
|
||||
|
||||
def test_success_fingerprints_only_current_reports(self):
|
||||
_, result, evidence = self.run_gate(
|
||||
'report:\n\t@mkdir -p target/llvm-cov\n\t@echo fresh > target/llvm-cov/report.json\n',
|
||||
['report'])
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertEqual(evidence['status'], 'passed')
|
||||
self.assertEqual(list(evidence['reports_sha256']), ['target/llvm-cov/report.json'])
|
||||
|
||||
def test_partial_report_is_retained_when_a_later_gate_fails(self):
|
||||
_, result, evidence = self.run_gate(
|
||||
'report:\n\t@mkdir -p target/llvm-cov\n\t@echo fresh > target/llvm-cov/report.json\nfail:\n\t@exit 1\n',
|
||||
['report', 'fail'])
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertEqual(evidence['status'], 'failed')
|
||||
self.assertIn('target/llvm-cov/report.json', evidence['reports_sha256'])
|
||||
Reference in New Issue
Block a user