feat(audio): add 32-voice PCM mixing and native music sequencing

Add self-contained SoundBanks and MusicBanks with per-sample PCM16 and IMA ADPCM storage, materialized to resident PCM16. Integrate instrument envelopes, automation, shared DSP and host lifecycle controls while preserving staged real-time execution.
This commit is contained in:
2026-11-20 22:15:24 +00:00
parent a32f834b2b
commit 8510c5b00d
40 changed files with 4708 additions and 509 deletions
+25 -9
View File
@@ -245,35 +245,49 @@ If a scene glyph dependency cannot be resolved:
### 4.3 Decoding validation and physical payload completeness
Complete-buffer/reader decoding and resumable decoding MUST use the same driver
format authority for glyph layout, sound metadata and PCM conversion, and scene
format authority for glyph layout, audio bank/sample descriptors and PCM conversion, and scene
headers, layers, tile ranges and decoded-size validation. Execution mode MUST NOT
change acceptance or semantic bank output for the same declared asset slice.
Every byte of the declared slice MUST be physically available. A short file read
MUST fail; it must not publish a partial sound or accept a physically complete
scene whose byte length is shorter than the declared asset size. This applies
even when the missing sound byte would have been an ignored odd PCM tail.
A physically present final unpaired PCM byte remains accepted and ignored.
including a missing byte in any audio header, descriptor or sample payload.
Audio payloads follow the versioned [audio bank format](05a-audio-bank-format.md); malformed frame counts and unpaired PCM bytes are rejected.
The shared worker retains resumable state and reads at most 4096 payload bytes
per decode step. Scene header and layer steps read their fixed headers; tile
steps use complete tile records. PCM steps end on sample boundaries except for
the final odd tail. Save work takes priority at the next safe step. Cancellation
steps use complete tile records. Audio preparation separately yields after at
most 4096 PCM bytes, one bounded IMA block, or 64 metadata/clock records. PCM
steps end on complete channel frames. Save work takes priority at the next safe
step. Cancellation
and target generation are checked before continuing and before staging a
complete bank; frame-boundary publication remains separate from decoding.
The conformance corpus checks explicit outputs and refusal outcomes across
buffer, reader, legacy load and resumable entry points. It covers empty supported
forms, chunk edges, palettes, truncated and trailing data, metadata, declared
sizes, scene ranges and nonfinite parallax. Ignored reserved fields remain
ignored; this validation sharing does not introduce new codecs or payload versions.
sizes, scene ranges and nonfinite parallax. Audio reserved bytes must be zero;
audio codecs and payload versions are defined in the audio bank format.
### 4.4 Audio banks
`SOUNDS` uses SBNK and `MUSIC` uses MBNK as defined in
[Audio banks and sequences](05a-audio-bank-format.md). Outer codec is NONE;
individual samples select PCM16_LE or IMA_ADPCM_WAV4. Complete validation and
PCM16 materialization precede commit. Audio `decoded_size` counts canonical
metadata/tables plus decoded PCM; actual allocation overhead is separate telemetry.
A single MusicBank is self-contained and has no external asset dependencies.
Replacement/unload invalidates its previous instance and stops its audio users;
last-owner destruction remains outside the callback.
## 5 Banks and Slots
The current runtime exposes bank types:
- `GLYPH`
- `SOUNDS`
- `SOUNDS` (16 SoundBank slots)
- `MUSIC` (16 independent MusicBank slots)
- `SCENE`
Assets are loaded into explicit slots identified by slot index at the public ABI boundary.
@@ -305,13 +319,15 @@ BankTelemetry {
Rules:
- the visible contract MUST NOT expose byte-oriented bank occupancy as the canonical summary;
- canonical bank names are `GLYPH` and `SOUNDS`;
- canonical bank names are `GLYPH`, `SOUNDS`, `MUSIC` and `SCENE`;
- detailed occupancy inspection remains host-owned and slot-based through slot references, not bank byte totals;
- public runtime specs MUST NOT treat `bank.slot_info` or JSON bank inspection payloads as the long-term guest ABI;
- any residual byte accounting MAY exist internally, but it is not part of the visible bank telemetry contract.
The rationale for this boundary is explained in [Host-Owned Debug Boundaries and Cheap Machine Diagnostics](../../../discussion/lessons/DSC-0012-perf-runtime-introspection-syscalls/LSN-0034-host-owned-debug-boundaries.md).
Guest bank selectors use GLYPH=0, SOUNDS=1, SCENE=2, MUSIC=3.
## 6 Load Lifecycle
The runtime asset manager exposes a staged lifecycle: