feat(audio): add 32-voice PCM mixing and native music sequencing

Add self-contained SoundBanks and MusicBanks with per-sample PCM16 and IMA ADPCM storage, materialized to resident PCM16. Integrate instrument envelopes, automation, shared DSP and host lifecycle controls while preserving staged real-time execution.
This commit is contained in:
2026-11-20 22:15:24 +00:00
parent a32f834b2b
commit 8510c5b00d
40 changed files with 4708 additions and 509 deletions
+67 -102
View File
@@ -1,70 +1,82 @@
# Audio Peripheral
Domain: virtual hardware: audio
Domain: virtual hardware: audio. Function: normative.
Function: normative
## Machine contract
Didactic companion: [Audio admission and lifecycle delivery](../../../discussion/lessons/DSC-0055-guest-audio-voice-control/LSN-0069-audio-admission-needs-a-delivery-boundary-and-independent-lifecycle-control.md).
The peripheral mixes **32 PCM voices** into stereo output at **48000 Hz**:
12 Music A, 12 Music B, and 8 SFX. Groups never borrow voices. Mono and stereo
samples each consume one logical voice. Music groups have independent resident
bank selection, sequence, tempo, loop, volume, pause, automation and effects.
A shared Rust engine performs desktop, headless and offline execution; the host
owns physical scheduling and the output device.
## 1 Scope and machine limits
The authoritative binary format, instrument/event model, numerical rules and new
high-level ABI are in [Audio banks and sequences](05a-audio-bank-format.md).
SoundBank and MusicBank each have 16 independent resident slots. A MusicBank
contains exactly one complete composition and every required instrument/sample.
Loading is explicit and atomic. PCM16_LE and IMA_ADPCM_WAV4 are selected per
sample, decoded completely during loading and resident as PCM16. The mixer never
decodes assets or loads a bank implicitly.
The AUDIO peripheral provides command-driven PCM playback on **16 explicitly selected voices**. Each voice plays one immutable mono PCM16 sample, with independent volume, stereo pan, playback rate and optional looping. The output target is **48,000 Hz stereo**, with saturating mixing. Hosts may use normalized floating-point device buffers to represent that output.
## SFX controls and status
The guest controls playback at logical-frame boundaries. The host renders samples independently of the 60 Hz game loop. These are separate clocks: a successful syscall admits a command, without promising that sound has already reached the device or a fixed latency in milliseconds.
Existing audio play/control identities keep their single current meaning; no
legacy voice/format path is provided. `audio.play(bank,sample,voice,volume,pan,
pitch,loop)` and `audio.play_sample(sample,voice,volume,pan,pitch)` address SFX
local voices **0..7**. The latter selects resident SoundBank 0 with looping off.
A play replaces that voice. Stop is immediate/idempotent and setters do not
start an inactive voice. No allocation/stealing of another SFX voice occurs.
This surface does not define playback handles, completion queries/events, automatic voice allocation, music sequencing, synthesis, streaming, envelopes or background audio producers.
Volume and pan/balance are 0..255; bank slots are 0..15 and sample indices
0..65535. Pitch is finite and positive; loop is exactly 0 or 1. Validate before
narrowing. Structural type/capability checks precede operational validation;
voice precedes resource lookup, and invalid commands never consume quota.
## 2 Guest ABI v1
All operations require AUDIO and return one status at five logical cycles:
0 OK (admitted asynchronously), 1 VOICE_INVALID (direct SFX play/stop/setters),
2 SAMPLE_NOT_FOUND, 3 ARG_RANGE_INVALID (including high-level group/bus/send selectors), 4 ASSET_NOT_FOUND (reserved),
5 NO_EFFECT (reserved), 6 BANK_INVALID,
7 COMMAND_LIMIT. Loading/decoding remains asset-manager work.
Every entry requires capability `AUDIO`, returns exactly one `status:int`, and has fixed syscall cost **5 cycles**, including operational rejection. Structural ABI misuse follows the VM trap policy. Mixing time is host work, not the syscall's logical cycle charge.
## Playback and signal
| Identity | ID | Arguments in order |
| --- | --- | --- |
| `audio.play_sample` | `0x3001` | `sample:int, voice:int, volume:int, pan:int, pitch:number` |
| `audio.play` | `0x3002` | `bank:int, sample:int, voice:int, volume:int, pan:int, pitch:number, loop:int` |
| `audio.stop` | `0x3003` | `voice:int` |
| `audio.set_volume` | `0x3004` | `voice:int, volume:int` |
| `audio.set_pan` | `0x3005` | `voice:int, pan:int` |
| `audio.set_pitch` | `0x3006` | `voice:int, pitch:number` |
Immutable samples are referenced without PCM copies per voice. Source-rate ratio
and pitch advance a fractional cursor with linear interpolation. Loop bounds are
validated decoded frames, start inclusive/end exclusive; playback begins at zero
and preserves an intro. Loop-off ends at the full sample length. Interpolation
wraps at loop-end when looping. A finite large rate must wrap safely without an
infinite position. Empty nonlooped samples are inactive without reading memory.
`number` means the existing VM numeric carriers accepted by audio playback, including integer and floating-point values; it is not a new ABI type. `play_sample` uses bank 0 and looping off. Both plays use internal priority 0. The public API has no priority argument.
Mono uses equal-power pan; stereo balance preserves its original two channels at
center. Instrument/track/group, Music Bus, SFX Bus and master gains compose before
final saturation. The three groups each have shared low-pass/delay processing and
selective sends; no per-voice DSP chain exists. Fade, envelope, LFO, portamento and
musical event timing use the audio timeline, independently of the game frame.
### 2.1 Ranges and validation
## Ownership, replacement and real-time processing
- Voice: integer **0..15**.
- Bank: integer **0..15**, referring to an installed sound bank. Validate before narrowing; 256 must never alias bank 0.
- Sample index: integer **0..65535**; the sample must exist in the selected bank.
- Volume: integer **0..255**. Zero silences output while playback position continues advancing.
- Pan: integer **0..255**, from fully left to fully right. The mixer uses equal-power gains `sqrt(1-pan/255)` and `sqrt(pan/255)`.
- Pitch: **finite and strictly positive**; 1 means original speed. NaN, infinity, zero and negative values are rejected. No 0.25..4 restriction is imposed. Setters and plays use the same range.
- Loop: integer **0 or 1**, meaning off or on. Other values are rejected, without normalization.
Unload/replacement invalidates the old bank instance. Its active groups/voices
stop on audio consumption, and stale queued plays cannot restart it. Failed
replacement preparation preserves the old bank. Developers can fade/stop first.
Old references stay valid until reclaimed safely outside the callback; buffers
already delivered to a physical device cannot be recalled. Music invalidation
clears only affected group effect history. Already mixed SFX delay tails may end
naturally while their source voices are stopped.
Structural type/shape/capability checks precede operational validation. Validate voice first, then other arguments/resources, then command admission. Rejection must not alter playback state or enqueue a command. When multiple operational errors exist, voice takes precedence; bank validity precedes sample lookup for bank playback.
### 2.2 Status values
| Value | Name | Meaning |
| --- | --- | --- |
| 0 | `OK` | Command admitted; application is asynchronous |
| 1 | `VOICE_INVALID` | Voice outside 0..15 |
| 2 | `SAMPLE_NOT_FOUND` | Sample absent from the selected installed bank |
| 3 | `ARG_RANGE_INVALID` | Invalid sample index, volume, pan, pitch or loop value |
| 4 | `ASSET_NOT_FOUND` | Reserved existing status |
| 5 | `NO_EFFECT` | Reserved existing status; inactive-voice controls do not use it |
| 6 | `BANK_INVALID` | Invalid or uninstalled sound bank |
| 7 | `COMMAND_LIMIT` | Current guest batch already admitted 1,024 commands |
Stop and setters can return 0, 1, 3 where applicable, or 7. Stopping an inactive voice or changing its parameters succeeds. No helper based on the driver's former activity mirror is exposed as a playback-completion query.
## 3 Ordering and voice reuse
Commands in a published batch apply in emission order before a subsequent output buffer. The API does not promise sample offsets within a buffer.
A play explicitly replaces the selected voice, resets its position, and supplies every playback parameter. There is no automatic stealing of another voice. `play(A), stop, play(B)` on one voice leaves B playing. A setter following play changes that playback without resetting position. Stop is idempotent, releases the sample and silences the voice. Setters on inactive voices do not establish defaults for future plays.
The callback must not allocate, deallocate a last runtime owner, block, perform
I/O, decode assets, or execute unbounded event/control work. Samples, MusicBanks,
invalidation tokens, command vectors and prepared DSP storage follow bounded
retirement/service rules. Host service runs while guest execution is paused or
backpressured. Absence/failure of an output device must not deadlock publication.
Headless conformance executes the shared engine with explicit frame advancement;
silent command consumption alone is not evidence of sequencer or PCM behavior.
Headless pause/resume publishes global audio pause state immediately, so explicit
audio rendering does not need another VM/graphics tick to observe that control.
## 4 Admission, publication and backpressure
A logical frame admits at most **1,024 commands**, shared by both plays and all four controls. Invalid calls do not consume quota. An excess call returns `COMMAND_LIMIT`; a rejected stop requires a guest retry. Quota depends on the logical frame, not the device callback speed or number of physical host ticks.
A logical frame admits at most **1,024 commands**, shared by guest SFX, music, bus and effect commands. Bank unload is an asset-owner action outside this admission quota, as specified in the audio-bank ABI. Invalid calls do not consume quota. An excess call returns `COMMAND_LIMIT`; a rejected stop requires a guest retry. Quota depends on the logical frame, not the device callback speed or number of physical host ticks.
Except for the lifecycle prefix publication defined below, commands remain staged until successful logical-frame completion: explicit `FRAME_SYNC` or the runtime's implicit frame end when the entry function returns (`EndOfRom`). An aborted/faulted frame or `HALT` cancels its unpublished commands. Debugger stepping preserves staging and publishes at either successful completion boundary.
@@ -82,58 +94,11 @@ Home and debugger pause stop progression of Game voices, preserving their positi
Reset, termination, crash or cartridge replacement ends the audio session, stops all its voices and invalidates its pending/staged commands. Internal session generations prevent old deliveries from restarting playback. Ending a session may cancel previously admitted commands. Silence is guaranteed from processing the transition in the mixer; buffers already submitted to a physical device cannot be recalled.
## 6 Samples, interpolation and ownership
Samples are immutable PCM16 mono, with a source sample rate and optional loop bounds. With looping off, playback ignores all loop markers and ends at the full sample length. With looping on, both markers must be present and satisfy `start < end <= sample length`; otherwise the loop interval is the full sample `[0, length)`. Playback starts at position zero, preserving the intro before the selected loop start, and wraps at the exclusive loop end. Empty samples terminate without reading PCM data.
## Qualification
Linear interpolation uses fractional playback position. With looping off, the next value is the next actual sample frame, or zero beyond the full sample length. With looping on, interpolation across the selected loop end uses the loop-start frame. Playback rate combines source/output sample-rate ratio and pitch. Loop steps may cross multiple repetitions; finite positive pitch, including `f64::MAX`, must not accidentally terminate a short loop or produce an infinite position.
Unloading or replacing a bank does not interrupt a sample already retained by an admitted command or mixer voice. New plays resolve the current bank. Memory may remain retained after the command is cancelled or playback ends, stops, is replaced, or its session ends, until the host service reclaims the retired owner. Logical inactivity and physical reclamation are separate. The driver must not maintain a second playback mirror that cannot observe completion.
## 7 Host responsibilities and evidence
The desktop host consumes batches, mixes PCM and schedules the physical output. If no output device is available, it consumes committed batches silently. A failed or stalled device is diagnosed and detached so pending publication cannot block the guest indefinitely. The desktop implementation treats two seconds without callback progress as a stalled output.
Headless execution consumes committed batches silently and preserves incomplete frame/callback staging. It validates admission and runtime behavior, without proving audible output. Synthetic mixer tests establish signal behavior; a real device smoke establishes stream initialization and callback consumption, while listening remains a separate observation. Bit-identical PCM across physical devices is not required.
### Bounded desktop reclamation
The successful desktop data callback MUST perform no allocation or deallocation
of runtime-owned memory, including command-vector storage and the last Sample/PCM
owner. This excludes CPAL/device internals and asynchronous driver error reporting.
It is not an absolute device-latency guarantee.
Initialization allocates exactly two retirement packets outside the callback.
Each has room for two returned command-vector owners and 3,088 retired Sample
references: one 1,024-command frame, one combined 2,048-command lifecycle delivery,
and sixteen existing voices. Fixed-capacity SPSC free and return lanes each hold
two packets and MUST NOT grow. A checked-out packet implies a free return slot;
returning it never waits or frees overflow ownership.
The callback MUST obtain a packet before taking a publication. It moves commands
out of vector slots, retaining the vector allocation in the packet, and moves
replaced/stopped/reset voice owners into retirement storage. Stale deliveries are
returned whole. Natural completion marks a voice inactive. Without an available
packet, no publication is taken, accepted commands remain pending under existing
backpressure, and at most sixteen inactive voice slots retain their Sample owners.
Inactive slots produce silence and do not advance. They are existing mixer slots,
not another playback mirror. Pending pause/session transitions apply when the
mixer next obtains a packet and processes control state; already emitted device
buffers cannot be recalled.
`HostAudio::service` MUST drain and recycle packets independently of guest progress,
including paused/backpressured execution, before the next guest attempt. It frees
retired references and clears returned commands outside the callback. Empty vector
storage can be reused through a bounded four-buffer transport cache; excess empty
storage is released on the service lane. These buffers retain no Sample owners.
There is no change to guest quotas, FIFO/prefix order, session epochs, sixteen-voice
capacity, or pause position.
Failed/stalled output is stopped and dropped before silent consumption takes over,
so only one consumer drains the publication transport. Teardown releases callback,
packet and pending-publication ownership outside the data callback. Headless/test
hosts continue consuming silently through the same transport, without constructing
an audio callback or requiring a device.
See [callback qualification](../../vm-arch/audio-callback-qualification.md) for
executed allocation, signal, ownership and device observations.
Functional, ownership and allocation tests must cover 32 **stereo** voices, two
independent compositions plus SFX, DSP/automation, malformed banks and continuous
replacement. Report cartridge bytes, decoded PCM and actual resident/retired/DSP
memory separately. Historical 16-voice measurements do not qualify this contract.
Hardware ARM/battery qualification requires an identified physical target.