feat(audio): add 32-voice PCM mixing and native music sequencing
Add self-contained SoundBanks and MusicBanks with per-sample PCM16 and IMA ADPCM storage, materialized to resident PCM16. Integrate instrument envelopes, automation, shared DSP and host lifecycle controls while preserving staged real-time execution.
This commit is contained in:
@@ -1,70 +1,82 @@
|
||||
# Audio Peripheral
|
||||
|
||||
Domain: virtual hardware: audio
|
||||
Domain: virtual hardware: audio. Function: normative.
|
||||
|
||||
Function: normative
|
||||
## Machine contract
|
||||
|
||||
Didactic companion: [Audio admission and lifecycle delivery](../../../discussion/lessons/DSC-0055-guest-audio-voice-control/LSN-0069-audio-admission-needs-a-delivery-boundary-and-independent-lifecycle-control.md).
|
||||
The peripheral mixes **32 PCM voices** into stereo output at **48000 Hz**:
|
||||
12 Music A, 12 Music B, and 8 SFX. Groups never borrow voices. Mono and stereo
|
||||
samples each consume one logical voice. Music groups have independent resident
|
||||
bank selection, sequence, tempo, loop, volume, pause, automation and effects.
|
||||
A shared Rust engine performs desktop, headless and offline execution; the host
|
||||
owns physical scheduling and the output device.
|
||||
|
||||
## 1 Scope and machine limits
|
||||
The authoritative binary format, instrument/event model, numerical rules and new
|
||||
high-level ABI are in [Audio banks and sequences](05a-audio-bank-format.md).
|
||||
SoundBank and MusicBank each have 16 independent resident slots. A MusicBank
|
||||
contains exactly one complete composition and every required instrument/sample.
|
||||
Loading is explicit and atomic. PCM16_LE and IMA_ADPCM_WAV4 are selected per
|
||||
sample, decoded completely during loading and resident as PCM16. The mixer never
|
||||
decodes assets or loads a bank implicitly.
|
||||
|
||||
The AUDIO peripheral provides command-driven PCM playback on **16 explicitly selected voices**. Each voice plays one immutable mono PCM16 sample, with independent volume, stereo pan, playback rate and optional looping. The output target is **48,000 Hz stereo**, with saturating mixing. Hosts may use normalized floating-point device buffers to represent that output.
|
||||
## SFX controls and status
|
||||
|
||||
The guest controls playback at logical-frame boundaries. The host renders samples independently of the 60 Hz game loop. These are separate clocks: a successful syscall admits a command, without promising that sound has already reached the device or a fixed latency in milliseconds.
|
||||
Existing audio play/control identities keep their single current meaning; no
|
||||
legacy voice/format path is provided. `audio.play(bank,sample,voice,volume,pan,
|
||||
pitch,loop)` and `audio.play_sample(sample,voice,volume,pan,pitch)` address SFX
|
||||
local voices **0..7**. The latter selects resident SoundBank 0 with looping off.
|
||||
A play replaces that voice. Stop is immediate/idempotent and setters do not
|
||||
start an inactive voice. No allocation/stealing of another SFX voice occurs.
|
||||
|
||||
This surface does not define playback handles, completion queries/events, automatic voice allocation, music sequencing, synthesis, streaming, envelopes or background audio producers.
|
||||
Volume and pan/balance are 0..255; bank slots are 0..15 and sample indices
|
||||
0..65535. Pitch is finite and positive; loop is exactly 0 or 1. Validate before
|
||||
narrowing. Structural type/capability checks precede operational validation;
|
||||
voice precedes resource lookup, and invalid commands never consume quota.
|
||||
|
||||
## 2 Guest ABI v1
|
||||
All operations require AUDIO and return one status at five logical cycles:
|
||||
0 OK (admitted asynchronously), 1 VOICE_INVALID (direct SFX play/stop/setters),
|
||||
2 SAMPLE_NOT_FOUND, 3 ARG_RANGE_INVALID (including high-level group/bus/send selectors), 4 ASSET_NOT_FOUND (reserved),
|
||||
5 NO_EFFECT (reserved), 6 BANK_INVALID,
|
||||
7 COMMAND_LIMIT. Loading/decoding remains asset-manager work.
|
||||
|
||||
Every entry requires capability `AUDIO`, returns exactly one `status:int`, and has fixed syscall cost **5 cycles**, including operational rejection. Structural ABI misuse follows the VM trap policy. Mixing time is host work, not the syscall's logical cycle charge.
|
||||
## Playback and signal
|
||||
|
||||
| Identity | ID | Arguments in order |
|
||||
| --- | --- | --- |
|
||||
| `audio.play_sample` | `0x3001` | `sample:int, voice:int, volume:int, pan:int, pitch:number` |
|
||||
| `audio.play` | `0x3002` | `bank:int, sample:int, voice:int, volume:int, pan:int, pitch:number, loop:int` |
|
||||
| `audio.stop` | `0x3003` | `voice:int` |
|
||||
| `audio.set_volume` | `0x3004` | `voice:int, volume:int` |
|
||||
| `audio.set_pan` | `0x3005` | `voice:int, pan:int` |
|
||||
| `audio.set_pitch` | `0x3006` | `voice:int, pitch:number` |
|
||||
Immutable samples are referenced without PCM copies per voice. Source-rate ratio
|
||||
and pitch advance a fractional cursor with linear interpolation. Loop bounds are
|
||||
validated decoded frames, start inclusive/end exclusive; playback begins at zero
|
||||
and preserves an intro. Loop-off ends at the full sample length. Interpolation
|
||||
wraps at loop-end when looping. A finite large rate must wrap safely without an
|
||||
infinite position. Empty nonlooped samples are inactive without reading memory.
|
||||
|
||||
`number` means the existing VM numeric carriers accepted by audio playback, including integer and floating-point values; it is not a new ABI type. `play_sample` uses bank 0 and looping off. Both plays use internal priority 0. The public API has no priority argument.
|
||||
Mono uses equal-power pan; stereo balance preserves its original two channels at
|
||||
center. Instrument/track/group, Music Bus, SFX Bus and master gains compose before
|
||||
final saturation. The three groups each have shared low-pass/delay processing and
|
||||
selective sends; no per-voice DSP chain exists. Fade, envelope, LFO, portamento and
|
||||
musical event timing use the audio timeline, independently of the game frame.
|
||||
|
||||
### 2.1 Ranges and validation
|
||||
## Ownership, replacement and real-time processing
|
||||
|
||||
- Voice: integer **0..15**.
|
||||
- Bank: integer **0..15**, referring to an installed sound bank. Validate before narrowing; 256 must never alias bank 0.
|
||||
- Sample index: integer **0..65535**; the sample must exist in the selected bank.
|
||||
- Volume: integer **0..255**. Zero silences output while playback position continues advancing.
|
||||
- Pan: integer **0..255**, from fully left to fully right. The mixer uses equal-power gains `sqrt(1-pan/255)` and `sqrt(pan/255)`.
|
||||
- Pitch: **finite and strictly positive**; 1 means original speed. NaN, infinity, zero and negative values are rejected. No 0.25..4 restriction is imposed. Setters and plays use the same range.
|
||||
- Loop: integer **0 or 1**, meaning off or on. Other values are rejected, without normalization.
|
||||
Unload/replacement invalidates the old bank instance. Its active groups/voices
|
||||
stop on audio consumption, and stale queued plays cannot restart it. Failed
|
||||
replacement preparation preserves the old bank. Developers can fade/stop first.
|
||||
Old references stay valid until reclaimed safely outside the callback; buffers
|
||||
already delivered to a physical device cannot be recalled. Music invalidation
|
||||
clears only affected group effect history. Already mixed SFX delay tails may end
|
||||
naturally while their source voices are stopped.
|
||||
|
||||
Structural type/shape/capability checks precede operational validation. Validate voice first, then other arguments/resources, then command admission. Rejection must not alter playback state or enqueue a command. When multiple operational errors exist, voice takes precedence; bank validity precedes sample lookup for bank playback.
|
||||
|
||||
### 2.2 Status values
|
||||
|
||||
| Value | Name | Meaning |
|
||||
| --- | --- | --- |
|
||||
| 0 | `OK` | Command admitted; application is asynchronous |
|
||||
| 1 | `VOICE_INVALID` | Voice outside 0..15 |
|
||||
| 2 | `SAMPLE_NOT_FOUND` | Sample absent from the selected installed bank |
|
||||
| 3 | `ARG_RANGE_INVALID` | Invalid sample index, volume, pan, pitch or loop value |
|
||||
| 4 | `ASSET_NOT_FOUND` | Reserved existing status |
|
||||
| 5 | `NO_EFFECT` | Reserved existing status; inactive-voice controls do not use it |
|
||||
| 6 | `BANK_INVALID` | Invalid or uninstalled sound bank |
|
||||
| 7 | `COMMAND_LIMIT` | Current guest batch already admitted 1,024 commands |
|
||||
|
||||
Stop and setters can return 0, 1, 3 where applicable, or 7. Stopping an inactive voice or changing its parameters succeeds. No helper based on the driver's former activity mirror is exposed as a playback-completion query.
|
||||
|
||||
## 3 Ordering and voice reuse
|
||||
|
||||
Commands in a published batch apply in emission order before a subsequent output buffer. The API does not promise sample offsets within a buffer.
|
||||
|
||||
A play explicitly replaces the selected voice, resets its position, and supplies every playback parameter. There is no automatic stealing of another voice. `play(A), stop, play(B)` on one voice leaves B playing. A setter following play changes that playback without resetting position. Stop is idempotent, releases the sample and silences the voice. Setters on inactive voices do not establish defaults for future plays.
|
||||
The callback must not allocate, deallocate a last runtime owner, block, perform
|
||||
I/O, decode assets, or execute unbounded event/control work. Samples, MusicBanks,
|
||||
invalidation tokens, command vectors and prepared DSP storage follow bounded
|
||||
retirement/service rules. Host service runs while guest execution is paused or
|
||||
backpressured. Absence/failure of an output device must not deadlock publication.
|
||||
Headless conformance executes the shared engine with explicit frame advancement;
|
||||
silent command consumption alone is not evidence of sequencer or PCM behavior.
|
||||
Headless pause/resume publishes global audio pause state immediately, so explicit
|
||||
audio rendering does not need another VM/graphics tick to observe that control.
|
||||
|
||||
## 4 Admission, publication and backpressure
|
||||
|
||||
A logical frame admits at most **1,024 commands**, shared by both plays and all four controls. Invalid calls do not consume quota. An excess call returns `COMMAND_LIMIT`; a rejected stop requires a guest retry. Quota depends on the logical frame, not the device callback speed or number of physical host ticks.
|
||||
A logical frame admits at most **1,024 commands**, shared by guest SFX, music, bus and effect commands. Bank unload is an asset-owner action outside this admission quota, as specified in the audio-bank ABI. Invalid calls do not consume quota. An excess call returns `COMMAND_LIMIT`; a rejected stop requires a guest retry. Quota depends on the logical frame, not the device callback speed or number of physical host ticks.
|
||||
|
||||
Except for the lifecycle prefix publication defined below, commands remain staged until successful logical-frame completion: explicit `FRAME_SYNC` or the runtime's implicit frame end when the entry function returns (`EndOfRom`). An aborted/faulted frame or `HALT` cancels its unpublished commands. Debugger stepping preserves staging and publishes at either successful completion boundary.
|
||||
|
||||
@@ -82,58 +94,11 @@ Home and debugger pause stop progression of Game voices, preserving their positi
|
||||
|
||||
Reset, termination, crash or cartridge replacement ends the audio session, stops all its voices and invalidates its pending/staged commands. Internal session generations prevent old deliveries from restarting playback. Ending a session may cancel previously admitted commands. Silence is guaranteed from processing the transition in the mixer; buffers already submitted to a physical device cannot be recalled.
|
||||
|
||||
## 6 Samples, interpolation and ownership
|
||||
|
||||
Samples are immutable PCM16 mono, with a source sample rate and optional loop bounds. With looping off, playback ignores all loop markers and ends at the full sample length. With looping on, both markers must be present and satisfy `start < end <= sample length`; otherwise the loop interval is the full sample `[0, length)`. Playback starts at position zero, preserving the intro before the selected loop start, and wraps at the exclusive loop end. Empty samples terminate without reading PCM data.
|
||||
## Qualification
|
||||
|
||||
Linear interpolation uses fractional playback position. With looping off, the next value is the next actual sample frame, or zero beyond the full sample length. With looping on, interpolation across the selected loop end uses the loop-start frame. Playback rate combines source/output sample-rate ratio and pitch. Loop steps may cross multiple repetitions; finite positive pitch, including `f64::MAX`, must not accidentally terminate a short loop or produce an infinite position.
|
||||
|
||||
Unloading or replacing a bank does not interrupt a sample already retained by an admitted command or mixer voice. New plays resolve the current bank. Memory may remain retained after the command is cancelled or playback ends, stops, is replaced, or its session ends, until the host service reclaims the retired owner. Logical inactivity and physical reclamation are separate. The driver must not maintain a second playback mirror that cannot observe completion.
|
||||
|
||||
## 7 Host responsibilities and evidence
|
||||
|
||||
The desktop host consumes batches, mixes PCM and schedules the physical output. If no output device is available, it consumes committed batches silently. A failed or stalled device is diagnosed and detached so pending publication cannot block the guest indefinitely. The desktop implementation treats two seconds without callback progress as a stalled output.
|
||||
|
||||
Headless execution consumes committed batches silently and preserves incomplete frame/callback staging. It validates admission and runtime behavior, without proving audible output. Synthetic mixer tests establish signal behavior; a real device smoke establishes stream initialization and callback consumption, while listening remains a separate observation. Bit-identical PCM across physical devices is not required.
|
||||
|
||||
### Bounded desktop reclamation
|
||||
|
||||
The successful desktop data callback MUST perform no allocation or deallocation
|
||||
of runtime-owned memory, including command-vector storage and the last Sample/PCM
|
||||
owner. This excludes CPAL/device internals and asynchronous driver error reporting.
|
||||
It is not an absolute device-latency guarantee.
|
||||
|
||||
Initialization allocates exactly two retirement packets outside the callback.
|
||||
Each has room for two returned command-vector owners and 3,088 retired Sample
|
||||
references: one 1,024-command frame, one combined 2,048-command lifecycle delivery,
|
||||
and sixteen existing voices. Fixed-capacity SPSC free and return lanes each hold
|
||||
two packets and MUST NOT grow. A checked-out packet implies a free return slot;
|
||||
returning it never waits or frees overflow ownership.
|
||||
|
||||
The callback MUST obtain a packet before taking a publication. It moves commands
|
||||
out of vector slots, retaining the vector allocation in the packet, and moves
|
||||
replaced/stopped/reset voice owners into retirement storage. Stale deliveries are
|
||||
returned whole. Natural completion marks a voice inactive. Without an available
|
||||
packet, no publication is taken, accepted commands remain pending under existing
|
||||
backpressure, and at most sixteen inactive voice slots retain their Sample owners.
|
||||
Inactive slots produce silence and do not advance. They are existing mixer slots,
|
||||
not another playback mirror. Pending pause/session transitions apply when the
|
||||
mixer next obtains a packet and processes control state; already emitted device
|
||||
buffers cannot be recalled.
|
||||
|
||||
`HostAudio::service` MUST drain and recycle packets independently of guest progress,
|
||||
including paused/backpressured execution, before the next guest attempt. It frees
|
||||
retired references and clears returned commands outside the callback. Empty vector
|
||||
storage can be reused through a bounded four-buffer transport cache; excess empty
|
||||
storage is released on the service lane. These buffers retain no Sample owners.
|
||||
There is no change to guest quotas, FIFO/prefix order, session epochs, sixteen-voice
|
||||
capacity, or pause position.
|
||||
|
||||
Failed/stalled output is stopped and dropped before silent consumption takes over,
|
||||
so only one consumer drains the publication transport. Teardown releases callback,
|
||||
packet and pending-publication ownership outside the data callback. Headless/test
|
||||
hosts continue consuming silently through the same transport, without constructing
|
||||
an audio callback or requiring a device.
|
||||
|
||||
See [callback qualification](../../vm-arch/audio-callback-qualification.md) for
|
||||
executed allocation, signal, ownership and device observations.
|
||||
Functional, ownership and allocation tests must cover 32 **stereo** voices, two
|
||||
independent compositions plus SFX, DSP/automation, malformed banks and continuous
|
||||
replacement. Report cartridge bytes, decoded PCM and actual resident/retired/DSP
|
||||
memory separately. Historical 16-voice measurements do not qualify this contract.
|
||||
Hardware ARM/battery qualification requires an identified physical target.
|
||||
|
||||
Reference in New Issue
Block a user