feat(audio): add 32-voice PCM mixing and native music sequencing
Add self-contained SoundBanks and MusicBanks with per-sample PCM16 and IMA ADPCM storage, materialized to resident PCM16. Integrate instrument envelopes, automation, shared DSP and host lifecycle controls while preserving staged real-time execution.
This commit is contained in:
@@ -1,70 +1,82 @@
|
||||
# Audio Peripheral
|
||||
|
||||
Domain: virtual hardware: audio
|
||||
Domain: virtual hardware: audio. Function: normative.
|
||||
|
||||
Function: normative
|
||||
## Machine contract
|
||||
|
||||
Didactic companion: [Audio admission and lifecycle delivery](../../../discussion/lessons/DSC-0055-guest-audio-voice-control/LSN-0069-audio-admission-needs-a-delivery-boundary-and-independent-lifecycle-control.md).
|
||||
The peripheral mixes **32 PCM voices** into stereo output at **48000 Hz**:
|
||||
12 Music A, 12 Music B, and 8 SFX. Groups never borrow voices. Mono and stereo
|
||||
samples each consume one logical voice. Music groups have independent resident
|
||||
bank selection, sequence, tempo, loop, volume, pause, automation and effects.
|
||||
A shared Rust engine performs desktop, headless and offline execution; the host
|
||||
owns physical scheduling and the output device.
|
||||
|
||||
## 1 Scope and machine limits
|
||||
The authoritative binary format, instrument/event model, numerical rules and new
|
||||
high-level ABI are in [Audio banks and sequences](05a-audio-bank-format.md).
|
||||
SoundBank and MusicBank each have 16 independent resident slots. A MusicBank
|
||||
contains exactly one complete composition and every required instrument/sample.
|
||||
Loading is explicit and atomic. PCM16_LE and IMA_ADPCM_WAV4 are selected per
|
||||
sample, decoded completely during loading and resident as PCM16. The mixer never
|
||||
decodes assets or loads a bank implicitly.
|
||||
|
||||
The AUDIO peripheral provides command-driven PCM playback on **16 explicitly selected voices**. Each voice plays one immutable mono PCM16 sample, with independent volume, stereo pan, playback rate and optional looping. The output target is **48,000 Hz stereo**, with saturating mixing. Hosts may use normalized floating-point device buffers to represent that output.
|
||||
## SFX controls and status
|
||||
|
||||
The guest controls playback at logical-frame boundaries. The host renders samples independently of the 60 Hz game loop. These are separate clocks: a successful syscall admits a command, without promising that sound has already reached the device or a fixed latency in milliseconds.
|
||||
Existing audio play/control identities keep their single current meaning; no
|
||||
legacy voice/format path is provided. `audio.play(bank,sample,voice,volume,pan,
|
||||
pitch,loop)` and `audio.play_sample(sample,voice,volume,pan,pitch)` address SFX
|
||||
local voices **0..7**. The latter selects resident SoundBank 0 with looping off.
|
||||
A play replaces that voice. Stop is immediate/idempotent and setters do not
|
||||
start an inactive voice. No allocation/stealing of another SFX voice occurs.
|
||||
|
||||
This surface does not define playback handles, completion queries/events, automatic voice allocation, music sequencing, synthesis, streaming, envelopes or background audio producers.
|
||||
Volume and pan/balance are 0..255; bank slots are 0..15 and sample indices
|
||||
0..65535. Pitch is finite and positive; loop is exactly 0 or 1. Validate before
|
||||
narrowing. Structural type/capability checks precede operational validation;
|
||||
voice precedes resource lookup, and invalid commands never consume quota.
|
||||
|
||||
## 2 Guest ABI v1
|
||||
All operations require AUDIO and return one status at five logical cycles:
|
||||
0 OK (admitted asynchronously), 1 VOICE_INVALID (direct SFX play/stop/setters),
|
||||
2 SAMPLE_NOT_FOUND, 3 ARG_RANGE_INVALID (including high-level group/bus/send selectors), 4 ASSET_NOT_FOUND (reserved),
|
||||
5 NO_EFFECT (reserved), 6 BANK_INVALID,
|
||||
7 COMMAND_LIMIT. Loading/decoding remains asset-manager work.
|
||||
|
||||
Every entry requires capability `AUDIO`, returns exactly one `status:int`, and has fixed syscall cost **5 cycles**, including operational rejection. Structural ABI misuse follows the VM trap policy. Mixing time is host work, not the syscall's logical cycle charge.
|
||||
## Playback and signal
|
||||
|
||||
| Identity | ID | Arguments in order |
|
||||
| --- | --- | --- |
|
||||
| `audio.play_sample` | `0x3001` | `sample:int, voice:int, volume:int, pan:int, pitch:number` |
|
||||
| `audio.play` | `0x3002` | `bank:int, sample:int, voice:int, volume:int, pan:int, pitch:number, loop:int` |
|
||||
| `audio.stop` | `0x3003` | `voice:int` |
|
||||
| `audio.set_volume` | `0x3004` | `voice:int, volume:int` |
|
||||
| `audio.set_pan` | `0x3005` | `voice:int, pan:int` |
|
||||
| `audio.set_pitch` | `0x3006` | `voice:int, pitch:number` |
|
||||
Immutable samples are referenced without PCM copies per voice. Source-rate ratio
|
||||
and pitch advance a fractional cursor with linear interpolation. Loop bounds are
|
||||
validated decoded frames, start inclusive/end exclusive; playback begins at zero
|
||||
and preserves an intro. Loop-off ends at the full sample length. Interpolation
|
||||
wraps at loop-end when looping. A finite large rate must wrap safely without an
|
||||
infinite position. Empty nonlooped samples are inactive without reading memory.
|
||||
|
||||
`number` means the existing VM numeric carriers accepted by audio playback, including integer and floating-point values; it is not a new ABI type. `play_sample` uses bank 0 and looping off. Both plays use internal priority 0. The public API has no priority argument.
|
||||
Mono uses equal-power pan; stereo balance preserves its original two channels at
|
||||
center. Instrument/track/group, Music Bus, SFX Bus and master gains compose before
|
||||
final saturation. The three groups each have shared low-pass/delay processing and
|
||||
selective sends; no per-voice DSP chain exists. Fade, envelope, LFO, portamento and
|
||||
musical event timing use the audio timeline, independently of the game frame.
|
||||
|
||||
### 2.1 Ranges and validation
|
||||
## Ownership, replacement and real-time processing
|
||||
|
||||
- Voice: integer **0..15**.
|
||||
- Bank: integer **0..15**, referring to an installed sound bank. Validate before narrowing; 256 must never alias bank 0.
|
||||
- Sample index: integer **0..65535**; the sample must exist in the selected bank.
|
||||
- Volume: integer **0..255**. Zero silences output while playback position continues advancing.
|
||||
- Pan: integer **0..255**, from fully left to fully right. The mixer uses equal-power gains `sqrt(1-pan/255)` and `sqrt(pan/255)`.
|
||||
- Pitch: **finite and strictly positive**; 1 means original speed. NaN, infinity, zero and negative values are rejected. No 0.25..4 restriction is imposed. Setters and plays use the same range.
|
||||
- Loop: integer **0 or 1**, meaning off or on. Other values are rejected, without normalization.
|
||||
Unload/replacement invalidates the old bank instance. Its active groups/voices
|
||||
stop on audio consumption, and stale queued plays cannot restart it. Failed
|
||||
replacement preparation preserves the old bank. Developers can fade/stop first.
|
||||
Old references stay valid until reclaimed safely outside the callback; buffers
|
||||
already delivered to a physical device cannot be recalled. Music invalidation
|
||||
clears only affected group effect history. Already mixed SFX delay tails may end
|
||||
naturally while their source voices are stopped.
|
||||
|
||||
Structural type/shape/capability checks precede operational validation. Validate voice first, then other arguments/resources, then command admission. Rejection must not alter playback state or enqueue a command. When multiple operational errors exist, voice takes precedence; bank validity precedes sample lookup for bank playback.
|
||||
|
||||
### 2.2 Status values
|
||||
|
||||
| Value | Name | Meaning |
|
||||
| --- | --- | --- |
|
||||
| 0 | `OK` | Command admitted; application is asynchronous |
|
||||
| 1 | `VOICE_INVALID` | Voice outside 0..15 |
|
||||
| 2 | `SAMPLE_NOT_FOUND` | Sample absent from the selected installed bank |
|
||||
| 3 | `ARG_RANGE_INVALID` | Invalid sample index, volume, pan, pitch or loop value |
|
||||
| 4 | `ASSET_NOT_FOUND` | Reserved existing status |
|
||||
| 5 | `NO_EFFECT` | Reserved existing status; inactive-voice controls do not use it |
|
||||
| 6 | `BANK_INVALID` | Invalid or uninstalled sound bank |
|
||||
| 7 | `COMMAND_LIMIT` | Current guest batch already admitted 1,024 commands |
|
||||
|
||||
Stop and setters can return 0, 1, 3 where applicable, or 7. Stopping an inactive voice or changing its parameters succeeds. No helper based on the driver's former activity mirror is exposed as a playback-completion query.
|
||||
|
||||
## 3 Ordering and voice reuse
|
||||
|
||||
Commands in a published batch apply in emission order before a subsequent output buffer. The API does not promise sample offsets within a buffer.
|
||||
|
||||
A play explicitly replaces the selected voice, resets its position, and supplies every playback parameter. There is no automatic stealing of another voice. `play(A), stop, play(B)` on one voice leaves B playing. A setter following play changes that playback without resetting position. Stop is idempotent, releases the sample and silences the voice. Setters on inactive voices do not establish defaults for future plays.
|
||||
The callback must not allocate, deallocate a last runtime owner, block, perform
|
||||
I/O, decode assets, or execute unbounded event/control work. Samples, MusicBanks,
|
||||
invalidation tokens, command vectors and prepared DSP storage follow bounded
|
||||
retirement/service rules. Host service runs while guest execution is paused or
|
||||
backpressured. Absence/failure of an output device must not deadlock publication.
|
||||
Headless conformance executes the shared engine with explicit frame advancement;
|
||||
silent command consumption alone is not evidence of sequencer or PCM behavior.
|
||||
Headless pause/resume publishes global audio pause state immediately, so explicit
|
||||
audio rendering does not need another VM/graphics tick to observe that control.
|
||||
|
||||
## 4 Admission, publication and backpressure
|
||||
|
||||
A logical frame admits at most **1,024 commands**, shared by both plays and all four controls. Invalid calls do not consume quota. An excess call returns `COMMAND_LIMIT`; a rejected stop requires a guest retry. Quota depends on the logical frame, not the device callback speed or number of physical host ticks.
|
||||
A logical frame admits at most **1,024 commands**, shared by guest SFX, music, bus and effect commands. Bank unload is an asset-owner action outside this admission quota, as specified in the audio-bank ABI. Invalid calls do not consume quota. An excess call returns `COMMAND_LIMIT`; a rejected stop requires a guest retry. Quota depends on the logical frame, not the device callback speed or number of physical host ticks.
|
||||
|
||||
Except for the lifecycle prefix publication defined below, commands remain staged until successful logical-frame completion: explicit `FRAME_SYNC` or the runtime's implicit frame end when the entry function returns (`EndOfRom`). An aborted/faulted frame or `HALT` cancels its unpublished commands. Debugger stepping preserves staging and publishes at either successful completion boundary.
|
||||
|
||||
@@ -82,58 +94,11 @@ Home and debugger pause stop progression of Game voices, preserving their positi
|
||||
|
||||
Reset, termination, crash or cartridge replacement ends the audio session, stops all its voices and invalidates its pending/staged commands. Internal session generations prevent old deliveries from restarting playback. Ending a session may cancel previously admitted commands. Silence is guaranteed from processing the transition in the mixer; buffers already submitted to a physical device cannot be recalled.
|
||||
|
||||
## 6 Samples, interpolation and ownership
|
||||
|
||||
Samples are immutable PCM16 mono, with a source sample rate and optional loop bounds. With looping off, playback ignores all loop markers and ends at the full sample length. With looping on, both markers must be present and satisfy `start < end <= sample length`; otherwise the loop interval is the full sample `[0, length)`. Playback starts at position zero, preserving the intro before the selected loop start, and wraps at the exclusive loop end. Empty samples terminate without reading PCM data.
|
||||
## Qualification
|
||||
|
||||
Linear interpolation uses fractional playback position. With looping off, the next value is the next actual sample frame, or zero beyond the full sample length. With looping on, interpolation across the selected loop end uses the loop-start frame. Playback rate combines source/output sample-rate ratio and pitch. Loop steps may cross multiple repetitions; finite positive pitch, including `f64::MAX`, must not accidentally terminate a short loop or produce an infinite position.
|
||||
|
||||
Unloading or replacing a bank does not interrupt a sample already retained by an admitted command or mixer voice. New plays resolve the current bank. Memory may remain retained after the command is cancelled or playback ends, stops, is replaced, or its session ends, until the host service reclaims the retired owner. Logical inactivity and physical reclamation are separate. The driver must not maintain a second playback mirror that cannot observe completion.
|
||||
|
||||
## 7 Host responsibilities and evidence
|
||||
|
||||
The desktop host consumes batches, mixes PCM and schedules the physical output. If no output device is available, it consumes committed batches silently. A failed or stalled device is diagnosed and detached so pending publication cannot block the guest indefinitely. The desktop implementation treats two seconds without callback progress as a stalled output.
|
||||
|
||||
Headless execution consumes committed batches silently and preserves incomplete frame/callback staging. It validates admission and runtime behavior, without proving audible output. Synthetic mixer tests establish signal behavior; a real device smoke establishes stream initialization and callback consumption, while listening remains a separate observation. Bit-identical PCM across physical devices is not required.
|
||||
|
||||
### Bounded desktop reclamation
|
||||
|
||||
The successful desktop data callback MUST perform no allocation or deallocation
|
||||
of runtime-owned memory, including command-vector storage and the last Sample/PCM
|
||||
owner. This excludes CPAL/device internals and asynchronous driver error reporting.
|
||||
It is not an absolute device-latency guarantee.
|
||||
|
||||
Initialization allocates exactly two retirement packets outside the callback.
|
||||
Each has room for two returned command-vector owners and 3,088 retired Sample
|
||||
references: one 1,024-command frame, one combined 2,048-command lifecycle delivery,
|
||||
and sixteen existing voices. Fixed-capacity SPSC free and return lanes each hold
|
||||
two packets and MUST NOT grow. A checked-out packet implies a free return slot;
|
||||
returning it never waits or frees overflow ownership.
|
||||
|
||||
The callback MUST obtain a packet before taking a publication. It moves commands
|
||||
out of vector slots, retaining the vector allocation in the packet, and moves
|
||||
replaced/stopped/reset voice owners into retirement storage. Stale deliveries are
|
||||
returned whole. Natural completion marks a voice inactive. Without an available
|
||||
packet, no publication is taken, accepted commands remain pending under existing
|
||||
backpressure, and at most sixteen inactive voice slots retain their Sample owners.
|
||||
Inactive slots produce silence and do not advance. They are existing mixer slots,
|
||||
not another playback mirror. Pending pause/session transitions apply when the
|
||||
mixer next obtains a packet and processes control state; already emitted device
|
||||
buffers cannot be recalled.
|
||||
|
||||
`HostAudio::service` MUST drain and recycle packets independently of guest progress,
|
||||
including paused/backpressured execution, before the next guest attempt. It frees
|
||||
retired references and clears returned commands outside the callback. Empty vector
|
||||
storage can be reused through a bounded four-buffer transport cache; excess empty
|
||||
storage is released on the service lane. These buffers retain no Sample owners.
|
||||
There is no change to guest quotas, FIFO/prefix order, session epochs, sixteen-voice
|
||||
capacity, or pause position.
|
||||
|
||||
Failed/stalled output is stopped and dropped before silent consumption takes over,
|
||||
so only one consumer drains the publication transport. Teardown releases callback,
|
||||
packet and pending-publication ownership outside the data callback. Headless/test
|
||||
hosts continue consuming silently through the same transport, without constructing
|
||||
an audio callback or requiring a device.
|
||||
|
||||
See [callback qualification](../../vm-arch/audio-callback-qualification.md) for
|
||||
executed allocation, signal, ownership and device observations.
|
||||
Functional, ownership and allocation tests must cover 32 **stereo** voices, two
|
||||
independent compositions plus SFX, DSP/automation, malformed banks and continuous
|
||||
replacement. Report cartridge bytes, decoded PCM and actual resident/retired/DSP
|
||||
memory separately. Historical 16-voice measurements do not qualify this contract.
|
||||
Hardware ARM/battery qualification requires an identified physical target.
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
# Audio bank and sequence format
|
||||
|
||||
Domain: virtual hardware / cartridge assets. Function: normative.
|
||||
|
||||
## Banks and materialization
|
||||
|
||||
`SOUNDS` and `MUSIC` each have 16 independent resident slots. Loading is explicit.
|
||||
The outer asset codec is `NONE`; sample descriptors select the audio codec.
|
||||
The runtime validates and materializes the entire bank before publishing it.
|
||||
Every sample is immutable interleaved PCM16 in resident memory. Glyph, palette,
|
||||
and scene payload formats are unchanged. There is no legacy audio payload path.
|
||||
|
||||
All integers below are little endian. Offsets are relative to the bank start.
|
||||
Reserved bytes must be zero. Counts, lengths, arithmetic, UTF-8, enums, references,
|
||||
and nonoverlapping payload bounds are validated before allocation/publication.
|
||||
`decoded_size` is the canonical metadata/table bytes plus decoded PCM bytes;
|
||||
actual resident allocation overhead is reported separately. No per-bank byte cap
|
||||
is imposed; allocation failure is an operational load error.
|
||||
|
||||
## Header (48 bytes)
|
||||
|
||||
| Offset | Type | Meaning |
|
||||
|---:|---|---|
|
||||
| 0 | bytes[4] | `SBNK` or `MBNK` |
|
||||
| 4 | u16 | version = 1 |
|
||||
| 6 | u16 | flags = 0 |
|
||||
| 8 | u32 | sample count |
|
||||
| 12 | u32 | instrument-zone count |
|
||||
| 16 | u32 | event count |
|
||||
| 20 | u32 | PPQ = 960 |
|
||||
| 24 | u32 | initial microseconds per quarter note, positive |
|
||||
| 28 | u32 | song duration in ticks |
|
||||
| 32 | u32 | song loop start tick, or 0xffffffff |
|
||||
| 36 | u32 | track count (0..256) |
|
||||
| 40 | u32 | UTF-8 song/bank name length |
|
||||
| 44 | u32 | reserved |
|
||||
|
||||
The name follows the header, then sample descriptors, instrument zones, events,
|
||||
and sample payloads, in that order. A SoundBank has at least one sample and no
|
||||
instruments, tracks, events, duration, or song loop. A MusicBank contains exactly
|
||||
one composition; all references are bank-local. Duration must be positive, loop
|
||||
start must precede duration, and the compiled loop must span at least one output
|
||||
frame. Events are sorted by tick; array order breaks equal-time ties.
|
||||
|
||||
## Sample descriptor (40 bytes)
|
||||
|
||||
| Offset | Type | Meaning |
|
||||
|---:|---|---|
|
||||
| 0 | u8 | codec: 0 = PCM16_LE, 1 = IMA_ADPCM_WAV4 |
|
||||
| 1 | u8 | channels: 1 or 2 |
|
||||
| 2 | u16 | flags: bit 0 means loop bounds present |
|
||||
| 4 | u32 | source sample rate, positive |
|
||||
| 8 | u32 | decoded frames per channel |
|
||||
| 12 | u32 | loop start frame |
|
||||
| 16 | u32 | exclusive loop end frame |
|
||||
| 20 | u32 | payload offset |
|
||||
| 24 | u32 | payload bytes |
|
||||
| 28 | u16 | IMA block alignment in bytes; zero for PCM |
|
||||
| 30 | u16 | IMA decoded frames per block; zero for PCM |
|
||||
| 32 | u8 | default volume (0..255) |
|
||||
| 33 | u8 | default pan/balance (0..255, center 128) |
|
||||
| 34 | u16 | reserved |
|
||||
| 36 | u32 | default rate multiplier, unsigned Q16.16, positive |
|
||||
|
||||
PCM payload size is exactly frames * channels * 2. Loops use decoded frames and
|
||||
must satisfy start < end <= frames. Without a loop flag both bounds are zero.
|
||||
Zero-length nonlooped samples are inert. No implicit downmix is performed.
|
||||
|
||||
IMA_ADPCM_WAV4 uses the WAVE IMA four-bit block layout, not RTP DVI4. Each block
|
||||
starts with a four-byte header per channel: initial PCM i16, step index u8 in
|
||||
0..88, and reserved zero. The initial PCM frame is emitted. Codes are decoded
|
||||
low nibble first; stereo alternates four code bytes (eight frames) for left and
|
||||
four for right. Each channel has independent predictor/index state. Predictor is
|
||||
saturated to i16 and index to 0..88 after each code. Block alignment is a positive
|
||||
multiple of 4*channels, at least 8*channels. Frames/block equals
|
||||
1 + (block_alignment - 4*channels)*2/channels. Every stored block is complete;
|
||||
only the final block may contain padding beyond the declared decoded frame count.
|
||||
Its unused codes are zero. The packer defaults to 256 bytes per channel (505
|
||||
frames/block); loops need not align to blocks because playback uses decoded PCM.
|
||||
The codec is lossy and selected explicitly for each sample. Encoder/decoder code
|
||||
must have documented project-compatible provenance and independent conformance
|
||||
vectors. Runtime does not parse a WAV/RIFF container.
|
||||
|
||||
## Instrument zone (64 bytes)
|
||||
|
||||
| Offset | Type | Meaning |
|
||||
|---:|---|---|
|
||||
| 0 | u16 | instrument ID |
|
||||
| 2 | u16 | sample index |
|
||||
| 4 | u8 | lowest note (0..127) |
|
||||
| 5 | u8 | highest note inclusive |
|
||||
| 6 | u8 | sample root note |
|
||||
| 7 | u8 | flags: bit 0 enables looping |
|
||||
| 8 | i16 | fine tuning in cents |
|
||||
| 10 | u8 | pan/balance |
|
||||
| 11 | u8 | gain |
|
||||
| 12 | u32 | attack duration in microseconds |
|
||||
| 16 | u32 | decay duration in microseconds |
|
||||
| 20 | u16 | sustain gain (0..65535) |
|
||||
| 22 | u16 | reserved |
|
||||
| 24 | u32 | release duration in microseconds |
|
||||
| 28 | u32 | vibrato frequency in millihertz |
|
||||
| 32 | u16 | vibrato depth in cents |
|
||||
| 34 | u16 | tremolo depth (0..65535) |
|
||||
| 36 | u32 | tremolo frequency in millihertz |
|
||||
| 40 | u16 | default delay send (0..65535) |
|
||||
| 42 | u8 | curve: 0 = all linear, 1 = linear attack/exponential decay/release |
|
||||
| 43 | bytes[21] | reserved |
|
||||
|
||||
Zones of the same instrument may not overlap. Note-on resolves exactly one zone.
|
||||
ADSR stages reach exact endpoints within their declared output-frame duration;
|
||||
zero duration transitions immediately. Sustain is constant. Note-off releases
|
||||
from the current gain. Exponential segments use the normalized curve
|
||||
(exp(-6*x)-exp(-6))/(1-exp(-6)), x in [0,1], with exact endpoints; table
|
||||
interpolation is permitted within 1e-4 absolute gain error.
|
||||
|
||||
## Event (32 bytes)
|
||||
|
||||
The record contains tick:u32, kind:u8, track:u8, flags:u16, a:u32, b:u32,
|
||||
c:i32, d:i32, duration_frames:u32, curve:u32. Curve is 0 linear or 1 smoothstep.
|
||||
Durations run on audio time, independently of later musical tempo changes.
|
||||
|
||||
| Kind | Operation | Operands |
|
||||
|---:|---|---|
|
||||
| 1 | note-on | a note identity, b instrument, c note, d velocity 0..255; flag 1 legato |
|
||||
| 2 | note-off | a note identity |
|
||||
| 3 | tempo | a positive microseconds/quarter; global |
|
||||
| 4 | track gain ramp | c destination 0..65535 |
|
||||
| 5 | track pan ramp | c destination 0..255 |
|
||||
| 6 | track pitch ramp | c bend in cents |
|
||||
| 7 | vibrato | a millihertz, c depth cents |
|
||||
| 8 | tremolo | a millihertz, c depth 0..65535 |
|
||||
| 9 | track delay send ramp | c destination 0..65535 |
|
||||
| 10 | instrument selection | a instrument ID; note-on b=0xffffffff inherits it |
|
||||
|
||||
Note-on duration_frames is portamento duration when legato is set. Compatible
|
||||
legato retains cursor/envelope on the most recent active note of that track;
|
||||
incompatible instrument/zone/sample retriggers. It never creates a hidden voice.
|
||||
Stealing selects the lowest free local index, oldest release, then oldest note;
|
||||
Release age starts at note-off; note age starts at note-on. Serialized event
|
||||
order resolves same-frame ages, with local index as final tie-breaker.
|
||||
Obsolete note-off cannot affect a reused voice. Authored note identities must
|
||||
distinguish simultaneously live logical notes of a track.
|
||||
|
||||
Events and tempo segments compile into exact rational frame times using u128
|
||||
checked arithmetic and denominator PPQ*1000000. Events execute at the first
|
||||
integer output frame not earlier than their represented time. Carry fractional
|
||||
remainder across tempo changes and song loops. At a loop boundary, notes from
|
||||
the previous iteration enter release and identities are renewed. A frame may
|
||||
execute at most 64 events per group; validate this bound, including the loop
|
||||
seam, during loading. Reject zero-time loops and malformed references.
|
||||
|
||||
## Processing, effects, and lifecycle
|
||||
|
||||
Three fixed groups own 12/12/8 voices. A stereo sample consumes one voice.
|
||||
Mono pan uses equal-power gains; stereo balance preserves L/R at center and
|
||||
attenuates only the opposite side toward an extreme. Mono normalized pan is
|
||||
p/256 for p<=128 and 0.5+(p-128)/254 above center; gains are sqrt(1-pan) and
|
||||
sqrt(pan). Stereo gains are [1,p/128] left of center and [(255-p)/127,1]
|
||||
right of center. Gain and pitch changes
|
||||
do not reset a cursor. Sum in floating point and clamp only at final output.
|
||||
|
||||
Each group has a shared low-pass and stereo delay; selective sends contribute
|
||||
to that group's delay only. Group gain follows dry/wet recombination. Music A/B
|
||||
feed Music Bus; SFX feeds SFX Bus; bus and master gains are independent.
|
||||
Each track initially uses each instrument's default send. A track send event
|
||||
sets an absolute destination: interpolate from each voice's current send to
|
||||
that destination, including interrupted transitions. New notes resolve their
|
||||
instrument default through the same transition. SFX send defaults to zero on
|
||||
new playback and is explicitly set with audio.set_send.
|
||||
|
||||
The filter is a one-pole low-pass: y += a*(x-y),
|
||||
a=1-exp(-2*pi*cutoff/48000). Cutoff zero bypasses filtering. Delay receives
|
||||
unfiltered selective sends; feedback is local to the delay. Output is
|
||||
dry*filtered_input + wet*delayed_input. Changing delay length clears history
|
||||
in O(1); no per-voice filter or delay is allocated.
|
||||
|
||||
Linear interpolation is used for PCM. Vibrato/tremolo use deterministic LFO
|
||||
phase on audio time; gain ramps support linear and smoothstep curves.
|
||||
|
||||
Delay storage is prepared for 2000 ms per group at 48 kHz, stereo f32: 2304000
|
||||
bytes total, independent of bank size. This is an explicit bounded processing
|
||||
budget, not a measured battery/ARM guarantee. Feedback must remain strictly
|
||||
inside (-1,1) after coefficient conversion. Reset invalidates history in O(1),
|
||||
without clearing all buffer bytes in the callback. Pause freezes group DSP and
|
||||
musical time. Music unload/replacement clears its group's history; stopping SFX
|
||||
of one bank allows already mixed shared delay history to finish naturally.
|
||||
|
||||
Resident banks carry invalidation tokens. Installing/unloading a slot invalidates
|
||||
its old instance; retained commands/voices check that token before generating
|
||||
more PCM. Failure to prepare a replacement leaves the old instance valid. Asset
|
||||
commit remains an owner-side frame-boundary operation, independent of audio
|
||||
consumption. No command may resurrect an invalid bank, including after pause.
|
||||
Samples, bank owners, command vectors and invalidation tokens are reclaimed by
|
||||
host service outside the callback; the callback never blocks or destroys a last
|
||||
owner. Already submitted device buffers are not recalled.
|
||||
|
||||
Retirement uses two preallocated packets. Each has capacity
|
||||
2*(3*1024+32+2) resource owners plus two command-vector owners. A staged play
|
||||
retains at most a sample and invalidation token; music owns the whole immutable
|
||||
bank, so note references can be released while that bank is still retained.
|
||||
Retire the bank only after releasing its note references. No callback operation
|
||||
can grow the packet; packet starvation retains inactive owners until service.
|
||||
|
||||
The existing 1024 frame/callback quotas, independent lifecycle slot, FIFO prefix
|
||||
publication, aborted-frame cancellation, session invalidation and bounded
|
||||
backpressure remain in force. Musical events do not consume VM frame quota.
|
||||
|
||||
## High-level ABI additions
|
||||
|
||||
All operations require AUDIO, return one status and cost five logical cycles.
|
||||
Existing audio play/control calls address only SFX local voices 0..7. Music groups
|
||||
are 0=A and 1=B; DSP groups are 0=A,1=B,2=SFX; buses are 0=Music,1=SFX,2=Master.
|
||||
|
||||
| ID | Operation | Arguments |
|
||||
|---|---|---|
|
||||
| 0x3010 | music.play | group, resident bank slot, loop 0/1 |
|
||||
| 0x3011 | music.stop | group |
|
||||
| 0x3012 | music.pause | group |
|
||||
| 0x3013 | music.resume | group |
|
||||
| 0x3014 | music.set_volume | group, gain 0..255 |
|
||||
| 0x3015 | music.fade_to | group, gain 0..255, milliseconds, curve 0/1 |
|
||||
| 0x3016 | music.crossfade | source group, target group, milliseconds, curve 0/1 |
|
||||
| 0x3020 | audio.set_bus_volume | bus, gain 0..255 |
|
||||
| 0x3021 | audio.set_effect | group, cutoff Hz (0=bypass), delay ms 0..2000, feedback number, wet number 0..1, dry number 0..1 |
|
||||
| 0x3022 | audio.set_send | SFX voice, gain 0..255 |
|
||||
| 0x3023 | audio.unload_bank | type 0=SoundBank/1=MusicBank, slot |
|
||||
| 0x3024 | audio.set_group_volume | group 0/1/2, gain 0..255 (independent of bus gain) |
|
||||
|
||||
Group/slot/voice errors and nonfinite/out-of-range arguments reject before
|
||||
admission without consuming quota. No command returns a playback handle or
|
||||
synchronously waits for the consumer. Crossfade schedules both ramps and stops
|
||||
the source at completion. If either group is inactive when the consumer applies
|
||||
the crossfade, it has no effect; admission success is not a consumer acknowledgement.
|
||||
Fade alone to
|
||||
zero keeps sequence time advancing. Unload is an independent bounded owner
|
||||
control, so an exhausted guest audio command quota cannot prevent invalidation.
|
||||
It returns AudioOpStatus, mapping unavailable owner control to BANK_INVALID. It
|
||||
is an asset-owner action: a later VM frame fault does not restore an unloaded bank.
|
||||
Music play/stop/replacement preserve the current group gain and cancel its old
|
||||
fade, so setting gain before play is effective. DSP configuration belongs to the
|
||||
group and persists across bank changes; its history follows the reset policy above.
|
||||
Session/epoch reset restores gains, buses and DSP configuration to defaults.
|
||||
|
||||
## Codec references and diagnostics
|
||||
|
||||
The IMA WAVE block header includes the first uncompressed sample and uses low
|
||||
nibble first. RTP DVI4 has a different framing/order; the distinction is
|
||||
documented in [RFC 3551 section 4.5.1](https://www.rfc-editor.org/rfc/rfc3551#section-4.5.1).
|
||||
The implementation is project-owned code under the project MIT license; no
|
||||
third-party decoder or additional codec library is linked. PCM WAV source
|
||||
layout follows [WAVEFORMATEX](https://learn.microsoft.com/en-us/windows/win32/api/mmreg/ns-mmreg-waveformatex).
|
||||
Only the packer reads WAV containers.
|
||||
|
||||
Owner-side bank diagnostics expose cartridge bytes, canonical decoded bytes,
|
||||
PCM length/allocation bytes per resident bank, and unique PCM totals across
|
||||
slots. Accounted metadata/total includes resident structs, strings and vector
|
||||
capacities, excluding Arc/allocator headers and asset-manager caches; allocator
|
||||
peak observations are separate actual allocation measurements. Materialization elapsed time includes cooperative yields; pipeline load
|
||||
latency is separately recorded by bank type. Mixer diagnostics expose active
|
||||
and maximum voices, frames and render time; the service lane measures retained
|
||||
PCM separately from resident slots. Retained PCM can overlap resident PCM; those
|
||||
reference counts must not be summed as additional memory. These are host diagnostics, not new guest
|
||||
byte-budget quotas. The measurement executable includes allocator peak tracking
|
||||
for replacement/load observations. Prepared engine memory includes fixed DSP
|
||||
capacity; transport, retirement packets and bank memory are additional.
|
||||
|
||||
Integer event timing and codec PCM output are deterministic. Floating-point
|
||||
mixing/DSP replay is tested with identical buffer partitions on a host; bitwise
|
||||
identity across different CPU/libm implementations is not qualified here.
|
||||
@@ -245,35 +245,49 @@ If a scene glyph dependency cannot be resolved:
|
||||
### 4.3 Decoding validation and physical payload completeness
|
||||
|
||||
Complete-buffer/reader decoding and resumable decoding MUST use the same driver
|
||||
format authority for glyph layout, sound metadata and PCM conversion, and scene
|
||||
format authority for glyph layout, audio bank/sample descriptors and PCM conversion, and scene
|
||||
headers, layers, tile ranges and decoded-size validation. Execution mode MUST NOT
|
||||
change acceptance or semantic bank output for the same declared asset slice.
|
||||
|
||||
Every byte of the declared slice MUST be physically available. A short file read
|
||||
MUST fail; it must not publish a partial sound or accept a physically complete
|
||||
scene whose byte length is shorter than the declared asset size. This applies
|
||||
even when the missing sound byte would have been an ignored odd PCM tail.
|
||||
A physically present final unpaired PCM byte remains accepted and ignored.
|
||||
including a missing byte in any audio header, descriptor or sample payload.
|
||||
Audio payloads follow the versioned [audio bank format](05a-audio-bank-format.md); malformed frame counts and unpaired PCM bytes are rejected.
|
||||
|
||||
The shared worker retains resumable state and reads at most 4096 payload bytes
|
||||
per decode step. Scene header and layer steps read their fixed headers; tile
|
||||
steps use complete tile records. PCM steps end on sample boundaries except for
|
||||
the final odd tail. Save work takes priority at the next safe step. Cancellation
|
||||
steps use complete tile records. Audio preparation separately yields after at
|
||||
most 4096 PCM bytes, one bounded IMA block, or 64 metadata/clock records. PCM
|
||||
steps end on complete channel frames. Save work takes priority at the next safe
|
||||
step. Cancellation
|
||||
and target generation are checked before continuing and before staging a
|
||||
complete bank; frame-boundary publication remains separate from decoding.
|
||||
|
||||
The conformance corpus checks explicit outputs and refusal outcomes across
|
||||
buffer, reader, legacy load and resumable entry points. It covers empty supported
|
||||
forms, chunk edges, palettes, truncated and trailing data, metadata, declared
|
||||
sizes, scene ranges and nonfinite parallax. Ignored reserved fields remain
|
||||
ignored; this validation sharing does not introduce new codecs or payload versions.
|
||||
sizes, scene ranges and nonfinite parallax. Audio reserved bytes must be zero;
|
||||
audio codecs and payload versions are defined in the audio bank format.
|
||||
|
||||
### 4.4 Audio banks
|
||||
|
||||
`SOUNDS` uses SBNK and `MUSIC` uses MBNK as defined in
|
||||
[Audio banks and sequences](05a-audio-bank-format.md). Outer codec is NONE;
|
||||
individual samples select PCM16_LE or IMA_ADPCM_WAV4. Complete validation and
|
||||
PCM16 materialization precede commit. Audio `decoded_size` counts canonical
|
||||
metadata/tables plus decoded PCM; actual allocation overhead is separate telemetry.
|
||||
A single MusicBank is self-contained and has no external asset dependencies.
|
||||
Replacement/unload invalidates its previous instance and stops its audio users;
|
||||
last-owner destruction remains outside the callback.
|
||||
|
||||
## 5 Banks and Slots
|
||||
|
||||
The current runtime exposes bank types:
|
||||
|
||||
- `GLYPH`
|
||||
- `SOUNDS`
|
||||
- `SOUNDS` (16 SoundBank slots)
|
||||
- `MUSIC` (16 independent MusicBank slots)
|
||||
- `SCENE`
|
||||
|
||||
Assets are loaded into explicit slots identified by slot index at the public ABI boundary.
|
||||
@@ -305,13 +319,15 @@ BankTelemetry {
|
||||
Rules:
|
||||
|
||||
- the visible contract MUST NOT expose byte-oriented bank occupancy as the canonical summary;
|
||||
- canonical bank names are `GLYPH` and `SOUNDS`;
|
||||
- canonical bank names are `GLYPH`, `SOUNDS`, `MUSIC` and `SCENE`;
|
||||
- detailed occupancy inspection remains host-owned and slot-based through slot references, not bank byte totals;
|
||||
- public runtime specs MUST NOT treat `bank.slot_info` or JSON bank inspection payloads as the long-term guest ABI;
|
||||
- any residual byte accounting MAY exist internally, but it is not part of the visible bank telemetry contract.
|
||||
|
||||
The rationale for this boundary is explained in [Host-Owned Debug Boundaries and Cheap Machine Diagnostics](../../../discussion/lessons/DSC-0012-perf-runtime-introspection-syscalls/LSN-0034-host-owned-debug-boundaries.md).
|
||||
|
||||
Guest bank selectors use GLYPH=0, SOUNDS=1, SCENE=2, MUSIC=3.
|
||||
|
||||
## 6 Load Lifecycle
|
||||
|
||||
The runtime asset manager exposes a staged lifecycle:
|
||||
|
||||
@@ -379,3 +379,7 @@ Native boundaries MUST respect the [Registry typed-value and ownership contract]
|
||||
## Fn Execution Boundary
|
||||
|
||||
Typed Fn callbacks execute as ordinary guest calls on the single VM context. Boot remains physical FunctionId 0; preparation requires an empty capture environment and existing argument checks. FRAME_SYNC preserves context for next-PC resumption; successful last-root RET completes entry and leaves rooted results before threshold GC. No native callback registration, retention or reentry API is introduced; raw HeapRef/ValueSnapshot copies do not establish guest roots or authorize cross-session imports. Syscalls remain non-first-class.
|
||||
|
||||
The `audio` and `music` modules both require the AUDIO capability. Their staged
|
||||
commands, numeric ranges, costs and audio-time semantics are defined in the
|
||||
[audio peripheral](05-audio-peripheral.md) and [bank/ABI tables](05a-audio-bank-format.md).
|
||||
|
||||
@@ -83,6 +83,7 @@ In the current state, no chapter under the active `docs/specs/runtime/` root is
|
||||
| [`03-memory-stack-heap-and-allocation.md`](03-memory-stack-heap-and-allocation.md) | VM memory model | normative |
|
||||
| [`04-gfx-peripheral.md`](04-gfx-peripheral.md) | virtual hardware: graphics | normative |
|
||||
| [`05-audio-peripheral.md`](05-audio-peripheral.md) | virtual hardware: audio | normative |
|
||||
| [`05a-audio-bank-format.md`](05a-audio-bank-format.md) | audio banks, codecs, sequencer and ABI | normative |
|
||||
| [`06-input-peripheral.md`](06-input-peripheral.md) | virtual hardware: input | normative |
|
||||
| [`07-touch-peripheral.md`](07-touch-peripheral.md) | virtual hardware: touch | normative |
|
||||
| [`08-save-memory-and-memcard.md`](08-save-memory-and-memcard.md) | virtual hardware: save memory | normative |
|
||||
|
||||
@@ -1,4 +1,11 @@
|
||||
# Desktop audio callback qualification
|
||||
# Historical desktop audio callback qualification
|
||||
|
||||
These observations and owner counts describe the previous 16-voice engine.
|
||||
They do not qualify the current 32-voice hardware contract. Current capacity,
|
||||
ownership and ABI are defined in the [audio bank contract](../specs/runtime/05a-audio-bank-format.md).
|
||||
The current executable measurement harness is `cargo run --release -p prometeu-drivers --example audio_measure`;
|
||||
physical/allocator tests live in `audio_callback_allocations.rs`.
|
||||
|
||||
|
||||
The desktop callback moves discarded Sample references and command-vector owners
|
||||
into two preallocated return packets. Host service reclaims them outside the data
|
||||
|
||||
Reference in New Issue
Block a user