# Audio bank and sequence format Domain: virtual hardware / cartridge assets. Function: normative. ## Banks and materialization `SOUNDS` and `MUSIC` each have 16 independent resident slots. Loading is explicit. The outer asset codec is `NONE`; sample descriptors select the audio codec. The runtime validates and materializes the entire bank before publishing it. Every sample is immutable interleaved PCM16 in resident memory. Glyph, palette, and scene payload formats are unchanged. There is no legacy audio payload path. All integers below are little endian. Offsets are relative to the bank start. Reserved bytes must be zero. Counts, lengths, arithmetic, UTF-8, enums, references, and nonoverlapping payload bounds are validated before allocation/publication. `decoded_size` is the canonical metadata/table bytes plus decoded PCM bytes; actual resident allocation overhead is reported separately. No per-bank byte cap is imposed; allocation failure is an operational load error. ## Header (48 bytes) | Offset | Type | Meaning | |---:|---|---| | 0 | bytes[4] | `SBNK` or `MBNK` | | 4 | u16 | version = 1 | | 6 | u16 | flags = 0 | | 8 | u32 | sample count | | 12 | u32 | instrument-zone count | | 16 | u32 | event count | | 20 | u32 | PPQ = 960 | | 24 | u32 | initial microseconds per quarter note, positive | | 28 | u32 | song duration in ticks | | 32 | u32 | song loop start tick, or 0xffffffff | | 36 | u32 | track count (0..256) | | 40 | u32 | UTF-8 song/bank name length | | 44 | u32 | reserved | The name follows the header, then sample descriptors, instrument zones, events, and sample payloads, in that order. A SoundBank has at least one sample and no instruments, tracks, events, duration, or song loop. A MusicBank contains exactly one composition; all references are bank-local. Duration must be positive, loop start must precede duration, and the compiled loop must span at least one output frame. Events are sorted by tick; array order breaks equal-time ties. ## Sample descriptor (40 bytes) | Offset | Type | Meaning | |---:|---|---| | 0 | u8 | codec: 0 = PCM16_LE, 1 = IMA_ADPCM_WAV4 | | 1 | u8 | channels: 1 or 2 | | 2 | u16 | flags: bit 0 means loop bounds present | | 4 | u32 | source sample rate, positive | | 8 | u32 | decoded frames per channel | | 12 | u32 | loop start frame | | 16 | u32 | exclusive loop end frame | | 20 | u32 | payload offset | | 24 | u32 | payload bytes | | 28 | u16 | IMA block alignment in bytes; zero for PCM | | 30 | u16 | IMA decoded frames per block; zero for PCM | | 32 | u8 | default volume (0..255) | | 33 | u8 | default pan/balance (0..255, center 128) | | 34 | u16 | reserved | | 36 | u32 | default rate multiplier, unsigned Q16.16, positive | PCM payload size is exactly frames * channels * 2. Loops use decoded frames and must satisfy start < end <= frames. Without a loop flag both bounds are zero. Zero-length nonlooped samples are inert. No implicit downmix is performed. IMA_ADPCM_WAV4 uses the WAVE IMA four-bit block layout, not RTP DVI4. Each block starts with a four-byte header per channel: initial PCM i16, step index u8 in 0..88, and reserved zero. The initial PCM frame is emitted. Codes are decoded low nibble first; stereo alternates four code bytes (eight frames) for left and four for right. Each channel has independent predictor/index state. Predictor is saturated to i16 and index to 0..88 after each code. Block alignment is a positive multiple of 4*channels, at least 8*channels. Frames/block equals 1 + (block_alignment - 4*channels)*2/channels. Every stored block is complete; only the final block may contain padding beyond the declared decoded frame count. Its unused codes are zero. The packer defaults to 256 bytes per channel (505 frames/block); loops need not align to blocks because playback uses decoded PCM. The codec is lossy and selected explicitly for each sample. Encoder/decoder code must have documented project-compatible provenance and independent conformance vectors. Runtime does not parse a WAV/RIFF container. ## Instrument zone (64 bytes) | Offset | Type | Meaning | |---:|---|---| | 0 | u16 | instrument ID | | 2 | u16 | sample index | | 4 | u8 | lowest note (0..127) | | 5 | u8 | highest note inclusive | | 6 | u8 | sample root note | | 7 | u8 | flags: bit 0 enables looping | | 8 | i16 | fine tuning in cents | | 10 | u8 | pan/balance | | 11 | u8 | gain | | 12 | u32 | attack duration in microseconds | | 16 | u32 | decay duration in microseconds | | 20 | u16 | sustain gain (0..65535) | | 22 | u16 | reserved | | 24 | u32 | release duration in microseconds | | 28 | u32 | vibrato frequency in millihertz | | 32 | u16 | vibrato depth in cents | | 34 | u16 | tremolo depth (0..65535) | | 36 | u32 | tremolo frequency in millihertz | | 40 | u16 | default delay send (0..65535) | | 42 | u8 | curve: 0 = all linear, 1 = linear attack/exponential decay/release | | 43 | bytes[21] | reserved | Zones of the same instrument may not overlap. Note-on resolves exactly one zone. ADSR stages reach exact endpoints within their declared output-frame duration; zero duration transitions immediately. Sustain is constant. Note-off releases from the current gain. Exponential segments use the normalized curve (exp(-6*x)-exp(-6))/(1-exp(-6)), x in [0,1], with exact endpoints; table interpolation is permitted within 1e-4 absolute gain error. ## Event (32 bytes) The record contains tick:u32, kind:u8, track:u8, flags:u16, a:u32, b:u32, c:i32, d:i32, duration_frames:u32, curve:u32. Curve is 0 linear or 1 smoothstep. Durations run on audio time, independently of later musical tempo changes. | Kind | Operation | Operands | |---:|---|---| | 1 | note-on | a note identity, b instrument, c note, d velocity 0..255; flag 1 legato | | 2 | note-off | a note identity | | 3 | tempo | a positive microseconds/quarter; global | | 4 | track gain ramp | c destination 0..65535 | | 5 | track pan ramp | c destination 0..255 | | 6 | track pitch ramp | c bend in cents | | 7 | vibrato | a millihertz, c depth cents | | 8 | tremolo | a millihertz, c depth 0..65535 | | 9 | track delay send ramp | c destination 0..65535 | | 10 | instrument selection | a instrument ID; note-on b=0xffffffff inherits it | Note-on duration_frames is portamento duration when legato is set. Compatible legato retains cursor/envelope on the most recent active note of that track; incompatible instrument/zone/sample retriggers. It never creates a hidden voice. Stealing selects the lowest free local index, oldest release, then oldest note; Release age starts at note-off; note age starts at note-on. Serialized event order resolves same-frame ages, with local index as final tie-breaker. Obsolete note-off cannot affect a reused voice. Authored note identities must distinguish simultaneously live logical notes of a track. Events and tempo segments compile into exact rational frame times using u128 checked arithmetic and denominator PPQ*1000000. Events execute at the first integer output frame not earlier than their represented time. Carry fractional remainder across tempo changes and song loops. At a loop boundary, notes from the previous iteration enter release and identities are renewed. A frame may execute at most 64 events per group; validate this bound, including the loop seam, during loading. Reject zero-time loops and malformed references. ## Processing, effects, and lifecycle Three fixed groups own 12/12/8 voices. A stereo sample consumes one voice. Mono pan uses equal-power gains; stereo balance preserves L/R at center and attenuates only the opposite side toward an extreme. Mono normalized pan is p/256 for p<=128 and 0.5+(p-128)/254 above center; gains are sqrt(1-pan) and sqrt(pan). Stereo gains are [1,p/128] left of center and [(255-p)/127,1] right of center. Gain and pitch changes do not reset a cursor. Sum in floating point and clamp only at final output. Each group has a shared low-pass and stereo delay; selective sends contribute to that group's delay only. Group gain follows dry/wet recombination. Music A/B feed Music Bus; SFX feeds SFX Bus; bus and master gains are independent. Each track initially uses each instrument's default send. A track send event sets an absolute destination: interpolate from each voice's current send to that destination, including interrupted transitions. New notes resolve their instrument default through the same transition. SFX send defaults to zero on new playback and is explicitly set with audio.set_send. The filter is a one-pole low-pass: y += a*(x-y), a=1-exp(-2*pi*cutoff/48000). Cutoff zero bypasses filtering. Delay receives unfiltered selective sends; feedback is local to the delay. Output is dry*filtered_input + wet*delayed_input. Changing delay length clears history in O(1); no per-voice filter or delay is allocated. Linear interpolation is used for PCM. Vibrato/tremolo use deterministic LFO phase on audio time; gain ramps support linear and smoothstep curves. Delay storage is prepared for 2000 ms per group at 48 kHz, stereo f32: 2304000 bytes total, independent of bank size. This is an explicit bounded processing budget, not a measured battery/ARM guarantee. Feedback must remain strictly inside (-1,1) after coefficient conversion. Reset invalidates history in O(1), without clearing all buffer bytes in the callback. Pause freezes group DSP and musical time. Music unload/replacement clears its group's history; stopping SFX of one bank allows already mixed shared delay history to finish naturally. Resident banks carry invalidation tokens. Installing/unloading a slot invalidates its old instance; retained commands/voices check that token before generating more PCM. Failure to prepare a replacement leaves the old instance valid. Asset commit remains an owner-side frame-boundary operation, independent of audio consumption. No command may resurrect an invalid bank, including after pause. Samples, bank owners, command vectors and invalidation tokens are reclaimed by host service outside the callback; the callback never blocks or destroys a last owner. Already submitted device buffers are not recalled. Retirement uses two preallocated packets. Each has capacity 2*(3*1024+32+2) resource owners plus two command-vector owners. A staged play retains at most a sample and invalidation token; music owns the whole immutable bank, so note references can be released while that bank is still retained. Retire the bank only after releasing its note references. No callback operation can grow the packet; packet starvation retains inactive owners until service. The existing 1024 frame/callback quotas, independent lifecycle slot, FIFO prefix publication, aborted-frame cancellation, session invalidation and bounded backpressure remain in force. Musical events do not consume VM frame quota. ## High-level ABI additions All operations require AUDIO, return one status and cost five logical cycles. Existing audio play/control calls address only SFX local voices 0..7. Music groups are 0=A and 1=B; DSP groups are 0=A,1=B,2=SFX; buses are 0=Music,1=SFX,2=Master. | ID | Operation | Arguments | |---|---|---| | 0x3010 | music.play | group, resident bank slot, loop 0/1 | | 0x3011 | music.stop | group | | 0x3012 | music.pause | group | | 0x3013 | music.resume | group | | 0x3014 | music.set_volume | group, gain 0..255 | | 0x3015 | music.fade_to | group, gain 0..255, milliseconds, curve 0/1 | | 0x3016 | music.crossfade | source group, target group, milliseconds, curve 0/1 | | 0x3020 | audio.set_bus_volume | bus, gain 0..255 | | 0x3021 | audio.set_effect | group, cutoff Hz (0=bypass), delay ms 0..2000, feedback number, wet number 0..1, dry number 0..1 | | 0x3022 | audio.set_send | SFX voice, gain 0..255 | | 0x3023 | audio.unload_bank | type 0=SoundBank/1=MusicBank, slot | | 0x3024 | audio.set_group_volume | group 0/1/2, gain 0..255 (independent of bus gain) | Group/slot/voice errors and nonfinite/out-of-range arguments reject before admission without consuming quota. No command returns a playback handle or synchronously waits for the consumer. Crossfade schedules both ramps and stops the source at completion. If either group is inactive when the consumer applies the crossfade, it has no effect; admission success is not a consumer acknowledgement. Fade alone to zero keeps sequence time advancing. Unload is an independent bounded owner control, so an exhausted guest audio command quota cannot prevent invalidation. It returns AudioOpStatus, mapping unavailable owner control to BANK_INVALID. It is an asset-owner action: a later VM frame fault does not restore an unloaded bank. Music play/stop/replacement preserve the current group gain and cancel its old fade, so setting gain before play is effective. DSP configuration belongs to the group and persists across bank changes; its history follows the reset policy above. Session/epoch reset restores gains, buses and DSP configuration to defaults. ## Codec references and diagnostics The IMA WAVE block header includes the first uncompressed sample and uses low nibble first. RTP DVI4 has a different framing/order; the distinction is documented in [RFC 3551 section 4.5.1](https://www.rfc-editor.org/rfc/rfc3551#section-4.5.1). The implementation is project-owned code under the project MIT license; no third-party decoder or additional codec library is linked. PCM WAV source layout follows [WAVEFORMATEX](https://learn.microsoft.com/en-us/windows/win32/api/mmreg/ns-mmreg-waveformatex). Only the packer reads WAV containers. Owner-side bank diagnostics expose cartridge bytes, canonical decoded bytes, PCM length/allocation bytes per resident bank, and unique PCM totals across slots. Accounted metadata/total includes resident structs, strings and vector capacities, excluding Arc/allocator headers and asset-manager caches; allocator peak observations are separate actual allocation measurements. Materialization elapsed time includes cooperative yields; pipeline load latency is separately recorded by bank type. Mixer diagnostics expose active and maximum voices, frames and render time; the service lane measures retained PCM separately from resident slots. Retained PCM can overlap resident PCM; those reference counts must not be summed as additional memory. These are host diagnostics, not new guest byte-budget quotas. The measurement executable includes allocator peak tracking for replacement/load observations. Prepared engine memory includes fixed DSP capacity; transport, retirement packets and bank memory are additional. Integer event timing and codec PCM output are deterministic. Floating-point mixing/DSP replay is tested with identical buffer partitions on a host; bitwise identity across different CPU/libm implementations is not qualified here.