Add simulator audio support

2026-09-23 19:13:39 -04:00
parent 982bba70b2
commit dcfd4e9bcc
15 changed files with 905 additions and 11 deletions
@@ -0,0 +1,93 @@
+# Simulator audio
+
+The desktop simulator exposes an SDL speaker and microphone through Tactility's
+standard `audio_stream_*` API. On macOS, SDL uses CoreAudio. Audio Settings controls
+the simulator's input/output volume, mute, and enabled state; these controls do not
+change macOS's system volume.
+
+## Running on macOS
+
+Build the simulator in the usual host build environment (with `python`, `lark`,
+and `pyyaml` available, and without `ESP_IDF_VERSION`):
+
+```sh
+cmake -S . -B buildsim
+cmake --build buildsim --target Tactility -j 8
+```
+
+Create a fresh application bundle (the script deliberately refuses to overwrite
+an existing bundle):
+
+```sh
+sh Buildscripts/release-simulator-macos-app.sh buildsim release/Tactility-audio.app
+open release/Tactility-audio.app
+```
+
+Alternatively, run `../buildsim/Tactility/Tactility` with `Data/` as the working
+directory. Both the executable and the application bundle include a microphone
+usage description. macOS asks for microphone access on the first actual recording
+request, not at simulator startup. If denied, enable access in **System Settings
+→ Privacy & Security → Microphone** and relaunch. For command-line launches, macOS
+may attribute the permission to the terminal or launching application.
+
+A Mac mini needs an external input device, such as a USB mic or headset. Without
+an input device, speaker output still works and input is reported unavailable.
+Connect the input before launch for predictable discovery/UI behavior.
+
+## Selecting devices
+
+By default, each stream opens the system's default device. Startup logs list SDL's
+device names. Optional environment variables select an exact name:
+
+```sh
+SIM_AUDIO_OUTPUT="Mac mini Speakers" SIM_AUDIO_INPUT="USB Microphone" \
+  release/Tactility-audio.app/Contents/MacOS/Tactility
+```
+
+- `SIM_AUDIO_OUTPUT`: exact output name, or `none` to disable output.
+- `SIM_AUDIO_INPUT`: exact input name, or `none` to disable input.
+- Unset or empty values use the system default.
+
+Selection is applied when opening a stream. An already-open stream does not
+automatically switch when the system default changes; close/reopen it or relaunch.
+A missing selected device causes an open failure rather than silently selecting a
+different device. Permission/device errors are logged under `SdlAudio`.
+
+## Formats and behavior
+
+- Signed 16-bit PCM; the shared stream module converts app sample rates and channel
+  counts to 48 kHz mono capture / stereo playback. SDL handles host format conversion.
+- One input and one output stream can be open simultaneously.
+- Read/write from a worker task, never the LVGL thread.
+- Audio callbacks use bounded lock-free buffers, with silence on playback underrun
+  and dropped incoming frames on capture overflow.
+- Reads/writes report partial byte counts on timeout, including converted streams.
+- Closing output drains its bounded buffer for up to 250 ms. Closing input does not
+  affect output, and vice versa.
+- No audio mixing or acoustic echo cancellation is added by this backend. Use
+  headphones when testing simultaneous microphone capture and playback.
+
+## Verification
+
+```sh
+cmake --build buildsim --target SimulatorAudioTests -j 8
+ctest --test-dir buildsim/Tests -R SimulatorAudio --output-on-failure
+```
+
+These tests use SDL's dummy and disk backends without requiring microphone access.
+They cover full duplex, 16/44.1/48 kHz app formats, bounded buffering, partial
+timeouts, enable/disable, unavailable inputs, and sample-level input/output gain
+and mute. They also cover disabling audio while a slow codec open is pending,
+as can happen during a microphone permission prompt. Disk fixtures and output
+files are temporary and removed after the run.
+
+To explicitly play a quiet half-second 440 Hz tone through real Mac audio:
+
+```sh
+SDL_AUDIODRIVER=coreaudio SIM_AUDIO_OUTPUT= \
+  buildsim/Tests/simulator/SimulatorAudioTests \
+  --no-skip --test-case="hardware output smoke test"
+```
+
+Physical microphone capture, the first-use permission prompt, and live device
+unplugging require a separate manual check with an input device attached.