fix(bookplayer): keep audio stream open across page turns + fade-in/out
Main / Build (AudioTest) (push) Has been cancelled
Main / Build (BibleVerse) (push) Has been cancelled
Main / Build (BookPlayer) (push) Has been cancelled
Main / Build (Brainfuck) (push) Has been cancelled
Main / Build (Breakout) (push) Has been cancelled
Main / Build (Calculator) (push) Has been cancelled
Main / Build (Diceware) (push) Has been cancelled
Main / Build (EpubReader) (push) Has been cancelled
Main / Build (EspNowBridge) (push) Has been cancelled
Main / Build (GPIO) (push) Has been cancelled
Main / Build (GameBoy) (push) Has been cancelled
Main / Build (GraphicsDemo) (push) Has been cancelled
Main / Build (HelloWorld) (push) Has been cancelled
Main / Build (M5UnitTest) (push) Has been cancelled
Main / Build (Magic8Ball) (push) Has been cancelled
Main / Build (McpScreen) (push) Has been cancelled
Main / Build (MediaKeys) (push) Has been cancelled
Main / Build (Mp3Player) (push) Has been cancelled
Main / Build (MystifyDemo) (push) Has been cancelled
Main / Build (PocketDungeon) (push) Has been cancelled
Main / Build (ReynaBot) (push) Has been cancelled
Main / Build (RobotArm) (push) Has been cancelled
Main / Build (SerialConsole) (push) Has been cancelled
Main / Build (Snake) (push) Has been cancelled
Main / Build (TamaTac) (push) Has been cancelled
Main / Build (TodoList) (push) Has been cancelled
Main / Build (TwoEleven) (push) Has been cancelled
Main / Build (VoiceRecorder) (push) Has been cancelled
Main / Bundle (push) Has been cancelled
Main / PublishApps (push) Has been cancelled

Close and re-creating the codec on every page change caused an audible pop
between pages. Now open_output_stream reuses an already-open stream when the
format matches, the codec is left open on a natural page finish (only torn
down on user stop / back / app close), and a short fade-in/out is applied at
each page boundary. Also fixes pre-existing device API renames (device_find_*
-> device_get_* out-param style).

Docs: record where the Tactility SDK actually lives and that the fresh CDN
0.8.0-dev SDK is broken for this setup (app-module resolution), with the
cached working copy workaround.
This commit is contained in:
Adolfo Reyna
2026-08-19 21:09:21 -04:00
parent 6e83272c8e
commit ef9d4e8de9
2 changed files with 105 additions and 19 deletions
+28 -10
View File
@@ -9,23 +9,42 @@ Tactility external ELF apps — loaded at runtime via firmware symbol table. Eac
Upstream: `https://github.com/TactilityProject/TactilityApps` Upstream: `https://github.com/TactilityProject/TactilityApps`
Personal Gitea mirror: `https://git.reynafamily.com/adolforeyna/tactility_apps` (`personal` remote) Personal Gitea mirror: `https://git.reynafamily.com/adolforeyna/tactility_apps` (`personal` remote)
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns. Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` and `.114` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
## Local Workstation ## Local Workstation
- Host: M2 Air (14,2) macOS 26.5.2 - Host: macOS 26.5.2
- Project: `/Users/adolforeyna/Projects/Tactility/apps` - Project: `/Users/adolforeyna/Projects/electronics/tactility/tactility_apps`
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf`, Python env `idf5.3_py3.9_env` at `~/.espressif/python_env/idf5.3_py3.9_env` - ESP-IDF: `/Users/adolforeyna/esp/esp-idf` (v5.5.2), Python env `idf5.5_py3.9_env` at `~/.espressif/python_env/idf5.5_py3.9_env` (NOT `idf5.3_py3.9_env` — that env does not exist on this machine)
- Tactility SDK: `/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK` (built locally, `--local-sdk`)
### Tactility SDK location (IMPORTANT)
- When you run `tactility.py ... build` **without** `--local-sdk`, it auto-downloads the SDK to the per-app cache:
`Apps/<Name>/.tactility/<version>-<platform>/TactilitySDK` (e.g. `Apps/BookPlayer/.tactility/0.8.0-dev-esp32s3/TactilitySDK`).
- **The freshly CDN-downloaded 0.8.0-dev SDK for esp32s3 is broken for this setup** — it fails at CMake with
`Failed to resolve component 'app-module' required by component 'TactilitySDK': unknown name` and also renames the
device API (`device_find_*` → `device_get_by_name/device_get_first_by_type` out-param style). It is a *different,
newer* SDK than the working one.
- **The known-good SDK is the locally-cached copy** already present in the other apps'
`.tactility/0.8.0-dev-esp32s3/TactilitySDK` (the one that carries a `Drivers/` folder). It builds clean.
- Workaround when a fresh download fails: copy the working cached SDK over the broken one:
```bash
SRC=Apps/PipecatVoice/.tactility/0.8.0-dev-esp32s3 # known-good copy
DST=Apps/MyApp/.tactility/0.8.0-dev-esp32s3
rm -rf "$DST/TactilitySDK"; cp -R "$SRC/TactilitySDK" "$DST/"
```
Then rebuild. (Resolving the root cause on the newer SDK is still TODO — consider pinning/downloading the cached zip
so this isn't needed per-app.)
### Hardware ### Hardware
| Board | IP | Port | Notes | | Board | IP | Port | Notes |
|-------|-----|------|-------| |-------|-----|------|-------|
| ES3C28P 2.8" color 240x320 | `192.168.68.112` | `/dev/cu.usbmodem101` | Primary deploy target, 16MB flash, OCT PSRAM, SD for ELF assets | | ES3C28P 2.8" color 240x320 | `192.168.68.112` | `/dev/cu.usbmodem101` | Primary deploy target, 16MB flash, OCT PSRAM, SD for ELF assets |
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111`? .1101 USB | `/dev/cu.usbmodem1101` | Secondary, compact density, mono threshold 60, font 18 | | ES3C28P 2.8" color 240x320 | `192.168.68.114` | `/dev/cu.usbmodem101` | Secondary S3 (used for recent BookPlayer deploys) |
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111` | `/dev/cu.usbmodem1101` | Secondary, compact density, SD threshold 60, font 18 |
User shorthand: "ending in 112" → `192.168.68.112`. User shorthand: "ending in 112" → `192.168.68.112`. Install is dashboard-port API (80), not dev port 6666.
## Build, Deploy, Run — Correct Env Wrapper ## Build, Deploy, Run — Correct Env Wrapper
@@ -34,11 +53,10 @@ Hermes desktop Python 3.11 venv pollutes `PYTHONPATH` → `tactility.py` crashes
Always: Always:
```bash ```bash
cd /Users/adolforeyna/Projects/Tactility/apps cd /Users/adolforeyna/Projects/electronics/tactility/tactility_apps
unset PYTHONPATH; unset PYTHONHOME unset PYTHONPATH; unset PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.3_py3.9_env export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh source /Users/adolforeyna/esp/esp-idf/export.sh
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK
# Build # Build
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp clean $IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp clean
+77 -9
View File
@@ -32,6 +32,10 @@
#define MAX_TITLE 128 #define MAX_TITLE 128
#define MAX_AUTHOR 128 #define MAX_AUTHOR 128
// Fade-in/out applied at the start/end of each page's narration to avoid an
// amplitude step (audible "pop") at page boundaries. In frames per channel.
#define FADE_FRAMES 4096
typedef enum { typedef enum {
STATE_IDLE, STATE_IDLE,
STATE_PLAYING, STATE_PLAYING,
@@ -83,6 +87,12 @@ typedef struct {
uint8_t* audio_buf; // Shared MP3 input and WAV buffer uint8_t* audio_buf; // Shared MP3 input and WAV buffer
mp3d_sample_t* pcm_buf; // MP3 decoded pcm buffer mp3d_sample_t* pcm_buf; // MP3 decoded pcm buffer
// Currently-open output stream format (so we can reuse it across page
// changes instead of tearing the codec down/recreating it every page).
uint32_t stream_rate;
uint8_t stream_channels;
uint8_t stream_bits;
TaskHandle_t playback_task_handle; TaskHandle_t playback_task_handle;
} AppCtx; } AppCtx;
@@ -116,14 +126,13 @@ static void scan_books(AppCtx* ctx);
/* ─── Audio-stream helpers ─── */ /* ─── Audio-stream helpers ─── */
static bool find_audio_stream_device(AppCtx* ctx) { static bool find_audio_stream_device(AppCtx* ctx) {
// Prefer device_find_first_by_type but keep compatibility with name lookup struct Device* dev = NULL;
struct Device* dev = device_find_by_name("audio-stream"); if (device_get_by_name("audio-stream", &dev) == ERROR_NONE && dev) {
if (dev) {
ctx->stream_dev = dev; ctx->stream_dev = dev;
return true; return true;
} }
dev = device_find_first_by_type(&AUDIO_STREAM_TYPE); dev = NULL;
if (dev) { if (device_get_first_by_type(&AUDIO_STREAM_TYPE, &dev) == ERROR_NONE && dev) {
ctx->stream_dev = dev; ctx->stream_dev = dev;
return true; return true;
} }
@@ -134,10 +143,20 @@ static void close_stream_if_open(AppCtx* ctx) {
if (ctx->stream_handle) { if (ctx->stream_handle) {
audio_stream_close(ctx->stream_handle); audio_stream_close(ctx->stream_handle);
ctx->stream_handle = NULL; ctx->stream_handle = NULL;
ctx->stream_rate = 0;
ctx->stream_channels = 0;
ctx->stream_bits = 0;
} }
} }
static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channels, uint8_t bits) { static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channels, uint8_t bits) {
// Reuse the already-open stream when the format hasn't changed. This keeps the
// codec alive across page changes, avoiding the audible pop caused by closing and
// re-initialising the audio hardware on every page turn.
if (ctx->stream_handle && ctx->stream_rate == sample_rate
&& ctx->stream_channels == channels && ctx->stream_bits == bits) {
return true;
}
close_stream_if_open(ctx); close_stream_if_open(ctx);
if (!ctx->stream_dev) return false; if (!ctx->stream_dev) return false;
struct AudioStreamConfig cfg = { struct AudioStreamConfig cfg = {
@@ -151,6 +170,9 @@ static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channe
ctx->stream_handle = NULL; ctx->stream_handle = NULL;
return false; return false;
} }
ctx->stream_rate = sample_rate;
ctx->stream_channels = channels;
ctx->stream_bits = bits;
ESP_LOGI(TAG, "audio_stream output opened: %u Hz %u ch %u-bit", (unsigned)sample_rate, channels, bits); ESP_LOGI(TAG, "audio_stream output opened: %u Hz %u ch %u-bit", (unsigned)sample_rate, channels, bits);
return true; return true;
} }
@@ -322,6 +344,7 @@ static void load_page(AppCtx* ctx, int page_index, bool start_audio) {
/* ─── Return to Book Picker Screen ─── */ /* ─── Return to Book Picker Screen ─── */
static void return_to_picker(AppCtx* ctx) { static void return_to_picker(AppCtx* ctx) {
wait_for_playback_task_to_exit(ctx); wait_for_playback_task_to_exit(ctx);
close_stream_if_open(ctx);
if (ctx->manifest_root) { if (ctx->manifest_root) {
cJSON_Delete(ctx->manifest_root); cJSON_Delete(ctx->manifest_root);
@@ -388,6 +411,7 @@ static void play_mp3(AppCtx* ctx) {
bool eof = false; bool eof = false;
int sample_rate = 0; int sample_rate = 0;
int channels = 0; int channels = 0;
size_t page_frames_written = 0;
ESP_LOGI(TAG, "Starting MP3 playback via audio-stream: %s (%d bytes)", ctx->current_audio_path, file_size); ESP_LOGI(TAG, "Starting MP3 playback via audio-stream: %s (%d bytes)", ctx->current_audio_path, file_size);
@@ -442,14 +466,30 @@ static void play_mp3(AppCtx* ctx) {
channels = info.channels; channels = info.channels;
} }
// Adjust Volume // Adjust Volume and apply a fade-in/out at the start/end of the page
// so the transition to the next page doesn't click.
int vol = ctx->volume; int vol = ctx->volume;
int16_t* samples_ptr = (int16_t*)ctx->pcm_buf; int16_t* samples_ptr = (int16_t*)ctx->pcm_buf;
size_t sample_count = (size_t)samples * info.channels; size_t sample_count = (size_t)samples * info.channels;
uint8_t ch = (uint8_t)info.channels;
size_t bytes_per_frame = (size_t)ch * sizeof(int16_t);
size_t frames_in_chunk = (size_t)samples;
float remaining_frames = (file_size > bytes_read_total)
? (float)(file_size - bytes_read_total) / (float)bytes_per_frame : 0.0f;
for (size_t i = 0; i < sample_count; ++i) { for (size_t i = 0; i < sample_count; ++i) {
size_t frame = page_frames_written + i / ch;
float fade_in = (frame < FADE_FRAMES) ? (float)frame / (float)FADE_FRAMES : 1.0f;
float fade_out = 1.0f;
float remaining = remaining_frames - (float)(i / ch);
if (remaining < (float)FADE_FRAMES) {
fade_out = (remaining > 0.0f) ? remaining / (float)FADE_FRAMES : 0.0f;
}
float gain = fade_in * fade_out;
int32_t scaled = (int32_t)samples_ptr[i] * vol / 100; int32_t scaled = (int32_t)samples_ptr[i] * vol / 100;
scaled = (int32_t)((float)scaled * gain);
samples_ptr[i] = (int16_t)scaled; samples_ptr[i] = (int16_t)scaled;
} }
page_frames_written += frames_in_chunk;
// Write via audio_stream (resampled to native 44100 internally) // Write via audio_stream (resampled to native 44100 internally)
size_t offset = 0; size_t offset = 0;
@@ -485,10 +525,16 @@ static void play_mp3(AppCtx* ctx) {
} }
fclose(file); fclose(file);
close_stream_if_open(ctx);
bool stopped_externally = (ctx->state == STATE_IDLE); bool stopped_externally = (ctx->state == STATE_IDLE);
if (stopped_externally) {
// User-initiated stop (prev/next, back, app close): tear the stream down.
close_stream_if_open(ctx);
}
// On a natural page finish we deliberately LEAVE the stream open so the next
// page reuses it, instead of closing+recreating the codec (which pops).
if (!stopped_externally) { if (!stopped_externally) {
tt_lvgl_lock(portMAX_DELAY); tt_lvgl_lock(portMAX_DELAY);
handle_audio_finished(ctx); handle_audio_finished(ctx);
@@ -567,6 +613,7 @@ static void play_wav(AppCtx* ctx) {
ESP_LOGI(TAG, "Starting WAV via audio-stream: %s (%u Hz, %u ch)", ctx->current_audio_path, (unsigned int)header.sample_rate, (unsigned int)header.channels); ESP_LOGI(TAG, "Starting WAV via audio-stream: %s (%u Hz, %u ch)", ctx->current_audio_path, (unsigned int)header.sample_rate, (unsigned int)header.channels);
size_t total_played = 0; size_t total_played = 0;
size_t page_frames_written = 0;
bool stream_open = true; bool stream_open = true;
while (total_played < data_size && ctx->state != STATE_IDLE) { while (total_played < data_size && ctx->state != STATE_IDLE) {
@@ -588,14 +635,29 @@ static void play_wav(AppCtx* ctx) {
size_t read_bytes = fread(ctx->audio_buf, 1, to_read, file); size_t read_bytes = fread(ctx->audio_buf, 1, to_read, file);
if (read_bytes == 0) break; if (read_bytes == 0) break;
// Scaling Volume // Scaling Volume + fade-in/out at page boundaries to avoid a click.
int vol = ctx->volume; int vol = ctx->volume;
int16_t* samples_ptr = (int16_t*)ctx->audio_buf; int16_t* samples_ptr = (int16_t*)ctx->audio_buf;
size_t sample_count = read_bytes / sizeof(int16_t); size_t sample_count = read_bytes / sizeof(int16_t);
uint8_t ch = (header.channels > 0) ? header.channels : 1;
size_t bytes_per_frame = (size_t)ch * sizeof(int16_t);
size_t frames_in_chunk = read_bytes / bytes_per_frame;
float remaining_frames = (data_size > total_played)
? (float)(data_size - total_played) / (float)bytes_per_frame : 0.0f;
for (size_t i = 0; i < sample_count; ++i) { for (size_t i = 0; i < sample_count; ++i) {
size_t frame = page_frames_written + i / ch;
float fade_in = (frame < FADE_FRAMES) ? (float)frame / (float)FADE_FRAMES : 1.0f;
float fade_out = 1.0f;
float remaining = remaining_frames - (float)(i / ch);
if (remaining < (float)FADE_FRAMES) {
fade_out = (remaining > 0.0f) ? remaining / (float)FADE_FRAMES : 0.0f;
}
float gain = fade_in * fade_out;
int32_t scaled = (int32_t)samples_ptr[i] * vol / 100; int32_t scaled = (int32_t)samples_ptr[i] * vol / 100;
scaled = (int32_t)((float)scaled * gain);
samples_ptr[i] = (int16_t)scaled; samples_ptr[i] = (int16_t)scaled;
} }
page_frames_written += frames_in_chunk;
// Write via audio_stream // Write via audio_stream
size_t offset = 0; size_t offset = 0;
@@ -629,10 +691,16 @@ static void play_wav(AppCtx* ctx) {
} }
fclose(file); fclose(file);
close_stream_if_open(ctx);
bool stopped_externally = (ctx->state == STATE_IDLE); bool stopped_externally = (ctx->state == STATE_IDLE);
if (stopped_externally) {
// User-initiated stop (prev/next, back, app close): tear the stream down.
close_stream_if_open(ctx);
}
// On a natural page finish we deliberately LEAVE the stream open so the next
// page reuses it, instead of closing+recreating the codec (which pops).
if (!stopped_externally) { if (!stopped_externally) {
tt_lvgl_lock(portMAX_DELAY); tt_lvgl_lock(portMAX_DELAY);
handle_audio_finished(ctx); handle_audio_finished(ctx);