fix(bookplayer): keep audio stream open across page turns + fade-in/out
Main / Build (AudioTest) (push) Has been cancelled
Main / Build (BibleVerse) (push) Has been cancelled
Main / Build (BookPlayer) (push) Has been cancelled
Main / Build (Brainfuck) (push) Has been cancelled
Main / Build (Breakout) (push) Has been cancelled
Main / Build (Calculator) (push) Has been cancelled
Main / Build (Diceware) (push) Has been cancelled
Main / Build (EpubReader) (push) Has been cancelled
Main / Build (EspNowBridge) (push) Has been cancelled
Main / Build (GPIO) (push) Has been cancelled
Main / Build (GameBoy) (push) Has been cancelled
Main / Build (GraphicsDemo) (push) Has been cancelled
Main / Build (HelloWorld) (push) Has been cancelled
Main / Build (M5UnitTest) (push) Has been cancelled
Main / Build (Magic8Ball) (push) Has been cancelled
Main / Build (McpScreen) (push) Has been cancelled
Main / Build (MediaKeys) (push) Has been cancelled
Main / Build (Mp3Player) (push) Has been cancelled
Main / Build (MystifyDemo) (push) Has been cancelled
Main / Build (PocketDungeon) (push) Has been cancelled
Main / Build (ReynaBot) (push) Has been cancelled
Main / Build (RobotArm) (push) Has been cancelled
Main / Build (SerialConsole) (push) Has been cancelled
Main / Build (Snake) (push) Has been cancelled
Main / Build (TamaTac) (push) Has been cancelled
Main / Build (TodoList) (push) Has been cancelled
Main / Build (TwoEleven) (push) Has been cancelled
Main / Build (VoiceRecorder) (push) Has been cancelled
Main / Bundle (push) Has been cancelled
Main / PublishApps (push) Has been cancelled
Main / Build (AudioTest) (push) Has been cancelled
Main / Build (BibleVerse) (push) Has been cancelled
Main / Build (BookPlayer) (push) Has been cancelled
Main / Build (Brainfuck) (push) Has been cancelled
Main / Build (Breakout) (push) Has been cancelled
Main / Build (Calculator) (push) Has been cancelled
Main / Build (Diceware) (push) Has been cancelled
Main / Build (EpubReader) (push) Has been cancelled
Main / Build (EspNowBridge) (push) Has been cancelled
Main / Build (GPIO) (push) Has been cancelled
Main / Build (GameBoy) (push) Has been cancelled
Main / Build (GraphicsDemo) (push) Has been cancelled
Main / Build (HelloWorld) (push) Has been cancelled
Main / Build (M5UnitTest) (push) Has been cancelled
Main / Build (Magic8Ball) (push) Has been cancelled
Main / Build (McpScreen) (push) Has been cancelled
Main / Build (MediaKeys) (push) Has been cancelled
Main / Build (Mp3Player) (push) Has been cancelled
Main / Build (MystifyDemo) (push) Has been cancelled
Main / Build (PocketDungeon) (push) Has been cancelled
Main / Build (ReynaBot) (push) Has been cancelled
Main / Build (RobotArm) (push) Has been cancelled
Main / Build (SerialConsole) (push) Has been cancelled
Main / Build (Snake) (push) Has been cancelled
Main / Build (TamaTac) (push) Has been cancelled
Main / Build (TodoList) (push) Has been cancelled
Main / Build (TwoEleven) (push) Has been cancelled
Main / Build (VoiceRecorder) (push) Has been cancelled
Main / Bundle (push) Has been cancelled
Main / PublishApps (push) Has been cancelled
Close and re-creating the codec on every page change caused an audible pop between pages. Now open_output_stream reuses an already-open stream when the format matches, the codec is left open on a natural page finish (only torn down on user stop / back / app close), and a short fade-in/out is applied at each page boundary. Also fixes pre-existing device API renames (device_find_* -> device_get_* out-param style). Docs: record where the Tactility SDK actually lives and that the fresh CDN 0.8.0-dev SDK is broken for this setup (app-module resolution), with the cached working copy workaround.
This commit is contained in:
@@ -9,23 +9,42 @@ Tactility external ELF apps — loaded at runtime via firmware symbol table. Eac
|
||||
Upstream: `https://github.com/TactilityProject/TactilityApps`
|
||||
Personal Gitea mirror: `https://git.reynafamily.com/adolforeyna/tactility_apps` (`personal` remote)
|
||||
|
||||
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
|
||||
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` and `.114` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
|
||||
|
||||
## Local Workstation
|
||||
|
||||
- Host: M2 Air (14,2) macOS 26.5.2
|
||||
- Project: `/Users/adolforeyna/Projects/Tactility/apps`
|
||||
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf`, Python env `idf5.3_py3.9_env` at `~/.espressif/python_env/idf5.3_py3.9_env`
|
||||
- Tactility SDK: `/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK` (built locally, `--local-sdk`)
|
||||
- Host: macOS 26.5.2
|
||||
- Project: `/Users/adolforeyna/Projects/electronics/tactility/tactility_apps`
|
||||
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf` (v5.5.2), Python env `idf5.5_py3.9_env` at `~/.espressif/python_env/idf5.5_py3.9_env` (NOT `idf5.3_py3.9_env` — that env does not exist on this machine)
|
||||
|
||||
### Tactility SDK location (IMPORTANT)
|
||||
|
||||
- When you run `tactility.py ... build` **without** `--local-sdk`, it auto-downloads the SDK to the per-app cache:
|
||||
`Apps/<Name>/.tactility/<version>-<platform>/TactilitySDK` (e.g. `Apps/BookPlayer/.tactility/0.8.0-dev-esp32s3/TactilitySDK`).
|
||||
- **The freshly CDN-downloaded 0.8.0-dev SDK for esp32s3 is broken for this setup** — it fails at CMake with
|
||||
`Failed to resolve component 'app-module' required by component 'TactilitySDK': unknown name` and also renames the
|
||||
device API (`device_find_*` → `device_get_by_name/device_get_first_by_type` out-param style). It is a *different,
|
||||
newer* SDK than the working one.
|
||||
- **The known-good SDK is the locally-cached copy** already present in the other apps'
|
||||
`.tactility/0.8.0-dev-esp32s3/TactilitySDK` (the one that carries a `Drivers/` folder). It builds clean.
|
||||
- Workaround when a fresh download fails: copy the working cached SDK over the broken one:
|
||||
```bash
|
||||
SRC=Apps/PipecatVoice/.tactility/0.8.0-dev-esp32s3 # known-good copy
|
||||
DST=Apps/MyApp/.tactility/0.8.0-dev-esp32s3
|
||||
rm -rf "$DST/TactilitySDK"; cp -R "$SRC/TactilitySDK" "$DST/"
|
||||
```
|
||||
Then rebuild. (Resolving the root cause on the newer SDK is still TODO — consider pinning/downloading the cached zip
|
||||
so this isn't needed per-app.)
|
||||
|
||||
### Hardware
|
||||
|
||||
| Board | IP | Port | Notes |
|
||||
|-------|-----|------|-------|
|
||||
| ES3C28P 2.8" color 240x320 | `192.168.68.112` | `/dev/cu.usbmodem101` | Primary deploy target, 16MB flash, OCT PSRAM, SD for ELF assets |
|
||||
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111`? .1101 USB | `/dev/cu.usbmodem1101` | Secondary, compact density, mono threshold 60, font 18 |
|
||||
| ES3C28P 2.8" color 240x320 | `192.168.68.114` | `/dev/cu.usbmodem101` | Secondary S3 (used for recent BookPlayer deploys) |
|
||||
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111` | `/dev/cu.usbmodem1101` | Secondary, compact density, SD threshold 60, font 18 |
|
||||
|
||||
User shorthand: "ending in 112" → `192.168.68.112`.
|
||||
User shorthand: "ending in 112" → `192.168.68.112`. Install is dashboard-port API (80), not dev port 6666.
|
||||
|
||||
## Build, Deploy, Run — Correct Env Wrapper
|
||||
|
||||
@@ -34,11 +53,10 @@ Hermes desktop Python 3.11 venv pollutes `PYTHONPATH` → `tactility.py` crashes
|
||||
Always:
|
||||
|
||||
```bash
|
||||
cd /Users/adolforeyna/Projects/Tactility/apps
|
||||
cd /Users/adolforeyna/Projects/electronics/tactility/tactility_apps
|
||||
unset PYTHONPATH; unset PYTHONHOME
|
||||
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.3_py3.9_env
|
||||
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
|
||||
source /Users/adolforeyna/esp/esp-idf/export.sh
|
||||
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK
|
||||
|
||||
# Build
|
||||
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp clean
|
||||
|
||||
@@ -32,6 +32,10 @@
|
||||
#define MAX_TITLE 128
|
||||
#define MAX_AUTHOR 128
|
||||
|
||||
// Fade-in/out applied at the start/end of each page's narration to avoid an
|
||||
// amplitude step (audible "pop") at page boundaries. In frames per channel.
|
||||
#define FADE_FRAMES 4096
|
||||
|
||||
typedef enum {
|
||||
STATE_IDLE,
|
||||
STATE_PLAYING,
|
||||
@@ -83,6 +87,12 @@ typedef struct {
|
||||
uint8_t* audio_buf; // Shared MP3 input and WAV buffer
|
||||
mp3d_sample_t* pcm_buf; // MP3 decoded pcm buffer
|
||||
|
||||
// Currently-open output stream format (so we can reuse it across page
|
||||
// changes instead of tearing the codec down/recreating it every page).
|
||||
uint32_t stream_rate;
|
||||
uint8_t stream_channels;
|
||||
uint8_t stream_bits;
|
||||
|
||||
TaskHandle_t playback_task_handle;
|
||||
} AppCtx;
|
||||
|
||||
@@ -116,14 +126,13 @@ static void scan_books(AppCtx* ctx);
|
||||
|
||||
/* ─── Audio-stream helpers ─── */
|
||||
static bool find_audio_stream_device(AppCtx* ctx) {
|
||||
// Prefer device_find_first_by_type but keep compatibility with name lookup
|
||||
struct Device* dev = device_find_by_name("audio-stream");
|
||||
if (dev) {
|
||||
struct Device* dev = NULL;
|
||||
if (device_get_by_name("audio-stream", &dev) == ERROR_NONE && dev) {
|
||||
ctx->stream_dev = dev;
|
||||
return true;
|
||||
}
|
||||
dev = device_find_first_by_type(&AUDIO_STREAM_TYPE);
|
||||
if (dev) {
|
||||
dev = NULL;
|
||||
if (device_get_first_by_type(&AUDIO_STREAM_TYPE, &dev) == ERROR_NONE && dev) {
|
||||
ctx->stream_dev = dev;
|
||||
return true;
|
||||
}
|
||||
@@ -134,10 +143,20 @@ static void close_stream_if_open(AppCtx* ctx) {
|
||||
if (ctx->stream_handle) {
|
||||
audio_stream_close(ctx->stream_handle);
|
||||
ctx->stream_handle = NULL;
|
||||
ctx->stream_rate = 0;
|
||||
ctx->stream_channels = 0;
|
||||
ctx->stream_bits = 0;
|
||||
}
|
||||
}
|
||||
|
||||
static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channels, uint8_t bits) {
|
||||
// Reuse the already-open stream when the format hasn't changed. This keeps the
|
||||
// codec alive across page changes, avoiding the audible pop caused by closing and
|
||||
// re-initialising the audio hardware on every page turn.
|
||||
if (ctx->stream_handle && ctx->stream_rate == sample_rate
|
||||
&& ctx->stream_channels == channels && ctx->stream_bits == bits) {
|
||||
return true;
|
||||
}
|
||||
close_stream_if_open(ctx);
|
||||
if (!ctx->stream_dev) return false;
|
||||
struct AudioStreamConfig cfg = {
|
||||
@@ -151,6 +170,9 @@ static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channe
|
||||
ctx->stream_handle = NULL;
|
||||
return false;
|
||||
}
|
||||
ctx->stream_rate = sample_rate;
|
||||
ctx->stream_channels = channels;
|
||||
ctx->stream_bits = bits;
|
||||
ESP_LOGI(TAG, "audio_stream output opened: %u Hz %u ch %u-bit", (unsigned)sample_rate, channels, bits);
|
||||
return true;
|
||||
}
|
||||
@@ -322,6 +344,7 @@ static void load_page(AppCtx* ctx, int page_index, bool start_audio) {
|
||||
/* ─── Return to Book Picker Screen ─── */
|
||||
static void return_to_picker(AppCtx* ctx) {
|
||||
wait_for_playback_task_to_exit(ctx);
|
||||
close_stream_if_open(ctx);
|
||||
|
||||
if (ctx->manifest_root) {
|
||||
cJSON_Delete(ctx->manifest_root);
|
||||
@@ -388,6 +411,7 @@ static void play_mp3(AppCtx* ctx) {
|
||||
bool eof = false;
|
||||
int sample_rate = 0;
|
||||
int channels = 0;
|
||||
size_t page_frames_written = 0;
|
||||
|
||||
ESP_LOGI(TAG, "Starting MP3 playback via audio-stream: %s (%d bytes)", ctx->current_audio_path, file_size);
|
||||
|
||||
@@ -442,14 +466,30 @@ static void play_mp3(AppCtx* ctx) {
|
||||
channels = info.channels;
|
||||
}
|
||||
|
||||
// Adjust Volume
|
||||
// Adjust Volume and apply a fade-in/out at the start/end of the page
|
||||
// so the transition to the next page doesn't click.
|
||||
int vol = ctx->volume;
|
||||
int16_t* samples_ptr = (int16_t*)ctx->pcm_buf;
|
||||
size_t sample_count = (size_t)samples * info.channels;
|
||||
uint8_t ch = (uint8_t)info.channels;
|
||||
size_t bytes_per_frame = (size_t)ch * sizeof(int16_t);
|
||||
size_t frames_in_chunk = (size_t)samples;
|
||||
float remaining_frames = (file_size > bytes_read_total)
|
||||
? (float)(file_size - bytes_read_total) / (float)bytes_per_frame : 0.0f;
|
||||
for (size_t i = 0; i < sample_count; ++i) {
|
||||
size_t frame = page_frames_written + i / ch;
|
||||
float fade_in = (frame < FADE_FRAMES) ? (float)frame / (float)FADE_FRAMES : 1.0f;
|
||||
float fade_out = 1.0f;
|
||||
float remaining = remaining_frames - (float)(i / ch);
|
||||
if (remaining < (float)FADE_FRAMES) {
|
||||
fade_out = (remaining > 0.0f) ? remaining / (float)FADE_FRAMES : 0.0f;
|
||||
}
|
||||
float gain = fade_in * fade_out;
|
||||
int32_t scaled = (int32_t)samples_ptr[i] * vol / 100;
|
||||
scaled = (int32_t)((float)scaled * gain);
|
||||
samples_ptr[i] = (int16_t)scaled;
|
||||
}
|
||||
page_frames_written += frames_in_chunk;
|
||||
|
||||
// Write via audio_stream (resampled to native 44100 internally)
|
||||
size_t offset = 0;
|
||||
@@ -485,10 +525,16 @@ static void play_mp3(AppCtx* ctx) {
|
||||
}
|
||||
|
||||
fclose(file);
|
||||
close_stream_if_open(ctx);
|
||||
|
||||
bool stopped_externally = (ctx->state == STATE_IDLE);
|
||||
|
||||
if (stopped_externally) {
|
||||
// User-initiated stop (prev/next, back, app close): tear the stream down.
|
||||
close_stream_if_open(ctx);
|
||||
}
|
||||
// On a natural page finish we deliberately LEAVE the stream open so the next
|
||||
// page reuses it, instead of closing+recreating the codec (which pops).
|
||||
|
||||
if (!stopped_externally) {
|
||||
tt_lvgl_lock(portMAX_DELAY);
|
||||
handle_audio_finished(ctx);
|
||||
@@ -567,6 +613,7 @@ static void play_wav(AppCtx* ctx) {
|
||||
ESP_LOGI(TAG, "Starting WAV via audio-stream: %s (%u Hz, %u ch)", ctx->current_audio_path, (unsigned int)header.sample_rate, (unsigned int)header.channels);
|
||||
|
||||
size_t total_played = 0;
|
||||
size_t page_frames_written = 0;
|
||||
bool stream_open = true;
|
||||
|
||||
while (total_played < data_size && ctx->state != STATE_IDLE) {
|
||||
@@ -588,14 +635,29 @@ static void play_wav(AppCtx* ctx) {
|
||||
size_t read_bytes = fread(ctx->audio_buf, 1, to_read, file);
|
||||
if (read_bytes == 0) break;
|
||||
|
||||
// Scaling Volume
|
||||
// Scaling Volume + fade-in/out at page boundaries to avoid a click.
|
||||
int vol = ctx->volume;
|
||||
int16_t* samples_ptr = (int16_t*)ctx->audio_buf;
|
||||
size_t sample_count = read_bytes / sizeof(int16_t);
|
||||
uint8_t ch = (header.channels > 0) ? header.channels : 1;
|
||||
size_t bytes_per_frame = (size_t)ch * sizeof(int16_t);
|
||||
size_t frames_in_chunk = read_bytes / bytes_per_frame;
|
||||
float remaining_frames = (data_size > total_played)
|
||||
? (float)(data_size - total_played) / (float)bytes_per_frame : 0.0f;
|
||||
for (size_t i = 0; i < sample_count; ++i) {
|
||||
size_t frame = page_frames_written + i / ch;
|
||||
float fade_in = (frame < FADE_FRAMES) ? (float)frame / (float)FADE_FRAMES : 1.0f;
|
||||
float fade_out = 1.0f;
|
||||
float remaining = remaining_frames - (float)(i / ch);
|
||||
if (remaining < (float)FADE_FRAMES) {
|
||||
fade_out = (remaining > 0.0f) ? remaining / (float)FADE_FRAMES : 0.0f;
|
||||
}
|
||||
float gain = fade_in * fade_out;
|
||||
int32_t scaled = (int32_t)samples_ptr[i] * vol / 100;
|
||||
scaled = (int32_t)((float)scaled * gain);
|
||||
samples_ptr[i] = (int16_t)scaled;
|
||||
}
|
||||
page_frames_written += frames_in_chunk;
|
||||
|
||||
// Write via audio_stream
|
||||
size_t offset = 0;
|
||||
@@ -629,10 +691,16 @@ static void play_wav(AppCtx* ctx) {
|
||||
}
|
||||
|
||||
fclose(file);
|
||||
close_stream_if_open(ctx);
|
||||
|
||||
bool stopped_externally = (ctx->state == STATE_IDLE);
|
||||
|
||||
if (stopped_externally) {
|
||||
// User-initiated stop (prev/next, back, app close): tear the stream down.
|
||||
close_stream_if_open(ctx);
|
||||
}
|
||||
// On a natural page finish we deliberately LEAVE the stream open so the next
|
||||
// page reuses it, instead of closing+recreating the codec (which pops).
|
||||
|
||||
if (!stopped_externally) {
|
||||
tt_lvgl_lock(portMAX_DELAY);
|
||||
handle_audio_finished(ctx);
|
||||
|
||||
Reference in New Issue
Block a user