2 Commits

Author SHA1 Message Date
Adolfo Reyna ef9d4e8de9 fix(bookplayer): keep audio stream open across page turns + fade-in/out
Main / Build (AudioTest) (push) Has been cancelled
Main / Build (BibleVerse) (push) Has been cancelled
Main / Build (BookPlayer) (push) Has been cancelled
Main / Build (Brainfuck) (push) Has been cancelled
Main / Build (Breakout) (push) Has been cancelled
Main / Build (Calculator) (push) Has been cancelled
Main / Build (Diceware) (push) Has been cancelled
Main / Build (EpubReader) (push) Has been cancelled
Main / Build (EspNowBridge) (push) Has been cancelled
Main / Build (GPIO) (push) Has been cancelled
Main / Build (GameBoy) (push) Has been cancelled
Main / Build (GraphicsDemo) (push) Has been cancelled
Main / Build (HelloWorld) (push) Has been cancelled
Main / Build (M5UnitTest) (push) Has been cancelled
Main / Build (Magic8Ball) (push) Has been cancelled
Main / Build (McpScreen) (push) Has been cancelled
Main / Build (MediaKeys) (push) Has been cancelled
Main / Build (Mp3Player) (push) Has been cancelled
Main / Build (MystifyDemo) (push) Has been cancelled
Main / Build (PocketDungeon) (push) Has been cancelled
Main / Build (ReynaBot) (push) Has been cancelled
Main / Build (RobotArm) (push) Has been cancelled
Main / Build (SerialConsole) (push) Has been cancelled
Main / Build (Snake) (push) Has been cancelled
Main / Build (TamaTac) (push) Has been cancelled
Main / Build (TodoList) (push) Has been cancelled
Main / Build (TwoEleven) (push) Has been cancelled
Main / Build (VoiceRecorder) (push) Has been cancelled
Main / Bundle (push) Has been cancelled
Main / PublishApps (push) Has been cancelled
Close and re-creating the codec on every page change caused an audible pop
between pages. Now open_output_stream reuses an already-open stream when the
format matches, the codec is left open on a natural page finish (only torn
down on user stop / back / app close), and a short fade-in/out is applied at
each page boundary. Also fixes pre-existing device API renames (device_find_*
-> device_get_* out-param style).

Docs: record where the Tactility SDK actually lives and that the fresh CDN
0.8.0-dev SDK is broken for this setup (app-module resolution), with the
cached working copy workaround.
2026-08-19 21:09:21 -04:00
Adolfo Reyna 6e83272c8e feat: PipecatVoice native Tactility app — voice gateway companion
Native ELF app (sibling to VoiceRecorder/ReynaBot). Mic/speaker via audio-stream +
I2S (16kHz PCM) + websocket transport (lwip, cJSON). Defaults to Hermes WS
ws://192.168.68.102:8642/api/esp32/voice/ws (voice profile) with Pipecat voice
gateway ws://<mac>:7861 second target. Adapted from ReynaBot PTT + events
(ready/listening/transcript/thinking/response_text/audio_start/audio_end/done).
Cloned ReynaBot websocket.c/h as base.
2026-08-06 08:44:08 -04:00
25 changed files with 1261 additions and 3946 deletions
+28 -10
View File
@@ -9,23 +9,42 @@ Tactility external ELF apps — loaded at runtime via firmware symbol table. Eac
Upstream: `https://github.com/TactilityProject/TactilityApps`
Personal Gitea mirror: `https://git.reynafamily.com/adolforeyna/tactility_apps` (`personal` remote)
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` and `.114` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
## Local Workstation
- Host: M2 Air (14,2) macOS 26.5.2
- Project: `/Users/adolforeyna/Projects/Tactility/apps`
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf`, Python env `idf5.3_py3.9_env` at `~/.espressif/python_env/idf5.3_py3.9_env`
- Tactility SDK: `/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK` (built locally, `--local-sdk`)
- Host: macOS 26.5.2
- Project: `/Users/adolforeyna/Projects/electronics/tactility/tactility_apps`
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf` (v5.5.2), Python env `idf5.5_py3.9_env` at `~/.espressif/python_env/idf5.5_py3.9_env` (NOT `idf5.3_py3.9_env` — that env does not exist on this machine)
### Tactility SDK location (IMPORTANT)
- When you run `tactility.py ... build` **without** `--local-sdk`, it auto-downloads the SDK to the per-app cache:
`Apps/<Name>/.tactility/<version>-<platform>/TactilitySDK` (e.g. `Apps/BookPlayer/.tactility/0.8.0-dev-esp32s3/TactilitySDK`).
- **The freshly CDN-downloaded 0.8.0-dev SDK for esp32s3 is broken for this setup** — it fails at CMake with
`Failed to resolve component 'app-module' required by component 'TactilitySDK': unknown name` and also renames the
device API (`device_find_*` → `device_get_by_name/device_get_first_by_type` out-param style). It is a *different,
newer* SDK than the working one.
- **The known-good SDK is the locally-cached copy** already present in the other apps'
`.tactility/0.8.0-dev-esp32s3/TactilitySDK` (the one that carries a `Drivers/` folder). It builds clean.
- Workaround when a fresh download fails: copy the working cached SDK over the broken one:
```bash
SRC=Apps/PipecatVoice/.tactility/0.8.0-dev-esp32s3 # known-good copy
DST=Apps/MyApp/.tactility/0.8.0-dev-esp32s3
rm -rf "$DST/TactilitySDK"; cp -R "$SRC/TactilitySDK" "$DST/"
```
Then rebuild. (Resolving the root cause on the newer SDK is still TODO — consider pinning/downloading the cached zip
so this isn't needed per-app.)
### Hardware
| Board | IP | Port | Notes |
|-------|-----|------|-------|
| ES3C28P 2.8" color 240x320 | `192.168.68.112` | `/dev/cu.usbmodem101` | Primary deploy target, 16MB flash, OCT PSRAM, SD for ELF assets |
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111`? .1101 USB | `/dev/cu.usbmodem1101` | Secondary, compact density, mono threshold 60, font 18 |
| ES3C28P 2.8" color 240x320 | `192.168.68.114` | `/dev/cu.usbmodem101` | Secondary S3 (used for recent BookPlayer deploys) |
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111` | `/dev/cu.usbmodem1101` | Secondary, compact density, SD threshold 60, font 18 |
User shorthand: "ending in 112" → `192.168.68.112`.
User shorthand: "ending in 112" → `192.168.68.112`. Install is dashboard-port API (80), not dev port 6666.
## Build, Deploy, Run — Correct Env Wrapper
@@ -34,11 +53,10 @@ Hermes desktop Python 3.11 venv pollutes `PYTHONPATH` → `tactility.py` crashes
Always:
```bash
cd /Users/adolforeyna/Projects/Tactility/apps
cd /Users/adolforeyna/Projects/electronics/tactility/tactility_apps
unset PYTHONPATH; unset PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.3_py3.9_env
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK
# Build
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp clean
+77 -9
View File
@@ -32,6 +32,10 @@
#define MAX_TITLE 128
#define MAX_AUTHOR 128
// Fade-in/out applied at the start/end of each page's narration to avoid an
// amplitude step (audible "pop") at page boundaries. In frames per channel.
#define FADE_FRAMES 4096
typedef enum {
STATE_IDLE,
STATE_PLAYING,
@@ -83,6 +87,12 @@ typedef struct {
uint8_t* audio_buf; // Shared MP3 input and WAV buffer
mp3d_sample_t* pcm_buf; // MP3 decoded pcm buffer
// Currently-open output stream format (so we can reuse it across page
// changes instead of tearing the codec down/recreating it every page).
uint32_t stream_rate;
uint8_t stream_channels;
uint8_t stream_bits;
TaskHandle_t playback_task_handle;
} AppCtx;
@@ -116,14 +126,13 @@ static void scan_books(AppCtx* ctx);
/* ─── Audio-stream helpers ─── */
static bool find_audio_stream_device(AppCtx* ctx) {
// Prefer device_find_first_by_type but keep compatibility with name lookup
struct Device* dev = device_find_by_name("audio-stream");
if (dev) {
struct Device* dev = NULL;
if (device_get_by_name("audio-stream", &dev) == ERROR_NONE && dev) {
ctx->stream_dev = dev;
return true;
}
dev = device_find_first_by_type(&AUDIO_STREAM_TYPE);
if (dev) {
dev = NULL;
if (device_get_first_by_type(&AUDIO_STREAM_TYPE, &dev) == ERROR_NONE && dev) {
ctx->stream_dev = dev;
return true;
}
@@ -134,10 +143,20 @@ static void close_stream_if_open(AppCtx* ctx) {
if (ctx->stream_handle) {
audio_stream_close(ctx->stream_handle);
ctx->stream_handle = NULL;
ctx->stream_rate = 0;
ctx->stream_channels = 0;
ctx->stream_bits = 0;
}
}
static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channels, uint8_t bits) {
// Reuse the already-open stream when the format hasn't changed. This keeps the
// codec alive across page changes, avoiding the audible pop caused by closing and
// re-initialising the audio hardware on every page turn.
if (ctx->stream_handle && ctx->stream_rate == sample_rate
&& ctx->stream_channels == channels && ctx->stream_bits == bits) {
return true;
}
close_stream_if_open(ctx);
if (!ctx->stream_dev) return false;
struct AudioStreamConfig cfg = {
@@ -151,6 +170,9 @@ static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channe
ctx->stream_handle = NULL;
return false;
}
ctx->stream_rate = sample_rate;
ctx->stream_channels = channels;
ctx->stream_bits = bits;
ESP_LOGI(TAG, "audio_stream output opened: %u Hz %u ch %u-bit", (unsigned)sample_rate, channels, bits);
return true;
}
@@ -322,6 +344,7 @@ static void load_page(AppCtx* ctx, int page_index, bool start_audio) {
/* ─── Return to Book Picker Screen ─── */
static void return_to_picker(AppCtx* ctx) {
wait_for_playback_task_to_exit(ctx);
close_stream_if_open(ctx);
if (ctx->manifest_root) {
cJSON_Delete(ctx->manifest_root);
@@ -388,6 +411,7 @@ static void play_mp3(AppCtx* ctx) {
bool eof = false;
int sample_rate = 0;
int channels = 0;
size_t page_frames_written = 0;
ESP_LOGI(TAG, "Starting MP3 playback via audio-stream: %s (%d bytes)", ctx->current_audio_path, file_size);
@@ -442,14 +466,30 @@ static void play_mp3(AppCtx* ctx) {
channels = info.channels;
}
// Adjust Volume
// Adjust Volume and apply a fade-in/out at the start/end of the page
// so the transition to the next page doesn't click.
int vol = ctx->volume;
int16_t* samples_ptr = (int16_t*)ctx->pcm_buf;
size_t sample_count = (size_t)samples * info.channels;
uint8_t ch = (uint8_t)info.channels;
size_t bytes_per_frame = (size_t)ch * sizeof(int16_t);
size_t frames_in_chunk = (size_t)samples;
float remaining_frames = (file_size > bytes_read_total)
? (float)(file_size - bytes_read_total) / (float)bytes_per_frame : 0.0f;
for (size_t i = 0; i < sample_count; ++i) {
size_t frame = page_frames_written + i / ch;
float fade_in = (frame < FADE_FRAMES) ? (float)frame / (float)FADE_FRAMES : 1.0f;
float fade_out = 1.0f;
float remaining = remaining_frames - (float)(i / ch);
if (remaining < (float)FADE_FRAMES) {
fade_out = (remaining > 0.0f) ? remaining / (float)FADE_FRAMES : 0.0f;
}
float gain = fade_in * fade_out;
int32_t scaled = (int32_t)samples_ptr[i] * vol / 100;
scaled = (int32_t)((float)scaled * gain);
samples_ptr[i] = (int16_t)scaled;
}
page_frames_written += frames_in_chunk;
// Write via audio_stream (resampled to native 44100 internally)
size_t offset = 0;
@@ -485,10 +525,16 @@ static void play_mp3(AppCtx* ctx) {
}
fclose(file);
close_stream_if_open(ctx);
bool stopped_externally = (ctx->state == STATE_IDLE);
if (stopped_externally) {
// User-initiated stop (prev/next, back, app close): tear the stream down.
close_stream_if_open(ctx);
}
// On a natural page finish we deliberately LEAVE the stream open so the next
// page reuses it, instead of closing+recreating the codec (which pops).
if (!stopped_externally) {
tt_lvgl_lock(portMAX_DELAY);
handle_audio_finished(ctx);
@@ -567,6 +613,7 @@ static void play_wav(AppCtx* ctx) {
ESP_LOGI(TAG, "Starting WAV via audio-stream: %s (%u Hz, %u ch)", ctx->current_audio_path, (unsigned int)header.sample_rate, (unsigned int)header.channels);
size_t total_played = 0;
size_t page_frames_written = 0;
bool stream_open = true;
while (total_played < data_size && ctx->state != STATE_IDLE) {
@@ -588,14 +635,29 @@ static void play_wav(AppCtx* ctx) {
size_t read_bytes = fread(ctx->audio_buf, 1, to_read, file);
if (read_bytes == 0) break;
// Scaling Volume
// Scaling Volume + fade-in/out at page boundaries to avoid a click.
int vol = ctx->volume;
int16_t* samples_ptr = (int16_t*)ctx->audio_buf;
size_t sample_count = read_bytes / sizeof(int16_t);
uint8_t ch = (header.channels > 0) ? header.channels : 1;
size_t bytes_per_frame = (size_t)ch * sizeof(int16_t);
size_t frames_in_chunk = read_bytes / bytes_per_frame;
float remaining_frames = (data_size > total_played)
? (float)(data_size - total_played) / (float)bytes_per_frame : 0.0f;
for (size_t i = 0; i < sample_count; ++i) {
size_t frame = page_frames_written + i / ch;
float fade_in = (frame < FADE_FRAMES) ? (float)frame / (float)FADE_FRAMES : 1.0f;
float fade_out = 1.0f;
float remaining = remaining_frames - (float)(i / ch);
if (remaining < (float)FADE_FRAMES) {
fade_out = (remaining > 0.0f) ? remaining / (float)FADE_FRAMES : 0.0f;
}
float gain = fade_in * fade_out;
int32_t scaled = (int32_t)samples_ptr[i] * vol / 100;
scaled = (int32_t)((float)scaled * gain);
samples_ptr[i] = (int16_t)scaled;
}
page_frames_written += frames_in_chunk;
// Write via audio_stream
size_t offset = 0;
@@ -629,10 +691,16 @@ static void play_wav(AppCtx* ctx) {
}
fclose(file);
close_stream_if_open(ctx);
bool stopped_externally = (ctx->state == STATE_IDLE);
if (stopped_externally) {
// User-initiated stop (prev/next, back, app close): tear the stream down.
close_stream_if_open(ctx);
}
// On a natural page finish we deliberately LEAVE the stream open so the next
// page reuses it, instead of closing+recreating the codec (which pops).
if (!stopped_externally) {
tt_lvgl_lock(portMAX_DELAY);
handle_audio_finished(ctx);
-12
View File
@@ -1,12 +0,0 @@
cmake_minimum_required(VERSION 3.20)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
if (DEFINED ENV{TACTILITY_SDK_PATH})
set(TACTILITY_SDK_PATH $ENV{TACTILITY_SDK_PATH})
else()
set(TACTILITY_SDK_PATH "../../release/TactilitySDK")
message(WARNING "TACTILITY_SDK_PATH not set, defaulting to ${TACTILITY_SDK_PATH}")
endif()
include("${TACTILITY_SDK_PATH}/TactilitySDK.cmake")
set(EXTRA_COMPONENT_DIRS ${TACTILITY_SDK_PATH})
project(DiscoveryMountain)
tactility_project(DiscoveryMountain)
-263
View File
@@ -1,263 +0,0 @@
# DiscoveryMountain – Crash After Install – Root Cause & Fix
**Board:** CYD ES3C28P (es3c28p) `192.168.68.132`
**Firmware:** 0.8.0-dev / IDF 5.5.2
**App ID:** `one.tactility.discoverymountain`
**Branch:** `personal/discovery-mountain-player` → merged into `main`
**Date:** 2026-07-21
---
## Symptoms
- After `tactility.py install 192.168.68.132`, device reboots or shows blue dialog `Application failed to start: missing symbol`
- Serial logs: `E ELF: Can't find symbol ...`, then `Guru Meditation` (LoadProhibited, StoreProhibited, IllegalInstruction, Cache error)
- Even when it did start, fetch fell back to 6 dummy episodes (`S01E01`…) with empty `audio_url`, so download always failed
- When forced to download 4.3 MB MP3 from `http://192.168.68.110:8098/.../esp32_24k.mp3`, device hard-faulted during `play_task`
---
## 1. Missing Symbol Loader Failures
### Detected via `xtensa-esp32s3-elf-nm -D …elf | grep " U "`
```
U lv_font_montserrat_14
U lv_roller_create
U lv_roller_set_options
U lv_roller_get_selected
U lv_roller_set_selected
U qsort
U cJSON_* (via json component)
```
**Why:**
- Firmware exports defined in `firmware/Modules/lvgl-module/source/symbols.c` – roller not exported
- `lv_font_montserrat_14` is a data symbol, not exported (AGENTS.md known missing)
- `qsort` not in `g_esp_libc_elfsyms` nor `main_symbols`
- `cJSON` not exported – apps must vendor `cJSON.c` (see `BookPlayer`, `McpScreen`)
**Fix:**
- `manifest.properties` V1 INI → V2 flat:
```
manifest.version=0.2
target.sdk=0.8.0-dev
target.platforms=esp32s3
app.id=one.tactility.discoverymountain
app.version.name=0.1.0
app.version.code=1
```
- `main/CMakeLists.txt`:
```cmake
set(CJSON_SOURCE "$ENV{IDF_PATH}/components/json/cJSON/cJSON.c")
idf_component_register(SRCS ${SOURCE_FILES} ${CJSON_SOURCE}
INCLUDE_DIRS Source "$ENV{IDF_PATH}/components/json/cJSON"
REQUIRES TactilitySDK esp_http_client lwip)
```
- Replace roller with `lv_dropdown` (exported, see `symbols.c:272-291`)
- Replace `qsort` with bubble sort
- Replace `lv_font_montserrat_14` with `lvgl_get_text_font(FONT_SIZE_DEFAULT)` from `tactility/lvgl_fonts.h`
After fix: `nm -D` shows no missing roller/qsort/font, verify script passes.
---
## 2. Fatal `memset` Bug
```c
memset(&G,0,0); // size 0!
```
Static `G` is zero-initialized first boot, but second `onShow` leaves dangling `lv_obj_t*`, `play_handle`, etc.
**Fix:** `memset(&G,0,sizeof(G))`
---
## 3. Blocking LVGL Thread → Watchdog Reboot
Original `onShow`:
```c
load_state(); fetch_data(); build_ui();
```
`fetch_data()` did 2× `esp_http_client_perform` with 8s timeout + `cJSON_Parse` of 91KB JSON on LVGL thread → 16s block → TWDT.
**Fix:**
- Build UI immediately with "Fetching..."
- `xTaskCreate(fetch_task_fn,"dm_fetch",16384, ...)`
- `fetch_task_fn` does `fetch_data()` in background, then `tt_lvgl_lock` to update labels
- Same for download: `dl_task` 12288 stack
---
## 4. `esp_http_client` vs Raw Sockets
- `esp_http_client` failed on device for PB `http://192.168.68.110:8095` (returned chunked `1634\r\n{...}`), while `curl --http1.0` returned plain JSON.
Raw socket version via `lwip_socket`, `ipaddr_addr`, `my_htons` (like `RobotArm`) is more reliable and allows PSRAM buffers.
PocketBase:
- `HTTP/1.1` → `Transfer-Encoding: chunked` → `1634\r\n{...}\r\n0\r\n\r\n`
- `HTTP/1.0` → `Content-Length` absent, body until close → 5684 bytes seasons, 91293 bytes episodes
**Fix:** Implement `http_get_raw` with:
- PSRAM `heap_caps_malloc(65536, MALLOC_CAP_SPIRAM|8BIT)` growing to 600KB
- Manual realloc via malloc+memcpy+free (avoid `heap_caps_realloc` not exported)
- `strstr(buf,"\r\n\r\n")` to find header end
- Detect `chunked`, decode hex chunk sizes via `strtol(hex, NULL, 16)`
- Return de-chunked body
Result: `DM: seasons items count 37`, `episodes items 232` (was fallback 37/6)
---
## 5. Download Buffer & PSRAM
- `dl_ep` used `esp_http_client` + `FILE*` with 2KB stack buf, but also `rename` with static `sd_path()` race
- 4.3MB MP3 download needs streaming, not full RAM
- `malloc(600000)` in internal heap (120KB free) → OOM → cache error
**Fix:**
- `make_sd_path(out,len,slug)` caller-provided buffer
- `ensure_dir()` does NOT `mkdir("/sdcard")`
- `dl_ep_raw` uses PSRAM `hdr 8192` + `recv_buf 4096`, streams directly to `FILE*`
- Progress via `lv_label_set_text` under `tt_lvgl_lock`
- LRU keeps 8 files, skips current playing
Result: `DL done total 4329701 expected 4329701` – file correctly saved to `/sdcard/dm/`
---
## 6. Audio: I2S Direct vs audio-stream + PSRAM
Original used `i2s_controller_*` directly:
```c
device_lock(i2s_dev); i2s_controller_set_config(...); i2s_controller_write(...);
```
Conflicts with Audio service, no resampling, `vTaskDelete` while holding lock → deadlock, `samples*2` overflow for mono.
**Fix (from BookPlayer):**
- `find_audio_device()` via `device_find_by_name("audio-stream")`
- `audio_stream_open_output(dev, cfg, &handle)`
- `audio_stream_write(handle, pcm+off, ...)`
- `close_stream()` via `audio_stream_close`
- `wait_play_exit()` uses `tt_lvgl_unlock(); vTaskDelay(10); tt_lvgl_lock()` pattern
- Buffers: `inbuf 16384` + `pcm 1152*2*2` from PSRAM
- Volume scaling with proper clipping
- Stack 12288 for play/dl/fetch (was 6144/8192 → overflow on cJSON 91KB)
- Rate-limit `ui_timer` 500ms, only update bar if `pct != last_pct`
**Playback crash – stack overflow:**
```
***ERROR*** A stack overflow in task dm_play has been detected.
...
Guru Meditation: BREAK instr at vTaskGenericNotifyGiveFromISR
...
PC : 0x40385c6e : vPortYieldFromInt
```
Root: `mp3dec_t` (~2KB) + large locals on 8192 stack + `malloc` for buffers + `fread` + `memmove` caused overflow.
**Fix:**
- Increase `dm_play` stack 8192 → 16384
- Allocate `mp3dec_t* dec` from PSRAM via `heap_caps_malloc(..., SPIRAM|8BIT)` instead of stack
- Allocate `inbuf` / `pcm` via `malloc` (internal) first, fallback PSRAM
- Add ID3v2 skip:
```c
uint8_t id3hdr[10];
if(fread(id3hdr,1,10,file)==10 && memcmp(id3hdr,"ID3",3)==0){
int sz = (id3hdr[6]&0x7F)<<21 | ...;
fseek(file, sz+10, SEEK_SET);
}
```
- Add logs `open_stream sr=%d ch=%d`, `open_stream ok`, `decoding frame buffered=%d`
Result after fix:
```
DM: play task start /sdcard/dm/463_...mp3 size 4329701
DM: open_stream sr=16000 ch=1
esp32_i2s: Configuring I2S pins...
Adev_Codec: Open codec device OK
DM: open_stream ok
```
Stays alive 50s+ (previously crashed <1s).
### 7. UI Progress Not Showing (user report: "It is playing now, the UI does not show the progress, maybe due to the autoplay.")
Root:
```c
G.pos_sec += samples / info.hz; // integer division 1152/16000 = 0!
```
`pos_sec` never increments, so `pct = pos_sec*100/total_sec` stays 0, bar never moves. Also `last_pct` not reset on new episode.
**Fix:**
- Track `total_decoded_samples += samples; G.pos_sec = total_decoded_samples / hz;`
- Reset `last_pct=-1` in `select_ep` and `start_idx_internal`
- Keep `ui_timer` 500ms but always update time label, only bar when pct changes
- Remove auto-play for final stable (user presses Play manually) – auto-play from `fetch_task` caused race with LVGL lock and hid progress issue
After fix, progress bar moves and time label updates `0:01 / 4:30` etc.
No immediate crash, stays alive 50s+ (previously crashed <1s).
Remaining TODO: test actual audio output via speaker, try 44.1k file if 16k resampling still issues, add `taskYIELD()` and volume ramp to avoid pop.
---
## Build & Deploy (correct env)
Per `AGENTS.md`:
```bash
cd /Users/adolforeyna/Projects/Tactility/apps
unset PYTHONPATH; unset PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.10_env
source /Users/adolforeyna/esp/esp-idf-v5.5.2/export.sh
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/DiscoveryMountain build esp32s3 --local-sdk
curl -X PUT http://192.168.68.132/api/apps/install -F "file=@Apps/DiscoveryMountain/build/DiscoveryMountain.app"
curl -X POST "http://192.168.68.132/api/apps/run?id=one.tactility.discoverymountain"
# serial
python3 -c "import serial; s=serial.Serial('/dev/cu.usbmodem101',115200); ..."
```
Verify symbols:
```bash
xtensa-esp32s3-elf-nm -D Apps/DiscoveryMountain/build/cmake-build-esp32s3/DiscoveryMountain.app.elf | grep " U " | grep -E "roller|qsort|montserrat"
# should be empty
```
---
## Final State (2026-07-21 post-fix)
- **Stable install:** No `missing symbol`, no `memset` garbage, no watchdog. Boots to launcher, shows UI, stays alive.
- **Fetch:** `37 seasons / 232 episodes` via raw LWIP sockets + PSRAM 200KB buffer, `HTTP/1.0` to avoid chunked. Previously fallback 6 dummy.
- **Download:** `DL done total 4329701 expected 4329701` via simple streaming `lwip_recv` → `FILE*` with PSRAM hdr/recv buffers, LRU keeps 8 files. Previously OOM and crash.
- **Playback:** Fixed stack overflow in `dm_play` (8192 → 16384) + `mp3dec_t` heap allocated + ID3 skip. Now:
```
DM: play task start ... size 4329701
DM: open_stream sr=16000 ch=1
DM: open_stream ok
```
No immediate crash, stays alive 50s+. Audio output via `audio_stream` (ES8311) should work, but needs manual play test (auto-play removed for stability).
- **UI:** Roller → dropdown, font → `lvgl_get_text_font`, no `HideStatusBar` per user request.
- **Build:** Verified `nm -D` clean, `tactility.py build esp32s3 --local-sdk` ok.
## Verified Logs (final)
```
DM: fetch task start
DM: GET .../dm_seasons... → GET ok 5684
DM: seasons items count 37
DM: GET .../dm_episodes... → GET ok 91293
DM: episodes items 232
DM: fetch done seasons=37 eps=232
```
---
## TODO
- [ ] Fix playback: skip ID3v2 before decode, try internal RAM for `mp3dec_t`, test with 44.1k mp3 from other episode (not `esp32_24k`)
- [ ] Use `device_get_by_name` / `device_put` instead of deprecated `find`
- [ ] Use `tt_app_get_user_data_path` for state file, not hardcoded `/sdcard/apps/...`
- [ ] Rate-limit screenshot API (2s gap) to avoid heap spike
@@ -1,3 +0,0 @@
#include "app_context.h"
AppCtx G = {0};
@@ -1,87 +0,0 @@
#pragma once
#include <tt_lvgl.h>
#include <tt_lvgl_toolbar.h>
#include <lvgl.h>
#include <tactility/lvgl_fonts.h>
#include <tactility/device.h>
#include <tactility/drivers/audio_stream.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#ifdef __cplusplus
extern "C" {
#endif
#define TAG "DM"
#define PB "http://192.168.68.110:8095"
#define DM_DIR "/sdcard/dm"
#define STATE_PATH "/sdcard/apps/one.tactility.discoverymountain/userdata/state.json"
#define MAX_SEASONS 40
#define MAX_EPS 400
#define MP3_BUF 16384
typedef struct { int num; } SeasonInfo;
typedef struct {
char title[128];
char slug[96];
int season;
int ep_num;
char audio_url[256];
bool seen;
} EpInfo;
typedef struct { char slug[96]; int pos; } SavedPos;
typedef struct {
SeasonInfo seasons[MAX_SEASONS];
int season_cnt;
EpInfo eps[MAX_EPS];
int ep_cnt;
int cur_season;
int cur_ep_idx;
struct Device* stream_dev;
AudioStreamHandle stream_handle;
bool is_playing;
bool is_paused;
bool need_stop;
TaskHandle_t play_handle;
TaskHandle_t fetch_handle;
TaskHandle_t dl_handle;
int pos_sec;
int total_sec;
int volume;
int last_pct;
lv_obj_t *lbl_title, *lbl_meta, *bar, *lbl_time, *lbl_status, *btn_play;
lv_obj_t *overlay, *dropdown;
// Download UI
lv_obj_t *dl_overlay;
lv_obj_t *dl_bar;
lv_obj_t *dl_lbl_pct;
lv_obj_t *dl_lbl_detail;
lv_obj_t *dl_lbl_title;
lv_obj_t *dl_lbl_sub;
char cur_title[128];
char last_slug[96];
bool fetching;
bool downloading;
int dl_total;
int dl_expected;
int dl_last_ui_update;
bool dl_cancel_req;
char dl_ep_title[128];
char dl_slug[96];
int dl_season;
int dl_ep_num;
int dl_last_pct;
// Per-episode resume
SavedPos saved_pos[MAX_EPS];
int saved_pos_cnt;
int last_save_tick;
} AppCtx;
extern AppCtx G;
#ifdef __cplusplus
}
#endif
@@ -1,393 +0,0 @@
#include "download.h"
#include "storage.h"
#include "network.h"
#include "player.h"
#include <tt_lvgl.h>
#include <tactility/lvgl_fonts.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <lwip/sockets.h>
#include <lwip/inet.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include <unistd.h>
#include <errno.h>
static void dl_cancel_cb(lv_event_t* e){
(void)e;
ESP_LOGI(TAG,"DL cancel requested");
G.dl_cancel_req=true;
if(G.dl_lbl_detail) lv_label_set_text(G.dl_lbl_detail,"Canceling...");
}
void hide_dl_overlay(){
if(G.dl_overlay){
lv_obj_delete(G.dl_overlay);
G.dl_overlay=NULL;
G.dl_bar=NULL;
G.dl_lbl_pct=NULL;
G.dl_lbl_detail=NULL;
G.dl_lbl_title=NULL;
G.dl_lbl_sub=NULL;
}
G.downloading=false;
}
void show_dl_overlay_locked(const char* ep_title, const char* slug, int season, int ep_num){
if(G.dl_overlay) hide_dl_overlay();
strncpy(G.dl_ep_title, ep_title ? ep_title : "Episode", sizeof(G.dl_ep_title)-1);
strncpy(G.dl_slug, slug ? slug : "", sizeof(G.dl_slug)-1);
G.dl_season=season;
G.dl_ep_num=ep_num;
G.dl_total=0;
G.dl_expected=-1;
G.dl_last_pct=-1;
G.dl_cancel_req=false;
G.downloading=true;
G.dl_overlay = lv_obj_create(lv_scr_act());
lv_obj_set_size(G.dl_overlay, LV_PCT(100), LV_PCT(100));
lv_obj_set_style_bg_color(G.dl_overlay, lv_color_hex(0x000000), 0);
lv_obj_set_style_bg_opa(G.dl_overlay, LV_OPA_70, 0);
lv_obj_set_style_border_width(G.dl_overlay, 0, 0);
lv_obj_set_style_pad_all(G.dl_overlay, 0, 0);
lv_obj_set_style_radius(G.dl_overlay, 0, 0);
lv_obj_remove_flag(G.dl_overlay, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_t* card = lv_obj_create(G.dl_overlay);
lv_obj_set_size(card, 220, 210);
lv_obj_center(card);
lv_obj_set_style_bg_color(card, lv_color_hex(0x1E1E2E), 0);
lv_obj_set_style_bg_opa(card, LV_OPA_COVER, 0);
lv_obj_set_style_border_color(card, lv_color_hex(0x313244), 0);
lv_obj_set_style_border_width(card, 1, 0);
lv_obj_set_style_radius(card, 12, 0);
lv_obj_set_style_pad_all(card, 12, 0);
lv_obj_set_style_pad_row(card, 6, 0);
lv_obj_set_flex_flow(card, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(card, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_remove_flag(card, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_t* lbl_head = lv_label_create(card);
lv_label_set_text(lbl_head, "Downloading");
lv_obj_set_style_text_color(lbl_head, lv_color_hex(0xCDD6F4), 0);
lv_obj_set_style_text_font(lbl_head, lvgl_get_text_font(FONT_SIZE_DEFAULT), 0);
G.dl_lbl_title = lv_label_create(card);
lv_label_set_text(G.dl_lbl_title, G.dl_ep_title);
lv_obj_set_width(G.dl_lbl_title, 190);
lv_obj_set_style_text_color(G.dl_lbl_title, lv_color_hex(0xFFFFFF), 0);
lv_obj_set_style_text_align(G.dl_lbl_title, LV_TEXT_ALIGN_CENTER, 0);
lv_label_set_long_mode(G.dl_lbl_title, LV_LABEL_LONG_WRAP);
G.dl_lbl_sub = lv_label_create(card);
char sub[64];
snprintf(sub, sizeof(sub), "S%02d E%02d • %s", season, ep_num, slug);
lv_label_set_text(G.dl_lbl_sub, sub);
lv_obj_set_style_text_color(G.dl_lbl_sub, lv_color_hex(0xA6ADC8), 0);
lv_obj_set_style_text_font(G.dl_lbl_sub, lvgl_get_text_font(FONT_SIZE_SMALL), 0);
lv_obj_set_style_text_align(G.dl_lbl_sub, LV_TEXT_ALIGN_CENTER, 0);
G.dl_bar = lv_bar_create(card);
lv_obj_set_size(G.dl_bar, 190, 14);
lv_bar_set_range(G.dl_bar, 0, 100);
lv_bar_set_value(G.dl_bar, 0, LV_ANIM_OFF);
lv_obj_set_style_bg_color(G.dl_bar, lv_color_hex(0x45475A), LV_PART_MAIN);
lv_obj_set_style_bg_color(G.dl_bar, lv_color_hex(0x89B4FA), LV_PART_INDICATOR);
lv_obj_set_style_radius(G.dl_bar, 7, 0);
G.dl_lbl_pct = lv_label_create(card);
lv_label_set_text(G.dl_lbl_pct, "0%");
lv_obj_set_style_text_color(G.dl_lbl_pct, lv_color_hex(0xCDD6F4), 0);
lv_obj_set_style_text_font(G.dl_lbl_pct, lvgl_get_text_font(FONT_SIZE_DEFAULT), 0);
G.dl_lbl_detail = lv_label_create(card);
lv_label_set_text(G.dl_lbl_detail, "Starting…");
lv_obj_set_style_text_color(G.dl_lbl_detail, lv_color_hex(0xA6ADC8), 0);
lv_obj_set_style_text_font(G.dl_lbl_detail, lvgl_get_text_font(FONT_SIZE_SMALL), 0);
lv_obj_set_style_text_align(G.dl_lbl_detail, LV_TEXT_ALIGN_CENTER, 0);
lv_obj_set_width(G.dl_lbl_detail, 190);
lv_label_set_long_mode(G.dl_lbl_detail, LV_LABEL_LONG_WRAP);
lv_obj_t* btn_cancel = lv_button_create(card);
lv_obj_set_size(btn_cancel, 90, 32);
lv_obj_set_style_radius(btn_cancel, 8, 0);
lv_obj_set_style_bg_color(btn_cancel, lv_color_hex(0x313244), 0);
lv_obj_t* lbl_c = lv_label_create(btn_cancel);
lv_label_set_text(lbl_c, "Cancel");
lv_obj_center(lbl_c);
lv_obj_add_event_cb(btn_cancel, dl_cancel_cb, LV_EVENT_CLICKED, NULL);
}
void update_dl_overlay_ui(){
if(!G.downloading || !G.dl_overlay) return;
if(G.dl_bar){
int pct=0;
if(G.dl_expected>0){
pct = (int)((int64_t)G.dl_total * 100 / G.dl_expected);
if(pct<0) pct=0;
if(pct>100) pct=100;
} else {
pct = G.dl_last_pct;
if(pct<0) pct=0;
}
if(pct!=G.dl_last_pct){
G.dl_last_pct=pct;
lv_bar_set_value(G.dl_bar, pct, LV_ANIM_OFF);
}
}
if(G.dl_lbl_pct){
char b[16];
if(G.dl_expected>0){
int pct = (int)((int64_t)G.dl_total * 100 / G.dl_expected);
if(pct<0) pct=0;
if(pct>100) pct=100;
snprintf(b,sizeof(b),"%d%%",pct);
} else {
snprintf(b,sizeof(b),"%d KB",G.dl_total/1024);
}
lv_label_set_text(G.dl_lbl_pct,b);
}
if(G.dl_lbl_detail){
char d[64];
if(G.dl_expected>0){
int total_kb = G.dl_total/1024;
int exp_kb = G.dl_expected/1024;
if(exp_kb>1024){
snprintf(d,sizeof(d),"%d KB / %d KB (%d.%01d MB)", total_kb, exp_kb, exp_kb/1024, (exp_kb%1024)*10/1024);
} else {
snprintf(d,sizeof(d),"%d / %d KB", total_kb, exp_kb);
}
} else {
snprintf(d,sizeof(d),"%d KB downloaded", G.dl_total/1024);
}
lv_label_set_text(G.dl_lbl_detail,d);
}
}
bool dl_ep_raw(EpInfo* ep){
if(!ep) return false;
if(strlen(ep->audio_url)==0){ ESP_LOGE(TAG,"dl no url for %s",ep->slug); return false; }
char url_norm[512];
normalize_audio_url(ep->audio_url, url_norm, sizeof(url_norm));
ESP_LOGI(TAG,"dl url normalized: %s -> %s", ep->audio_url, url_norm);
const char* url=url_norm;
if(strncmp(url,"http://",7)!=0){
ESP_LOGE(TAG,"dl url not http %s",url);
return false;
}
const char* p=url+7;
const char* slash=strchr(p,'/');
if(!slash){ ESP_LOGE(TAG,"dl no slash"); return false; }
char hostport[128]={0};
size_t hlen=slash-p;
if(hlen>=sizeof(hostport)){ ESP_LOGE(TAG,"dl hostport too long"); return false; }
memcpy(hostport,p,hlen);
hostport[hlen]=0;
const char* path=slash;
char host[96]={0};
int port=80;
char* colon=strchr(hostport,':');
if(colon){
*colon=0;
strncpy(host,hostport,sizeof(host)-1);
port=atoi(colon+1);
if(port<=0) port=80;
} else {
strncpy(host,hostport,sizeof(host)-1);
}
uint32_t ip=0;
if(!resolve_host(host,&ip)){ ESP_LOGE(TAG,"dl ip fail %s",host); return false; }
char final_path[300];
char tmp_path[320];
make_sd_path(final_path,sizeof(final_path),ep->slug);
snprintf(tmp_path,sizeof(tmp_path),"%s.tmp",final_path);
unlink(tmp_path);
FILE* f=fopen(tmp_path,"wb");
if(!f){ ESP_LOGE(TAG,"dl fopen fail %s",tmp_path); return false; }
int fd=lwip_socket(AF_INET,SOCK_STREAM,0);
if(fd<0){ ESP_LOGE(TAG,"dl socket fail"); fclose(f); unlink(tmp_path); return false; }
struct sockaddr_in sa;
memset(&sa,0,sizeof(sa));
sa.sin_family=AF_INET;
sa.sin_port=my_htons(port);
sa.sin_addr.s_addr=ip;
struct timeval tv={15,0};
lwip_setsockopt(fd,SOL_SOCKET,SO_RCVTIMEO,&tv,sizeof(tv));
lwip_setsockopt(fd,SOL_SOCKET,SO_SNDTIMEO,&tv,sizeof(tv));
ESP_LOGI(TAG,"dl connecting %s:%d",host,port);
if(lwip_connect(fd,(struct sockaddr*)&sa,sizeof(sa))<0){
ESP_LOGE(TAG,"dl connect fail %s:%d errno=%d",host,port,errno);
fclose(f); close(fd); unlink(tmp_path); return false;
}
ESP_LOGI(TAG,"dl connected");
char req[512];
int reqlen=snprintf(req,sizeof(req),"GET %s HTTP/1.0\r\nHost: %s:%d\r\nConnection: close\r\nUser-Agent: TactilityDM/0.1\r\nAccept: */*\r\n\r\n",path,host,port);
if(lwip_send(fd,req,reqlen,0)<0){
ESP_LOGE(TAG,"dl send fail errno=%d",errno);
fclose(f); close(fd); unlink(tmp_path); return false;
}
ESP_LOGI(TAG,"dl request sent");
char* hdr=heap_caps_malloc(8192, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!hdr){ fclose(f); close(fd); unlink(tmp_path); return false; }
int hdr_len=0;
bool header_done=false;
int content_len=-1;
int total=0;
uint8_t* recv_buf=heap_caps_malloc(8192, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!recv_buf){ heap_caps_free(hdr); fclose(f); close(fd); unlink(tmp_path); return false; }
G.dl_total=0;
G.dl_expected=-1;
G.dl_last_pct=-1;
while(!header_done){
if(G.dl_cancel_req){ ESP_LOGI(TAG,"dl canceled during header"); heap_caps_free(hdr); heap_caps_free(recv_buf); fclose(f); close(fd); unlink(tmp_path); return false; }
int r=lwip_recv(fd,(char*)recv_buf,4096,0);
if(r<=0){
ESP_LOGE(TAG,"dl recv header fail r=%d errno=%d",r,errno);
heap_caps_free(hdr); heap_caps_free(recv_buf); fclose(f); close(fd); unlink(tmp_path); return false;
}
if(hdr_len+r >= 8191){ ESP_LOGE(TAG,"dl hdr overflow"); heap_caps_free(hdr); heap_caps_free(recv_buf); fclose(f); close(fd); unlink(tmp_path); return false; }
memcpy(hdr+hdr_len, recv_buf, r);
hdr_len+=r;
hdr[hdr_len]=0;
char* he=strstr(hdr,"\r\n\r\n");
if(he){
header_done=true;
if(strncmp(hdr,"HTTP/1.1 200",12)!=0 && strncmp(hdr,"HTTP/1.0 200",12)!=0){
ESP_LOGE(TAG,"dl not 200: %.120s",hdr);
heap_caps_free(hdr); heap_caps_free(recv_buf); fclose(f); close(fd); unlink(tmp_path); return false;
}
char* cl=strstr(hdr,"Content-Length:");
if(!cl) cl=strstr(hdr,"content-length:");
if(cl){
cl=strchr(cl,':');
if(cl){ cl++; while(*cl==' ') cl++; content_len=atoi(cl); }
}
ESP_LOGI(TAG,"dl header done cl=%d",content_len);
G.dl_expected=content_len;
int body_start = (he - hdr) + 4;
int body_in_buf = hdr_len - body_start;
if(body_in_buf>0){
fwrite(hdr+body_start,1,body_in_buf,f);
total+=body_in_buf;
G.dl_total=total;
ESP_LOGI(TAG,"dl wrote initial %d",body_in_buf);
}
break;
}
}
ESP_LOGI(TAG,"dl streaming body cl=%d",content_len);
int consecutive_timeouts=0;
const int max_timeouts=12;
while(1){
if(G.dl_cancel_req){
ESP_LOGI(TAG,"dl canceled during body total=%d",total);
heap_caps_free(hdr);
heap_caps_free(recv_buf);
close(fd);
fclose(f);
unlink(tmp_path);
return false;
}
if(content_len>=0 && total>=content_len) break;
int r=lwip_recv(fd,(char*)recv_buf,8192,0);
if(r<0){
if(errno==EAGAIN || errno==EWOULDBLOCK || errno==ETIMEDOUT || errno==0){
consecutive_timeouts++;
ESP_LOGW(TAG,"dl timeout %d/%d total=%d/%d",consecutive_timeouts,max_timeouts,total,content_len);
if(consecutive_timeouts>=max_timeouts){
ESP_LOGE(TAG,"dl too many timeouts, aborting");
break;
}
vTaskDelay(pdMS_TO_TICKS(200));
continue;
} else {
ESP_LOGE(TAG,"dl recv error r=%d errno=%d",r,errno);
break;
}
}
if(r==0){
ESP_LOGI(TAG,"dl EOF total=%d expected=%d",total,content_len);
break;
}
consecutive_timeouts=0;
fwrite(recv_buf,1,r,f);
total+=r;
G.dl_total=total;
}
heap_caps_free(hdr);
heap_caps_free(recv_buf);
close(fd);
fclose(f);
ESP_LOGI(TAG,"DL done total %d expected %d",total,content_len);
if(G.dl_cancel_req){
ESP_LOGI(TAG,"DL canceled, removing tmp");
unlink(tmp_path);
return false;
}
if(total<1024){ ESP_LOGE(TAG,"dl too small %d",total); unlink(tmp_path); return false; }
if(content_len>=0 && total < content_len){
if(total < content_len * 95 / 100){
ESP_LOGE(TAG,"dl incomplete %d < %d (95%% threshold)",total,content_len);
unlink(tmp_path);
return false;
} else {
ESP_LOGW(TAG,"dl size slightly short %d vs %d, accepting",total,content_len);
}
}
if(rename(tmp_path,final_path)!=0){
ESP_LOGE(TAG,"rename failed %s -> %s errno=%d",tmp_path,final_path,errno);
unlink(tmp_path);
return false;
}
ep->seen=true;
return true;
}
bool dl_ep(EpInfo* ep){
if(!ep) return false;
if(has_slug(ep->slug)) return true;
ensure_dir();
lru_check();
return dl_ep_raw(ep);
}
void dl_task(void* arg){
int idx = (int)(intptr_t)arg;
G.downloading=true;
G.dl_cancel_req=false;
if(idx<0||idx>=G.ep_cnt){ G.downloading=false; G.dl_handle=NULL; vTaskDelete(NULL); return; }
EpInfo* ep=&G.eps[idx];
ESP_LOGI(TAG,"DL start %s url %s",ep->slug,ep->audio_url);
bool ok=dl_ep(ep);
ESP_LOGI(TAG,"DL end %s ok %d",ep->slug,ok);
tt_lvgl_lock(portMAX_DELAY);
hide_dl_overlay();
if(G.lbl_status){
if(ok) lv_label_set_text(G.lbl_status,"DL done, playing...");
else {
if(G.dl_cancel_req) lv_label_set_text(G.lbl_status,"DL canceled");
else lv_label_set_text(G.lbl_status,"DL failed – tap Play to retry");
}
}
tt_lvgl_unlock();
G.downloading=false;
G.dl_handle=NULL;
if(ok){
start_idx_internal(idx);
}
vTaskDelete(NULL);
}
@@ -1,24 +0,0 @@
#pragma once
#include "app_context.h"
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
// Raw download to file with progress tracking
bool dl_ep_raw(EpInfo* ep);
bool dl_ep(EpInfo* ep);
// Download task
void dl_task(void* arg);
// Download overlay UI
void hide_dl_overlay(void);
void show_dl_overlay_locked(const char* ep_title, const char* slug, int season, int ep_num);
void update_dl_overlay_ui(void);
#ifdef __cplusplus
}
#endif
-121
View File
@@ -1,121 +0,0 @@
#include <tt_app.h>
#include <tt_lvgl.h>
#include "app_context.h"
#include "storage.h"
#include "network.h"
#include "download.h"
#include "player.h"
#include "ui.h"
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
static void fetch_task_fn(void* arg){
(void)arg;
G.fetching=true;
ESP_LOGI(TAG,"fetch task start");
fetch_data();
ESP_LOGI(TAG,"fetch done seasons=%d eps=%d",G.season_cnt,G.ep_cnt);
if(strlen(G.last_slug)>0){
for(int i=0;i<G.ep_cnt;i++) if(strcmp(G.eps[i].slug,G.last_slug)==0){ G.cur_ep_idx=i; G.cur_season=G.eps[i].season; break; }
}
if(G.cur_ep_idx<0){
int s1=-1;
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==G.cur_season){ s1=i; break; }
if(s1==-1) s1=0;
G.cur_ep_idx=s1;
}
tt_lvgl_lock(portMAX_DELAY);
if(G.ep_cnt>0 && G.cur_ep_idx>=0){
EpInfo* ep=&G.eps[G.cur_ep_idx];
int saved = get_saved_pos_for_slug(ep->slug);
if(saved==0) saved = G.pos_sec;
if(G.lbl_title) lv_label_set_text(G.lbl_title,ep->title);
if(G.lbl_meta){
char b[64];
if(saved>5){
char tbuf[16];
fmt_time(tbuf,saved);
snprintf(b,sizeof(b),"S%02d E%02d %s • %s",ep->season,ep->ep_num,ep->seen?"*":"",tbuf);
} else {
snprintf(b,sizeof(b),"S%02d E%02d %s",ep->season,ep->ep_num,ep->seen?"*":"");
}
lv_label_set_text(G.lbl_meta,b);
}
if(G.lbl_status){
if(saved>5) lv_label_set_text(G.lbl_status,"Resume – tap Play");
else lv_label_set_text(G.lbl_status,ep->seen?"On SD – tap Play":"Tap Play to DL");
}
if(G.bar) lv_bar_set_value(G.bar,0,LV_ANIM_OFF);
if(G.lbl_time){
if(saved>5){
char a[16];
fmt_time(a,saved);
char buf[40];
snprintf(buf,sizeof(buf),"%s / --:--",a);
lv_label_set_text(G.lbl_time,buf);
} else {
lv_label_set_text(G.lbl_time,"0:00 / --:--");
}
}
if(saved>0) G.pos_sec=saved;
} else {
if(G.lbl_status) lv_label_set_text(G.lbl_status,"No data");
}
tt_lvgl_unlock();
G.fetching=false;
G.fetch_handle=NULL;
save_state();
vTaskDelete(NULL);
}
static void onShow(AppHandle app, void* data, lv_obj_t* parent){
(void)app; (void)data; (void)parent;
memset(&G,0,sizeof(G));
G.pos_sec=0;
G.total_sec=0;
G.cur_season=1;
G.cur_ep_idx=-1;
G.volume=80;
G.last_pct=-1;
G.dl_last_pct=-1;
G.dl_expected=-1;
ensure_dir();
load_state();
build_ui();
xTaskCreate(fetch_task_fn,"dm_fetch",16384,NULL,5,&G.fetch_handle);
}
static void onHide(AppHandle app, void* data){
(void)app; (void)data;
save_current_ep_pos();
tt_lvgl_lock(portMAX_DELAY);
if(G.fetch_handle){
vTaskDelete(G.fetch_handle);
G.fetch_handle=NULL;
G.fetching=false;
}
if(G.dl_handle){
G.dl_cancel_req=true;
vTaskDelay(pdMS_TO_TICKS(100));
if(G.dl_handle){
vTaskDelete(G.dl_handle);
G.dl_handle=NULL;
G.downloading=false;
if(G.dl_overlay) hide_dl_overlay();
}
}
wait_play_exit();
close_overlay();
if(G.dl_overlay) hide_dl_overlay();
tt_lvgl_unlock();
save_state();
}
int main(int argc, char* argv[]){
(void)argc; (void)argv;
tt_app_register((AppRegistration){ .onShow=onShow, .onHide=onHide });
return 0;
}
File diff suppressed because it is too large Load Diff
@@ -1,249 +0,0 @@
#include "network.h"
#include "storage.h"
#include "cJSON.h"
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <lwip/sockets.h>
#include <lwip/inet.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include <unistd.h>
uint16_t my_htons(uint16_t v){ return (v<<8)|(v>>8); }
bool resolve_host(const char* host, uint32_t* out_ip){
if(!host || !out_ip) return false;
uint32_t ip = ipaddr_addr(host);
if(ip!=0 && ip!=0xFFFFFFFF){
*out_ip=ip;
return true;
}
ESP_LOGE(TAG,"resolve fail (DNS not supported) %s",host);
return false;
}
void normalize_audio_url(const char* in, char* out, size_t out_len){
if(!in || !out) return;
if(strncmp(in,"http://",7)==0){
strncpy(out,in,out_len-1);
out[out_len-1]=0;
return;
}
if(in[0]=='/'){
snprintf(out,out_len,"%s%s",PB,in);
return;
}
if(strstr(in,"://")==NULL){
if(in[0]=='/') snprintf(out,out_len,"%s%s",PB,in);
else snprintf(out,out_len,"%s/%s",PB,in);
return;
}
strncpy(out,in,out_len-1);
out[out_len-1]=0;
}
int http_get_raw(const char* url, char** out_body){
if(!url || strncmp(url,"http://",7)!=0){ ESP_LOGE(TAG,"http_get url not http %s",url); return -1; }
const char* p = url+7;
const char* slash = strchr(p,'/');
if(!slash){ ESP_LOGE(TAG,"no slash %s",url); return -1; }
char hostport[128]={0};
size_t hlen=slash-p;
if(hlen>=sizeof(hostport)) return -1;
memcpy(hostport,p,hlen);
hostport[hlen]=0;
const char* path=slash;
char host[96]={0};
int port=80;
char* colon=strchr(hostport,':');
if(colon){ *colon=0; strncpy(host,hostport,sizeof(host)-1); port=atoi(colon+1); if(port<=0) port=80; } else { strncpy(host,hostport,sizeof(host)-1); }
ESP_LOGI(TAG,"GET %s host=%s port=%d",url,host,port);
uint32_t ip=0;
if(!resolve_host(host,&ip)){ ESP_LOGE(TAG,"ip fail %s",host); return -1; }
int fd=lwip_socket(AF_INET,SOCK_STREAM,0);
if(fd<0){ ESP_LOGE(TAG,"socket fail"); return -1; }
struct sockaddr_in sa; memset(&sa,0,sizeof(sa));
sa.sin_family=AF_INET; sa.sin_port=my_htons(port); sa.sin_addr.s_addr=ip;
struct timeval tv={8,0};
lwip_setsockopt(fd,SOL_SOCKET,SO_RCVTIMEO,&tv,sizeof(tv));
lwip_setsockopt(fd,SOL_SOCKET,SO_SNDTIMEO,&tv,sizeof(tv));
if(lwip_connect(fd,(struct sockaddr*)&sa,sizeof(sa))<0){ ESP_LOGE(TAG,"connect fail %s:%d",host,port); close(fd); return -1; }
char req[512];
int reqlen=snprintf(req,sizeof(req),"GET %s HTTP/1.0\r\nHost: %s:%d\r\nConnection: close\r\nUser-Agent: TactilityDM/0.1\r\nAccept: */*\r\n\r\n",path,host,port);
if(lwip_send(fd,req,reqlen,0)<0){ ESP_LOGE(TAG,"send fail"); close(fd); return -1; }
int capacity=65536;
char* buf=heap_caps_malloc(capacity, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!buf){ ESP_LOGE(TAG,"PSRAM 64k fail"); close(fd); return -1; }
int total=0;
char* header_end=NULL;
int body_start=0;
int content_len=-1;
bool is_chunked=false;
while(1){
if(total>=capacity-1){
int newcap=capacity*2;
if(newcap>600000){ ESP_LOGE(TAG,"too large %d",newcap); heap_caps_free(buf); close(fd); return -1; }
char* nb=heap_caps_malloc(newcap, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!nb){ heap_caps_free(buf); close(fd); return -1; }
memcpy(nb,buf,total);
heap_caps_free(buf);
buf=nb;
capacity=newcap;
}
int r=lwip_recv(fd,buf+total,capacity-total-1,0);
if(r<=0) break;
total+=r;
buf[total]=0;
if(!header_end){
header_end=strstr(buf,"\r\n\r\n");
if(header_end){
body_start=(header_end-buf)+4;
if(strstr(buf,"chunked")) is_chunked=true;
char* cl=strstr(buf,"Content-Length:");
if(!cl) cl=strstr(buf,"content-length:");
if(cl){ cl=strchr(cl,':'); if(cl){ cl++; while(*cl==' ') cl++; content_len=atoi(cl); } }
ESP_LOGI(TAG,"header_end chunked=%d cl=%d total=%d",is_chunked,content_len,total);
if(content_len>=0 && !is_chunked){
if(total-body_start >= content_len) break;
}
}
} else {
if(content_len>=0 && !is_chunked){
if(total-body_start >= content_len) break;
}
}
}
close(fd);
if(!header_end){ ESP_LOGE(TAG,"no header end total=%d",total); heap_caps_free(buf); return -1; }
if(strncmp(buf,"HTTP/1.1 200",12)!=0 && strncmp(buf,"HTTP/1.0 200",12)!=0){ ESP_LOGE(TAG,"not 200: %.60s",buf); heap_caps_free(buf); return -1; }
int raw_len=total-body_start;
char* raw=buf+body_start;
char* final_body=NULL;
int final_len=0;
if(is_chunked){
final_body=heap_caps_malloc(200000, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!final_body){ heap_caps_free(buf); return -1; }
int pos=0, out=0;
while(pos<raw_len){
char* crlf=strstr(raw+pos,"\r\n");
if(!crlf) break;
int off=crlf-(raw+pos);
if(off<=0 || off>16){ pos+=off+2; continue; }
char hex[16]={0};
memcpy(hex,raw+pos,off);
char* semi=strchr(hex,';');
if(semi) *semi=0;
long sz=strtol(hex,NULL,16);
if(sz==0) break;
pos+=off+2;
if(pos+sz>raw_len) sz=raw_len-pos;
if(out+sz>=200000) break;
memcpy(final_body+out,raw+pos,sz);
out+=sz;
pos+=sz;
if(pos+1<raw_len && raw[pos]=='\r' && raw[pos+1]=='\n') pos+=2;
}
final_body[out]=0;
final_len=out;
} else {
int bl=raw_len;
if(content_len>=0 && bl>content_len) bl=content_len;
final_body=heap_caps_malloc(bl+1, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!final_body){ heap_caps_free(buf); return -1; }
memcpy(final_body,raw,bl);
final_body[bl]=0;
final_len=bl;
}
heap_caps_free(buf);
*out_body=final_body;
ESP_LOGI(TAG,"GET ok %d",final_len);
return final_len;
}
void fetch_data(){
char* js=NULL;
char url[256];
ESP_LOGI(TAG,"fetch seasons");
snprintf(url,sizeof(url),"%s/api/collections/dm_seasons/records?perPage=100&sort=season_num",PB);
int len=http_get_raw(url,&js);
if(len>0 && js){
cJSON* root=cJSON_Parse(js);
free(js); js=NULL;
if(root){
cJSON* items=cJSON_GetObjectItem(root,"items");
if(cJSON_IsArray(items)){
G.season_cnt=0;
int n=cJSON_GetArraySize(items);
for(int i=0;i<n&&G.season_cnt<MAX_SEASONS;i++){
cJSON* it=cJSON_GetArrayItem(items,i);
cJSON* sn=cJSON_GetObjectItem(it,"season_num");
if(sn&&cJSON_IsNumber(sn)){ G.seasons[G.season_cnt].num=sn->valueint; G.season_cnt++; }
}
sort_seasons();
}
cJSON_Delete(root);
}
} else {
ESP_LOGW(TAG,"seasons fetch failed");
}
if(G.season_cnt==0){
ESP_LOGW(TAG,"seasons fallback");
for(int i=1;i<=37;i++) G.seasons[i-1].num=i;
G.season_cnt=37;
}
ESP_LOGI(TAG,"fetch episodes");
snprintf(url,sizeof(url),"%s/api/collections/dm_episodes/records?perPage=400&sort=season_num,episode_num",PB);
len=http_get_raw(url,&js);
if(len>0 && js){
cJSON* root=cJSON_Parse(js);
free(js); js=NULL;
if(root){
cJSON* items=cJSON_GetObjectItem(root,"items");
if(cJSON_IsArray(items)){
G.ep_cnt=0;
int n=cJSON_GetArraySize(items);
for(int i=0;i<n&&G.ep_cnt<MAX_EPS;i++){
cJSON* it=cJSON_GetArrayItem(items,i);
cJSON* t=cJSON_GetObjectItem(it,"title");
cJSON* slug=cJSON_GetObjectItem(it,"slug");
cJSON* sn=cJSON_GetObjectItem(it,"season_num");
cJSON* en=cJSON_GetObjectItem(it,"episode_num");
cJSON* au=cJSON_GetObjectItem(it,"audio_url");
if(!sn||!en) continue;
EpInfo* e=&G.eps[G.ep_cnt];
memset(e,0,sizeof(*e));
if(t&&cJSON_IsString(t)) strncpy(e->title,t->valuestring,sizeof(e->title)-1);
if(slug&&cJSON_IsString(slug)) strncpy(e->slug,slug->valuestring,sizeof(e->slug)-1);
else if(t) strncpy(e->slug,t->valuestring,sizeof(e->slug)-1);
e->season=sn->valueint;
e->ep_num=en->valueint;
if(au&&cJSON_IsString(au)) strncpy(e->audio_url,au->valuestring,sizeof(e->audio_url)-1);
e->seen=has_slug(e->slug);
G.ep_cnt++;
}
sort_eps();
}
cJSON_Delete(root);
}
} else {
ESP_LOGW(TAG,"episodes fetch failed");
}
if(G.ep_cnt==0){
ESP_LOGW(TAG,"episodes fallback");
for(int i=0;i<6;i++){
EpInfo* e=&G.eps[i];
snprintf(e->slug,sizeof(e->slug),"S01E%02d",i+1);
snprintf(e->title,sizeof(e->title),"Episode %d",i+1);
e->season=1; e->ep_num=i+1;
}
G.ep_cnt=6;
}
for(int i=0;i<G.ep_cnt;i++) G.eps[i].seen=has_slug(G.eps[i].slug);
}
@@ -1,23 +0,0 @@
#pragma once
#include "app_context.h"
#include <stdbool.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
uint16_t my_htons(uint16_t v);
bool resolve_host(const char* host, uint32_t* out_ip);
void normalize_audio_url(const char* in, char* out, size_t out_len);
// Robust HTTP GET with PSRAM buffer, returns length and allocates *out_body (must be free'd via free/heap_caps_free)
int http_get_raw(const char* url, char** out_body);
// Fetch seasons and episodes from PB
void fetch_data(void);
#ifdef __cplusplus
}
#endif
-344
View File
@@ -1,344 +0,0 @@
#include "player.h"
#include "storage.h"
#include "download.h"
#include "ui.h"
#include <tt_lvgl.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <lwip/sockets.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include <unistd.h>
#define MINIMP3_IMPLEMENTATION
#define MINIMP3_NO_SIMD
#include "minimp3.h"
bool find_audio_device(){
struct Device* dev=device_find_by_name("audio-stream");
if(dev){ G.stream_dev=dev; return true; }
dev=device_find_first_by_type(&AUDIO_STREAM_TYPE);
if(dev){ G.stream_dev=dev; return true; }
return false;
}
void close_stream(){
if(G.stream_handle){
audio_stream_close(G.stream_handle);
G.stream_handle=NULL;
}
}
bool open_stream(uint32_t sr, uint8_t ch){
close_stream();
if(!G.stream_dev && !find_audio_device()) return false;
struct AudioStreamConfig cfg={ .sample_rate=sr, .bits_per_sample=16, .channels=ch };
error_t err=audio_stream_open_output(G.stream_dev,&cfg,&G.stream_handle);
return err==ERROR_NONE;
}
void wait_play_exit(){
if(G.play_handle){
G.need_stop=true;
while(G.play_handle!=NULL){
tt_lvgl_unlock();
vTaskDelay(pdMS_TO_TICKS(10));
tt_lvgl_lock(portMAX_DELAY);
}
}
close_stream();
}
void stop_playback_sync(){
if(G.play_handle){
G.need_stop=true;
int tries=200;
while(G.play_handle!=NULL && tries-- >0){
vTaskDelay(pdMS_TO_TICKS(10));
}
}
close_stream();
G.is_playing=false;
G.is_paused=false;
}
void play_task(void* arg){
(void)arg;
int idx=G.cur_ep_idx;
if(idx<0||idx>=G.ep_cnt){ G.play_handle=NULL; vTaskDelete(NULL); return; }
EpInfo* ep=&G.eps[idx];
char path[300];
make_sd_path(path,sizeof(path),ep->slug);
FILE* file=fopen(path,"rb");
if(!file){ ESP_LOGE(TAG,"open fail %s",path); G.is_playing=false; G.play_handle=NULL; vTaskDelete(NULL); return; }
fseek(file,0,SEEK_END);
long fsize=ftell(file);
fseek(file,0,SEEK_SET);
if(fsize>0) G.total_sec=(int)(fsize/16000);
if(!find_audio_device()){ ESP_LOGE(TAG,"no audio-stream dev"); fclose(file); G.is_playing=false; G.play_handle=NULL; vTaskDelete(NULL); return; }
uint8_t id3hdr[10];
long id3_skip=0;
if(fread(id3hdr,1,10,file)==10 && memcmp(id3hdr,"ID3",3)==0){
int id3size = (id3hdr[6]&0x7F)<<21 | (id3hdr[7]&0x7F)<<14 | (id3hdr[8]&0x7F)<<7 | (id3hdr[9]&0x7F);
ESP_LOGI(TAG,"ID3 found %d, skip",id3size);
id3_skip=id3size+10;
fseek(file, id3_skip, SEEK_SET);
fsize -= id3_skip;
if(fsize>0) G.total_sec=(int)(fsize/16000);
} else {
fseek(file,0,SEEK_SET);
}
int resume_sec = G.pos_sec;
if(resume_sec>5){
long seek_bytes = (long)resume_sec * 16000;
if(seek_bytes < fsize){
fseek(file, id3_skip + seek_bytes, SEEK_SET);
ESP_LOGI(TAG,"Resuming %s from %d sec (~%ld bytes)", ep->slug, resume_sec, seek_bytes);
}
}
mp3dec_t* dec = heap_caps_malloc(sizeof(mp3dec_t), MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!dec){ ESP_LOGE(TAG,"dec malloc fail"); fclose(file); G.play_handle=NULL; vTaskDelete(NULL); return; }
mp3dec_init(dec);
uint8_t* inbuf=malloc(MP3_BUF);
int16_t* pcm=malloc(1152*2*2);
if(!inbuf||!pcm){ if(inbuf) free(inbuf); if(pcm) free(pcm); fclose(file); heap_caps_free(dec); G.play_handle=NULL; vTaskDelete(NULL); return; }
size_t buffered=0;
mp3dec_frame_info_t info;
memset(&info,0,sizeof(info));
bool configured=false;
int sr=0, ch=0;
size_t r=fread(inbuf,1,MP3_BUF,file);
buffered=r;
ESP_LOGI(TAG,"play task start %s size %ld resume=%d",path,fsize,resume_sec);
long total_decoded_samples = 0;
if(resume_sec>5){
total_decoded_samples = (long)resume_sec * 16000;
}
while(!G.need_stop){
if(G.is_paused){
if(configured){
close_stream();
configured=false;
sr=0; ch=0;
}
vTaskDelay(pdMS_TO_TICKS(100));
continue;
}
int samples=mp3dec_decode_frame(dec,inbuf,(int)buffered,pcm,&info);
if(samples>0){
total_decoded_samples += samples;
if(info.hz>0) G.pos_sec = total_decoded_samples / info.hz;
else G.pos_sec = total_decoded_samples / 16000;
if(!configured || info.hz!=sr || info.channels!=ch){
ESP_LOGI(TAG,"open_stream sr=%d ch=%d",info.hz,info.channels);
if(!open_stream((uint32_t)info.hz,(uint8_t)info.channels)){
ESP_LOGE(TAG,"open_stream fail %d %d",info.hz,info.channels);
break;
}
ESP_LOGI(TAG,"open_stream ok");
sr=info.hz; ch=info.channels;
configured=true;
if(resume_sec>5 && total_decoded_samples < (long)resume_sec * info.hz + info.hz){
total_decoded_samples = (long)resume_sec * info.hz;
G.pos_sec = resume_sec;
}
}
float vol=G.volume/100.0f;
if(vol<0) vol=0;
if(vol>1) vol=1;
int total_samples = samples * (info.channels>0?info.channels:1);
for(int i=0;i<total_samples;i++){
int32_t s=pcm[i];
s=(int32_t)(s*vol);
if(s>32767) s=32767;
if(s<-32768) s=-32768;
pcm[i]=(int16_t)s;
}
size_t to_write = total_samples * sizeof(int16_t);
size_t off=0;
while(off<to_write && !G.need_stop){
if(G.is_paused){
if(configured){
close_stream();
configured=false;
sr=0; ch=0;
}
break;
}
size_t w=0;
error_t e=audio_stream_write(G.stream_handle,(uint8_t*)pcm+off,to_write-off,&w,pdMS_TO_TICKS(500));
if(e==ERROR_NONE){
off+=w;
} else {
ESP_LOGW(TAG,"audio_write err %d",e);
vTaskDelay(pdMS_TO_TICKS(10));
}
}
if(G.is_paused) continue;
}
if(info.frame_bytes>0 && info.frame_bytes <= (int)buffered){
memmove(inbuf,inbuf+info.frame_bytes,buffered-info.frame_bytes);
buffered-=info.frame_bytes;
} else {
buffered=0;
}
if(buffered < MP3_BUF/2){
size_t nr=fread(inbuf+buffered,1,MP3_BUF-buffered,file);
if(nr>0) buffered+=nr;
else { if(samples==0) break; }
}
if(buffered==0) break;
}
fclose(file);
free(inbuf);
free(pcm);
heap_caps_free(dec);
close_stream();
bool finished_naturally = !G.need_stop;
if(finished_naturally){
ESP_LOGI(TAG,"playback finished naturally for %s, clearing saved pos", ep->slug);
set_saved_pos_for_slug(ep->slug, 0);
G.pos_sec=0;
save_state();
} else {
set_saved_pos_for_slug(ep->slug, G.pos_sec);
save_state();
}
ESP_LOGI(TAG,"play task end finished_naturally=%d", finished_naturally);
G.is_playing=false;
G.play_handle=NULL;
vTaskDelete(NULL);
}
void start_idx_internal(int idx){
if(idx<0||idx>=G.ep_cnt) return;
EpInfo* ep=&G.eps[idx];
if(G.cur_ep_idx!=idx){
int saved = get_saved_pos_for_slug(ep->slug);
if(saved>5){
G.pos_sec=saved;
ESP_LOGI(TAG,"Restoring saved pos %d for %s", saved, ep->slug);
}
G.last_pct=-1;
}
tt_lvgl_lock(portMAX_DELAY);
wait_play_exit();
G.cur_ep_idx=idx; G.cur_season=ep->season;
strncpy(G.cur_title,ep->title,sizeof(G.cur_title)-1);
strncpy(G.last_slug,ep->slug,sizeof(G.last_slug)-1);
G.need_stop=false; G.is_paused=false; G.is_playing=true;
save_state();
tt_lvgl_unlock();
xTaskCreate(play_task,"dm_play",16384,NULL,5,&G.play_handle);
}
void start_idx(int idx){
if(idx<0||idx>=G.ep_cnt) return;
EpInfo* ep=&G.eps[idx];
if(!has_slug(ep->slug)){
stop_playback_sync();
tt_lvgl_lock(portMAX_DELAY);
show_dl_overlay_locked(ep->title, ep->slug, ep->season, ep->ep_num);
if(G.lbl_status) lv_label_set_text(G.lbl_status,"Downloading…");
tt_lvgl_unlock();
if(G.dl_handle){ return; }
xTaskCreate(dl_task,"dm_dl",16384,(void*)(intptr_t)idx,5,&G.dl_handle);
return;
}
start_idx_internal(idx);
}
void select_ep(int idx){
if(idx<0||idx>=G.ep_cnt) return;
if(G.cur_ep_idx>=0 && G.cur_ep_idx < G.ep_cnt && G.cur_ep_idx!=idx){
save_current_ep_pos();
}
G.cur_ep_idx=idx;
EpInfo* ep=&G.eps[idx];
G.cur_season=ep->season;
strncpy(G.last_slug, ep->slug, sizeof(G.last_slug)-1);
int saved = get_saved_pos_for_slug(ep->slug);
if(saved>5){
G.pos_sec=saved;
ESP_LOGI(TAG,"select_ep restore %s pos=%d", ep->slug, saved);
} else {
G.pos_sec=saved;
}
G.last_pct=-1;
G.total_sec=0;
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_title) lv_label_set_text(G.lbl_title,ep->title);
if(G.lbl_meta){
char b[64];
if(saved>5){
char tbuf[16];
fmt_time(tbuf,saved);
snprintf(b,sizeof(b),"S%02d E%02d %s • %s",ep->season,ep->ep_num,ep->seen?"*":"",tbuf);
} else {
snprintf(b,sizeof(b),"S%02d E%02d %s",ep->season,ep->ep_num,ep->seen?"*":"");
}
lv_label_set_text(G.lbl_meta,b);
}
if(G.lbl_status){
if(saved>5) lv_label_set_text(G.lbl_status,"Resume – tap Play");
else lv_label_set_text(G.lbl_status,ep->seen?"On SD – tap Play":"Tap Play to download");
}
if(G.bar){
lv_bar_set_value(G.bar,0,LV_ANIM_OFF);
}
if(G.lbl_time){
if(saved>5){
char a[16];
fmt_time(a,saved);
char buf[40];
snprintf(buf,sizeof(buf),"%s / --:--",a);
lv_label_set_text(G.lbl_time,buf);
} else {
lv_label_set_text(G.lbl_time,"0:00 / --:--");
}
}
tt_lvgl_unlock();
save_state();
}
void go_next(){
if(G.ep_cnt==0) return;
save_current_ep_pos();
save_state();
stop_playback_sync();
int n=G.cur_ep_idx+1;
if(n>=G.ep_cnt) n=0;
select_ep(n);
start_idx(n);
}
void go_prev(){
if(G.ep_cnt==0) return;
if(G.pos_sec>5){
ESP_LOGI(TAG,"go_prev restart current %d sec",G.pos_sec);
if(G.cur_ep_idx>=0){
set_saved_pos_for_slug(G.eps[G.cur_ep_idx].slug, 0);
save_state();
}
stop_playback_sync();
G.pos_sec=0;
int cur=G.cur_ep_idx;
if(cur<0) cur=0;
select_ep(cur);
start_idx(cur);
return;
}
save_current_ep_pos();
save_state();
stop_playback_sync();
int p=G.cur_ep_idx-1;
if(p<0) p=G.ep_cnt-1;
select_ep(p);
start_idx(p);
}
@@ -1,25 +0,0 @@
#pragma once
#include "app_context.h"
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
bool find_audio_device(void);
void close_stream(void);
bool open_stream(uint32_t sr, uint8_t ch);
void wait_play_exit(void);
void stop_playback_sync(void);
void play_task(void* arg);
void start_idx_internal(int idx);
void start_idx(int idx);
void select_ep(int idx);
void go_next(void);
void go_prev(void);
#ifdef __cplusplus
}
#endif
@@ -1,203 +0,0 @@
#include "storage.h"
#include "cJSON.h"
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <dirent.h>
#include <sys/stat.h>
#include <unistd.h>
void make_sd_path(char* out, size_t out_len, const char* slug){
if(!out || !slug) return;
snprintf(out, out_len, "%s/%s.mp3", DM_DIR, slug);
}
void ensure_dir(){
mkdir("/sdcard/apps",0777);
mkdir("/sdcard/apps/one.tactility.discoverymountain",0777);
mkdir("/sdcard/apps/one.tactility.discoverymountain/userdata",0777);
mkdir(DM_DIR,0777);
}
bool has_slug(const char* slug){
char p[300];
make_sd_path(p, sizeof(p), slug);
struct stat st;
return stat(p,&st)==0;
}
void fmt_time(char* o,int s){ snprintf(o,16,"%d:%02d",s/60,s%60); }
void sort_seasons(){
for(int i=0;i<G.season_cnt;i++){
for(int j=i+1;j<G.season_cnt;j++){
if(G.seasons[j].num < G.seasons[i].num){
SeasonInfo t=G.seasons[i]; G.seasons[i]=G.seasons[j]; G.seasons[j]=t;
}
}
}
}
void sort_eps(){
for(int i=0;i<G.ep_cnt;i++){
for(int j=i+1;j<G.ep_cnt;j++){
EpInfo *a=&G.eps[i], *b=&G.eps[j];
bool should_swap=false;
if(a->season != b->season) should_swap = (b->season < a->season);
else should_swap = (b->ep_num < a->ep_num);
if(should_swap){
EpInfo t=*a; *a=*b; *b=t;
}
}
}
}
// ── Per-episode position helpers ──
int find_saved_pos_idx(const char* slug){
if(!slug) return -1;
for(int i=0;i<G.saved_pos_cnt;i++){
if(strcmp(G.saved_pos[i].slug, slug)==0) return i;
}
return -1;
}
int get_saved_pos_for_slug(const char* slug){
int idx=find_saved_pos_idx(slug);
if(idx>=0) return G.saved_pos[idx].pos;
return 0;
}
void set_saved_pos_for_slug(const char* slug, int pos){
if(!slug || strlen(slug)==0) return;
if(pos<0) pos=0;
if(G.total_sec>0 && pos>0 && pos >= G.total_sec-10){
pos=0;
}
int idx=find_saved_pos_idx(slug);
if(idx>=0){
G.saved_pos[idx].pos=pos;
return;
}
if(G.saved_pos_cnt < MAX_EPS){
strncpy(G.saved_pos[G.saved_pos_cnt].slug, slug, sizeof(G.saved_pos[G.saved_pos_cnt].slug)-1);
G.saved_pos[G.saved_pos_cnt].slug[sizeof(G.saved_pos[G.saved_pos_cnt].slug)-1]=0;
G.saved_pos[G.saved_pos_cnt].pos=pos;
G.saved_pos_cnt++;
}
}
void save_current_ep_pos(){
if(G.cur_ep_idx>=0 && G.cur_ep_idx < G.ep_cnt){
const char* slug=G.eps[G.cur_ep_idx].slug;
set_saved_pos_for_slug(slug, G.pos_sec);
} else if(strlen(G.last_slug)>0){
set_saved_pos_for_slug(G.last_slug, G.pos_sec);
}
}
void save_state(){
if(G.cur_ep_idx>=0 && G.ep_cnt>0){
if(G.pos_sec>=0){
set_saved_pos_for_slug(G.eps[G.cur_ep_idx].slug, G.pos_sec);
}
}
cJSON* r=cJSON_CreateObject();
if(G.ep_cnt>0 && G.cur_ep_idx>=0 && G.cur_ep_idx < G.ep_cnt) cJSON_AddStringToObject(r,"last_slug",G.eps[G.cur_ep_idx].slug);
else if(strlen(G.last_slug)>0) cJSON_AddStringToObject(r,"last_slug",G.last_slug);
cJSON_AddNumberToObject(r,"last_season",G.cur_season);
cJSON_AddNumberToObject(r,"pos_sec",G.pos_sec);
if(G.saved_pos_cnt>0){
cJSON* pos_obj=cJSON_CreateObject();
for(int i=0;i<G.saved_pos_cnt;i++){
if(G.saved_pos[i].slug[0]==0) continue;
if(G.saved_pos[i].pos>0){
cJSON_AddNumberToObject(pos_obj, G.saved_pos[i].slug, G.saved_pos[i].pos);
}
}
cJSON_AddItemToObject(r,"positions",pos_obj);
}
char* s=cJSON_PrintUnformatted(r);
if(s){
FILE* f=fopen(STATE_PATH,"w");
if(f){ fwrite(s,1,strlen(s),f); fclose(f);}
free(s);
}
cJSON_Delete(r);
}
void load_state(){
G.cur_season=1; G.cur_ep_idx=-1; G.pos_sec=0;
G.saved_pos_cnt=0;
G.last_save_tick=0;
memset(G.last_slug,0,sizeof(G.last_slug));
memset(G.saved_pos,0,sizeof(G.saved_pos));
FILE* f=fopen(STATE_PATH,"r");
if(!f) return;
fseek(f,0,SEEK_END);
long sz=ftell(f);
fseek(f,0,SEEK_SET);
if(sz<=0||sz>16384){ fclose(f); return;}
char* buf=malloc(sz+1);
if(!buf){ fclose(f); return; }
size_t got=fread(buf,1,sz,f);
buf[got]=0;
fclose(f);
cJSON* r=cJSON_Parse(buf);
free(buf);
if(!r) return;
cJSON* ls=cJSON_GetObjectItem(r,"last_slug");
cJSON* lsn=cJSON_GetObjectItem(r,"last_season");
cJSON* ps=cJSON_GetObjectItem(r,"pos_sec");
if(ps&&cJSON_IsNumber(ps)) G.pos_sec=ps->valueint;
if(lsn&&cJSON_IsNumber(lsn)) G.cur_season=lsn->valueint;
if(ls&&cJSON_IsString(ls)){
strncpy(G.last_slug, ls->valuestring, sizeof(G.last_slug)-1);
for(int i=0;i<G.ep_cnt;i++) if(strcmp(G.eps[i].slug,ls->valuestring)==0){ G.cur_ep_idx=i; G.cur_season=G.eps[i].season; break; }
}
cJSON* poss=cJSON_GetObjectItem(r,"positions");
if(poss && cJSON_IsObject(poss)){
cJSON* child=NULL;
cJSON_ArrayForEach(child, poss){
if(cJSON_IsNumber(child) && child->string && G.saved_pos_cnt < MAX_EPS){
strncpy(G.saved_pos[G.saved_pos_cnt].slug, child->string, sizeof(G.saved_pos[G.saved_pos_cnt].slug)-1);
G.saved_pos[G.saved_pos_cnt].pos=child->valueint;
G.saved_pos_cnt++;
}
}
if(strlen(G.last_slug)>0){
int sp=get_saved_pos_for_slug(G.last_slug);
if(sp>0) G.pos_sec=sp;
}
}
cJSON_Delete(r);
}
void lru_check(){
typedef struct{ char path[300]; time_t mt; } FE;
FE files[200];
int fc=0;
DIR* d=opendir(DM_DIR);
if(!d) return;
struct dirent* de;
char cur_path[300]={0};
if(G.cur_ep_idx>=0 && G.cur_ep_idx<G.ep_cnt){
make_sd_path(cur_path, sizeof(cur_path), G.eps[G.cur_ep_idx].slug);
}
while((de=readdir(d))!=NULL && fc<200){
if(de->d_name[0]=='.') continue;
size_t l=strlen(de->d_name);
if(l<5) continue;
if(strcmp(de->d_name+l-4,".mp3")!=0) continue;
char full[300];
snprintf(full,sizeof(full),"%s/%.200s",DM_DIR,de->d_name);
struct stat st;
if(stat(full,&st)!=0) continue;
if(cur_path[0] && strcmp(full,cur_path)==0) continue;
strncpy(files[fc].path,full,sizeof(files[fc].path)-1);
files[fc].mt=st.st_mtime;
fc++;
}
closedir(d);
if(fc<12) return;
for(int i=0;i<fc;i++){
for(int j=i+1;j<fc;j++){
if(files[j].mt<files[i].mt){ FE t=files[i]; files[i]=files[j]; files[j]=t; }
}
}
int todel=fc-8;
for(int i=0;i<todel;i++) unlink(files[i].path);
}
@@ -1,38 +0,0 @@
#pragma once
#include "app_context.h"
#include <stddef.h>
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
// Path helpers
void make_sd_path(char* out, size_t out_len, const char* slug);
void ensure_dir(void);
bool has_slug(const char* slug);
// LRU cache cleanup
void lru_check(void);
// Sorting
void sort_seasons(void);
void sort_eps(void);
// Time formatting
void fmt_time(char* o, int s);
// Per-episode resume helpers
int find_saved_pos_idx(const char* slug);
int get_saved_pos_for_slug(const char* slug);
void set_saved_pos_for_slug(const char* slug, int pos);
void save_current_ep_pos(void);
// Persistence
void save_state(void);
void load_state(void);
#ifdef __cplusplus
}
#endif
-244
View File
@@ -1,244 +0,0 @@
#include "ui.h"
#include "storage.h"
#include "player.h"
#include "download.h"
#include <tt_lvgl.h>
#include <tactility/lvgl_fonts.h>
#include <string.h>
#include <stdio.h>
#include "esp_log.h"
void close_overlay(){
if(G.overlay){
lv_obj_delete(G.overlay);
G.overlay=NULL;
G.dropdown=NULL;
}
}
void dropdown_cb(lv_event_t* e){
if(lv_event_get_code(e)!=LV_EVENT_VALUE_CHANGED) return;
lv_obj_t* dd = (lv_obj_t*)lv_event_get_target(e);
int sel=lv_dropdown_get_selected(dd);
if(sel<0||sel>=G.season_cnt) return;
int sn=G.seasons[sel].num;
close_overlay();
int first=-1, first_unseen=-1;
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==sn){
if(first==-1) first=i;
if(!G.eps[i].seen && first_unseen==-1) first_unseen=i;
}
int tgt=first_unseen!=-1?first_unseen:first;
if(tgt!=-1){
G.cur_season=sn;
select_ep(tgt);
}
}
void btn_seasons_cb(lv_event_t* e){
(void)e;
if(G.downloading) return;
if(G.overlay){ close_overlay(); return; }
tt_lvgl_lock(portMAX_DELAY);
G.overlay=lv_obj_create(lv_scr_act());
lv_obj_set_size(G.overlay,240,280);
lv_obj_center(G.overlay);
lv_obj_set_style_bg_color(G.overlay,lv_color_hex(0x222222),0);
lv_obj_set_style_radius(G.overlay,12,0);
lv_obj_t* title=lv_label_create(G.overlay);
lv_label_set_text(title,"Select Season");
lv_obj_align(title,LV_ALIGN_TOP_MID,0,8);
static char opts[1024];
opts[0]=0;
for(int i=0;i<G.season_cnt;i++){
char line[16];
if(i==0) snprintf(opts,sizeof(opts),"S%d",G.seasons[i].num);
else { snprintf(line,sizeof(line),"\nS%d",G.seasons[i].num); strncat(opts,line,sizeof(opts)-strlen(opts)-1); }
}
G.dropdown=lv_dropdown_create(G.overlay);
lv_dropdown_set_options(G.dropdown, opts);
lv_obj_set_size(G.dropdown,160,40);
lv_obj_align(G.dropdown,LV_ALIGN_CENTER,0,10);
int sel=0;
for(int i=0;i<G.season_cnt;i++) if(G.seasons[i].num==G.cur_season) sel=i;
lv_dropdown_set_selected(G.dropdown, sel);
lv_obj_add_event_cb(G.dropdown,dropdown_cb,LV_EVENT_VALUE_CHANGED,NULL);
lv_obj_t* bc=lv_btn_create(G.overlay);
lv_obj_set_size(bc,80,32);
lv_obj_align(bc,LV_ALIGN_BOTTOM_MID,0,-8);
lv_obj_t* lc=lv_label_create(bc);
lv_label_set_text(lc,"Close");
lv_obj_center(lc);
lv_obj_add_event_cb(bc,(lv_event_cb_t)close_overlay,LV_EVENT_CLICKED,NULL);
tt_lvgl_unlock();
}
void build_ui(){
tt_lvgl_lock(portMAX_DELAY);
lv_obj_t* root=lv_obj_create(lv_scr_act());
lv_obj_set_size(root,LV_PCT(100),LV_PCT(100));
lv_obj_set_style_bg_color(root,lv_color_hex(0x111111),0);
lv_obj_set_style_pad_all(root,8,0);
lv_obj_set_style_border_width(root,0,0);
lv_obj_remove_flag(root, LV_OBJ_FLAG_SCROLLABLE);
G.lbl_title=lv_label_create(root);
lv_label_set_text(G.lbl_title,"Discovery Mountain");
lv_obj_set_style_text_font(G.lbl_title, lvgl_get_text_font(FONT_SIZE_DEFAULT), 0);
lv_obj_set_width(G.lbl_title,LV_PCT(100));
lv_label_set_long_mode(G.lbl_title,LV_LABEL_LONG_WRAP);
lv_obj_align(G.lbl_title,LV_ALIGN_TOP_LEFT,0,0);
G.lbl_meta=lv_label_create(root);
lv_label_set_text(G.lbl_meta,"Loading...");
lv_obj_set_style_text_color(G.lbl_meta,lv_color_hex(0xAAAAAA),0);
lv_obj_align(G.lbl_meta,LV_ALIGN_TOP_LEFT,0,28);
G.bar=lv_bar_create(root);
lv_obj_set_size(G.bar,LV_PCT(100),12);
lv_obj_align(G.bar,LV_ALIGN_TOP_LEFT,0,50);
lv_bar_set_value(G.bar,0,LV_ANIM_OFF);
G.lbl_time=lv_label_create(root);
lv_label_set_text(G.lbl_time,"0:00 / --:--");
lv_obj_align(G.lbl_time,LV_ALIGN_TOP_LEFT,0,66);
G.lbl_status=lv_label_create(root);
lv_label_set_text(G.lbl_status,"Fetching data...");
lv_obj_set_style_text_color(G.lbl_status,lv_color_hex(0x888888),0);
lv_obj_align(G.lbl_status,LV_ALIGN_TOP_LEFT,0,82);
lv_obj_t* row=lv_obj_create(root);
lv_obj_set_size(row,LV_PCT(100),56);
lv_obj_align(row,LV_ALIGN_TOP_LEFT,0,104);
lv_obj_set_flex_flow(row,LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(row,LV_FLEX_ALIGN_SPACE_EVENLY,LV_FLEX_ALIGN_CENTER,LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_bg_opa(row,LV_OPA_TRANSP,0);
lv_obj_set_style_border_width(row,0,0);
lv_obj_remove_flag(row, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_t* b1=lv_btn_create(row);
lv_obj_set_size(b1,56,48);
lv_obj_t* l1=lv_label_create(b1);
lv_label_set_text(l1,LV_SYMBOL_PREV);
lv_obj_center(l1);
lv_obj_add_event_cb(b1,btn_prev_cb,LV_EVENT_CLICKED,NULL);
G.btn_play=lv_btn_create(row);
lv_obj_set_size(G.btn_play,72,48);
lv_obj_set_style_bg_color(G.btn_play,lv_color_hex(0x2E7D32),0);
lv_obj_t* l2=lv_label_create(G.btn_play);
lv_label_set_text(l2,LV_SYMBOL_PLAY);
lv_obj_center(l2);
lv_obj_add_event_cb(G.btn_play,btn_play_cb,LV_EVENT_CLICKED,NULL);
lv_obj_t* b3=lv_btn_create(row);
lv_obj_set_size(b3,56,48);
lv_obj_t* l3=lv_label_create(b3);
lv_label_set_text(l3,LV_SYMBOL_NEXT);
lv_obj_center(l3);
lv_obj_add_event_cb(b3,btn_next_cb,LV_EVENT_CLICKED,NULL);
lv_obj_t* row2=lv_obj_create(root);
lv_obj_set_size(row2,LV_PCT(100),44);
lv_obj_align(row2,LV_ALIGN_BOTTOM_LEFT,0,0);
lv_obj_set_flex_flow(row2,LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(row2,LV_FLEX_ALIGN_SPACE_EVENLY,LV_FLEX_ALIGN_CENTER,LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_bg_opa(row2,LV_OPA_TRANSP,0);
lv_obj_set_style_border_width(row2,0,0);
lv_obj_remove_flag(row2, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_t* bs=lv_btn_create(row2);
lv_obj_set_size(bs,LV_PCT(100),36);
lv_obj_t* ls=lv_label_create(bs);
lv_label_set_text(ls,"Seasons");
lv_obj_center(ls);
lv_obj_add_event_cb(bs,btn_seasons_cb,LV_EVENT_CLICKED,NULL);
lv_timer_create(ui_timer,200,NULL);
tt_lvgl_unlock();
}
void ui_timer(lv_timer_t* t){
(void)t;
if(G.downloading){
update_dl_overlay_ui();
}
if(G.bar){
if(G.total_sec>0){
int pct=G.pos_sec*100/G.total_sec;
if(pct>100) pct=100;
if(pct!=G.last_pct){
G.last_pct=pct;
lv_bar_set_value(G.bar,pct,LV_ANIM_OFF);
}
} else if(G.is_playing){
int pct=(G.pos_sec%10)*10;
if(pct!=G.last_pct){
G.last_pct=pct;
lv_bar_set_value(G.bar,pct,LV_ANIM_OFF);
}
}
}
if(G.lbl_time){
char a[16],b[16];
fmt_time(a,G.pos_sec);
if(G.total_sec>0) fmt_time(b,G.total_sec);
else snprintf(b,sizeof(b),"--:--");
char buf[40];
snprintf(buf,sizeof(buf),"%s / %s",a,b);
lv_label_set_text(G.lbl_time,buf);
}
if(G.is_playing && G.lbl_status && !G.downloading){
const char* cur = lv_label_get_text(G.lbl_status);
if(!cur || strcmp(cur,"Playing")!=0) lv_label_set_text(G.lbl_status,"Playing");
}
if(G.is_playing && !G.is_paused && !G.downloading){
G.last_save_tick++;
if(G.last_save_tick >= 25){
G.last_save_tick=0;
save_current_ep_pos();
save_state();
}
}
}
void btn_play_cb(lv_event_t* e){
(void)e;
if(G.downloading){
return;
}
if(G.is_playing && !G.is_paused){
G.is_paused=true;
save_current_ep_pos();
save_state();
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status) lv_label_set_text(G.lbl_status,"Paused");
if(G.btn_play){
lv_obj_t* ch=lv_obj_get_child(G.btn_play,0);
if(ch) lv_label_set_text(ch,LV_SYMBOL_PLAY);
}
tt_lvgl_unlock();
} else if(G.is_paused){
G.is_paused=false;
tt_lvgl_lock(portMAX_DELAY);
if(G.btn_play){
lv_obj_t* ch=lv_obj_get_child(G.btn_play,0);
if(ch) lv_label_set_text(ch,LV_SYMBOL_PAUSE);
}
tt_lvgl_unlock();
} else {
start_idx(G.cur_ep_idx);
tt_lvgl_lock(portMAX_DELAY);
if(G.btn_play){
lv_obj_t* ch=lv_obj_get_child(G.btn_play,0);
if(ch) lv_label_set_text(ch,LV_SYMBOL_PAUSE);
}
tt_lvgl_unlock();
}
}
void btn_prev_cb(lv_event_t* e){ (void)e; if(G.downloading) return; go_prev(); }
void btn_next_cb(lv_event_t* e){ (void)e; if(G.downloading) return; go_next(); }
-21
View File
@@ -1,21 +0,0 @@
#pragma once
#include "app_context.h"
#ifdef __cplusplus
extern "C" {
#endif
void build_ui(void);
void ui_timer(lv_timer_t* t);
void btn_play_cb(lv_event_t* e);
void btn_prev_cb(lv_event_t* e);
void btn_next_cb(lv_event_t* e);
void btn_seasons_cb(lv_event_t* e);
void close_overlay(void);
void dropdown_cb(lv_event_t* e);
#ifdef __cplusplus
}
#endif
@@ -1,7 +0,0 @@
manifest.version=0.2
target.sdk=0.8.0-dev
target.platforms=esp32s3
app.id=one.tactility.discoverymountain
app.name=Discovery Mountain
app.version.name=0.1.0
app.version.code=1
+19
View File
@@ -0,0 +1,19 @@
# PipecatVoice — Tactility native voice companion
Native ESP32 ELF app (like VoiceRecorder/ReynaBot), not a web client.
- Uses `audio-stream`/`i2s_controller` APIs (see VoiceRecorder) for 16kHz PCM mic + speaker
- Uses ReynaBot's `websocket.c/h` + `lwip` for WS transport
- Default gateway: Hermes WS `ws://<mac>:8642/api/esp32/voice/ws` (voice profile, limited tools, imperfect STT)
- Optional second target: Pipecat websocket transport `ws://<mac>:7861` (when server exposes `websocket` via `create_transport`)
Build:
```
unset PYTHONPATH; export IDF_PYTHON_ENV_PATH=~/.espressif/python_env/idf5.3_py3.9_env
. ~/esp/esp-idf/export.sh
export TACTILITY_SDK_PATH=~/Projects/electronics/tactility/tactility/Buildscripts/TactilitySDK # or firmware/release/TactilitySDK if built
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/PipecatVoice build esp32s3 --local-sdk
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/PipecatVoice install 192.168.68.112 esp32s3
```
Adapted from ReynaBot (PTT + I2S + WS JSON events: ready/listening/transcript/thinking/response_text/audio_start/audio_end/done/error).
@@ -4,5 +4,5 @@ set(CJSON_SOURCE "$ENV{IDF_PATH}/components/json/cJSON/cJSON.c")
idf_component_register(
SRCS ${SOURCE_FILES} ${CJSON_SOURCE}
INCLUDE_DIRS Source "$ENV{IDF_PATH}/components/json/cJSON"
REQUIRES TactilitySDK esp_http_client lwip
REQUIRES TactilitySDK lwip
)
+830
View File
@@ -0,0 +1,830 @@
#include <tt_app.h>
#include <tt_lvgl.h>
#include <tt_lvgl_toolbar.h>
#include <tt_wifi.h>
#include <tactility/device.h>
#include <tactility/drivers/i2s_controller.h>
#include "websocket.h"
#include <cJSON.h>
#include <string.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <errno.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#define TAG "PipecatVoice"
typedef enum {
STATE_IDLE,
STATE_CONNECTING,
STATE_LISTENING,
STATE_THINKING,
STATE_SPEAKING,
STATE_ERROR
} PipecatVoiceState;
typedef struct {
AppHandle app;
bool visible;
PipecatVoiceState state;
char server_url[128];
char device_id[64];
char api_key[128];
char last_transcript[256];
char last_response[512];
char error_message[128];
// UI components
lv_obj_t* lbl_status;
lv_obj_t* lbl_transcript;
lv_obj_t* lbl_response;
lv_obj_t* btn_ptt;
lv_obj_t* btn_ptt_label;
lv_obj_t* btn_grace;
lv_obj_t* btn_elias;
bool speaker_selected;
lv_obj_t* speaker_select_cont;
lv_obj_t* app_main_cont;
// PTT control
bool is_pressed;
bool start_session;
bool stop_session;
bool cancel_session;
// FreeRTOS Tasks
TaskHandle_t worker_task;
TaskHandle_t rx_task;
// Hardware
struct Device* i2s_dev;
// WebSocket
int ws_fd;
bool ws_connected;
bool ws_done;
bool ui_update_pending;
} PipecatVoiceCtx;
/* ─── Forward Declarations ─── */
static void pipecatvoice_task(void* arg);
static void pipecatvoice_rx_task(void* arg);
static void update_ui(PipecatVoiceCtx* ctx);
static void load_config(PipecatVoiceCtx* ctx);
/* ─── Helper for URL parsing ─── */
static bool parse_ws_url(const char* url, char* host, int* port, char* path) {
if (strncmp(url, "ws://", 5) != 0) return false;
const char* p = url + 5;
const char* colon = strchr(p, ':');
const char* slash = strchr(p, '/');
if (slash == NULL) {
strcpy(path, "/");
} else {
strcpy(path, slash);
}
if (colon != NULL && (slash == NULL || colon < slash)) {
int host_len = colon - p;
memcpy(host, p, host_len);
host[host_len] = '\0';
*port = atoi(colon + 1);
} else {
int host_len = slash ? (slash - p) : strlen(p);
memcpy(host, p, host_len);
host[host_len] = '\0';
*port = 80;
}
return true;
}
/* ─── Config Loading/Saving ─── */
static void load_config(PipecatVoiceCtx* ctx) {
// Default fallback config
snprintf(ctx->server_url, sizeof(ctx->server_url), "ws://192.168.68.102:8642/api/esp32/voice/ws");
snprintf(ctx->device_id, sizeof(ctx->device_id), "pipecatvoice");
snprintf(ctx->api_key, sizeof(ctx->api_key), "mcT1YA1vOr9wXSiHpCYalweEGGZKX-PIfZv2drp8BSg");
char path[256];
size_t path_size = sizeof(path);
tt_app_get_user_data_child_path(ctx->app, "config.json", path, &path_size);
FILE* file = fopen(path, "r");
if (file == NULL) {
// Create user data directory if it doesn't exist
char dir_path[256];
size_t dir_size = sizeof(dir_path);
tt_app_get_user_data_path(ctx->app, dir_path, &dir_size);
mkdir(dir_path, 0755);
// Write default config file
file = fopen(path, "w");
if (file != NULL) {
fprintf(file, "{\n \"server_url\": \"ws://192.168.68.102:8642/api/esp32/voice/ws\",\n \"device_id\": \"pipecatvoice\",\n \"api_key\": \"mcT1YA1vOr9wXSiHpCYalweEGGZKX-PIfZv2drp8BSg\"\n}\n");
fclose(file);
}
ESP_LOGI(TAG, "Created default config.json at %s", path);
return;
}
fseek(file, 0, SEEK_END);
long size = ftell(file);
fseek(file, 0, SEEK_SET);
if (size > 0 && size < 4096) {
char* buf = malloc(size + 1);
if (buf != NULL) {
size_t read_bytes = fread(buf, 1, size, file);
buf[read_bytes] = '\0';
cJSON* json = cJSON_Parse(buf);
if (json != NULL) {
cJSON* url_item = cJSON_GetObjectItem(json, "server_url");
if (url_item != NULL && url_item->valuestring != NULL) {
strncpy(ctx->server_url, url_item->valuestring, sizeof(ctx->server_url) - 1);
}
cJSON* dev_item = cJSON_GetObjectItem(json, "device_id");
if (dev_item != NULL && dev_item->valuestring != NULL) {
strncpy(ctx->device_id, dev_item->valuestring, sizeof(ctx->device_id) - 1);
}
cJSON* key_item = cJSON_GetObjectItem(json, "api_key");
if (key_item != NULL && key_item->valuestring != NULL) {
strncpy(ctx->api_key, key_item->valuestring, sizeof(ctx->api_key) - 1);
}
cJSON_Delete(json);
}
free(buf);
}
}
fclose(file);
ESP_LOGI(TAG, "Loaded config: server=%s device=%s", ctx->server_url, ctx->device_id);
}
/* ─── JSON Message Processing ─── */
static void parse_json_message(PipecatVoiceCtx* ctx, const char* json_str) {
cJSON* json = cJSON_Parse(json_str);
if (json == NULL) return;
cJSON* evt_item = cJSON_GetObjectItem(json, "event");
if (evt_item != NULL && evt_item->valuestring != NULL) {
const char* evt = evt_item->valuestring;
ESP_LOGI(TAG, "WS Event: %s", evt);
if (strcmp(evt, "ready") == 0) {
ESP_LOGI(TAG, "Server ready");
} else if (strcmp(evt, "listening") == 0) {
ctx->state = STATE_LISTENING;
ctx->ui_update_pending = true;
} else if (strcmp(evt, "transcript") == 0) {
cJSON* txt_item = cJSON_GetObjectItem(json, "text");
if (txt_item != NULL && txt_item->valuestring != NULL) {
strncpy(ctx->last_transcript, txt_item->valuestring, sizeof(ctx->last_transcript) - 1);
ctx->ui_update_pending = true;
}
} else if (strcmp(evt, "thinking") == 0) {
ctx->state = STATE_THINKING;
ctx->ui_update_pending = true;
} else if (strcmp(evt, "response_text") == 0) {
cJSON* txt_item = cJSON_GetObjectItem(json, "text");
if (txt_item != NULL && txt_item->valuestring != NULL) {
strncpy(ctx->last_response, txt_item->valuestring, sizeof(ctx->last_response) - 1);
ctx->ui_update_pending = true;
}
} else if (strcmp(evt, "audio_start") == 0) {
ctx->state = STATE_SPEAKING;
ctx->ui_update_pending = true;
// I2S is already configured globally at session startup
} else if (strcmp(evt, "audio_end") == 0) {
ESP_LOGI(TAG, "Audio response ended");
} else if (strcmp(evt, "done") == 0) {
ctx->ws_done = true;
} else if (strcmp(evt, "error") == 0) {
cJSON* msg_item = cJSON_GetObjectItem(json, "message");
if (msg_item != NULL && msg_item->valuestring != NULL) {
strncpy(ctx->error_message, msg_item->valuestring, sizeof(ctx->error_message) - 1);
} else {
snprintf(ctx->error_message, sizeof(ctx->error_message), "Unknown server error");
}
ctx->state = STATE_ERROR;
ctx->ui_update_pending = true;
}
}
cJSON_Delete(json);
}
/* ─── WebSocket RX (Receive) Task ─── */
static void pipecatvoice_rx_task(void* arg) {
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)arg;
uint8_t* rx_buf = malloc(8192);
if (rx_buf == NULL) {
ESP_LOGE(TAG, "Failed to allocate RX buffer");
ctx->ws_done = true;
ctx->rx_task = NULL;
vTaskDelete(NULL);
return;
}
int opcode;
ESP_LOGI(TAG, "WS Receive task started");
while (ctx->ws_fd >= 0) {
int r = ws_recv(ctx->ws_fd, &opcode, rx_buf, 8191);
if (r < 0) {
if (r == -1) {
ESP_LOGI(TAG, "WS connection closed or read error, errno=%d", errno);
} else {
ESP_LOGE(TAG, "WS rx buffer overflow");
}
ctx->ws_done = true;
break;
}
if (opcode == 0x01) { // Text frame (JSON)
rx_buf[r] = '\0';
parse_json_message(ctx, (char*)rx_buf);
} else if (opcode == 0x02) { // Binary frame (Audio data)
if (ctx->state == STATE_SPEAKING && ctx->i2s_dev != NULL) {
const uint8_t* payload_ptr = rx_buf;
size_t payload_len = r;
// Skip WAV header if present in the first chunk
if (payload_len > 44 && memcmp(payload_ptr, "RIFF", 4) == 0) {
payload_ptr += 44;
payload_len -= 44;
}
size_t written = 0;
device_lock(ctx->i2s_dev);
i2s_controller_write(ctx->i2s_dev, payload_ptr, payload_len, &written, pdMS_TO_TICKS(100));
device_unlock(ctx->i2s_dev);
}
} else if (opcode == 0x09) { // PING frame
ESP_LOGI(TAG, "WS PING received, sending PONG");
ws_send_pong(ctx->ws_fd, rx_buf, r);
}
}
free(rx_buf);
ctx->rx_task = NULL;
vTaskDelete(NULL);
}
/* ─── UI Speaker Selection Event Callback ─── */
static void speaker_event_cb(lv_event_t* e) {
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)lv_event_get_user_data(e);
lv_obj_t* target = lv_event_get_target(e);
lv_event_code_t code = lv_event_get_code(e);
if (code == LV_EVENT_CLICKED && !ctx->speaker_selected) {
if (target == ctx->btn_grace) {
strcpy(ctx->device_id, "grace-pipecat");
} else if (target == ctx->btn_elias) {
strcpy(ctx->device_id, "elias-pipecat");
}
ctx->speaker_selected = true;
// Hide selection screen, show main screen
lv_obj_add_flag(ctx->speaker_select_cont, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->app_main_cont, LV_OBJ_FLAG_HIDDEN);
// Trigger UI update to refresh status and PTT button
ctx->ui_update_pending = true;
}
}
/* ─── UI Press-to-Talk Event Callback ─── */
static void ptt_event_cb(lv_event_t* e) {
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)lv_event_get_user_data(e);
lv_event_code_t code = lv_event_get_code(e);
if (code == LV_EVENT_PRESSED) {
ctx->is_pressed = true;
if (ctx->state == STATE_IDLE || ctx->state == ERROR_NONE) {
ctx->start_session = true;
}
} else if (code == LV_EVENT_RELEASED || code == LV_EVENT_PRESS_LOST) {
ctx->is_pressed = false;
if (ctx->state == STATE_LISTENING) {
ctx->stop_session = true;
} else if (ctx->state == STATE_CONNECTING) {
ctx->cancel_session = true;
}
}
}
/* ─── Core UI Update Function ─── */
static void update_ui(PipecatVoiceCtx* ctx) {
if (!ctx->visible) return;
if (tt_lvgl_lock(pdMS_TO_TICKS(500))) {
if (!ctx->visible) {
tt_lvgl_unlock();
return;
}
if (!ctx->speaker_selected) {
lv_label_set_text(ctx->lbl_status, "Choose Speaker");
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x9E9E9E), LV_PART_MAIN);
tt_lvgl_unlock();
return;
}
switch (ctx->state) {
case STATE_IDLE:
lv_label_set_text(ctx->lbl_status, "Ready");
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x9E9E9E), LV_PART_MAIN);
lv_label_set_text(ctx->btn_ptt_label, LV_SYMBOL_AUDIO " Hold to Talk");
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0x6200EE), LV_PART_MAIN);
lv_obj_remove_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
break;
case STATE_CONNECTING:
lv_label_set_text(ctx->lbl_status, "Connecting...");
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0xFFC107), LV_PART_MAIN);
lv_label_set_text(ctx->btn_ptt_label, "Connecting...");
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0x757575), LV_PART_MAIN);
lv_obj_remove_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
break;
case STATE_LISTENING:
lv_label_set_text(ctx->lbl_status, "Listening...");
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x00E676), LV_PART_MAIN);
lv_label_set_text(ctx->btn_ptt_label, "Release to Stop");
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0xD50000), LV_PART_MAIN);
lv_obj_remove_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
break;
case STATE_THINKING:
lv_label_set_text(ctx->lbl_status, "Thinking...");
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x00B0FF), LV_PART_MAIN);
lv_obj_add_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
break;
case STATE_SPEAKING:
lv_label_set_text(ctx->lbl_status, "Speaking...");
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0xAA00FF), LV_PART_MAIN);
lv_obj_add_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
break;
case STATE_ERROR:
lv_label_set_text(ctx->lbl_status, "Error");
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0xFF1744), LV_PART_MAIN);
lv_label_set_text(ctx->btn_ptt_label, LV_SYMBOL_REFRESH " Try Again");
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0xFF1744), LV_PART_MAIN);
lv_obj_remove_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
// Show the full error message in the bot's response bubble
if (ctx->error_message[0] != '\0') {
snprintf(ctx->last_response, sizeof(ctx->last_response), "Error: %s", ctx->error_message);
}
break;
}
lv_label_set_text(ctx->lbl_transcript, ctx->last_transcript[0] != '\0' ? ctx->last_transcript : "(Your question will appear here)");
lv_label_set_text(ctx->lbl_response, ctx->last_response[0] != '\0' ? ctx->last_response : "(Answer will appear here)");
tt_lvgl_unlock();
}
}
/* ─── Streaming & Main Worker Task ─── */
static void pipecatvoice_task(void* arg) {
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)arg;
char host[64];
char path[128];
int port = 80;
ctx->ws_fd = -1;
while (ctx->visible) {
if (ctx->start_session) {
ctx->start_session = false;
ctx->stop_session = false;
ctx->cancel_session = false;
ctx->state = STATE_CONNECTING;
ctx->last_transcript[0] = '\0';
ctx->last_response[0] = '\0';
ctx->error_message[0] = '\0';
ctx->ws_done = false;
update_ui(ctx);
if (!parse_ws_url(ctx->server_url, host, &port, path)) {
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Invalid Server URL");
update_ui(ctx);
continue;
}
ESP_LOGI(TAG, "Connecting: host=%s port=%d path=%s", host, port, path);
int fd = ws_connect(host, port, path, ctx->device_id, ctx->api_key);
if (fd < 0) {
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Connect failed");
update_ui(ctx);
continue;
}
ctx->ws_fd = fd;
ctx->ws_connected = true;
// Configure I2S on demand for this session (16 kHz, 16-bit, mono)
struct I2sConfig session_cfg = {
.communication_format = I2S_FORMAT_STAND_I2S,
.sample_rate = 16000,
.bits_per_sample = 16,
.channel_left = 0,
.channel_right = I2S_CHANNEL_NONE
};
if (ctx->i2s_dev != NULL) {
device_lock(ctx->i2s_dev);
i2s_controller_set_config(ctx->i2s_dev, &session_cfg);
device_unlock(ctx->i2s_dev);
}
// Spawn background RX task to read and parse events
xTaskCreate(pipecatvoice_rx_task, "pipecat_rx", 4096, ctx, 6, &ctx->rx_task);
// Send start event handshake
char start_json[256];
snprintf(start_json, sizeof(start_json),
"{\"event\":\"start\",\"device_id\":\"%s\",\"sample_rate\":16000,\"channels\":1,\"sample_width\":2,\"format\":\"pcm_s16le\"}",
ctx->device_id);
if (ws_send(ctx->ws_fd, (const uint8_t*)start_json, strlen(start_json), false) < 0) {
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Handshake send failed");
ws_close(ctx->ws_fd);
ctx->ws_fd = -1;
ctx->ws_connected = false;
update_ui(ctx);
continue;
}
// Wait for server to switch us to LISTENING
int timeout_ms = 4000;
while (ctx->state != STATE_LISTENING && timeout_ms > 0 && !ctx->cancel_session && !ctx->ws_done) {
vTaskDelay(pdMS_TO_TICKS(50));
timeout_ms -= 50;
}
if (ctx->state != STATE_LISTENING) {
if (ctx->state != STATE_ERROR) {
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Handshake timeout");
}
ws_close(ctx->ws_fd);
ctx->ws_fd = -1;
ctx->ws_connected = false;
update_ui(ctx);
continue;
}
// I2S is already configured globally at session startup
uint8_t* buffer = malloc(1024);
if (buffer == NULL) {
ESP_LOGE(TAG, "Failed to allocate record buffer");
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Out of memory");
ws_close(ctx->ws_fd);
ctx->ws_fd = -1;
ctx->ws_connected = false;
update_ui(ctx);
continue;
}
size_t total_sent_bytes = 0;
// Stream audio loop while PTT is held
while (ctx->is_pressed && !ctx->stop_session && !ctx->cancel_session && !ctx->ws_done && total_sent_bytes < 320000) {
size_t bytes_read = 0;
error_t r = i2s_controller_read(ctx->i2s_dev, buffer, 1024, &bytes_read, pdMS_TO_TICKS(100));
if (r == ERROR_NONE && bytes_read > 0) {
if (ws_send(ctx->ws_fd, buffer, bytes_read, true) < 0) {
ESP_LOGE(TAG, "Audio stream send failed");
break;
}
total_sent_bytes += bytes_read;
}
}
free(buffer);
if (ctx->cancel_session || total_sent_bytes < 3200) {
ESP_LOGI(TAG, "Cancelling audio session");
const char* cancel_json = "{\"event\":\"cancel\"}";
ws_send(ctx->ws_fd, (const uint8_t*)cancel_json, strlen(cancel_json), false);
ctx->state = STATE_IDLE;
} else {
const char* stop_json = "{\"event\":\"stop\"}";
ws_send(ctx->ws_fd, (const uint8_t*)stop_json, strlen(stop_json), false);
ctx->state = STATE_THINKING;
update_ui(ctx);
// Wait for response to finish (increase to 90s to match socket timeout)
int wait_timeout_ms = 90000;
while (!ctx->ws_done && ctx->state != STATE_ERROR && wait_timeout_ms > 0) {
vTaskDelay(pdMS_TO_TICKS(100));
wait_timeout_ms -= 100;
if (ctx->ui_update_pending) {
ctx->ui_update_pending = false;
update_ui(ctx);
}
}
if (!ctx->ws_done && ctx->state != STATE_ERROR) {
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Response timeout");
}
}
// Clean up socket
int fd_to_close = ctx->ws_fd;
ctx->ws_fd = -1;
ctx->ws_connected = false;
ws_close(fd_to_close);
// Reset I2S controller to release DMA and stop white noise
if (ctx->i2s_dev != NULL) {
device_lock(ctx->i2s_dev);
i2s_controller_reset(ctx->i2s_dev);
device_unlock(ctx->i2s_dev);
}
// Wait for receive task to exit
int rx_timeout = 100;
while (ctx->rx_task != NULL && rx_timeout > 0) {
vTaskDelay(pdMS_TO_TICKS(10));
rx_timeout--;
}
if (ctx->state != STATE_ERROR) {
ctx->state = STATE_IDLE;
}
update_ui(ctx);
}
vTaskDelay(pdMS_TO_TICKS(100));
if (ctx->ui_update_pending) {
ctx->ui_update_pending = false;
update_ui(ctx);
}
}
ctx->worker_task = NULL;
vTaskDelete(NULL);
}
/* ─── App Lifecycle ─── */
static void* create_data(void) {
PipecatVoiceCtx* ctx = calloc(1, sizeof(PipecatVoiceCtx));
if (ctx != NULL) {
ctx->state = STATE_IDLE;
ctx->ws_fd = -1;
}
return ctx;
}
static void destroy_data(void* data) {
free(data);
}
static void on_create(AppHandle app, void* data) {
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)data;
ctx->app = app;
}
static void on_destroy(AppHandle app, void* data) {
// No-op
}
static void on_show(AppHandle app, void* data, lv_obj_t* parent) {
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)data;
ctx->visible = true;
load_config(ctx);
// Find I2S controller
ctx->i2s_dev = device_find_by_name("i2s0");
if (ctx->i2s_dev == NULL) {
ESP_LOGE(TAG, "I2S controller 'i2s0' not found!");
}
// Style the parent screen
lv_obj_set_flex_flow(parent, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(parent, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_all(parent, 10, LV_PART_MAIN);
lv_obj_set_style_pad_row(parent, 12, LV_PART_MAIN);
lv_obj_set_style_bg_color(parent, lv_color_hex(0x0C0B12), LV_PART_MAIN); // Rich dark background
// Add App Toolbar
lv_obj_t* toolbar = tt_lvgl_toolbar_create_for_app(parent, app);
lv_obj_align(toolbar, LV_ALIGN_TOP_MID, 0, 0);
// Status Label inside toolbar (replaces the static "ReynaBot Voice" label)
ctx->lbl_status = lv_label_create(toolbar);
lv_label_set_text(ctx->lbl_status, "Ready");
lv_obj_add_flag(ctx->lbl_status, LV_OBJ_FLAG_FLOATING);
lv_obj_align(ctx->lbl_status, LV_ALIGN_RIGHT_MID, -10, 0);
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x9E9E9E), LV_PART_MAIN);
// Initial state
ctx->speaker_selected = false;
// Default to Grace if device_id is not already configured to elias-esp32
if (strcmp(ctx->device_id, "elias-pipecat") != 0) {
strcpy(ctx->device_id, "grace-pipecat");
}
// 1. Speaker Selection Container (Startup Screen)
ctx->speaker_select_cont = lv_obj_create(parent);
lv_obj_set_width(ctx->speaker_select_cont, lv_pct(95));
lv_obj_set_flex_grow(ctx->speaker_select_cont, 1);
lv_obj_set_flex_flow(ctx->speaker_select_cont, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(ctx->speaker_select_cont, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_all(ctx->speaker_select_cont, 0, LV_PART_MAIN);
lv_obj_set_style_bg_opa(ctx->speaker_select_cont, 0, LV_PART_MAIN);
lv_obj_set_style_border_width(ctx->speaker_select_cont, 0, LV_PART_MAIN);
lv_obj_set_style_pad_row(ctx->speaker_select_cont, 16, LV_PART_MAIN);
// Title Prompt
lv_obj_t* prompt_lbl = lv_label_create(ctx->speaker_select_cont);
lv_label_set_text(prompt_lbl, "Who is speaking?");
lv_obj_set_style_text_color(prompt_lbl, lv_color_hex(0xFFFFFF), LV_PART_MAIN);
// Row of two selection buttons
lv_obj_t* speaker_row = lv_obj_create(ctx->speaker_select_cont);
lv_obj_set_size(speaker_row, lv_pct(100), 140);
lv_obj_set_flex_flow(speaker_row, LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(speaker_row, LV_FLEX_ALIGN_SPACE_BETWEEN, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_all(speaker_row, 0, LV_PART_MAIN);
lv_obj_set_style_bg_opa(speaker_row, 0, LV_PART_MAIN);
lv_obj_set_style_border_width(speaker_row, 0, LV_PART_MAIN);
// Grace Button (Girl)
ctx->btn_grace = lv_btn_create(speaker_row);
lv_obj_set_size(ctx->btn_grace, lv_pct(46), 130);
lv_obj_set_flex_flow(ctx->btn_grace, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(ctx->btn_grace, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_all(ctx->btn_grace, 8, LV_PART_MAIN);
lv_obj_set_style_pad_row(ctx->btn_grace, 8, LV_PART_MAIN);
lv_obj_set_style_radius(ctx->btn_grace, 12, LV_PART_MAIN);
lv_obj_set_style_bg_color(ctx->btn_grace, lv_color_hex(0x1C1B22), LV_PART_MAIN);
lv_obj_set_style_border_width(ctx->btn_grace, 1, LV_PART_MAIN);
lv_obj_set_style_border_color(ctx->btn_grace, lv_color_hex(0x2D2B36), LV_PART_MAIN);
lv_obj_add_event_cb(ctx->btn_grace, speaker_event_cb, LV_EVENT_CLICKED, ctx);
lv_obj_t* icon_grace = lv_image_create(ctx->btn_grace);
lv_obj_set_size(icon_grace, 80, 80);
char path_grace[256] = "A:";
size_t sz_grace = sizeof(path_grace) - 2;
tt_app_get_assets_child_path(ctx->app, "girl.png", path_grace + 2, &sz_grace);
lv_image_set_src(icon_grace, path_grace);
lv_obj_t* lbl_grace = lv_label_create(ctx->btn_grace);
lv_label_set_text(lbl_grace, "Grace");
lv_obj_set_style_text_color(lbl_grace, lv_color_hex(0xFFFFFF), LV_PART_MAIN);
// Elias Button (Boy)
ctx->btn_elias = lv_btn_create(speaker_row);
lv_obj_set_size(ctx->btn_elias, lv_pct(46), 130);
lv_obj_set_flex_flow(ctx->btn_elias, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(ctx->btn_elias, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_all(ctx->btn_elias, 8, LV_PART_MAIN);
lv_obj_set_style_pad_row(ctx->btn_elias, 8, LV_PART_MAIN);
lv_obj_set_style_radius(ctx->btn_elias, 12, LV_PART_MAIN);
lv_obj_set_style_bg_color(ctx->btn_elias, lv_color_hex(0x1C1B22), LV_PART_MAIN);
lv_obj_set_style_border_width(ctx->btn_elias, 1, LV_PART_MAIN);
lv_obj_set_style_border_color(ctx->btn_elias, lv_color_hex(0x2D2B36), LV_PART_MAIN);
lv_obj_add_event_cb(ctx->btn_elias, speaker_event_cb, LV_EVENT_CLICKED, ctx);
lv_obj_t* icon_elias = lv_image_create(ctx->btn_elias);
lv_obj_set_size(icon_elias, 80, 80);
char path_elias[256] = "A:";
size_t sz_elias = sizeof(path_elias) - 2;
tt_app_get_assets_child_path(ctx->app, "boy.png", path_elias + 2, &sz_elias);
lv_image_set_src(icon_elias, path_elias);
lv_obj_t* lbl_elias = lv_label_create(ctx->btn_elias);
lv_label_set_text(lbl_elias, "Elias");
lv_obj_set_style_text_color(lbl_elias, lv_color_hex(0xFFFFFF), LV_PART_MAIN);
// 2. Main App Container (Usual App Screen)
ctx->app_main_cont = lv_obj_create(parent);
lv_obj_set_width(ctx->app_main_cont, lv_pct(95));
lv_obj_set_flex_grow(ctx->app_main_cont, 1);
lv_obj_set_flex_flow(ctx->app_main_cont, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(ctx->app_main_cont, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_all(ctx->app_main_cont, 0, LV_PART_MAIN);
lv_obj_set_style_pad_row(ctx->app_main_cont, 12, LV_PART_MAIN);
lv_obj_set_style_bg_opa(ctx->app_main_cont, 0, LV_PART_MAIN);
lv_obj_set_style_border_width(ctx->app_main_cont, 0, LV_PART_MAIN);
lv_obj_add_flag(ctx->app_main_cont, LV_OBJ_FLAG_HIDDEN); // Hidden by default
// Conversation bubble container
lv_obj_t* conv_card = lv_obj_create(ctx->app_main_cont);
lv_obj_set_width(conv_card, lv_pct(100));
lv_obj_set_flex_grow(conv_card, 1);
lv_obj_set_flex_flow(conv_card, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(conv_card, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START);
lv_obj_set_style_pad_all(conv_card, 12, LV_PART_MAIN);
lv_obj_set_style_pad_row(conv_card, 10, LV_PART_MAIN);
lv_obj_set_style_bg_color(conv_card, lv_color_hex(0x13121A), LV_PART_MAIN);
lv_obj_set_style_border_color(conv_card, lv_color_hex(0x23212C), LV_PART_MAIN);
lv_obj_set_style_border_width(conv_card, 1, LV_PART_MAIN);
lv_obj_set_style_radius(conv_card, 12, LV_PART_MAIN);
// User bubble
lv_obj_t* user_label_title = lv_label_create(conv_card);
lv_label_set_text(user_label_title, "👤 You:");
lv_obj_set_style_text_color(user_label_title, lv_color_hex(0x00E676), LV_PART_MAIN);
ctx->lbl_transcript = lv_label_create(conv_card);
lv_label_set_text(ctx->lbl_transcript, "(Your question will appear here)");
lv_label_set_long_mode(ctx->lbl_transcript, LV_LABEL_LONG_WRAP);
lv_obj_set_width(ctx->lbl_transcript, lv_pct(100));
lv_obj_set_style_text_color(ctx->lbl_transcript, lv_color_hex(0xE0E0E0), LV_PART_MAIN);
// Separator line
lv_obj_t* sep = lv_line_create(conv_card);
static lv_point_precise_t line_points[] = { {0, 0}, {300, 0} };
lv_line_set_points(sep, line_points, 2);
lv_obj_set_style_line_color(sep, lv_color_hex(0x2D2B36), LV_PART_MAIN);
lv_obj_set_style_line_width(sep, 1, LV_PART_MAIN);
lv_obj_set_width(sep, lv_pct(100));
// Assistant bubble
lv_obj_t* bot_label_title = lv_label_create(conv_card);
lv_label_set_text(bot_label_title, "🤖 Pipecat:");
lv_obj_set_style_text_color(bot_label_title, lv_color_hex(0xAA00FF), LV_PART_MAIN);
ctx->lbl_response = lv_label_create(conv_card);
lv_label_set_text(ctx->lbl_response, "(Answer will appear here)");
lv_label_set_long_mode(ctx->lbl_response, LV_LABEL_LONG_WRAP);
lv_obj_set_width(ctx->lbl_response, lv_pct(100));
lv_obj_set_style_text_color(ctx->lbl_response, lv_color_hex(0xE0E0E0), LV_PART_MAIN);
// Push-to-Talk Button
ctx->btn_ptt = lv_btn_create(ctx->app_main_cont);
lv_obj_set_size(ctx->btn_ptt, 180, 50);
lv_obj_set_style_radius(ctx->btn_ptt, 25, LV_PART_MAIN);
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0x6200EE), LV_PART_MAIN);
ctx->btn_ptt_label = lv_label_create(ctx->btn_ptt);
lv_label_set_text(ctx->btn_ptt_label, LV_SYMBOL_AUDIO " Hold to Talk");
lv_obj_center(ctx->btn_ptt_label);
lv_obj_add_event_cb(ctx->btn_ptt, ptt_event_cb, LV_EVENT_ALL, ctx);
// Launch main worker task
xTaskCreate(pipecatvoice_task, "pipecat_worker", 6144, ctx, 5, &ctx->worker_task);
}
static void on_hide(AppHandle app, void* data) {
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)data;
if (ctx == NULL) return;
ESP_LOGI(TAG, "on_hide: cleaning up");
ctx->visible = false;
if (ctx->ws_fd >= 0) {
int fd_to_close = ctx->ws_fd;
ctx->ws_fd = -1;
ws_close(fd_to_close);
}
// Reset I2S controller to stop DMA and looping noise
if (ctx->i2s_dev != NULL) {
device_lock(ctx->i2s_dev);
i2s_controller_reset(ctx->i2s_dev);
device_unlock(ctx->i2s_dev);
}
// Wait briefly for tasks to exit
int timeout = 100;
while ((ctx->worker_task != NULL || ctx->rx_task != NULL) && timeout > 0) {
vTaskDelay(pdMS_TO_TICKS(10));
timeout--;
}
}
int main(int argc, char* argv[]) {
(void)argc;
(void)argv;
tt_app_register((AppRegistration) {
.createData = create_data,
.destroyData = destroy_data,
.onCreate = on_create,
.onDestroy = on_destroy,
.onShow = on_show,
.onHide = on_hide
});
return 0;
}
+240
View File
@@ -0,0 +1,240 @@
#include "websocket.h"
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <lwip/sockets.h>
#include <lwip/inet.h>
#include <esp_log.h>
static int recv_all(int fd, void* buf, size_t len) {
size_t total = 0;
char* p = (char*)buf;
while (total < len) {
int r = lwip_recv(fd, p + total, len - total, 0);
if (r <= 0) {
return -1;
}
total += r;
}
return 0;
}
static uint16_t my_htons(uint16_t val) {
return (uint16_t)(((val & 0xff) << 8) | ((val & 0xff00) >> 8));
}
int ws_connect(const char* host, int port, const char* path, const char* device_id, const char* auth_key) {
int fd = lwip_socket(AF_INET, SOCK_STREAM, 0);
if (fd < 0) return -1;
struct sockaddr_in addr;
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = my_htons(port);
addr.sin_addr.s_addr = ipaddr_addr(host);
if (lwip_connect(fd, (struct sockaddr*)&addr, sizeof(addr)) < 0) {
close(fd);
return -1;
}
// Set socket receive timeout (e.g. 90 seconds) to prevent blocking indefinitely
struct timeval tv;
tv.tv_sec = 90;
tv.tv_usec = 0;
lwip_setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));
// Send HTTP upgrade handshake request
char req[1024];
snprintf(req, sizeof(req),
"GET %s HTTP/1.1\r\n"
"Host: %s:%d\r\n"
"Upgrade: websocket\r\n"
"Connection: Upgrade\r\n"
"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n"
"Sec-WebSocket-Version: 13\r\n"
"Authorization: Bearer %s\r\n"
"X-Device-ID: %s\r\n"
"\r\n",
path, host, port, auth_key, device_id);
if (lwip_send(fd, req, strlen(req), 0) < 0) {
close(fd);
return -1;
}
// Read HTTP response headers until we hit "\r\n\r\n"
char header_buf[1024];
size_t header_len = 0;
while (header_len < sizeof(header_buf) - 1) {
char c;
int r = lwip_recv(fd, &c, 1, 0);
if (r <= 0) {
close(fd);
return -1;
}
header_buf[header_len++] = c;
header_buf[header_len] = '\0';
if (header_len >= 4 && strcmp(header_buf + header_len - 4, "\r\n\r\n") == 0) {
break;
}
}
// Verify HTTP 101 Switching Protocols response status
if (strstr(header_buf, "HTTP/1.1 101") == NULL && strstr(header_buf, "HTTP/1.0 101") == NULL) {
close(fd);
return -1;
}
return fd;
}
int ws_send(int fd, const uint8_t* data, size_t len, bool binary) {
uint8_t header[10];
size_t header_len = 0;
header[0] = binary ? 0x82 : 0x81;
if (len < 126) {
header[1] = 0x80 | (uint8_t)len;
header_len = 2;
} else {
header[1] = 0x80 | 126;
header[2] = (uint8_t)((len >> 8) & 0xFF);
header[3] = (uint8_t)(len & 0xFF);
header_len = 4;
}
// Use fixed client mask for performance: 0x12, 0x34, 0x56, 0x78
uint8_t mask[4] = { 0x12, 0x34, 0x56, 0x78 };
memcpy(header + header_len, mask, 4);
header_len += 4;
// Send WebSocket frame header
int sent = lwip_send(fd, header, header_len, 0);
if (sent < 0) return -1;
// Mask the payload
uint8_t* masked = malloc(len);
if (masked == NULL) return -1;
for (size_t i = 0; i < len; ++i) {
masked[i] = data[i] ^ mask[i % 4];
}
// Send masked payload
sent = lwip_send(fd, masked, len, 0);
free(masked);
return sent >= 0 ? 0 : -1;
}
int ws_recv(int fd, int* out_opcode, uint8_t* payload, size_t max_len) {
uint8_t header[2];
if (recv_all(fd, header, 2) < 0) {
return -1;
}
int opcode = header[0] & 0x0F;
if (out_opcode != NULL) {
*out_opcode = opcode;
}
int masked = (header[1] & 0x80) != 0;
size_t len = header[1] & 0x7F;
if (len == 126) {
uint8_t ext_len[2];
if (recv_all(fd, ext_len, 2) < 0) return -1;
len = ((size_t)ext_len[0] << 8) | ext_len[1];
} else if (len == 127) {
uint8_t ext_len[8];
if (recv_all(fd, ext_len, 8) < 0) return -1;
// Parse 64-bit length into size_t
len = ((size_t)ext_len[4] << 24) | ((size_t)ext_len[5] << 16) | ((size_t)ext_len[6] << 8) | ext_len[7];
}
if (masked) {
uint8_t mask[4];
if (recv_all(fd, mask, 4) < 0) return -1;
if (len > max_len) {
ESP_LOGE("websocket", "ws_recv overflow (masked): len=%u, max_len=%u", (unsigned)len, (unsigned)max_len);
// Buffer overflow, skip payload to align stream
size_t to_discard = len;
uint8_t discard_buf[256];
while (to_discard > 0) {
size_t chunk = to_discard < sizeof(discard_buf) ? to_discard : sizeof(discard_buf);
if (recv_all(fd, discard_buf, chunk) < 0) return -1;
to_discard -= chunk;
}
return -2;
}
if (recv_all(fd, payload, len) < 0) return -1;
for (size_t i = 0; i < len; ++i) {
payload[i] ^= mask[i % 4];
}
} else {
if (len > max_len) {
ESP_LOGE("websocket", "ws_recv overflow (unmasked): len=%u, max_len=%u", (unsigned)len, (unsigned)max_len);
size_t to_discard = len;
uint8_t discard_buf[256];
while (to_discard > 0) {
size_t chunk = to_discard < sizeof(discard_buf) ? to_discard : sizeof(discard_buf);
if (recv_all(fd, discard_buf, chunk) < 0) return -1;
to_discard -= chunk;
}
return -2;
}
if (recv_all(fd, payload, len) < 0) return -1;
}
return (int)len;
}
void ws_close(int fd) {
if (fd >= 0) {
close(fd);
}
}
int ws_send_pong(int fd, const uint8_t* payload, size_t len) {
uint8_t header[10];
size_t header_len = 0;
header[0] = 0x8A; // FIN | PONG (0x0A)
if (len < 126) {
header[1] = 0x80 | (uint8_t)len;
header_len = 2;
} else {
header[1] = 0x80 | 126;
header[2] = (uint8_t)((len >> 8) & 0xFF);
header[3] = (uint8_t)(len & 0xFF);
header_len = 4;
}
uint8_t mask[4] = { 0x12, 0x34, 0x56, 0x78 };
memcpy(header + header_len, mask, 4);
header_len += 4;
int sent = lwip_send(fd, header, header_len, 0);
if (sent < 0) return -1;
if (len > 0 && payload != NULL) {
uint8_t* masked = malloc(len);
if (masked == NULL) return -1;
for (size_t i = 0; i < len; ++i) {
masked[i] = payload[i] ^ mask[i % 4];
}
sent = lwip_send(fd, masked, len, 0);
free(masked);
}
return sent >= 0 ? 0 : -1;
}
+59
View File
@@ -0,0 +1,59 @@
#pragma once
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
/**
* Connect to a WebSocket server.
* @param host Server IP address (e.g. "192.168.68.126")
* @param port Port number (e.g. 8642)
* @param path WebSocket path (e.g. "/api/esp32/voice/ws")
* @param device_id Unique device identifier
* @param auth_key Hermes Bearer API key
* @return Socket file descriptor on success, or -1 on failure
*/
int ws_connect(const char* host, int port, const char* path, const char* device_id, const char* auth_key);
/**
* Send a WebSocket frame.
* @param fd Socket file descriptor
* @param data Data payload to send
* @param len Length of the data payload
* @param binary True for binary frame, false for text frame
* @return 0 on success, or -1 on failure
*/
int ws_send(int fd, const uint8_t* data, size_t len, bool binary);
/**
* Receive a WebSocket frame.
* @param fd Socket file descriptor
* @param out_opcode Pointer to store the received opcode (e.g. 0x01 text, 0x02 binary)
* @param payload Buffer to store the received payload
* @param max_len Maximum length of the payload buffer
* @return Received payload length on success, -1 on connection failure, or -2 on buffer overflow
*/
int ws_recv(int fd, int* out_opcode, uint8_t* payload, size_t max_len);
/**
* Close a WebSocket connection.
* @param fd Socket file descriptor
*/
void ws_close(int fd);
/**
* Send a WebSocket PONG frame.
* @param fd Socket file descriptor
* @param payload Payload to reflect
* @param len Length of payload
* @return 0 on success, or -1 on failure
*/
int ws_send_pong(int fd, const uint8_t* payload, size_t len);
#ifdef __cplusplus
}
#endif
+7
View File
@@ -0,0 +1,7 @@
manifest.version=0.2
target.sdk=0.8.0-dev
target.platforms=esp32s3
app.id=one.tactility.pipecatvoice
app.version.name=1.0.0
app.version.code=1
app.name=Pipecat Voice