16 Commits

Author SHA1 Message Date
Adolfo 7ef8e272fc feat(mp3,voice): fix scroll anim audio glitch, add seek/volume/history UI
- Mp3Player:
  * Disable LV_LABEL_LONG_SCROLL_CIRCULAR during playback (flag ENABLE_TITLE_SCROLL_ANIM_WHEN_IDLE) to avoid screen invalidation starving audio_stream
  * Single-line controls [-10s][Play/Pause][Stop][+10s] icon-only, volume slider at bottom, no card/toolbar, time label MM:SS / MM:SS updating every sec
  * Seek ±10s with proportional byte calc and correct sample count (was byte vs sample bug)
  * Safe history persistence: heap buffer (not stack), recursive mkdir, play_history.txt and volume.txt in user data
  * History only for >=10min audios, updated on pause/stop/hide, auto-resume latest
  * Full-screen history screen with Back button, open via top-right list icon, playing file highlighted CHECKED, tap to resume long or restart short

- VoiceRecorder:
  * Fix crash from lv_label_set_long_mode on icon child in list buttons (force DOT only on last child label)
  * Force DOT on status and memo list to avoid scroll anim during audio
  * Throttle UI updates (1Hz rec, %/sec change play)

Tested on 192.168.68.107, no crash, heap free ~15KB, screenshot verified
2026-07-30 15:00:34 -04:00
Adolfo eb364cb34f Scale Immich photos to cover the display 2026-07-29 22:00:18 -04:00
Adolfo eaccebc95d feat(DM+SdDDL): sequential full-season download queue, fix close crash, return true
- DM: add dl_missing_* fields to AppCtx for sequential queue
- DM player: count_missing_in_season, find_next_missing, download_episode, download_missing_season_start/next
- DM main onResult: auto-continue queue when dl_missing_active, updates status DL X/Y, finishes with done ok/fail
- SdDownloader: file exists returns true (Already on SD) with total=file size, not false
- SdDownloader: set_result_and_close minimal bundle (only success bool) to avoid internal heap OOM crash on close
- SdDownloader: Cancel button becomes Close after done, calls tt_app_stop() directly (manual close per request, auto-close unreliable)
- Fixes crash on close observed in demo mode (file exists) and after download done

Tested on 192.168.68.133: SdDownloader shows Already on SD + Close, DM shows S10 5/6 after sequential, no crash on open/close cycles
2026-07-25 22:49:28 -04:00
Adolfo 3778cb3399 feat(DM): library UI with episode list, SD status, cache, close button, stable single-file DL
- DiscoveryMountain: new UI optimized for episode list (not player anymore)
  - Shows episodes for current season, indicates if on SD card (green=On SD, gray=Not DL)
  - Per-episode action: download single via SdDownloader or play via Mp3Player
  - DL Missing button for missing eps in season (currently single first missing, batch reverted)
  - Season selector screen with cached/total counts
  - Top-right X close button
  - Seasons/episodes cached to cache.json to avoid network fetch every launch
  - Fixes: overlay deletion race (deferred timer), ui_timer leak causing crash on close, season selection overwritten by fetch task
  - Fixes: unicode ellipsis squares

- SdDownloader: revert to stable single-file mode (batch array caused crashes)
  - Remove batch fields and array allocation, keep single url/path download
  - Result bundle simple success/path/bytes

- Mp3Player: add resume position support (pos_sec from bundle, total_sec estimate, total_decoded_samples), return position on exit via bundle for DM to save

Tested on 192.168.68.133: opens, shows S10 2/6, Play/DL buttons, close works, no crash on open/close cycles, cache shows 'Loaded from cache, refreshing...'
2026-07-25 22:06:43 -04:00
Adolfo 6a1fe5dda6 feat(ImmichElias): add person selector like Bible books, 4 persons Elias/Grace/Alicia/Adolfo, picker UI with toolbar, player bottom-only, random pagination, cache, fullscreen 2026-07-25 16:29:24 -04:00
Adolfo d06f8ddbb5 fix(ImmichElias): black screen fix, 320x240 request for PSRAM stability (480x320 OOM), fullscreen widget 480x320 bg, bottom-only UI, tap toggle 2026-07-25 16:14:34 -04:00
Adolfo a2bb8e5132 feat(ImmichElias): 2x render 480x320 fullscreen bg, display detection, bottom-only UI, PSRAM 2026-07-25 16:11:29 -04:00
Adolfo 9ee0599bec fix(ImmichElias): black screen fix, 240x180 request for PSRAM memory safety, fullscreen bg widget 480x320, bottom-only UI, tap toggle, cache random pagination 2026-07-25 16:09:45 -04:00
Adolfo c3689e82cc fix(ImmichElias): black screen fix, clear hidden flag, fullscreen 480x320 bg, bottom-only UI tap toggle, filter VIDEO, random pagination 100, cache 2026-07-25 16:05:59 -04:00
Adolfo 4c71c77bf6 fix(ImmichElias): show >50 photos via random pagination, filter IMAGE only, increase to 100, JSON 128KB, slideshow + cache 2026-07-25 16:00:18 -04:00
Adolfo 0595e149d4 fix(ImmichElias): bottom-only UI like Bible, fullscreen 480x320 bg, tap toggles chrome, PSRAM, random, cache with offline fallback, filter IMAGE only, proxy fallback 320x240, slideshow 60s, crash fixes for touch 2026-07-25 15:53:37 -04:00
Adolfo 995297314d fix(ImmichElias): bottom-only UI like Bible, fullscreen bg, tap to toggle chrome, dynamic 480x320, cache, random, slideshow 60s, crash fixes 2026-07-25 15:47:08 -04:00
Adolfo 349034b8ba fix(ImmichElias): fullscreen background 480x320, BookPlayer UI, no crop when hidden, PSRAM for images, random, cache offline, slideshow 60s, crash fixes 2026-07-25 15:39:08 -04:00
Adolfo 3085bac5f1 fix(ImmichElias): merge dynamic sizing 480x320 + crash fix, BookPlayer UI, random, cache offline, slideshow 60s 2026-07-25 15:24:29 -04:00
Hermes Reyna Bot b03cb2b244 fix(ImmichElias): PNG via resize proxy 8106 for real photos, fixes WebP crash, device 133 yellow case
- Proxy already fixed on FamReynaServer 192.168.68.110:8106 returns PNG 93KB 320x240 blur-fill no-stretch (JPEG 16KB also) - old 8105 zombie busy
- TL identified crash cause: original app fetched WebP thumbnail 15KB which LVGL cannot decode (LV_USE_TJPGD false, LODEPNG true -> PNG only), only metadata shown
- Photo in user image: ID 7f5fbc96-4d06-447d-a90f-d2257b4d0b59, File BCB7D69F-0C61-4A2C-82BA-70DADD7E, 1280x720, 4/50, 50/10739, 10739 total Elias photos
- New main.c 219+: RESIZE_PROXY_URL 8106, /data/immich_cur.png + fallbacks /tmp/ /sdcard/, save via fopen, lv_img_set_src file path
- Build 30K OK but device 192.168.68.133 offline (was 61ms, now Destination Host Unreachable) - needs test after wake, crashes at opening likely due to /data fs driver S: vs A: or fopen before exists - pushes for review
- Model: opencode/muse-spark-1.1 via api.meta.ai same key, tmux visible engineer window 2:engineer on iMac .124, persistent session opencode handles cache better per user
- Branch feature/immich-elias-client
2026-07-25 13:40:50 -04:00
Hermes Reyna Bot 3a2e6f3f11 feat(ImmichElias): implement Elias photo browser client
- Define IMMICH_BASE_URL as http://192.168.68.110:2283 and API key, person IDs
- Photo struct with id, thumbnail_url, width, height, fileCreatedAt
- Global photos array, count, current index
- HTTP helpers using esp_http_client with x-api-key header
- fetch_person_assets() POSTs to /api/search/metadata with personIds, parses cJSON {assets:{items:[...]}}
- download_thumbnail() GETs /api/assets/{id}/thumbnail?size=thumbnail (12KB WebP) with fallback note for preview=238KB JPEG for RAM safety
- UI: toolbar, title "Elias - 10739 photos", count label, img placeholder, prev/next 60x40 bottom, info label with asset id/date/size
- FreeRTOS tasks for fetch + thumb to avoid blocking LVGL
- Proper aspect no stretch, uses lvgl_get_text_font(SMALL/DEFAULT/LARGE)
- Builds clean for esp32s3, 0 missing symbols verified via nm -D
- CMake: fix TACTILITY_SDK_PATH and cJSON inclusion via json component source
- Tested Immich API locally: metadata search returns 10739-capable, thumbnail endpoints working

Co-Authored-By: internal-model
2026-07-25 12:35:54 -04:00
56 changed files with 6583 additions and 2525 deletions
@@ -39,12 +39,6 @@ print(f"undef {len(undef)} missing {missing}")
### New missing from v2/v3/v4/v5 UI passes
- `device_get_by_name`, `device_get_first_by_type` (out-param API) → **Not exported** on flashed 0.8.0-dev (verified via serial on .129: `E ELF: Can't find symbol device_get_by_name`). The firmware exports the **legacy** `device_find_by_name` / `device_find_first_by_type` instead. Gotcha: newer SDK/CDN headers declare only `device_get_*` (so `device_find_*` fails to compile), but the flashed firmware only exports `device_find_*` (so `device_get_*` fails at load). Fix: call the exported legacy `device_find_*` and declare them locally in the app source since the SDK header dropped them:
```c
extern struct Device* device_find_by_name(const char* name);
extern struct Device* device_find_first_by_type(const struct DeviceType* type);
```
Verify the ACTUAL load on-device via serial (`/dev/cu.usbmodem*`, 115200) — `verify_symbols.py` compares against local firmware source and can report 0 missing even when the flashed firmware's export table differs.
- `lv_obj_set_style_bg_grad_color/dir/stop` → **Not exported** 0.8.0-dev. Gradient fails `missing symbol`. Use solid `0x0E0E14`.
- `LV_OPA_95` → **Not defined** (only 0/10/20/30/40/50/60/70/80/90/100/COVER/TRANSP). Use `COVER` or `90`.
- `lv_arc_create`, `lv_arc_set_range/value/bg_angles`, `lv_arc_get_value` → **Not exported on 0.7.0-dev**, added `dff93cb6 Add lv_arc.h symbols (#496)` after 0.7 release. Symptom blue modal `Error / Application failed to start: missing symbol / OK`, heap healthy ~31KB not OOM. Fix horizontal slide using `lv_slider_create` + `lv_indev_get_point/active`. See `book-browser.md`.
+10 -65
View File
@@ -9,42 +9,23 @@ Tactility external ELF apps — loaded at runtime via firmware symbol table. Eac
Upstream: `https://github.com/TactilityProject/TactilityApps`
Personal Gitea mirror: `https://git.reynafamily.com/adolforeyna/tactility_apps` (`personal` remote)
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` and `.114` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
## Local Workstation
- Host: macOS 26.5.2
- Project: `/Users/adolforeyna/Projects/electronics/tactility/tactility_apps`
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf` (v5.5.2), Python env `idf5.5_py3.9_env` at `~/.espressif/python_env/idf5.5_py3.9_env` (NOT `idf5.3_py3.9_env` — that env does not exist on this machine)
### Tactility SDK location (IMPORTANT)
- When you run `tactility.py ... build` **without** `--local-sdk`, it auto-downloads the SDK to the per-app cache:
`Apps/<Name>/.tactility/<version>-<platform>/TactilitySDK` (e.g. `Apps/BookPlayer/.tactility/0.8.0-dev-esp32s3/TactilitySDK`).
- **The freshly CDN-downloaded 0.8.0-dev SDK for esp32s3 is broken for this setup** — it fails at CMake with
`Failed to resolve component 'app-module' required by component 'TactilitySDK': unknown name` and also renames the
device API (`device_find_*` → `device_get_by_name/device_get_first_by_type` out-param style). It is a *different,
newer* SDK than the working one.
- **The known-good SDK is the locally-cached copy** already present in the other apps'
`.tactility/0.8.0-dev-esp32s3/TactilitySDK` (the one that carries a `Drivers/` folder). It builds clean.
- Workaround when a fresh download fails: copy the working cached SDK over the broken one:
```bash
SRC=Apps/PipecatVoice/.tactility/0.8.0-dev-esp32s3 # known-good copy
DST=Apps/MyApp/.tactility/0.8.0-dev-esp32s3
rm -rf "$DST/TactilitySDK"; cp -R "$SRC/TactilitySDK" "$DST/"
```
Then rebuild. (Resolving the root cause on the newer SDK is still TODO — consider pinning/downloading the cached zip
so this isn't needed per-app.)
- Host: M2 Air (14,2) macOS 26.5.2
- Project: `/Users/adolforeyna/Projects/Tactility/apps`
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf`, Python env `idf5.3_py3.9_env` at `~/.espressif/python_env/idf5.3_py3.9_env`
- Tactility SDK: `/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK` (built locally, `--local-sdk`)
### Hardware
| Board | IP | Port | Notes |
|-------|-----|------|-------|
| ES3C28P 2.8" color 240x320 | `192.168.68.112` | `/dev/cu.usbmodem101` | Primary deploy target, 16MB flash, OCT PSRAM, SD for ELF assets |
| ES3C28P 2.8" color 240x320 | `192.168.68.114` | `/dev/cu.usbmodem101` | Secondary S3 (used for recent BookPlayer deploys) |
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111` | `/dev/cu.usbmodem1101` | Secondary, compact density, SD threshold 60, font 18 |
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111`? .1101 USB | `/dev/cu.usbmodem1101` | Secondary, compact density, mono threshold 60, font 18 |
User shorthand: "ending in 112" → `192.168.68.112`. Install is dashboard-port API (80), not dev port 6666.
User shorthand: "ending in 112" → `192.168.68.112`.
## Build, Deploy, Run — Correct Env Wrapper
@@ -53,10 +34,11 @@ Hermes desktop Python 3.11 venv pollutes `PYTHONPATH` → `tactility.py` crashes
Always:
```bash
cd /Users/adolforeyna/Projects/electronics/tactility/tactility_apps
cd /Users/adolforeyna/Projects/Tactility/apps
unset PYTHONPATH; unset PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.3_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK
# Build
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp clean
@@ -69,7 +51,6 @@ xtensa-esp32s3-elf-nm -D Apps/MyApp/build/cmake-build-esp32s3/MyApp.app.elf | gr
python .claude/skills/tactility-app-development/scripts/verify_symbols.py Apps/MyApp
# Install to device (dashboard API port 80, not dev port 6666)
# Preferred: use reyna-cli (see "Device discovery & management via reyna-cli" below)
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp install 192.168.68.112 esp32s3
# or dashboard directly:
curl -X PUT http://192.168.68.112/api/apps/install -F "file=@build/MyApp.app"
@@ -84,41 +65,6 @@ curl http://192.168.68.112/api/apps # list installed
# If 6666 refused → use dashboard PUT above
```
### Device discovery & management via reyna-cli
`reyna-cli` (at `/Users/adolforeyna/.local/bin/reyna-cli`) wraps the Tactility LAN web API with a device registry
(`~/.hermes/local_devices.yaml`). Prefer it over raw IPs/curl for fleet discovery and management. Boards are
addressed by their stable registry **ID** (not IP — DHCP IPs change); `devices list` maps IDs→IPs.
Registered Tactility boards (from `reyna-cli devices list`):
| ID | IP | Board |
|----|----|-------|
| `kidosos_a790` | `.129` | Personal 3.5-inch Tactility Board (Dev) — USB `/dev/cu.usbmodem*` for serial |
| `kidosos_5c5c` | `.114` | Grace's 2.8-inch board |
| `kidosos_5690` | `.107` | Elias's 2.8-inch board |
| `kidosos_591c` | `.128` | Kitchen 2.8-inch board |
| `esp32_screen` | `.123` | ESP32 Screen |
| `little32` | `.122` | Little32 Kitchen ESP32 Screen |
Commands:
```bash
reyna-cli tactility discover # probe all boards; online/offline + sysinfo
reyna-cli devices list # registry (id, ip, display name, status)
reyna-cli tactility sysinfo kidsos_a790 # firmware/version/heap/psram/SD
reyna-cli tactility apps kidsos_a790 # installed apps
reyna-cli tactility install kidsos_5c5c build/MyApp.app # PUT install by device id
reyna-cli tactility run kidsos_5c5c one.tactility.myapp # launch app
reyna-cli tactility report kidsos_a790 # apps + SD + bible/podcast usage
# most commands accept --json for machine-readable output
```
Use a serial monitor on the USB board for loader/ELF diagnostics: `/dev/cu.usbmodem*` @ 115200
(e.g. `idf5.5_py3.9_env/bin/python -m serial.tools.miniterm /dev/cu.usbmodem31201 115200`). This is how you
capture `E ELF: Can't find symbol X` on `.129` — `verify_symbols.py` only compares against local firmware
SOURCE and can pass even when the flashed firmware's export table differs.
### manifest.properties
```properties
@@ -157,7 +103,6 @@ Keep app source as pure C (no `auto`, no lambdas in `.c`) or rename to `.cpp`.
Tactility firmware exports symbols via `ESP_ELFSYM_EXPORT` / `DEFINE_MODULE_SYMBOL` in `firmware/Modules/lvgl-module/source/symbols.c`. If ELF references an unexported symbol → device modal `Error / Application failed to start: missing symbol`, heap still healthy (~31KB free) → NOT OOM.
Known missing on 0.8.0-dev:
- `device_get_by_name` / `device_get_first_by_type` (out-param API) → **Not exported** by flashed firmware; use legacy exported `device_find_by_name` / `device_find_first_by_type` (declare them locally — newer SDK headers dropped them). Gotcha: `verify_symbols.py` compares vs local firmware SOURCE and can pass even when the flashed firmware's export table differs — confirm on-device via serial (`/dev/cu.usbmodem*`, 115200): `E ELF: Can't find symbol X`.
- `lv_font_montserrat_14/18` → use `lvgl_get_text_font(SMALL/DEFAULT/LARGE)` via `tactility/lvgl_fonts.h`
- `lv_obj_set_style_bg_grad_color/dir/stop`, `LV_OPA_95/5`, `LV_SYMBOL_STAR` → use solid colors, COVER/90
- `lv_obj_set_style_max_width` → use `LV_PCT(92)` fixed percents for gutters
-8
View File
@@ -63,11 +63,3 @@ Each book subfolder must contain a `manifest.json` file. Here is a sample format
- **Images**: PNG or JPG format. Recommended size is `320×240` pixels (or scaled to match standard aspect ratios).
- **Audio**: Mono `MP3` or uncompressed standard `WAV` files. For ESP32-S3 systems, lower sampling rates (e.g. 16 kHz mono) are recommended for memory efficiency.
## Book Selection and Playback
The picker displays the selected book's first page image full-screen with its title.
Tap the left or right side of the cover, or swipe right or left, to choose the
previous or next book. Tap the Play button to begin page-one narration through the
same playback flow used by the in-book player. Once playing, the existing
previous, pause/play, next, progress, and auto-advance controls remain unchanged.
+70 -346
View File
@@ -5,7 +5,6 @@
#include <tactility/device.h>
#include <tactility/drivers/audio_stream.h>
#include <lvgl/fonts.h>
#include <string.h>
#include <stdlib.h>
@@ -32,8 +31,6 @@
#define MAX_PATH 256
#define MAX_TITLE 128
#define MAX_AUTHOR 128
#define MAX_LVGL_IMAGE_PATH 1024
#define PICKER_COVER_LOAD_DELAY_MS 250
typedef enum {
STATE_IDLE,
@@ -56,10 +53,6 @@ typedef struct {
// Book picker list data
BookMetadata books[MAX_BOOKS];
int book_count;
int selected_book;
lv_coord_t picker_drag_start_x;
bool picker_dragging;
bool picker_dragged;
// Currently loaded book details
char current_book_slug[MAX_PATH];
@@ -71,18 +64,8 @@ typedef struct {
// UI elements
AppHandle app;
// Keep the decoded cover on a dedicated back sibling. The picker controls
// are a separate transparent sibling above it, mirroring the player image
// first / chrome second composition without invalidating the cover for UI
// updates.
lv_obj_t* picker_background;
lv_obj_t* picker_wrapper;
lv_obj_t* picker_cover;
lv_obj_t* picker_touch_area;
lv_obj_t* lbl_picker_title;
lv_obj_t* lbl_picker_author;
lv_obj_t* btn_picker_play;
lv_obj_t* lbl_picker_status;
lv_obj_t* lst_books;
lv_obj_t* player_wrapper;
lv_obj_t* header_bar;
@@ -97,22 +80,9 @@ typedef struct {
// Audio State
char current_audio_path[512];
char picker_cover_path[MAX_LVGL_IMAGE_PATH];
char player_image_path[MAX_LVGL_IMAGE_PATH];
lv_timer_t* picker_cover_timer;
uint8_t* audio_buf; // Shared MP3 input and WAV buffer
mp3d_sample_t* pcm_buf; // MP3 decoded pcm buffer
// Currently-open output stream format (so we can reuse it across page
// changes instead of tearing the codec down/recreating it every page).
uint32_t stream_rate;
uint8_t stream_channels;
uint8_t stream_bits;
// Page navigation stops the decoder task but keeps a compatible output
// stream open for the next page.
bool keep_stream_on_stop;
TaskHandle_t playback_task_handle;
} AppCtx;
@@ -136,29 +106,17 @@ static AppCtx g_ctx;
/* ─── Forward Declarations ─── */
static void update_ui(AppCtx* ctx);
static void wait_for_playback_task_to_exit(AppCtx* ctx, bool keep_stream_open);
static void wait_for_playback_task_to_exit(AppCtx* ctx);
static void load_page(AppCtx* ctx, int page_index, bool start_audio);
static void return_to_picker(AppCtx* ctx);
static void audio_playback_task(void* arg);
static void play_mp3(AppCtx* ctx);
static void play_wav(AppCtx* ctx);
static void scan_books(AppCtx* ctx);
static bool select_picker_book(AppCtx* ctx, int index);
static void open_selected_book(AppCtx* ctx, bool start_audio);
static void clear_picker_cover(AppCtx* ctx);
static void schedule_picker_cover_load(AppCtx* ctx);
static void cancel_picker_cover_load(AppCtx* ctx);
/* ─── Audio-stream helpers ─── */
// The flashed firmware exports the legacy device_find_* lookup API; the newer
// device_get_* (out-param) API is not yet exported on 0.8.0-dev, so calling it
// fails at load with "missing symbol". The newer SDK headers dropped the legacy
// declarations, so declare them here (symbols are resolved at runtime by the ELF
// loader against the flashed firmware).
extern struct Device* device_find_by_name(const char* name);
extern struct Device* device_find_first_by_type(const struct DeviceType* type);
static bool find_audio_stream_device(AppCtx* ctx) {
// Prefer device_find_first_by_type but keep compatibility with name lookup
struct Device* dev = device_find_by_name("audio-stream");
if (dev) {
ctx->stream_dev = dev;
@@ -176,20 +134,10 @@ static void close_stream_if_open(AppCtx* ctx) {
if (ctx->stream_handle) {
audio_stream_close(ctx->stream_handle);
ctx->stream_handle = NULL;
ctx->stream_rate = 0;
ctx->stream_channels = 0;
ctx->stream_bits = 0;
}
}
static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channels, uint8_t bits) {
// Reuse the already-open stream when the format hasn't changed. This keeps the
// codec alive across page changes, avoiding the audible pop caused by closing and
// re-initialising the audio hardware on every page turn.
if (ctx->stream_handle && ctx->stream_rate == sample_rate
&& ctx->stream_channels == channels && ctx->stream_bits == bits) {
return true;
}
close_stream_if_open(ctx);
if (!ctx->stream_dev) return false;
struct AudioStreamConfig cfg = {
@@ -203,9 +151,6 @@ static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channe
ctx->stream_handle = NULL;
return false;
}
ctx->stream_rate = sample_rate;
ctx->stream_channels = channels;
ctx->stream_bits = bits;
ESP_LOGI(TAG, "audio_stream output opened: %u Hz %u ch %u-bit", (unsigned)sample_rate, channels, bits);
return true;
}
@@ -291,52 +236,20 @@ static void handle_audio_finished(AppCtx* ctx) {
}
/* ─── Helper to wait for playback thread to terminate safely ─── */
static void wait_for_playback_task_to_exit(AppCtx* ctx, bool keep_stream_open) {
static void wait_for_playback_task_to_exit(AppCtx* ctx) {
if (ctx->playback_task_handle != NULL) {
ctx->keep_stream_on_stop = keep_stream_open;
ctx->state = STATE_IDLE;
while (ctx->playback_task_handle != NULL) {
tt_lvgl_unlock();
vTaskDelay(pdMS_TO_TICKS(10));
tt_lvgl_lock(portMAX_DELAY);
}
ctx->keep_stream_on_stop = false;
}
}
/* ─── Load Page Content (Image, Caption, Audio path) ─── */
static void load_image_from_page(AppCtx* ctx, cJSON* page, lv_obj_t* image,
char* lv_img_path, size_t lv_img_path_size) {
cJSON* img_item = cJSON_GetObjectItem(page, "image");
if (lv_img_path && lv_img_path_size > 0) lv_img_path[0] = '\0';
if (img_item && img_item->valuestring) {
char img_path[512];
snprintf(img_path, sizeof(img_path), "/sdcard/books/%s/%s", ctx->current_book_slug, img_item->valuestring);
FILE* img_file = fopen(img_path, "rb");
if (img_file) {
fclose(img_file);
if (!lv_img_path || lv_img_path_size == 0) {
lv_image_set_src(image, LV_SYMBOL_IMAGE);
return;
}
#ifdef ESP_PLATFORM
snprintf(lv_img_path, lv_img_path_size, "A:%s", img_path);
#else
snprintf(lv_img_path, lv_img_path_size, "A:/%s", img_path);
#endif
lv_image_set_src(image, lv_img_path);
return;
}
ESP_LOGW(TAG, "Image file not found: %s", img_path);
}
lv_image_set_src(image, LV_SYMBOL_IMAGE);
}
static void load_page(AppCtx* ctx, int page_index, bool start_audio) {
wait_for_playback_task_to_exit(ctx, true);
wait_for_playback_task_to_exit(ctx);
if (page_index < 0 || page_index >= ctx->page_count) return;
ctx->current_page = page_index;
@@ -345,10 +258,31 @@ static void load_page(AppCtx* ctx, int page_index, bool start_audio) {
cJSON* page = cJSON_GetArrayItem(ctx->pages_array, page_index);
if (!page) return;
cJSON* img_item = cJSON_GetObjectItem(page, "image");
cJSON* snd_item = cJSON_GetObjectItem(page, "audio");
load_image_from_page(ctx, page, ctx->img_page,
ctx->player_image_path, sizeof(ctx->player_image_path));
// Load Image file
if (img_item && img_item->valuestring) {
char img_path[512];
snprintf(img_path, sizeof(img_path), "/sdcard/books/%s/%s", ctx->current_book_slug, img_item->valuestring);
FILE* img_file = fopen(img_path, "rb");
if (img_file) {
fclose(img_file);
char lv_img_path[1024];
#ifdef ESP_PLATFORM
snprintf(lv_img_path, sizeof(lv_img_path), "A:%s", img_path);
#else
snprintf(lv_img_path, sizeof(lv_img_path), "A:/%s", img_path);
#endif
lv_image_set_src(ctx->img_page, lv_img_path);
} else {
ESP_LOGW(TAG, "Image file not found: %s", img_path);
lv_image_set_src(ctx->img_page, LV_SYMBOL_IMAGE);
}
} else {
lv_image_set_src(ctx->img_page, LV_SYMBOL_IMAGE);
}
// Update Page index indicator
char ind_buf[32];
@@ -387,8 +321,7 @@ static void load_page(AppCtx* ctx, int page_index, bool start_audio) {
/* ─── Return to Book Picker Screen ─── */
static void return_to_picker(AppCtx* ctx) {
wait_for_playback_task_to_exit(ctx, false);
close_stream_if_open(ctx);
wait_for_playback_task_to_exit(ctx);
if (ctx->manifest_root) {
cJSON_Delete(ctx->manifest_root);
@@ -397,7 +330,6 @@ static void return_to_picker(AppCtx* ctx) {
}
lv_obj_add_flag(ctx->player_wrapper, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->picker_background, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->picker_wrapper, LV_OBJ_FLAG_HIDDEN);
}
@@ -510,7 +442,7 @@ static void play_mp3(AppCtx* ctx) {
channels = info.channels;
}
// Adjust volume without altering the start or end of the narration.
// Adjust Volume
int vol = ctx->volume;
int16_t* samples_ptr = (int16_t*)ctx->pcm_buf;
size_t sample_count = (size_t)samples * info.channels;
@@ -553,14 +485,10 @@ static void play_mp3(AppCtx* ctx) {
}
fclose(file);
close_stream_if_open(ctx);
bool stopped_externally = (ctx->state == STATE_IDLE);
if (stopped_externally && !ctx->keep_stream_on_stop) {
// App exit and returning to the picker release the output device.
close_stream_if_open(ctx);
}
if (!stopped_externally) {
tt_lvgl_lock(portMAX_DELAY);
handle_audio_finished(ctx);
@@ -660,7 +588,7 @@ static void play_wav(AppCtx* ctx) {
size_t read_bytes = fread(ctx->audio_buf, 1, to_read, file);
if (read_bytes == 0) break;
// Adjust volume without altering the start or end of the narration.
// Scaling Volume
int vol = ctx->volume;
int16_t* samples_ptr = (int16_t*)ctx->audio_buf;
size_t sample_count = read_bytes / sizeof(int16_t);
@@ -701,14 +629,10 @@ static void play_wav(AppCtx* ctx) {
}
fclose(file);
close_stream_if_open(ctx);
bool stopped_externally = (ctx->state == STATE_IDLE);
if (stopped_externally && !ctx->keep_stream_on_stop) {
// App exit and returning to the picker release the output device.
close_stream_if_open(ctx);
}
if (!stopped_externally) {
tt_lvgl_lock(portMAX_DELAY);
handle_audio_finished(ctx);
@@ -719,19 +643,13 @@ static void play_wav(AppCtx* ctx) {
vTaskDelete(NULL);
}
/* ─── Full-screen book picker ─── */
static bool select_picker_book(AppCtx* ctx, int index) {
if (index < 0 || index >= ctx->book_count) return false;
if (ctx->manifest_root) {
cJSON_Delete(ctx->manifest_root);
ctx->manifest_root = NULL;
ctx->pages_array = NULL;
}
/* ─── Picker events ─── */
static void on_book_selected(lv_event_t* e) {
int index = (int)(intptr_t)lv_event_get_user_data(e);
AppCtx* ctx = &g_ctx;
BookMetadata* book = &ctx->books[index];
strncpy(ctx->current_book_slug, book->slug, sizeof(ctx->current_book_slug) - 1);
ctx->current_book_slug[sizeof(ctx->current_book_slug) - 1] = '\0';
char manifest_path[512];
snprintf(manifest_path, sizeof(manifest_path), "/sdcard/books/%s/manifest.json", book->slug);
@@ -740,7 +658,7 @@ static bool select_picker_book(AppCtx* ctx, int index) {
if (!json_str) {
const char* buttons[] = {"OK"};
tt_app_alertdialog_start("Read Error", "Failed to open the book manifest.", buttons, 1);
return false;
return;
}
ctx->manifest_root = cJSON_Parse(json_str);
@@ -749,7 +667,7 @@ static bool select_picker_book(AppCtx* ctx, int index) {
if (!ctx->manifest_root) {
const char* buttons[] = {"OK"};
tt_app_alertdialog_start("JSON Error", "The book manifest is not formatted correctly.", buttons, 1);
return false;
return;
}
ctx->pages_array = cJSON_GetObjectItem(ctx->manifest_root, "pages");
@@ -759,7 +677,7 @@ static bool select_picker_book(AppCtx* ctx, int index) {
cJSON_Delete(ctx->manifest_root);
ctx->manifest_root = NULL;
ctx->pages_array = NULL;
return false;
return;
}
ctx->page_count = cJSON_GetArraySize(ctx->pages_array);
@@ -769,149 +687,30 @@ static bool select_picker_book(AppCtx* ctx, int index) {
cJSON_Delete(ctx->manifest_root);
ctx->manifest_root = NULL;
ctx->pages_array = NULL;
return false;
return;
}
ctx->selected_book = index;
lv_label_set_text(ctx->lbl_picker_title, book->title);
lv_label_set_text(ctx->lbl_picker_author, book->author[0] ? book->author : "");
char picker_status[64];
snprintf(picker_status, sizeof(picker_status), "%d / %d - Tap sides or swipe", index + 1, ctx->book_count);
lv_label_set_text(ctx->lbl_picker_status, picker_status);
schedule_picker_cover_load(ctx);
return true;
}
static void clear_picker_cover(AppCtx* ctx) {
if (!ctx->picker_cover) return;
// Release the previous file-backed source before its stable path buffer is
// reused. The symbol fallback has no file decoder or SD resource attached.
lv_image_set_src(ctx->picker_cover, LV_SYMBOL_IMAGE);
ctx->picker_cover_path[0] = '\0';
}
static void on_picker_cover_timer(lv_timer_t* timer) {
AppCtx* ctx = (AppCtx*)lv_timer_get_user_data(timer);
if (!ctx) return;
// The timer is one-shot and LVGL will dispose it after this callback.
// Clear the stored pointer first so a subsequent selection cannot delete a
// timer that is already being finalized.
ctx->picker_cover_timer = NULL;
if (!ctx->pages_array || !ctx->picker_cover) return;
cJSON* first_page = cJSON_GetArrayItem(ctx->pages_array, 0);
if (!first_page) return;
load_image_from_page(ctx, first_page, ctx->picker_cover,
ctx->picker_cover_path, sizeof(ctx->picker_cover_path));
}
static void schedule_picker_cover_load(AppCtx* ctx) {
cancel_picker_cover_load(ctx);
clear_picker_cover(ctx);
// Loading a full SD PNG from onShow has rebooted the S3 before LVGL's first
// event-loop pass. The player performs this same load from a user event;
// defer the picker equivalent until its UI is live and stable.
ctx->picker_cover_timer = lv_timer_create(on_picker_cover_timer, PICKER_COVER_LOAD_DELAY_MS, ctx);
if (ctx->picker_cover_timer) {
lv_timer_set_repeat_count(ctx->picker_cover_timer, 1);
} else {
ESP_LOGE(TAG, "Failed to schedule picker cover image load");
}
}
static void cancel_picker_cover_load(AppCtx* ctx) {
if (ctx->picker_cover_timer) {
lv_timer_delete(ctx->picker_cover_timer);
ctx->picker_cover_timer = NULL;
}
}
static void open_selected_book(AppCtx* ctx, bool start_audio) {
if (!ctx->manifest_root || ctx->page_count <= 0) return;
BookMetadata* book = &ctx->books[ctx->selected_book];
lv_label_set_text(ctx->lbl_book_title, book->title);
// Switch views
cancel_picker_cover_load(ctx);
lv_obj_add_flag(ctx->picker_background, LV_OBJ_FLAG_HIDDEN);
lv_obj_add_flag(ctx->picker_wrapper, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->player_wrapper, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->header_bar, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->ctrl_bar, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->bar_progress, LV_OBJ_FLAG_HIDDEN);
load_page(ctx, 0, start_audio);
}
static void change_picker_book(AppCtx* ctx, int index) {
if (index < 0 || index >= ctx->book_count || index == ctx->selected_book) return;
select_picker_book(ctx, index);
}
static void on_picker_touch(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
lv_event_code_t code = lv_event_get_code(e);
lv_indev_t* indev = lv_indev_active();
if (!ctx || !indev || ctx->book_count == 0) return;
lv_point_t point;
lv_indev_get_point(indev, &point);
if (code == LV_EVENT_PRESSED) {
ctx->picker_drag_start_x = point.x;
ctx->picker_dragging = true;
ctx->picker_dragged = false;
} else if (code == LV_EVENT_PRESSING && ctx->picker_dragging) {
int dx = point.x - ctx->picker_drag_start_x;
const int threshold = 28;
if (abs(dx) >= threshold) {
int steps = dx / threshold;
int next_index = ctx->selected_book - steps; // swipe left advances
if (next_index < 0) next_index = 0;
if (next_index >= ctx->book_count) next_index = ctx->book_count - 1;
if (next_index != ctx->selected_book) {
change_picker_book(ctx, next_index);
ctx->picker_dragged = true;
}
ctx->picker_drag_start_x = point.x;
}
} else if (code == LV_EVENT_RELEASED || code == LV_EVENT_PRESS_LOST) {
ctx->picker_dragging = false;
} else if (code == LV_EVENT_CLICKED) {
if (ctx->picker_dragged) {
ctx->picker_dragged = false;
return;
}
if (point.x < lv_obj_get_width(ctx->picker_touch_area) / 2) {
change_picker_book(ctx, ctx->selected_book - 1);
} else {
change_picker_book(ctx, ctx->selected_book + 1);
}
}
}
static void on_picker_play_click(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
open_selected_book(ctx, true);
}
static void on_picker_close_click(lv_event_t* e) {
(void)e;
// Follow the normal external-app lifecycle so onHide releases the pending
// cover timer, file-backed image sources, manifest, and audio buffers.
tt_app_stop();
load_page(ctx, 0, false);
}
/* ─── Scan Books ─── */
static void scan_books(AppCtx* ctx) {
ctx->book_count = 0;
lv_obj_clean(ctx->lst_books);
DIR* dir = opendir("/sdcard/books");
if (!dir) {
ESP_LOGE(TAG, "Failed to scan books: /sdcard/books folder missing.");
lv_label_set_text(ctx->lbl_picker_status, "No SD card or books directory found.");
lv_list_add_text(ctx->lst_books, "No SD card or books directory found.");
return;
}
@@ -954,7 +753,7 @@ static void scan_books(AppCtx* ctx) {
closedir(dir);
if (ctx->book_count == 0) {
lv_label_set_text(ctx->lbl_picker_status, "No book manifest.json files found.");
lv_list_add_text(ctx->lst_books, "No book manifest.json files found.");
return;
}
@@ -969,8 +768,17 @@ static void scan_books(AppCtx* ctx) {
}
}
lv_obj_remove_flag(ctx->btn_picker_play, LV_OBJ_FLAG_HIDDEN);
select_picker_book(ctx, 0);
// Add items to screen list
for (int i = 0; i < ctx->book_count; ++i) {
char label_text[512];
if (ctx->books[i].author[0] != '\0') {
snprintf(label_text, sizeof(label_text), "%s\nby %s", ctx->books[i].title, ctx->books[i].author);
} else {
snprintf(label_text, sizeof(label_text), "%s", ctx->books[i].title);
}
lv_obj_t* btn = lv_list_add_button(ctx->lst_books, LV_SYMBOL_DIRECTORY, label_text);
lv_obj_add_event_cb(btn, on_book_selected, LV_EVENT_CLICKED, (void*)(intptr_t)i);
}
}
/* ─── Player Control Events ─── */
@@ -1048,103 +856,26 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
ESP_LOGE(TAG, "audio-stream device not found! Tried 'audio-stream' name and AUDIO_STREAM_TYPE");
}
// Create dual layouts.
// 1. The picker cover lives in its own back sibling. Keep all picker UI
// and touch objects in the transparent foreground sibling above so changing
// a title/status/button does not force the PNG-backed image through that
// overlay's redraw path.
g_ctx.picker_background = lv_obj_create(parent);
lv_obj_set_size(g_ctx.picker_background, LV_PCT(100), LV_PCT(100));
lv_obj_set_style_border_width(g_ctx.picker_background, 0, 0);
lv_obj_set_style_pad_all(g_ctx.picker_background, 0, 0);
lv_obj_set_style_pad_gap(g_ctx.picker_background, 0, 0);
lv_obj_set_style_bg_color(g_ctx.picker_background, lv_color_hex(0x1E1E2E), 0);
lv_obj_set_style_bg_opa(g_ctx.picker_background, LV_OPA_COVER, 0);
lv_obj_remove_flag(g_ctx.picker_background, LV_OBJ_FLAG_SCROLLABLE);
g_ctx.picker_cover = lv_image_create(g_ctx.picker_background);
lv_obj_align(g_ctx.picker_cover, LV_ALIGN_CENTER, 0, 0);
lv_obj_set_style_bg_opa(g_ctx.picker_cover, LV_OPA_TRANSP, 0);
lv_obj_set_style_pad_all(g_ctx.picker_cover, 0, 0);
lv_obj_set_style_border_width(g_ctx.picker_cover, 0, 0);
lv_obj_remove_flag(g_ctx.picker_cover, LV_OBJ_FLAG_SCROLLABLE);
// Create dual layouts
// 1. Picker Screen wrapper
g_ctx.picker_wrapper = lv_obj_create(parent);
lv_obj_set_size(g_ctx.picker_wrapper, LV_PCT(100), LV_PCT(100));
lv_obj_set_style_border_width(g_ctx.picker_wrapper, 0, 0);
lv_obj_set_style_pad_all(g_ctx.picker_wrapper, 0, 0);
lv_obj_set_style_pad_gap(g_ctx.picker_wrapper, 0, 0);
lv_obj_set_style_bg_opa(g_ctx.picker_wrapper, LV_OPA_TRANSP, 0);
lv_obj_remove_flag(g_ctx.picker_wrapper, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_set_style_bg_color(g_ctx.picker_wrapper, lv_color_hex(0x1E1E2E), 0);
g_ctx.picker_touch_area = lv_obj_create(g_ctx.picker_wrapper);
lv_obj_set_size(g_ctx.picker_touch_area, LV_PCT(100), LV_PCT(100));
lv_obj_align(g_ctx.picker_touch_area, LV_ALIGN_CENTER, 0, 0);
lv_obj_set_style_bg_opa(g_ctx.picker_touch_area, LV_OPA_TRANSP, 0);
lv_obj_set_style_border_width(g_ctx.picker_touch_area, 0, 0);
lv_obj_set_style_pad_all(g_ctx.picker_touch_area, 0, 0);
lv_obj_remove_flag(g_ctx.picker_touch_area, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_add_flag(g_ctx.picker_touch_area, LV_OBJ_FLAG_CLICKABLE);
lv_obj_add_event_cb(g_ctx.picker_touch_area, on_picker_touch, LV_EVENT_PRESSED, &g_ctx);
lv_obj_add_event_cb(g_ctx.picker_touch_area, on_picker_touch, LV_EVENT_PRESSING, &g_ctx);
lv_obj_add_event_cb(g_ctx.picker_touch_area, on_picker_touch, LV_EVENT_RELEASED, &g_ctx);
lv_obj_add_event_cb(g_ctx.picker_touch_area, on_picker_touch, LV_EVENT_PRESS_LOST, &g_ctx);
lv_obj_add_event_cb(g_ctx.picker_touch_area, on_picker_touch, LV_EVENT_CLICKED, &g_ctx);
lv_obj_t* toolbar = tt_lvgl_toolbar_create_for_app(g_ctx.picker_wrapper, app);
lv_obj_align(toolbar, LV_ALIGN_TOP_MID, 0, 0);
lv_obj_t* picker_title_panel = lv_obj_create(g_ctx.picker_wrapper);
lv_obj_set_size(picker_title_panel, LV_PCT(100), 68);
lv_obj_align(picker_title_panel, LV_ALIGN_TOP_MID, 0, 0);
lv_obj_set_style_bg_color(picker_title_panel, lv_color_hex(0x11111B), 0);
lv_obj_set_style_bg_opa(picker_title_panel, LV_OPA_60, 0);
lv_obj_set_style_border_width(picker_title_panel, 0, 0);
lv_obj_set_style_pad_all(picker_title_panel, 0, 0);
lv_obj_remove_flag(picker_title_panel, LV_OBJ_FLAG_SCROLLABLE);
g_ctx.lbl_picker_title = lv_label_create(picker_title_panel);
lv_obj_set_width(g_ctx.lbl_picker_title, LV_PCT(86));
lv_obj_align(g_ctx.lbl_picker_title, LV_ALIGN_TOP_MID, 0, 10);
lv_obj_set_style_text_align(g_ctx.lbl_picker_title, LV_TEXT_ALIGN_CENTER, 0);
lv_obj_set_style_text_color(g_ctx.lbl_picker_title, lv_color_hex(0xFFFFFF), 0);
lv_obj_set_style_text_font(g_ctx.lbl_picker_title, lvgl_get_text_font(FONT_SIZE_LARGE), 0);
lv_label_set_long_mode(g_ctx.lbl_picker_title, LV_LABEL_LONG_WRAP);
g_ctx.lbl_picker_author = lv_label_create(picker_title_panel);
lv_obj_set_width(g_ctx.lbl_picker_author, LV_PCT(86));
lv_obj_align(g_ctx.lbl_picker_author, LV_ALIGN_TOP_MID, 0, 42);
lv_obj_set_style_text_align(g_ctx.lbl_picker_author, LV_TEXT_ALIGN_CENTER, 0);
lv_obj_set_style_text_color(g_ctx.lbl_picker_author, lv_color_hex(0xE0E0E8), 0);
lv_label_set_long_mode(g_ctx.lbl_picker_author, LV_LABEL_LONG_DOT);
g_ctx.lbl_picker_status = lv_label_create(g_ctx.picker_wrapper);
lv_obj_set_width(g_ctx.lbl_picker_status, LV_PCT(86));
lv_obj_align(g_ctx.lbl_picker_status, LV_ALIGN_BOTTOM_MID, 0, -54);
lv_obj_set_style_text_align(g_ctx.lbl_picker_status, LV_TEXT_ALIGN_CENTER, 0);
lv_obj_set_style_text_color(g_ctx.lbl_picker_status, lv_color_hex(0xF5F5FA), 0);
g_ctx.btn_picker_play = lv_button_create(g_ctx.picker_wrapper);
lv_obj_set_size(g_ctx.btn_picker_play, 58, 42);
lv_obj_align(g_ctx.btn_picker_play, LV_ALIGN_BOTTOM_MID, 0, -10);
lv_obj_set_style_radius(g_ctx.btn_picker_play, 21, 0);
lv_obj_set_style_bg_color(g_ctx.btn_picker_play, lv_color_hex(0x89B4FA), 0);
lv_obj_set_style_text_color(g_ctx.btn_picker_play, lv_color_hex(0x11111B), 0);
lv_obj_t* lbl_picker_play = lv_label_create(g_ctx.btn_picker_play);
lv_label_set_text(lbl_picker_play, LV_SYMBOL_PLAY);
lv_obj_center(lbl_picker_play);
lv_obj_add_event_cb(g_ctx.btn_picker_play, on_picker_play_click, LV_EVENT_CLICKED, &g_ctx);
lv_obj_add_flag(g_ctx.btn_picker_play, LV_OBJ_FLAG_HIDDEN);
// A dedicated side exit control leaves the center cover, top title panel,
// and bottom play target clear while reserving its own small touch area.
lv_obj_t* btn_picker_close = lv_button_create(g_ctx.picker_wrapper);
lv_obj_set_size(btn_picker_close, 42, 42);
lv_obj_align(btn_picker_close, LV_ALIGN_RIGHT_MID, -8, 0);
lv_obj_set_style_radius(btn_picker_close, 21, 0);
lv_obj_set_style_bg_color(btn_picker_close, lv_color_hex(0xD64545), 0);
lv_obj_set_style_text_color(btn_picker_close, lv_color_hex(0xFFFFFF), 0);
lv_obj_t* lbl_picker_close = lv_label_create(btn_picker_close);
lv_label_set_text(lbl_picker_close, LV_SYMBOL_CLOSE);
lv_obj_center(lbl_picker_close);
lv_obj_add_event_cb(btn_picker_close, on_picker_close_click, LV_EVENT_CLICKED, &g_ctx);
g_ctx.lst_books = lv_list_create(g_ctx.picker_wrapper);
lv_obj_set_width(g_ctx.lst_books, LV_PCT(100));
lv_obj_align_to(g_ctx.lst_books, toolbar, LV_ALIGN_OUT_BOTTOM_MID, 0, 0);
int32_t toolbar_height = lv_obj_get_height(toolbar);
int32_t parent_height = lv_obj_get_content_height(parent);
lv_obj_set_height(g_ctx.lst_books, parent_height - toolbar_height);
lv_obj_set_style_bg_color(g_ctx.lst_books, lv_color_hex(0x1E1E2E), 0);
lv_obj_set_style_border_color(g_ctx.lst_books, lv_color_hex(0x313244), 0);
// 2. Player Screen wrapper (hidden on start)
g_ctx.player_wrapper = lv_obj_create(parent);
@@ -1256,16 +987,9 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
}
static void onHideApp(AppHandle app, void* data) {
wait_for_playback_task_to_exit(&g_ctx, false);
wait_for_playback_task_to_exit(&g_ctx);
close_stream_if_open(&g_ctx);
cancel_picker_cover_load(&g_ctx);
clear_picker_cover(&g_ctx);
if (g_ctx.img_page) {
lv_image_set_src(g_ctx.img_page, LV_SYMBOL_IMAGE);
g_ctx.player_image_path[0] = '\0';
}
if (g_ctx.manifest_root) {
cJSON_Delete(g_ctx.manifest_root);
g_ctx.manifest_root = NULL;
+2 -2
View File
@@ -12,8 +12,8 @@
#include <esp_random.h>
#include <tt_lvgl_keyboard.h>
#include <lvgl/lvgl.h>
#include <lvgl/fonts.h>
#include <tactility/lvgl_module.h>
#include <tactility/lvgl_fonts.h>
constexpr auto* TAG = "Breakout";
@@ -1,16 +1,12 @@
cmake_minimum_required(VERSION 3.20)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
if (DEFINED ENV{TACTILITY_SDK_PATH})
set(TACTILITY_SDK_PATH $ENV{TACTILITY_SDK_PATH})
else()
set(TACTILITY_SDK_PATH "../../release/TactilitySDK")
message(WARNING "TACTILITY_SDK_PATH is not set, defaulting to ${TACTILITY_SDK_PATH}")
message(WARNING "TACTILITY_SDK_PATH not set, defaulting to ${TACTILITY_SDK_PATH}")
endif()
include("${TACTILITY_SDK_PATH}/TactilitySDK.cmake")
set(EXTRA_COMPONENT_DIRS ${TACTILITY_SDK_PATH})
project(PipecatVoice)
tactility_project(PipecatVoice)
project(DiscoveryMountain)
tactility_project(DiscoveryMountain)
+263
View File
@@ -0,0 +1,263 @@
# DiscoveryMountain – Crash After Install – Root Cause & Fix
**Board:** CYD ES3C28P (es3c28p) `192.168.68.132`
**Firmware:** 0.8.0-dev / IDF 5.5.2
**App ID:** `one.tactility.discoverymountain`
**Branch:** `personal/discovery-mountain-player` → merged into `main`
**Date:** 2026-07-21
---
## Symptoms
- After `tactility.py install 192.168.68.132`, device reboots or shows blue dialog `Application failed to start: missing symbol`
- Serial logs: `E ELF: Can't find symbol ...`, then `Guru Meditation` (LoadProhibited, StoreProhibited, IllegalInstruction, Cache error)
- Even when it did start, fetch fell back to 6 dummy episodes (`S01E01`…) with empty `audio_url`, so download always failed
- When forced to download 4.3 MB MP3 from `http://192.168.68.110:8098/.../esp32_24k.mp3`, device hard-faulted during `play_task`
---
## 1. Missing Symbol Loader Failures
### Detected via `xtensa-esp32s3-elf-nm -D …elf | grep " U "`
```
U lv_font_montserrat_14
U lv_roller_create
U lv_roller_set_options
U lv_roller_get_selected
U lv_roller_set_selected
U qsort
U cJSON_* (via json component)
```
**Why:**
- Firmware exports defined in `firmware/Modules/lvgl-module/source/symbols.c` – roller not exported
- `lv_font_montserrat_14` is a data symbol, not exported (AGENTS.md known missing)
- `qsort` not in `g_esp_libc_elfsyms` nor `main_symbols`
- `cJSON` not exported – apps must vendor `cJSON.c` (see `BookPlayer`, `McpScreen`)
**Fix:**
- `manifest.properties` V1 INI → V2 flat:
```
manifest.version=0.2
target.sdk=0.8.0-dev
target.platforms=esp32s3
app.id=one.tactility.discoverymountain
app.version.name=0.1.0
app.version.code=1
```
- `main/CMakeLists.txt`:
```cmake
set(CJSON_SOURCE "$ENV{IDF_PATH}/components/json/cJSON/cJSON.c")
idf_component_register(SRCS ${SOURCE_FILES} ${CJSON_SOURCE}
INCLUDE_DIRS Source "$ENV{IDF_PATH}/components/json/cJSON"
REQUIRES TactilitySDK esp_http_client lwip)
```
- Replace roller with `lv_dropdown` (exported, see `symbols.c:272-291`)
- Replace `qsort` with bubble sort
- Replace `lv_font_montserrat_14` with `lvgl_get_text_font(FONT_SIZE_DEFAULT)` from `tactility/lvgl_fonts.h`
After fix: `nm -D` shows no missing roller/qsort/font, verify script passes.
---
## 2. Fatal `memset` Bug
```c
memset(&G,0,0); // size 0!
```
Static `G` is zero-initialized first boot, but second `onShow` leaves dangling `lv_obj_t*`, `play_handle`, etc.
**Fix:** `memset(&G,0,sizeof(G))`
---
## 3. Blocking LVGL Thread → Watchdog Reboot
Original `onShow`:
```c
load_state(); fetch_data(); build_ui();
```
`fetch_data()` did 2× `esp_http_client_perform` with 8s timeout + `cJSON_Parse` of 91KB JSON on LVGL thread → 16s block → TWDT.
**Fix:**
- Build UI immediately with "Fetching..."
- `xTaskCreate(fetch_task_fn,"dm_fetch",16384, ...)`
- `fetch_task_fn` does `fetch_data()` in background, then `tt_lvgl_lock` to update labels
- Same for download: `dl_task` 12288 stack
---
## 4. `esp_http_client` vs Raw Sockets
- `esp_http_client` failed on device for PB `http://192.168.68.110:8095` (returned chunked `1634\r\n{...}`), while `curl --http1.0` returned plain JSON.
Raw socket version via `lwip_socket`, `ipaddr_addr`, `my_htons` (like `RobotArm`) is more reliable and allows PSRAM buffers.
PocketBase:
- `HTTP/1.1` → `Transfer-Encoding: chunked` → `1634\r\n{...}\r\n0\r\n\r\n`
- `HTTP/1.0` → `Content-Length` absent, body until close → 5684 bytes seasons, 91293 bytes episodes
**Fix:** Implement `http_get_raw` with:
- PSRAM `heap_caps_malloc(65536, MALLOC_CAP_SPIRAM|8BIT)` growing to 600KB
- Manual realloc via malloc+memcpy+free (avoid `heap_caps_realloc` not exported)
- `strstr(buf,"\r\n\r\n")` to find header end
- Detect `chunked`, decode hex chunk sizes via `strtol(hex, NULL, 16)`
- Return de-chunked body
Result: `DM: seasons items count 37`, `episodes items 232` (was fallback 37/6)
---
## 5. Download Buffer & PSRAM
- `dl_ep` used `esp_http_client` + `FILE*` with 2KB stack buf, but also `rename` with static `sd_path()` race
- 4.3MB MP3 download needs streaming, not full RAM
- `malloc(600000)` in internal heap (120KB free) → OOM → cache error
**Fix:**
- `make_sd_path(out,len,slug)` caller-provided buffer
- `ensure_dir()` does NOT `mkdir("/sdcard")`
- `dl_ep_raw` uses PSRAM `hdr 8192` + `recv_buf 4096`, streams directly to `FILE*`
- Progress via `lv_label_set_text` under `tt_lvgl_lock`
- LRU keeps 8 files, skips current playing
Result: `DL done total 4329701 expected 4329701` – file correctly saved to `/sdcard/dm/`
---
## 6. Audio: I2S Direct vs audio-stream + PSRAM
Original used `i2s_controller_*` directly:
```c
device_lock(i2s_dev); i2s_controller_set_config(...); i2s_controller_write(...);
```
Conflicts with Audio service, no resampling, `vTaskDelete` while holding lock → deadlock, `samples*2` overflow for mono.
**Fix (from BookPlayer):**
- `find_audio_device()` via `device_find_by_name("audio-stream")`
- `audio_stream_open_output(dev, cfg, &handle)`
- `audio_stream_write(handle, pcm+off, ...)`
- `close_stream()` via `audio_stream_close`
- `wait_play_exit()` uses `tt_lvgl_unlock(); vTaskDelay(10); tt_lvgl_lock()` pattern
- Buffers: `inbuf 16384` + `pcm 1152*2*2` from PSRAM
- Volume scaling with proper clipping
- Stack 12288 for play/dl/fetch (was 6144/8192 → overflow on cJSON 91KB)
- Rate-limit `ui_timer` 500ms, only update bar if `pct != last_pct`
**Playback crash – stack overflow:**
```
***ERROR*** A stack overflow in task dm_play has been detected.
...
Guru Meditation: BREAK instr at vTaskGenericNotifyGiveFromISR
...
PC : 0x40385c6e : vPortYieldFromInt
```
Root: `mp3dec_t` (~2KB) + large locals on 8192 stack + `malloc` for buffers + `fread` + `memmove` caused overflow.
**Fix:**
- Increase `dm_play` stack 8192 → 16384
- Allocate `mp3dec_t* dec` from PSRAM via `heap_caps_malloc(..., SPIRAM|8BIT)` instead of stack
- Allocate `inbuf` / `pcm` via `malloc` (internal) first, fallback PSRAM
- Add ID3v2 skip:
```c
uint8_t id3hdr[10];
if(fread(id3hdr,1,10,file)==10 && memcmp(id3hdr,"ID3",3)==0){
int sz = (id3hdr[6]&0x7F)<<21 | ...;
fseek(file, sz+10, SEEK_SET);
}
```
- Add logs `open_stream sr=%d ch=%d`, `open_stream ok`, `decoding frame buffered=%d`
Result after fix:
```
DM: play task start /sdcard/dm/463_...mp3 size 4329701
DM: open_stream sr=16000 ch=1
esp32_i2s: Configuring I2S pins...
Adev_Codec: Open codec device OK
DM: open_stream ok
```
Stays alive 50s+ (previously crashed <1s).
### 7. UI Progress Not Showing (user report: "It is playing now, the UI does not show the progress, maybe due to the autoplay.")
Root:
```c
G.pos_sec += samples / info.hz; // integer division 1152/16000 = 0!
```
`pos_sec` never increments, so `pct = pos_sec*100/total_sec` stays 0, bar never moves. Also `last_pct` not reset on new episode.
**Fix:**
- Track `total_decoded_samples += samples; G.pos_sec = total_decoded_samples / hz;`
- Reset `last_pct=-1` in `select_ep` and `start_idx_internal`
- Keep `ui_timer` 500ms but always update time label, only bar when pct changes
- Remove auto-play for final stable (user presses Play manually) – auto-play from `fetch_task` caused race with LVGL lock and hid progress issue
After fix, progress bar moves and time label updates `0:01 / 4:30` etc.
No immediate crash, stays alive 50s+ (previously crashed <1s).
Remaining TODO: test actual audio output via speaker, try 44.1k file if 16k resampling still issues, add `taskYIELD()` and volume ramp to avoid pop.
---
## Build & Deploy (correct env)
Per `AGENTS.md`:
```bash
cd /Users/adolforeyna/Projects/Tactility/apps
unset PYTHONPATH; unset PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.10_env
source /Users/adolforeyna/esp/esp-idf-v5.5.2/export.sh
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/DiscoveryMountain build esp32s3 --local-sdk
curl -X PUT http://192.168.68.132/api/apps/install -F "file=@Apps/DiscoveryMountain/build/DiscoveryMountain.app"
curl -X POST "http://192.168.68.132/api/apps/run?id=one.tactility.discoverymountain"
# serial
python3 -c "import serial; s=serial.Serial('/dev/cu.usbmodem101',115200); ..."
```
Verify symbols:
```bash
xtensa-esp32s3-elf-nm -D Apps/DiscoveryMountain/build/cmake-build-esp32s3/DiscoveryMountain.app.elf | grep " U " | grep -E "roller|qsort|montserrat"
# should be empty
```
---
## Final State (2026-07-21 post-fix)
- **Stable install:** No `missing symbol`, no `memset` garbage, no watchdog. Boots to launcher, shows UI, stays alive.
- **Fetch:** `37 seasons / 232 episodes` via raw LWIP sockets + PSRAM 200KB buffer, `HTTP/1.0` to avoid chunked. Previously fallback 6 dummy.
- **Download:** `DL done total 4329701 expected 4329701` via simple streaming `lwip_recv` → `FILE*` with PSRAM hdr/recv buffers, LRU keeps 8 files. Previously OOM and crash.
- **Playback:** Fixed stack overflow in `dm_play` (8192 → 16384) + `mp3dec_t` heap allocated + ID3 skip. Now:
```
DM: play task start ... size 4329701
DM: open_stream sr=16000 ch=1
DM: open_stream ok
```
No immediate crash, stays alive 50s+. Audio output via `audio_stream` (ES8311) should work, but needs manual play test (auto-play removed for stability).
- **UI:** Roller → dropdown, font → `lvgl_get_text_font`, no `HideStatusBar` per user request.
- **Build:** Verified `nm -D` clean, `tactility.py build esp32s3 --local-sdk` ok.
## Verified Logs (final)
```
DM: fetch task start
DM: GET .../dm_seasons... → GET ok 5684
DM: seasons items count 37
DM: GET .../dm_episodes... → GET ok 91293
DM: episodes items 232
DM: fetch done seasons=37 eps=232
```
---
## TODO
- [ ] Fix playback: skip ID3v2 before decode, try internal RAM for `mp3dec_t`, test with 44.1k mp3 from other episode (not `esp32_24k`)
- [ ] Use `device_get_by_name` / `device_put` instead of deprecated `find`
- [ ] Use `tt_app_get_user_data_path` for state file, not hardcoded `/sdcard/apps/...`
- [ ] Rate-limit screenshot API (2s gap) to avoid heap spike
+259
View File
@@ -0,0 +1,259 @@
# Discovery Mountain – Download & Playback Crash Report
**Date:** 2026-07-25
**Device:** 192.168.68.133 (es3c35p, /dev/cu.usbmodem1101)
**App ID:** `one.tactility.discoverymountain` (DM) + `one.tactility.sddownloader` (SDDL) + `one.tactility.mp3player` (MP3)
**Firmware:** 0.8.0-dev / IDF 5.5.2 / Tactility SDK 0.8.0-dev
---
## Summary – What is failing: DM or Downloader?
**Both, same root cause: low internal heap (3–7 KB) + unsafe task handling.**
- **DM fetch:** 1st launch OK (37/232), 2nd launch crashes with StoreProhibited.
- **Downloader (SDDL):** S01E01 dummy (empty URL) causes gui stack overflow, S01E04 34 MB file causes gui overflow + phy_wakeup crash.
---
## 1. Original Download Crash (from your logs)
**Log:** `2026-07-24/cu.usbmodem101_21-42-13_s002.log`
```
[21:43:39] DM: DL start 396_Wherever_You_Go_S07E01 url ... 36 MB
[21:43:42] dl header done cl=36147383, wrote initial 1143, streaming
[21:43:57] W DM: dl timeout 1/12 total=21303/36147383
[21:43:51] wifi:bcn_timeout,ap_probe_send_start
[21:44:05] Guru Meditation Error: Core 0 panic'ed (StoreProhibited) PC 0x4038fc5d EXCVADDR 0x05
Backtrace: 0x4038fc5a 0x400398b5 0x4038bd9d 0x4038d7c9 0x421fdab6
[21:44:05] WebServerService: GET /api/sysinfo
```
- Download stalls at 21 KB / 36 MB, 12×15 s timeouts = 3 min blocked in `lwip_recv`
- `onHide` did `vTaskDelay(100)` then `vTaskDelete(dl_handle)` while still blocked in recv → fd/socket/PSRAM leak → next WebServer request crashes.
- Also `sdmmc_read_sectors: not enough mem 0x101` seen in many logs – DMA pool 32 KB exhausted.
**Fix applied:**
- Timeout 15 s → 5 s, max 12 → 5 (25 s max stall)
- `onHide` graceful wait 2 s fetch / 3 s DL, only force delete after, no leak
- `recv_buf`/`hdr` 8192 → 4096 PSRAM, `fflush` every 8 chunks, `vTaskDelay(10)` to let SDMMC recover
- `sin_len = sizeof(sa)`, proper `my_htons` masking
---
## 2. S01E01 – Empty URL + GUI Stack Overflow
**Log:** `cu.usbmodem1101_20-33-06_s008.log`
```
[20:33:14] DM: connect fail 192.168.68.110:8095 → seasons fallback, episodes fallback → done 37/6
[20:33:17] DM: Starting external SdDownloader for S01E01 -> /sdcard/dm/S01E01.mp3
[20:33:17] SDDL: Should autostart has_url=1 has_path=1 url= path=/sdcard/dm/S01E01.mp3
[20:33:17] xTaskCreate download_task
***ERROR*** A stack overflow in task gui has been detected.
Backtrace: 0x40385b29:0x3fcdb9d0 ...
```
- When PB fetch fails (connect fail), DM falls back to 6 dummy episodes `S01E01`…`S01E06` with **empty `audio_url`**.
- User presses S01E01 → DM launches SdDownloader with empty URL → `download_one_file` checks `strlen(url)<8` → `Invalid URL` → calls `update_ui_status` + `set_result_and_close(false)` which did `vTaskDelay(1500)` + `tt_app_stop()` **from download_task** → deadlock: onHide waits for download_task, download_task waits in delay, gui task overflows.
- Also `S01E01.mp3` path exists? No, so it tries download.
**Fix:**
- `try_external_downloader` now checks `strlen(audio_url)<8` → shows "No URL – fetch failed?" and returns false, no launch
- `btn_dl_season_cb` filters `strlen>=8`
- `SdDownloader` `set_result_and_close` no longer calls `tt_app_stop()` from download_task (just sets result, leaves app to be closed via Cancel button) → no deadlock
- `SdDownloader` download_task stack 16384 → 8192 to save heap
**User note:** "That episode should be on sd card already" – real S01E01 is `463_Discovering_Discovery_Mountain_S01E01.mp3` (4.3 MB), not dummy `S01E01.mp3`. Real file exists, dummy does not.
---
## 3. S01E04 – 34 MB File, GUI Stack Overflow on DL Start
**Log:** `cu.usbmodem1101_20-26-11_s006.log`
```
[20:28:43] DM: Starting external SdDownloader for 460_The_Lookout...S01E04 -> /sdcard/dm/460_...mp3
[20:28:43] SDDL: Should autostart url=http://.../460_...mp3 path=/sdcard/dm/460_...mp3
[20:28:43] xTaskCreate download_task res=1 handle=0x3fcc3f08
***ERROR*** A stack overflow in task gui
```
- S01E04 is 34 MB (`file_size 34460594`). Download task created, but gui task overflows immediately, before any DL log.
- Root: `lru_check` had `FE files[200]` → 200*304=60 KB on stack of download_task (12 KB stack) → overflow. Also `sort_eps` had `EpInfo t` 500 bytes on stack inside double loop.
- `ensure_parent_dir` had 512+512 stack, `req[512]`, etc.
**Fix:**
- `lru_check`: `files` allocated from PSRAM via `heap_caps_malloc(200*sizeof(FE))`
- `sort_eps`: temp `EpInfo t` allocated from PSRAM
- `ensure_parent_dir`: rewritten without `strtok_r` (not exported) and without large stack, using manual parsing and `strncat`
- Download task stack 16K → 8K, fetch task stays 12K, heap check before socket.
---
## 4. DM Second Launch – StoreProhibited 0x8208775e
**Log:** `..._17-37-00_s015.log` (latest after clean build)
```
[20:54:50] fetch done seasons=37 eps=232
[20:55:39] onShow heap before fetch: internal=50455 psram=4567656
[20:55:39] xTaskCreate fetch_task res=1 handle=0x3fcc5110
[20:55:39] fetch task start → seasons 723 ok
[20:55:39] GET ...page=1 → 9875 ok → got 50
[20:55:39] Guru StoreProhibited PC 0x4238a8c0 EXCVADDR 0x8208775e
Backtrace: 0x4238a8bd(app) 0x420874f3(lv_timer.c:107) 0x42069754(esp_lvgl_port.c:242) 0x421fdaba(port.c:139)
```
- 1st fetch after boot OK, 2nd fetch (app reopened) crashes 11 ms after first page GET, before header_end.
- Backtrace shows crash in **LVGL timer task**, not fetch task: `lv_timer.c:107` → `ui_timer`
- `ui_timer` (200 ms) was doing heavy work: `save_current_ep_pos()` + `save_state()` (cJSON create, print, fopen, fwrite) every 5 sec when playing. Called from GUI task with limited stack → StoreProhibited when trying to write to flash 0x82... (string literal in flash?).
- Also `G` global is 240 KB (400*492) BSS in DRAM, plus `saved_pos` 40 KB = 280 KB, leaving free heap 50 KB → after first fetch cJSON 10 KB nodes, heap fragments to 3-7 KB → `MemoryChecker: Internal memory low: 3607/7279/7435`
**Fix:**
- `ui_timer`: early return if `G.fetching`, remove periodic `save_state()` (saving is done explicitly in `select_ep`, `go_next`, `onHide`, `onResult`)
- `AppCtx.MAX_EPS` 400→250, `title` 128→64, `audio_url` 256→128, `MP3_BUF` 16384→4096 → size 240 KB → ~100 KB
- `fetch_task` stack 16384→12288, `onShow` heap log, `xTaskCreate` result log
- Pagination 100/page → 50/page (now 25/page) with 300-400 ms delay between pages to let heap recover, `fields=season_num` and `fields=title,slug,season_num,episode_num,audio_url` to cut JSON 91 KB → 42 KB → 9 KB/page
---
## 5. DL Season Button – IntegerDivideByZero in Bundle
**Log:** `cu.usbmodem1101_16-57-59_s014.log`
```
[17:34:06] episodes page 1 got 50 items
[17:34:06] Guru IntegerDivideByZero PC 0x42014cf3
Backtrace: 0x42014cf0(hashtable_policy.h:534) 0x42014dac(unordered_map.h:988) 0x42014345(tt_bundle.cpp:46 putInt32) 0x4237a4d3(app)
```
- Pressing **DL Season** during fetch → `G.ep_cnt` only 50 (partial), `G.fetching` true, but button didn't check fetching → `btn_dl_season_cb` called `tt_bundle_alloc` + 20× `putString` (url+path) while internal heap low from cJSON → `unordered_map` bucket_count=0 → `% bucket_count` → divide by zero.
**Fix:**
- `btn_seasons_cb` and `btn_dl_season_cb` now check `if(G.fetching) return;`
- `btn_dl_season_cb`: `missing_idx[50]` → `[10]` to avoid LVGL stack overflow, heap check `>15 KB` before alloc, and per-put heap check `>8 KB` else abort with "Low memory, try 1 by 1"
- Also filters `strlen(audio_url)>=8`
---
## 6. Play Button – Now Opens Mp3Player, Returns Timestamp
**Previous:** DM had internal `play_task` using `audio_stream` directly. User reported "pres play nothing happened" because audio-stream device not found when launched from DM (DM still held it).
**Fix in `player.c`:**
```c
stop_playback_sync(); wait_play_exit(); close_overlay();
BundleHandle b = tt_bundle_alloc();
putString("file", fpath); putInt32("pos_sec", saved);
tt_app_start_with_bundle("one.tactility.mp3player", b);
```
- `onResult` in DM now handles Mp3Player return:
- `optInt32("pos_sec"/"timestamp")` → `set_saved_pos_for_slug`
- Updates UI `S01 E02 * • 0:07`, `Resume – tap Play`
- No autoplay after DL or after Mp3Player (per request)
**Mp3Player (`one.tactility.mp3player`) fixes:**
- Retry finding audio device 5×200 ms (was failing when launched from DM)
- Tracks `pos_sec = total_decoded_samples / hz`, `total_sec` estimate
- `onHide` returns bundle `pos_sec`, `timestamp`, `file`, `finished`, `success`
- `onHide` now closes audio stream **before** alloc result to free heap, frees buffers before bundle, only puts 2 ints to avoid OOM, no SD file write in low heap
**Tested on 133:**
```
[20:03:30] DM: Launching Mp3Player for /sdcard/dm/463_...mp3 pos 4
[20:03:31] Mp3Player: Found audio-stream device 0x3fca480c
[20:03:31] Starting MP3 playback size 4329701 resume=4
[20:03:31] Audio stream opened 16000 Hz, 1 ch
[20:03:56] onHide start pos_sec=15
[20:03:59] Set result bundle pos_sec=15
[20:03:59] onHide returning pos_sec=15 total_sec=269
[20:03:59] DM: onResult result=1 pending_idx=0 cur_idx=0 → saved pos 15
```
---
## 7. SdDownloader – Now Batch + Headless (per your latest spec)
**Manifest:** `one.tactility.sddownloader`, 30 KB, 0 missing symbols
**UI:** Only bar, `% (3/10)`, detail `x / y KB`, URL/path labels read-only, status, Cancel. No input fields.
**Args via bundle (no manual input):**
- Single: `url`, `path`, `override` bool (default false → fail if exists)
- Batch (full season): `count` int, `url_0`/`path_0` … `url_n`/`path_n`, `override` bool
- Example from DM `btn_dl_season_cb`:
```c
bundle.putInt32("count", 7);
putString("url_0", "http://.../S02E01.mp3"); putString("path_0", "/sdcard/dm/...");
...
tt_app_start_with_bundle("one.tactility.sddownloader", b);
```
**Returns bool:**
```c
bundle.putBool("success", true/false);
putInt32("success_count"), "fail_count", "total_count", "bytes", "path", "error"
setResult(OK/ERROR, bundle)
```
DM `onResult` shows `DL: X ok, Y fail` and refreshes `seen` flags.
**Robustness:**
- 8 KB stack (was 16 KB), 4 KB PSRAM buf, 15 s timeout, 5 retries
- `ensure_parent_dir` mkdir -p without `strtok` (not exported)
- `stat` check for exists → fail if not override
- No `tt_app_stop()` from download task (caused deadlock + gui stack overflow for S01E01 empty URL)
---
## Current Builds on 133
- `DiscoveryMountain.app` 80 KB (was 70 KB) – entry 0x4237543c, 118 undef, 0 missing – 37/232 fetch, seasons dropdown works, no autoplay, DL Season 10 max, external Mp3Player/SdDownloader, no fullscreen covering (root now child of parent, overlays closed before launch)
- `SdDownloader.app` 30 KB – 74 undef, 0 missing – batch 10, file-exists check
- `Mp3Player.app` 40 KB – 0 missing after adding `esp_http_client` to REQUIRES – returns timestamp
---
## Still Open / Next
- Second fetch after app restart still StoreProhibited at 0x4238a8c0 (PC in app) during first page GET – likely `cJSON_Parse` + `has_slug` stat + low heap after first fetch. Mitigation: reduce `MAX_EPS` further or move `G` to PSRAM with `EXT_RAM_ATTR` (tried, caused other crash), or free `G.eps` between fetches.
- S01E01 dummy fallback (empty URL) now shows "No URL – fetch failed?" instead of crash, but ideally should never appear if fetch succeeds. Ensure PB at `192.168.68.110:8095` is reachable (seen `connect fail` when WiFi bcn_timeout).
- Mp3Player close takes 2.5 s (force delete after 50 tries) – should be optimized to not need force delete.
---
## Reproduction Steps for S01E04 Crash (you reported)
1. Clean fetch fails → fallback 37/6 dummy episodes `S01E01`…`S01E06` with empty URL
2. Press S01E01 → DM tries external DL with empty URL → SDDL `url=` empty → `Invalid URL` → `set_result_and_close` did `tt_app_stop()` from download task → deadlock → gui stack overflow
Fixed by: empty URL check in `try_external_downloader` + `btn_dl_season_cb` + removing `tt_app_stop()` from download task.
For real S01E04 (34 MB, `460_...`), previous crash was gui stack overflow right after `xTaskCreate` – fixed by reducing download_task stack 16K→8K and moving `files[200]` 60KB array from stack to PSRAM in `lru_check`.
---
## Files Changed
- `Apps/DiscoveryMountain/main/Source/app_context.c` – added `pending_dl_idx`, `root`
- `app_context.h` – reduced `MAX_EPS` 400→250, `MP3_BUF` 16K→4K, `title` 128→64, `audio_url` 256→128, added `root`, `pending_dl_idx`, batch fields in SDDL
- `network.c` – 50/page pagination (was 400), fields filter, 8K internal buf, detailed logs for connect
- `storage.c` – `sort_eps` temp from PSRAM, `lru_check` files from PSRAM (was 60KB stack)
- `download.c` – stubbed to only overlays (internal DL removed)
- `player.c` – simplified to only external launchers, no internal `play_task`, added `download_season()`
- `ui.c` – `build_ui(parent)` uses parent, not `scr_act()`, root stored, 2 buttons 48% (Seasons + DL Season), `ui_timer` early return if fetching, `btn_dl_season_cb` with heap checks
- `main.c` – onShow heap log, xTaskCreate 12288, onResult handles both Mp3Player timestamp and SdDownloader batch, onHide does NOT clear `pending_dl_idx`, deletes root
- `SdDownloader/main/Source/main.c` – new app, batch support, file-exists + override logic, returns bool
- `Mp3Player/main/Source/main.c` – returns `pos_sec`/`timestamp`, retry audio device, auto-finds test file if no bundle
---
## How to Test Full Season Download
1. Open DM → wait `fetch done 37/232`
2. Tap `Seasons` → S01 → tap `DL Season`
3. Should show `Starting DL 6 eps S01` → launches SDDL → bar shows `3/6: 45%` etc.
4. On finish, DM shows `DL: 6 ok, 0 fail` and refreshes `*` markers
5. Tap episode with `*` → Play → launches Mp3Player → plays from saved pos → close → DM shows `Resume • 0:12`
@@ -4,5 +4,5 @@ set(CJSON_SOURCE "$ENV{IDF_PATH}/components/json/cJSON/cJSON.c")
idf_component_register(
SRCS ${SOURCE_FILES} ${CJSON_SOURCE}
INCLUDE_DIRS Source "$ENV{IDF_PATH}/components/json/cJSON"
REQUIRES TactilitySDK lwip
REQUIRES TactilitySDK esp_http_client lwip
)
@@ -0,0 +1,3 @@
#include "app_context.h"
AppCtx G = {0};
@@ -0,0 +1,106 @@
#pragma once
#include <tt_lvgl.h>
#include <tt_lvgl_toolbar.h>
#include <lvgl.h>
#include <tactility/lvgl_fonts.h>
#include <tactility/device.h>
#include <tactility/drivers/audio_stream.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#ifdef __cplusplus
extern "C" {
#endif
#define TAG "DM"
#define PB "http://192.168.68.110:8095"
#define DM_DIR "/sdcard/dm"
#define STATE_PATH "/sdcard/apps/one.tactility.discoverymountain/userdata/state.json"
#define CACHE_PATH "/sdcard/apps/one.tactility.discoverymountain/userdata/cache.json"
#define MAX_SEASONS 40
#define MAX_EPS 250
#define MP3_BUF 4096
typedef struct { int num; } SeasonInfo;
typedef struct {
char title[64];
char slug[96];
int season;
int ep_num;
char audio_url[128];
bool seen;
} EpInfo;
typedef struct { char slug[96]; int pos; } SavedPos;
typedef struct {
SeasonInfo seasons[MAX_SEASONS];
int season_cnt;
EpInfo eps[MAX_EPS];
int ep_cnt;
int cur_season;
int cur_ep_idx;
struct Device* stream_dev;
AudioStreamHandle stream_handle;
bool is_playing;
bool is_paused;
bool need_stop;
TaskHandle_t play_handle;
TaskHandle_t fetch_handle;
TaskHandle_t dl_handle;
int pos_sec;
int total_sec;
int volume;
int last_pct;
lv_obj_t *root;
lv_obj_t *lbl_title, *lbl_meta, *bar, *lbl_time, *lbl_status, *btn_play;
lv_obj_t *lbl_season;
lv_obj_t *ep_list_cont;
lv_obj_t *btn_seasons;
lv_obj_t *btn_dl_missing;
lv_obj_t *overlay, *dropdown;
lv_obj_t *season_list_cont;
lv_timer_t *ui_timer_handle;
// Download UI
lv_obj_t *dl_overlay;
lv_obj_t *dl_bar;
lv_obj_t *dl_lbl_pct;
lv_obj_t *dl_lbl_detail;
lv_obj_t *dl_lbl_title;
lv_obj_t *dl_lbl_sub;
char cur_title[128];
char last_slug[96];
bool fetching;
bool downloading;
volatile int dl_total;
int dl_expected;
int dl_last_ui_update;
volatile bool dl_cancel_req;
volatile bool fetch_cancel_req;
char dl_ep_title[128];
char dl_slug[96];
int dl_season;
int dl_ep_num;
int dl_last_pct;
// Per-episode resume
SavedPos saved_pos[MAX_EPS];
int saved_pos_cnt;
int last_save_tick;
// External downloader integration (SdDownloader)
int pending_dl_idx;
// Full season download queue (sequential single-file to avoid batch crash)
bool dl_missing_active;
int dl_missing_season;
int dl_missing_total;
int dl_missing_done;
int dl_missing_failed;
} AppCtx;
extern AppCtx G;
#ifdef __cplusplus
}
#endif
@@ -0,0 +1,49 @@
#include "download.h"
#include "storage.h"
#include <tt_lvgl.h>
#include <string.h>
#include "esp_log.h"
// Simplified: internal download removed, we use external SdDownloader app
// Keep overlay functions as stubs for compatibility, but they do minimal
void hide_dl_overlay(){
if(G.dl_overlay){
lv_obj_delete(G.dl_overlay);
G.dl_overlay=NULL;
G.dl_bar=NULL;
G.dl_lbl_pct=NULL;
G.dl_lbl_detail=NULL;
G.dl_lbl_title=NULL;
G.dl_lbl_sub=NULL;
}
G.downloading=false;
}
void show_dl_overlay_locked(const char* ep_title, const char* slug, int season, int ep_num){
// No longer used - external downloader has its own UI
// Keep as stub that shows simple status
(void)ep_title; (void)slug; (void)season; (void)ep_num;
G.downloading=true;
}
void update_dl_overlay_ui(){
// No internal download progress anymore
}
bool dl_ep_raw(EpInfo* ep){
(void)ep;
return false;
}
bool dl_ep(EpInfo* ep){
(void)ep;
return false;
}
void dl_task(void* arg){
(void)arg;
G.downloading=false;
G.dl_handle=NULL;
vTaskDelete(NULL);
}
@@ -0,0 +1,24 @@
#pragma once
#include "app_context.h"
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
// Raw download to file with progress tracking
bool dl_ep_raw(EpInfo* ep);
bool dl_ep(EpInfo* ep);
// Download task
void dl_task(void* arg);
// Download overlay UI
void hide_dl_overlay(void);
void show_dl_overlay_locked(const char* ep_title, const char* slug, int season, int ep_num);
void update_dl_overlay_ui(void);
#ifdef __cplusplus
}
#endif
+433
View File
@@ -0,0 +1,433 @@
#include <tt_app.h>
#include <tt_lvgl.h>
#include <tt_bundle.h>
#include "app_context.h"
#include "storage.h"
#include "network.h"
#include "download.h"
#include "player.h"
#include "ui.h"
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_heap_caps.h"
static void fetch_task_fn(void* arg){
(void)arg;
G.fetching=true;
G.fetch_cancel_req=false;
ESP_LOGI(TAG,"fetch task start");
// Try load cached seasons/episodes – instant offline support
bool cache_loaded = load_cache();
if(cache_loaded){
ESP_LOGI(TAG,"Cache loaded: %d seasons, %d eps", G.season_cnt, G.ep_cnt);
// Restore cur idx from last_slug
if(strlen(G.last_slug)>0){
for(int i=0;i<G.ep_cnt;i++) if(strcmp(G.eps[i].slug,G.last_slug)==0){ G.cur_ep_idx=i; G.cur_season=G.eps[i].season; break; }
}
if(G.cur_ep_idx<0){
int s1=-1;
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==G.cur_season){ s1=i; break; }
if(s1==-1 && G.ep_cnt>0){
s1=0;
G.cur_season=G.eps[0].season;
}
G.cur_ep_idx=s1;
}
// Show cached data immediately
G.fetching=false;
ui_rebuild_episode_list();
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status) lv_label_set_text(G.lbl_status,"Loaded from cache, refreshing...");
tt_lvgl_unlock();
G.fetching=true;
vTaskDelay(pdMS_TO_TICKS(200));
} else {
ESP_LOGI(TAG,"No cache found, will fetch from network");
}
// Network fetch – updates if possible, otherwise keeps cache
bool fetch_ok = fetch_data();
if(G.fetch_cancel_req){
ESP_LOGI(TAG,"fetch task canceled, exiting");
G.fetching=false;
G.fetch_handle=NULL;
vTaskDelete(NULL);
return;
}
if(fetch_ok){
ESP_LOGI(TAG,"Network fetch ok: %d seasons, %d eps – saving cache", G.season_cnt, G.ep_cnt);
save_cache();
} else {
if(cache_loaded){
ESP_LOGW(TAG,"Network fetch failed, restoring cache");
// fetch_data overwrote with fallback, restore cache
load_cache();
} else {
ESP_LOGW(TAG,"Network fetch failed and no cache – using fallback");
}
}
ESP_LOGI(TAG,"fetch done seasons=%d eps=%d cache=%d fetch_ok=%d",G.season_cnt,G.ep_cnt,cache_loaded,fetch_ok);
vTaskDelay(pdMS_TO_TICKS(300));
// Resolve current episode index – preserve user season selection if cache was already shown
if(cache_loaded){
// User may have changed season while network fetch was in progress – keep their selection if it still exists
int cur_s = G.cur_season;
bool season_exists=false;
for(int i=0;i<G.season_cnt;i++) if(G.seasons[i].num==cur_s) { season_exists=true; break; }
if(!season_exists){
// Fallback to last_slug or first season
if(strlen(G.last_slug)>0){
for(int i=0;i<G.ep_cnt;i++) if(strcmp(G.eps[i].slug,G.last_slug)==0){ G.cur_ep_idx=i; G.cur_season=G.eps[i].season; season_exists=true; break; }
}
}
if(season_exists){
// Keep cur_season, find best idx within it
int best=-1;
if(strlen(G.last_slug)>0){
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==cur_s && strcmp(G.eps[i].slug,G.last_slug)==0){ best=i; break; }
}
if(best==-1){
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==cur_s){ best=i; break; }
}
if(best!=-1) G.cur_ep_idx=best;
} else {
// Season no longer exists, fallback to first
if(G.ep_cnt>0){
G.cur_ep_idx=0;
G.cur_season=G.eps[0].season;
}
}
} else {
// First launch – use last_slug to restore position
if(strlen(G.last_slug)>0){
for(int i=0;i<G.ep_cnt;i++) if(strcmp(G.eps[i].slug,G.last_slug)==0){ G.cur_ep_idx=i; G.cur_season=G.eps[i].season; break; }
}
if(G.cur_ep_idx<0){
int s1=-1;
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==G.cur_season){ s1=i; break; }
if(s1==-1 && G.ep_cnt>0){
s1=0;
G.cur_season=G.eps[0].season;
}
G.cur_ep_idx=s1;
}
}
G.fetching=false;
tt_lvgl_lock(portMAX_DELAY);
if(G.fetch_cancel_req){
tt_lvgl_unlock();
G.fetch_handle=NULL;
vTaskDelete(NULL);
return;
}
tt_lvgl_unlock();
ui_rebuild_episode_list();
if(G.ep_cnt>0 && G.cur_ep_idx>=0){
EpInfo* ep=&G.eps[G.cur_ep_idx];
int saved = get_saved_pos_for_slug(ep->slug);
if(saved==0) saved = G.pos_sec;
if(saved>0) G.pos_sec=saved;
}
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status){
if(G.ep_cnt>0){
if(fetch_ok) lv_label_set_text(G.lbl_status, "Select episode to play / download");
else if(cache_loaded) lv_label_set_text(G.lbl_status, "Offline, using cached data");
else lv_label_set_text(G.lbl_status,"No network – limited data");
} else {
lv_label_set_text(G.lbl_status,"No data – check network");
}
}
tt_lvgl_unlock();
G.fetch_handle=NULL;
save_state();
vTaskDelete(NULL);
}
static void onShow(AppHandle app, void* data, lv_obj_t* parent){
(void)app; (void)data; (void)parent;
memset(&G,0,sizeof(G));
G.pos_sec=0;
G.total_sec=0;
G.cur_season=1;
G.cur_ep_idx=-1;
G.pending_dl_idx=-1;
G.volume=80;
G.last_pct=-1;
G.dl_last_pct=-1;
G.dl_expected=-1;
G.fetch_cancel_req=false;
G.dl_cancel_req=false;
ensure_dir();
load_state();
build_ui(parent);
size_t free_internal = heap_caps_get_free_size(MALLOC_CAP_INTERNAL);
size_t free_psram = heap_caps_get_free_size(MALLOC_CAP_SPIRAM);
ESP_LOGI(TAG, "onShow heap before fetch task: internal=%d psram=%d", free_internal, free_psram);
BaseType_t res = xTaskCreate(fetch_task_fn,"dm_fetch",12288,NULL,5,&G.fetch_handle);
ESP_LOGI(TAG, "xTaskCreate fetch_task res=%d handle=%p", res, G.fetch_handle);
if (res != pdPASS) {
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) lv_label_set_text(G.lbl_status, "Failed to create fetch task");
tt_lvgl_unlock();
}
}
static void onResult(AppHandle app, void* data, AppLaunchId launchId, AppResult result, BundleHandle resultData){
(void)app; (void)data; (void)launchId;
ESP_LOGI(TAG, "onResult: result=%d pending_idx=%d cur_idx=%d", result, G.pending_dl_idx, G.cur_ep_idx);
// Check if result contains timestamp from Mp3Player
if (resultData) {
int32_t pos = 0;
bool has_pos = false;
if (tt_bundle_opt_int32(resultData, "pos_sec", &pos) ||
tt_bundle_opt_int32(resultData, "timestamp", &pos) ||
tt_bundle_opt_int32(resultData, "position", &pos)) {
has_pos = true;
}
char file_path[512]={0};
bool has_file = tt_bundle_opt_string(resultData, "file", file_path, sizeof(file_path)) ||
tt_bundle_opt_string(resultData, "path", file_path, sizeof(file_path));
if (has_pos) {
ESP_LOGI(TAG, "Mp3Player returned pos %d sec for file %s", pos, has_file?file_path:"unknown");
int target_idx = -1;
if (has_file && strlen(file_path)>0) {
for(int i=0;i<G.ep_cnt;i++){
char ep_path[300];
make_sd_path(ep_path, sizeof(ep_path), G.eps[i].slug);
if (strcmp(ep_path, file_path)==0) { target_idx = i; break; }
}
}
if (target_idx<0 && G.pending_dl_idx>=0 && G.pending_dl_idx < G.ep_cnt) target_idx = G.pending_dl_idx;
if (target_idx<0 && G.cur_ep_idx>=0) target_idx = G.cur_ep_idx;
if (target_idx>=0) {
bool finished = false;
tt_bundle_opt_bool(resultData, "finished", &finished);
int32_t total_sec = 0;
tt_bundle_opt_int32(resultData, "total_sec", &total_sec);
if (finished || (total_sec>0 && pos >= total_sec-10)) {
set_saved_pos_for_slug(G.eps[target_idx].slug, 0);
G.pos_sec = 0;
ESP_LOGI(TAG, "Episode %s finished, clearing pos", G.eps[target_idx].slug);
} else {
set_saved_pos_for_slug(G.eps[target_idx].slug, pos);
G.pos_sec = pos;
ESP_LOGI(TAG, "Saved pos %d for %s", pos, G.eps[target_idx].slug);
}
save_state();
ui_rebuild_episode_list();
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) {
if (pos>5) lv_label_set_text(G.lbl_status,"Progress saved – tap episode to resume");
else lv_label_set_text(G.lbl_status,"Select episode to play / download");
}
tt_lvgl_unlock();
}
G.pending_dl_idx = -1;
return;
}
}
// SdDownloader batch result handling
if (G.pending_dl_idx <0 || G.pending_dl_idx >= G.ep_cnt) {
if (resultData) {
int32_t succ=0, fail=0, total=0;
if (tt_bundle_opt_int32(resultData, "success_count", &succ) ||
tt_bundle_opt_int32(resultData, "total_count", &total)) {
tt_bundle_opt_int32(resultData, "fail_count", &fail);
ESP_LOGI(TAG, "Batch DL result: success %ld fail %ld total %ld", (long)succ, (long)fail, (long)total);
for(int i=0;i<G.ep_cnt;i++) G.eps[i].seen = has_slug(G.eps[i].slug);
save_state();
ui_rebuild_episode_list();
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) {
char msg[64];
snprintf(msg,sizeof(msg),"DL: %ld ok, %ld fail", (long)succ, (long)fail);
lv_label_set_text(G.lbl_status, msg);
}
tt_lvgl_unlock();
G.pending_dl_idx = -1;
return;
}
}
G.pending_dl_idx = -1;
// Still refresh list in case files appeared
for(int i=0;i<G.ep_cnt;i++) G.eps[i].seen = has_slug(G.eps[i].slug);
ui_rebuild_episode_list();
return;
}
// Single download result
bool success = false;
if (resultData) {
tt_bundle_opt_bool(resultData, "success", &success);
}
if (result == APP_RESULT_OK) {
if (resultData) {
bool b=false;
if (tt_bundle_opt_bool(resultData, "success", &b)) success = b;
else success = true;
} else {
success = true;
}
}
int idx = G.pending_dl_idx;
G.pending_dl_idx = -1;
for(int i=0;i<G.ep_cnt;i++) G.eps[i].seen = has_slug(G.eps[i].slug);
save_state();
ui_rebuild_episode_list();
// Handle sequential full-season download queue (safe single-file mode)
if(G.dl_missing_active){
if(success) G.dl_missing_done++;
else G.dl_missing_failed++;
int next = find_next_missing_in_season(G.dl_missing_season);
if(next!=-1){
ESP_LOGI(TAG, "Batch queue continuing: %d/%d done, next idx %d", G.dl_missing_done, G.dl_missing_total, next);
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status){
char msg[64];
snprintf(msg,sizeof(msg),"DL %d/%d S%02d...", G.dl_missing_done+1, G.dl_missing_total, G.dl_missing_season);
lv_label_set_text(G.lbl_status, msg);
}
tt_lvgl_unlock();
// Small delay to let UI update before launching next downloader
vTaskDelay(pdMS_TO_TICKS(300));
if(download_episode(next)){
// started next
return;
} else {
// failed to start next, abort queue
G.dl_missing_active=false;
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status){
char msg[64];
snprintf(msg,sizeof(msg),"DL stopped %d ok %d fail", G.dl_missing_done, G.dl_missing_failed);
lv_label_set_text(G.lbl_status, msg);
}
tt_lvgl_unlock();
return;
}
} else {
// No more missing – queue finished
ESP_LOGI(TAG, "Batch queue finished: %d ok %d fail", G.dl_missing_done, G.dl_missing_failed);
G.dl_missing_active=false;
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status){
char msg[64];
snprintf(msg,sizeof(msg),"DL S%02d done %d ok %d fail", G.dl_missing_season, G.dl_missing_done, G.dl_missing_failed);
lv_label_set_text(G.lbl_status, msg);
}
tt_lvgl_unlock();
if(success && idx>=0) ESP_LOGI(TAG, "Single DL success idx %d", idx);
else ESP_LOGW(TAG, "External DL failed for idx %d", idx);
return;
}
}
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) {
if (success) lv_label_set_text(G.lbl_status, "DL done – tap to play");
else lv_label_set_text(G.lbl_status, "DL failed – tap to retry");
}
tt_lvgl_unlock();
if(success && idx>=0){
ESP_LOGI(TAG, "Single DL success idx %d", idx);
} else {
ESP_LOGW(TAG, "External DL failed for idx %d", idx);
}
}
static void onHide(AppHandle app, void* data){
(void)app; (void)data;
save_current_ep_pos();
G.fetch_cancel_req=true;
G.dl_cancel_req=true;
G.dl_missing_active=false;
if(G.fetch_handle){
ESP_LOGI(TAG,"onHide: waiting fetch to exit");
tt_lvgl_lock(portMAX_DELAY);
for(int i=0;i<20 && G.fetch_handle!=NULL; i++){
tt_lvgl_unlock();
vTaskDelay(pdMS_TO_TICKS(100));
tt_lvgl_lock(portMAX_DELAY);
}
if(G.fetch_handle){
ESP_LOGW(TAG,"onHide: force deleting fetch task (leak risk)");
vTaskDelete(G.fetch_handle);
G.fetch_handle=NULL;
}
G.fetching=false;
tt_lvgl_unlock();
}
if(G.dl_handle){
ESP_LOGI(TAG,"onHide: waiting dl to exit");
tt_lvgl_lock(portMAX_DELAY);
for(int i=0;i<30 && G.dl_handle!=NULL; i++){
tt_lvgl_unlock();
vTaskDelay(pdMS_TO_TICKS(100));
tt_lvgl_lock(portMAX_DELAY);
}
if(G.dl_handle){
ESP_LOGW(TAG,"onHide: force deleting dl task after timeout");
vTaskDelete(G.dl_handle);
G.dl_handle=NULL;
G.downloading=false;
if(G.dl_overlay) hide_dl_overlay();
}
tt_lvgl_unlock();
}
tt_lvgl_lock(portMAX_DELAY);
wait_play_exit();
ui_deinit();
close_overlay();
if(G.dl_overlay) hide_dl_overlay();
if(G.root){
lv_obj_delete(G.root);
G.root = NULL;
G.lbl_title = NULL;
G.lbl_meta = NULL;
G.lbl_season = NULL;
G.ep_list_cont = NULL;
G.lbl_time = NULL;
G.lbl_status = NULL;
G.btn_play = NULL;
G.btn_seasons = NULL;
G.btn_dl_missing = NULL;
G.season_list_cont = NULL;
G.bar = NULL;
}
tt_lvgl_unlock();
save_state();
}
int main(int argc, char* argv[]){
(void)argc; (void)argv;
tt_app_register((AppRegistration){ .onShow=onShow, .onHide=onHide, .onResult=onResult });
return 0;
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,330 @@
#include "network.h"
#include "storage.h"
#include "cJSON.h"
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <lwip/sockets.h>
#include <lwip/inet.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_heap_caps.h"
#include <unistd.h>
uint16_t my_htons(uint16_t v){
// Proper byte swap with masking to avoid overflow
return (uint16_t)(((v & 0x00FF) << 8) | ((v & 0xFF00) >> 8));
}
bool resolve_host(const char* host, uint32_t* out_ip){
if(!host || !out_ip) return false;
uint32_t ip = ipaddr_addr(host);
if(ip!=0 && ip!=0xFFFFFFFF){
*out_ip=ip;
return true;
}
ESP_LOGE(TAG,"resolve fail (DNS not supported) %s",host);
return false;
}
void normalize_audio_url(const char* in, char* out, size_t out_len){
if(!in || !out) return;
if(strncmp(in,"http://",7)==0){
strncpy(out,in,out_len-1);
out[out_len-1]=0;
return;
}
if(in[0]=='/'){
snprintf(out,out_len,"%s%s",PB,in);
return;
}
if(strstr(in,"://")==NULL){
if(in[0]=='/') snprintf(out,out_len,"%s%s",PB,in);
else snprintf(out,out_len,"%s/%s",PB,in);
return;
}
strncpy(out,in,out_len-1);
out[out_len-1]=0;
}
int http_get_raw(const char* url, char** out_body){
if(!url || strncmp(url,"http://",7)!=0){ ESP_LOGE(TAG,"http_get url not http %s",url); return -1; }
const char* p = url+7;
const char* slash = strchr(p,'/');
if(!slash){ ESP_LOGE(TAG,"no slash %s",url); return -1; }
char hostport[128]={0};
size_t hlen=slash-p;
if(hlen>=sizeof(hostport)) return -1;
memcpy(hostport,p,hlen);
hostport[hlen]=0;
const char* path=slash;
char host[96]={0};
int port=80;
char* colon=strchr(hostport,':');
if(colon){ *colon=0; strncpy(host,hostport,sizeof(host)-1); port=atoi(colon+1); if(port<=0) port=80; } else { strncpy(host,hostport,sizeof(host)-1); }
ESP_LOGI(TAG,"GET %s host=%s port=%d",url,host,port);
uint32_t ip=0;
ESP_LOGI(TAG,"Resolving host %s",host);
if(!resolve_host(host,&ip)){ ESP_LOGE(TAG,"ip fail %s",host); return -1; }
ESP_LOGI(TAG,"Resolved %s -> ip 0x%08x", host, ip);
int fd=lwip_socket(AF_INET,SOCK_STREAM,0);
ESP_LOGI(TAG,"Socket fd=%d", fd);
if(fd<0){ ESP_LOGE(TAG,"socket fail"); return -1; }
struct sockaddr_in sa;
ESP_LOGI(TAG,"Memset sa");
memset(&sa,0,sizeof(sa));
ESP_LOGI(TAG,"Set sa len/family/port/addr");
sa.sin_len = sizeof(sa);
sa.sin_family=AF_INET;
sa.sin_port=my_htons(port);
sa.sin_addr.s_addr=ip;
ESP_LOGI(TAG,"SetSockOpt RCV");
struct timeval tv={5,0};
int opt_res = lwip_setsockopt(fd,SOL_SOCKET,SO_RCVTIMEO,&tv,sizeof(tv));
ESP_LOGI(TAG,"SetSockOpt RCV res=%d", opt_res);
ESP_LOGI(TAG,"SetSockOpt SND");
opt_res = lwip_setsockopt(fd,SOL_SOCKET,SO_SNDTIMEO,&tv,sizeof(tv));
ESP_LOGI(TAG,"SetSockOpt SND res=%d", opt_res);
ESP_LOGI(TAG,"Connecting to %s:%d",host,port);
int conn_res = lwip_connect(fd,(struct sockaddr*)&sa,sizeof(sa));
ESP_LOGI(TAG,"Connect res=%d", conn_res);
if(conn_res<0){ ESP_LOGE(TAG,"connect fail %s:%d",host,port); close(fd); return -1; }
ESP_LOGI(TAG,"Connected, sending request");
char req[512];
int reqlen=snprintf(req,sizeof(req),"GET %s HTTP/1.0\r\nHost: %s:%d\r\nConnection: close\r\nUser-Agent: TactilityDM/0.1\r\nAccept: */*\r\n\r\n",path,host,port);
if(lwip_send(fd,req,reqlen,0)<0){ ESP_LOGE(TAG,"send fail"); close(fd); return -1; }
ESP_LOGI(TAG,"Request sent, waiting for response");
int capacity=8192; // further reduced to 8K internal to avoid PSRAM StoreProhibited crash (seen 0x8208775e)
char* buf=malloc(capacity);
if(!buf){
buf=heap_caps_malloc(capacity, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!buf){ ESP_LOGE(TAG,"buf 8k fail"); close(fd); return -1; }
}
int total=0;
char* header_end=NULL;
int body_start=0;
int content_len=-1;
bool is_chunked=false;
while(1){
// Graceful cancel support to avoid force vTaskDelete leaking PSRAM/SD resources
if(G.fetch_cancel_req){
ESP_LOGI(TAG,"http_get_raw canceled by user");
heap_caps_free(buf);
close(fd);
return -1;
}
if(total>=capacity-1){
int newcap=capacity*2;
if(newcap>600000){ ESP_LOGE(TAG,"too large %d",newcap); heap_caps_free(buf); close(fd); return -1; }
char* nb=heap_caps_malloc(newcap, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!nb){ heap_caps_free(buf); close(fd); return -1; }
memcpy(nb,buf,total);
heap_caps_free(buf);
buf=nb;
capacity=newcap;
}
int r=lwip_recv(fd,buf+total,capacity-total-1,0);
if(r<=0){
if(r<0 && (errno==EAGAIN || errno==EWOULDBLOCK || errno==ETIMEDOUT)){
ESP_LOGW(TAG,"http_get_raw timeout total=%d",total);
// On timeout, if fetching canceled, abort
if(G.fetch_cancel_req){
ESP_LOGI(TAG,"http_get_raw timeout + canceled");
heap_caps_free(buf);
close(fd);
return -1;
}
// Continue loop – LWIP will retry until EOF
// But if we already have header and content_len satisfied, break
if(header_end && content_len>=0 && !is_chunked && total-body_start >= content_len) break;
// If no header yet and timeout, treat as failure to avoid 15s hang
if(!header_end && total==0){
ESP_LOGE(TAG,"http_get_raw timeout before header");
heap_caps_free(buf);
close(fd);
return -1;
}
// Otherwise continue to try again
continue;
}
break;
}
total+=r;
buf[total]=0;
if(!header_end){
header_end=strstr(buf,"\r\n\r\n");
if(header_end){
body_start=(header_end-buf)+4;
if(strstr(buf,"chunked")) is_chunked=true;
char* cl=strstr(buf,"Content-Length:");
if(!cl) cl=strstr(buf,"content-length:");
if(cl){ cl=strchr(cl,':'); if(cl){ cl++; while(*cl==' ') cl++; content_len=atoi(cl); } }
ESP_LOGI(TAG,"header_end chunked=%d cl=%d total=%d",is_chunked,content_len,total);
if(content_len>=0 && !is_chunked){
if(total-body_start >= content_len) break;
}
}
} else {
if(content_len>=0 && !is_chunked){
if(total-body_start >= content_len) break;
}
}
}
close(fd);
if(!header_end){ ESP_LOGE(TAG,"no header end total=%d",total); heap_caps_free(buf); return -1; }
if(strncmp(buf,"HTTP/1.1 200",12)!=0 && strncmp(buf,"HTTP/1.0 200",12)!=0){ ESP_LOGE(TAG,"not 200: %.60s",buf); heap_caps_free(buf); return -1; }
int raw_len=total-body_start;
char* raw=buf+body_start;
char* final_body=NULL;
int final_len=0;
if(is_chunked){
final_body=heap_caps_malloc(200000, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!final_body){ heap_caps_free(buf); return -1; }
int pos=0, out=0;
while(pos<raw_len){
char* crlf=strstr(raw+pos,"\r\n");
if(!crlf) break;
int off=crlf-(raw+pos);
if(off<=0 || off>16){ pos+=off+2; continue; }
char hex[16]={0};
memcpy(hex,raw+pos,off);
char* semi=strchr(hex,';');
if(semi) *semi=0;
long sz=strtol(hex,NULL,16);
if(sz==0) break;
pos+=off+2;
if(pos+sz>raw_len) sz=raw_len-pos;
if(out+sz>=200000) break;
memcpy(final_body+out,raw+pos,sz);
out+=sz;
pos+=sz;
if(pos+1<raw_len && raw[pos]=='\r' && raw[pos+1]=='\n') pos+=2;
}
final_body[out]=0;
final_len=out;
} else {
int bl=raw_len;
if(content_len>=0 && bl>content_len) bl=content_len;
final_body=heap_caps_malloc(bl+1, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!final_body){ heap_caps_free(buf); return -1; }
memcpy(final_body,raw,bl);
final_body[bl]=0;
final_len=bl;
}
heap_caps_free(buf);
*out_body=final_body;
ESP_LOGI(TAG,"GET ok %d",final_len);
return final_len;
}
bool fetch_data(){
char* js=NULL;
char url[320];
bool seasons_ok=false;
bool episodes_ok=false;
ESP_LOGI(TAG,"fetch seasons");
// Reduce payload via fields filter – drastically lowers cJSON malloc pressure (internal heap low warning)
snprintf(url,sizeof(url),"%s/api/collections/dm_seasons/records?perPage=100&sort=season_num&fields=season_num",PB);
int len=http_get_raw(url,&js);
if(len>0 && js){
cJSON* root=cJSON_Parse(js);
free(js); js=NULL;
if(root){
cJSON* items=cJSON_GetObjectItem(root,"items");
if(cJSON_IsArray(items)){
G.season_cnt=0;
int n=cJSON_GetArraySize(items);
for(int i=0;i<n&&G.season_cnt<MAX_SEASONS;i++){
cJSON* it=cJSON_GetArrayItem(items,i);
cJSON* sn=cJSON_GetObjectItem(it,"season_num");
if(sn&&cJSON_IsNumber(sn)){ G.seasons[G.season_cnt].num=sn->valueint; G.season_cnt++; }
}
sort_seasons();
if(G.season_cnt>0) seasons_ok=true;
}
cJSON_Delete(root);
}
} else {
ESP_LOGW(TAG,"seasons fetch failed");
}
if(G.season_cnt==0){
ESP_LOGW(TAG,"seasons fallback");
for(int i=1;i<=37;i++) G.seasons[i-1].num=i;
G.season_cnt=37;
}
ESP_LOGI(TAG,"fetch episodes paginated to reduce internal heap (prev low 7KB)");
G.ep_cnt=0;
for(int page=1; page<=10; page++){
if(G.fetch_cancel_req) break;
if(G.ep_cnt>=MAX_EPS) break;
snprintf(url,sizeof(url),"%s/api/collections/dm_episodes/records?perPage=25&page=%d&sort=season_num,episode_num&fields=title,slug,season_num,episode_num,audio_url",PB,page);
len=http_get_raw(url,&js);
if(len<=0 || !js){
ESP_LOGW(TAG,"episodes page %d fetch failed",page);
if(js){ free(js); js=NULL; }
break;
}
cJSON* root=cJSON_Parse(js);
free(js); js=NULL;
if(!root){
ESP_LOGW(TAG,"episodes page %d JSON parse fail",page);
break;
}
cJSON* items=cJSON_GetObjectItem(root,"items");
if(!cJSON_IsArray(items)){
cJSON_Delete(root);
break;
}
int n=cJSON_GetArraySize(items);
if(n==0){
cJSON_Delete(root);
break;
}
ESP_LOGI(TAG,"episodes page %d got %d items (total so far %d)",page,n,G.ep_cnt);
for(int i=0;i<n&&G.ep_cnt<MAX_EPS;i++){
cJSON* it=cJSON_GetArrayItem(items,i);
cJSON* t=cJSON_GetObjectItem(it,"title");
cJSON* slug=cJSON_GetObjectItem(it,"slug");
cJSON* sn=cJSON_GetObjectItem(it,"season_num");
cJSON* en=cJSON_GetObjectItem(it,"episode_num");
cJSON* au=cJSON_GetObjectItem(it,"audio_url");
if(!sn||!en) continue;
EpInfo* e=&G.eps[G.ep_cnt];
memset(e,0,sizeof(*e));
if(t&&cJSON_IsString(t)) strncpy(e->title,t->valuestring,sizeof(e->title)-1);
if(slug&&cJSON_IsString(slug)) strncpy(e->slug,slug->valuestring,sizeof(e->slug)-1);
else if(t) strncpy(e->slug,t->valuestring,sizeof(e->slug)-1);
e->season=sn->valueint;
e->ep_num=en->valueint;
if(au&&cJSON_IsString(au)) strncpy(e->audio_url,au->valuestring,sizeof(e->audio_url)-1);
e->seen=has_slug(e->slug);
G.ep_cnt++;
}
episodes_ok=true;
cJSON_Delete(root);
// Yield between pages to let MemoryChecker recover
vTaskDelay(pdMS_TO_TICKS(400));
if(n<25) break; // last page
}
// Final sort after all pages
if(G.ep_cnt>0) sort_eps();
if(G.ep_cnt==0){
ESP_LOGW(TAG,"episodes fallback");
for(int i=0;i<6;i++){
EpInfo* e=&G.eps[i];
snprintf(e->slug,sizeof(e->slug),"S01E%02d",i+1);
snprintf(e->title,sizeof(e->title),"Episode %d",i+1);
e->season=1; e->ep_num=i+1;
}
G.ep_cnt=6;
}
for(int i=0;i<G.ep_cnt;i++) G.eps[i].seen=has_slug(G.eps[i].slug);
return (seasons_ok && episodes_ok);
}
@@ -0,0 +1,23 @@
#pragma once
#include "app_context.h"
#include <stdbool.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
uint16_t my_htons(uint16_t v);
bool resolve_host(const char* host, uint32_t* out_ip);
void normalize_audio_url(const char* in, char* out, size_t out_len);
// Robust HTTP GET with PSRAM buffer, returns length and allocates *out_body (must be free'd via free/heap_caps_free)
int http_get_raw(const char* url, char** out_body);
// Fetch seasons and episodes from PB – returns true if network fetch succeeded with real data
bool fetch_data(void);
#ifdef __cplusplus
}
#endif
+292
View File
@@ -0,0 +1,292 @@
#include "player.h"
#include "storage.h"
#include "ui.h"
#include "download.h"
#include <tt_lvgl.h>
#include <tt_app.h>
#include <tt_bundle.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_heap_caps.h"
void wait_play_exit(void) {
// No internal playback anymore, nothing to wait
}
void stop_playback_sync(void) {
// No internal playback, nothing to stop
G.is_playing = false;
G.is_paused = false;
}
static bool try_external_downloader(int idx) {
if(idx<0||idx>=G.ep_cnt) return false;
EpInfo* ep=&G.eps[idx];
if (strlen(ep->audio_url) < 8) {
ESP_LOGW(TAG, "Skipping DL for %s: empty URL", ep->slug);
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) lv_label_set_text(G.lbl_status, "No URL – fetch failed?");
tt_lvgl_unlock();
return false;
}
char fpath[300];
make_sd_path(fpath, sizeof(fpath), ep->slug);
tt_lvgl_lock(portMAX_DELAY);
close_overlay();
if(G.dl_overlay) hide_dl_overlay();
tt_lvgl_unlock();
BundleHandle b = tt_bundle_alloc();
if (!b) {
ESP_LOGW(TAG, "Failed to alloc bundle for external DL");
return false;
}
tt_bundle_put_string(b, "url", ep->audio_url);
tt_bundle_put_string(b, "path", fpath);
tt_bundle_put_bool(b, "override", false);
ESP_LOGI(TAG, "Starting external SdDownloader for %s -> %s", ep->slug, fpath);
G.pending_dl_idx = idx;
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) lv_label_set_text(G.lbl_status, "Starting downloader...");
tt_lvgl_unlock();
tt_app_start_with_bundle("one.tactility.sddownloader", b);
return true;
}
int count_missing_in_season(int season){
int cnt=0;
for(int i=0;i<G.ep_cnt;i++){
if(G.eps[i].season==season && !G.eps[i].seen && strlen(G.eps[i].audio_url)>=8) cnt++;
}
return cnt;
}
int find_next_missing_in_season(int season){
for(int i=0;i<G.ep_cnt;i++){
if(G.eps[i].season==season && !G.eps[i].seen && strlen(G.eps[i].audio_url)>=8) return i;
}
return -1;
}
bool download_episode(int idx){
return try_external_downloader(idx);
}
bool download_missing_next(void){
if(!G.dl_missing_active) return false;
int next = find_next_missing_in_season(G.dl_missing_season);
if(next==-1){
// No more missing – done
G.dl_missing_active=false;
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status){
char msg[64];
snprintf(msg,sizeof(msg),"DL S%02d done %d ok %d fail", G.dl_missing_season, G.dl_missing_done, G.dl_missing_failed);
lv_label_set_text(G.lbl_status, msg);
}
tt_lvgl_unlock();
return false;
}
// Update UI for next file
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status){
char msg[64];
snprintf(msg,sizeof(msg),"DL %d/%d S%02d...", G.dl_missing_done+1, G.dl_missing_total, G.dl_missing_season);
lv_label_set_text(G.lbl_status, msg);
}
tt_lvgl_unlock();
return try_external_downloader(next);
}
bool download_missing_season_start(int season){
int missing = count_missing_in_season(season);
if(missing==0){
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status) lv_label_set_text(G.lbl_status,"Season already cached");
tt_lvgl_unlock();
G.dl_missing_active=false;
return true;
}
ESP_LOGI(TAG, "Starting sequential DL for S%02d, %d missing", season, missing);
G.dl_missing_active=true;
G.dl_missing_season=season;
G.dl_missing_total=missing;
G.dl_missing_done=0;
G.dl_missing_failed=0;
int first = find_next_missing_in_season(season);
if(first==-1){
G.dl_missing_active=false;
return true;
}
// Update status
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status){
char msg[64];
snprintf(msg,sizeof(msg),"DL 1/%d S%02d...", missing, season);
lv_label_set_text(G.lbl_status, msg);
}
tt_lvgl_unlock();
return try_external_downloader(first);
}
static bool try_external_downloader_batch(int season) {
// Sequential queue – safe single-file mode
return download_missing_season_start(season);
}
void start_idx_internal(int idx){
if(idx<0||idx>=G.ep_cnt) return;
EpInfo* ep=&G.eps[idx];
char fpath[300];
make_sd_path(fpath, sizeof(fpath), ep->slug);
if(!has_slug(ep->slug)){
ESP_LOGW(TAG, "start_idx_internal called but file missing %s", ep->slug);
start_idx(idx);
return;
}
int saved = 0;
if(G.cur_ep_idx!=idx){
saved = get_saved_pos_for_slug(ep->slug);
if(saved>5){
ESP_LOGI(TAG,"Restoring saved pos %d for %s", saved, ep->slug);
}
} else {
saved = G.pos_sec;
}
tt_lvgl_lock(portMAX_DELAY);
close_overlay();
if(G.dl_overlay) hide_dl_overlay();
tt_lvgl_unlock();
BundleHandle b = tt_bundle_alloc();
if(b){
tt_bundle_put_string(b, "file", fpath);
tt_bundle_put_string(b, "path", fpath);
tt_bundle_put_int32(b, "pos_sec", saved);
tt_bundle_put_int32(b, "position", saved);
tt_bundle_put_int32(b, "volume", G.volume);
ESP_LOGI(TAG, "Launching Mp3Player for %s pos %d", fpath, saved);
G.pending_dl_idx = idx;
G.cur_ep_idx=idx; G.cur_season=ep->season;
strncpy(G.cur_title,ep->title,sizeof(G.cur_title)-1);
strncpy(G.last_slug,ep->slug,sizeof(G.last_slug)-1);
save_state();
tt_app_start_with_bundle("one.tactility.mp3player", b);
return;
}
ESP_LOGW(TAG, "Failed to alloc bundle for Mp3Player");
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status) lv_label_set_text(G.lbl_status,"Failed to launch player");
tt_lvgl_unlock();
}
void start_idx(int idx){
if(idx<0||idx>=G.ep_cnt) return;
EpInfo* ep=&G.eps[idx];
if(!has_slug(ep->slug)){
// File not on SD, download via external downloader
if (try_external_downloader(idx)) {
return;
}
// Fallback: show message, cannot download internally anymore
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status) lv_label_set_text(G.lbl_status,"Install SdDownloader");
tt_lvgl_unlock();
return;
}
start_idx_internal(idx);
}
void select_ep(int idx){
if(idx<0||idx>=G.ep_cnt) return;
if(G.cur_ep_idx>=0 && G.cur_ep_idx < G.ep_cnt && G.cur_ep_idx!=idx){
save_current_ep_pos();
}
G.cur_ep_idx=idx;
EpInfo* ep=&G.eps[idx];
G.cur_season=ep->season;
strncpy(G.last_slug, ep->slug, sizeof(G.last_slug)-1);
int saved = get_saved_pos_for_slug(ep->slug);
G.pos_sec=saved;
G.last_pct=-1;
G.total_sec=0;
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_title) lv_label_set_text(G.lbl_title,ep->title);
if(G.lbl_meta){
char b[64];
if(saved>5){
char tbuf[16];
fmt_time(tbuf,saved);
snprintf(b,sizeof(b),"S%02d E%02d %s • %s",ep->season,ep->ep_num,ep->seen?"*":"",tbuf);
} else {
snprintf(b,sizeof(b),"S%02d E%02d %s",ep->season,ep->ep_num,ep->seen?"*":"");
}
lv_label_set_text(G.lbl_meta,b);
}
if(G.lbl_status){
if(saved>5) lv_label_set_text(G.lbl_status,"Resume – tap Play");
else lv_label_set_text(G.lbl_status,ep->seen?"On SD – tap Play":"Tap Play to download");
}
if(G.bar){
lv_bar_set_value(G.bar,0,LV_ANIM_OFF);
}
if(G.lbl_time){
if(saved>5){
char a[16];
fmt_time(a,saved);
char buf[40];
snprintf(buf,sizeof(buf),"%s / --:--",a);
lv_label_set_text(G.lbl_time,buf);
} else {
lv_label_set_text(G.lbl_time,"0:00 / --:--");
}
}
tt_lvgl_unlock();
save_state();
}
void go_next(){
if(G.ep_cnt==0) return;
save_current_ep_pos();
save_state();
int n=G.cur_ep_idx+1;
if(n>=G.ep_cnt) n=0;
select_ep(n);
// Do not auto-play next, per user request: no autoplay
}
void go_prev(){
if(G.ep_cnt==0) return;
if(G.pos_sec>5){
if(G.cur_ep_idx>=0){
set_saved_pos_for_slug(G.eps[G.cur_ep_idx].slug, 0);
save_state();
}
G.pos_sec=0;
int cur=G.cur_ep_idx;
if(cur<0) cur=0;
select_ep(cur);
return;
}
save_current_ep_pos();
save_state();
int p=G.cur_ep_idx-1;
if(p<0) p=G.ep_cnt-1;
select_ep(p);
}
// Called from UI for season batch download
bool download_season(int season) {
return try_external_downloader_batch(season);
}
@@ -0,0 +1,30 @@
#pragma once
#include "app_context.h"
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
// Simplified: no internal audio, only external apps
void wait_play_exit(void);
void stop_playback_sync(void);
void start_idx_internal(int idx);
void start_idx(int idx);
void select_ep(int idx);
void go_next(void);
void go_prev(void);
bool download_season(int season);
// New single-file download API
bool download_episode(int idx);
bool download_missing_season_start(int season);
bool download_missing_next(void); // continue queue, returns false if done
int count_missing_in_season(int season);
int find_next_missing_in_season(int season);
#ifdef __cplusplus
}
#endif
@@ -0,0 +1,321 @@
#include "storage.h"
#include "cJSON.h"
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <dirent.h>
#include <sys/stat.h>
#include <unistd.h>
#include "esp_heap_caps.h"
void make_sd_path(char* out, size_t out_len, const char* slug){
if(!out || !slug) return;
snprintf(out, out_len, "%s/%s.mp3", DM_DIR, slug);
}
void ensure_dir(){
mkdir("/sdcard/apps",0777);
mkdir("/sdcard/apps/one.tactility.discoverymountain",0777);
mkdir("/sdcard/apps/one.tactility.discoverymountain/userdata",0777);
mkdir(DM_DIR,0777);
}
bool has_slug(const char* slug){
char p[300];
make_sd_path(p, sizeof(p), slug);
struct stat st;
return stat(p,&st)==0;
}
void fmt_time(char* o,int s){ snprintf(o,16,"%d:%02d",s/60,s%60); }
void sort_seasons(){
for(int i=0;i<G.season_cnt;i++){
for(int j=i+1;j<G.season_cnt;j++){
if(G.seasons[j].num < G.seasons[i].num){
SeasonInfo t=G.seasons[i]; G.seasons[i]=G.seasons[j]; G.seasons[j]=t;
}
}
}
}
void sort_eps(){
// Use PSRAM for temp to avoid stack overflow (EpInfo ~300 bytes, was on stack)
EpInfo *t = heap_caps_malloc(sizeof(EpInfo), MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!t) return;
for(int i=0;i<G.ep_cnt;i++){
for(int j=i+1;j<G.ep_cnt;j++){
EpInfo *a=&G.eps[i], *b=&G.eps[j];
bool should_swap=false;
if(a->season != b->season) should_swap = (b->season < a->season);
else should_swap = (b->ep_num < a->ep_num);
if(should_swap){
*t=*a; *a=*b; *b=*t;
}
}
}
heap_caps_free(t);
}
// ── Per-episode position helpers ──
int find_saved_pos_idx(const char* slug){
if(!slug) return -1;
for(int i=0;i<G.saved_pos_cnt;i++){
if(strcmp(G.saved_pos[i].slug, slug)==0) return i;
}
return -1;
}
int get_saved_pos_for_slug(const char* slug){
int idx=find_saved_pos_idx(slug);
if(idx>=0) return G.saved_pos[idx].pos;
return 0;
}
void set_saved_pos_for_slug(const char* slug, int pos){
if(!slug || strlen(slug)==0) return;
if(pos<0) pos=0;
if(G.total_sec>0 && pos>0 && pos >= G.total_sec-10){
pos=0;
}
int idx=find_saved_pos_idx(slug);
if(idx>=0){
G.saved_pos[idx].pos=pos;
return;
}
if(G.saved_pos_cnt < MAX_EPS){
strncpy(G.saved_pos[G.saved_pos_cnt].slug, slug, sizeof(G.saved_pos[G.saved_pos_cnt].slug)-1);
G.saved_pos[G.saved_pos_cnt].slug[sizeof(G.saved_pos[G.saved_pos_cnt].slug)-1]=0;
G.saved_pos[G.saved_pos_cnt].pos=pos;
G.saved_pos_cnt++;
}
}
void save_current_ep_pos(){
if(G.cur_ep_idx>=0 && G.cur_ep_idx < G.ep_cnt){
const char* slug=G.eps[G.cur_ep_idx].slug;
set_saved_pos_for_slug(slug, G.pos_sec);
} else if(strlen(G.last_slug)>0){
set_saved_pos_for_slug(G.last_slug, G.pos_sec);
}
}
void save_state(){
if(G.cur_ep_idx>=0 && G.ep_cnt>0){
if(G.pos_sec>=0){
set_saved_pos_for_slug(G.eps[G.cur_ep_idx].slug, G.pos_sec);
}
}
cJSON* r=cJSON_CreateObject();
if(G.ep_cnt>0 && G.cur_ep_idx>=0 && G.cur_ep_idx < G.ep_cnt) cJSON_AddStringToObject(r,"last_slug",G.eps[G.cur_ep_idx].slug);
else if(strlen(G.last_slug)>0) cJSON_AddStringToObject(r,"last_slug",G.last_slug);
cJSON_AddNumberToObject(r,"last_season",G.cur_season);
cJSON_AddNumberToObject(r,"pos_sec",G.pos_sec);
if(G.saved_pos_cnt>0){
cJSON* pos_obj=cJSON_CreateObject();
for(int i=0;i<G.saved_pos_cnt;i++){
if(G.saved_pos[i].slug[0]==0) continue;
if(G.saved_pos[i].pos>0){
cJSON_AddNumberToObject(pos_obj, G.saved_pos[i].slug, G.saved_pos[i].pos);
}
}
cJSON_AddItemToObject(r,"positions",pos_obj);
}
char* s=cJSON_PrintUnformatted(r);
if(s){
FILE* f=fopen(STATE_PATH,"w");
if(f){ fwrite(s,1,strlen(s),f); fclose(f);}
free(s);
}
cJSON_Delete(r);
}
void load_state(){
G.cur_season=1; G.cur_ep_idx=-1; G.pos_sec=0;
G.saved_pos_cnt=0;
G.last_save_tick=0;
memset(G.last_slug,0,sizeof(G.last_slug));
memset(G.saved_pos,0,sizeof(G.saved_pos));
FILE* f=fopen(STATE_PATH,"r");
if(!f) return;
fseek(f,0,SEEK_END);
long sz=ftell(f);
fseek(f,0,SEEK_SET);
if(sz<=0||sz>16384){ fclose(f); return;}
char* buf=malloc(sz+1);
if(!buf){ fclose(f); return; }
size_t got=fread(buf,1,sz,f);
buf[got]=0;
fclose(f);
cJSON* r=cJSON_Parse(buf);
free(buf);
if(!r) return;
cJSON* ls=cJSON_GetObjectItem(r,"last_slug");
cJSON* lsn=cJSON_GetObjectItem(r,"last_season");
cJSON* ps=cJSON_GetObjectItem(r,"pos_sec");
if(ps&&cJSON_IsNumber(ps)) G.pos_sec=ps->valueint;
if(lsn&&cJSON_IsNumber(lsn)) G.cur_season=lsn->valueint;
if(ls&&cJSON_IsString(ls)){
strncpy(G.last_slug, ls->valuestring, sizeof(G.last_slug)-1);
for(int i=0;i<G.ep_cnt;i++) if(strcmp(G.eps[i].slug,ls->valuestring)==0){ G.cur_ep_idx=i; G.cur_season=G.eps[i].season; break; }
}
cJSON* poss=cJSON_GetObjectItem(r,"positions");
if(poss && cJSON_IsObject(poss)){
cJSON* child=NULL;
cJSON_ArrayForEach(child, poss){
if(cJSON_IsNumber(child) && child->string && G.saved_pos_cnt < MAX_EPS){
strncpy(G.saved_pos[G.saved_pos_cnt].slug, child->string, sizeof(G.saved_pos[G.saved_pos_cnt].slug)-1);
G.saved_pos[G.saved_pos_cnt].pos=child->valueint;
G.saved_pos_cnt++;
}
}
if(strlen(G.last_slug)>0){
int sp=get_saved_pos_for_slug(G.last_slug);
if(sp>0) G.pos_sec=sp;
}
}
cJSON_Delete(r);
}
bool save_cache(void){
if(G.ep_cnt==0 || G.season_cnt==0) return false;
cJSON* root=cJSON_CreateObject();
if(!root) return false;
cJSON* seasons_arr=cJSON_CreateArray();
if(!seasons_arr){ cJSON_Delete(root); return false; }
for(int i=0;i<G.season_cnt && i<MAX_SEASONS;i++){
cJSON_AddItemToArray(seasons_arr, cJSON_CreateNumber(G.seasons[i].num));
}
cJSON_AddItemToObject(root,"seasons",seasons_arr);
cJSON* eps_arr=cJSON_CreateArray();
if(!eps_arr){ cJSON_Delete(root); return false; }
for(int i=0;i<G.ep_cnt && i<MAX_EPS;i++){
EpInfo* e=&G.eps[i];
if(e->slug[0]==0) continue;
cJSON* obj=cJSON_CreateObject();
if(!obj) continue;
cJSON_AddStringToObject(obj,"title", e->title);
cJSON_AddStringToObject(obj,"slug", e->slug);
cJSON_AddNumberToObject(obj,"season_num", e->season);
cJSON_AddNumberToObject(obj,"episode_num", e->ep_num);
cJSON_AddStringToObject(obj,"audio_url", e->audio_url);
cJSON_AddItemToArray(eps_arr, obj);
}
cJSON_AddItemToObject(root,"episodes",eps_arr);
char* s=cJSON_PrintUnformatted(root);
cJSON_Delete(root);
if(!s) return false;
FILE* f=fopen(CACHE_PATH,"w");
bool ok=false;
if(f){
size_t len=strlen(s);
size_t wrote=fwrite(s,1,len,f);
fclose(f);
ok = (wrote==len);
}
free(s);
return ok;
}
bool load_cache(void){
FILE* f=fopen(CACHE_PATH,"r");
if(!f) return false;
fseek(f,0,SEEK_END);
long sz=ftell(f);
fseek(f,0,SEEK_SET);
if(sz<=0 || sz>200000){ fclose(f); return false; }
// Use PSRAM for large json to avoid internal heap pressure
char* buf=heap_caps_malloc(sz+1, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!buf){
buf=malloc(sz+1);
if(!buf){ fclose(f); return false; }
}
size_t got=fread(buf,1,sz,f);
buf[got]=0;
fclose(f);
cJSON* root=cJSON_Parse(buf);
heap_caps_free(buf);
if(!root) return false;
cJSON* seasons_arr=cJSON_GetObjectItem(root,"seasons");
cJSON* eps_arr=cJSON_GetObjectItem(root,"episodes");
if(!cJSON_IsArray(seasons_arr) || !cJSON_IsArray(eps_arr)){
cJSON_Delete(root);
return false;
}
int scnt=cJSON_GetArraySize(seasons_arr);
G.season_cnt=0;
for(int i=0;i<scnt && G.season_cnt<MAX_SEASONS;i++){
cJSON* it=cJSON_GetArrayItem(seasons_arr,i);
if(cJSON_IsNumber(it)){
G.seasons[G.season_cnt].num=it->valueint;
G.season_cnt++;
}
}
int ecnt=cJSON_GetArraySize(eps_arr);
G.ep_cnt=0;
for(int i=0;i<ecnt && G.ep_cnt<MAX_EPS;i++){
cJSON* it=cJSON_GetArrayItem(eps_arr,i);
if(!cJSON_IsObject(it)) continue;
cJSON* t=cJSON_GetObjectItem(it,"title");
cJSON* slug=cJSON_GetObjectItem(it,"slug");
cJSON* sn=cJSON_GetObjectItem(it,"season_num");
cJSON* en=cJSON_GetObjectItem(it,"episode_num");
cJSON* au=cJSON_GetObjectItem(it,"audio_url");
if(!sn || !en) continue;
EpInfo* e=&G.eps[G.ep_cnt];
memset(e,0,sizeof(*e));
if(t && cJSON_IsString(t)) strncpy(e->title, t->valuestring, sizeof(e->title)-1);
if(slug && cJSON_IsString(slug)) strncpy(e->slug, slug->valuestring, sizeof(e->slug)-1);
else if(t && cJSON_IsString(t)) strncpy(e->slug, t->valuestring, sizeof(e->slug)-1);
e->season=sn->valueint;
e->ep_num=en->valueint;
if(au && cJSON_IsString(au)) strncpy(e->audio_url, au->valuestring, sizeof(e->audio_url)-1);
e->seen=has_slug(e->slug);
G.ep_cnt++;
}
cJSON_Delete(root);
if(G.season_cnt>0) sort_seasons();
if(G.ep_cnt>0) sort_eps();
// Refresh seen in case SD changed
for(int i=0;i<G.ep_cnt;i++) G.eps[i].seen=has_slug(G.eps[i].slug);
return (G.season_cnt>0 && G.ep_cnt>0);
}
void lru_check(){
typedef struct{ char path[300]; time_t mt; } FE;
// Allocate from PSRAM to avoid stack overflow (200*~304=60KB on stack would overflow 12KB task)
FE *files = heap_caps_malloc(200 * sizeof(FE), MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if(!files) return;
int fc=0;
DIR* d=opendir(DM_DIR);
if(!d){ heap_caps_free(files); return; }
struct dirent* de;
char cur_path[300]={0};
if(G.cur_ep_idx>=0 && G.cur_ep_idx<G.ep_cnt){
make_sd_path(cur_path, sizeof(cur_path), G.eps[G.cur_ep_idx].slug);
}
while((de=readdir(d))!=NULL && fc<200){
if(de->d_name[0]=='.') continue;
size_t l=strlen(de->d_name);
if(l<5) continue;
if(strcmp(de->d_name+l-4,".mp3")!=0) continue;
char full[300];
snprintf(full,sizeof(full),"%s/%.200s",DM_DIR,de->d_name);
struct stat st;
if(stat(full,&st)!=0) continue;
if(cur_path[0] && strcmp(full,cur_path)==0) continue;
strncpy(files[fc].path,full,sizeof(files[fc].path)-1);
files[fc].mt=st.st_mtime;
fc++;
}
closedir(d);
if(fc<12){ heap_caps_free(files); return; }
for(int i=0;i<fc;i++){
for(int j=i+1;j<fc;j++){
if(files[j].mt<files[i].mt){ FE t=files[i]; files[i]=files[j]; files[j]=t; }
}
}
int todel=fc-8;
for(int i=0;i<todel;i++) unlink(files[i].path);
heap_caps_free(files);
}
@@ -0,0 +1,42 @@
#pragma once
#include "app_context.h"
#include <stddef.h>
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
// Path helpers
void make_sd_path(char* out, size_t out_len, const char* slug);
void ensure_dir(void);
bool has_slug(const char* slug);
// LRU cache cleanup
void lru_check(void);
// Sorting
void sort_seasons(void);
void sort_eps(void);
// Time formatting
void fmt_time(char* o, int s);
// Per-episode resume helpers
int find_saved_pos_idx(const char* slug);
int get_saved_pos_for_slug(const char* slug);
void set_saved_pos_for_slug(const char* slug, int pos);
void save_current_ep_pos(void);
// Persistence
void save_state(void);
void load_state(void);
// Seasons / episodes cache (PB data, rarely changes)
bool save_cache(void);
bool load_cache(void);
#ifdef __cplusplus
}
#endif
+554
View File
@@ -0,0 +1,554 @@
#include "ui.h"
#include "storage.h"
#include "player.h"
#include "download.h"
#include <tt_lvgl.h>
#include <tt_app.h>
#include <tt_bundle.h>
#include <tactility/lvgl_fonts.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include "esp_log.h"
#include "esp_heap_caps.h"
void close_overlay(){
if(G.overlay){
lv_obj_delete(G.overlay);
G.overlay=NULL;
G.dropdown=NULL;
G.season_list_cont=NULL;
}
}
static void overlay_close_timer_cb(lv_timer_t* t){
close_overlay();
lv_timer_delete(t);
}
static void close_overlay_safe(void){
// Defer deletion to avoid deleting ancestor of current event target (season buttons inside overlay)
lv_timer_create(overlay_close_timer_cb, 20, NULL);
}
// Forward decls
static void ep_action_cb(lv_event_t* e);
static void season_select_cb(lv_event_t* e);
static void rebuild_episode_list_locked(void);
static void clear_container(lv_obj_t* cont){
if(!cont) return;
uint32_t child_cnt = lv_obj_get_child_cnt(cont);
while(child_cnt>0){
lv_obj_t* child = lv_obj_get_child(cont, 0);
if(child) lv_obj_delete(child);
else break;
child_cnt = lv_obj_get_child_cnt(cont);
}
}
static void btn_close_cb(lv_event_t* e){
(void)e;
if(G.overlay){
close_overlay();
return;
}
tt_app_stop();
}
static void get_season_stats(int season, int* total, int* cached){
int t=0,c=0;
for(int i=0;i<G.ep_cnt;i++){
if(G.eps[i].season==season){
t++;
if(G.eps[i].seen) c++;
}
}
if(total) *total=t;
if(cached) *cached=c;
}
void ui_update_status_locked(const char* msg){
if(G.lbl_status && msg) lv_label_set_text(G.lbl_status, msg);
}
void ui_update_header_locked(void){
if(!G.lbl_title && !G.lbl_season) return;
if(G.fetching){
if(G.lbl_season) lv_label_set_text(G.lbl_season, "Fetching...");
return;
}
if(G.season_cnt==0){
if(G.lbl_season) lv_label_set_text(G.lbl_season, "No seasons");
return;
}
int total=0,cached=0;
get_season_stats(G.cur_season,&total,&cached);
char buf[64];
if(total>0){
snprintf(buf,sizeof(buf),"S%02d • %d/%d on SD • %d eps", G.cur_season, cached, total, total);
}else{
snprintf(buf,sizeof(buf),"S%02d • %d eps", G.cur_season, total);
}
if(G.lbl_season) lv_label_set_text(G.lbl_season, buf);
if(G.lbl_title){
// Keep app name
// optionally update title to count
lv_label_set_text(G.lbl_title, "Discovery Mountain");
}
}
static void rebuild_episode_list_locked(void){
if(!G.ep_list_cont) return;
clear_container(G.ep_list_cont);
if(G.fetching){
lv_obj_t* lbl=lv_label_create(G.ep_list_cont);
lv_label_set_text(lbl,"Fetching episodes...");
lv_obj_set_style_text_color(lbl, lv_color_hex(0xAAAAAA),0);
lv_obj_align(lbl,LV_ALIGN_TOP_MID,0,20);
return;
}
if(G.ep_cnt==0){
lv_obj_t* lbl=lv_label_create(G.ep_list_cont);
lv_label_set_text(lbl,"No episodes");
lv_obj_set_style_text_color(lbl, lv_color_hex(0xAAAAAA),0);
return;
}
// Count episodes for cur season
int count=0;
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==G.cur_season) count++;
if(count==0){
lv_obj_t* lbl=lv_label_create(G.ep_list_cont);
lv_label_set_text(lbl,"No episodes in season");
lv_obj_set_style_text_color(lbl, lv_color_hex(0xAAAAAA),0);
// Also show all if fallback?
// list first few from all
return;
}
for(int i=0;i<G.ep_cnt;i++){
if(G.eps[i].season!=G.cur_season) continue;
EpInfo* ep=&G.eps[i];
// Row container – optimized for low internal heap (4 objs per ep)
lv_obj_t* row=lv_obj_create(G.ep_list_cont);
lv_obj_set_size(row, LV_PCT(100), 50);
lv_obj_set_style_bg_color(row, ep->seen ? lv_color_hex(0x1E2A22) : lv_color_hex(0x222222),0);
lv_obj_set_style_bg_opa(row, LV_OPA_COVER,0);
lv_obj_set_style_radius(row,8,0);
lv_obj_set_style_pad_all(row,6,0);
lv_obj_set_style_border_width(row,1,0);
lv_obj_set_style_border_color(row, ep->seen ? lv_color_hex(0x2E7D32) : lv_color_hex(0x333333),0);
lv_obj_set_flex_flow(row, LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(row, LV_FLEX_ALIGN_SPACE_BETWEEN, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_remove_flag(row, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_set_style_pad_gap(row,6,0);
lv_obj_add_flag(row, LV_OBJ_FLAG_CLICKABLE);
lv_obj_add_event_cb(row, ep_action_cb, LV_EVENT_CLICKED, (void*)(intptr_t)i);
// Left single label with title (truncated) – we embed meta as second line via \n to save objects
lv_obj_t* ltitle=lv_label_create(row);
lv_obj_set_flex_grow(ltitle,1);
// Truncate title to ~26 chars for display
char short_title[30];
strncpy(short_title, ep->title, sizeof(short_title)-1);
short_title[sizeof(short_title)-1]=0;
if(strlen(ep->title) > 26){
short_title[23]='.';
short_title[24]='.';
short_title[25]='.';
short_title[26]=0;
}
char line1[64];
snprintf(line1,sizeof(line1),"E%02d: %s", ep->ep_num, short_title);
// Second line status
char line2[32];
int saved = get_saved_pos_for_slug(ep->slug);
if(ep->seen){
if(saved>5){
char tb[16]; fmt_time(tb,saved);
snprintf(line2,sizeof(line2),"On SD %s", tb);
}else{
snprintf(line2,sizeof(line2),"On SD");
}
}else{
snprintf(line2,sizeof(line2),"Not DL");
}
char tbuf[96];
snprintf(tbuf,sizeof(tbuf),"%s\n%s", line1, line2);
lv_label_set_text(ltitle, tbuf);
lv_obj_set_width(ltitle, LV_PCT(100));
lv_label_set_long_mode(ltitle, LV_LABEL_LONG_DOT);
lv_obj_set_style_text_font(ltitle, lvgl_get_text_font(FONT_SIZE_SMALL),0);
lv_obj_set_style_text_color(ltitle, ep->seen ? lv_color_hex(0xE0E0E0) : lv_color_hex(0xCCCCCC),0);
// Right side: button
lv_obj_t* btn=lv_btn_create(row);
lv_obj_set_size(btn, 52, 34);
if(ep->seen){
lv_obj_set_style_bg_color(btn, lv_color_hex(0x2E7D32),0);
}else{
lv_obj_set_style_bg_color(btn, lv_color_hex(0x1E88E5),0);
}
lv_obj_set_style_radius(btn,6,0);
lv_obj_t* bl=lv_label_create(btn);
lv_label_set_text(bl, ep->seen ? LV_SYMBOL_PLAY : LV_SYMBOL_DOWNLOAD);
lv_obj_center(bl);
lv_obj_add_event_cb(btn, ep_action_cb, LV_EVENT_CLICKED, (void*)(intptr_t)i);
}
// Scroll to top after rebuild
lv_obj_scroll_to_y(G.ep_list_cont, 0, LV_ANIM_OFF);
}
void ui_rebuild_episode_list(void){
tt_lvgl_lock(portMAX_DELAY);
if(G.ep_list_cont) rebuild_episode_list_locked();
ui_update_header_locked();
tt_lvgl_unlock();
}
static void ep_action_cb(lv_event_t* e){
if(lv_event_get_code(e)!=LV_EVENT_CLICKED) return;
// Retrieve idx from user data
int idx = (int)(intptr_t)lv_event_get_user_data(e);
// Fallback: try event target's user data via current target?
// lv_event_get_user_data should already give us the idx
if(idx<0 || idx>=G.ep_cnt) return;
if(G.downloading || G.fetching) return;
EpInfo* ep=&G.eps[idx];
ESP_LOGI(TAG,"ep_action idx %d S%02dE%02d seen %d", idx, ep->season, ep->ep_num, ep->seen);
if(ep->season != G.cur_season){
// Shouldn't happen but update cur season to match
G.cur_season = ep->season;
}
// Use start_idx which handles both DL and Play
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status){
if(ep->seen) lv_label_set_text(G.lbl_status,"Starting player...");
else lv_label_set_text(G.lbl_status,"Starting downloader...");
}
tt_lvgl_unlock();
start_idx(idx);
}
static void season_select_cb(lv_event_t* e){
if(lv_event_get_code(e)!=LV_EVENT_CLICKED) return;
int season_num = (int)(intptr_t)lv_event_get_user_data(e);
if(season_num<=0) return;
ESP_LOGI(TAG,"Season selected S%02d", season_num);
// Defer overlay deletion – deleting ancestor during event is unsafe
close_overlay_safe();
G.cur_season = season_num;
// Save state immediately
tt_lvgl_lock(portMAX_DELAY);
ui_update_header_locked();
rebuild_episode_list_locked();
if(G.lbl_status){
char buf[40];
snprintf(buf,sizeof(buf),"Switched to S%02d", season_num);
lv_label_set_text(G.lbl_status, buf);
}
tt_lvgl_unlock();
save_state();
// Jump to first unseen or first in season
int first=-1, first_unseen=-1;
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==season_num){
if(first==-1) first=i;
if(!G.eps[i].seen && first_unseen==-1) first_unseen=i;
}
int tgt = first_unseen!=-1?first_unseen:first;
if(tgt!=-1){
// Optional: select ep updates title but we already rebuilt list; keep cur_ep_idx
G.cur_ep_idx = tgt;
strncpy(G.last_slug, G.eps[tgt].slug, sizeof(G.last_slug)-1);
save_state();
}
}
void dropdown_cb(lv_event_t* e){
if(lv_event_get_code(e)!=LV_EVENT_VALUE_CHANGED) return;
lv_obj_t* dd = (lv_obj_t*)lv_event_get_target(e);
int sel=lv_dropdown_get_selected(dd);
if(sel<0||sel>=G.season_cnt) return;
int sn=G.seasons[sel].num;
close_overlay_safe();
int first=-1, first_unseen=-1;
for(int i=0;i<G.ep_cnt;i++) if(G.eps[i].season==sn){
if(first==-1) first=i;
if(!G.eps[i].seen && first_unseen==-1) first_unseen=i;
}
int tgt=first_unseen!=-1?first_unseen:first;
if(tgt!=-1){
G.cur_season=sn;
G.cur_ep_idx=tgt;
strncpy(G.last_slug, G.eps[tgt].slug, sizeof(G.last_slug)-1);
tt_lvgl_lock(portMAX_DELAY);
ui_update_header_locked();
rebuild_episode_list_locked();
tt_lvgl_unlock();
save_state();
}
}
static void close_btn_cb(lv_event_t* e){
(void)e;
close_overlay_safe();
}
void btn_seasons_cb(lv_event_t* e){
(void)e;
if(G.downloading) return;
if(G.fetching) return;
if(G.overlay){ close_overlay(); return; }
tt_lvgl_lock(portMAX_DELAY);
// Create overlay centered modal – parented to root if possible to avoid lingering on app background
lv_obj_t* ovl_parent = G.root ? G.root : lv_scr_act();
G.overlay=lv_obj_create(ovl_parent);
lv_obj_set_size(G.overlay, 240, 280);
lv_obj_center(G.overlay);
lv_obj_set_style_bg_color(G.overlay,lv_color_hex(0x222222),0);
lv_obj_set_style_radius(G.overlay,12,0);
lv_obj_set_style_border_width(G.overlay,1,0);
lv_obj_set_style_border_color(G.overlay, lv_color_hex(0x444444),0);
lv_obj_set_style_pad_all(G.overlay,8,0);
lv_obj_set_flex_flow(G.overlay, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(G.overlay, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_gap(G.overlay,6,0);
lv_obj_t* title=lv_label_create(G.overlay);
lv_label_set_text(title,"Select Season");
lv_obj_set_style_text_font(title, lvgl_get_text_font(FONT_SIZE_DEFAULT),0);
// Scrollable list container
G.season_list_cont=lv_obj_create(G.overlay);
lv_obj_set_size(G.season_list_cont, LV_PCT(100), 200);
lv_obj_set_flex_grow(G.season_list_cont,1);
lv_obj_set_style_bg_color(G.season_list_cont, lv_color_hex(0x1A1A1A),0);
lv_obj_set_style_radius(G.season_list_cont,8,0);
lv_obj_set_style_pad_all(G.season_list_cont,4,0);
lv_obj_set_flex_flow(G.season_list_cont, LV_FLEX_FLOW_COLUMN);
lv_obj_set_style_pad_gap(G.season_list_cont,4,0);
// Ensure scrollable
lv_obj_add_flag(G.season_list_cont, LV_OBJ_FLAG_SCROLLABLE);
for(int i=0;i<G.season_cnt;i++){
int sn=G.seasons[i].num;
int total,cached;
get_season_stats(sn,&total,&cached);
lv_obj_t* btn=lv_btn_create(G.season_list_cont);
lv_obj_set_size(btn, LV_PCT(100), 38);
lv_obj_set_style_bg_color(btn, sn==G.cur_season ? lv_color_hex(0x2E7D32) : lv_color_hex(0x333333),0);
lv_obj_set_style_radius(btn,6,0);
lv_obj_set_style_pad_all(btn,6,0);
lv_obj_add_event_cb(btn, season_select_cb, LV_EVENT_CLICKED, (void*)(intptr_t)sn);
char blabel[40];
if(total>0){
if(sn==G.cur_season) snprintf(blabel,sizeof(blabel),"S%02d %d/%d " LV_SYMBOL_OK, sn, cached, total);
else snprintf(blabel,sizeof(blabel),"S%02d %d/%d", sn, cached, total);
}else{
if(sn==G.cur_season) snprintf(blabel,sizeof(blabel),"S%02d " LV_SYMBOL_OK, sn);
else snprintf(blabel,sizeof(blabel),"S%02d", sn);
}
lv_obj_t* l1=lv_label_create(btn);
lv_label_set_text(l1, blabel);
lv_obj_set_style_text_font(l1, lvgl_get_text_font(FONT_SIZE_SMALL),0);
lv_obj_center(l1);
}
lv_obj_t* bc=lv_btn_create(G.overlay);
lv_obj_set_size(bc, LV_PCT(60), 36);
lv_obj_set_style_bg_color(bc, lv_color_hex(0x444444),0);
lv_obj_t* lc=lv_label_create(bc);
lv_label_set_text(lc,"Close");
lv_obj_center(lc);
lv_obj_add_event_cb(bc,close_btn_cb,LV_EVENT_CLICKED,NULL);
tt_lvgl_unlock();
}
void btn_dl_season_cb(lv_event_t* e){
(void)e;
if(G.downloading) return;
if(G.fetching) return;
if(G.ep_cnt==0) return;
int season = G.cur_season;
ESP_LOGI(TAG, "DL Missing S%02d pressed", season);
tt_lvgl_lock(portMAX_DELAY);
close_overlay();
if(G.dl_overlay) hide_dl_overlay();
if(G.lbl_status){
char buf[40];
snprintf(buf,sizeof(buf),"Checking S%02d missing...", season);
lv_label_set_text(G.lbl_status, buf);
}
tt_lvgl_unlock();
if (!download_season(season)) {
tt_lvgl_lock(portMAX_DELAY);
if(G.lbl_status) lv_label_set_text(G.lbl_status,"DL Season failed – no missing?");
tt_lvgl_unlock();
}
}
void build_ui(lv_obj_t* parent){
tt_lvgl_lock(portMAX_DELAY);
lv_obj_t* root = lv_obj_create(parent);
G.root = root;
lv_obj_set_size(root,LV_PCT(100),LV_PCT(100));
lv_obj_set_style_bg_color(root,lv_color_hex(0x111111),0);
lv_obj_set_style_pad_all(root,6,0);
lv_obj_set_style_pad_gap(root,6,0);
lv_obj_set_style_border_width(root,0,0);
lv_obj_set_flex_flow(root, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(root, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START);
lv_obj_remove_flag(root, LV_OBJ_FLAG_SCROLLABLE);
// Header container
lv_obj_t* header=lv_obj_create(root);
lv_obj_set_size(header, LV_PCT(100), LV_SIZE_CONTENT);
lv_obj_set_style_bg_opa(header, LV_OPA_TRANSP,0);
lv_obj_set_style_border_width(header,0,0);
lv_obj_set_style_pad_all(header,2,0);
lv_obj_set_style_pad_gap(header,6,0);
lv_obj_set_flex_flow(header, LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(header, LV_FLEX_ALIGN_SPACE_BETWEEN, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_remove_flag(header, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_t* header_left=lv_obj_create(header);
lv_obj_set_size(header_left, LV_PCT(80), LV_SIZE_CONTENT);
lv_obj_set_style_bg_opa(header_left, LV_OPA_TRANSP,0);
lv_obj_set_style_border_width(header_left,0,0);
lv_obj_set_style_pad_all(header_left,0,0);
lv_obj_set_flex_flow(header_left, LV_FLEX_FLOW_COLUMN);
lv_obj_set_style_pad_gap(header_left,2,0);
lv_obj_remove_flag(header_left, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_set_flex_grow(header_left,1);
G.lbl_title=lv_label_create(header_left);
lv_label_set_text(G.lbl_title,"Discovery Mountain");
lv_obj_set_style_text_font(G.lbl_title, lvgl_get_text_font(FONT_SIZE_DEFAULT), 0);
lv_obj_set_style_text_color(G.lbl_title, lv_color_hex(0xFFFFFF),0);
G.lbl_season=lv_label_create(header_left);
lv_label_set_text(G.lbl_season,"Loading seasons...");
lv_obj_set_style_text_font(G.lbl_season, lvgl_get_text_font(FONT_SIZE_SMALL),0);
lv_obj_set_style_text_color(G.lbl_season,lv_color_hex(0xAAAAAA),0);
// Close button top right
lv_obj_t* btn_close=lv_btn_create(header);
lv_obj_set_size(btn_close, 36, 32);
lv_obj_set_style_bg_color(btn_close, lv_color_hex(0x333333),0);
lv_obj_set_style_radius(btn_close,6,0);
lv_obj_t* l_close=lv_label_create(btn_close);
lv_label_set_text(l_close, LV_SYMBOL_CLOSE);
lv_obj_set_style_text_font(l_close, lvgl_get_text_font(FONT_SIZE_SMALL),0);
lv_obj_center(l_close);
lv_obj_add_event_cb(btn_close, btn_close_cb, LV_EVENT_CLICKED, NULL);
// Action row: Seasons + DL Missing
lv_obj_t* row=lv_obj_create(root);
lv_obj_set_size(row,LV_PCT(100),44);
lv_obj_set_flex_flow(row,LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(row,LV_FLEX_ALIGN_SPACE_EVENLY,LV_FLEX_ALIGN_CENTER,LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_bg_opa(row,LV_OPA_TRANSP,0);
lv_obj_set_style_border_width(row,0,0);
lv_obj_set_style_pad_all(row,0,0);
lv_obj_set_style_pad_gap(row,8,0);
lv_obj_remove_flag(row, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_t* bs=lv_btn_create(row);
lv_obj_set_size(bs,LV_PCT(48),36);
G.btn_seasons=bs;
lv_obj_set_style_bg_color(bs, lv_color_hex(0x333333),0);
lv_obj_set_style_radius(bs,6,0);
lv_obj_t* ls=lv_label_create(bs);
lv_label_set_text(ls,"Seasons");
lv_obj_set_style_text_font(ls, lvgl_get_text_font(FONT_SIZE_SMALL),0);
lv_obj_center(ls);
lv_obj_add_event_cb(bs,btn_seasons_cb,LV_EVENT_CLICKED,NULL);
lv_obj_t* bd=lv_btn_create(row);
lv_obj_set_size(bd,LV_PCT(48),36);
G.btn_dl_missing=bd;
lv_obj_set_style_bg_color(bd, lv_color_hex(0x1E88E5), 0);
lv_obj_set_style_radius(bd,6,0);
lv_obj_t* ld=lv_label_create(bd);
lv_label_set_text(ld,"DL Missing");
lv_obj_set_style_text_font(ld, lvgl_get_text_font(FONT_SIZE_SMALL),0);
lv_obj_center(ld);
lv_obj_add_event_cb(bd,btn_dl_season_cb,LV_EVENT_CLICKED,NULL);
// Episode list container – fills remaining space
G.ep_list_cont=lv_obj_create(root);
lv_obj_set_size(G.ep_list_cont, LV_PCT(100), LV_PCT(100));
lv_obj_set_flex_grow(G.ep_list_cont, 1);
lv_obj_set_style_bg_color(G.ep_list_cont, lv_color_hex(0x0E0E0E),0);
lv_obj_set_style_bg_opa(G.ep_list_cont, LV_OPA_COVER,0);
lv_obj_set_style_radius(G.ep_list_cont,8,0);
lv_obj_set_style_pad_all(G.ep_list_cont,4,0);
lv_obj_set_style_pad_gap(G.ep_list_cont,4,0);
lv_obj_set_flex_flow(G.ep_list_cont, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(G.ep_list_cont, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START);
// Ensure scrollable
lv_obj_add_flag(G.ep_list_cont, LV_OBJ_FLAG_SCROLLABLE);
lv_obj_set_scrollbar_mode(G.ep_list_cont, LV_SCROLLBAR_MODE_AUTO);
lv_obj_t* hint=lv_label_create(G.ep_list_cont);
lv_label_set_text(hint,"Loading episodes...");
lv_obj_set_style_text_color(hint, lv_color_hex(0x888888),0);
lv_obj_set_style_text_font(hint, lvgl_get_text_font(FONT_SIZE_SMALL),0);
// Status bar at bottom
G.lbl_status=lv_label_create(root);
lv_label_set_text(G.lbl_status,"Fetching data...");
lv_obj_set_style_text_color(G.lbl_status,lv_color_hex(0x888888),0);
lv_obj_set_style_text_font(G.lbl_status, lvgl_get_text_font(FONT_SIZE_SMALL),0);
lv_obj_set_width(G.lbl_status, LV_PCT(100));
lv_label_set_long_mode(G.lbl_status, LV_LABEL_LONG_DOT);
// Legacy placeholders nulled
G.lbl_meta=NULL;
G.bar=NULL;
G.lbl_time=NULL;
G.btn_play=NULL;
G.dl_overlay=NULL;
G.ui_timer_handle=NULL;
G.ui_timer_handle = lv_timer_create(ui_timer,300,NULL);
tt_lvgl_unlock();
}
void ui_deinit(void){
if(G.ui_timer_handle){
lv_timer_delete(G.ui_timer_handle);
G.ui_timer_handle=NULL;
}
}
void ui_timer(lv_timer_t* t){
(void)t;
// If app is closing (root null) skip
if(!G.root) return;
if(G.fetching){
return;
}
if(G.downloading && G.dl_overlay){
update_dl_overlay_ui();
}
}
void btn_play_cb(lv_event_t* e){ (void)e; /* legacy, not used */ }
void btn_prev_cb(lv_event_t* e){ (void)e; /* legacy */ }
void btn_next_cb(lv_event_t* e){ (void)e; /* legacy */ }
+28
View File
@@ -0,0 +1,28 @@
#pragma once
#include "app_context.h"
#ifdef __cplusplus
extern "C" {
#endif
void build_ui(lv_obj_t* parent);
void ui_timer(lv_timer_t* t);
void btn_play_cb(lv_event_t* e);
void btn_prev_cb(lv_event_t* e);
void btn_next_cb(lv_event_t* e);
void btn_seasons_cb(lv_event_t* e);
void btn_dl_season_cb(lv_event_t* e);
void close_overlay(void);
void dropdown_cb(lv_event_t* e);
// New library UI helpers
void ui_rebuild_episode_list(void);
void ui_update_header_locked(void);
void ui_update_status_locked(const char* msg);
void ui_deinit(void);
#ifdef __cplusplus
}
#endif
@@ -0,0 +1,7 @@
manifest.version=0.2
target.sdk=0.8.0-dev
target.platforms=esp32s3
app.id=one.tactility.discoverymountain
app.name=Discovery Mountain
app.version.name=0.1.0
app.version.code=1
+16
View File
@@ -0,0 +1,16 @@
cmake_minimum_required(VERSION 3.20)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
if (DEFINED ENV{TACTILITY_SDK_PATH})
set(TACTILITY_SDK_PATH $ENV{TACTILITY_SDK_PATH})
else()
set(TACTILITY_SDK_PATH "../../release/TactilitySDK")
message(WARNING "⚠️ TACTILITY_SDK_PATH environment variable is not set, defaulting to ${TACTILITY_SDK_PATH}")
endif()
include("${TACTILITY_SDK_PATH}/TactilitySDK.cmake")
set(EXTRA_COMPONENT_DIRS ${TACTILITY_SDK_PATH})
project(ImmichElias)
tactility_project(ImmichElias)
@@ -4,5 +4,6 @@ set(CJSON_SOURCE "$ENV{IDF_PATH}/components/json/cJSON/cJSON.c")
idf_component_register(
SRCS ${SOURCE_FILES} ${CJSON_SOURCE}
INCLUDE_DIRS Source "$ENV{IDF_PATH}/components/json/cJSON"
REQUIRES TactilitySDK lwip
REQUIRES TactilitySDK esp_http_client
)
target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-format-truncation -Wno-unused-but-set-variable -Wno-unused-function -Wno-unused-variable)
File diff suppressed because it is too large Load Diff
+7
View File
@@ -0,0 +1,7 @@
manifest.version=0.2
target.sdk=0.8.0-dev
target.platforms=esp32s3,esp32p4
app.id=one.tactility.imichelias
app.version.name=0.1.0
app.version.code=1
app.name=Immich Elias
-17
View File
@@ -1,17 +0,0 @@
# Live Captions
ESP32-S3 external app that streams 16 kHz signed-16-bit mono microphone PCM to the Mac mini Hermes voice gateway and renders its live draft/final captions. It stores **only final captions** on the device SD card in `/sdcard/captions/YYYY-MM-DD.txt`.
## Configuration
Before packaging, place `config.json` in the app's user-data directory (not source control):
```json
{
"server_url": "ws://192.168.68.102:8642/api/esp32/voice/ws",
"device_id": "your-registered-device-id",
"api_key": "device-profile-key"
}
```
The app never retries automatically: connection failure remains `FAILED` until the user selects **Start** again. **Stop** sends the gateway `stop` event, waits for the final caption, appends it to that day’s text file, then returns to idle.
-328
View File
@@ -1,328 +0,0 @@
/* Live Captions: stream mic PCM to Mac mini and save final text on the SD card. */
#include <tt_app.h>
#include <tt_lvgl.h>
#include <tt_lvgl_toolbar.h>
#include <tactility/device.h>
#include <tactility/drivers/audio_stream.h>
#include <cJSON.h>
#include <esp_log.h>
#include <esp_random.h>
#include <freertos/FreeRTOS.h>
#include <freertos/semphr.h>
#include <freertos/task.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <sys/stat.h>
#include "websocket.h"
/* These legacy names are exported by the flashed 0.8.0-dev firmware. */
struct Device* device_find_by_name(const char* name);
struct Device* device_find_first_by_type(const struct DeviceType* type);
#define TAG "LiveCaptions"
#define DEFAULT_ENDPOINT "ws://192.168.68.102:8645/api/esp32/captions/ws"
#define DEFAULT_DEVICE_ID "tactility-14c19d1a790"
#define PCM_BUFFER_BYTES 1024U
#define EVENT_BUFFER_BYTES 4096U
#define DISPLAY_WORD_LIMIT 50
typedef enum {
CAPTION_IDLE,
CAPTION_CONNECTING,
CAPTION_LISTENING,
CAPTION_PROCESSING,
CAPTION_FAILED,
} CaptionState;
typedef struct {
AppHandle app;
volatile bool visible;
volatile bool capture_audio;
volatile bool stop_requested;
volatile bool session_active;
volatile bool socket_failed;
int fd;
CaptionState state;
char endpoint[128];
char device_id[64];
char api_key[128];
char detail[96];
char caption[768];
char last_final[768];
struct Device* stream_dev;
AudioStreamHandle input_handle;
TaskHandle_t worker;
TaskHandle_t receiver;
SemaphoreHandle_t socket_lock;
SemaphoreHandle_t audio_lock;
lv_obj_t* caption_label;
lv_obj_t* status_label;
} CaptionContext;
static void update_ui(CaptionContext* ctx) {
if (!ctx->visible || !tt_lvgl_lock(pdMS_TO_TICKS(100))) return;
lv_label_set_text(ctx->caption_label, ctx->caption);
const char* status = ctx->state == CAPTION_LISTENING ? "Connected" : (ctx->state == CAPTION_CONNECTING ? "Connecting" : (ctx->state == CAPTION_FAILED ? "Failed — Reconnecting" : "Connecting"));
lv_label_set_text(ctx->status_label, status);
tt_lvgl_unlock();
}
static void set_state(CaptionContext* ctx, CaptionState state, const char* detail) {
ctx->state = state;
snprintf(ctx->detail, sizeof(ctx->detail), "%s", detail ? detail : "");
update_ui(ctx);
}
static bool find_audio_stream_device(CaptionContext* ctx) {
ctx->stream_dev = device_find_by_name("audio-stream");
if (ctx->stream_dev == NULL) ctx->stream_dev = device_find_first_by_type(&AUDIO_STREAM_TYPE);
return ctx->stream_dev != NULL;
}
static bool open_input_stream(CaptionContext* ctx) {
if (ctx->stream_dev == NULL) return false;
if (ctx->input_handle != NULL) return true;
struct AudioStreamConfig config = {.sample_rate = 16000, .bits_per_sample = 16, .channels = 1};
if (audio_stream_open_input(ctx->stream_dev, &config, &ctx->input_handle) != ERROR_NONE) return false;
audio_stream_set_mute(ctx->stream_dev, AUDIO_CODEC_DIR_INPUT, false);
audio_stream_set_volume(ctx->stream_dev, AUDIO_CODEC_DIR_INPUT, 100.0f);
return true;
}
static void close_input_stream(CaptionContext* ctx) {
if (ctx->input_handle != NULL) {
audio_stream_close(ctx->input_handle);
ctx->input_handle = NULL;
}
}
static int send_locked(CaptionContext* ctx, const uint8_t* data, size_t length, bool binary) {
if (ctx->fd < 0 || xSemaphoreTake(ctx->socket_lock, pdMS_TO_TICKS(500)) != pdTRUE) return -1;
int result = ws_send(ctx->fd, data, length, binary);
xSemaphoreGive(ctx->socket_lock);
return result;
}
static bool parse_endpoint(const char* url, char* host, size_t host_size, int* port, char* path, size_t path_size) {
if (url == NULL || strncmp(url, "ws://", 5) != 0) return false;
const char* authority = url + 5;
const char* slash = strchr(authority, '/');
const char* end = slash ? slash : authority + strlen(authority);
const char* colon = NULL;
for (const char* p = authority; p < end; ++p) if (*p == ':') colon = p;
size_t host_len = (size_t)((colon ? colon : end) - authority);
if (host_len == 0 || host_len >= host_size) return false;
memcpy(host, authority, host_len); host[host_len] = '\0';
*port = 80;
if (colon != NULL) {
*port = atoi(colon + 1);
if (*port < 1 || *port > 65535) return false;
}
const char* wire_path = slash ? slash : "/";
if (strlen(wire_path) >= path_size) return false;
snprintf(path, path_size, "%s", wire_path);
return true;
}
static void append_final_caption(const char* text) {
if (text == NULL || !*text) return;
mkdir("/sdcard/captions", 0755);
time_t now = time(NULL);
struct tm local;
localtime_r(&now, &local);
char filename[32];
strftime(filename, sizeof(filename), "%Y-%m-%d.txt", &local);
char stamp[16];
strftime(stamp, sizeof(stamp), "%H:%M:%S", &local);
char path[128];
snprintf(path, sizeof(path), "/sdcard/captions/%s", filename);
FILE* log = fopen(path, "a");
if (log == NULL) { ESP_LOGE(TAG, "could not append %s", path); return; }
fprintf(log, "[%s] %s\n", stamp, text);
fclose(log);
}
static bool is_space_char(char value) { return value == ' ' || value == '\n' || value == '\r' || value == ' '; }
static void copy_recent_words(char* output, size_t output_size, const char* text) {
const char* starts[DISPLAY_WORD_LIMIT];
int count = 0;
bool in_word = false;
for (const char* cursor = text; *cursor; ++cursor) {
if (is_space_char(*cursor)) { in_word = false; continue; }
if (!in_word) { starts[count % DISPLAY_WORD_LIMIT] = cursor; ++count; in_word = true; }
}
const char* first = count > DISPLAY_WORD_LIMIT ? starts[count % DISPLAY_WORD_LIMIT] : text;
snprintf(output, output_size, "%s", first);
}
static void display_caption(CaptionContext* ctx, const char* text, bool final) {
if (text == NULL || !*text) return;
copy_recent_words(ctx->caption, sizeof(ctx->caption), text);
if (final && strcmp(ctx->last_final, text) != 0) {
snprintf(ctx->last_final, sizeof(ctx->last_final), "%s", text);
append_final_caption(text);
ctx->state = CAPTION_IDLE;
}
update_ui(ctx);
}
static void handle_event(CaptionContext* ctx, const char* json) {
cJSON* root = cJSON_Parse(json);
if (root == NULL) return;
cJSON* event = cJSON_GetObjectItem(root, "event");
cJSON* text = cJSON_GetObjectItem(root, "text");
if (!cJSON_IsString(event)) { cJSON_Delete(root); return; }
const char* name = event->valuestring;
if (strcmp(name, "ready") == 0) {
set_state(ctx, CAPTION_CONNECTING, "Starting caption stream");
} else if (strcmp(name, "listening") == 0) {
set_state(ctx, CAPTION_LISTENING, "Listening — press Stop when finished");
} else if (strcmp(name, "state") == 0) {
cJSON* remote_state = cJSON_GetObjectItem(root, "state");
if (cJSON_IsString(remote_state) && strcmp(remote_state->valuestring, "listening") == 0) {
set_state(ctx, CAPTION_LISTENING, "Listening");
} else if (cJSON_IsString(remote_state) && strcmp(remote_state->valuestring, "processing") == 0) {
set_state(ctx, CAPTION_PROCESSING, "Captioning…");
}
} else if (strcmp(name, "draft") == 0 || strcmp(name, "interim_transcript") == 0 || strcmp(name, "review") == 0) {
if (cJSON_IsString(text)) display_caption(ctx, text->valuestring, false);
} else if (strcmp(name, "transcript") == 0 || strcmp(name, "final") == 0) {
cJSON* is_final = cJSON_GetObjectItem(root, "isFinal");
if (cJSON_IsString(text) && (!cJSON_IsBool(is_final) || cJSON_IsTrue(is_final) || strcmp(name, "final") == 0)) {
display_caption(ctx, text->valuestring, true);
}
} else if (strcmp(name, "thinking") == 0) {
set_state(ctx, CAPTION_PROCESSING, "Final captioning…");
} else if (strcmp(name, "error") == 0) {
ctx->socket_failed = true;
set_state(ctx, CAPTION_FAILED, "Fail to connect");
}
cJSON_Delete(root);
}
static void receiver_task(void* argument) {
CaptionContext* ctx = argument;
uint8_t* buffer = malloc(EVENT_BUFFER_BYTES + 1U);
if (buffer == NULL) { ctx->socket_failed = true; ctx->receiver = NULL; vTaskDelete(NULL); }
while (ctx->visible && ctx->fd >= 0) {
int opcode = 0; bool complete = false;
int received = ws_recv(ctx->fd, &opcode, &complete, buffer, EVENT_BUFFER_BYTES);
if (received < 0 || !complete) break;
if (opcode == 0x01) { buffer[received] = '\0'; handle_event(ctx, (const char*)buffer); }
else if (opcode == 0x09) {
if (xSemaphoreTake(ctx->socket_lock, pdMS_TO_TICKS(500)) == pdTRUE) {
ws_send_pong(ctx->fd, buffer, (size_t)received); xSemaphoreGive(ctx->socket_lock);
}
} else if (opcode == 0x08) break;
}
ctx->session_active = false;
ctx->receiver = NULL;
vTaskDelete(NULL);
}
static void load_config(CaptionContext* ctx) {
snprintf(ctx->endpoint, sizeof(ctx->endpoint), "%s", DEFAULT_ENDPOINT);
snprintf(ctx->device_id, sizeof(ctx->device_id), "%s", DEFAULT_DEVICE_ID);
ctx->api_key[0] = '\0';
char path[256]; size_t size = sizeof(path);
tt_app_get_user_data_child_path(ctx->app, "config.json", path, &size);
FILE* file = fopen(path, "r");
if (file == NULL) return;
char raw[512]; size_t bytes = fread(raw, 1, sizeof(raw) - 1, file); fclose(file); raw[bytes] = '\0';
cJSON* root = cJSON_Parse(raw);
cJSON* endpoint = root ? cJSON_GetObjectItem(root, "server_url") : NULL;
cJSON* device = root ? cJSON_GetObjectItem(root, "device_id") : NULL;
cJSON* key = root ? cJSON_GetObjectItem(root, "api_key") : NULL;
if (cJSON_IsString(endpoint)) snprintf(ctx->endpoint, sizeof(ctx->endpoint), "%s", endpoint->valuestring);
if (cJSON_IsString(device)) snprintf(ctx->device_id, sizeof(ctx->device_id), "%s", device->valuestring);
if (cJSON_IsString(key)) snprintf(ctx->api_key, sizeof(ctx->api_key), "%s", key->valuestring);
cJSON_Delete(root);
}
static void worker_task(void* argument) {
CaptionContext* ctx = argument;
char host[64], path[96]; int port = 0;
if (!parse_endpoint(ctx->endpoint, host, sizeof(host), &port, path, sizeof(path))) {
set_state(ctx, CAPTION_FAILED, "Fail to connect"); ctx->worker = NULL; vTaskDelete(NULL);
}
set_state(ctx, CAPTION_CONNECTING, "Connecting to Mac mini");
ctx->fd = ws_connect(host, port, path, ctx->device_id, ctx->api_key);
if (ctx->fd < 0) { set_state(ctx, CAPTION_FAILED, "Fail to connect"); ctx->worker = NULL; vTaskDelete(NULL); }
char start[256];
snprintf(start, sizeof(start), "{\"v\":1,\"event\":\"start\",\"session_id\":\"cap-%08lx\",\"device_id\":\"%s\",\"audio\":{\"format\":\"pcm_s16le\",\"sample_rate\":16000,\"channels\":1,\"sample_width\":2}}", (unsigned long)esp_random(), ctx->device_id);
if (send_locked(ctx, (const uint8_t*)start, strlen(start), false) != 0 || !open_input_stream(ctx)) {
set_state(ctx, CAPTION_FAILED, "Fail to connect"); ws_close(ctx->fd); ctx->fd = -1; ctx->worker = NULL; vTaskDelete(NULL);
}
ctx->session_active = true;
xTaskCreate(receiver_task, "caption_rx", 6144, ctx, 6, &ctx->receiver);
set_state(ctx, CAPTION_LISTENING, "Listening — press Stop when finished");
uint8_t pcm[PCM_BUFFER_BYTES];
while (ctx->visible && ctx->capture_audio && !ctx->socket_failed) {
size_t bytes = 0;
xSemaphoreTake(ctx->audio_lock, portMAX_DELAY);
error_t result = audio_stream_read(ctx->input_handle, pcm, sizeof(pcm), &bytes, pdMS_TO_TICKS(100));
xSemaphoreGive(ctx->audio_lock);
if (result == ERROR_NONE && bytes > 0 && (bytes % 2U) == 0 && send_locked(ctx, pcm, bytes, true) != 0) ctx->socket_failed = true;
}
xSemaphoreTake(ctx->audio_lock, portMAX_DELAY); close_input_stream(ctx); xSemaphoreGive(ctx->audio_lock);
if (ctx->stop_requested && !ctx->socket_failed) {
const char* stop = "{\"event\":\"stop\"}";
send_locked(ctx, (const uint8_t*)stop, strlen(stop), false);
set_state(ctx, CAPTION_PROCESSING, "Final captioning…");
for (unsigned i = 0; ctx->session_active && i < 150; ++i) vTaskDelay(pdMS_TO_TICKS(100));
}
if (ctx->fd >= 0) { ws_send_close(ctx->fd); ws_close(ctx->fd); ctx->fd = -1; }
bool reconnect = ctx->socket_failed && ctx->visible;
if (reconnect) set_state(ctx, CAPTION_FAILED, "Reconnecting");
else if (ctx->state == CAPTION_PROCESSING) set_state(ctx, CAPTION_IDLE, "");
ctx->worker = NULL;
if (reconnect) {
vTaskDelay(pdMS_TO_TICKS(3000));
if (ctx->visible) { ctx->socket_failed = false; ctx->capture_audio = true; xTaskCreate(worker_task, "caption_tx", 8192, ctx, 5, &ctx->worker); }
}
vTaskDelete(NULL);
}
static void* create_data(void) { CaptionContext* ctx = calloc(1, sizeof(*ctx)); if (ctx) ctx->fd = -1; return ctx; }
static void destroy_data(void* data) { free(data); }
static void on_create(AppHandle app, void* data) { ((CaptionContext*)data)->app = app; }
static void on_show(AppHandle app, void* data, lv_obj_t* parent) {
CaptionContext* ctx = data; ctx->visible = true; load_config(ctx); find_audio_stream_device(ctx);
ctx->socket_lock = xSemaphoreCreateMutex(); ctx->audio_lock = xSemaphoreCreateMutex();
lv_obj_t* toolbar = tt_lvgl_toolbar_create_for_app(parent, app);
lv_obj_align(toolbar, LV_ALIGN_TOP_MID, 0, 0);
ctx->caption_label = lv_label_create(parent);
lv_obj_set_width(ctx->caption_label, lv_pct(88));
lv_label_set_long_mode(ctx->caption_label, LV_LABEL_LONG_WRAP);
lv_obj_set_style_text_align(ctx->caption_label, LV_TEXT_ALIGN_CENTER, 0);
lv_obj_align(ctx->caption_label, LV_ALIGN_CENTER, 0, 0);
ctx->status_label = lv_label_create(parent);
lv_obj_align(ctx->status_label, LV_ALIGN_BOTTOM_MID, 0, -10);
ctx->caption[0] = '\0';
if (ctx->stream_dev != NULL && ctx->socket_lock != NULL && ctx->audio_lock != NULL) {
ctx->capture_audio = true; ctx->stop_requested = false; ctx->socket_failed = false;
xTaskCreate(worker_task, "caption_tx", 8192, ctx, 5, &ctx->worker);
}
}
static void on_hide(AppHandle app, void* data) {
(void)app; CaptionContext* ctx = data; ctx->visible = false; ctx->capture_audio = false; ctx->stop_requested = false;
if (ctx->fd >= 0) { ws_send_close(ctx->fd); ws_close(ctx->fd); ctx->fd = -1; }
for (unsigned i = 0; (ctx->worker || ctx->receiver) && i < 100; ++i) vTaskDelay(pdMS_TO_TICKS(10));
close_input_stream(ctx);
if (ctx->socket_lock) { vSemaphoreDelete(ctx->socket_lock); ctx->socket_lock = NULL; }
if (ctx->audio_lock) { vSemaphoreDelete(ctx->audio_lock); ctx->audio_lock = NULL; }
}
int main(int argc, char* argv[]) {
(void)argc; (void)argv;
tt_app_register((AppRegistration){.createData=create_data,.destroyData=destroy_data,.onCreate=on_create,.onShow=on_show,.onHide=on_hide});
return 0;
}
-167
View File
@@ -1,167 +0,0 @@
#include "websocket.h"
#include <string.h>
#include <stdio.h>
#include <unistd.h>
#include <esp_log.h>
#include <esp_random.h>
#include <lwip/inet.h>
#include <lwip/sockets.h>
#define TAG "PipecatVoiceWs"
#define WS_HEADER_LIMIT 1024U
#define WS_CONTROL_LIMIT 125U
static int send_all(int fd, const uint8_t* data, size_t length) {
size_t sent = 0;
while (sent < length) {
int result = lwip_send(fd, data + sent, length - sent, 0);
if (result <= 0) return -1;
sent += (size_t)result;
}
return 0;
}
static int recv_all(int fd, uint8_t* data, size_t length) {
size_t received = 0;
while (received < length) {
int result = lwip_recv(fd, data + received, length - received, 0);
if (result <= 0) return -1;
received += (size_t)result;
}
return 0;
}
static int discard(int fd, uint64_t length) {
uint8_t buffer[256];
while (length > 0) {
size_t chunk = length > sizeof(buffer) ? sizeof(buffer) : (size_t)length;
if (recv_all(fd, buffer, chunk) < 0) return -1;
length -= chunk;
}
return 0;
}
static int send_frame(int fd, uint8_t opcode, const uint8_t* payload, size_t length) {
if (length > 65535U || ((opcode & 0x08U) && length > WS_CONTROL_LIMIT)) return -1;
uint8_t header[8];
size_t header_length = 2;
header[0] = 0x80U | opcode;
if (length < 126U) {
header[1] = 0x80U | (uint8_t)length;
} else {
header[1] = 0x80U | 126U;
header[2] = (uint8_t)(length >> 8U);
header[3] = (uint8_t)length;
header_length = 4;
}
uint8_t mask[4];
uint32_t random = esp_random();
memcpy(mask, &random, sizeof(mask));
memcpy(header + header_length, mask, sizeof(mask));
header_length += sizeof(mask);
if (send_all(fd, header, header_length) < 0) return -1;
uint8_t chunk[512];
size_t offset = 0;
while (offset < length) {
size_t count = length - offset > sizeof(chunk) ? sizeof(chunk) : length - offset;
for (size_t i = 0; i < count; ++i) chunk[i] = payload[offset + i] ^ mask[(offset + i) % sizeof(mask)];
if (send_all(fd, chunk, count) < 0) return -1;
offset += count;
}
return 0;
}
int ws_connect(const char* host, int port, const char* path, const char* device_id, const char* api_key) {
if (host == NULL || path == NULL || device_id == NULL || api_key == NULL || port < 1 || port > 65535) return -1;
int fd = lwip_socket(AF_INET, SOCK_STREAM, 0);
if (fd < 0) {
ESP_LOGW(TAG, "socket create failed");
return -1;
}
struct sockaddr_in address = {0};
address.sin_family = AF_INET;
address.sin_port = htons((uint16_t)port);
address.sin_addr.s_addr = ipaddr_addr(host);
if (address.sin_addr.s_addr == IPADDR_NONE) {
ESP_LOGW(TAG, "endpoint address parse failed");
close(fd);
return -1;
}
if (lwip_connect(fd, (struct sockaddr*)&address, sizeof(address)) < 0) {
ESP_LOGW(TAG, "TCP connect failed");
close(fd);
return -1;
}
struct timeval timeout = {.tv_sec = 15, .tv_usec = 0};
lwip_setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout));
char request[WS_HEADER_LIMIT];
int request_length = snprintf(request, sizeof(request),
"GET %s HTTP/1.1\r\nHost: %s:%d\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n"
"Sec-WebSocket-Key: MDEyMzQ1Njc4OWFiY2RlZg==\r\nSec-WebSocket-Version: 13\r\n"
"Authorization: Bearer %s\r\nX-Device-ID: %s\r\n\r\n",
path, host, port, api_key, device_id);
if (request_length < 0 || (size_t)request_length >= sizeof(request) || send_all(fd, (const uint8_t*)request, (size_t)request_length) < 0) {
ESP_LOGW(TAG, "WebSocket upgrade request failed");
close(fd);
return -1;
}
char response[WS_HEADER_LIMIT];
size_t length = 0;
while (length + 1 < sizeof(response)) {
if (recv_all(fd, (uint8_t*)&response[length], 1) < 0) {
ESP_LOGW(TAG, "WebSocket upgrade response failed");
close(fd);
return -1;
}
response[++length] = '\0';
if (length >= 4 && memcmp(response + length - 4, "\r\n\r\n", 4) == 0) break;
}
if (length + 1 >= sizeof(response) || strstr(response, " 101 ") == NULL) {
ESP_LOGW(TAG, "WebSocket upgrade rejected");
close(fd);
return -1;
}
ESP_LOGI(TAG, "WebSocket upgrade accepted");
return fd;
}
int ws_send(int fd, const uint8_t* data, size_t length, bool binary) {
if (fd < 0 || data == NULL || length == 0) return -1;
return send_frame(fd, binary ? 0x02U : 0x01U, data, length);
}
int ws_recv(int fd, int* opcode, bool* final, uint8_t* payload, size_t maximum) {
uint8_t header[2];
if (fd < 0 || recv_all(fd, header, sizeof(header)) < 0) return -1;
uint64_t length = header[1] & 0x7fU;
if (length == 126U) {
uint8_t extended[2];
if (recv_all(fd, extended, sizeof(extended)) < 0) return -1;
length = ((uint64_t)extended[0] << 8U) | extended[1];
} else if (length == 127U) {
uint8_t extended[8];
if (recv_all(fd, extended, sizeof(extended)) < 0) return -1;
length = 0;
for (size_t i = 0; i < sizeof(extended); ++i) length = (length << 8U) | extended[i];
}
bool masked = (header[1] & 0x80U) != 0;
uint8_t mask[4] = {0};
if (masked && recv_all(fd, mask, sizeof(mask)) < 0) return -1;
uint8_t frame_opcode = header[0] & 0x0fU;
if (((frame_opcode & 0x08U) && (length > WS_CONTROL_LIMIT || !(header[0] & 0x80U))) || length > maximum) {
if (discard(fd, length) < 0) return -1;
return -2;
}
if (length > 0 && recv_all(fd, payload, (size_t)length) < 0) return -1;
if (masked) for (size_t i = 0; i < (size_t)length; ++i) payload[i] ^= mask[i % sizeof(mask)];
if (opcode) *opcode = frame_opcode;
if (final) *final = (header[0] & 0x80U) != 0;
return (int)length;
}
int ws_send_pong(int fd, const uint8_t* payload, size_t length) { return send_frame(fd, 0x0aU, payload, length); }
int ws_send_close(int fd) { return send_frame(fd, 0x08U, NULL, 0); }
void ws_close(int fd) { if (fd >= 0) close(fd); }
-62
View File
@@ -1,62 +0,0 @@
#pragma once
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
/**
* Connect to a WebSocket server.
* @param host Server IP address (e.g. "192.168.68.126")
* @param port Port number (e.g. 8642)
* @param path WebSocket path (e.g. "/api/esp32/voice/ws")
* @param device_id Unique device identifier
* @param api_key Optional profile API key; never compiled into firmware
* @return Socket file descriptor on success, or -1 on failure
*/
int ws_connect(const char* host, int port, const char* path, const char* device_id, const char* api_key);
/**
* Send a WebSocket frame.
* @param fd Socket file descriptor
* @param data Data payload to send
* @param len Length of the data payload
* @param binary True for binary frame, false for text frame
* @return 0 on success, or -1 on failure
*/
int ws_send(int fd, const uint8_t* data, size_t len, bool binary);
/**
* Receive a WebSocket frame.
* @param fd Socket file descriptor
* @param out_opcode Pointer to store the received opcode (e.g. 0x01 text, 0x02 binary)
* @param payload Buffer to store the received payload
* @param max_len Maximum length of the payload buffer
* @return Received payload length on success, -1 on connection failure, or -2 on buffer overflow
*/
int ws_recv(int fd, int* out_opcode, bool* out_final, uint8_t* payload, size_t max_len);
/**
* Close a WebSocket connection.
* @param fd Socket file descriptor
*/
void ws_close(int fd);
/**
* Send a WebSocket PONG frame.
* @param fd Socket file descriptor
* @param payload Payload to reflect
* @param len Length of payload
* @return 0 on success, or -1 on failure
*/
int ws_send_pong(int fd, const uint8_t* payload, size_t len);
/** Send a clean WebSocket close control frame before closing the socket. */
int ws_send_close(int fd);
#ifdef __cplusplus
}
#endif
+2 -1
View File
@@ -2,5 +2,6 @@ file(GLOB_RECURSE SOURCE_FILES Source/*.c)
idf_component_register(
SRCS ${SOURCE_FILES}
REQUIRES TactilitySDK
REQUIRES TactilitySDK esp_http_client
)
target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-deprecated-declarations -Wno-unused-but-set-variable -Wno-format-truncation)
+135 -142
View File
@@ -7,9 +7,6 @@
#include <tactility/drivers/audio_stream.h>
#include <tactility/drivers/audio_codec.h>
/* Exported by firmware 0.8.0-dev although absent from the CDN SDK header. */
struct Device* device_find_by_name(const char* name);
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
@@ -32,7 +29,7 @@ struct Device* device_find_by_name(const char* name);
#define ENABLE_TITLE_SCROLL_ANIM_WHEN_IDLE 1
#define MAX_HISTORY 20
#define SEEK_SECONDS 15
#define SEEK_SECONDS 10
typedef enum {
STATE_IDLE,
@@ -65,12 +62,12 @@ typedef struct {
lv_obj_t* btn_stop;
lv_obj_t* btn_seek_back;
lv_obj_t* btn_seek_fwd;
lv_obj_t* slider_volume;
lv_obj_t* list_history;
lv_obj_t* history_container;
lv_obj_t* center_box;
lv_obj_t* volume_box;
lv_obj_t* slider_volume;
lv_obj_t* bottom_box;
lv_obj_t* btn_history;
// Playback state variables
FILE* file;
@@ -247,6 +244,11 @@ static void save_history(AppCtx* ctx) {
static void update_history_entry(AppCtx* ctx, const char* filepath, int pos_sec, int total_sec) {
if (!filepath || filepath[0]=='\0') return;
// Only track audios >=10min (600 sec) as requested
if (total_sec > 0 && total_sec < 600) {
ESP_LOGI(TAG, "Skip history: audio too short (%d sec < 600)", total_sec);
return;
}
time_t now = time(NULL);
int idx = -1;
for (int i=0;i<ctx->history_count;i++) {
@@ -311,13 +313,25 @@ static void hide_history_screen(AppCtx* ctx);
static void load_and_play_file(AppCtx* ctx, const char* filepath, int start_pos);
static void on_history_selected(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
int idx = (int)(intptr_t)lv_event_get_user_data(e);
// Actually user_data is ctx, we need index via event param? Use lv_event_get_user_data returns ctx, but we need index.
// We'll store index in btn's user_data via lv_obj_set_user_data? Simpler: use event param as index via lv_event_get_param?
// For now, get index from button's index in list_history
// The callback is set with (void*)index, not ctx, so get it:
int index = (int)(intptr_t)lv_event_get_user_data(e);
// But we also need ctx global - use g_ctx
AppCtx* c = &g_ctx;
if (index < 0 || index >= c->history_count) return;
HistoryEntry* he = &c->history[index];
int resume_pos = he->pos_sec;
if (resume_pos <= 5) resume_pos = 0;
if (he->total_sec > 0 && resume_pos >= he->total_sec - 10) resume_pos = 0;
int resume_pos = 0;
if (he->total_sec >= 600) {
resume_pos = he->pos_sec;
if (resume_pos <= 5) resume_pos = 0;
if (he->total_sec >0 && resume_pos >= he->total_sec - 10) resume_pos = 0;
} else {
resume_pos = 0; // short files start from beginning
}
ESP_LOGI(TAG, "History tap %d: %s resume %d (total %d)", index, he->filepath, resume_pos, he->total_sec);
// Hide history full screen and go back to player
hide_history_screen(c);
@@ -335,7 +349,7 @@ static void refresh_history_list(AppCtx* ctx) {
HistoryEntry* he = &ctx->history[i];
const char* slash = strrchr(he->filepath, '/');
const char* fname = slash ? slash+1 : he->filepath;
char label[640];
char label[128];
if (he->total_sec >= 600) {
int pos_m = he->pos_sec / 60;
int pos_s = he->pos_sec % 60;
@@ -373,7 +387,6 @@ static void show_history_screen(AppCtx* ctx) {
lv_obj_move_foreground(ctx->history_container);
// Hide player UI
lv_obj_add_flag(ctx->center_box, LV_OBJ_FLAG_HIDDEN);
lv_obj_add_flag(ctx->volume_box, LV_OBJ_FLAG_HIDDEN);
lv_obj_add_flag(ctx->bottom_box, LV_OBJ_FLAG_HIDDEN);
}
@@ -383,7 +396,6 @@ static void hide_history_screen(AppCtx* ctx) {
// For full-screen mode, we hide history container and show player
lv_obj_add_flag(ctx->history_container, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->center_box, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->volume_box, LV_OBJ_FLAG_HIDDEN);
lv_obj_remove_flag(ctx->bottom_box, LV_OBJ_FLAG_HIDDEN);
// Restore small size for when it's shown inline (if needed)
lv_obj_set_size(ctx->history_container, lv_pct(92), lv_pct(28));
@@ -527,7 +539,7 @@ static void mp3_playback_task(void* arg) {
ctx->finished_naturally = false;
ctx->last_progress_pct = -1;
ctx->seek_request_sec = -1;
// Handle ID3 and resume seek
uint8_t id3hdr[10];
ctx->id3_skip = 0;
@@ -758,18 +770,6 @@ static void mp3_playback_task(void* arg) {
vTaskDelete(NULL);
}
static void wait_for_playback_task_to_exit(AppCtx* ctx) {
if (ctx->playback_task_handle == NULL) return;
ctx->state = STATE_IDLE;
while (ctx->playback_task_handle != NULL) {
// Let the playback worker take LVGL for its final teardown update.
tt_lvgl_unlock();
vTaskDelay(pdMS_TO_TICKS(10));
tt_lvgl_lock(portMAX_DELAY);
}
}
/* ─── Callbacks ─── */
static void on_play_pause_click(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
@@ -782,10 +782,14 @@ static void on_play_pause_click(lv_event_t* e) {
} else if (ctx->state == STATE_PLAYING) {
ctx->state = STATE_PAUSED;
update_ui(ctx);
update_history_entry(ctx, ctx->filepath, ctx->pos_sec, ctx->total_sec);
save_history(ctx);
refresh_history_list(ctx);
ESP_LOGI(TAG, "History updated on pause: %s pos %d total %d", ctx->filepath, ctx->pos_sec, ctx->total_sec);
// Update history on pause, only for 10min+ audios
bool is_long = (ctx->total_sec >= 600) || (ctx->file_data_size >= 600*16000L);
if (is_long) {
update_history_entry(ctx, ctx->filepath, ctx->pos_sec, ctx->total_sec);
save_history(ctx);
refresh_history_list(ctx);
ESP_LOGI(TAG, "History updated on pause: %s pos %d total %d", ctx->filepath, ctx->pos_sec, ctx->total_sec);
}
} else if (ctx->state == STATE_PAUSED) {
ctx->state = STATE_PLAYING;
update_ui(ctx);
@@ -795,23 +799,18 @@ static void on_play_pause_click(lv_event_t* e) {
static void on_stop_click(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
if (ctx->state == STATE_IDLE) return;
update_history_entry(ctx, ctx->filepath, ctx->pos_sec, ctx->total_sec);
save_history(ctx);
refresh_history_list(ctx);
ESP_LOGI(TAG, "History updated on stop: %s pos %d total %d", ctx->filepath, ctx->pos_sec, ctx->total_sec);
// Update history on stop for 10min+ audios
bool is_long = (ctx->total_sec >= 600) || (ctx->file_data_size >= 600*16000L);
if (is_long) {
update_history_entry(ctx, ctx->filepath, ctx->pos_sec, ctx->total_sec);
save_history(ctx);
refresh_history_list(ctx);
ESP_LOGI(TAG, "History updated on stop: %s pos %d total %d", ctx->filepath, ctx->pos_sec, ctx->total_sec);
}
ctx->state = STATE_IDLE;
update_ui(ctx);
}
static void on_volume_slider_changed(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
if (!ctx || !ctx->slider_volume) return;
ctx->volume = lv_slider_get_value(ctx->slider_volume);
if (ctx->volume < 0) ctx->volume = 0;
if (ctx->volume > 100) ctx->volume = 100;
save_volume(ctx);
}
static void on_seek_back_click(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
request_seek(ctx, -SEEK_SECONDS);
@@ -822,15 +821,22 @@ static void on_seek_fwd_click(lv_event_t* e) {
request_seek(ctx, SEEK_SECONDS);
}
static void on_close_click(lv_event_t* e) {
(void)e;
wait_for_playback_task_to_exit(&g_ctx);
tt_app_stop();
static void on_volume_slider_changed(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
lv_obj_t* slider = lv_event_get_target(e);
int vol = lv_slider_get_value(slider);
ctx->volume = vol;
save_volume(ctx);
}
static void on_library_click(lv_event_t* e) {
(void)e;
show_history_screen(&g_ctx);
static void on_close_click(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
tt_app_stop(ctx->app_handle);
}
static void on_history_open_click(lv_event_t* e) {
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
show_history_screen(ctx);
}
static void on_history_cancel_click(lv_event_t* e) {
@@ -838,34 +844,6 @@ static void on_history_cancel_click(lv_event_t* e) {
hide_history_screen(ctx);
}
/* Consumed once by the host-side device test; never exposed in the player UI. */
static void on_dev_autoclose_timer(lv_timer_t* timer) {
lv_timer_del(timer);
ESP_LOGI(TAG, "DEV_TEST invoking the normal Close path");
on_close_click(NULL);
}
static void schedule_dev_autoclose_if_requested(AppCtx* ctx) {
char marker_path[512] = {0};
size_t marker_path_size = sizeof(marker_path);
tt_app_get_user_data_child_path(ctx->app_handle, "mp3player_dev_autoclose_ms", marker_path, &marker_path_size);
if (marker_path_size == 0) return;
FILE* marker = fopen(marker_path, "r");
if (!marker) return;
int delay_ms = 0;
int parsed = fscanf(marker, "%d", &delay_ms);
fclose(marker);
unlink(marker_path);
if (parsed != 1 || delay_ms < 1000 || delay_ms > 30000) {
ESP_LOGW(TAG, "DEV_TEST ignored invalid autoclose request");
return;
}
ESP_LOGI(TAG, "DEV_TEST scheduling normal Close in %d ms", delay_ms);
lv_timer_create(on_dev_autoclose_timer, delay_ms, NULL);
}
/* ─── App Lifecycle ─── */
static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
memset(&g_ctx, 0, sizeof(g_ctx));
@@ -877,7 +855,9 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
g_ctx.pcm_buf = (mp3d_sample_t*)malloc(MINIMP3_MAX_SAMPLES_PER_FRAME * sizeof(mp3d_sample_t));
for (int tries=0; tries<5; tries++) {
struct Device* dev = device_find_by_name("audio-stream0");
struct Device* dev = device_find_by_name("audio-stream");
if (dev) { g_ctx.stream_dev = dev; break; }
dev = device_find_first_by_type(&AUDIO_STREAM_TYPE);
if (dev) { g_ctx.stream_dev = dev; break; }
ESP_LOGW(TAG, "Audio-stream not found, retry %d/5", tries);
vTaskDelay(pdMS_TO_TICKS(200));
@@ -941,7 +921,7 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
strncpy(g_ctx.filepath, last_path, sizeof(g_ctx.filepath)-1);
g_ctx.filepath[sizeof(g_ctx.filepath)-1]=0;
g_ctx.start_pos_sec = last_pos;
ESP_LOGI(TAG, "Resuming last play %s at %d sec from history", g_ctx.filepath, last_pos);
ESP_LOGI(TAG, "Resuming last 10min+ play %s at %d sec from history", g_ctx.filepath, last_pos);
has_file_param = true;
}
}
@@ -965,11 +945,34 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
}
}
// ─── UI Layout ───
// Full-screen player with only the requested bottom control row.
// ─── UI Layout (redesigned per request) ───
// No toolbar, no card container, volume + controls at bottom
lv_obj_set_style_bg_color(parent, lv_color_hex(0x11111B), 0);
lv_obj_set_style_bg_opa(parent, LV_OPA_COVER, 0);
// Close button top-left since toolbar removed
lv_obj_t* btn_close = lv_btn_create(parent);
lv_obj_set_size(btn_close, 36, 28);
lv_obj_align(btn_close, LV_ALIGN_TOP_LEFT, 6, 6);
lv_obj_set_style_radius(btn_close, 8, 0);
lv_obj_set_style_bg_color(btn_close, lv_color_hex(0x313244), 0);
lv_obj_t* lbl_close = lv_label_create(btn_close);
lv_label_set_text(lbl_close, LV_SYMBOL_CLOSE);
lv_obj_center(lbl_close);
lv_obj_add_event_cb(btn_close, on_close_click, LV_EVENT_CLICKED, &g_ctx);
// History button top-right
lv_obj_t* btn_hist = lv_btn_create(parent);
lv_obj_set_size(btn_hist, 36, 28);
lv_obj_align(btn_hist, LV_ALIGN_TOP_RIGHT, -6, 6);
lv_obj_set_style_radius(btn_hist, 8, 0);
lv_obj_set_style_bg_color(btn_hist, lv_color_hex(0x313244), 0);
lv_obj_t* lbl_hist_btn = lv_label_create(btn_hist);
lv_label_set_text(lbl_hist_btn, LV_SYMBOL_LIST);
lv_obj_center(lbl_hist_btn);
lv_obj_add_event_cb(btn_hist, on_history_open_click, LV_EVENT_CLICKED, &g_ctx);
g_ctx.btn_history = btn_hist;
// Center content vertically (title, status, progress, time)
lv_obj_t* center_box = lv_obj_create(parent);
lv_obj_remove_style_all(center_box);
@@ -979,7 +982,11 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
lv_obj_set_flex_align(center_box, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_gap(center_box, 12, 0);
g_ctx.center_box = center_box;
lv_obj_t* icon = lv_label_create(center_box);
lv_label_set_text(icon, LV_SYMBOL_AUDIO);
lv_obj_set_style_text_color(icon, lv_color_hex(0x89B4FA), 0);
g_ctx.lbl_title = lv_label_create(center_box);
lv_obj_set_width(g_ctx.lbl_title, lv_pct(95));
lv_obj_set_style_text_align(g_ctx.lbl_title, LV_TEXT_ALIGN_CENTER, 0);
@@ -1060,67 +1067,58 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
// Populate history list
refresh_history_list(&g_ctx);
// Visible volume row restored above the fixed six-button bottom row.
lv_obj_t* vol_box = lv_obj_create(parent);
lv_obj_remove_style_all(vol_box);
lv_obj_set_size(vol_box, lv_pct(92), 26);
lv_obj_align(vol_box, LV_ALIGN_BOTTOM_MID, 0, -60);
g_ctx.volume_box = vol_box;
lv_obj_set_flex_flow(vol_box, LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(vol_box, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_column(vol_box, 8, 0);
lv_obj_t* lbl_volume = lv_label_create(vol_box);
lv_label_set_text(lbl_volume, "Volume");
lv_obj_set_style_text_color(lbl_volume, lv_color_hex(0xCDD6F4), 0);
lv_obj_set_style_text_font(lbl_volume, lv_theme_get_font_small(vol_box), 0);
g_ctx.slider_volume = lv_slider_create(vol_box);
lv_obj_set_size(g_ctx.slider_volume, 170, 12);
lv_slider_set_range(g_ctx.slider_volume, 0, 100);
lv_slider_set_value(g_ctx.slider_volume, g_ctx.volume, LV_ANIM_OFF);
lv_obj_set_style_bg_color(g_ctx.slider_volume, lv_color_hex(0x313244), LV_PART_MAIN);
lv_obj_set_style_bg_color(g_ctx.slider_volume, lv_color_hex(0x89B4FA), LV_PART_INDICATOR);
lv_obj_add_event_cb(g_ctx.slider_volume, on_volume_slider_changed, LV_EVENT_VALUE_CHANGED, &g_ctx);
// Bottom-only controls: Close, -15s, Play/Pause, Stop, +15s, Folder.
// Bottom controls container (volume + playback)
lv_obj_t* bottom_box = lv_obj_create(parent);
lv_obj_remove_style_all(bottom_box);
lv_obj_set_size(bottom_box, lv_pct(100), 52);
lv_obj_align(bottom_box, LV_ALIGN_BOTTOM_MID, 0, -5);
lv_obj_set_size(bottom_box, lv_pct(100), 90);
lv_obj_align(bottom_box, LV_ALIGN_BOTTOM_MID, 0, -10);
g_ctx.bottom_box = bottom_box;
lv_obj_set_flex_flow(bottom_box, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(bottom_box, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_gap(bottom_box, 10, 0);
lv_obj_set_style_pad_all(bottom_box, 5, 0);
// Volume slider at very bottom as requested
lv_obj_t* vol_box = lv_obj_create(bottom_box);
lv_obj_remove_style_all(vol_box);
lv_obj_set_size(vol_box, lv_pct(92), 20);
lv_obj_set_flex_flow(vol_box, LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(vol_box, LV_FLEX_ALIGN_SPACE_BETWEEN, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_t* lbl_vol_icon = lv_label_create(vol_box);
lv_label_set_text(lbl_vol_icon, LV_SYMBOL_VOLUME_MAX);
lv_obj_set_style_text_color(lbl_vol_icon, lv_color_hex(0xA6ADC8), 0);
g_ctx.slider_volume = lv_slider_create(vol_box);
lv_obj_set_size(g_ctx.slider_volume, lv_pct(82), 6);
lv_slider_set_range(g_ctx.slider_volume, 0, 100);
lv_slider_set_value(g_ctx.slider_volume, g_ctx.volume, LV_ANIM_OFF);
lv_obj_set_style_bg_color(g_ctx.slider_volume, lv_color_hex(0x45475A), LV_PART_MAIN);
lv_obj_set_style_bg_color(g_ctx.slider_volume, lv_color_hex(0x89B4FA), LV_PART_INDICATOR);
lv_obj_set_style_bg_color(g_ctx.slider_volume, lv_color_hex(0xCDD6F4), LV_PART_KNOB);
lv_obj_add_event_cb(g_ctx.slider_volume, on_volume_slider_changed, LV_EVENT_VALUE_CHANGED, &g_ctx);
// Single line controls: -10s, Play/Pause, Stop, +10s
lv_obj_t* ctrl_box = lv_obj_create(bottom_box);
lv_obj_remove_style_all(ctrl_box);
lv_obj_set_size(ctrl_box, lv_pct(98), 42);
lv_obj_set_size(ctrl_box, lv_pct(96), 40);
lv_obj_set_flex_flow(ctrl_box, LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(ctrl_box, LV_FLEX_ALIGN_SPACE_EVENLY, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_t* btn_close = lv_btn_create(ctrl_box);
lv_obj_set_size(btn_close, 34, 36);
lv_obj_set_style_radius(btn_close, 12, 0);
lv_obj_set_style_bg_color(btn_close, lv_color_hex(0x313244), 0);
lv_obj_t* lbl_close = lv_label_create(btn_close);
lv_label_set_text(lbl_close, LV_SYMBOL_CLOSE);
lv_obj_center(lbl_close);
lv_obj_add_event_cb(btn_close, on_close_click, LV_EVENT_CLICKED, &g_ctx);
// -15s
// -10s
g_ctx.btn_seek_back = lv_btn_create(ctrl_box);
lv_obj_set_size(g_ctx.btn_seek_back, 46, 36);
lv_obj_set_size(g_ctx.btn_seek_back, 58, 36);
lv_obj_set_style_radius(g_ctx.btn_seek_back, 12, 0);
lv_obj_set_style_bg_color(g_ctx.btn_seek_back, lv_color_hex(0x313244), 0);
lv_obj_set_style_text_color(g_ctx.btn_seek_back, lv_color_hex(0xCDD6F4), 0);
lv_obj_t* lbl_back = lv_label_create(g_ctx.btn_seek_back);
lv_label_set_text(lbl_back, "-15s");
lv_label_set_text(lbl_back, "-10s");
lv_obj_center(lbl_back);
lv_obj_add_event_cb(g_ctx.btn_seek_back, on_seek_back_click, LV_EVENT_CLICKED, &g_ctx);
// Play/Pause icon only
g_ctx.btn_play_pause = lv_btn_create(ctrl_box);
lv_obj_set_size(g_ctx.btn_play_pause, 44, 36);
lv_obj_set_size(g_ctx.btn_play_pause, 56, 36);
lv_obj_set_style_radius(g_ctx.btn_play_pause, 18, 0);
lv_obj_set_style_bg_color(g_ctx.btn_play_pause, lv_color_hex(0x89B4FA), 0);
lv_obj_set_style_text_color(g_ctx.btn_play_pause, lv_color_hex(0x11111B), 0);
@@ -1128,9 +1126,10 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
lv_label_set_text(lbl_play, LV_SYMBOL_PLAY);
lv_obj_center(lbl_play);
lv_obj_add_event_cb(g_ctx.btn_play_pause, on_play_pause_click, LV_EVENT_CLICKED, &g_ctx);
// Stop icon only
g_ctx.btn_stop = lv_btn_create(ctrl_box);
lv_obj_set_size(g_ctx.btn_stop, 38, 36);
lv_obj_set_size(g_ctx.btn_stop, 50, 36);
lv_obj_set_style_radius(g_ctx.btn_stop, 12, 0);
lv_obj_set_style_bg_color(g_ctx.btn_stop, lv_color_hex(0xF38BA8), 0);
lv_obj_set_style_text_color(g_ctx.btn_stop, lv_color_hex(0x11111B), 0);
@@ -1139,26 +1138,16 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
lv_obj_center(lbl_stop);
lv_obj_add_event_cb(g_ctx.btn_stop, on_stop_click, LV_EVENT_CLICKED, &g_ctx);
// +15s
// +10s
g_ctx.btn_seek_fwd = lv_btn_create(ctrl_box);
lv_obj_set_size(g_ctx.btn_seek_fwd, 46, 36);
lv_obj_set_size(g_ctx.btn_seek_fwd, 58, 36);
lv_obj_set_style_radius(g_ctx.btn_seek_fwd, 12, 0);
lv_obj_set_style_bg_color(g_ctx.btn_seek_fwd, lv_color_hex(0x313244), 0);
lv_obj_set_style_text_color(g_ctx.btn_seek_fwd, lv_color_hex(0xCDD6F4), 0);
lv_obj_t* lbl_fwd = lv_label_create(g_ctx.btn_seek_fwd);
lv_label_set_text(lbl_fwd, "+15s");
lv_label_set_text(lbl_fwd, "+10s");
lv_obj_center(lbl_fwd);
lv_obj_add_event_cb(g_ctx.btn_seek_fwd, on_seek_fwd_click, LV_EVENT_CLICKED, &g_ctx);
// Folder opens the persisted in-app playback history.
lv_obj_t* btn_hist = lv_btn_create(ctrl_box);
lv_obj_set_size(btn_hist, 34, 36);
lv_obj_set_style_radius(btn_hist, 12, 0);
lv_obj_set_style_bg_color(btn_hist, lv_color_hex(0x313244), 0);
lv_obj_t* lbl_hist_btn = lv_label_create(btn_hist);
lv_label_set_text(lbl_hist_btn, LV_SYMBOL_DIRECTORY);
lv_obj_center(lbl_hist_btn);
lv_obj_add_event_cb(btn_hist, on_library_click, LV_EVENT_CLICKED, &g_ctx);
if (!g_ctx.stream_dev) {
lv_label_set_text(g_ctx.lbl_status, "Error: Audio Not Found");
@@ -1179,7 +1168,6 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
g_ctx.state = STATE_PLAYING;
update_ui(&g_ctx);
xTaskCreate(mp3_playback_task, "mp3_play", 6144, &g_ctx, 6, &g_ctx.playback_task_handle);
schedule_dev_autoclose_if_requested(&g_ctx);
}
}
}
@@ -1211,11 +1199,16 @@ static void onHideApp(AppHandle app, void* data) {
vTaskDelay(pdMS_TO_TICKS(100));
}
// Persist the active track for normal relaunch, regardless of duration.
// Save volume and history (only for 10min+ audios as requested)
save_volume(&g_ctx);
update_history_entry(&g_ctx, g_ctx.filepath, g_ctx.pos_sec, g_ctx.total_sec);
save_history(&g_ctx);
ESP_LOGI(TAG, "History saved on hide: %s pos %d total %d", g_ctx.filepath, g_ctx.pos_sec, g_ctx.total_sec);
bool is_long = (g_ctx.total_sec >= 600) || (g_ctx.file_data_size >= 600*16000L);
if (is_long) {
update_history_entry(&g_ctx, g_ctx.filepath, g_ctx.pos_sec, g_ctx.total_sec);
save_history(&g_ctx);
ESP_LOGI(TAG, "History saved on hide: %s pos %d total %d (is_long=%d)", g_ctx.filepath, g_ctx.pos_sec, g_ctx.total_sec, is_long);
} else {
ESP_LOGI(TAG, "Skip history save on hide: not long audio total=%d data=%ld", g_ctx.total_sec, g_ctx.file_data_size);
}
if (g_ctx.input_buf) {
free(g_ctx.input_buf);
-30
View File
@@ -1,30 +0,0 @@
# Pipecat Voice
Minimal native Tactility client for the LAN voice-adapter protocol v1. Opening the app immediately connects to `ws://192.168.68.102:8644/api/esp32/voice/ws`, sends the versioned `start` declaration, and continuously streams 16 kHz mono signed-16-bit PCM. There is no Connect button, speaker/transport picker, PTT control, text entry, transcript display, or credential on the device.
The Mac-hosted adapter owns VAD, STT, Pipecat/Hermes orchestration, TTS, and credentials. It returns metadata followed by a single PCM response frame; the app pauses capture, plays that response at the declared rate through Tactility `i2s0`, then resumes capture. The app only displays connection/streaming/reconnect/configuration state and uses the standard toolbar to exit.
## Configuration
`config.json` in app user data can override the non-secret endpoint and allowed adapter device id:
```json
{"server_url":"ws://192.168.68.102:8644/api/esp32/voice/ws","device_id":"tactility-14c19d1a790"}
```
Only `ws://` private-LAN endpoints are accepted; loopback endpoints are rejected. Invalid configuration is a terminal actionable state. Network failures use bounded 1, 2, 4, 8, 16, then 30-second reconnect delays. Protocol and payload violations close the session and reconnect; stale audio is never queued.
## Build and test
```sh
cc -std=c11 -Wall -Wextra -Werror -I main/Source tests/test_voice_protocol.c main/Source/voice_protocol.c -o /tmp/pipecatvoice-protocol-test
/tmp/pipecatvoice-protocol-test
unset PYTHONPATH PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.3_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/PipecatVoice build esp32s3 --local-sdk
```
The compatible adapter is documented at `/Users/adolforeyna/Projects/voice-assistant/hermes-esp32-voice-gateway/docs/lan-ws-pipecat-adapter.md`. It is the only supported endpoint; Pipecat `:7861` is not an optional device transport.
@@ -1,89 +0,0 @@
# Pipecat SmallWebRTC ESP32 feasibility spike
Date: 2026-08-10
## Decision
**NO-GO for a Tactility external ELF on the current SDK; GO only for a separate, full ESP-IDF firmware application.**
Pipecat has an official native ESP32 client, so the required transport is real and source-proven. It cannot presently be used as a Tactility runtime ELF without a firmware/SDK integration project: the client is a full ESP-IDF firmware with private, static component dependencies and system configuration that the ELF loader does not provide or export. Do not replace the rejected raw-WebSocket implementation with another transport until that integration is designed and proven.
This is not a proposal to flash anything. No device was deployed or flashed during this spike.
## Source-pinned native client
| Item | Evidence |
| --- | --- |
| Client | `https://github.com/pipecat-ai/pipecat-esp32`, commit `e70e3b1f0576e502af9e390434e1a6e8a5cd0d2e`, cloned with all recursive submodules |
| Top-level licence | MIT (`LICENSE`, copyright Daily/OpenAI) |
| Pipecat server | local installed `pipecat-ai 1.7.0`, Python 3.11 virtual environment |
| Server ESP32 support | `SmallWebRTCRequestHandler(..., esp32_mode=True, host=...)` munges the SDP; `smallwebrtc_sdp_munging()` removes SHA-384/SHA-512 fingerprints and retains only the chosen host's ICE candidates |
| ESP-IDF build used | local ESP-IDF `v5.5.2`, environment `idf5.5_py3.9_env` |
| Native build result | upstream `esp32-s3-box-3` built successfully, including `peer`, `srtp`, `esp-libopus`, Wi-Fi, HTTP client, DTLS/SRTP, Opus, and the ESP-BOX-3 BSP |
The official client is designed for ESP32-S3 and uses the `libpeer` API. Its `PeerConfiguration` sets `CODEC_OPUS`, creates a peer connection, installs ICE/data/audio callbacks, and invokes `peer_connection_create_offer()`. This is the source-proven native WebRTC implementation; it owns ICE, DTLS-SRTP, RTP, and Opus rather than hand-implementing any of them.
## Required SmallWebRTC contract
The product transport is SmallWebRTC HTTP signaling plus WebRTC media, never the old raw WebSocket PCM/JSON protocol:
1. `POST /start` with `transport: "webrtc"`, `enableDefaultIceServers: false`, and optional `body`; retain the returned `sessionId`.
2. `POST /sessions/{sessionId}/api/offer` with `{ "sdp": ..., "type": "offer", "pc_id": optional, "restart_pc": optional, "requestData": optional }`; Pipecat returns SDP answer, type, and `pc_id`.
3. `PATCH /sessions/{sessionId}/api/offer` with `{ "pc_id": ..., "candidates": [{ "candidate": ..., "sdp_mid": ..., "sdp_mline_index": ... }] }` for trickle ICE. An empty candidate is the end-of-candidates marker.
4. Use the negotiated WebRTC audio track continuously. The runner starts the bot after the offer is processed.
The local Pipecat source also supports the direct `/api/offer` route used by the current official ESP32 example. The session form above is the approved application contract because it supports Pipecat runner session lifecycle. The live endpoint returned HTTP 200 to `/status`, but it was not restarted with `--esp32`; therefore no live offer/candidate exchange is represented as ESP32 validation.
## Audio adapter boundary
The official client source (`media.cpp`) uses 16 kHz, mono, signed 16-bit PCM (`640` bytes = 320 samples = 20 ms) and encodes it as Opus for `peer_connection_send_audio()`. Inbound WebRTC audio reaches the `onaudiotrack` callback as Opus, is decoded to the same PCM shape, and is written to the speaker codec.
For a future firmware-level integration, Tactility must keep ownership at the following boundary (no hard-coded board pins):
- acquire the existing Tactility `audio_stream` / `i2s_controller` service;
- pull fixed 20 ms frames, 16 kHz mono S16LE, into the native client encoder;
- feed decoded remote S16LE frames to the existing output service;
- serialize I/O ownership, keep bounded queues, and drop stale audio rather than accumulating latency;
- close peer/media callbacks before releasing the audio device.
The current kernel exports `audio_stream_open_input`, `audio_stream_open_output`, `audio_stream_read`, `audio_stream_write`, `audio_stream_close`, and `i2s_controller_read`/`i2s_controller_write`. Those APIs are the usable boundary, not a reason to configure physical pins in the app.
## Full-firmware build evidence
The following was run in a temporary checkout; non-secret placeholder Wi-Fi values were used and no flash command was run:
```text
cd /tmp/pipecat-esp32-spike
# cloned pipecat-esp32 at e70e3b1... and initialized all recursive submodules
cd esp32-s3-box-3
unset PYTHONPATH PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
export WIFI_SSID=spike
export WIFI_PASSWORD=spike
export PIPECAT_SMALLWEBRTC_URL=http://192.168.68.112:7860/api/offer
source /Users/adolforeyna/esp/esp-idf/export.sh
idf.py build
```
Actual result: `src.elf` and `src.bin` were produced; the IDF build ended with `Project build complete`. `src.bin` is **1,493,408 bytes** and the upstream 1.5 MiB app partition reported **79,712 bytes (5%) free**. `xtensa-esp32s3-elf-size src.elf` reported text `1,304,360`, data `201,012`, bss `2,863,205` (total `4,368,577`). The linked firmware has no undefined dynamic symbols.
This is important capacity evidence: even before adapting it to the target board and Tactility services, the supported client nearly fills its own dedicated application partition and has a 2.86 MiB BSS footprint.
## Why this does not link as a Tactility ELF
Tactility's `TactilitySDK.cmake` calls `project_elf()`. Its loader CMake builds a PIC shared ELF with `-nostartfiles -nostdlib -shared -e app_main`, and links only `main` plus explicitly listed `ELF_COMPONENTS` / `ELF_LIBS`. The current PipecatVoice component declares only `REQUIRES TactilitySDK lwip`.
The upstream client instead requires full firmware components including `peer`, `srtp`, `esp-libopus`, `esp_http_client`, `esp_wifi`, `nvs_flash`, `esp_psram`, `esp_netif`, mbedTLS, and ESP-BOX-3 BSP. The official `peer` static archive has unresolved references to the linked firmware environment such as `mbedtls_ssl_conf_dtls_srtp_protection_profiles`, `mbedtls_ssl_config_defaults`, `lwip_inet_ntop`, and socket/ICE helpers. The Tactility kernel export table contains the audio service APIs listed above but no `peer_connection`, `opus_*`, `srtp_*`, `mbedtls_*`, `esp_http_client*`, `esp_wifi*`, or `esp_netif*` exports.
Attempting the ordinary app build also hit a concrete local SDK packaging blocker before linking: `tactility.py Apps/PipecatVoice build esp32s3 --local-sdk` reported that `Buildscripts/TactilitySDK/0.8.0-dev-esp32s3/TactilitySDK` is missing. This must be corrected for later normal ELF builds, but it is distinct from the component/loader incompatibility.
Therefore copying the client sources or merely adding `REQUIRES peer` would not make a runnable ELF: it would either fail to find the private IDF component libraries during the ELF link or produce imports that the firmware loader cannot resolve. Statically embedding all dependencies is unproven and high-risk because of ELF size, duplicate runtime/library state, SDK configuration, and Wi-Fi/codec ownership conflicts.
## Security, licensing, and follow-up gate
- Do not log SDP, ICE details, credentials, or raw audio. The source-level HTTP helper currently logs offer/answer in debug mode; any reused code must remove that logging.
- The upstream defaults deliberately disable TLS certificate verification for its demo. Production must use HTTPS with a pinned/validated trust chain; do not inherit that setting.
- Preserve MIT notices for Pipecat ESP32 and audit each pinned submodule separately (`libpeer`, SRTP/libSRTP, Opus, and Espressif managed components have their own licenses).
- Do not use the existing PipecatVoice raw WebSocket code or its historical configuration as a fallback; it is protocol-incompatible with SmallWebRTC.
A firmware-level project must first export/package the required WebRTC dependency set, prove an external ELF link with zero unresolved loader symbols (or move the client into firmware), set deterministic memory budgets, and then perform an `--esp32` SmallWebRTC live offer/ICE/media test. Only after that gate may the approved minimal auto-start UI be implemented.
-320
View File
@@ -1,320 +0,0 @@
#include <tt_app.h>
#include <tt_lvgl.h>
#include <tt_lvgl_toolbar.h>
#include <tactility/device.h>
#include <tactility/drivers/audio_stream.h>
#include <cJSON.h>
#include <esp_log.h>
#include <esp_random.h>
#include <freertos/FreeRTOS.h>
#include <freertos/semphr.h>
#include <freertos/task.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "voice_protocol.h"
#include "websocket.h"
/* Exported by firmware 0.8.0-dev although absent from the CDN SDK header. */
struct Device* device_find_by_name(const char* name);
struct Device* device_find_first_by_type(const struct DeviceType* type);
#define TAG "PipecatVoice"
#define DEFAULT_ENDPOINT "ws://192.168.68.102:8644/api/esp32/voice/ws"
#define DEFAULT_DEVICE_ID "tactility-14c19d1a790"
typedef struct {
AppHandle app;
volatile bool visible;
volatile bool streaming;
volatile bool playing;
volatile bool socket_failed;
int fd;
PvState state;
unsigned retry_attempt;
size_t expected_audio_bytes;
char endpoint[128];
char device_id[64];
char api_key[128];
char detail[96];
lv_obj_t* state_label;
lv_obj_t* detail_label;
struct Device* stream_dev;
AudioStreamHandle input_handle;
AudioStreamHandle output_handle;
TaskHandle_t worker;
TaskHandle_t receiver;
SemaphoreHandle_t socket_lock;
SemaphoreHandle_t audio_lock;
} VoiceContext;
static void update_ui(VoiceContext* ctx) {
if (!ctx->visible || !tt_lvgl_lock(pdMS_TO_TICKS(100))) return;
lv_label_set_text(ctx->state_label, pv_state_label(ctx->state));
lv_obj_set_style_text_color(ctx->state_label,
ctx->state == PV_STREAMING ? lv_color_hex(0x32c86e) :
ctx->state == PV_FAILED ? lv_color_hex(0xd94a4a) : lv_color_hex(0xe0b64a), LV_PART_MAIN);
lv_label_set_text(ctx->detail_label, ctx->detail);
tt_lvgl_unlock();
}
static void set_state(VoiceContext* ctx, PvState state, const char* detail) {
ctx->state = state;
snprintf(ctx->detail, sizeof(ctx->detail), "%s", detail);
update_ui(ctx);
}
static bool open_input_stream(VoiceContext* ctx) {
if (ctx->stream_dev == NULL) return false;
if (ctx->output_handle) { audio_stream_close(ctx->output_handle); ctx->output_handle = NULL; }
if (ctx->input_handle) return true;
struct AudioStreamConfig cfg = {
.sample_rate = 16000,
.bits_per_sample = 16,
.channels = 1,
};
if (audio_stream_open_input(ctx->stream_dev, &cfg, &ctx->input_handle) != ERROR_NONE) {
ESP_LOGW(TAG, "audio_stream_open_input failed");
return false;
}
audio_stream_set_mute(ctx->stream_dev, AUDIO_CODEC_DIR_INPUT, false);
audio_stream_set_volume(ctx->stream_dev, AUDIO_CODEC_DIR_INPUT, 100.0f);
return true;
}
static bool open_output_stream(VoiceContext* ctx) {
if (ctx->stream_dev == NULL) return false;
if (ctx->input_handle) { audio_stream_close(ctx->input_handle); ctx->input_handle = NULL; }
if (ctx->output_handle) return true;
struct AudioStreamConfig cfg = {
.sample_rate = 16000,
.bits_per_sample = 16,
.channels = 1,
};
if (audio_stream_open_output(ctx->stream_dev, &cfg, &ctx->output_handle) != ERROR_NONE) {
ESP_LOGW(TAG, "audio_stream_open_output failed");
return false;
}
audio_stream_set_mute(ctx->stream_dev, AUDIO_CODEC_DIR_OUTPUT, false);
audio_stream_set_volume(ctx->stream_dev, AUDIO_CODEC_DIR_OUTPUT, 80.0f);
return true;
}
static void close_audio(VoiceContext* ctx) {
if (ctx->output_handle) { audio_stream_close(ctx->output_handle); ctx->output_handle = NULL; }
if (ctx->input_handle) { audio_stream_close(ctx->input_handle); ctx->input_handle = NULL; }
}
static int send_locked(VoiceContext* ctx, const uint8_t* data, size_t length, bool binary) {
if (ctx->fd < 0 || xSemaphoreTake(ctx->socket_lock, pdMS_TO_TICKS(500)) != pdTRUE) return -1;
int result = ws_send(ctx->fd, data, length, binary);
xSemaphoreGive(ctx->socket_lock);
return result;
}
static void handle_event(VoiceContext* ctx, const char* text) {
cJSON* root = cJSON_Parse(text);
if (root == NULL) return;
cJSON* version = cJSON_GetObjectItem(root, "v");
cJSON* event = cJSON_GetObjectItem(root, "event");
if (!cJSON_IsNumber(version) || version->valueint != PV_PROTOCOL_VERSION || !cJSON_IsString(event)) {
cJSON_Delete(root);
return;
}
if (strcmp(event->valuestring, "ready") == 0) {
ctx->streaming = true;
} else if (strcmp(event->valuestring, "state") == 0) {
cJSON* state = cJSON_GetObjectItem(root, "state");
if (cJSON_IsString(state) && strcmp(state->valuestring, "listening") == 0) {
ctx->streaming = true;
set_state(ctx, PV_STREAMING, "Continuous microphone streaming");
}
} else if (strcmp(event->valuestring, "audio") == 0) {
cJSON* format = cJSON_GetObjectItem(root, "format");
cJSON* rate = cJSON_GetObjectItem(root, "sample_rate");
cJSON* channels = cJSON_GetObjectItem(root, "channels");
cJSON* width = cJSON_GetObjectItem(root, "sample_width");
cJSON* length = cJSON_GetObjectItem(root, "byte_length");
if (cJSON_IsString(format) && cJSON_IsNumber(rate) && cJSON_IsNumber(channels) && cJSON_IsNumber(width) &&
cJSON_IsNumber(length) && pv_valid_downstream_audio(format->valuestring, rate->valueint, channels->valueint,
width->valueint, (size_t)length->valueint)) {
ctx->expected_audio_bytes = (size_t)length->valueint;
ctx->playing = true;
set_state(ctx, PV_STREAMING, "Playing response; microphone paused");
} else {
ctx->socket_failed = true;
}
} else if (strcmp(event->valuestring, "error") == 0) {
/* The server-provided message is intentionally not copied to display/logs. */
ctx->socket_failed = true;
}
cJSON_Delete(root);
}
static void receiver_task(void* argument) {
VoiceContext* ctx = argument;
uint8_t* buffer = malloc(PV_DOWNSTREAM_MAX + 1U);
if (buffer == NULL) { ctx->socket_failed = true; ctx->receiver = NULL; vTaskDelete(NULL); }
while (ctx->visible && ctx->fd >= 0) {
int opcode = 0;
bool final = false;
int received = ws_recv(ctx->fd, &opcode, &final, buffer, PV_DOWNSTREAM_MAX);
if (received < 0 || !final) { ctx->socket_failed = true; break; }
if (opcode == 0x01) {
buffer[received] = '\0';
handle_event(ctx, (const char*)buffer);
} else if (opcode == 0x02) {
if (!ctx->playing || !pv_binary_matches_metadata(ctx->expected_audio_bytes, (size_t)received)) { ctx->socket_failed = true; break; }
xSemaphoreTake(ctx->audio_lock, portMAX_DELAY);
bool ok = open_output_stream(ctx);
size_t written = 0;
if (ok) ok = audio_stream_write(ctx->output_handle, buffer, (size_t)received, &written, pdMS_TO_TICKS(3000)) == ERROR_NONE;
open_input_stream(ctx);
xSemaphoreGive(ctx->audio_lock);
if (!ok || written != (size_t)received) { ctx->socket_failed = true; break; }
ctx->expected_audio_bytes = 0;
ctx->playing = false;
} else if (opcode == 0x09) {
if (xSemaphoreTake(ctx->socket_lock, pdMS_TO_TICKS(500)) == pdTRUE) {
ws_send_pong(ctx->fd, buffer, (size_t)received);
xSemaphoreGive(ctx->socket_lock);
}
} else if (opcode == 0x08) {
ctx->socket_failed = true;
break;
}
}
free(buffer);
ctx->receiver = NULL;
vTaskDelete(NULL);
}
static void close_session(VoiceContext* ctx) {
int fd = ctx->fd;
ctx->fd = -1;
ctx->streaming = false;
ctx->playing = false;
ctx->expected_audio_bytes = 0;
if (fd >= 0) {
if (xSemaphoreTake(ctx->socket_lock, pdMS_TO_TICKS(100)) == pdTRUE) {
ws_send_close(fd);
xSemaphoreGive(ctx->socket_lock);
}
ws_close(fd);
}
close_audio(ctx);
}
static void worker_task(void* argument) {
VoiceContext* ctx = argument;
PvEndpoint endpoint;
if (!pv_parse_endpoint(ctx->endpoint, &endpoint)) {
set_state(ctx, PV_FAILED, "Set a private-LAN ws:// endpoint in config.json");
ctx->worker = NULL;
vTaskDelete(NULL);
}
uint8_t pcm[1024];
while (ctx->visible) {
set_state(ctx, ctx->retry_attempt ? PV_RECONNECTING : PV_CONNECTING,
ctx->retry_attempt ? "Retrying voice gateway" : "Connecting to voice gateway");
ctx->socket_failed = false;
ESP_LOGI(TAG, "Opening voice gateway session");
ctx->fd = ws_connect(endpoint.host, endpoint.port, endpoint.path, ctx->device_id, ctx->api_key);
if (ctx->fd >= 0) {
char session[64];
snprintf(session, sizeof(session), "pv-%08lx", (unsigned long)esp_random());
char start[256];
if (pv_make_start_json(start, sizeof(start), session, ctx->device_id) &&
send_locked(ctx, (const uint8_t*)start, strlen(start), false) == 0) {
if (!open_input_stream(ctx)) {
set_state(ctx, PV_FAILED, "Audio stream unavailable");
ctx->socket_failed = true;
} else {
xTaskCreate(receiver_task, "pv_rx", 6144, ctx, 6, &ctx->receiver);
uint32_t stable_ticks = 0;
while (ctx->visible && !ctx->socket_failed) {
if (!ctx->streaming || ctx->playing) { vTaskDelay(pdMS_TO_TICKS(20)); continue; }
xSemaphoreTake(ctx->audio_lock, portMAX_DELAY);
bool opened = open_input_stream(ctx);
size_t read = 0;
error_t read_result = opened ? audio_stream_read(ctx->input_handle, pcm, sizeof(pcm), &read, pdMS_TO_TICKS(100)) : ERROR_RESOURCE;
xSemaphoreGive(ctx->audio_lock);
if (!opened) { ctx->socket_failed = true; break; }
if (read_result == ERROR_NONE && pv_valid_pcm_chunk(read) && send_locked(ctx, pcm, read, true) < 0) ctx->socket_failed = true;
if (++stable_ticks >= 300) ctx->retry_attempt = 0;
}
}
}
}
close_session(ctx);
if (!ctx->visible) break;
uint32_t delay = pv_retry_delay_seconds(ctx->retry_attempt++);
ESP_LOGW(TAG, "Voice gateway session unavailable; retry in %lu seconds", (unsigned long)delay);
set_state(ctx, PV_RECONNECTING, "Gateway unavailable; retry scheduled");
for (uint32_t second = 0; ctx->visible && second < delay; ++second) vTaskDelay(pdMS_TO_TICKS(1000));
}
ctx->worker = NULL;
vTaskDelete(NULL);
}
static void load_config(VoiceContext* ctx) {
snprintf(ctx->endpoint, sizeof(ctx->endpoint), "%s", DEFAULT_ENDPOINT);
snprintf(ctx->device_id, sizeof(ctx->device_id), "%s", DEFAULT_DEVICE_ID);
ctx->api_key[0] = '\0';
char path[256]; size_t path_size = sizeof(path);
tt_app_get_user_data_child_path(ctx->app, "config.json", path, &path_size);
FILE* file = fopen(path, "r");
if (file == NULL) return;
char json[512]; size_t bytes = fread(json, 1, sizeof(json) - 1, file); fclose(file); json[bytes] = '\0';
cJSON* root = cJSON_Parse(json);
cJSON* endpoint = root ? cJSON_GetObjectItem(root, "server_url") : NULL;
cJSON* device = root ? cJSON_GetObjectItem(root, "device_id") : NULL;
cJSON* key = root ? cJSON_GetObjectItem(root, "api_key") : NULL;
if (cJSON_IsString(endpoint)) snprintf(ctx->endpoint, sizeof(ctx->endpoint), "%s", endpoint->valuestring);
if (cJSON_IsString(device)) snprintf(ctx->device_id, sizeof(ctx->device_id), "%s", device->valuestring);
if (cJSON_IsString(key)) snprintf(ctx->api_key, sizeof(ctx->api_key), "%s", key->valuestring);
cJSON_Delete(root);
}
static void* create_data(void) { VoiceContext* ctx = calloc(1, sizeof(*ctx)); if (ctx) ctx->fd = -1; return ctx; }
static void destroy_data(void* data) { free(data); }
static void on_create(AppHandle app, void* data) { ((VoiceContext*)data)->app = app; }
static void on_destroy(AppHandle app, void* data) { (void)app; (void)data; }
static void on_show(AppHandle app, void* data, lv_obj_t* parent) {
VoiceContext* ctx = data; ctx->visible = true; load_config(ctx);
ctx->stream_dev = device_find_by_name("audio-stream");
if (ctx->stream_dev == NULL) ctx->stream_dev = device_find_first_by_type(&AUDIO_STREAM_TYPE);
ctx->socket_lock = xSemaphoreCreateMutex();
ctx->audio_lock = xSemaphoreCreateMutex();
lv_obj_t* toolbar = tt_lvgl_toolbar_create_for_app(parent, app);
lv_obj_align(toolbar, LV_ALIGN_TOP_MID, 0, 0);
ctx->state_label = lv_label_create(parent);
lv_obj_align(ctx->state_label, LV_ALIGN_CENTER, 0, -30);
ctx->detail_label = lv_label_create(parent);
lv_obj_set_width(ctx->detail_label, lv_pct(88)); lv_label_set_long_mode(ctx->detail_label, LV_LABEL_LONG_WRAP);
lv_obj_set_style_text_align(ctx->detail_label, LV_TEXT_ALIGN_CENTER, LV_PART_MAIN);
lv_obj_align(ctx->detail_label, LV_ALIGN_CENTER, 0, 25);
if (ctx->stream_dev == NULL || ctx->socket_lock == NULL || ctx->audio_lock == NULL) set_state(ctx, PV_FAILED, "Audio service unavailable");
else xTaskCreate(worker_task, "pv_worker", 7168, ctx, 5, &ctx->worker);
}
static void on_hide(AppHandle app, void* data) {
(void)app; VoiceContext* ctx = data; ctx->visible = false; close_session(ctx);
for (unsigned i = 0; (ctx->worker || ctx->receiver) && i < 100; ++i) vTaskDelay(pdMS_TO_TICKS(10));
close_audio(ctx);
if (ctx->socket_lock) { vSemaphoreDelete(ctx->socket_lock); ctx->socket_lock = NULL; }
if (ctx->audio_lock) { vSemaphoreDelete(ctx->audio_lock); ctx->audio_lock = NULL; }
}
int main(int argc, char* argv[]) {
(void)argc; (void)argv;
tt_app_register((AppRegistration){.createData=create_data,.destroyData=destroy_data,.onCreate=on_create,.onDestroy=on_destroy,.onShow=on_show,.onHide=on_hide});
return 0;
}
@@ -1,113 +0,0 @@
#include "voice_protocol.h"
#include <stdio.h>
#include <string.h>
/* NOTE: Do not use ctype.h (isalnum/isalpha/isdigit/isspace) in this app. Those
* functions read the `_ctype_` table, which is resolved from the flashed firmware
* at runtime; the firmware's table does not behave correctly for side-loaded ELF
* apps, so isalnum('a') can return false. Use explicit ASCII range checks instead. */
static bool is_digit(unsigned char c) { return c >= '0' && c <= '9'; }
static bool is_space(unsigned char c) {
return c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\v' || c == '\f';
}
static bool is_alnum(unsigned char c) {
return is_digit(c) || (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z');
}
static bool copy_part(char* destination, size_t destination_size, const char* start, size_t length) {
if (length == 0 || length >= destination_size) return false;
memcpy(destination, start, length);
destination[length] = '\0';
return true;
}
static bool is_loopback(const char* host) {
return strcmp(host, "localhost") == 0 || strcmp(host, "::1") == 0 || strncmp(host, "127.", 4) == 0;
}
static bool valid_identifier(const char* value) {
if (value == NULL || *value == '\0') return false;
for (const unsigned char* p = (const unsigned char*)value; *p; ++p) {
if (!is_alnum(*p) && *p != '-' && *p != '_' && *p != '.') return false;
}
return true;
}
bool pv_parse_endpoint(const char* url, PvEndpoint* endpoint) {
if (url == NULL || endpoint == NULL || strncmp(url, "ws://", 5) != 0) return false;
const char* authority = url + 5;
const char* path = strchr(authority, '/');
const char* authority_end = path ? path : authority + strlen(authority);
const char* colon = NULL;
for (const char* p = authority; p < authority_end; ++p) {
if (*p == ':') {
if (colon != NULL) return false;
colon = p;
}
if (is_space((unsigned char)*p) || *p == '@' || *p == '?' || *p == '#') return false;
}
size_t host_length = (size_t)((colon ? colon : authority_end) - authority);
if (!copy_part(endpoint->host, sizeof(endpoint->host), authority, host_length) || is_loopback(endpoint->host)) return false;
endpoint->port = 80;
if (colon != NULL) {
unsigned long port = 0;
for (const char* p = colon + 1; p < authority_end; ++p) {
if (!is_digit((unsigned char)*p)) return false;
port = port * 10U + (unsigned long)(*p - '0');
if (port > 65535U) return false;
}
if (port == 0) return false;
endpoint->port = (uint16_t)port;
}
return path == NULL ? copy_part(endpoint->path, sizeof(endpoint->path), "/", 1)
: copy_part(endpoint->path, sizeof(endpoint->path), path, strlen(path));
}
bool pv_make_start_json(char* out, size_t out_size, const char* session_id, const char* device_id) {
if (out == NULL || !valid_identifier(session_id) || !valid_identifier(device_id)) return false;
int written = snprintf(out, out_size,
"{\"v\":1,\"event\":\"start\",\"session_id\":\"%s\",\"device_id\":\"%s\",\"audio\":{\"format\":\"pcm_s16le\",\"sample_rate\":16000,\"channels\":1,\"sample_width\":2}}",
session_id, device_id);
return written > 0 && (size_t)written < out_size;
}
bool pv_valid_pcm_chunk(size_t bytes) {
return bytes > 0 && bytes <= PV_PCM_CHUNK_MAX && (bytes % 2U) == 0;
}
bool pv_valid_downstream_audio(const char* format, int sample_rate, int channels, int sample_width, size_t byte_length) {
return format != NULL && strcmp(format, "pcm_s16le") == 0 && sample_rate > 0 && sample_rate <= 48000 &&
channels == 1 && sample_width == 2 && byte_length > 0 && byte_length <= PV_DOWNSTREAM_MAX &&
(byte_length % 2U) == 0;
}
bool pv_binary_matches_metadata(size_t expected_bytes, size_t received_bytes) {
return expected_bytes > 0 && expected_bytes == received_bytes;
}
uint32_t pv_retry_delay_seconds(unsigned attempt) {
uint32_t delay = 1;
while (attempt > 0 && delay < 30) {
delay *= 2;
--attempt;
}
return delay > 30 ? 30 : delay;
}
PvState pv_disconnect_state(bool endpoint_valid) {
return endpoint_valid ? PV_RECONNECTING : PV_FAILED;
}
const char* pv_state_label(PvState state) {
switch (state) {
case PV_CONNECTING: return "CONNECTING";
case PV_STREAMING: return "STREAMING";
case PV_RECONNECTING: return "RECONNECTING";
case PV_FAILED: return "FAILED";
default: return "FAILED";
}
}
@@ -1,31 +0,0 @@
#pragma once
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#define PV_PROTOCOL_VERSION 1
#define PV_PCM_CHUNK_MAX 16384U
#define PV_DOWNSTREAM_MAX 65536U
typedef enum {
PV_CONNECTING,
PV_STREAMING,
PV_RECONNECTING,
PV_FAILED,
} PvState;
typedef struct {
char host[64];
char path[96];
uint16_t port;
} PvEndpoint;
bool pv_parse_endpoint(const char* url, PvEndpoint* endpoint);
bool pv_make_start_json(char* out, size_t out_size, const char* session_id, const char* device_id);
bool pv_valid_pcm_chunk(size_t bytes);
bool pv_valid_downstream_audio(const char* format, int sample_rate, int channels, int sample_width, size_t byte_length);
bool pv_binary_matches_metadata(size_t expected_bytes, size_t received_bytes);
uint32_t pv_retry_delay_seconds(unsigned attempt);
PvState pv_disconnect_state(bool endpoint_valid);
const char* pv_state_label(PvState state);
-167
View File
@@ -1,167 +0,0 @@
#include "websocket.h"
#include <string.h>
#include <stdio.h>
#include <unistd.h>
#include <esp_log.h>
#include <esp_random.h>
#include <lwip/inet.h>
#include <lwip/sockets.h>
#define TAG "PipecatVoiceWs"
#define WS_HEADER_LIMIT 1024U
#define WS_CONTROL_LIMIT 125U
static int send_all(int fd, const uint8_t* data, size_t length) {
size_t sent = 0;
while (sent < length) {
int result = lwip_send(fd, data + sent, length - sent, 0);
if (result <= 0) return -1;
sent += (size_t)result;
}
return 0;
}
static int recv_all(int fd, uint8_t* data, size_t length) {
size_t received = 0;
while (received < length) {
int result = lwip_recv(fd, data + received, length - received, 0);
if (result <= 0) return -1;
received += (size_t)result;
}
return 0;
}
static int discard(int fd, uint64_t length) {
uint8_t buffer[256];
while (length > 0) {
size_t chunk = length > sizeof(buffer) ? sizeof(buffer) : (size_t)length;
if (recv_all(fd, buffer, chunk) < 0) return -1;
length -= chunk;
}
return 0;
}
static int send_frame(int fd, uint8_t opcode, const uint8_t* payload, size_t length) {
if (length > 65535U || ((opcode & 0x08U) && length > WS_CONTROL_LIMIT)) return -1;
uint8_t header[8];
size_t header_length = 2;
header[0] = 0x80U | opcode;
if (length < 126U) {
header[1] = 0x80U | (uint8_t)length;
} else {
header[1] = 0x80U | 126U;
header[2] = (uint8_t)(length >> 8U);
header[3] = (uint8_t)length;
header_length = 4;
}
uint8_t mask[4];
uint32_t random = esp_random();
memcpy(mask, &random, sizeof(mask));
memcpy(header + header_length, mask, sizeof(mask));
header_length += sizeof(mask);
if (send_all(fd, header, header_length) < 0) return -1;
uint8_t chunk[512];
size_t offset = 0;
while (offset < length) {
size_t count = length - offset > sizeof(chunk) ? sizeof(chunk) : length - offset;
for (size_t i = 0; i < count; ++i) chunk[i] = payload[offset + i] ^ mask[(offset + i) % sizeof(mask)];
if (send_all(fd, chunk, count) < 0) return -1;
offset += count;
}
return 0;
}
int ws_connect(const char* host, int port, const char* path, const char* device_id, const char* api_key) {
if (host == NULL || path == NULL || device_id == NULL || api_key == NULL || port < 1 || port > 65535) return -1;
int fd = lwip_socket(AF_INET, SOCK_STREAM, 0);
if (fd < 0) {
ESP_LOGW(TAG, "socket create failed");
return -1;
}
struct sockaddr_in address = {0};
address.sin_family = AF_INET;
address.sin_port = htons((uint16_t)port);
address.sin_addr.s_addr = ipaddr_addr(host);
if (address.sin_addr.s_addr == IPADDR_NONE) {
ESP_LOGW(TAG, "endpoint address parse failed");
close(fd);
return -1;
}
if (lwip_connect(fd, (struct sockaddr*)&address, sizeof(address)) < 0) {
ESP_LOGW(TAG, "TCP connect failed");
close(fd);
return -1;
}
struct timeval timeout = {.tv_sec = 15, .tv_usec = 0};
lwip_setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof(timeout));
char request[WS_HEADER_LIMIT];
int request_length = snprintf(request, sizeof(request),
"GET %s HTTP/1.1\r\nHost: %s:%d\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n"
"Sec-WebSocket-Key: MDEyMzQ1Njc4OWFiY2RlZg==\r\nSec-WebSocket-Version: 13\r\n"
"Authorization: Bearer %s\r\nX-Device-ID: %s\r\n\r\n",
path, host, port, api_key, device_id);
if (request_length < 0 || (size_t)request_length >= sizeof(request) || send_all(fd, (const uint8_t*)request, (size_t)request_length) < 0) {
ESP_LOGW(TAG, "WebSocket upgrade request failed");
close(fd);
return -1;
}
char response[WS_HEADER_LIMIT];
size_t length = 0;
while (length + 1 < sizeof(response)) {
if (recv_all(fd, (uint8_t*)&response[length], 1) < 0) {
ESP_LOGW(TAG, "WebSocket upgrade response failed");
close(fd);
return -1;
}
response[++length] = '\0';
if (length >= 4 && memcmp(response + length - 4, "\r\n\r\n", 4) == 0) break;
}
if (length + 1 >= sizeof(response) || strstr(response, " 101 ") == NULL) {
ESP_LOGW(TAG, "WebSocket upgrade rejected");
close(fd);
return -1;
}
ESP_LOGI(TAG, "WebSocket upgrade accepted");
return fd;
}
int ws_send(int fd, const uint8_t* data, size_t length, bool binary) {
if (fd < 0 || data == NULL || length == 0) return -1;
return send_frame(fd, binary ? 0x02U : 0x01U, data, length);
}
int ws_recv(int fd, int* opcode, bool* final, uint8_t* payload, size_t maximum) {
uint8_t header[2];
if (fd < 0 || recv_all(fd, header, sizeof(header)) < 0) return -1;
uint64_t length = header[1] & 0x7fU;
if (length == 126U) {
uint8_t extended[2];
if (recv_all(fd, extended, sizeof(extended)) < 0) return -1;
length = ((uint64_t)extended[0] << 8U) | extended[1];
} else if (length == 127U) {
uint8_t extended[8];
if (recv_all(fd, extended, sizeof(extended)) < 0) return -1;
length = 0;
for (size_t i = 0; i < sizeof(extended); ++i) length = (length << 8U) | extended[i];
}
bool masked = (header[1] & 0x80U) != 0;
uint8_t mask[4] = {0};
if (masked && recv_all(fd, mask, sizeof(mask)) < 0) return -1;
uint8_t frame_opcode = header[0] & 0x0fU;
if (((frame_opcode & 0x08U) && (length > WS_CONTROL_LIMIT || !(header[0] & 0x80U))) || length > maximum) {
if (discard(fd, length) < 0) return -1;
return -2;
}
if (length > 0 && recv_all(fd, payload, (size_t)length) < 0) return -1;
if (masked) for (size_t i = 0; i < (size_t)length; ++i) payload[i] ^= mask[i % sizeof(mask)];
if (opcode) *opcode = frame_opcode;
if (final) *final = (header[0] & 0x80U) != 0;
return (int)length;
}
int ws_send_pong(int fd, const uint8_t* payload, size_t length) { return send_frame(fd, 0x0aU, payload, length); }
int ws_send_close(int fd) { return send_frame(fd, 0x08U, NULL, 0); }
void ws_close(int fd) { if (fd >= 0) close(fd); }
-62
View File
@@ -1,62 +0,0 @@
#pragma once
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>
#ifdef __cplusplus
extern "C" {
#endif
/**
* Connect to a WebSocket server.
* @param host Server IP address (e.g. "192.168.68.126")
* @param port Port number (e.g. 8642)
* @param path WebSocket path (e.g. "/api/esp32/voice/ws")
* @param device_id Unique device identifier
* @param api_key Optional profile API key; never compiled into firmware
* @return Socket file descriptor on success, or -1 on failure
*/
int ws_connect(const char* host, int port, const char* path, const char* device_id, const char* api_key);
/**
* Send a WebSocket frame.
* @param fd Socket file descriptor
* @param data Data payload to send
* @param len Length of the data payload
* @param binary True for binary frame, false for text frame
* @return 0 on success, or -1 on failure
*/
int ws_send(int fd, const uint8_t* data, size_t len, bool binary);
/**
* Receive a WebSocket frame.
* @param fd Socket file descriptor
* @param out_opcode Pointer to store the received opcode (e.g. 0x01 text, 0x02 binary)
* @param payload Buffer to store the received payload
* @param max_len Maximum length of the payload buffer
* @return Received payload length on success, -1 on connection failure, or -2 on buffer overflow
*/
int ws_recv(int fd, int* out_opcode, bool* out_final, uint8_t* payload, size_t max_len);
/**
* Close a WebSocket connection.
* @param fd Socket file descriptor
*/
void ws_close(int fd);
/**
* Send a WebSocket PONG frame.
* @param fd Socket file descriptor
* @param payload Payload to reflect
* @param len Length of payload
* @return 0 on success, or -1 on failure
*/
int ws_send_pong(int fd, const uint8_t* payload, size_t len);
/** Send a clean WebSocket close control frame before closing the socket. */
int ws_send_close(int fd);
#ifdef __cplusplus
}
#endif
-7
View File
@@ -1,7 +0,0 @@
manifest.version=0.2
target.sdk=0.8.0-dev
target.platforms=esp32s3
app.id=one.tactility.pipecatvoice
app.version.name=1.0.0
app.version.code=1
app.name=Pipecat Voice
@@ -1,52 +0,0 @@
#include "voice_protocol.h"
#include <assert.h>
#include <stdio.h>
#include <string.h>
static void test_endpoint_validation(void) {
PvEndpoint endpoint;
assert(pv_parse_endpoint("ws://192.168.68.102:8644/api/esp32/voice/ws", &endpoint));
assert(strcmp(endpoint.host, "192.168.68.102") == 0);
assert(endpoint.port == 8644);
assert(strcmp(endpoint.path, "/api/esp32/voice/ws") == 0);
assert(!pv_parse_endpoint("ws://127.0.0.1:8642/api", &endpoint));
assert(!pv_parse_endpoint("ws://localhost:8642/api", &endpoint));
assert(!pv_parse_endpoint("wss://192.168.68.102/api", &endpoint));
assert(!pv_parse_endpoint("ws://192.168.68.102:0/api", &endpoint));
}
static void test_start_and_pcm_boundaries(void) {
char json[256];
assert(pv_make_start_json(json, sizeof(json), "session-01", "tactility-14c19d1a790"));
assert(strstr(json, "\"v\":1") != NULL);
assert(strstr(json, "\"pcm_s16le\"") != NULL);
assert(!pv_make_start_json(json, sizeof(json), "bad session", "device"));
assert(pv_valid_pcm_chunk(2));
assert(pv_valid_pcm_chunk(PV_PCM_CHUNK_MAX));
assert(!pv_valid_pcm_chunk(0));
assert(!pv_valid_pcm_chunk(3));
assert(!pv_valid_pcm_chunk(PV_PCM_CHUNK_MAX + 2));
assert(pv_valid_downstream_audio("pcm_s16le", 24000, 1, 2, 48000));
assert(!pv_valid_downstream_audio("wav", 24000, 1, 2, 48000));
assert(!pv_valid_downstream_audio("pcm_s16le", 24000, 2, 2, 48000));
assert(!pv_valid_downstream_audio("pcm_s16le", 24000, 1, 2, PV_DOWNSTREAM_MAX + 2));
assert(pv_binary_matches_metadata(48000, 48000));
assert(!pv_binary_matches_metadata(48000, 47998));
}
static void test_retry_and_state(void) {
const uint32_t expected[] = {1, 2, 4, 8, 16, 30, 30};
for (unsigned i = 0; i < sizeof(expected) / sizeof(expected[0]); ++i) assert(pv_retry_delay_seconds(i) == expected[i]);
assert(pv_disconnect_state(true) == PV_RECONNECTING);
assert(pv_disconnect_state(false) == PV_FAILED);
assert(strcmp(pv_state_label(PV_STREAMING), "STREAMING") == 0);
}
int main(void) {
test_endpoint_validation();
test_start_and_pcm_boundaries();
test_retry_and_state();
puts("voice_protocol tests passed");
return 0;
}
+77 -217
View File
@@ -1,32 +1,24 @@
#include <tt_app.h>
#include <tt_lvgl.h>
#include <tt_lvgl_toolbar.h>
#include <tt_wifi.h>
#include <tactility/device.h>
#include <tactility/drivers/audio_stream.h>
#include <tactility/drivers/i2s_controller.h>
#include "websocket.h"
#include <cJSON.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <errno.h>
/* Exported by Tactility firmware 0.8.0-dev; absent from the CDN SDK header. */
struct Device* device_find_by_name(const char* name);
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_random.h"
#define TAG "ReynaBot"
#define AUDIO_SAMPLE_RATE 16000U
#define AUDIO_BITS_PER_SAMPLE 16U
#define MAX_PCM_CHUNK_BYTES 16384U
#define MAX_RESPONSE_AUDIO_BYTES 65536U
typedef enum {
STATE_IDLE,
@@ -72,19 +64,10 @@ typedef struct {
TaskHandle_t worker_task;
TaskHandle_t rx_task;
// Hardware/audio service
struct Device* audio_stream_dev;
AudioStreamHandle input_handle;
AudioStreamHandle output_handle;
uint32_t output_sample_rate;
uint8_t output_channels;
uint8_t output_bits_per_sample;
bool output_stream_pending;
size_t expected_audio_bytes;
uint32_t expected_audio_rate;
uint8_t expected_audio_channels;
uint8_t expected_audio_bits;
bool stop_sent;
// Hardware
struct Device* i2s_dev;
// WebSocket
int ws_fd;
bool ws_connected;
bool ws_done;
@@ -98,77 +81,6 @@ static void reynabot_rx_task(void* arg);
static void update_ui(ReynaBotCtx* ctx);
static void load_config(ReynaBotCtx* ctx);
static bool find_audio_stream_device(ReynaBotCtx* ctx) {
struct Device* dev = device_find_by_name("audio-stream");
if (dev != NULL) {
ctx->audio_stream_dev = dev;
return true;
}
// The 0.8.0-dev firmware registers this shared service by name. The
// CDN SDK headers do not expose the deprecated type-search helper, so do
// not reference it from a dynamically linked app.
return false;
}
static void close_input_stream(ReynaBotCtx* ctx) {
if (ctx->input_handle != NULL) {
audio_stream_close(ctx->input_handle);
ctx->input_handle = NULL;
}
}
static void close_output_stream(ReynaBotCtx* ctx) {
if (ctx->output_handle != NULL) {
audio_stream_close(ctx->output_handle);
ctx->output_handle = NULL;
}
ctx->output_stream_pending = false;
}
static bool open_input_stream(ReynaBotCtx* ctx) {
if (ctx->audio_stream_dev == NULL) return false;
close_input_stream(ctx);
struct AudioStreamConfig config = {
.sample_rate = 16000,
.bits_per_sample = 16,
.channels = 1,
};
error_t err = audio_stream_open_input(ctx->audio_stream_dev, &config, &ctx->input_handle);
if (err != ERROR_NONE) {
ctx->input_handle = NULL;
ESP_LOGE(TAG, "audio_stream_open_input failed: %d", err);
return false;
}
audio_stream_set_mute(ctx->audio_stream_dev, AUDIO_CODEC_DIR_INPUT, false);
audio_stream_set_volume(ctx->audio_stream_dev, AUDIO_CODEC_DIR_INPUT, 100.0f);
ESP_LOGI(TAG, "Microphone opened via audio-stream at 16000 Hz mono");
return true;
}
static bool open_output_stream(ReynaBotCtx* ctx, uint32_t sample_rate, uint8_t channels, uint8_t bits_per_sample) {
if (ctx->audio_stream_dev == NULL || sample_rate == 0 || channels == 0 || bits_per_sample != 16) return false;
close_output_stream(ctx);
struct AudioStreamConfig config = {
.sample_rate = sample_rate,
.bits_per_sample = bits_per_sample,
.channels = channels,
};
error_t err = audio_stream_open_output(ctx->audio_stream_dev, &config, &ctx->output_handle);
if (err != ERROR_NONE) {
ctx->output_handle = NULL;
ESP_LOGE(TAG, "audio_stream_open_output failed: %d rate=%u ch=%u", err, (unsigned)sample_rate, channels);
return false;
}
audio_stream_set_mute(ctx->audio_stream_dev, AUDIO_CODEC_DIR_OUTPUT, false);
audio_stream_set_volume(ctx->audio_stream_dev, AUDIO_CODEC_DIR_OUTPUT, 100.0f);
ctx->output_sample_rate = sample_rate;
ctx->output_channels = channels;
ctx->output_bits_per_sample = bits_per_sample;
ctx->output_stream_pending = true;
ESP_LOGI(TAG, "Speaker opened via audio-stream at %u Hz mono", (unsigned)sample_rate);
return true;
}
/* ─── Helper for URL parsing ─── */
static bool parse_ws_url(const char* url, char* host, int* port, char* path) {
if (strncmp(url, "ws://", 5) != 0) return false;
@@ -199,9 +111,9 @@ static bool parse_ws_url(const char* url, char* host, int* port, char* path) {
/* ─── Config Loading/Saving ─── */
static void load_config(ReynaBotCtx* ctx) {
// Default fallback config
snprintf(ctx->server_url, sizeof(ctx->server_url), "ws://192.168.68.112:8643/api/esp32/voice/ws");
snprintf(ctx->server_url, sizeof(ctx->server_url), "ws://192.168.68.126:8643/api/esp32/voice/ws");
snprintf(ctx->device_id, sizeof(ctx->device_id), "reynabot_screen");
ctx->api_key[0] = '\0';
snprintf(ctx->api_key, sizeof(ctx->api_key), "hmek_sXB7921bZ9FXTVKARExqUZ7ttBxtEoURHRU0JCB-gNY");
char path[256];
size_t path_size = sizeof(path);
@@ -218,7 +130,7 @@ static void load_config(ReynaBotCtx* ctx) {
// Write default config file
file = fopen(path, "w");
if (file != NULL) {
fprintf(file, "{\n \"server_url\": \"ws://192.168.68.112:8643/api/esp32/voice/ws\",\n \"device_id\": \"reynabot_screen\",\n \"api_key\": \"\"\n}\n");
fprintf(file, "{\n \"server_url\": \"ws://192.168.68.126:8643/api/esp32/voice/ws\",\n \"device_id\": \"reynabot_screen\",\n \"api_key\": \"mcT1YA1vOr9wXSiHpCYalweEGGZKX-PIfZv2drp8BSg\"\n}\n");
fclose(file);
}
ESP_LOGI(TAG, "Created default config.json at %s", path);
@@ -269,21 +181,6 @@ static void parse_json_message(ReynaBotCtx* ctx, const char* json_str) {
if (strcmp(evt, "ready") == 0) {
ESP_LOGI(TAG, "Server ready");
} else if (strcmp(evt, "state") == 0) {
cJSON* state_item = cJSON_GetObjectItem(json, "state");
if (state_item != NULL && cJSON_IsString(state_item)) {
if (strcmp(state_item->valuestring, "listening") == 0) {
if (ctx->stop_sent) {
ctx->ws_done = true;
} else {
ctx->state = STATE_LISTENING;
}
ctx->ui_update_pending = true;
} else if (strcmp(state_item->valuestring, "processing") == 0) {
ctx->state = STATE_THINKING;
ctx->ui_update_pending = true;
}
}
} else if (strcmp(evt, "listening") == 0) {
ctx->state = STATE_LISTENING;
ctx->ui_update_pending = true;
@@ -302,42 +199,11 @@ static void parse_json_message(ReynaBotCtx* ctx, const char* json_str) {
strncpy(ctx->last_response, txt_item->valuestring, sizeof(ctx->last_response) - 1);
ctx->ui_update_pending = true;
}
} else if (strcmp(evt, "audio") == 0) {
cJSON* format_item = cJSON_GetObjectItem(json, "format");
cJSON* rate_item = cJSON_GetObjectItem(json, "sample_rate");
cJSON* channels_item = cJSON_GetObjectItem(json, "channels");
cJSON* width_item = cJSON_GetObjectItem(json, "sample_width");
cJSON* length_item = cJSON_GetObjectItem(json, "byte_length");
bool valid = format_item != NULL && cJSON_IsString(format_item) &&
strcmp(format_item->valuestring, "pcm_s16le") == 0 &&
rate_item != NULL && cJSON_IsNumber(rate_item) && rate_item->valueint > 0 && rate_item->valueint <= 48000 &&
channels_item != NULL && cJSON_IsNumber(channels_item) && channels_item->valueint == 1 &&
width_item != NULL && cJSON_IsNumber(width_item) && width_item->valueint == 2 &&
length_item != NULL && cJSON_IsNumber(length_item) && length_item->valueint > 0 &&
length_item->valueint <= MAX_RESPONSE_AUDIO_BYTES && (length_item->valueint % 2) == 0;
if (valid) {
ctx->expected_audio_rate = (uint32_t)rate_item->valueint;
ctx->expected_audio_channels = (uint8_t)channels_item->valueint;
ctx->expected_audio_bits = (uint8_t)width_item->valueint;
ctx->expected_audio_bytes = (size_t)length_item->valueint;
if (!open_output_stream(ctx, ctx->expected_audio_rate, ctx->expected_audio_channels, ctx->expected_audio_bits)) {
snprintf(ctx->error_message, sizeof(ctx->error_message), "Audio output unavailable");
ctx->state = STATE_ERROR;
ctx->expected_audio_bytes = 0;
} else {
ctx->state = STATE_SPEAKING;
}
ctx->ui_update_pending = true;
} else {
snprintf(ctx->error_message, sizeof(ctx->error_message), "Invalid audio metadata");
ctx->state = STATE_ERROR;
ctx->expected_audio_bytes = 0;
ctx->ui_update_pending = true;
}
} else if (strcmp(evt, "audio_start") == 0) {
/* Legacy event: metadata must still arrive as `audio` before binary PCM. */
ctx->state = STATE_SPEAKING;
ctx->ui_update_pending = true;
// I2S is already configured globally at session startup
} else if (strcmp(evt, "audio_end") == 0) {
ESP_LOGI(TAG, "Audio response ended");
} else if (strcmp(evt, "done") == 0) {
@@ -359,7 +225,7 @@ static void parse_json_message(ReynaBotCtx* ctx, const char* json_str) {
/* ─── WebSocket RX (Receive) Task ─── */
static void reynabot_rx_task(void* arg) {
ReynaBotCtx* ctx = (ReynaBotCtx*)arg;
uint8_t* rx_buf = malloc(MAX_RESPONSE_AUDIO_BYTES + 1U);
uint8_t* rx_buf = malloc(8192);
if (rx_buf == NULL) {
ESP_LOGE(TAG, "Failed to allocate RX buffer");
ctx->ws_done = true;
@@ -372,7 +238,7 @@ static void reynabot_rx_task(void* arg) {
ESP_LOGI(TAG, "WS Receive task started");
while (ctx->ws_fd >= 0) {
int r = ws_recv(ctx->ws_fd, &opcode, rx_buf, MAX_RESPONSE_AUDIO_BYTES);
int r = ws_recv(ctx->ws_fd, &opcode, rx_buf, 8191);
if (r < 0) {
if (r == -1) {
ESP_LOGI(TAG, "WS connection closed or read error, errno=%d", errno);
@@ -386,31 +252,21 @@ static void reynabot_rx_task(void* arg) {
if (opcode == 0x01) { // Text frame (JSON)
rx_buf[r] = '\0';
parse_json_message(ctx, (char*)rx_buf);
} else if (opcode == 0x02) { // Audio response PCM
if (ctx->output_handle == NULL || ctx->expected_audio_bytes == 0 || (size_t)r != ctx->expected_audio_bytes) {
ESP_LOGE(TAG, "Audio frame does not match metadata: got=%d expected=%u", r, (unsigned)ctx->expected_audio_bytes);
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Invalid audio frame");
ctx->ui_update_pending = true;
} else {
size_t written_total = 0;
while (written_total < (size_t)r) {
size_t written = 0;
error_t err = audio_stream_write(ctx->output_handle, rx_buf + written_total,
(size_t)r - written_total, &written, pdMS_TO_TICKS(3000));
if (err != ERROR_NONE || written == 0) {
ESP_LOGE(TAG, "audio_stream_write failed: %d written=%u", err, (unsigned)written);
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Audio playback failed");
ctx->ui_update_pending = true;
break;
}
written_total += written;
} else if (opcode == 0x02) { // Binary frame (Audio data)
if (ctx->state == STATE_SPEAKING && ctx->i2s_dev != NULL) {
const uint8_t* payload_ptr = rx_buf;
size_t payload_len = r;
// Skip WAV header if present in the first chunk
if (payload_len > 44 && memcmp(payload_ptr, "RIFF", 4) == 0) {
payload_ptr += 44;
payload_len -= 44;
}
close_output_stream(ctx);
ctx->expected_audio_bytes = 0;
ctx->state = STATE_THINKING;
ctx->ui_update_pending = true;
size_t written = 0;
device_lock(ctx->i2s_dev);
i2s_controller_write(ctx->i2s_dev, payload_ptr, payload_len, &written, pdMS_TO_TICKS(100));
device_unlock(ctx->i2s_dev);
}
} else if (opcode == 0x09) { // PING frame
ESP_LOGI(TAG, "WS PING received, sending PONG");
@@ -552,11 +408,8 @@ static void reynabot_task(void* arg) {
ctx->start_session = false;
ctx->stop_session = false;
ctx->cancel_session = false;
ctx->stop_sent = false;
ctx->expected_audio_bytes = 0;
close_input_stream(ctx);
close_output_stream(ctx);
ctx->state = STATE_CONNECTING;
ctx->last_transcript[0] = '\0';
ctx->last_response[0] = '\0';
ctx->error_message[0] = '\0';
@@ -581,18 +434,29 @@ static void reynabot_task(void* arg) {
ctx->ws_fd = fd;
ctx->ws_connected = true;
// Spawn background RX task to read and parse events.
// Configure I2S on demand for this session (16 kHz, 16-bit, mono)
struct I2sConfig session_cfg = {
.communication_format = I2S_FORMAT_STAND_I2S,
.sample_rate = 16000,
.bits_per_sample = 16,
.channel_left = 0,
.channel_right = I2S_CHANNEL_NONE
};
if (ctx->i2s_dev != NULL) {
device_lock(ctx->i2s_dev);
i2s_controller_set_config(ctx->i2s_dev, &session_cfg);
device_unlock(ctx->i2s_dev);
}
// Spawn background RX task to read and parse events
xTaskCreate(reynabot_rx_task, "reynabot_rx", 4096, ctx, 6, &ctx->rx_task);
// The Kids LAN adapter requires protocol v1 and a fresh session id.
char session_id[80];
snprintf(session_id, sizeof(session_id), "reynabot-%08x%08x",
(unsigned)esp_random(), (unsigned)esp_random());
char start_json[384];
// Send start event handshake
char start_json[256];
snprintf(start_json, sizeof(start_json),
"{\"v\":1,\"event\":\"start\",\"session_id\":\"%s\",\"device_id\":\"%s\",\"audio\":{\"format\":\"pcm_s16le\",\"sample_rate\":16000,\"channels\":1,\"sample_width\":2}}",
session_id, ctx->device_id);
"{\"event\":\"start\",\"device_id\":\"%s\",\"sample_rate\":16000,\"channels\":1,\"sample_width\":2,\"format\":\"pcm_s16le\"}",
ctx->device_id);
if (ws_send(ctx->ws_fd, (const uint8_t*)start_json, strlen(start_json), false) < 0) {
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Handshake send failed");
@@ -622,20 +486,11 @@ static void reynabot_task(void* arg) {
continue;
}
if (!open_input_stream(ctx)) {
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Microphone unavailable");
ws_close(ctx->ws_fd);
ctx->ws_fd = -1;
ctx->ws_connected = false;
update_ui(ctx);
continue;
}
uint8_t* buffer = malloc(MAX_PCM_CHUNK_BYTES);
// I2S is already configured globally at session startup
uint8_t* buffer = malloc(1024);
if (buffer == NULL) {
ESP_LOGE(TAG, "Failed to allocate record buffer");
close_input_stream(ctx);
ctx->state = STATE_ERROR;
snprintf(ctx->error_message, sizeof(ctx->error_message), "Out of memory");
ws_close(ctx->ws_fd);
@@ -645,13 +500,12 @@ static void reynabot_task(void* arg) {
continue;
}
size_t total_sent_bytes = 0;
// Stream 16 kHz mono PCM while PTT is held.
// Stream audio loop while PTT is held
while (ctx->is_pressed && !ctx->stop_session && !ctx->cancel_session && !ctx->ws_done && total_sent_bytes < 320000) {
size_t bytes_read = 0;
error_t r = audio_stream_read(ctx->input_handle, buffer, MAX_PCM_CHUNK_BYTES,
&bytes_read, pdMS_TO_TICKS(200));
if (r == ERROR_NONE && bytes_read > 0 && bytes_read <= MAX_PCM_CHUNK_BYTES && (bytes_read % 2U) == 0) {
error_t r = i2s_controller_read(ctx->i2s_dev, buffer, 1024, &bytes_read, pdMS_TO_TICKS(100));
if (r == ERROR_NONE && bytes_read > 0) {
if (ws_send(ctx->ws_fd, buffer, bytes_read, true) < 0) {
ESP_LOGE(TAG, "Audio stream send failed");
break;
@@ -659,20 +513,17 @@ static void reynabot_task(void* arg) {
total_sent_bytes += bytes_read;
}
}
close_input_stream(ctx);
free(buffer);
if (ctx->cancel_session || total_sent_bytes < 3200) {
ESP_LOGI(TAG, "Cancelling audio session");
const char* cancel_json = "{\"event\":\"cancel\"}";
ws_send(ctx->ws_fd, (const uint8_t*)cancel_json, strlen(cancel_json), false);
ctx->stop_sent = true;
ctx->state = STATE_IDLE;
} else {
const char* stop_json = "{\"event\":\"stop\"}";
ws_send(ctx->ws_fd, (const uint8_t*)stop_json, strlen(stop_json), false);
ctx->stop_sent = true;
ctx->state = STATE_THINKING;
update_ui(ctx);
@@ -699,9 +550,13 @@ static void reynabot_task(void* arg) {
ctx->ws_connected = false;
ws_close(fd_to_close);
close_input_stream(ctx);
close_output_stream(ctx);
// Reset I2S controller to release DMA and stop white noise
if (ctx->i2s_dev != NULL) {
device_lock(ctx->i2s_dev);
i2s_controller_reset(ctx->i2s_dev);
device_unlock(ctx->i2s_dev);
}
// Wait for receive task to exit
int rx_timeout = 100;
while (ctx->rx_task != NULL && rx_timeout > 0) {
@@ -756,9 +611,10 @@ static void on_show(AppHandle app, void* data, lv_obj_t* parent) {
load_config(ctx);
// Find the firmware Audio System service; it owns codec/native-rate conversion.
if (!find_audio_stream_device(ctx)) {
ESP_LOGE(TAG, "audio-stream device not found!");
// Find I2S controller
ctx->i2s_dev = device_find_by_name("i2s0");
if (ctx->i2s_dev == NULL) {
ESP_LOGE(TAG, "I2S controller 'i2s0' not found!");
}
// Style the parent screen
@@ -944,9 +800,13 @@ static void on_hide(AppHandle app, void* data) {
ws_close(fd_to_close);
}
close_input_stream(ctx);
close_output_stream(ctx);
// Reset I2S controller to stop DMA and looping noise
if (ctx->i2s_dev != NULL) {
device_lock(ctx->i2s_dev);
i2s_controller_reset(ctx->i2s_dev);
device_unlock(ctx->i2s_dev);
}
// Wait briefly for tasks to exit
int timeout = 100;
while ((ctx->worker_task != NULL || ctx->rx_task != NULL) && timeout > 0) {
+2 -2
View File
@@ -56,8 +56,8 @@ int ws_connect(const char* host, int port, const char* path, const char* device_
"Connection: Upgrade\r\n"
"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n"
"Sec-WebSocket-Version: 13\r\n"
"Authorization: Bearer %s\\r\\n"
"X-Device-ID: %s\\r\\n"
"Authorization: Bearer %s\r\n"
"X-Device-ID: %s\r\n"
"\r\n",
path, host, port, auth_key, device_id);
+2 -2
View File
@@ -10,8 +10,8 @@ extern "C" {
/**
* Connect to a WebSocket server.
* @param host Server IP address (for example, the Mac LAN host 192.168.68.112)
* @param port Port number (e.g. 8643 for the Kids gateway)
* @param host Server IP address (e.g. "192.168.68.126")
* @param port Port number (e.g. 8642)
* @param path WebSocket path (e.g. "/api/esp32/voice/ws")
* @param device_id Unique device identifier
* @param auth_key Hermes Bearer API key
@@ -1,16 +1,12 @@
cmake_minimum_required(VERSION 3.20)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
if (DEFINED ENV{TACTILITY_SDK_PATH})
set(TACTILITY_SDK_PATH $ENV{TACTILITY_SDK_PATH})
else()
set(TACTILITY_SDK_PATH "../../release/TactilitySDK")
message(WARNING "TACTILITY_SDK_PATH is not set, defaulting to ${TACTILITY_SDK_PATH}")
message(WARNING "TACTILITY_SDK_PATH not set, defaulting to ${TACTILITY_SDK_PATH}")
endif()
include("${TACTILITY_SDK_PATH}/TactilitySDK.cmake")
set(EXTRA_COMPONENT_DIRS ${TACTILITY_SDK_PATH})
project(LiveCaptions)
tactility_project(LiveCaptions)
project(SdDownloader)
tactility_project(SdDownloader)
+6
View File
@@ -0,0 +1,6 @@
file(GLOB_RECURSE SOURCE_FILES Source/*.c)
idf_component_register(
SRCS ${SOURCE_FILES}
INCLUDE_DIRS Source
REQUIRES TactilitySDK esp_http_client
)
+460
View File
@@ -0,0 +1,460 @@
#include <tt_app.h>
#include <tt_lvgl.h>
#include <tt_bundle.h>
#include <tactility/lvgl_fonts.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <unistd.h>
#include <errno.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_http_client.h"
#include "esp_heap_caps.h"
#define TAG "SDDL"
#define DEFAULT_URL "http://192.168.68.110:8098/463_Discovering_Discovery_Mountain_S01E01/discovery_mountain_s01e01_esp32_24k.mp3"
#define DEFAULT_PATH "/sdcard/download/test2.mp3"
typedef struct {
lv_obj_t *bar;
lv_obj_t *lbl_url;
lv_obj_t *lbl_path;
lv_obj_t *lbl_pct;
lv_obj_t *lbl_detail;
lv_obj_t *lbl_status;
lv_obj_t *btn_cancel;
char url[512];
char path[512];
bool override_existing;
AppHandle app_handle;
TaskHandle_t dl_handle;
volatile bool downloading;
volatile bool cancel_req;
volatile int total;
volatile int expected;
int last_pct;
bool result_success;
} AppCtx;
static AppCtx G;
static void ensure_parent_dir(const char *filepath) {
if (!filepath) return;
char path_copy[512];
strncpy(path_copy, filepath, sizeof(path_copy)-1);
path_copy[sizeof(path_copy)-1]=0;
char *last_slash = strrchr(path_copy, '/');
if (!last_slash) return;
*last_slash = 0;
if (strlen(path_copy)==0) return;
char cur[512] = "";
if (path_copy[0]=='/') strcpy(cur, "/");
char *start = path_copy[0]=='/' ? path_copy+1 : path_copy;
int len = strlen(start);
int i = 0;
while (i <= len) {
int j = i;
while (j < len && start[j] != '/') j++;
if (j > i) {
size_t cur_len = strlen(cur);
if (cur_len > 0 && cur[cur_len-1] != '/') {
if (cur_len + 1 < sizeof(cur)) {
cur[cur_len] = '/';
cur[cur_len+1] = 0;
}
}
char saved = start[j];
start[j] = 0;
strncat(cur, start+i, sizeof(cur)-strlen(cur)-1);
start[j] = saved;
mkdir(cur, 0777);
}
i = j+1;
}
}
static void update_ui_status(const char *txt) {
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) lv_label_set_text(G.lbl_status, txt);
tt_lvgl_unlock();
}
static void set_result_and_close(bool success) {
G.result_success = success;
size_t free_internal = heap_caps_get_free_size(MALLOC_CAP_INTERNAL);
BundleHandle result_bundle = NULL;
if (free_internal > 8000) {
result_bundle = tt_bundle_alloc();
}
if (result_bundle) {
tt_bundle_put_bool(result_bundle, "success", success);
// Keep bundle minimal to avoid low heap crash (previous put_string path 512B caused OOM)
tt_app_set_result(G.app_handle, success ? APP_RESULT_OK : APP_RESULT_ERROR, result_bundle);
} else {
// Low heap – return OK without bundle (DM will still refresh via has_slug)
if (free_internal <= 8000) {
ESP_LOGW(TAG, "Low heap %d, returning result without bundle", free_internal);
}
tt_app_set_result(G.app_handle, success ? APP_RESULT_OK : APP_RESULT_ERROR, NULL);
}
// Change Cancel button to Close once done (manual close, auto-close unreliable)
tt_lvgl_lock(portMAX_DELAY);
if (G.btn_cancel) {
lv_obj_t* lbl = lv_obj_get_child(G.btn_cancel, 0);
if (lbl) lv_label_set_text(lbl, "Close");
}
tt_lvgl_unlock();
}
static bool download_one_file(const char *url, const char *path) {
ESP_LOGI(TAG, "DL one file url=%s path=%s override=%d", url, path, G.override_existing);
struct stat st;
if (stat(path, &st)==0) {
if (!G.override_existing) {
ESP_LOGW(TAG, "File exists and override=false: %s (returning true as cached)", path);
G.total = st.st_size;
update_ui_status("Already on SD");
return true;
} else {
ESP_LOGI(TAG, "File exists but override=true, overwriting %s", path);
unlink(path);
}
}
ensure_parent_dir(path);
size_t free_internal = heap_caps_get_free_size(MALLOC_CAP_INTERNAL);
ESP_LOGI(TAG, "Heap before DL: internal=%d", free_internal);
if (free_internal < 10000) {
vTaskDelay(pdMS_TO_TICKS(1000));
}
if (strlen(url) < 8 || strncmp(url, "http://", 7)!=0) {
update_ui_status("Invalid URL");
return false;
}
char tmp_path[520];
snprintf(tmp_path, sizeof(tmp_path), "%s.tmp", path);
unlink(tmp_path);
FILE *f = fopen(tmp_path, "wb");
if (!f) {
ESP_LOGE(TAG, "fopen fail %s errno=%d", tmp_path, errno);
update_ui_status("Failed to open file");
return false;
}
esp_http_client_config_t cfg = {
.url = url,
.timeout_ms = 15000,
.buffer_size = 4096,
.buffer_size_tx = 1024,
.keep_alive_enable = false,
};
esp_http_client_handle_t client = esp_http_client_init(&cfg);
if (!client) {
fclose(f);
unlink(tmp_path);
update_ui_status("HTTP init failed");
return false;
}
esp_err_t err = esp_http_client_open(client, 0);
if (err != ESP_OK) {
esp_http_client_cleanup(client);
fclose(f);
unlink(tmp_path);
update_ui_status("HTTP open failed");
return false;
}
int content_len = esp_http_client_fetch_headers(client);
int status = esp_http_client_get_status_code(client);
ESP_LOGI(TAG, "HTTP status %d, content_len %d", status, content_len);
if (status != 200) {
char msg[64];
snprintf(msg, sizeof(msg), "HTTP %d", status);
update_ui_status(msg);
esp_http_client_close(client);
esp_http_client_cleanup(client);
fclose(f);
unlink(tmp_path);
return false;
}
G.expected = content_len;
uint8_t *buf = heap_caps_malloc(4096, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if (!buf) {
buf = malloc(4096);
if (!buf) {
esp_http_client_close(client);
esp_http_client_cleanup(client);
fclose(f);
unlink(tmp_path);
update_ui_status("No memory");
return false;
}
}
int total = 0;
int timeouts = 0;
bool ok = true;
while (1) {
if (G.cancel_req) { ok = false; break; }
if (content_len >=0 && total >= content_len) break;
int r = esp_http_client_read(client, (char*)buf, 4096);
if (r < 0) {
timeouts++;
if (timeouts >= 5) { ok = false; break; }
vTaskDelay(pdMS_TO_TICKS(200));
continue;
}
if (r == 0) break;
timeouts = 0;
if (fwrite(buf, 1, r, f) != (size_t)r) { ok = false; break; }
total += r;
G.total = total;
int pct = 0;
if (content_len > 0) {
pct = (int)((int64_t)total * 100 / content_len);
if (pct<0) pct=0;
if (pct>100) pct=100;
}
if (pct != G.last_pct) {
G.last_pct = pct;
tt_lvgl_lock(portMAX_DELAY);
if (G.bar) lv_bar_set_value(G.bar, pct, LV_ANIM_OFF);
if (G.lbl_detail) {
char d[64];
if (content_len>0) snprintf(d,sizeof(d),"%d / %d KB", total/1024, content_len/1024);
else snprintf(d,sizeof(d),"%d KB", total/1024);
lv_label_set_text(G.lbl_detail, d);
}
if (G.lbl_pct) {
char b[32];
if (content_len>0) snprintf(b,sizeof(b),"%d%%",pct);
else snprintf(b,sizeof(b),"%d KB", total/1024);
lv_label_set_text(G.lbl_pct, b);
}
tt_lvgl_unlock();
}
if ((total / 4096) % 8 == 0) {
fflush(f);
vTaskDelay(pdMS_TO_TICKS(10));
}
}
heap_caps_free(buf);
esp_http_client_close(client);
esp_http_client_cleanup(client);
fclose(f);
if (G.cancel_req) { unlink(tmp_path); update_ui_status("Canceled"); return false; }
if (!ok || total < 512) { unlink(tmp_path); update_ui_status("Failed – error"); return false; }
if (content_len >=0 && total < content_len * 95 / 100) { unlink(tmp_path); update_ui_status("Failed – incomplete"); return false; }
if (rename(tmp_path, path) != 0) { unlink(tmp_path); update_ui_status("Rename failed"); return false; }
return true;
}
static void download_task(void *arg) {
(void)arg;
G.downloading = true;
G.cancel_req = false;
G.last_pct = -1;
char url[512];
char path[512];
strncpy(url, G.url, sizeof(url)-1);
strncpy(path, G.path, sizeof(path)-1);
url[sizeof(url)-1]=0;
path[sizeof(path)-1]=0;
bool ok = download_one_file(url, path);
if (ok) {
char msg[64];
snprintf(msg, sizeof(msg), "Done! %d KB", G.total/1024);
update_ui_status(msg);
}
G.downloading = false;
G.dl_handle = NULL;
set_result_and_close(ok);
vTaskDelete(NULL);
}
static void btn_cancel_cb(lv_event_t *e) {
(void)e;
if (!G.downloading) {
tt_app_stop();
return;
}
G.cancel_req = true;
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) lv_label_set_text(G.lbl_status, "Canceling...");
tt_lvgl_unlock();
}
static void build_ui(void) {
tt_lvgl_lock(portMAX_DELAY);
lv_obj_t *root = lv_obj_create(lv_scr_act());
lv_obj_set_size(root, LV_PCT(100), LV_PCT(100));
lv_obj_set_style_pad_all(root, 8, 0);
lv_obj_set_style_bg_color(root, lv_color_hex(0x111111), 0);
lv_obj_set_style_border_width(root, 0, 0);
lv_obj_set_flex_flow(root, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(root, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
lv_obj_set_style_pad_row(root, 6, 0);
lv_obj_t *title = lv_label_create(root);
lv_label_set_text(title, "Downloading...");
lv_obj_set_style_text_font(title, lvgl_get_text_font(FONT_SIZE_LARGE), 0);
lv_obj_set_style_text_color(title, lv_color_hex(0xFFFFFF), 0);
G.lbl_url = lv_label_create(root);
lv_label_set_text(G.lbl_url, G.url);
lv_obj_set_width(G.lbl_url, LV_PCT(100));
lv_label_set_long_mode(G.lbl_url, LV_LABEL_LONG_WRAP);
lv_obj_set_style_text_font(G.lbl_url, lvgl_get_text_font(FONT_SIZE_SMALL), 0);
lv_obj_set_style_text_color(G.lbl_url, lv_color_hex(0x888888), 0);
G.lbl_path = lv_label_create(root);
lv_label_set_text(G.lbl_path, G.path);
lv_obj_set_width(G.lbl_path, LV_PCT(100));
lv_label_set_long_mode(G.lbl_path, LV_LABEL_LONG_WRAP);
lv_obj_set_style_text_font(G.lbl_path, lvgl_get_text_font(FONT_SIZE_SMALL), 0);
lv_obj_set_style_text_color(G.lbl_path, lv_color_hex(0xAAAAAA), 0);
G.bar = lv_bar_create(root);
lv_obj_set_size(G.bar, LV_PCT(100), 14);
lv_bar_set_range(G.bar, 0, 100);
lv_bar_set_value(G.bar, 0, LV_ANIM_OFF);
lv_obj_set_style_bg_color(G.bar, lv_color_hex(0x333333), LV_PART_MAIN);
lv_obj_set_style_bg_color(G.bar, lv_color_hex(0x2E7D32), LV_PART_INDICATOR);
lv_obj_t *row = lv_obj_create(root);
lv_obj_set_size(row, LV_PCT(100), LV_SIZE_CONTENT);
lv_obj_set_style_bg_opa(row, LV_OPA_TRANSP, 0);
lv_obj_set_style_border_width(row, 0, 0);
lv_obj_set_style_pad_all(row, 0, 0);
lv_obj_set_flex_flow(row, LV_FLEX_FLOW_ROW);
lv_obj_set_flex_align(row, LV_FLEX_ALIGN_SPACE_BETWEEN, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
G.lbl_pct = lv_label_create(row);
lv_label_set_text(G.lbl_pct, "0%");
lv_obj_set_style_text_font(G.lbl_pct, lvgl_get_text_font(FONT_SIZE_DEFAULT), 0);
G.lbl_detail = lv_label_create(row);
lv_label_set_text(G.lbl_detail, "-");
lv_obj_set_style_text_font(G.lbl_detail, lvgl_get_text_font(FONT_SIZE_SMALL), 0);
lv_obj_set_style_text_color(G.lbl_detail, lv_color_hex(0x888888), 0);
G.lbl_status = lv_label_create(root);
lv_label_set_text(G.lbl_status, "Starting...");
lv_obj_set_width(G.lbl_status, LV_PCT(100));
lv_label_set_long_mode(G.lbl_status, LV_LABEL_LONG_WRAP);
lv_obj_set_style_text_color(G.lbl_status, lv_color_hex(0xCCCCCC), 0);
G.btn_cancel = lv_button_create(root);
lv_obj_set_size(G.btn_cancel, 100, 40);
lv_obj_set_style_bg_color(G.btn_cancel, lv_color_hex(0x444444), 0);
lv_obj_t *lbl = lv_label_create(G.btn_cancel);
lv_label_set_text(lbl, "Cancel");
lv_obj_center(lbl);
lv_obj_add_event_cb(G.btn_cancel, btn_cancel_cb, LV_EVENT_CLICKED, NULL);
tt_lvgl_unlock();
}
static void onShow(AppHandle app, void* data, lv_obj_t* parent) {
(void)data; (void)parent;
memset(&G, 0, sizeof(G));
G.app_handle = app;
G.last_pct = -1;
BundleHandle bundle = tt_app_get_parameters(app);
bool has_url = false;
bool has_path = false;
if (bundle) {
char url_buf[512] = {0};
char path_buf[512] = {0};
bool override_val = false;
if (tt_bundle_opt_string(bundle, "url", url_buf, sizeof(url_buf)) ||
tt_bundle_opt_string(bundle, "file", url_buf, sizeof(url_buf))) {
strncpy(G.url, url_buf, sizeof(G.url)-1);
has_url = true;
}
if (tt_bundle_opt_string(bundle, "path", path_buf, sizeof(path_buf)) ||
tt_bundle_opt_string(bundle, "dest", path_buf, sizeof(path_buf)) ||
tt_bundle_opt_string(bundle, "file_dest", path_buf, sizeof(path_buf))) {
strncpy(G.path, path_buf, sizeof(G.path)-1);
has_path = true;
}
if (tt_bundle_opt_bool(bundle, "override", &override_val)) {
G.override_existing = override_val;
} else {
bool ow2 = false;
if (tt_bundle_opt_bool(bundle, "overwrite", &ow2)) G.override_existing = ow2;
}
}
if (!has_url) strncpy(G.url, DEFAULT_URL, sizeof(G.url)-1);
if (!has_path) strncpy(G.path, DEFAULT_PATH, sizeof(G.path)-1);
mkdir("/sdcard", 0777);
mkdir("/sdcard/download", 0777);
mkdir("/sdcard/dm", 0777);
build_ui();
bool should_autostart = has_url && has_path;
if (!has_url && !has_path) {
should_autostart = true;
tt_lvgl_lock(portMAX_DELAY);
if (G.lbl_status) lv_label_set_text(G.lbl_status, "Demo mode – no args, using defaults");
tt_lvgl_unlock();
}
ESP_LOGI(TAG, "Autostart=%d url=%s path=%s override=%d", should_autostart, G.url, G.path, G.override_existing);
if (should_autostart) {
BaseType_t res = xTaskCreate(download_task, "sd_dl", 8192, NULL, 5, &G.dl_handle);
ESP_LOGI(TAG, "xTaskCreate res=%d handle=%p", res, G.dl_handle);
if (res != pdPASS) update_ui_status("Failed to create DL task");
} else {
update_ui_status("Missing url/path args");
}
}
static void onHide(AppHandle app, void* data) {
(void)app; (void)data;
if (G.dl_handle) {
G.cancel_req = true;
for (int i=0;i<20 && G.dl_handle!=NULL; i++) vTaskDelay(pdMS_TO_TICKS(100));
if (G.dl_handle) { vTaskDelete(G.dl_handle); G.dl_handle = NULL; G.downloading = false; }
}
}
int main(int argc, char* argv[]) {
(void)argc; (void)argv;
tt_app_register((AppRegistration){ .onShow = onShow, .onHide = onHide });
return 0;
}
@@ -1,7 +1,7 @@
manifest.version=0.2
target.sdk=0.8.0-dev
target.platforms=esp32s3
app.id=one.tactility.livecaptions
app.id=one.tactility.sddownloader
app.name=SD Downloader
app.version.name=1.0.0
app.version.code=1
app.name=Live Captions
+123 -14
View File
@@ -30,6 +30,18 @@
#define CHUNK_BYTES (CHUNK_SAMPLES * (BITS_PER_SAMPLE / 8)) // 1024 bytes
#define AUDIO_BUF_SIZE 4096
// ─── Scroll animation flags ───
// LVGL label scroll animation (LV_LABEL_LONG_SCROLL_*) causes continuous
// invalidations and screen refreshes that can starve the audio_stream task.
// LVGL's lv_list_add_button() internally sets SCROLL_CIRCULAR by default (see
// lv_list.c), so we must override it for memo names.
//
// User request: apply flag only to recordings names on the list.
#define ENABLE_SCROLL_ANIM_WHEN_IDLE 1
#define ENABLE_SCROLL_ANIM_DURING_AUDIO 0
#define ENABLE_MEMO_LIST_SCROLL_ANIM 0 // 0 = DOT truncation (no anim, safe for audio), 1 = SCROLL_CIRCULAR
#define ENABLE_MEMO_LIST_SCROLL_DURING_AUDIO 0 // Force DOT during recording/playback regardless of above
#define MAX_MEMOS 50
#define MAX_FILENAME_LEN 64
@@ -208,6 +220,8 @@ static void record_task(void* arg) {
size_t total_written = 0;
uint32_t start_time = xTaskGetTickCount();
uint32_t last_ui_sec = 0;
bool first_ui = true;
while (ctx->state == STATE_RECORDING) {
size_t bytes_read = 0;
@@ -223,13 +237,18 @@ static void record_task(void* arg) {
}
uint32_t elapsed_sec = (xTaskGetTickCount() - start_time) / configTICK_RATE_HZ;
char status_buf[64];
snprintf(status_buf, sizeof(status_buf), "🔴 Recording... %02u:%02u", (unsigned)(elapsed_sec / 60), (unsigned)(elapsed_sec % 60));
// Throttle UI updates to once per second to reduce LVGL invalidations during recording
if (first_ui || elapsed_sec != last_ui_sec) {
first_ui = false;
last_ui_sec = elapsed_sec;
char status_buf[64];
snprintf(status_buf, sizeof(status_buf), "🔴 Recording... %02u:%02u", (unsigned)(elapsed_sec / 60), (unsigned)(elapsed_sec % 60));
tt_lvgl_lock(portMAX_DELAY);
lv_label_set_text(ctx->lbl_status, status_buf);
lv_bar_set_value(ctx->bar_progress, (int)((elapsed_sec * 100 / 300)) % 100, LV_ANIM_OFF);
tt_lvgl_unlock();
tt_lvgl_lock(portMAX_DELAY);
lv_label_set_text(ctx->lbl_status, status_buf);
lv_bar_set_value(ctx->bar_progress, (int)((elapsed_sec * 100 / 300)) % 100, LV_ANIM_OFF);
tt_lvgl_unlock();
}
}
audio_stream_close(ctx->input_handle);
@@ -349,6 +368,9 @@ static void play_task(void* arg) {
}
bool out_open = true;
int last_pct = -1;
uint32_t last_elapsed_sec = 0;
bool first_ui = true;
while (total_played < data_size && ctx->state != STATE_IDLE) {
if (ctx->state == STATE_PAUSED) {
@@ -407,15 +429,21 @@ static void play_task(void* arg) {
uint32_t elapsed_sec = total_played / bytes_per_sec;
uint32_t total_sec = data_size / bytes_per_sec;
char status_buf[64];
snprintf(status_buf, sizeof(status_buf), "🔊 Playing... %02u:%02u / %02u:%02u",
(unsigned)(elapsed_sec / 60), (unsigned)(elapsed_sec % 60),
(unsigned)(total_sec / 60), (unsigned)(total_sec % 60));
// Throttle UI updates – only when % or second changes, to reduce LVGL work during audio
if (first_ui || pct != last_pct || elapsed_sec != last_elapsed_sec) {
first_ui = false;
last_pct = pct;
last_elapsed_sec = elapsed_sec;
char status_buf[64];
snprintf(status_buf, sizeof(status_buf), "🔊 Playing... %02u:%02u / %02u:%02u",
(unsigned)(elapsed_sec / 60), (unsigned)(elapsed_sec % 60),
(unsigned)(total_sec / 60), (unsigned)(total_sec % 60));
tt_lvgl_lock(portMAX_DELAY);
lv_label_set_text(ctx->lbl_status, status_buf);
lv_bar_set_value(ctx->bar_progress, pct, LV_ANIM_OFF);
tt_lvgl_unlock();
tt_lvgl_lock(portMAX_DELAY);
lv_label_set_text(ctx->lbl_status, status_buf);
lv_bar_set_value(ctx->bar_progress, pct, LV_ANIM_OFF);
tt_lvgl_unlock();
}
}
fclose(f);
@@ -549,17 +577,91 @@ static void refresh_memo_list(AppCtx* ctx) {
for (int i = 0; i < ctx->memo_count; i++) {
lv_obj_t* btn = lv_list_add_btn(ctx->lst_memos, LV_SYMBOL_AUDIO, ctx->memos[i]);
lv_obj_add_event_cb(btn, on_memo_selected, LV_EVENT_CLICKED, ctx);
// ─── Fix: LVGL list buttons default to SCROLL_CIRCULAR (see lv_list.c)
// This causes constant invalidations that starve audio_stream.
// User wants flag to disable this for recording names.
// Safe handling: last child is always the label (icon is first if present)
uint32_t child_cnt = lv_obj_get_child_cnt(btn);
if (child_cnt > 0) {
lv_obj_t* label = lv_obj_get_child(btn, child_cnt - 1);
if (label) {
#if ENABLE_MEMO_LIST_SCROLL_ANIM
// Allow scrolling when flag enabled – but force DOT during audio if that flag is off
#if ENABLE_MEMO_LIST_SCROLL_DURING_AUDIO
lv_label_set_long_mode(label, LV_LABEL_LONG_SCROLL_CIRCULAR);
#else
// During audio we still force DOT; this func runs at list refresh (idle time)
// To respect flag for idle, we set SCROLL here; update_ui will force DOT during audio if needed
lv_label_set_long_mode(label, LV_LABEL_LONG_SCROLL_CIRCULAR);
#endif
#else
lv_label_set_long_mode(label, LV_LABEL_LONG_DOT);
#endif
}
}
if (i == ctx->selected_index) {
lv_obj_add_state(btn, LV_STATE_CHECKED);
}
}
}
// Helper to force DOT on memo list during audio playback/recording if needed
static void set_memo_list_scroll_enabled(AppCtx* ctx, bool enable) {
if (!ctx->lst_memos) return;
#if ENABLE_MEMO_LIST_SCROLL_DURING_AUDIO
// If flag allows scroll during audio, do nothing
if (enable) return;
#endif
// Force all memo labels to DOT (no animation) when audio active
uint32_t list_child_cnt = lv_obj_get_child_cnt(ctx->lst_memos);
for (uint32_t i = 0; i < list_child_cnt; i++) {
lv_obj_t* btn = lv_obj_get_child(ctx->lst_memos, i);
if (!btn) continue;
uint32_t btn_child_cnt = lv_obj_get_child_cnt(btn);
if (btn_child_cnt == 0) continue;
lv_obj_t* label = lv_obj_get_child(btn, btn_child_cnt - 1);
if (label) {
lv_label_set_long_mode(label, enable ?
#if ENABLE_MEMO_LIST_SCROLL_ANIM
LV_LABEL_LONG_SCROLL_CIRCULAR
#else
LV_LABEL_LONG_DOT
#endif
: LV_LABEL_LONG_DOT);
}
}
}
// Helper to control label scrolling based on audio state
static void set_status_scroll_enabled(AppCtx* ctx, bool enable) {
if (!ctx->lbl_status) return;
if (enable) {
#if ENABLE_SCROLL_ANIM_WHEN_IDLE
// Allow scrolling when idle if flag says so, otherwise DOT
// We intentionally keep status as DOT even when idle to avoid
// unnecessary redraws, but this hook lets future tweaks enable it.
lv_label_set_long_mode(ctx->lbl_status, LV_LABEL_LONG_DOT);
#else
lv_label_set_long_mode(ctx->lbl_status, LV_LABEL_LONG_DOT);
#endif
} else {
#if ENABLE_SCROLL_ANIM_DURING_AUDIO
lv_label_set_long_mode(ctx->lbl_status, LV_LABEL_LONG_SCROLL_CIRCULAR);
#else
lv_label_set_long_mode(ctx->lbl_status, LV_LABEL_LONG_DOT);
#endif
}
}
static void update_ui(AppCtx* ctx) {
if (ctx->lbl_status == NULL) return;
switch (ctx->state) {
case STATE_RECORDING:
set_status_scroll_enabled(ctx, false);
set_memo_list_scroll_enabled(ctx, false);
lv_obj_add_state(ctx->btn_record, LV_STATE_DISABLED);
lv_obj_add_state(ctx->btn_play_pause, LV_STATE_DISABLED);
lv_obj_clear_state(ctx->btn_stop, LV_STATE_DISABLED);
@@ -567,6 +669,8 @@ static void update_ui(AppCtx* ctx) {
break;
case STATE_PLAYING:
set_status_scroll_enabled(ctx, false);
set_memo_list_scroll_enabled(ctx, false);
lv_obj_add_state(ctx->btn_record, LV_STATE_DISABLED);
lv_obj_clear_state(ctx->btn_play_pause, LV_STATE_DISABLED);
lv_label_set_text(lv_obj_get_child(ctx->btn_play_pause, 0), LV_SYMBOL_PAUSE);
@@ -575,6 +679,8 @@ static void update_ui(AppCtx* ctx) {
break;
case STATE_PAUSED:
set_status_scroll_enabled(ctx, false);
set_memo_list_scroll_enabled(ctx, false);
lv_obj_add_state(ctx->btn_record, LV_STATE_DISABLED);
lv_obj_clear_state(ctx->btn_play_pause, LV_STATE_DISABLED);
lv_label_set_text(lv_obj_get_child(ctx->btn_play_pause, 0), LV_SYMBOL_PLAY);
@@ -584,6 +690,8 @@ static void update_ui(AppCtx* ctx) {
case STATE_IDLE:
default:
set_status_scroll_enabled(ctx, true);
set_memo_list_scroll_enabled(ctx, true);
lv_obj_clear_state(ctx->btn_record, LV_STATE_DISABLED);
if (ctx->selected_index >= 0 && ctx->selected_index < ctx->memo_count) {
@@ -642,6 +750,7 @@ static void onShowApp(AppHandle app, void* data, lv_obj_t* parent) {
lv_obj_set_width(g_ctx.lbl_status, lv_pct(95));
lv_obj_set_style_text_align(g_ctx.lbl_status, LV_TEXT_ALIGN_CENTER, 0);
lv_obj_set_style_text_color(g_ctx.lbl_status, lv_color_hex(0xCDD6F4), 0);
lv_label_set_long_mode(g_ctx.lbl_status, LV_LABEL_LONG_DOT);
lv_label_set_text(g_ctx.lbl_status, "Scanning memos...");
g_ctx.bar_progress = lv_bar_create(card);
-4
View File
@@ -15,10 +15,6 @@
#include <cstring>
#include "esp_log.h"
// The deployed 0.8.0-dev firmware still exports this legacy lookup API, while
// the current SDK headers only expose device_get_by_name().
extern "C" Device* device_find_by_name(const char* name);
#ifndef M_PI
#define M_PI 3.14159265358979323846f
#endif
-51
View File
@@ -1,51 +0,0 @@
#!/usr/bin/env python3
"""Validate MP3 Player close-and-resume evidence captured from serial."""
from __future__ import annotations
import argparse
import re
from pathlib import Path
def evaluate_resume_trace(trace: str, fixture: str, minimum_position: int = 6) -> dict[str, int]:
"""Require real playback, persisted position, and same-file relaunch evidence."""
escaped = re.escape(fixture)
initial = re.search(rf"Starting MP3 playback: {escaped} .* resume=0\b", trace)
stream_opened = re.search(r"Audio stream opened: \d+ Hz, \d+ channels", trace)
persisted = re.search(rf"History saved on hide: {escaped} pos (\d+) total \d+", trace)
resumed = re.search(rf"Resuming last play {escaped} at (\d+) sec from history", trace)
playback_matches = list(re.finditer(rf"Starting MP3 playback: {escaped} .* resume=(\d+)\b", trace))
resumed_playback = playback_matches[-1] if playback_matches else None
if "Playback task stuck, force deleting" in trace:
raise RuntimeError("forced playback task termination invalidates the device test")
if not initial or not stream_opened:
raise RuntimeError("missing verified playback evidence for the fixture")
if not persisted or not resumed or not resumed_playback:
raise RuntimeError("missing persisted or resumed playback evidence")
persisted_position = int(persisted.group(1))
resumed_position = int(resumed.group(1))
resumed_playback_position = int(resumed_playback.group(1))
if persisted_position < minimum_position:
raise RuntimeError(f"persisted position {persisted_position} is below {minimum_position}")
if resumed_position != persisted_position or resumed_playback_position != persisted_position:
raise RuntimeError("relaunch did not resume the persisted position on the same file")
return {"persisted_position": persisted_position, "resumed_position": resumed_position}
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--trace", type=Path, required=True, help="Captured serial log")
parser.add_argument("--fixture", required=True, help="Absolute fixture path logged by MP3 Player")
parser.add_argument("--minimum-position", type=int, default=6)
args = parser.parse_args()
result = evaluate_resume_trace(args.trace.read_text(encoding="utf-8", errors="replace"), args.fixture, args.minimum_position)
print(f"PASS persisted_position={result['persisted_position']} resumed_position={result['resumed_position']}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
-51
View File
@@ -1,51 +0,0 @@
"""Source-level contract for the Live Captions external Tactility app.
The app is ELF-loaded firmware code, so this test guards the device protocol and
persistence requirements before an ESP32-S3 build/install test is available.
"""
from pathlib import Path
import unittest
ROOT = Path(__file__).resolve().parents[1]
APP = ROOT / "Apps" / "LiveCaptions"
class LiveCaptionsContractTests(unittest.TestCase):
def test_manifest_identifies_the_captions_app(self):
manifest = (APP / "manifest.properties").read_text()
self.assertIn("app.id=one.tactility.livecaptions", manifest)
self.assertIn("app.name=Live Captions", manifest)
self.assertIn("target.platforms=esp32s3", manifest)
def test_app_streams_audio_and_logs_final_caption_to_daily_text_file(self):
source = (APP / "main" / "Source" / "main.c").read_text()
self.assertIn("audio_stream_open_input", source)
self.assertIn("/sdcard/captions", source)
self.assertIn('"%Y-%m-%d.txt"', source)
self.assertIn("append_final_caption", source)
self.assertIn('"draft"', source)
self.assertIn('"interim_transcript"', source)
self.assertIn('"transcript"', source)
self.assertIn('"final"', source)
self.assertIn("const char* stop", source)
self.assertNotIn("i2s_controller_", source)
def test_ui_autostarts_and_keeps_only_the_latest_thirty_words(self):
source = (APP / "main" / "Source" / "main.c").read_text()
self.assertIn("#define DISPLAY_WORD_LIMIT 50", source)
self.assertIn("copy_recent_words", source)
self.assertIn("tt_lvgl_toolbar_create_for_app", source)
self.assertNotIn("lv_btn_create", source)
self.assertNotIn("start_button", source)
self.assertIn("xTaskCreate(worker_task", source)
def test_connection_failure_stays_failed_until_user_starts_again(self):
source = (APP / "main" / "Source" / "main.c").read_text()
self.assertIn("CAPTION_FAILED", source)
self.assertIn("Fail to connect", source)
self.assertNotIn("retry_attempt", source)
self.assertNotIn("retry scheduled", source)
if __name__ == "__main__":
unittest.main()
-82
View File
@@ -1,82 +0,0 @@
from pathlib import Path
MP3_SOURCE = Path(__file__).parents[1] / "Apps" / "Mp3Player" / "main" / "Source" / "main.c"
def test_mp3_player_uses_the_firmware_audio_stream_device_name():
source = MP3_SOURCE.read_text(encoding="utf-8")
assert 'device_find_by_name("audio-stream0")' in source
assert 'device_find_by_name("audio-stream")' not in source
assert "device_get_first_by_type" not in source
assert "device_find_first_by_type" not in source
def test_mp3_player_restores_a_visible_persisted_volume_control_above_the_six_button_row():
source = MP3_SOURCE.read_text(encoding="utf-8")
assert "lv_obj_t* card = lv_obj_create(parent);" not in source
assert "lv_label_set_text(icon, LV_SYMBOL_AUDIO);" not in source
assert "lv_obj_t* vol_box = lv_obj_create(parent);" in source
assert "g_ctx.slider_volume = lv_slider_create(vol_box);" in source
assert "lv_slider_set_value(g_ctx.slider_volume, g_ctx.volume, LV_ANIM_OFF);" in source
assert "on_volume_slider_changed" in source
assert "save_volume(ctx);" in source
assert "lv_obj_set_size(bottom_box, lv_pct(100), 52);" in source
assert "lv_obj_t* btn_close = lv_btn_create(ctrl_box);" in source
assert "lv_obj_t* btn_hist = lv_btn_create(ctrl_box);" in source
assert "lv_label_set_text(lbl_close, LV_SYMBOL_CLOSE);" in source
assert "lv_label_set_text(lbl_hist_btn, LV_SYMBOL_DIRECTORY);" in source
assert "lv_label_set_text(lbl_back, \"-15s\");" in source
assert "lv_label_set_text(lbl_fwd, \"+15s\");" in source
def test_mp3_player_folder_button_returns_to_the_files_list():
source = MP3_SOURCE.read_text(encoding="utf-8")
library_callback = source.split("static void on_library_click", 1)[1].split("/* ─── App Lifecycle", 1)[0]
assert "tt_app_stop();" not in library_callback
assert "show_history_screen(&g_ctx);" in library_callback
assert "load_first_sd_mp3" not in library_callback
def test_mp3_player_history_view_and_fifteen_second_seek_are_present():
source = MP3_SOURCE.read_text(encoding="utf-8")
assert "#define SEEK_SECONDS 15" in source
assert 'tt_app_get_user_data_child_path(app, "play_history.txt"' in source
assert "static void show_history_screen" in source
assert "static void on_history_selected" in source
assert "char label[640];" in source
assert "request_seek(ctx, -SEEK_SECONDS);" in source
assert "request_seek(ctx, SEEK_SECONDS);" in source
def test_mp3_player_persists_and_resumes_short_tracks_too():
source = MP3_SOURCE.read_text(encoding="utf-8")
history_selector = source.split("static void on_history_selected", 1)[1].split("static void refresh_history_list", 1)[0]
assert "Skip history: audio too short" not in source
assert "Resuming last 10min+ play" not in source
assert "Resuming last play %s at %d sec from history" in source
assert "if (he->total_sec >= 600)" not in history_selector
assert "resume_pos = he->pos_sec;" in history_selector
def test_mp3_player_close_waits_for_playback_before_stopping_the_app():
source = MP3_SOURCE.read_text(encoding="utf-8")
close_callback = source.split("static void on_close_click", 1)[1].split("static void on_library_click", 1)[0]
assert "wait_for_playback_task_to_exit(&g_ctx);" in close_callback
assert close_callback.index("wait_for_playback_task_to_exit(&g_ctx);") < close_callback.index("tt_app_stop();")
assert "tt_lvgl_unlock();" in source
assert "tt_lvgl_lock(portMAX_DELAY);" in source
def test_mp3_player_has_a_consumed_dev_autoclose_hook_for_device_tests():
source = MP3_SOURCE.read_text(encoding="utf-8")
assert '"mp3player_dev_autoclose_ms"' in source
assert "on_close_click(NULL);" in source
assert "unlink(marker_path);" in source
-69
View File
@@ -1,69 +0,0 @@
import importlib.util
from pathlib import Path
RUNNER_PATH = Path(__file__).parents[1] / "scripts" / "mp3_player_device_runner.py"
def load_runner():
spec = importlib.util.spec_from_file_location("mp3_player_device_runner", RUNNER_PATH)
module = importlib.util.module_from_spec(spec)
assert spec.loader is not None
spec.loader.exec_module(module)
return module
def test_resume_trace_requires_playback_persistence_and_same_path_resume():
runner = load_runner()
fixture = "/sdcard/download/test.mp3"
trace = "\n".join(
[
"I Mp3Player: Starting MP3 playback: /sdcard/download/test.mp3 (size: 480000 bytes) resume=0",
"I Mp3Player: Audio stream opened: 16000 Hz, 1 channels",
"I Mp3Player: History saved on hide: /sdcard/download/test.mp3 pos 9 total 30",
"I Mp3Player: Resuming last play /sdcard/download/test.mp3 at 9 sec from history",
"I Mp3Player: Starting MP3 playback: /sdcard/download/test.mp3 (size: 480000 bytes) resume=9",
]
)
result = runner.evaluate_resume_trace(trace, fixture, minimum_position=6)
assert result == {"persisted_position": 9, "resumed_position": 9}
def test_resume_trace_rejects_a_different_file_or_no_playback_evidence():
runner = load_runner()
fixture = "/sdcard/download/test.mp3"
try:
runner.evaluate_resume_trace(
"I Mp3Player: Starting MP3 playback: /sdcard/other.mp3 (size: 1 bytes) resume=0",
fixture,
minimum_position=6,
)
except RuntimeError as exc:
assert "playback" in str(exc).lower()
else:
raise AssertionError("missing playback evidence must fail")
def test_resume_trace_rejects_forced_playback_task_termination():
runner = load_runner()
fixture = "/sdcard/download/test.mp3"
trace = "\n".join(
[
f"I Mp3Player: Starting MP3 playback: {fixture} (size: 480000 bytes) resume=0",
"I Mp3Player: Audio stream opened: 16000 Hz, 1 channels",
f"I Mp3Player: History saved on hide: {fixture} pos 9 total 180",
"W Mp3Player: Playback task stuck, force deleting",
f"I Mp3Player: Resuming last play {fixture} at 9 sec from history",
f"I Mp3Player: Starting MP3 playback: {fixture} (size: 480000 bytes) resume=9",
]
)
try:
runner.evaluate_resume_trace(trace, fixture, minimum_position=6)
except RuntimeError as exc:
assert "forced" in str(exc).lower()
else:
raise AssertionError("forced task termination must fail the device test")