tactility_apps/Apps/DiscoveryMountain/CRASH_ANALYSIS.md

DiscoveryMountain – Crash After Install – Root Cause & Fix

Board: CYD ES3C28P (es3c28p) 192.168.68.132
Firmware: 0.8.0-dev / IDF 5.5.2
App ID: one.tactility.discoverymountain
Branch: personal/discovery-mountain-player → merged into main
Date: 2026-07-21

---

Symptoms

• After tactility.py install 192.168.68.132, device reboots or shows blue dialog Application failed to start: missing symbol
• Serial logs: E ELF: Can't find symbol ..., then Guru Meditation (LoadProhibited, StoreProhibited, IllegalInstruction, Cache error)
• Even when it did start, fetch fell back to 6 dummy episodes (S01E01…) with empty audio_url, so download always failed
• When forced to download 4.3 MB MP3 from http://192.168.68.110:8098/.../esp32_24k.mp3, device hard-faulted during play_task

---

1. Missing Symbol Loader Failures

Detected via xtensa-esp32s3-elf-nm -D …elf | grep " U "

U lv_font_montserrat_14
U lv_roller_create
U lv_roller_set_options
U lv_roller_get_selected
U lv_roller_set_selected
U qsort
U cJSON_* (via json component)

Why:

• Firmware exports defined in firmware/Modules/lvgl-module/source/symbols.c – roller not exported
• lv_font_montserrat_14 is a data symbol, not exported (AGENTS.md known missing)
• qsort not in g_esp_libc_elfsyms nor main_symbols
• cJSON not exported – apps must vendor cJSON.c (see BookPlayer, McpScreen)

Fix:

• manifest.properties V1 INI → V2 flat:

  manifest.version=0.2
  target.sdk=0.8.0-dev
  target.platforms=esp32s3
  app.id=one.tactility.discoverymountain
  app.version.name=0.1.0
  app.version.code=1
  

• main/CMakeLists.txt:

  set(CJSON_SOURCE "$ENV{IDF_PATH}/components/json/cJSON/cJSON.c")
  idf_component_register(SRCS ${SOURCE_FILES} ${CJSON_SOURCE}
                         INCLUDE_DIRS Source "$ENV{IDF_PATH}/components/json/cJSON"
                         REQUIRES TactilitySDK esp_http_client lwip)
  

• Replace roller with lv_dropdown (exported, see symbols.c:272-291)
• Replace qsort with bubble sort
• Replace lv_font_montserrat_14 with lvgl_get_text_font(FONT_SIZE_DEFAULT) from tactility/lvgl_fonts.h

After fix: nm -D shows no missing roller/qsort/font, verify script passes.

---

2. Fatal memset Bug

memset(&G,0,0); // size 0!

Static G is zero-initialized first boot, but second onShow leaves dangling lv_obj_t*, play_handle, etc.

Fix: memset(&G,0,sizeof(G))

---

3. Blocking LVGL Thread → Watchdog Reboot

Original onShow:

load_state(); fetch_data(); build_ui();

fetch_data() did 2× esp_http_client_perform with 8s timeout + cJSON_Parse of 91KB JSON on LVGL thread → 16s block → TWDT.

Fix:

• Build UI immediately with "Fetching..."
• xTaskCreate(fetch_task_fn,"dm_fetch",16384, ...)
• fetch_task_fn does fetch_data() in background, then tt_lvgl_lock to update labels
• Same for download: dl_task 12288 stack

---

4. esp_http_client vs Raw Sockets

• esp_http_client failed on device for PB http://192.168.68.110:8095 (returned chunked 1634\r\n{...}), while curl --http1.0 returned plain JSON.

Raw socket version via lwip_socket, ipaddr_addr, my_htons (like RobotArm) is more reliable and allows PSRAM buffers.

PocketBase:

• HTTP/1.1 → Transfer-Encoding: chunked → 1634\r\n{...}\r\n0\r\n\r\n
• HTTP/1.0 → Content-Length absent, body until close → 5684 bytes seasons, 91293 bytes episodes

Fix: Implement http_get_raw with:

• PSRAM heap_caps_malloc(65536, MALLOC_CAP_SPIRAM|8BIT) growing to 600KB
• Manual realloc via malloc+memcpy+free (avoid heap_caps_realloc not exported)
• strstr(buf,"\r\n\r\n") to find header end
• Detect chunked, decode hex chunk sizes via strtol(hex, NULL, 16)
• Return de-chunked body

Result: DM: seasons items count 37, episodes items 232 (was fallback 37/6)

---

5. Download Buffer & PSRAM

• dl_ep used esp_http_client + FILE* with 2KB stack buf, but also rename with static sd_path() race
• 4.3MB MP3 download needs streaming, not full RAM
• malloc(600000) in internal heap (120KB free) → OOM → cache error

Fix:

• make_sd_path(out,len,slug) caller-provided buffer
• ensure_dir() does NOT mkdir("/sdcard")
• dl_ep_raw uses PSRAM hdr 8192 + recv_buf 4096, streams directly to FILE*
• Progress via lv_label_set_text under tt_lvgl_lock
• LRU keeps 8 files, skips current playing

Result: DL done total 4329701 expected 4329701 – file correctly saved to /sdcard/dm/

---

6. Audio: I2S Direct vs audio-stream + PSRAM

Original used i2s_controller_* directly:

device_lock(i2s_dev); i2s_controller_set_config(...); i2s_controller_write(...);

Conflicts with Audio service, no resampling, vTaskDelete while holding lock → deadlock, samples*2 overflow for mono.

Fix (from BookPlayer):

• find_audio_device() via device_find_by_name("audio-stream")
• audio_stream_open_output(dev, cfg, &handle)
• audio_stream_write(handle, pcm+off, ...)
• close_stream() via audio_stream_close
• wait_play_exit() uses tt_lvgl_unlock(); vTaskDelay(10); tt_lvgl_lock() pattern
• Buffers: inbuf 16384 + pcm 1152*2*2 from PSRAM
• Volume scaling with proper clipping
• Stack 12288 for play/dl/fetch (was 6144/8192 → overflow on cJSON 91KB)
• Rate-limit ui_timer 500ms, only update bar if pct != last_pct

Remaining playback crash:

Guru Meditation: BREAK instr at vTaskGenericNotifyGiveFromISR
play task start ... size 4329701

Occurs right after play_task start. Likely:

• mp3dec_decode_frame on file with ID3 tag (ID3... seen in curl) – needs ID3 skip
• PSRAM inbuf/pcm unaligned for minimp3
• 24kHz sample rate not supported by ES8311 native, though audio-stream should resample

Mitigation: allocate decoder from PSRAM, add ID3 skip, add logs before/after decode, test with 44.1k file.

---

Build & Deploy (correct env)

Per AGENTS.md:

cd /Users/adolforeyna/Projects/Tactility/apps
unset PYTHONPATH; unset PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.10_env
source /Users/adolforeyna/esp/esp-idf-v5.5.2/export.sh
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK

$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/DiscoveryMountain build esp32s3 --local-sdk
curl -X PUT http://192.168.68.132/api/apps/install -F "file=@Apps/DiscoveryMountain/build/DiscoveryMountain.app"
curl -X POST "http://192.168.68.132/api/apps/run?id=one.tactility.discoverymountain"
# serial
python3 -c "import serial; s=serial.Serial('/dev/cu.usbmodem101',115200); ..."

Verify symbols:

xtensa-esp32s3-elf-nm -D Apps/DiscoveryMountain/build/cmake-build-esp32s3/DiscoveryMountain.app.elf | grep " U " | grep -E "roller|qsort|montserrat"
# should be empty

---

Final State

• Stable: App installs, shows UI, fetches 37 seasons / 232 episodes, stays alive (no reboot)
• Download: Works (4.3MB file) via raw socket + PSRAM
• Playback: Crashes on mp3dec_decode_frame or audio_stream_write ISR – needs further ID3 skip + maybe use internal RAM for decoder + test 44.1k file
• Fullscreen: Not needed per user, so no flags=HideStatusBar

---

TODO

• Fix playback: skip ID3v2 before decode, try internal RAM for mp3dec_t, test with 44.1k mp3 from other episode (not esp32_24k)
• Use device_get_by_name / device_put instead of deprecated find
• Use tt_app_get_user_data_path for state file, not hardcoded /sdcard/apps/...
• Rate-limit screenshot API (2s gap) to avoid heap spike