tactility_apps/AGENTS.md

AGENTS.md — Tactility Apps

This file is for AI agents (and humans) working in this repo. It documents local workflow, owned hardware, and in-repo skills.

Repo

Tactility external ELF apps — loaded at runtime via firmware symbol table. Each app: Apps/<Name>/manifest.properties + main/CMakeLists.txt + Source/*.c.

Upstream: https://github.com/TactilityProject/TactilityApps Personal Gitea mirror: https://git.reynafamily.com/adolforeyna/tactility_apps (personal remote)

Dev board IP: 192.168.68.112 (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. .111 and .114 also online. OS image is 0.8.0-dev so manifest should use sdk=0.8.0-dev or loader warns.

Local Workstation

• Host: macOS 26.5.2
• Project: /Users/adolforeyna/Projects/electronics/tactility/tactility_apps
• ESP-IDF: /Users/adolforeyna/esp/esp-idf (v5.5.2), Python env idf5.5_py3.9_env at ~/.espressif/python_env/idf5.5_py3.9_env (NOT idf5.3_py3.9_env — that env does not exist on this machine)

Tactility SDK location (IMPORTANT)

• When you run tactility.py ... build without --local-sdk, it auto-downloads the SDK to the per-app cache: Apps/<Name>/.tactility/<version>-<platform>/TactilitySDK (e.g. Apps/BookPlayer/.tactility/0.8.0-dev-esp32s3/TactilitySDK).
• The freshly CDN-downloaded 0.8.0-dev SDK for esp32s3 is broken for this setup — it fails at CMake with Failed to resolve component 'app-module' required by component 'TactilitySDK': unknown name and also renames the device API (device_find_* → device_get_by_name/device_get_first_by_type out-param style). It is a different, newer SDK than the working one.
• The known-good SDK is the locally-cached copy already present in the other apps' .tactility/0.8.0-dev-esp32s3/TactilitySDK (the one that carries a Drivers/ folder). It builds clean.
• Workaround when a fresh download fails: copy the working cached SDK over the broken one:

  SRC=Apps/PipecatVoice/.tactility/0.8.0-dev-esp32s3   # known-good copy
  DST=Apps/MyApp/.tactility/0.8.0-dev-esp32s3
  rm -rf "$DST/TactilitySDK"; cp -R "$SRC/TactilitySDK" "$DST/"
  

  Then rebuild. (Resolving the root cause on the newer SDK is still TODO — consider pinning/downloading the cached zip so this isn't needed per-app.)

Hardware

Board	IP	Port	Notes
ES3C28P 2.8" color 240x320	192.168.68.112	/dev/cu.usbmodem101	Primary deploy target, 16MB flash, OCT PSRAM, SD for ELF assets
ES3C28P 2.8" color 240x320	192.168.68.114	/dev/cu.usbmodem101	Secondary S3 (used for recent BookPlayer deploys)
Waveshare RLCD 4.2" mono 400x300 ST7305	192.168.68.111	/dev/cu.usbmodem1101	Secondary, compact density, SD threshold 60, font 18

User shorthand: "ending in 112" → 192.168.68.112. Install is dashboard-port API (80), not dev port 6666.

Build, Deploy, Run — Correct Env Wrapper

Hermes desktop Python 3.11 venv pollutes PYTHONPATH → tactility.py crashes with TypeError: unsupported operand type(s) for | (urllib3 uses X | Y 3.10+). Also idf.py crashes with pydantic_core mismatch.

Always:

cd /Users/adolforeyna/Projects/electronics/tactility/tactility_apps
unset PYTHONPATH; unset PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh

# Build
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp clean
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp build esp32s3 --local-sdk --verbose
# → build/MyApp.app (tar), build/cmake-build-esp32s3/MyApp.app.elf

# Symbol check (0 missing = good, any = "Application failed to start: missing symbol" blue OK dialog)
xtensa-esp32s3-elf-nm -D Apps/MyApp/build/cmake-build-esp32s3/MyApp.app.elf | grep " U "
# or automated:
python .claude/skills/tactility-app-development/scripts/verify_symbols.py Apps/MyApp

# Install to device (dashboard API port 80, not dev port 6666)
# Preferred: use reyna-cli (see "Device discovery & management via reyna-cli" below)
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp install 192.168.68.112 esp32s3
# or dashboard directly:
curl -X PUT http://192.168.68.112/api/apps/install -F "file=@build/MyApp.app"

# Run
curl -X POST "http://192.168.68.112/api/apps/run?id=one.tactility.myapp"
# screenshot QA
curl -s http://192.168.68.112/api/screenshot -o /tmp/fb.png && open /tmp/fb.png
curl http://192.168.68.112/api/apps   # list installed

# Dev server fallback (needs Developer mode enabled in Settings, port 6666)
# If 6666 refused → use dashboard PUT above

Device discovery & management via reyna-cli

reyna-cli (at /Users/adolforeyna/.local/bin/reyna-cli) wraps the Tactility LAN web API with a device registry (~/.hermes/local_devices.yaml). Prefer it over raw IPs/curl for fleet discovery and management. Boards are addressed by their stable registry ID (not IP — DHCP IPs change); devices list maps IDs→IPs.

Registered Tactility boards (from reyna-cli devices list):

ID	IP	Board
kidosos_a790	.129	Personal 3.5-inch Tactility Board (Dev) — USB /dev/cu.usbmodem* for serial
kidosos_5c5c	.114	Grace's 2.8-inch board
kidosos_5690	.107	Elias's 2.8-inch board
kidosos_591c	.128	Kitchen 2.8-inch board
esp32_screen	.123	ESP32 Screen
little32	.122	Little32 Kitchen ESP32 Screen

Commands:

reyna-cli tactility discover                       # probe all boards; online/offline + sysinfo
reyna-cli devices list                             # registry (id, ip, display name, status)
reyna-cli tactility sysinfo kidsos_a790            # firmware/version/heap/psram/SD
reyna-cli tactility apps kidsos_a790               # installed apps
reyna-cli tactility install kidsos_5c5c build/MyApp.app   # PUT install by device id
reyna-cli tactility run kidsos_5c5c one.tactility.myapp   # launch app
reyna-cli tactility report kidsos_a790             # apps + SD + bible/podcast usage
# most commands accept --json for machine-readable output

Use a serial monitor on the USB board for loader/ELF diagnostics: /dev/cu.usbmodem* @ 115200 (e.g. idf5.5_py3.9_env/bin/python -m serial.tools.miniterm /dev/cu.usbmodem31201 115200). This is how you capture E ELF: Can't find symbol X on .129 — verify_symbols.py only compares against local firmware SOURCE and can pass even when the flashed firmware's export table differs.

manifest.properties

[manifest]
version=0.1
[target]
sdk=0.8.0-dev
platforms=esp32s3,esp32p4
[app]
id=one.tactility.myapp
versionName=1.0.0
versionCode=1
name=My App

Always pass esp32s3 platform arg to install when only that ELF is built.

CMake Component Template

file(GLOB_RECURSE SOURCE_FILES Source/*.c*)
# optionally include shared libs:
# file(GLOB_RECURSE GAMEKIT Source/*.c* ../../Libraries/GameKit/Source/*.c* ...)
idf_component_register(
  SRCS ${SOURCE_FILES}
  INCLUDE_DIRS "Source" "../../Libraries/GameKit/Include" ...
  REQUIRES TactilitySDK esp_driver_i2s ...
)
target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-format-truncation -Wno-unused-but-set-variable)

Keep app source as pure C (no auto, no lambdas in .c) or rename to .cpp.

ELF Loader — Missing Symbol Pattern

Tactility firmware exports symbols via ESP_ELFSYM_EXPORT / DEFINE_MODULE_SYMBOL in firmware/Modules/lvgl-module/source/symbols.c. If ELF references an unexported symbol → device modal Error / Application failed to start: missing symbol, heap still healthy (~31KB free) → NOT OOM.

Known missing on 0.8.0-dev:

• device_get_by_name / device_get_first_by_type (out-param API) → Not exported by flashed firmware; use legacy exported device_find_by_name / device_find_first_by_type (declare them locally — newer SDK headers dropped them). Gotcha: verify_symbols.py compares vs local firmware SOURCE and can pass even when the flashed firmware's export table differs — confirm on-device via serial (/dev/cu.usbmodem*, 115200): E ELF: Can't find symbol X.
• lv_font_montserrat_14/18 → use lvgl_get_text_font(SMALL/DEFAULT/LARGE) via tactility/lvgl_fonts.h
• lv_obj_set_style_bg_grad_color/dir/stop, LV_OPA_95/5, LV_SYMBOL_STAR → use solid colors, COVER/90
• lv_obj_set_style_max_width → use LV_PCT(92) fixed percents for gutters
• lv_obj_set_scrollbar_mode, lv_arc_create (pre PR #496 on 0.7), lv_roller_create
• lv_anim_* → exported since 0.6, safe for 0.7/0.8
• lv_binfont_create → exported on 0.8.0-dev but VFS fragile, prefer embedded C fonts via lv_font_conv --format lvgl

If OOM not missing-symbol: heap total 310KB free 275B min_free 23 → 66 buttons case = too many LVGL objects, not PSRAM (ELF lives in PSRAM, LVGL objs internal heap only). Fix: reduce obj count, horizontal slide pattern.

App Patterns in This Fleet

Pattern	Summary
Immersive full-screen	Single screen, header+toolbar hidden default, tap toggles chrome, LV_OBJ_FLAG_HIDDEN. No tt_lvgl_toolbar_create_for_app → saves 40px but need custom exit.
QVGA intro layout	Header + flex row = left scrollable column (GOAL+CONTROLS cards 90% width) + right 44px rail (play 38px accent + close 30px muted icons). Fixes clipped bottom on 320x198 usable.
Tutorial teach-by-play	Dungeon 2-row TUTORIAL_ROWS=2, floor 0→@+>, 1→@+b+$+>, real from 2. isTutorial() skip monster moves, renderRows + HIDDEN flag, re-center board.
Adaptive scaling	apply_verse_scaling() strlen→ font LARGE/DEFAULT/SMALL + line_space + pad + letter_spacing. Long (>350c) → compact small.
Book browser	Clean slide ~10 objs: [X left][slider flex][N/66 right], center LARGE 50% bigger tappable, swipe 28px threshold, anim 220-300ms via lv_anim_*, fade ref name. Avoid 66 buttons → OOM.
Asset splitting	4.8MB XML bible → per-book .bin (NUL-joined verses) + .idx (BIBK header u16+u32+offsets) + books.json. tt_app_get_assets_path() + fallback /sdcard/bibles/.
Persistence	tt_app_get_user_data_child_path(app, "progress.txt", buf, len) → snprintf. Atomic write on change.
Exit paths	(1) InputKind::Cancel/Menu Q/ESC → tt_app_stop(), (2) X button 36×28, (3) long-press LV_EVENT_LONG_PRESSED → pause overlay Resume/Exit.
GameKit Input	attachInputToCurrent() — re-attach GameKit::attachInput(root, scene) after container changes, non-clickable info boxes so taps bubble, `CLICKABLE
Screenshot rate-limit	/api/screenshot 2s+ between calls or ConnectionResetError 54 heap spike.

In-Repo Skill

Skill	When to use
tactility-app-development	ELF app build/env wrapper, missing-symbol loader triage, asset packaging (per-book binary), immersive LVGL UI, toolbar-less app exit, GameKit/GameInput bubbling pitfall, screenshot rate-limit, QVGA intro rail vs wide buttons, tutorial level pattern, audio apps RLCD pin conflict

Key references

• tactility-app-development/references/build-env.md — Hermes venv PYTHONPATH error transcript + fix
• tactility-app-development/references/symbols.md — exported vs missing list, OOM vs missing-symbol diagnosis, heap sizes
• tactility-app-development/references/deploy.md — dashboard vs :6666 install, PUT vs POST, platform arg
• tactility-app-development/references/immersive-ui.md — editorial v4 warm #F2F0E8, gap 10, fade 220/300ms
• tactility-app-development/references/intro-fullscreen.md — fullscreen no-toolbar, GameKit bubbling fix, QVGA rail, v7 GOAL+CONTROLS + icons
• tactility-app-development/references/pocket-dungeon.md — GameKit+SfxEngine GLOB_RECURSE, 112 discovery, 9×7 grid, tutorial 2-row
• tactility-app-development/references/black-bar-crop.md — header 44→36 TRANSP, Georgia down-tier LARGE→22, 15px lift
• tactility-app-development/references/sdk-mismatch-missing-symbol.md — sdk bump + max_width removal
• tactility-app-development/scripts/verify_symbols.py — undefined symbol checker

Load via skill_view(name='tactility-app-development') in agent sessions.