tactility_apps/AGENTS.md

# AGENTS.md — Tactility Apps

This file is for AI agents (and humans) working in this repo. It documents local workflow, owned hardware, and in-repo skills.

## Repo

Tactility external ELF apps — loaded at runtime via firmware symbol table. Each app: `Apps/<Name>/manifest.properties` + `main/CMakeLists.txt` + `Source/*.c`.

Upstream: `https://github.com/TactilityProject/TactilityApps`
Personal Gitea mirror: `https://git.reynafamily.com/adolforeyna/tactility_apps` (`personal` remote)

Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` and `.114` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.

## Local Workstation

- Host: macOS 26.5.2
- Project: `/Users/adolforeyna/Projects/electronics/tactility/tactility_apps`
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf` (v5.5.2), Python env `idf5.5_py3.9_env` at `~/.espressif/python_env/idf5.5_py3.9_env` (NOT `idf5.3_py3.9_env` — that env does not exist on this machine)

### Tactility SDK location (IMPORTANT)

- When you run `tactility.py ... build` **without** `--local-sdk`, it auto-downloads the SDK to the per-app cache:
  `Apps/<Name>/.tactility/<version>-<platform>/TactilitySDK` (e.g. `Apps/BookPlayer/.tactility/0.8.0-dev-esp32s3/TactilitySDK`).
- **The freshly CDN-downloaded 0.8.0-dev SDK for esp32s3 is broken for this setup** — it fails at CMake with
  `Failed to resolve component 'app-module' required by component 'TactilitySDK': unknown name` and also renames the
  device API (`device_find_*` → `device_get_by_name/device_get_first_by_type` out-param style). It is a *different,
  newer* SDK than the working one.
- **The known-good SDK is the locally-cached copy** already present in the other apps'
  `.tactility/0.8.0-dev-esp32s3/TactilitySDK` (the one that carries a `Drivers/` folder). It builds clean.
- Workaround when a fresh download fails: copy the working cached SDK over the broken one:
  ```bash
  SRC=Apps/PipecatVoice/.tactility/0.8.0-dev-esp32s3   # known-good copy
  DST=Apps/MyApp/.tactility/0.8.0-dev-esp32s3
  rm -rf "$DST/TactilitySDK"; cp -R "$SRC/TactilitySDK" "$DST/"
  ```
  Then rebuild. (Resolving the root cause on the newer SDK is still TODO — consider pinning/downloading the cached zip
  so this isn't needed per-app.)

### Hardware

| Board | IP | Port | Notes |
|-------|-----|------|-------|
| ES3C28P 2.8" color 240x320 | `192.168.68.112` | `/dev/cu.usbmodem101` | Primary deploy target, 16MB flash, OCT PSRAM, SD for ELF assets |
| ES3C28P 2.8" color 240x320 | `192.168.68.114` | `/dev/cu.usbmodem101` | Secondary S3 (used for recent BookPlayer deploys) |
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111` | `/dev/cu.usbmodem1101` | Secondary, compact density, SD threshold 60, font 18 |

User shorthand: "ending in 112" → `192.168.68.112`. Install is dashboard-port API (80), not dev port 6666.

## Build, Deploy, Run — Correct Env Wrapper

Hermes desktop Python 3.11 venv pollutes `PYTHONPATH` → `tactility.py` crashes with `TypeError: unsupported operand type(s) for |` (urllib3 uses `X | Y` 3.10+). Also `idf.py` crashes with `pydantic_core` mismatch.

Always:

```bash
cd /Users/adolforeyna/Projects/electronics/tactility/tactility_apps
unset PYTHONPATH; unset PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh

# Build
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp clean
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp build esp32s3 --local-sdk --verbose
# → build/MyApp.app (tar), build/cmake-build-esp32s3/MyApp.app.elf

# Symbol check (0 missing = good, any = "Application failed to start: missing symbol" blue OK dialog)
xtensa-esp32s3-elf-nm -D Apps/MyApp/build/cmake-build-esp32s3/MyApp.app.elf | grep " U "
# or automated:
python .claude/skills/tactility-app-development/scripts/verify_symbols.py Apps/MyApp

# Install to device (dashboard API port 80, not dev port 6666)
# Preferred: use reyna-cli (see "Device discovery & management via reyna-cli" below)
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp install 192.168.68.112 esp32s3
# or dashboard directly:
curl -X PUT http://192.168.68.112/api/apps/install -F "file=@build/MyApp.app"

# Run
curl -X POST "http://192.168.68.112/api/apps/run?id=one.tactility.myapp"
# screenshot QA
curl -s http://192.168.68.112/api/screenshot -o /tmp/fb.png && open /tmp/fb.png
curl http://192.168.68.112/api/apps   # list installed

# Dev server fallback (needs Developer mode enabled in Settings, port 6666)
# If 6666 refused → use dashboard PUT above
```

### Device discovery & management via reyna-cli

`reyna-cli` (at `/Users/adolforeyna/.local/bin/reyna-cli`) wraps the Tactility LAN web API with a device registry
(`~/.hermes/local_devices.yaml`). Prefer it over raw IPs/curl for fleet discovery and management. Boards are
addressed by their stable registry **ID** (not IP — DHCP IPs change); `devices list` maps IDs→IPs.

Registered Tactility boards (from `reyna-cli devices list`):

| ID | IP | Board |
|----|----|-------|
| `kidosos_a790` | `.129` | Personal 3.5-inch Tactility Board (Dev) — USB `/dev/cu.usbmodem*` for serial |
| `kidosos_5c5c` | `.114` | Grace's 2.8-inch board |
| `kidosos_5690` | `.107` | Elias's 2.8-inch board |
| `kidosos_591c` | `.128` | Kitchen 2.8-inch board |
| `esp32_screen` | `.123` | ESP32 Screen |
| `little32` | `.122` | Little32 Kitchen ESP32 Screen |

Commands:

```bash
reyna-cli tactility discover                       # probe all boards; online/offline + sysinfo
reyna-cli devices list                             # registry (id, ip, display name, status)
reyna-cli tactility sysinfo kidsos_a790            # firmware/version/heap/psram/SD
reyna-cli tactility apps kidsos_a790               # installed apps
reyna-cli tactility install kidsos_5c5c build/MyApp.app   # PUT install by device id
reyna-cli tactility run kidsos_5c5c one.tactility.myapp   # launch app
reyna-cli tactility report kidsos_a790             # apps + SD + bible/podcast usage
# most commands accept --json for machine-readable output
```

Use a serial monitor on the USB board for loader/ELF diagnostics: `/dev/cu.usbmodem*` @ 115200
(e.g. `idf5.5_py3.9_env/bin/python -m serial.tools.miniterm /dev/cu.usbmodem31201 115200`). This is how you
capture `E ELF: Can't find symbol X` on `.129` — `verify_symbols.py` only compares against local firmware
SOURCE and can pass even when the flashed firmware's export table differs.

### manifest.properties

```properties
[manifest]
version=0.1
[target]
sdk=0.8.0-dev
platforms=esp32s3,esp32p4
[app]
id=one.tactility.myapp
versionName=1.0.0
versionCode=1
name=My App
```

Always pass `esp32s3` platform arg to `install` when only that ELF is built.

### CMake Component Template

```cmake
file(GLOB_RECURSE SOURCE_FILES Source/*.c*)
# optionally include shared libs:
# file(GLOB_RECURSE GAMEKIT Source/*.c* ../../Libraries/GameKit/Source/*.c* ...)
idf_component_register(
  SRCS ${SOURCE_FILES}
  INCLUDE_DIRS "Source" "../../Libraries/GameKit/Include" ...
  REQUIRES TactilitySDK esp_driver_i2s ...
)
target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-format-truncation -Wno-unused-but-set-variable)
```

Keep app source as pure C (no `auto`, no lambdas in `.c`) or rename to `.cpp`.

### ELF Loader — Missing Symbol Pattern

Tactility firmware exports symbols via `ESP_ELFSYM_EXPORT` / `DEFINE_MODULE_SYMBOL` in `firmware/Modules/lvgl-module/source/symbols.c`. If ELF references an unexported symbol → device modal `Error / Application failed to start: missing symbol`, heap still healthy (~31KB free) → NOT OOM.

Known missing on 0.8.0-dev:
- `device_get_by_name` / `device_get_first_by_type` (out-param API) → **Not exported** by flashed firmware; use legacy exported `device_find_by_name` / `device_find_first_by_type` (declare them locally — newer SDK headers dropped them). Gotcha: `verify_symbols.py` compares vs local firmware SOURCE and can pass even when the flashed firmware's export table differs — confirm on-device via serial (`/dev/cu.usbmodem*`, 115200): `E ELF: Can't find symbol X`.
- `lv_font_montserrat_14/18` → use `lvgl_get_text_font(SMALL/DEFAULT/LARGE)` via `tactility/lvgl_fonts.h`
- `lv_obj_set_style_bg_grad_color/dir/stop`, `LV_OPA_95/5`, `LV_SYMBOL_STAR` → use solid colors, COVER/90
- `lv_obj_set_style_max_width` → use `LV_PCT(92)` fixed percents for gutters
- `lv_obj_set_scrollbar_mode`, `lv_arc_create` (pre PR #496 on 0.7), `lv_roller_create`
- `lv_anim_*` → exported since 0.6, safe for 0.7/0.8
- `lv_binfont_create` → exported on 0.8.0-dev but VFS fragile, prefer embedded C fonts via `lv_font_conv --format lvgl`

If OOM not missing-symbol: heap `total 310KB free 275B min_free 23` → 66 buttons case = too many LVGL objects, not PSRAM (ELF lives in PSRAM, LVGL objs internal heap only). Fix: reduce obj count, horizontal slide pattern.

### App Patterns in This Fleet

| Pattern | Summary |
|---------|---------|
| **Immersive full-screen** | Single screen, header+toolbar hidden default, tap toggles chrome, `LV_OBJ_FLAG_HIDDEN`. No `tt_lvgl_toolbar_create_for_app` → saves 40px but need custom exit. |
| **QVGA intro layout** | Header + flex row = left scrollable column (GOAL+CONTROLS cards 90% width) + right 44px rail (play 38px accent + close 30px muted icons). Fixes clipped bottom on 320x198 usable. |
| **Tutorial teach-by-play** | Dungeon 2-row `TUTORIAL_ROWS=2`, floor 0→@+>, 1→@+b+$+>, real from 2. `isTutorial()` skip monster moves, `renderRows` + `HIDDEN` flag, re-center board. |
| **Adaptive scaling** | `apply_verse_scaling()` strlen→ font LARGE/DEFAULT/SMALL + line_space + pad + letter_spacing. Long (>350c) → compact small. |
| **Book browser** | Clean slide ~10 objs: [X left][slider flex][N/66 right], center LARGE 50% bigger tappable, swipe 28px threshold, anim 220-300ms via `lv_anim_*`, fade ref name. Avoid 66 buttons → OOM. |
| **Asset splitting** | 4.8MB XML bible → per-book `.bin` (NUL-joined verses) + `.idx` (BIBK header u16+u32+offsets) + `books.json`. `tt_app_get_assets_path()` + fallback `/sdcard/bibles/`. |
| **Persistence** | `tt_app_get_user_data_child_path(app, "progress.txt", buf, len)` → `snprintf`. Atomic write on change. |
| **Exit paths** | (1) `InputKind::Cancel/Menu` Q/ESC → `tt_app_stop()`, (2) X button 36×28, (3) long-press `LV_EVENT_LONG_PRESSED` → pause overlay Resume/Exit. |
| **GameKit Input** | `attachInputToCurrent()` — re-attach `GameKit::attachInput(root, scene)` after container changes, non-clickable info boxes so taps bubble, `CLICKABLE|GESTURE_BUBBLE` on containers, stop_bubbling on start button. |
| **Screenshot rate-limit** | `/api/screenshot` 2s+ between calls or `ConnectionResetError 54` heap spike. |

## In-Repo Skill

| Skill | When to use |
|-------|-------------|
| `tactility-app-development` | ELF app build/env wrapper, missing-symbol loader triage, asset packaging (per-book binary), immersive LVGL UI, toolbar-less app exit, GameKit/GameInput bubbling pitfall, screenshot rate-limit, QVGA intro rail vs wide buttons, tutorial level pattern, audio apps RLCD pin conflict |

### Key references

- `tactility-app-development/references/build-env.md` — Hermes venv PYTHONPATH error transcript + fix
- `tactility-app-development/references/symbols.md` — exported vs missing list, OOM vs missing-symbol diagnosis, heap sizes
- `tactility-app-development/references/deploy.md` — dashboard vs :6666 install, PUT vs POST, platform arg
- `tactility-app-development/references/immersive-ui.md` — editorial v4 warm #F2F0E8, gap 10, fade 220/300ms
- `tactility-app-development/references/intro-fullscreen.md` — fullscreen no-toolbar, GameKit bubbling fix, QVGA rail, v7 GOAL+CONTROLS + icons
- `tactility-app-development/references/pocket-dungeon.md` — GameKit+SfxEngine GLOB_RECURSE, 112 discovery, 9×7 grid, tutorial 2-row
- `tactility-app-development/references/black-bar-crop.md` — header 44→36 TRANSP, Georgia down-tier LARGE→22, 15px lift
- `tactility-app-development/references/sdk-mismatch-missing-symbol.md` — sdk bump + `max_width` removal
- `tactility-app-development/scripts/verify_symbols.py` — undefined symbol checker

Load via `skill_view(name='tactility-app-development')` in agent sessions.