tactility/.claude/skills/tactility-firmware-development/SKILL.md

---
name: tactility-firmware-development
description: Use when merging current Tactility firmware, preserving local ES3C28P/ES3C35P customizations, building/flashing ESP32 firmware, or validating external apps against it.
---

# Tactility firmware development

## Scope and success criteria

Use this skill for firmware upgrades, especially when upstream changes the app
loader, dashboard API, SDK export, or device model. Before editing, record the
active branch, `git status --short`, selected `Devices/<id>`, serial port, and
the expected device identity. Preserve user work and local board
customizations; do not reset or overwrite a dirty tree.

Success is not merely a successful flash: the board must boot, mount storage,
return `/api/sysinfo`, expose required internal apps, and accept an external
app rebuilt against this exact firmware.

## Upgrade and customization audit

1. Fetch and compare the intended upstream branch before merging. Treat
   `upstream/main` as distinct from experimental release/IDF branches unless
   the task explicitly asks for one of those branches.
2. Merge without discarding local work (for example, `git merge --autostash
   upstream/main` after inspecting status). Resolve conflicts by preserving
   required ES3C28P and ES3C35P device definitions, partition selection,
   display/font customizations, and product features.
3. Compare the pre-upgrade customization commit against the resulting worktree.
   Check new app registrations, CMake/source inclusion, settings pages,
   web-server endpoints, and board-specific `sdkconfig` entries—not just files
   that happen to conflict.
4. Commit the resulting firmware customization as a focused, reviewable
   commit after `git diff --check`.

## Multi-binary external-app compatibility

Upstream app packaging changed to `bin/<platform>/<binary>.elf`. The old
`elf/<platform>.elf` archive layout produces a loader `Not executable` error.

- Manifest v0.2: `bin/esp32s3/app.elf`
- Manifest v0.3: `bin/esp32s3/<app.0.binary>.elf`

When this change arrives, update the companion app tool and rebuild every app
from a fresh SDK exported from this firmware. Do not attempt to repair a
packaged ELF on the SD card: its layout and ABI must both be regenerated.

## Build and flash

Use the installed IDF 5.5 environment; host virtual environments can corrupt
both Python dependencies and IDF tooling:

```zsh
unset VIRTUAL_ENV PYTHONPATH PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh

# Select/check the intended board before this step.
python device.py <device-id>
idf.py build
idf.py -p /dev/cu.usbmodemXXXX flash
```

`Tactility/CMakeLists.txt` uses a non-configure-dependent source glob. After
adding a new `.c`/`.cpp` file, run `idf.py reconfigure build`; otherwise a
linker error for a newly referenced symbol may only mean CMake has not
discovered the source file yet.

## Boot and feature acceptance

Capture serial at 115200 after flash or reset. Confirm the board name, SD-card
mount, HTTP-server start, Wi-Fi address, and any feature-specific startup logs.
Then query the resolved device:

```zsh
curl -fsS http://<ip>/api/sysinfo
curl -fsS http://<ip>/api/apps
```

For MCP settings, verify the `McpSettings` internal app appears in `/api/apps`.
When enabled, verify MCP stream startup logs. For a feature restored from an
older customization, validate its registration path and persisted settings,
not only its source files.

## External-app acceptance gate

Use the companion app skill's exact-firmware wrapper:

```zsh
cd /path/to/tactility_apps
scripts/build_current_firmware_app.sh Apps/MyApp --firmware /path/to/tactility
```

Inspect the resulting tar member path, install through port-80 dashboard API,
launch it, and read serial. Required evidence is the loader's `Loading
.../bin/...`, an ELF entry address, `Task started`, and app-specific startup
logs. HTTP 200 or a package simply appearing in `/api/apps` is insufficient.

## Host simulator (buildsim) + web viewer

The POSIX simulator runs the real firmware (LVGL, services, web server) on
macOS/Linux with an SDL backend. On current firmware, `Main.cpp` runs SDL's
event loop on the process main thread while FreeRTOS runs on a separate thread.
This is required by AppKit and supports a native macOS window as well as the
web viewer. `SDL_VIDEODRIVER=dummy` remains useful for headless automation.

Code locations:

- `Devices/simulator/Source/module.cpp` — display resolution + `SIM_DISPLAY_W/H`
- `Devices/simulator/Source/drivers/sdl_display.{h,cpp}` — SDL backend
- `Devices/simulator/Source/drivers/sdl_input.{h,cpp}` — pointer/key state +
  web touch-injection override
- `Tactility/Source/service/webserver/WebServerService.cpp` — `/sim` viewer,
  `/sim/api/*` aliases, `POST /api/sim/touch`, `GET /api/screenshot?fast=`
- `Tactility/Private/Tactility/service/webserver/WebServerService.h` — handler decls

### Build and run

```zsh
# one-time host deps (outside any ESP-IDF env)
mkdir -p /tmp/simbin && ln -sf "$(which python3)" /tmp/simbin/python
pip3 install --break-system-packages lark pyyaml   # devicetree compiler

cd /path/to/tactility
export PATH="/tmp/simbin:$PATH"
env -u ESP_IDF_VERSION -u IDF_PATH cmake -S . -B buildsim -DCMAKE_BUILD_TYPE=Release
env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target Tactility -j "$(sysctl -n hw.ncpu)"

# POSIX SDK for host apps (arm64)
env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target TactilityKernel lvgl minitar minmea \
  app-module crypt-module gps-module http-module lvgl-module lvgl-window-manager-module service-module
env -u ESP_IDF_VERSION -u IDF_PATH python3 Buildscripts/release-sdk-posix.py /tmp/sim-sdk

# release (MUST run from the firmware root: release-simulator.sh uses relative
# version.txt / Data paths)
sh Buildscripts/release-simulator.sh buildsim /tmp/simrun
# Native macOS window + web API. Run this from the release directory because
# data/ and system/ are relative to the process working directory.
(cd /tmp/simrun && SIM_DISPLAY_W=480 SIM_DISPLAY_H=320 \
  nohup ./Tactility > /tmp/sim_gui.log 2>&1 &)
# For CI/headless operation, set SDL_VIDEODRIVER=dummy instead.
curl -s --max-time 5 http://127.0.0.1/api/sysinfo | head -c 120
```

Display resolution: `SIM_DISPLAY_W/H` env (default **480x320 landscape**,
matching on-device screenshots). ES3C35P panel is 320x480 portrait in DTS but
presents 480x320 landscape; ES3C28P is 320x240. The chosen geometry is logged
as `Simulator Sim display WxH`. `SDL_VIDEODRIVER=dummy` is expected to log one
`SdlDisplay Failed to create SDL window: Couldn't find matching render
driver` line — LVGL still renders and screenshots work. A native macOS launch
must not emit that line.

### Web viewer, touch, screenshots

- `GET /sim` → 301 to `/sim/` (trailing slash required so the page's relative
  `api/` URLs resolve under `/sim/`). Viewer polls `api/screenshot?fast=1`
  every 500 ms, footer shows live `naturalWidth×naturalHeight`, click/tap
  POSTs `api/sim/touch?x=&y=`.
- `POST /api/sim/touch?x=123&y=456[&down=0|1]` (also `/sim/api/sim/touch` via
  alias). Coordinates are LVGL logical pixels. `down=1` (default) presses and
  **auto-releases after 1500 ms** (`SIM_TOUCH_HOLD_MS` in `sdl_input.cpp`),
  long enough for LVGL indev polls to register a click. Simulator-only: 404 on
  ESP32 (`#ifndef ESP_PLATFORM`).
- `GET /api/screenshot?fast=1` (default): `lv_snapshot_take` (RGB888) → in-place
  BGR→RGB swap → `lodepng_encode24` **to memory** → chunked HTTP. No filesystem
  touch, ~8 ms/shot. `?fast=0` keeps the legacy `webscreenshotN.png` file path
  (slot scan + accumulation — avoid for viewer loops).
- lodepng include in `.cpp`: `#define LODEPNG_NO_COMPILE_CPP` before
  `#include "src/libs/lodepng/lodepng.h"`, otherwise its C++ `std::vector`
  overloads collide with the C declarations (`conflicting types for 'encode'`).
- MCP includes and `settings::mcp` reads are `#ifdef ESP_PLATFORM`-gated; the
  sim has no `McpSystem`.

### Tailscale viewer

```zsh
tailscale serve --bg --set-path=/simagent http://127.0.0.1:80/
# open: https://<node>/simagent/sim/
```

The serve target must be `/` (not `/sim`): the page resolves `api/` against
its own directory, so at `/simagent/sim/` fetches go to `/simagent/sim/api/…`,
which tailscale strips to `/sim/api/…` and the firmware's `/sim/api/*`
aliases (GET+POST, registered in `startServer()`) handle. Absolute `/api/…`
URLs would 404 at the edge (no `/api` mount there).

### Simulator pitfalls

- **Rebuild ≠ redeploy.** `cmake --build buildsim` updates `buildsim/` only.
  Re-run `release-simulator.sh`, restart the process, then retest. A stale
  `/tmp/simrun/Tactility` serves old handlers with new logs nowhere to be found.
- **One simulator owns port 80.** Do not launch a second instance while another
  simulator is listening: it will initialize LVGL but fail `bind/listen`, so its
  app API and viewer target the other process. Identify the listener with
  `lsof -nP -iTCP:80 -sTCP:LISTEN`, stop only the intended simulator, then
  release/restart it before installing or running POSIX apps.
- **Input is cross-thread on macOS.** SDL event pumping occurs on the real main
  thread; LVGL and web touch injection run elsewhere. Keep all shared pointer,
  key-queue, and touch-override state under `sdl_input.cpp`'s mutex.
- **C array `sizeof` decay.** A helper like
  `f(HttpServerRequest*, char uri[256])` sees `sizeof(uri) == 8`, truncating
  `get_uri` output to 7 chars (`/api/sy`, `/sim/ap` 404s). Pass the size
  explicitly: `f(request, buf, sizeof(buf))`.
- **Log truncation.** `LOG_QUEUE_MESSAGE_MAX_LENGTH` is 256 (`TactilityKernel/
  private/tactility/log_queue.h`), including color/timestamp prefix. Long URIs
  and messages truncate — don't over-interpret a short path in the log.
- **Auth.** The viewer and API handlers enforce `validateRequestAuth` like any
  other endpoint; failures surface as 401/404, not viewer bugs.