tactility/.claude/skills/tactility-firmware-development/SKILL.md

name, description

name	description
tactility-firmware-development	Use when merging current Tactility firmware, preserving local ES3C28P/ES3C35P customizations, building/flashing ESP32 firmware, or validating external apps against it.

Tactility firmware development

Scope and success criteria

Use this skill for firmware upgrades, especially when upstream changes the app loader, dashboard API, SDK export, or device model. Before editing, record the active branch, git status --short, selected Devices/<id>, serial port, and the expected device identity. Preserve user work and local board customizations; do not reset or overwrite a dirty tree.

Success is not merely a successful flash: the board must boot, mount storage, return /api/sysinfo, expose required internal apps, and accept an external app rebuilt against this exact firmware.

Upgrade and customization audit

1. Fetch and compare the intended upstream branch before merging. Treat upstream/main as distinct from experimental release/IDF branches unless the task explicitly asks for one of those branches.
2. Merge without discarding local work (for example, git merge --autostash upstream/main after inspecting status). Resolve conflicts by preserving required ES3C28P and ES3C35P device definitions, partition selection, display/font customizations, and product features.
3. Compare the pre-upgrade customization commit against the resulting worktree. Check new app registrations, CMake/source inclusion, settings pages, web-server endpoints, and board-specific sdkconfig entries—not just files that happen to conflict.
4. Commit the resulting firmware customization as a focused, reviewable commit after git diff --check.

Multi-binary external-app compatibility

Upstream app packaging changed to bin/<platform>/<binary>.elf. The old elf/<platform>.elf archive layout produces a loader Not executable error.

• Manifest v0.2: bin/esp32s3/app.elf
• Manifest v0.3: bin/esp32s3/<app.0.binary>.elf

When this change arrives, update the companion app tool and rebuild every app from a fresh SDK exported from this firmware. Do not attempt to repair a packaged ELF on the SD card: its layout and ABI must both be regenerated.

Build and flash

Use the installed IDF 5.5 environment; host virtual environments can corrupt both Python dependencies and IDF tooling:

unset VIRTUAL_ENV PYTHONPATH PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh

# Select/check the intended board before this step.
python device.py <device-id>
idf.py build
idf.py -p /dev/cu.usbmodemXXXX flash

Tactility/CMakeLists.txt uses a non-configure-dependent source glob. After adding a new .c/.cpp file, run idf.py reconfigure build; otherwise a linker error for a newly referenced symbol may only mean CMake has not discovered the source file yet.

Boot and feature acceptance

Capture serial at 115200 after flash or reset. Confirm the board name, SD-card mount, HTTP-server start, Wi-Fi address, and any feature-specific startup logs. Then query the resolved device:

curl -fsS http://<ip>/api/sysinfo
curl -fsS http://<ip>/api/apps

For MCP settings, verify the McpSettings internal app appears in /api/apps. When enabled, verify MCP stream startup logs. For a feature restored from an older customization, validate its registration path and persisted settings, not only its source files.

External-app acceptance gate

Use the companion app skill's exact-firmware wrapper:

cd /path/to/tactility_apps
scripts/build_current_firmware_app.sh Apps/MyApp --firmware /path/to/tactility

Inspect the resulting tar member path, install through port-80 dashboard API, launch it, and read serial. Required evidence is the loader's Loading .../bin/..., an ELF entry address, Task started, and app-specific startup logs. HTTP 200 or a package simply appearing in /api/apps is insufficient.

Host simulator (buildsim) + web viewer

The POSIX simulator runs the real firmware (LVGL, services, web server) on macOS/Linux with an SDL backend. On current firmware, Main.cpp runs SDL's event loop on the process main thread while FreeRTOS runs on a separate thread. This is required by AppKit and supports a native macOS window as well as the web viewer. SDL_VIDEODRIVER=dummy remains useful for headless automation.

Code locations:

• Devices/simulator/Source/module.cpp — display resolution + SIM_DISPLAY_W/H
• Devices/simulator/Source/drivers/sdl_display.{h,cpp} — SDL backend
• Devices/simulator/Source/drivers/sdl_input.{h,cpp} — pointer/key state + web touch-injection override
• Tactility/Source/service/webserver/WebServerService.cpp — /sim viewer, /sim/api/* aliases, POST /api/sim/touch, GET /api/screenshot?fast=
• Tactility/Private/Tactility/service/webserver/WebServerService.h — handler decls

Build and run

# one-time host deps (outside any ESP-IDF env)
mkdir -p /tmp/simbin && ln -sf "$(which python3)" /tmp/simbin/python
pip3 install --break-system-packages lark pyyaml   # devicetree compiler

cd /path/to/tactility
export PATH="/tmp/simbin:$PATH"
env -u ESP_IDF_VERSION -u IDF_PATH cmake -S . -B buildsim -DCMAKE_BUILD_TYPE=Release
env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target Tactility -j "$(sysctl -n hw.ncpu)"

# POSIX SDK for host apps (arm64)
env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target TactilityKernel lvgl minitar minmea \
  app-module crypt-module gps-module http-module lvgl-module lvgl-window-manager-module service-module
env -u ESP_IDF_VERSION -u IDF_PATH python3 Buildscripts/release-sdk-posix.py /tmp/sim-sdk

# release (MUST run from the firmware root: release-simulator.sh uses relative
# version.txt / Data paths)
sh Buildscripts/release-simulator.sh buildsim /tmp/simrun
# Native macOS window + web API. Run this from the release directory because
# data/ and system/ are relative to the process working directory.
(cd /tmp/simrun && SIM_DISPLAY_W=480 SIM_DISPLAY_H=320 \
  nohup ./Tactility > /tmp/sim_gui.log 2>&1 &)
# For CI/headless operation, set SDL_VIDEODRIVER=dummy instead.
curl -s --max-time 5 http://127.0.0.1/api/sysinfo | head -c 120

Display resolution: SIM_DISPLAY_W/H env (default 480x320 landscape, matching on-device screenshots). ES3C35P panel is 320x480 portrait in DTS but presents 480x320 landscape; ES3C28P is 320x240. The chosen geometry is logged as Simulator Sim display WxH. SDL_VIDEODRIVER=dummy is expected to log one SdlDisplay Failed to create SDL window: Couldn't find matching render driver line — LVGL still renders and screenshots work. A native macOS launch must not emit that line.

Web viewer, touch, screenshots

• GET /sim → 301 to /sim/ (trailing slash required so the page's relative api/ URLs resolve under /sim/). Viewer polls api/screenshot?fast=1 every 500 ms, footer shows live naturalWidth×naturalHeight, click/tap POSTs api/sim/touch?x=&y=.
• POST /api/sim/touch?x=123&y=456[&down=0|1] (also /sim/api/sim/touch via alias). Coordinates are LVGL logical pixels. down=1 (default) presses and auto-releases after 1500 ms (SIM_TOUCH_HOLD_MS in sdl_input.cpp), long enough for LVGL indev polls to register a click. Simulator-only: 404 on ESP32 (#ifndef ESP_PLATFORM).
• GET /api/screenshot?fast=1 (default): lv_snapshot_take (RGB888) → in-place BGR→RGB swap → lodepng_encode24 to memory → chunked HTTP. No filesystem touch, ~8 ms/shot. ?fast=0 keeps the legacy webscreenshotN.png file path (slot scan + accumulation — avoid for viewer loops).
• lodepng include in .cpp: #define LODEPNG_NO_COMPILE_CPP before #include "src/libs/lodepng/lodepng.h", otherwise its C++ std::vector overloads collide with the C declarations (conflicting types for 'encode').
• MCP includes and settings::mcp reads are #ifdef ESP_PLATFORM-gated; the sim has no McpSystem.

Tailscale viewer

tailscale serve --bg --set-path=/simagent http://127.0.0.1:80/
# open: https://<node>/simagent/sim/

The serve target must be / (not /sim): the page resolves api/ against its own directory, so at /simagent/sim/ fetches go to /simagent/sim/api/…, which tailscale strips to /sim/api/… and the firmware's /sim/api/* aliases (GET+POST, registered in startServer()) handle. Absolute /api/… URLs would 404 at the edge (no /api mount there).

Simulator pitfalls

• Rebuild ≠ redeploy. cmake --build buildsim updates buildsim/ only. Re-run release-simulator.sh, restart the process, then retest. A stale /tmp/simrun/Tactility serves old handlers with new logs nowhere to be found.
• One simulator owns port 80. Do not launch a second instance while another simulator is listening: it will initialize LVGL but fail bind/listen, so its app API and viewer target the other process. Identify the listener with lsof -nP -iTCP:80 -sTCP:LISTEN, stop only the intended simulator, then release/restart it before installing or running POSIX apps.
• Input is cross-thread on macOS. SDL event pumping occurs on the real main thread; LVGL and web touch injection run elsewhere. Keep all shared pointer, key-queue, and touch-override state under sdl_input.cpp's mutex.
• C array sizeof decay. A helper like f(HttpServerRequest*, char uri[256]) sees sizeof(uri) == 8, truncating get_uri output to 7 chars (/api/sy, /sim/ap 404s). Pass the size explicitly: f(request, buf, sizeof(buf)).
• Log truncation. LOG_QUEUE_MESSAGE_MAX_LENGTH is 256 (TactilityKernel/ private/tactility/log_queue.h), including color/timestamp prefix. Long URIs and messages truncate — don't over-interpret a short path in the log.
• Auth. The viewer and API handlers enforce validateRequestAuth like any other endpoint; failures surface as 401/404, not viewer bugs.