tactility/.claude/skills/tactility-firmware-development/SKILL.md

name, description

name	description
tactility-firmware-development	Use when merging current Tactility firmware, preserving local ES3C28P/ES3C35P customizations, building/flashing ESP32 firmware, or validating external apps against it.

Tactility firmware development

Scope and success criteria

Use this skill for firmware upgrades, especially when upstream changes the app loader, dashboard API, SDK export, or device model. Before editing, record the active branch, git status --short, selected Devices/<id>, serial port, and the expected device identity. Preserve user work and local board customizations; do not reset or overwrite a dirty tree.

Success is not merely a successful flash: the board must boot, mount storage, return /api/sysinfo, expose required internal apps, and accept an external app rebuilt against this exact firmware.

Upgrade and customization audit

1. Fetch and compare the intended upstream branch before merging. Treat upstream/main as distinct from experimental release/IDF branches unless the task explicitly asks for one of those branches.
2. Merge without discarding local work (for example, git merge --autostash upstream/main after inspecting status). Resolve conflicts by preserving required ES3C28P and ES3C35P device definitions, partition selection, display/font customizations, and product features.
3. Compare the pre-upgrade customization commit against the resulting worktree. Check new app registrations, CMake/source inclusion, settings pages, web-server endpoints, and board-specific sdkconfig entries—not just files that happen to conflict.
4. Commit the resulting firmware customization as a focused, reviewable commit after git diff --check.

Multi-binary external-app compatibility

Upstream app packaging changed to bin/<platform>/<binary>.elf. The old elf/<platform>.elf archive layout produces a loader Not executable error.

• Manifest v0.2: bin/esp32s3/app.elf
• Manifest v0.3: bin/esp32s3/<app.0.binary>.elf

When this change arrives, update the companion app tool and rebuild every app from a fresh SDK exported from this firmware. Do not attempt to repair a packaged ELF on the SD card: its layout and ABI must both be regenerated.

Build and flash

Use the installed IDF 5.5 environment; host virtual environments can corrupt both Python dependencies and IDF tooling:

unset VIRTUAL_ENV PYTHONPATH PYTHONHOME
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
source /Users/adolforeyna/esp/esp-idf/export.sh

# Select/check the intended board before this step.
python device.py <device-id>
idf.py build
idf.py -p /dev/cu.usbmodemXXXX flash

Tactility/CMakeLists.txt uses a non-configure-dependent source glob. After adding a new .c/.cpp file, run idf.py reconfigure build; otherwise a linker error for a newly referenced symbol may only mean CMake has not discovered the source file yet.

Boot and feature acceptance

Capture serial at 115200 after flash or reset. Confirm the board name, SD-card mount, HTTP-server start, Wi-Fi address, and any feature-specific startup logs. Then query the resolved device:

curl -fsS http://<ip>/api/sysinfo
curl -fsS http://<ip>/api/apps

For MCP settings, verify the McpSettings internal app appears in /api/apps. When enabled, verify MCP stream startup logs. For a feature restored from an older customization, validate its registration path and persisted settings, not only its source files.

External-app acceptance gate

Use the companion app skill's exact-firmware wrapper:

cd /path/to/tactility_apps
scripts/build_current_firmware_app.sh Apps/MyApp --firmware /path/to/tactility

Inspect the resulting tar member path, install through port-80 dashboard API, launch it, and read serial. Required evidence is the loader's Loading .../bin/..., an ELF entry address, Task started, and app-specific startup logs. HTTP 200 or a package simply appearing in /api/apps is insufficient.

Host simulator (buildsim) + web viewer

The POSIX simulator runs the real firmware (LVGL, services, web server) on macOS/Linux with an SDL backend. macOS has no visible window (upstream .github/workflows/build-simulator.yml: "macOS simulator currently fails due to main thread requirement for rendering" — AppKit menu init must happen on the process main thread, but FreeRTOS-POSIX parks main in sigwait and runs everything on pthreads). The supported loop is headless + web viewer: screenshots render server-side regardless of any window.

Code locations:

• Devices/simulator/Source/module.cpp — display resolution + SIM_DISPLAY_W/H
• Devices/simulator/Source/drivers/sdl_display.{h,cpp} — SDL backend
• Devices/simulator/Source/drivers/sdl_input.{h,cpp} — pointer/key state + web touch-injection override
• Tactility/Source/service/webserver/WebServerService.cpp — /sim viewer, /sim/api/* aliases, POST /api/sim/touch, GET /api/screenshot?fast=
• Tactility/Private/Tactility/service/webserver/WebServerService.h — handler decls

Build and run

# one-time host deps (outside any ESP-IDF env)
mkdir -p /tmp/simbin && ln -sf "$(which python3)" /tmp/simbin/python
pip3 install --break-system-packages lark pyyaml   # devicetree compiler

cd /path/to/tactility
export PATH="/tmp/simbin:$PATH"
env -u ESP_IDF_VERSION -u IDF_PATH cmake -S . -B buildsim -DCMAKE_BUILD_TYPE=Release
env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target Tactility -j "$(sysctl -n hw.ncpu)"

# POSIX SDK for host apps (arm64)
env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target TactilityKernel lvgl minitar minmea \
  app-module crypt-module gps-module http-module lvgl-module lvgl-window-manager-module service-module
env -u ESP_IDF_VERSION -u IDF_PATH python3 Buildscripts/release-sdk-posix.py /tmp/sim-sdk

# release + headless run (MUST run from the firmware root: release-simulator.sh
# uses relative version.txt / Data paths)
sh Buildscripts/release-simulator.sh buildsim /tmp/simrun
(cd /tmp/simrun && SIM_DISPLAY_W=480 SIM_DISPLAY_H=320 SDL_VIDEODRIVER=dummy \
  nohup ./Tactility > /tmp/sim_web.log 2>&1 &)
curl -s --max-time 5 http://127.0.0.1/api/sysinfo | head -c 120

Display resolution: SIM_DISPLAY_W/H env (default 480x320 landscape, matching on-device screenshots). ES3C35P panel is 320x480 portrait in DTS but presents 480x320 landscape; ES3C28P is 320x240. The chosen geometry is logged as Simulator Sim display WxH. SDL_VIDEODRIVER=dummy is expected to log one SdlDisplay Failed to create SDL window: Couldn't find matching render driver line — LVGL still renders and screenshots work.

Web viewer, touch, screenshots

• GET /sim → 301 to /sim/ (trailing slash required so the page's relative api/ URLs resolve under /sim/). Viewer polls api/screenshot?fast=1 every 500 ms, footer shows live naturalWidth×naturalHeight, click/tap POSTs api/sim/touch?x=&y=.
• POST /api/sim/touch?x=123&y=456[&down=0|1] (also /sim/api/sim/touch via alias). Coordinates are LVGL logical pixels. down=1 (default) presses and auto-releases after 1500 ms (SIM_TOUCH_HOLD_MS in sdl_input.cpp), long enough for LVGL indev polls to register a click. Simulator-only: 404 on ESP32 (#ifndef ESP_PLATFORM).
• GET /api/screenshot?fast=1 (default): lv_snapshot_take (RGB888) → in-place BGR→RGB swap → lodepng_encode24 to memory → chunked HTTP. No filesystem touch, ~8 ms/shot. ?fast=0 keeps the legacy webscreenshotN.png file path (slot scan + accumulation — avoid for viewer loops).
• lodepng include in .cpp: #define LODEPNG_NO_COMPILE_CPP before #include "src/libs/lodepng/lodepng.h", otherwise its C++ std::vector overloads collide with the C declarations (conflicting types for 'encode').
• MCP includes and settings::mcp reads are #ifdef ESP_PLATFORM-gated; the sim has no McpSystem.

Tailscale viewer

tailscale serve --bg --set-path=/simagent http://127.0.0.1:80/
# open: https://<node>/simagent/sim/

The serve target must be / (not /sim): the page resolves api/ against its own directory, so at /simagent/sim/ fetches go to /simagent/sim/api/…, which tailscale strips to /sim/api/… and the firmware's /sim/api/* aliases (GET+POST, registered in startServer()) handle. Absolute /api/… URLs would 404 at the edge (no /api mount there).

Simulator pitfalls

• Rebuild ≠ redeploy. cmake --build buildsim updates buildsim/ only. Re-run release-simulator.sh, restart the process, then retest. A stale /tmp/simrun/Tactility serves old handlers with new logs nowhere to be found.
• C array sizeof decay. A helper like f(HttpServerRequest*, char uri[256]) sees sizeof(uri) == 8, truncating get_uri output to 7 chars (/api/sy, /sim/ap 404s). Pass the size explicitly: f(request, buf, sizeof(buf)).
• Log truncation. LOG_QUEUE_MESSAGE_MAX_LENGTH is 256 (TactilityKernel/ private/tactility/log_queue.h), including color/timestamp prefix. Long URIs and messages truncate — don't over-interpret a short path in the log.
• Auth. The viewer and API handlers enforce validateRequestAuth like any other endpoint; failures surface as 401/404, not viewer bugs.