--- name: tactility-firmware-development description: Use when merging current Tactility firmware, preserving local ES3C28P/ES3C35P customizations, building/flashing ESP32 firmware, or validating external apps against it. --- # Tactility firmware development ## Scope and success criteria Use this skill for firmware upgrades, especially when upstream changes the app loader, dashboard API, SDK export, or device model. Before editing, record the active branch, `git status --short`, selected `Devices/`, serial port, and the expected device identity. Preserve user work and local board customizations; do not reset or overwrite a dirty tree. Success is not merely a successful flash: the board must boot, mount storage, return `/api/sysinfo`, expose required internal apps, and accept an external app rebuilt against this exact firmware. ## Upgrade and customization audit 1. Fetch and compare the intended upstream branch before merging. Treat `upstream/main` as distinct from experimental release/IDF branches unless the task explicitly asks for one of those branches. 2. Merge without discarding local work (for example, `git merge --autostash upstream/main` after inspecting status). Resolve conflicts by preserving required ES3C28P and ES3C35P device definitions, partition selection, display/font customizations, and product features. 3. Compare the pre-upgrade customization commit against the resulting worktree. Check new app registrations, CMake/source inclusion, settings pages, web-server endpoints, and board-specific `sdkconfig` entries—not just files that happen to conflict. 4. Commit the resulting firmware customization as a focused, reviewable commit after `git diff --check`. ## Multi-binary external-app compatibility Upstream app packaging changed to `bin//.elf`. The old `elf/.elf` archive layout produces a loader `Not executable` error. - Manifest v0.2: `bin/esp32s3/app.elf` - Manifest v0.3: `bin/esp32s3/.elf` When this change arrives, update the companion app tool and rebuild every app from a fresh SDK exported from this firmware. Do not attempt to repair a packaged ELF on the SD card: its layout and ABI must both be regenerated. ## Build and flash Use the installed IDF 5.5 environment; host virtual environments can corrupt both Python dependencies and IDF tooling: ```zsh unset VIRTUAL_ENV PYTHONPATH PYTHONHOME export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env source /Users/adolforeyna/esp/esp-idf/export.sh # Select/check the intended board before this step. python device.py idf.py build idf.py -p /dev/cu.usbmodemXXXX flash ``` `Tactility/CMakeLists.txt` uses a non-configure-dependent source glob. After adding a new `.c`/`.cpp` file, run `idf.py reconfigure build`; otherwise a linker error for a newly referenced symbol may only mean CMake has not discovered the source file yet. ## Boot and feature acceptance Capture serial at 115200 after flash or reset. Confirm the board name, SD-card mount, HTTP-server start, Wi-Fi address, and any feature-specific startup logs. Then query the resolved device: ```zsh curl -fsS http:///api/sysinfo curl -fsS http:///api/apps ``` For MCP settings, verify the `McpSettings` internal app appears in `/api/apps`. When enabled, verify MCP stream startup logs. For a feature restored from an older customization, validate its registration path and persisted settings, not only its source files. ## External-app acceptance gate Use the companion app skill's exact-firmware wrapper: ```zsh cd /path/to/tactility_apps scripts/build_current_firmware_app.sh Apps/MyApp --firmware /path/to/tactility ``` Inspect the resulting tar member path, install through port-80 dashboard API, launch it, and read serial. Required evidence is the loader's `Loading .../bin/...`, an ELF entry address, `Task started`, and app-specific startup logs. HTTP 200 or a package simply appearing in `/api/apps` is insufficient. ## Host simulator (buildsim) + web viewer The POSIX simulator runs the real firmware (LVGL, services, web server) on macOS/Linux with an SDL backend. On current firmware, `Main.cpp` runs SDL's event loop on the process main thread while FreeRTOS runs on a separate thread. This is required by AppKit and supports a native macOS window as well as the web viewer. `SDL_VIDEODRIVER=dummy` remains useful for headless automation. Code locations: - `Devices/simulator/Source/module.cpp` — display resolution + `SIM_DISPLAY_W/H` - `Devices/simulator/Source/drivers/sdl_display.{h,cpp}` — SDL backend - `Devices/simulator/Source/drivers/sdl_input.{h,cpp}` — pointer/key state + web touch-injection override - `Tactility/Source/service/webserver/WebServerService.cpp` — `/sim` viewer, `/sim/api/*` aliases, `POST /api/sim/touch`, `GET /api/screenshot?fast=` - `Tactility/Private/Tactility/service/webserver/WebServerService.h` — handler decls ### Build and run ```zsh # one-time host deps (outside any ESP-IDF env) mkdir -p /tmp/simbin && ln -sf "$(which python3)" /tmp/simbin/python pip3 install --break-system-packages lark pyyaml # devicetree compiler cd /path/to/tactility export PATH="/tmp/simbin:$PATH" env -u ESP_IDF_VERSION -u IDF_PATH cmake -S . -B buildsim -DCMAKE_BUILD_TYPE=Release env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target Tactility -j "$(sysctl -n hw.ncpu)" # POSIX SDK for host apps (arm64) env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target TactilityKernel lvgl minitar minmea \ app-module crypt-module gps-module http-module lvgl-module lvgl-window-manager-module service-module env -u ESP_IDF_VERSION -u IDF_PATH python3 Buildscripts/release-sdk-posix.py /tmp/sim-sdk # release (MUST run from the firmware root: release-simulator.sh uses relative # version.txt / Data paths) sh Buildscripts/release-simulator.sh buildsim /tmp/simrun # Native macOS window + web API. Run this from the release directory because # data/ and system/ are relative to the process working directory. (cd /tmp/simrun && SIM_DISPLAY_W=480 SIM_DISPLAY_H=320 \ nohup ./Tactility > /tmp/sim_gui.log 2>&1 &) # For CI/headless operation, set SDL_VIDEODRIVER=dummy instead. curl -s --max-time 5 http://127.0.0.1/api/sysinfo | head -c 120 ``` Display resolution: `SIM_DISPLAY_W/H` env (default **480x320 landscape**, matching on-device screenshots). ES3C35P panel is 320x480 portrait in DTS but presents 480x320 landscape; ES3C28P is 320x240. The chosen geometry is logged as `Simulator Sim display WxH`. `SDL_VIDEODRIVER=dummy` is expected to log one `SdlDisplay Failed to create SDL window: Couldn't find matching render driver` line — LVGL still renders and screenshots work. A native macOS launch must not emit that line. ### Web viewer, touch, screenshots - `GET /sim` → 301 to `/sim/` (trailing slash required so the page's relative `api/` URLs resolve under `/sim/`). Viewer polls `api/screenshot?fast=1` every 500 ms, footer shows live `naturalWidth×naturalHeight`, click/tap POSTs `api/sim/touch?x=&y=`. - `POST /api/sim/touch?x=123&y=456[&down=0|1]` (also `/sim/api/sim/touch` via alias). Coordinates are LVGL logical pixels. `down=1` (default) presses and **auto-releases after 1500 ms** (`SIM_TOUCH_HOLD_MS` in `sdl_input.cpp`), long enough for LVGL indev polls to register a click. Simulator-only: 404 on ESP32 (`#ifndef ESP_PLATFORM`). - `GET /api/screenshot?fast=1` (default): `lv_snapshot_take` (RGB888) → in-place BGR→RGB swap → `lodepng_encode24` **to memory** → chunked HTTP. No filesystem touch, ~8 ms/shot. `?fast=0` keeps the legacy `webscreenshotN.png` file path (slot scan + accumulation — avoid for viewer loops). - lodepng include in `.cpp`: `#define LODEPNG_NO_COMPILE_CPP` before `#include "src/libs/lodepng/lodepng.h"`, otherwise its C++ `std::vector` overloads collide with the C declarations (`conflicting types for 'encode'`). - MCP includes and `settings::mcp` reads are `#ifdef ESP_PLATFORM`-gated; the sim has no `McpSystem`. ### Tailscale viewer ```zsh tailscale serve --bg --set-path=/simagent http://127.0.0.1:80/ # open: https:///simagent/sim/ ``` The serve target must be `/` (not `/sim`): the page resolves `api/` against its own directory, so at `/simagent/sim/` fetches go to `/simagent/sim/api/…`, which tailscale strips to `/sim/api/…` and the firmware's `/sim/api/*` aliases (GET+POST, registered in `startServer()`) handle. Absolute `/api/…` URLs would 404 at the edge (no `/api` mount there). ### Simulator pitfalls - **Rebuild ≠ redeploy.** `cmake --build buildsim` updates `buildsim/` only. Re-run `release-simulator.sh`, restart the process, then retest. A stale `/tmp/simrun/Tactility` serves old handlers with new logs nowhere to be found. - **One simulator owns port 80.** Do not launch a second instance while another simulator is listening: it will initialize LVGL but fail `bind/listen`, so its app API and viewer target the other process. Identify the listener with `lsof -nP -iTCP:80 -sTCP:LISTEN`, stop only the intended simulator, then release/restart it before installing or running POSIX apps. - **Input is cross-thread on macOS.** SDL event pumping occurs on the real main thread; LVGL and web touch injection run elsewhere. Keep all shared pointer, key-queue, and touch-override state under `sdl_input.cpp`'s mutex. - **C array `sizeof` decay.** A helper like `f(HttpServerRequest*, char uri[256])` sees `sizeof(uri) == 8`, truncating `get_uri` output to 7 chars (`/api/sy`, `/sim/ap` 404s). Pass the size explicitly: `f(request, buf, sizeof(buf))`. - **Log truncation.** `LOG_QUEUE_MESSAGE_MAX_LENGTH` is 256 (`TactilityKernel/ private/tactility/log_queue.h`), including color/timestamp prefix. Long URIs and messages truncate — don't over-interpret a short path in the log. - **Auth.** The viewer and API handlers enforce `validateRequestAuth` like any other endpoint; failures surface as 401/404, not viewer bugs.