feat(sim): web viewer /sim, touch injection, fast screenshot, SIM_DISPLAY_W/H
This commit is contained in:
@@ -95,3 +95,104 @@ Inspect the resulting tar member path, install through port-80 dashboard API,
|
||||
launch it, and read serial. Required evidence is the loader's `Loading
|
||||
.../bin/...`, an ELF entry address, `Task started`, and app-specific startup
|
||||
logs. HTTP 200 or a package simply appearing in `/api/apps` is insufficient.
|
||||
|
||||
## Host simulator (buildsim) + web viewer
|
||||
|
||||
The POSIX simulator runs the real firmware (LVGL, services, web server) on
|
||||
macOS/Linux with an SDL backend. macOS has no visible window (upstream
|
||||
`.github/workflows/build-simulator.yml`: "macOS simulator currently fails due
|
||||
to main thread requirement for rendering" — AppKit menu init must happen on
|
||||
the process main thread, but FreeRTOS-POSIX parks main in `sigwait` and runs
|
||||
everything on pthreads). The supported loop is **headless + web viewer**:
|
||||
screenshots render server-side regardless of any window.
|
||||
|
||||
Code locations:
|
||||
|
||||
- `Devices/simulator/Source/module.cpp` — display resolution + `SIM_DISPLAY_W/H`
|
||||
- `Devices/simulator/Source/drivers/sdl_display.{h,cpp}` — SDL backend
|
||||
- `Devices/simulator/Source/drivers/sdl_input.{h,cpp}` — pointer/key state +
|
||||
web touch-injection override
|
||||
- `Tactility/Source/service/webserver/WebServerService.cpp` — `/sim` viewer,
|
||||
`/sim/api/*` aliases, `POST /api/sim/touch`, `GET /api/screenshot?fast=`
|
||||
- `Tactility/Private/Tactility/service/webserver/WebServerService.h` — handler decls
|
||||
|
||||
### Build and run
|
||||
|
||||
```zsh
|
||||
# one-time host deps (outside any ESP-IDF env)
|
||||
mkdir -p /tmp/simbin && ln -sf "$(which python3)" /tmp/simbin/python
|
||||
pip3 install --break-system-packages lark pyyaml # devicetree compiler
|
||||
|
||||
cd /path/to/tactility
|
||||
export PATH="/tmp/simbin:$PATH"
|
||||
env -u ESP_IDF_VERSION -u IDF_PATH cmake -S . -B buildsim -DCMAKE_BUILD_TYPE=Release
|
||||
env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target Tactility -j "$(sysctl -n hw.ncpu)"
|
||||
|
||||
# POSIX SDK for host apps (arm64)
|
||||
env -u ESP_IDF_VERSION -u IDF_PATH cmake --build buildsim --target TactilityKernel lvgl minitar minmea \
|
||||
app-module crypt-module gps-module http-module lvgl-module lvgl-window-manager-module service-module
|
||||
env -u ESP_IDF_VERSION -u IDF_PATH python3 Buildscripts/release-sdk-posix.py /tmp/sim-sdk
|
||||
|
||||
# release + headless run (MUST run from the firmware root: release-simulator.sh
|
||||
# uses relative version.txt / Data paths)
|
||||
sh Buildscripts/release-simulator.sh buildsim /tmp/simrun
|
||||
(cd /tmp/simrun && SIM_DISPLAY_W=480 SIM_DISPLAY_H=320 SDL_VIDEODRIVER=dummy \
|
||||
nohup ./Tactility > /tmp/sim_web.log 2>&1 &)
|
||||
curl -s --max-time 5 http://127.0.0.1/api/sysinfo | head -c 120
|
||||
```
|
||||
|
||||
Display resolution: `SIM_DISPLAY_W/H` env (default **480x320 landscape**,
|
||||
matching on-device screenshots). ES3C35P panel is 320x480 portrait in DTS but
|
||||
presents 480x320 landscape; ES3C28P is 320x240. The chosen geometry is logged
|
||||
as `Simulator Sim display WxH`. `SDL_VIDEODRIVER=dummy` is expected to log one
|
||||
`SdlDisplay Failed to create SDL window: Couldn't find matching render
|
||||
driver` line — LVGL still renders and screenshots work.
|
||||
|
||||
### Web viewer, touch, screenshots
|
||||
|
||||
- `GET /sim` → 301 to `/sim/` (trailing slash required so the page's relative
|
||||
`api/` URLs resolve under `/sim/`). Viewer polls `api/screenshot?fast=1`
|
||||
every 500 ms, footer shows live `naturalWidth×naturalHeight`, click/tap
|
||||
POSTs `api/sim/touch?x=&y=`.
|
||||
- `POST /api/sim/touch?x=123&y=456[&down=0|1]` (also `/sim/api/sim/touch` via
|
||||
alias). Coordinates are LVGL logical pixels. `down=1` (default) presses and
|
||||
**auto-releases after 1500 ms** (`SIM_TOUCH_HOLD_MS` in `sdl_input.cpp`),
|
||||
long enough for LVGL indev polls to register a click. Simulator-only: 404 on
|
||||
ESP32 (`#ifndef ESP_PLATFORM`).
|
||||
- `GET /api/screenshot?fast=1` (default): `lv_snapshot_take` (RGB888) → in-place
|
||||
BGR→RGB swap → `lodepng_encode24` **to memory** → chunked HTTP. No filesystem
|
||||
touch, ~8 ms/shot. `?fast=0` keeps the legacy `webscreenshotN.png` file path
|
||||
(slot scan + accumulation — avoid for viewer loops).
|
||||
- lodepng include in `.cpp`: `#define LODEPNG_NO_COMPILE_CPP` before
|
||||
`#include "src/libs/lodepng/lodepng.h"`, otherwise its C++ `std::vector`
|
||||
overloads collide with the C declarations (`conflicting types for 'encode'`).
|
||||
- MCP includes and `settings::mcp` reads are `#ifdef ESP_PLATFORM`-gated; the
|
||||
sim has no `McpSystem`.
|
||||
|
||||
### Tailscale viewer
|
||||
|
||||
```zsh
|
||||
tailscale serve --bg --set-path=/simagent http://127.0.0.1:80/
|
||||
# open: https://<node>/simagent/sim/
|
||||
```
|
||||
|
||||
The serve target must be `/` (not `/sim`): the page resolves `api/` against
|
||||
its own directory, so at `/simagent/sim/` fetches go to `/simagent/sim/api/…`,
|
||||
which tailscale strips to `/sim/api/…` and the firmware's `/sim/api/*`
|
||||
aliases (GET+POST, registered in `startServer()`) handle. Absolute `/api/…`
|
||||
URLs would 404 at the edge (no `/api` mount there).
|
||||
|
||||
### Simulator pitfalls
|
||||
|
||||
- **Rebuild ≠ redeploy.** `cmake --build buildsim` updates `buildsim/` only.
|
||||
Re-run `release-simulator.sh`, restart the process, then retest. A stale
|
||||
`/tmp/simrun/Tactility` serves old handlers with new logs nowhere to be found.
|
||||
- **C array `sizeof` decay.** A helper like
|
||||
`f(HttpServerRequest*, char uri[256])` sees `sizeof(uri) == 8`, truncating
|
||||
`get_uri` output to 7 chars (`/api/sy`, `/sim/ap` 404s). Pass the size
|
||||
explicitly: `f(request, buf, sizeof(buf))`.
|
||||
- **Log truncation.** `LOG_QUEUE_MESSAGE_MAX_LENGTH` is 256 (`TactilityKernel/
|
||||
private/tactility/log_queue.h`), including color/timestamp prefix. Long URIs
|
||||
and messages truncate — don't over-interpret a short path in the log.
|
||||
- **Auth.** The viewer and API handlers enforce `validateRequestAuth` like any
|
||||
other endpoint; failures surface as 401/404, not viewer bugs.
|
||||
|
||||
Reference in New Issue
Block a user