# MCP System — Screensaver + WebServer + Video Streaming

## File Map
- `Private/Tactility/mcp/McpSystem.h` — global `McpSystemState` singleton
- `Source/mcp/McpSystem.cpp` — drawing (clear/text/RGB565/BMP/PBM), screenshot PBM base64, audio, sensors/battery/ble/SD file ops, video stream task, app listing
- `Source/service/webserver/McpHandler.cpp` — TOOLS_JSON, handle_rpc, handleApiMcp, handleApiScreenRaw
- `Source/service/webserver/WebServerService.cpp` — wildcard `/api/*` dispatches to MCP before auth, screenshot handler `/api/screenshot` (500ms timeout after fix)
- `Source/service/screenshot/ScreenshotTask.cpp` — `makeScreenshot` uses lvgl::lock(300ms after fix)
- `Source/service/displayidle/McpScreensaver.{h,cpp}` — canvas full-screen overlay, framebuffer SPIRAM, registers in McpSystemState
- `Include/Tactility/settings/McpSettings.h` + `Source/settings/McpSettings.cpp` — persisted at `getUserDataPath()+"/settings/mcp.properties"` = `/sdcard/tactility/...` on SD boards, `/data/tactility/...` internal. Old hardcoded `/data/service/mcp/settings.properties` bug fixed 2026-07-13.

## State Flow
1. WebServer `onStart()` loads `webServerSettings` + `mcpSettings` under `g_settingsMutex`, `serverEnabled = webEnabled || mcpEnabled`
2. `setEnabled(true)` → `startServer()` → `HttpServer(80, handlers, 12288)` → `httpd_start`
3. If `mcpEnabled` after HTTP up → `startVideoStreamServer()` → `xTaskCreatePinnedToCore(mcp_video_stream, 8192, prio5, core1)` allocates mono 15000 + color 153600 buffers SPIRAM
4. LLM draw → `drawRgb565/clear/drawText` → `ensureOverrideScreen()` → `displayidle::findService()->startMcpScreensaver()` → LVGL lock 200ms → create overlay canvas → register `drawArea`+`framebuffer`
5. Video stream client connects 8081/8083 → raw recv loop → `ensureOverrideScreen()` + `put_pixel` + `lv_obj_invalidate`

## Screensaver Integration (RLCD + MCP Persistence + Deadlock + Wake 50d86de4)

- Previously `DisplayIdle::onStart()` early returned on `!supportsBacklightDuty()` (ST7305) — killed timer + settings → MCP manual activation broken
- Now timer always starts, settings always loaded, `supportsBacklightDuty()` check only gates auto-backlight logic and duty calls in tick() and `startMcpScreensaver()`
- **Wake fix 50d86de4:** tick() timeout/wake logic must run for ALL displays, not gated by `supportsBacklight`. Only `setBacklightDuty()` guarded. Prevents RLCD locked state after dim.
- **Critical guard 6235fa05 — MCP flash-then-back <1s:** `tick()` every 50ms after `startMcpScreensaver()` sets `displayDimmed=true`, if `inactive_ms < 100ms` → `stopScreensaver()` deletes overlay. Fix: `bool isMcpActive = (drawArea != nullptr || overrideActive)` under mutex, guard all auto-stop `if (displayDimmed && !isMcpActive)`. Count `!isMcpActive>=3`.
- **Deadlock fix 3629ffef:** MCP video task locks `state.mutex -> lvgl`, tick must NOT do `lvgl -> mutex`. Read `isMcpActive` OUTSIDE LVGL lock.
- See `display-idle.md` for full tick flow final.

## RLCD 2-Button UX

User: "if i press the select button first it gets lock in there, so i need to press the change button first".

- Change GPIO18 KEY = LV_KEY_NEXT cycles focus
- Select GPIO0 BOOT = LV_KEY_ENTER activates
- No focus yet → Select does nothing → appears locked. Mitigation: auto-focus first btn on onShow.

## Serial Log Debugging — RLCD Crash Triage Recipe

```python
import serial
port="/dev/cu.usbmodem101"
ser=serial.Serial(port, 115200, timeout=1)
ser.dtr=False; ser.rts=False
buf=""
start=time.time()
while time.time()-start < 15:
  data=ser.read(2048)
  if data:
    txt=data.decode('utf-8', errors='replace')
    buf+=txt
    print(txt, end='')
ser.close()
open("/tmp/serial.log","w").write(buf)
```

Look for:
- ESP-ROM + rst:0x15 = normal hard reset after flash
- Tactility v0.8.0-dev on WaveShare ESP32-S3-RLCD-4.2 = correct target
- Monochrome/RLCD detected = RLCD timer fix applied
- Mono TCP Video stream server started 8081 + Color 8083 = video OK
- Guru Meditation / Backtrace / Stack canary / abort() = crash, decode via `xtensa-esp32s3-elf-addr2line -pfiaC -e Tactility.elf <addr>`
- `i2c: CONFLICT! driver_ng` = legacy i2c vs i2c_master both registered in platform-esp32, benign after migration to `esp32-i2c-master` DTS but still warns; ES8311 now initializes after fix
- File lock function not set = benign
- No crash 15s + AP started + launcher → freeze likely deadlock not crash, check lock order (3629ffef)
- `WaveshareRLCD: ES8311 codec initialized` = I2C master fix 3770ac89 works
- `Hal: ST7305 started` time: should be ~2160ms with 150ms reset delay, vs 1828ms before (no delay) — freeze on fast boot fixed

## Screenshot Lock Contention — "could not acquire LVGL lock" (fixed 2026-07-14 6f095081)

**API:** `GET /api/screenshot` in `WebServerService.cpp:1522` — `httpd_resp_send_err(..., "could not acquire LVGL lock")` — this is what MCP reports as "Screenshot failed: could not acquire LVGL lock"

**Root causes:**
- Handler used `lvgl::lock(100)` — RLCD ST7305 SPI 10MHz full_refresh 120k pixels ~100ms/copy, 100ms too short
- `ScreenshotTask::makeScreenshot` used 50ms
- `mcp_video_stream_task` loop 5ms streaming hogs LVGL lock → UI frozen appearance "screen is still showing the freezed image"

**Fixes 6f095081 (build 0x2b79e0):**
- `WebServerService.cpp` 100→500ms: `lvgl::lock(pdMS_TO_TICKS(500))` + error log "within 500ms"
- `ScreenshotTask.cpp` 50→300ms
- `McpSystem.cpp` video stream 50ms idle→100ms, 5ms streaming→30ms with comment "RLCD ST7305 SPI is slow and shared"
- DisplayIdle: for monochrome don't auto-enter screensaver (BouncingBalls heavy)
- ST7305 RAM retains frozen image until power cycle — RTS reset not enough, need USB unplug 3s
- Verification 14 checks PASS hermes-verify-jw_55oxm.py
- See also `references/rlcd-initialization.md` + `references/display-idle.md` 6f095081

## Video Stream Protocol
- Mono: TCP 8081, 15000-byte frames, each byte 8 monochrome pixels RLCD packing, unpacked black/white with x_off/y_off centering
- Color: TCP 8083, packet: 16-byte header `magic=RAW\x01 (4) + x BE u16 + y BE u16 + w BE u16 + h BE u16 + payload_len BE u32`, payload BE RGB565
- Stats: `framesDrawn`, `tcpBytesReceived`, `lastDrawMs`, `lastFps`, `overrideActive`

## Adding New MCP Tool
1. Add function in `McpSystem.h/cpp`
2. Add JSON schema entry in `TOOLS_JSON` in `McpHandler.cpp`
3. Add `if (strcmp(name,"my_tool")==0)` branch
4. PSRAM malloc handling — use `heap_caps_malloc(SPIRAM)` + fallback `malloc`, free with `heap_caps_free`

## Known Gotchas
- `psram_malloc` fallback: `heap_caps_malloc(..., SPIRAM)` → if null `malloc`, free must be `heap_caps_free`
- `base64_decode` uses `malloc` not PSRAM — limit checked via `MCP_JSON_BODY_LIMIT 256KB` and `MCP_RAW_BODY_LIMIT 512KB`
- `ensureOverrideScreen()` blocking `vTaskDelay(50)*10=500ms` poll waiting for canvas — must not be called from LVGL task or deadlock
- Audio `audioBusy` spin flag prevents concurrent play/record
- Screenshot API `/api/screenshot` requires `TT_FEATURE_SCREENSHOT_ENABLED` — disabled saves build size but breaks MCP screenshot tool? Check sdkconfig