Migrate MCP and utilities to CircuitPython, including color GIF and volume fixes

.gitignore

@@ -1,3 +1,13 @@
+# Local-only CircuitPython configuration / secrets
+settings.toml
+.env
+
+# Firmware downloads and generated bundles
+firmware/
+*.bin
+*.uf2
+*.zip
+
 # Temporary generated media and screenshots
 *.pbm
 *.png
@@ -8,5 +18,23 @@
 __pycache__/
 *.pyc
 
+# iPhone app build output
+iphone_app/.manual_build/
+iphone_app/.manual_pkg/
+iphone_app/.theos/
+iphone_app/packages/
+
 # OS generated files
 .DS_Store
+
+# Log files
+desktop_client/*.log
+
+# Helper scripts and datasheets
+deploy_main_serial.py
+dtr_reset.py
+hard_reset.py
+pcm2wav.py
+plot_audio.py
+*.pdf
+

README.md

@@ -2,6 +2,8 @@
 
 This repository contains the MicroPython firmware and host-side bridge script to expose the **Waveshare ESP32-S3-RLCD-4.2** development board as a physical, Model Context Protocol (MCP)-compliant agentic companion. 
 
+It also includes a native jailbroken iPhone 6 target in [`iphone_app/`](iphone_app/README.md). The iPhone implementation exposes the same MCP-style screen, audio, camera, file, and monochrome streaming features, plus RGB565 color streaming and Hermes voice turns.
+
 With this setup, local AI harnesses (such as Claude Desktop, OpenClaude, Cursor, or Hermes) can directly control the board's screen, NeoPixels, audio codec, microphone array, sensors, and remote filesystem.
 
 ---
@@ -43,17 +45,21 @@ To bypass the memory and protocol constraints of the microcontroller, we use a h
 *   **[boot.py](boot.py)**: Automatically connects to Wi-Fi using credentials in `wifi_config.py` and renders connection logs on the screen.
 *   **[main.py](main.py)**: Background execution loop. Handlers for buttons (manual dashboard refresh, LED state cycle), NeoPixel animation loop, and non-blocking MCP server requests.
 *   **[mcp_server.py](mcp_server.py)**: The lightweight JSON-RPC server implementing the MCP tools list and call handlers.
-*   **[audio_util.py](audio_util.py)**: Audio drivers for the ES7210 microphone array and the ES8311 speaker DAC. Implements sine-wave tone generation (`play_tone`), voice recording (`record_audio`), and file-based audio playing (`play_wav`).
-*   **[rlcd.py](rlcd.py)**: Low-level FrameBuffer driver for the 4.2" Reflective LCD, including PBM loaders and screenshot exporters.
-*   **[sd_util.py](sd_util.py)**: Mounting and filesystem management utility for the onboard microSD card slot.
-*   **Hardware Drivers**: 
-    *   [shtc3_util.py](shtc3_util.py) (Temperature & Humidity)
-    *   [rtc_util.py](rtc_util.py) (Hardware Real-Time Clock)
-    *   [battery_util.py](battery_util.py) (Voltage & Capacity reader)
-    *   [rgb_led_util.py](rgb_led_util.py) (WS2812 NeoPixel animations)
-    *   [button_util.py](button_util.py) (Key & Boot button debouncer)
-    *   [ble_util.py](ble_util.py) (Passive BLE scanning & UART)
 *   **[wifi_config.py](wifi_config.py)**: Wi-Fi credentials. **(Do not commit real credentials to Git)**.
+*   **`lib/`**: Hardware drivers and utility modules automatically searched by MicroPython:
+    *   **[audio_util.py](lib/audio_util.py)**: Audio drivers for the ES7210 microphone array and the ES8311 speaker DAC. Implements sine-wave tone generation (`play_tone`), voice recording (`record_audio`), and file-based audio playing (`play_wav`).
+    *   **[rlcd.py](lib/rlcd.py)**: Low-level FrameBuffer driver for the 4.2" Reflective LCD, including PBM loaders and screenshot exporters.
+    *   **[sd_util.py](lib/sd_util.py)**: Mounting and filesystem management utility for the onboard microSD card slot.
+    *   **[shtc3_util.py](lib/shtc3_util.py)**: Temperature & Humidity sensor utility.
+    *   **[rtc_util.py](lib/rtc_util.py)**: PCF85063 Hardware Real-Time Clock driver.
+    *   **[battery_util.py](lib/battery_util.py)**: Battery voltage and capacity reader.
+    *   **[rgb_led_util.py](lib/rgb_led_util.py)**: WS2812 NeoPixel animation manager.
+    *   **[button_util.py](lib/button_util.py)**: Boot/Key button debouncer utility.
+    *   **[ble_util.py](lib/ble_util.py)**: Passive BLE scanning and BLE UART interface.
+    *   **[ft6336u.py](lib/ft6336u.py)**: I2C Capacitive touchscreen controller driver.
+    *   **[ili9341.py](lib/ili9341.py)**: ILI9341 LCD driver.
+    *   **[st7796.py](lib/st7796.py)**: ST7796 LCD driver.
+    *   **[download_util.py](lib/download_util.py)**: Helper for streaming downloads to the local filesystem.
 
 ### Host-Side files
 *   **[mcp_bridge.py](mcp_bridge.py)**: The stdio-to-HTTP LAN bridge connecting the LLM client to the board. Handles image resizing, dithering, and formatting.
@@ -133,3 +139,9 @@ Here is a summary of the MCP tools exposed by the bridge:
 | `write_file` | `path`, `content` | Writes text file to board flash. |
 | `read_file` | `path` | Reads text file from board flash. |
 | `execute_python` | `code` | Executes arbitrary Python code dynamically. |
+
+---
+
+## 5. Hardware & Troubleshooting Findings
+
+For detailed technical findings regarding the board's hardware (e.g. I2S bit-depth configuration, I2C bus lockup recovery, pinout configuration, and ES7210 microphone clock/OSR registers to resolve popping sound issues), refer to [hardware_findings.md](hardware_findings.md).

boot.py

@@ -1,41 +1,34 @@
 # This file is executed on every boot (including wake-boot from deepsleep)
 import time
 import sys
+import board_config
+import wifi_config
 
 try:
     import network
     has_network = True
 except ImportError:
     has_network = False
-if sys.platform == 'rp2':
-    import st7796 as display_module
-else:
-    import rlcd as display_module
-from machine import Pin, SPI
-import wifi_config
 
 def connect_wifi():
     if sys.platform == 'rp2':
         # Skip display and connection setup on RP2 (no network/Wi-Fi hardware)
-        # Keep the sleep window to make REPL interruption easy
         time.sleep(1.5)
         return
 
-    # Initialize display to show connection progress
-    display = None
+    # Use the pre-initialized display from board_config
+    display = board_config.display_instance
+    if display:
         try:
-        # ESP32-S3 configuration
-        spi = SPI(1, baudrate=20000000, polarity=0, phase=0, sck=Pin(11), mosi=Pin(12))
-        display = display_module.RLCD(spi, cs=Pin(40), dc=Pin(5), rst=Pin(41))
             display.clear(0)
-        
-        display.text("ESP32-S3-RLCD-4.2", 10, 10, 1)
-        display.line(10, 20, 390, 20, 1)
+            board_name = "ESP32-S3 " + str(board_config.BOARD_TYPE)
+            display.text(board_name, 10, 10, 1)
+            display.line(10, 20, display.width - 10, 20, 1)
             display.text("Connecting to Wi-Fi...", 10, 35, 1)
             display.text(f"SSID: {wifi_config.WIFI_SSID}", 10, 50, 1)
             display.show()
         except Exception as e:
-        print("Display init failed in boot.py:", e)
+            print("Display setup failed in boot.py:", e)
 
     # Initialize Wi-Fi Station
     if not has_network:
@@ -62,34 +55,45 @@ def connect_wifi():
             timeout -= 1
             print("Connecting...")
             if display:
+                try:
                     display.text(".", dot_x, 70, 1)
                     dot_x += 10
                     display.show()
+                except:
+                    pass
                 
     if wlan.isconnected():
         ip = wlan.ifconfig()[0]
         print(f"Wi-Fi Connected! IP Address: {ip}")
         if display:
+            try:
                 display.clear(0)
-            display.text("ESP32-S3-RLCD-4.2", 10, 10, 1)
-            display.line(10, 20, 390, 20, 1)
+                board_name = "ESP32-S3 " + str(board_config.BOARD_TYPE)
+                display.text(board_name, 10, 10, 1)
+                display.line(10, 20, display.width - 10, 20, 1)
                 display.text("Wi-Fi Status: Connected!", 10, 35, 1)
                 display.text(f"SSID: {wifi_config.WIFI_SSID}", 10, 50, 1)
                 display.text(f"IP:   {ip}", 10, 65, 1)
                 display.text("Starting MCP Server...", 10, 90, 1)
                 display.show()
                 time.sleep(1.5)
+            except:
+                pass
     else:
         print("Wi-Fi Connection Failed.")
         if display:
+            try:
                 display.clear(0)
-            display.text("ESP32-S3-RLCD-4.2", 10, 10, 1)
-            display.line(10, 20, 390, 20, 1)
-            display.text("Wi-Fi Status: Connection Failed!", 10, 35, 1)
+                board_name = "ESP32-S3 " + str(board_config.BOARD_TYPE)
+                display.text(board_name, 10, 10, 1)
+                display.line(10, 20, display.width - 10, 20, 1)
+                display.text("Wi-Fi Status: Failed!", 10, 35, 1)
                 display.text("Check credentials in:", 10, 55, 1)
                 display.text("wifi_config.py", 20, 70, 1)
                 display.text("Starting offline...", 10, 95, 1)
                 display.show()
                 time.sleep(2)
+            except:
+                pass
 
 connect_wifi()

circuitpython/README.md

@@ -0,0 +1,83 @@
+# CircuitPython migration prototype for Waveshare ESP32-S3-RLCD-4.2
+
+This folder is an initial CircuitPython port attempt for the existing MicroPython ESP32-S3-RLCD project.
+
+## Goal
+
+Move the board firmware toward CircuitPython so we can use built-in CircuitPython media modules:
+
+- `audiomp3` for MP3 playback over I2S
+- `gifio` for frame-by-frame GIF decoding
+
+The current MicroPython project already has working low-level knowledge for:
+
+- ST7305 reflective LCD init and 2x4 packed monochrome framebuffer
+- ES8311 speaker DAC over I2C + I2S
+- GPIO46 speaker amp enable
+- ESP32-S3 board pins
+
+## Current status
+
+This is not a full replacement firmware yet. It is a bootable/iterable prototype layout:
+
+- `code.py` — demo entrypoint for CircuitPython
+- `lib/rlcd_cp.py` — custom ST7305/RLCD driver using `busio`/`digitalio`
+- `lib/audio_cp.py` — ES8311 + I2S MP3 playback helper
+- `lib/gif_player.py` — GIF-to-RLCD playback helper using `gifio`
+
+## Why a custom display driver is needed
+
+Antigravity’s review confirmed the important constraint: ST7305/RLCD is not a normal `displayio` panel here. The existing `rlcd.py` packs pixels into 15,000 bytes where each byte represents a 2×4 pixel block, with vertical inversion. CircuitPython does not appear to have a native `displayio` ST7305 driver for this exact layout, so this port keeps a Python-side canvas and manually packs it before SPI transfer.
+
+## Hardware pins copied from the working MicroPython firmware
+
+Display SPI:
+
+- SCK: IO11
+- MOSI: IO12
+- CS: IO40
+- DC: IO5
+- RST: IO41
+
+Audio/I2C:
+
+- I2C SDA: IO13
+- I2C SCL: IO14
+- I2S BCLK: IO9
+- I2S LRCK/WS: IO45
+- I2S DOUT: IO8
+- MCLK PWM: IO16
+- Speaker amp enable: IO46
+
+## Copy to CIRCUITPY
+
+After flashing a compatible CircuitPython build for the ESP32-S3 N16R8-style board:
+
+```bash
+cp circuitpython/code.py /media/$USER/CIRCUITPY/code.py
+mkdir -p /media/$USER/CIRCUITPY/lib
+cp circuitpython/lib/*.py /media/$USER/CIRCUITPY/lib/
+```
+
+Optional test assets:
+
+```bash
+cp demo.mp3 /media/$USER/CIRCUITPY/demo.mp3
+cp demo.gif /media/$USER/CIRCUITPY/demo.gif
+```
+
+## Expected next hardware tests
+
+1. Boot with only `code.py` and libraries copied.
+2. Confirm display init + dashboard text appears.
+3. Copy a tiny 320px-wide-or-smaller monochrome/low-color GIF and test GIF playback. `gifio` cannot load 400px-wide GIFs on current CircuitPython builds; the prototype centers 320px GIFs on the 400px screen.
+4. Copy a short MP3 and test I2S/ES8311 playback.
+5. If display is scrambled, compare `pack()` output against MicroPython `rlcd.py` and host `notetaker.py` mapping.
+
+## Known risks
+
+- CircuitPython may not expose every `board.IO##` name on the selected build. If so, update `PINS` in `code.py` with the names present in `dir(board)`.
+- `pwmio.PWMOut` at 12.288 MHz on IO16 may not work on all CircuitPython ESP32-S3 builds. If MCLK fails, MP3 playback may be silent even if I2S is writing.
+- Pure Python display packing loops may be slow. This prototype favors correctness first; optimize with lookup tables after the first visible frame works.
+- GIF playback on a 1-bit reflective LCD will be low-framerate and monochrome-thresholded.
+- CircuitPython SPI configuration is lock-scoped, so `rlcd_cp.py` configures 20 MHz SPI after every lock.

circuitpython/code.py

@@ -0,0 +1,439 @@
+"""CircuitPython entrypoint for Waveshare ESP32-S3-RLCD-4.2 and Hosyond ESP32-S3 Touchscreen.
+
+Performs I2C scanning to auto-detect the connected board type, dynamically initializes
+all peripheral drivers, connects to Wi-Fi, and starts the MCP server and video stream servers.
+"""
+
+import gc
+import os
+import time
+
+import board
+import busio
+import digitalio
+import wifi
+import socketpool
+import adafruit_requests
+
+# Import drivers
+from audio_cp import BoardAudio
+from rlcd_cp import RLCD
+from ili9341_cp import ILI9341
+from ft6336u_cp import FT6336U
+from battery_cp import BatteryMonitor
+from shtc3_cp import SHTC3
+from rtc_cp import PCF85063
+from rgb_led_cp import BoardLED
+from button_cp import BoardButtons
+from ble_cp import BLEUART
+from video_stream_cp import VideoStreamServer
+from mcp_server_cp import MCPServer
+import wifi_config
+
+# Global variables for hardware
+board_type = None  # 'HOSYOND' or 'WAVESHARE_RLCD'
+display = None
+touch = None
+audio = None
+sensor = None
+rtc_chip = None
+battery = None
+led = None
+buttons = None
+ble_uart = None
+ip_addr = "Offline"
+
+def detect_and_init():
+    global board_type, display, touch, audio, sensor, rtc_chip, battery, led, buttons, ble_uart
+
+    print("Scanning for board type...")
+
+    # Try Hosyond pins first (SDA=IO16, SCL=IO15)
+    try:
+        i2c = busio.I2C(scl=board.IO15, sda=board.IO16)
+        while not i2c.try_lock():
+            pass
+        devices = i2c.scan()
+        i2c.unlock()
+        i2c.deinit()
+    except Exception as e:
+        devices = []
+
+    if 0x38 in devices:
+        print("Detected Board: HOSYOND (Color Screen + Touch)")
+        board_type = 'HOSYOND'
+
+        # Display: ILI9341 (320x240)
+        spi = busio.SPI(clock=board.IO12, MOSI=board.IO11)
+        display = ILI9341(
+            spi,
+            cs=digitalio.DigitalInOut(board.IO10),
+            dc=digitalio.DigitalInOut(board.IO46),
+            bl=board.IO45,
+            rst=None,
+        )
+
+        # Touch: FT6336U
+        i2c_bus = busio.I2C(scl=board.IO15, sda=board.IO16)
+        touch = FT6336U(
+            i2c_bus,
+            rst_pin=digitalio.DigitalInOut(board.IO18),
+            int_pin=digitalio.DigitalInOut(board.IO17),
+            width=320,
+            height=240,
+            swap_xy=True,
+            invert_x=False,
+            invert_y=True,
+        )
+
+        # Audio: ES8311
+        audio = BoardAudio(
+            i2c_bus,
+            bit_clock=board.IO5,
+            word_select=board.IO7,
+            data=board.IO8,
+            mclk=board.IO4,
+            amp=board.IO1,
+            amp_active_level=0,  # Active Low
+        )
+
+        battery = BatteryMonitor(board.IO9)
+        led = BoardLED(board.IO48)
+        buttons = BoardButtons(board.IO0, key_pin=None)
+        
+        ble_name = "ESP32-S3-Touch"
+        try:
+            ble_uart = BLEUART(name=ble_name)
+        except Exception as ble_err:
+            print(f"BLE init failed: {ble_err}")
+        return
+
+    # Try Waveshare RLCD pins (SDA=IO13, SCL=IO14)
+    try:
+        i2c = busio.I2C(scl=board.IO14, sda=board.IO13)
+        while not i2c.try_lock():
+            pass
+        devices = i2c.scan()
+        i2c.unlock()
+        i2c.deinit()
+    except Exception as e:
+        devices = []
+
+    if 0x70 in devices or 0x51 in devices:
+        print("Detected Board: WAVESHARE_RLCD (Monochrome RLCD)")
+        board_type = 'WAVESHARE_RLCD'
+
+        # Display: RLCD (400x300)
+        spi = busio.SPI(clock=board.IO11, MOSI=board.IO12)
+        display = RLCD(
+            spi,
+            cs=digitalio.DigitalInOut(board.IO40),
+            dc=digitalio.DigitalInOut(board.IO5),
+            rst=digitalio.DigitalInOut(board.IO41),
+        )
+
+        # Audio: ES8311
+        i2c_bus = busio.I2C(scl=board.IO14, sda=board.IO13)
+        audio = BoardAudio(
+            i2c_bus,
+            bit_clock=board.IO9,
+            word_select=board.IO45,
+            data=board.IO8,
+            mclk=board.IO16,
+            amp=board.IO46,
+            amp_active_level=1,  # Active High
+        )
+
+        battery = BatteryMonitor(board.IO9)
+        led = BoardLED(board.IO38)
+        buttons = BoardButtons(board.IO0, key_pin=board.IO47)
+
+        # Environment & Clock
+        try:
+            sensor = SHTC3(i2c_bus)
+        except Exception as e:
+            print("Failed to init SHTC3:", e)
+        try:
+            rtc_chip = PCF85063(i2c_bus)
+        except Exception as e:
+            print("Failed to init PCF85063:", e)
+
+        ble_name = "ESP32-S3-RLCD"
+        try:
+            ble_uart = BLEUART(name=ble_name)
+        except Exception as ble_err:
+            print(f"BLE init failed: {ble_err}")
+        return
+
+    # Fallback to Hosyond
+    print("Board scan failed. Falling back to HOSYOND defaults...")
+    board_type = 'HOSYOND'
+    spi = busio.SPI(clock=board.IO12, MOSI=board.IO11)
+    display = ILI9341(
+        spi,
+        cs=digitalio.DigitalInOut(board.IO10),
+        dc=digitalio.DigitalInOut(board.IO46),
+        bl=board.IO45,
+        rst=None,
+    )
+    i2c_bus = busio.I2C(scl=board.IO15, sda=board.IO16)
+    try:
+        touch = FT6336U(
+            i2c_bus,
+            rst_pin=digitalio.DigitalInOut(board.IO18),
+            int_pin=digitalio.DigitalInOut(board.IO17),
+            width=320,
+            height=240,
+            swap_xy=True,
+            invert_x=False,
+            invert_y=True,
+        )
+    except Exception as te:
+        print("Fallback touch init failed:", te)
+    audio = BoardAudio(
+        i2c_bus,
+        bit_clock=board.IO5,
+        word_select=board.IO7,
+        data=board.IO8,
+        mclk=board.IO4,
+        amp=board.IO1,
+        amp_active_level=0,
+    )
+    battery = BatteryMonitor(board.IO9)
+    led = BoardLED(board.IO48)
+    buttons = BoardButtons(board.IO0)
+    
+    ble_name = "ESP32-S3-Touch"
+    try:
+        ble_uart = BLEUART(name=ble_name)
+    except Exception as ble_err:
+        print(f"BLE init failed: {ble_err}")
+
+def main():
+    global ip_addr
+    detect_and_init()
+
+    # Draw initial dashboard
+    display.clear(0)
+    display.text("CircuitPython Active", 10, 10, 1)
+    display.text("Connecting Wi-Fi...", 10, 30, 1)
+    display.show()
+
+    # Sync system clock from RTC chip if available
+    if rtc_chip:
+        try:
+            rtc_chip.sync_to_system()
+        except Exception as e:
+            print("Failed to sync clock:", e)
+
+    # Connect to Wi-Fi
+    WIFI_SSID = getattr(wifi_config, "WIFI_SSID", "FamReynaMesh")
+    WIFI_PASS = getattr(wifi_config, "WIFI_PASS", "Gloria2020")
+    print(f"Connecting to Wi-Fi SSID '{WIFI_SSID}'...")
+    try:
+        wifi.radio.connect(WIFI_SSID, WIFI_PASS)
+        ip_addr = str(wifi.radio.ipv4_address)
+        print(f"Wi-Fi Connected! IP Address: {ip_addr}")
+    except Exception as wifi_err:
+        print(f"Wi-Fi Connection failed: {wifi_err}")
+        ip_addr = "Disconnected"
+
+    # Initialize socket pool and request session
+    pool = socketpool.SocketPool(wifi.radio)
+    session = adafruit_requests.Session(pool)
+
+    # Start Video Stream Server
+    vstream = VideoStreamServer(display, pool, tcp_port=8081, udp_port=8082, color_port=8083)
+    vstream.start()
+
+    # Start MCP Server
+    mcp = MCPServer(
+        display, led, battery, sensor, rtc_chip, ble_uart,
+        pool=pool, vstream=vstream, touch=touch, session=session, audio=audio
+    )
+    mcp.start(port=80)
+
+    # Sync time from NTP if connected
+    if ip_addr != "Disconnected":
+        try:
+            mcp._call_tool("sync_time", {})
+        except Exception as ntp_err:
+            print(f"NTP Time sync failed: {ntp_err}")
+
+    # Set default LED mode: Green Breathing
+    led_modes = [
+        ("Red (Breathing)", lambda l: l.set_color(40, 0, 0), "breath"),
+        ("Green (Breathing)", lambda l: l.set_color(0, 40, 0), "breath"),
+        ("Blue (Breathing)", lambda l: l.set_color(0, 0, 40), "breath"),
+        ("Cyan (Breathing)", lambda l: l.set_color(0, 30, 30), "breath"),
+        ("Magenta (Breathing)", lambda l: l.set_color(30, 0, 30), "breath"),
+        ("Rainbow Cycle", lambda l: l.set_color(30, 30, 30), "rainbow"),
+        ("LED Off", lambda l: l.off(), "off")
+    ]
+    local_led_mode_idx = 1 # Green breathing
+    led_modes[local_led_mode_idx][1](led)
+    mcp.active_led_mode = led_modes[local_led_mode_idx][2]
+
+    # Button callbacks
+    last_action_str = "Boot finished."
+    force_dashboard_redraw = True
+
+    def on_key_click():
+        nonlocal local_led_mode_idx, last_action_str, force_dashboard_redraw
+        local_led_mode_idx = (local_led_mode_idx + 1) % len(led_modes)
+        mode_name, color_fn, mode_type = led_modes[local_led_mode_idx]
+        color_fn(led)
+        mcp.active_led_mode = mode_type
+        print(f"Local Button: Cycle LED -> {mode_name}")
+        last_action_str = f"Local Button: {mode_name}"
+        mcp.override_active = False
+        force_dashboard_redraw = True
+
+    def on_boot_click():
+        nonlocal last_action_str, force_dashboard_redraw
+        print("Local Button: Force Dashboard refresh.")
+        last_action_str = "Dashboard Refreshed"
+        mcp.override_active = False
+        force_dashboard_redraw = True
+
+    if buttons:
+        buttons.boot.on_click(on_boot_click)
+        if buttons.key:
+            buttons.key.on_click(on_key_click)
+
+    last_dashboard_update = 0
+    dashboard_update_interval_s = 5
+    last_led_update = 0
+    last_wifi_check = 0
+
+    print("ESP32 CircuitPython main loop running...")
+
+    while True:
+        now = time.monotonic()
+
+        # A. Check Wi-Fi connection and reconnect if lost (every 10s)
+        if now - last_wifi_check >= 10.0:
+            last_wifi_check = now
+            if not wifi.radio.connected:
+                print("Wi-Fi connection lost. Attempting reconnect...")
+                last_action_str = "Wi-Fi Disconnected"
+                if ip_addr != "Disconnected":
+                    ip_addr = "Disconnected"
+                    force_dashboard_redraw = True
+                try:
+                    wifi.radio.connect(WIFI_SSID, WIFI_PASS)
+                except Exception as e:
+                    print("Wi-Fi reconnect trigger failed:", e)
+            elif ip_addr == "Disconnected":
+                ip_addr = str(wifi.radio.ipv4_address)
+                print(f"Wi-Fi Connected! IP Address: {ip_addr}")
+                last_action_str = f"Wi-Fi Connected: {ip_addr}"
+                force_dashboard_redraw = True
+                try:
+                    mcp._call_tool("sync_time", {})
+                except:
+                    pass
+
+        # B. Check for incoming MCP JSON-RPC TCP/UDP queries
+        mcp.update()
+
+        # C. Check for incoming Video Streaming TCP/UDP frames
+        vstream.update()
+
+        # D. Update button debouncers
+        if buttons:
+            buttons.update()
+
+        # E. Update BLE UART connection poll
+        if ble_uart:
+            ble_uart.update()
+
+        # F. Draw Local Dashboard periodically
+        # Do not draw if screen is overridden by MCP commands (e.g. draw_text, draw_image) or active video stream
+        if not mcp.override_active and not vstream.active:
+            if force_dashboard_redraw or (now - last_dashboard_update >= dashboard_update_interval_s):
+                force_dashboard_redraw = False
+                last_dashboard_update = now
+
+                # SHTC3 Sensor
+                t, h = sensor.read_sensor() if sensor else (None, None)
+                t_str = f"{t:.1f} C" if t is not None else "N/A"
+                h_str = f"{h:.1f} %" if h is not None else "N/A"
+
+                # Battery
+                bat_v = battery.read_voltage() if battery else None
+                bat_p = battery.read_percentage() if battery else 0
+                bat_str = f"{bat_v:.2f}V ({bat_p}%)" if bat_v is not None else "N/A"
+
+                # Time
+                dt = rtc_chip.get_datetime() if rtc_chip else None
+                if dt:
+                    time_str = f"{dt[0]:04d}-{dt[1]:02d}-{dt[2]:02d} {dt[4]:02d}:{dt[5]:02d}:{dt[6]:02d}"
+                else:
+                    time_str = "N/A (RTC Error)"
+
+                display.clear(0)
+                line_w = display.width - 10
+
+                if board_type == 'WAVESHARE_RLCD':
+                    title_text = "Waveshare ESP32-S3-RLCD Server"
+                    display.text(title_text, 10, 10, 1)
+                    display.line(10, 20, line_w, 20, 1)
+                    display.text_large("ENVIRONMENT", 15, 30, scale=2, color=1)
+                    display.text(f"Temp        : {t_str}", 25, 55, 1)
+                    display.text(f"Humid       : {h_str}", 25, 70, 1)
+                    display.line(10, 95, line_w, 95, 1)
+                    display.text_large("MCP NET CONNECTION", 15, 105, scale=2, color=1)
+                    display.text(f"IP Address  : {ip_addr}", 25, 130, 1)
+                    display.text(f"Port / Path : 80 /api/mcp", 25, 145, 1)
+                    display.text(f"BLE Name    : ESP32-S3-RLCD", 25, 160, 1)
+                    display.line(10, 185, line_w, 185, 1)
+                    display.text_large("SYSTEM STATUS", 15, 195, scale=2, color=1)
+                    display.text(f"Battery     : {bat_str}", 25, 220, 1)
+                    display.text(f"Time        : {time_str}", 25, 235, 1)
+                    display.line(10, 255, line_w, 255, 1)
+                    display.text(f"Status: {last_action_str}", 15, 265, 1)
+                else:
+                    title_text = "Hosyond ESP32-S3 Server"
+                    display.text(title_text, 10, 8, 1)
+                    display.line(10, 18, line_w, 18, 1)
+                    # Left Column
+                    display.text("SYSTEM & ENV", 10, 28, 1)
+                    display.line(10, 38, 150, 38, 1)
+                    display.text(f"Temp : {t_str}", 10, 46, 1)
+                    display.text(f"Hum  : {h_str}", 10, 58, 1)
+                    display.text(f"Bat  : {bat_str}", 10, 70, 1)
+                    display.text(f"Time : {time_str[11:19]}", 10, 82, 1)
+                    display.text(f"Date : {time_str[0:10]}", 10, 94, 1)
+                    # Right Column
+                    display.text("MCP NETWORK", 170, 28, 1)
+                    display.line(170, 38, line_w, 38, 1)
+                    display.text(f"IP : {ip_addr}", 170, 46, 1)
+                    display.text("Port: 80/api/mcp", 170, 58, 1)
+                    display.text("BLE : Touch", 170, 70, 1)
+                    # Bottom Status
+                    display.line(10, 115, line_w, 115, 1)
+                    display.text(f"Status: {last_action_str}", 10, 125, 1)
+
+                display.show()
+                gc.collect()
+
+        # G. Update NeoPixel LED animations smoothly (every 50ms)
+        if now - last_led_update >= 0.05:
+            last_led_update = now
+            mode = mcp.active_led_mode
+            if mode == "breath":
+                led.update_breathing(1.5)
+            elif mode == "rainbow":
+                led.update_rainbow(0.4)
+            elif mode == "off":
+                led.off()
+
+        # H. Sleep to prevent CPU hogging, poll faster if streaming is active
+        if vstream.active:
+            time.sleep(0.002)
+        else:
+            time.sleep(0.020)
+
+if __name__ == "__main__":
+    main()

circuitpython/deploy_only.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+set -euo pipefail
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+CIRCUITPY_MOUNT="${CIRCUITPY_MOUNT:-}"
+
+if [[ -z "$CIRCUITPY_MOUNT" ]]; then
+  for candidate in \
+    "/media/$USER/CIRCUITPY" \
+    "/run/media/$USER/CIRCUITPY" \
+    "/media/$(id -un)/CIRCUITPY" \
+    "/Volumes/CIRCUITPY"; do
+    if [[ -d "$candidate" ]]; then
+      CIRCUITPY_MOUNT="$candidate"
+      break
+    fi
+  done
+fi
+
+if [[ -z "$CIRCUITPY_MOUNT" || ! -d "$CIRCUITPY_MOUNT" ]]; then
+  echo "CIRCUITPY mount not found. Set CIRCUITPY_MOUNT=/path/to/CIRCUITPY" >&2
+  exit 1
+fi
+
+mkdir -p "$CIRCUITPY_MOUNT/lib"
+cp "$ROOT/circuitpython/code.py" "$CIRCUITPY_MOUNT/code.py"
+cp "$ROOT/circuitpython/lib/"*.py "$CIRCUITPY_MOUNT/lib/"
+cp "$ROOT/circuitpython/lib/"*.mpy "$CIRCUITPY_MOUNT/lib/"
+sync
+find "$CIRCUITPY_MOUNT" -maxdepth 2 -type f | sort

circuitpython/flash_and_deploy.sh

@@ -0,0 +1,74 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+PORT="${PORT:-/dev/cu.usbmodem101}"
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+FIRMWARE="$ROOT/firmware/adafruit-circuitpython-espressif_esp32s3_devkitc_1_n8r8-en_US-10.2.1.bin"
+CIRCUITPY_MOUNT="${CIRCUITPY_MOUNT:-}"
+
+if [[ ! -f "$FIRMWARE" ]]; then
+  echo "Firmware not found: $FIRMWARE" >&2
+  exit 1
+fi
+
+if [[ ! -r "$PORT" || ! -w "$PORT" ]]; then
+  cat >&2 <<EOF
+Cannot access $PORT as $(id -un).
+
+Temporary fix from a local terminal:
+  sudo chmod 666 $PORT
+
+Permanent fix:
+  sudo usermod -aG dialout $(id -un)
+  # then log out/in, or start a new shell with: newgrp dialout
+EOF
+  exit 2
+fi
+
+echo "==> Probing ESP32-S3 on $PORT"
+python3 -m esptool --chip esp32s3 --port "$PORT" flash_id
+
+echo "==> Erasing flash"
+python3 -m esptool --chip esp32s3 --port "$PORT" erase_flash
+
+echo "==> Flashing CircuitPython N8R8 firmware"
+python3 -m esptool --chip esp32s3 --port "$PORT" --baud 460800 write_flash -z 0x0 "$FIRMWARE"
+
+echo "==> Waiting for CIRCUITPY mount"
+for _ in $(seq 1 60); do
+  if [[ -n "$CIRCUITPY_MOUNT" && -d "$CIRCUITPY_MOUNT" ]]; then
+    break
+  fi
+  for candidate in \
+    "/media/$USER/CIRCUITPY" \
+    "/run/media/$USER/CIRCUITPY" \
+    "/media/$(id -un)/CIRCUITPY" \
+    "/Volumes/CIRCUITPY"; do
+    if [[ -d "$candidate" ]]; then
+      CIRCUITPY_MOUNT="$candidate"
+      break 2
+    fi
+  done
+  sleep 1
+done
+
+if [[ -z "$CIRCUITPY_MOUNT" || ! -d "$CIRCUITPY_MOUNT" ]]; then
+  cat >&2 <<EOF
+Flashing finished, but CIRCUITPY did not auto-mount.
+Reset the board once, then rerun just deployment with:
+  CIRCUITPY_MOUNT=/path/to/CIRCUITPY bash circuitpython/deploy_only.sh
+EOF
+  exit 3
+fi
+
+echo "==> Deploying prototype to $CIRCUITPY_MOUNT"
+mkdir -p "$CIRCUITPY_MOUNT/lib"
+cp "$ROOT/circuitpython/code.py" "$CIRCUITPY_MOUNT/code.py"
+cp "$ROOT/circuitpython/lib/"*.py "$CIRCUITPY_MOUNT/lib/"
+cp "$ROOT/circuitpython/lib/"*.mpy "$CIRCUITPY_MOUNT/lib/"
+sync
+
+echo "==> Deployed. Files on CIRCUITPY:"
+find "$CIRCUITPY_MOUNT" -maxdepth 2 -type f | sort
+
+echo "==> Done. Reset the board if it does not reload automatically."

circuitpython/lib/audio_cp.py

@@ -0,0 +1,210 @@
+"""CircuitPython audio helpers for Waveshare ESP32-S3-RLCD-4.2.
+
+Ports the ES8311 setup sequence from MicroPython audio_util.py and adds MP3
+playback using CircuitPython's audiomp3 + audiobusio stack.
+"""
+
+import time
+import array
+import math
+
+import audiobusio
+import audiomp3
+import audiocore
+import digitalio
+import pwmio
+
+
+
+class ES8311:
+    ADDR = 0x18
+
+    def __init__(self, i2c):
+        self.i2c = i2c
+
+    def _write(self, register, value):
+        # CircuitPython I2CDevice is not used to keep this dependency-free.
+        while not self.i2c.try_lock():
+            pass
+        try:
+            self.i2c.writeto(self.ADDR, bytes([register & 0xFF, value & 0xFF]))
+        finally:
+            self.i2c.unlock()
+
+    def init(self, sample_rate=44100):
+        """Initialize ES8311 for I2S DAC playback.
+
+        The register sequence is the existing MicroPython project's known-good
+        sequence. The original comments targeted 16 kHz WAV playback, but the
+        I2S peripheral is configured with the MP3 decoder sample rate at play
+        time. If MP3 output is silent, hardware-test this sequence first with a
+        16 kHz WAV/tone and then tune codec dividers for the MP3 sample rate.
+        """
+        del sample_rate  # Kept for API clarity; sequence currently fixed.
+        self._write(0x00, 0x1F)
+        time.sleep(0.01)
+        self._write(0x00, 0x00)
+        time.sleep(0.01)
+        self._write(0x01, 0x3F)
+        self._write(0x02, 0x48)
+        self._write(0x03, 0x10)
+        self._write(0x04, 0x20)
+        self._write(0x05, 0x00)
+        self._write(0x06, 0x03)
+        self._write(0x07, 0x00)
+        self._write(0x08, 0xFF)
+        self._write(0x09, 0x0C)
+        self._write(0x0A, 0x0C)
+        self._write(0x0D, 0x01)
+        self._write(0x0E, 0x02)
+        self._write(0x12, 0x00)
+        self._write(0x13, 0x10)
+        self._write(0x1C, 0x6A)
+        self._write(0x37, 0x08)
+        self._write(0x32, 0xBF)
+        self._write(0x31, 0x00)
+        self._write(0x00, 0x80)
+
+    def set_volume(self, volume):
+        volume = max(0, min(100, int(volume)))
+        if volume <= 0:
+            reg_val = 0
+        elif volume <= 50:
+            # 1..50 maps to 0..191 (0xBF = 0dB)
+            reg_val = int((volume / 50.0) * 191)
+        else:
+            # 51..100 maps to 192..255 (0xFF = +32dB)
+            reg_val = 191 + int(((volume - 50) / 50.0) * (255 - 191))
+        self._write(0x32, reg_val)
+
+
+class BoardAudio:
+    def __init__(self, i2c, *, bit_clock, word_select, data, mclk, amp, amp_active_level=1):
+        self.i2c = i2c
+        self.bit_clock_pin = bit_clock
+        self.word_select_pin = word_select
+        self.data_pin = data
+        self.mclk_pin = mclk
+        self.amp_pin = amp
+        self.amp_active_level = amp_active_level
+        self._mclk = None
+        self._amp = digitalio.DigitalInOut(amp)
+        self._amp.switch_to_output(value=not amp_active_level)
+        self.codec = ES8311(i2c)
+
+    def start_mclk(self):
+        if self._mclk is None:
+            # External MCLK for ES8311. CircuitPython I2SOut on ESP32-S3 does
+            # not accept a main_clock parameter, so keep this independent.
+            self._mclk = pwmio.PWMOut(
+                self.mclk_pin,
+                frequency=12288000,
+                duty_cycle=32768,
+                variable_frequency=False,
+              )
+
+    def stop_mclk(self):
+        if self._mclk is not None:
+            self._mclk.deinit()
+            self._mclk = None
+
+    def play_mp3(self, filename, volume=60):
+        """Play an MP3 file from CIRCUITPY storage over the onboard speaker."""
+        self.start_mclk()
+        self.codec.init()
+        self.codec.set_volume(volume)
+        decoder = None
+        audio = None
+        f = None
+        try:
+            f = open(filename, "rb")
+            decoder = audiomp3.MP3Decoder(f)
+            audio = audiobusio.I2SOut(
+                bit_clock=self.bit_clock_pin,
+                word_select=self.word_select_pin,
+                data=self.data_pin,
+            )
+            self._amp.value = self.amp_active_level
+            audio.play(decoder)
+            while audio.playing:
+                time.sleep(0.05)
+            return True
+        finally:
+            self._amp.value = not self.amp_active_level
+            if audio is not None:
+                audio.deinit()
+            if decoder is not None:
+                decoder.deinit()
+            if f is not None:
+                f.close()
+
+    def play_wav(self, filename, volume=60):
+        """Play a WAV file from CIRCUITPY storage over the onboard speaker."""
+        self.start_mclk()
+        self.codec.init()
+        self.codec.set_volume(volume)
+        wav = None
+        audio = None
+        f = None
+        try:
+            f = open(filename, "rb")
+            wav = audiocore.WaveFile(f)
+            audio = audiobusio.I2SOut(
+                bit_clock=self.bit_clock_pin,
+                word_select=self.word_select_pin,
+                data=self.data_pin,
+            )
+            self._amp.value = self.amp_active_level
+            audio.play(wav)
+            while audio.playing:
+                time.sleep(0.05)
+            return True
+        except Exception as e:
+            print(f"Error playing WAV: {e}")
+            return False
+        finally:
+            self._amp.value = not self.amp_active_level
+            if audio is not None:
+                audio.deinit()
+            if wav is not None:
+                wav.deinit()
+            if f is not None:
+                f.close()
+
+    def play_tone(self, frequency=440, duration_ms=1000, volume=50):
+        """Generate and play a pure sine wave tone on the speaker."""
+        self.start_mclk()
+        self.codec.init()
+        self.codec.set_volume(volume)
+
+        sample_rate = 16000
+        length = int(sample_rate / frequency)
+        if length < 4:
+            length = 4
+
+        sine_wave = array.array("h", [0] * length)
+        for i in range(length):
+            sine_wave[i] = int(math.sin(2 * math.pi * i / length) * 32767)
+
+        sample = audiocore.RawSample(sine_wave, sample_rate=sample_rate)
+        audio = None
+        try:
+            audio = audiobusio.I2SOut(
+                bit_clock=self.bit_clock_pin,
+                word_select=self.word_select_pin,
+                data=self.data_pin,
+            )
+            self._amp.value = self.amp_active_level
+            audio.play(sample, loop=True)
+            time.sleep(duration_ms / 1000.0)
+            audio.stop()
+            return True
+        except Exception as e:
+            print(f"Error playing tone: {e}")
+            return False
+        finally:
+            self._amp.value = not self.amp_active_level
+            if audio is not None:
+                audio.deinit()
+            sample.deinit()
+

circuitpython/lib/battery_cp.py

@@ -0,0 +1,42 @@
+"""CircuitPython Battery Monitor utility.
+
+Wraps analogio.AnalogIn to read voltage via a 2x divider and estimate capacity.
+"""
+
+import analogio
+import board
+
+class BatteryMonitor:
+    def __init__(self, pin=board.IO9):
+        self.adc = analogio.AnalogIn(pin)
+
+    def read_voltage(self):
+        try:
+            # AnalogIn value is 16-bit (0-65535). Map to 3.3V reference.
+            # Divider is 2x, so actual voltage is scale * 2.
+            raw = self.adc.value
+            voltage = (raw / 65535.0) * 3.3 * 2.0
+            return round(voltage, 3)
+        except Exception as e:
+            print(f"Error reading battery ADC: {e}")
+            return None
+
+    def read_percentage(self):
+        voltage = self.read_voltage()
+        if voltage is None:
+            return 0
+        v_min = 3.0
+        v_max = 4.2
+        if voltage <= v_min:
+            return 0
+        if voltage >= v_max:
+            return 100
+        pct = (voltage - v_min) / (v_max - v_min) * 100.0
+        return int(pct)
+
+    def get_status_summary(self):
+        v = self.read_voltage()
+        p = self.read_percentage()
+        if v is None:
+            return "Battery: Error"
+        return f"Battery: {v:.2f}V ({p}%)"

circuitpython/lib/ble_cp.py

@@ -0,0 +1,101 @@
+"""CircuitPython BLE UART utility.
+
+Wraps the adafruit_ble library to advertise Nordic UART Service and scan for nearby devices.
+"""
+
+import time
+from adafruit_ble import BLERadio
+from adafruit_ble.services.nordic import UARTService
+from adafruit_ble.advertising.standard import ProvideServicesAdvertisement
+
+class BLEUART:
+    def __init__(self, ble=None, name="ESP32-S3-RLCD"):
+        self.ble = BLERadio()
+        self.ble.name = name
+        self.uart = UARTService()
+        self.advertisement = ProvideServicesAdvertisement(self.uart)
+        self.rx_callback = None
+        self._is_advertising = False
+        self._start_advertise()
+
+    def _start_advertise(self):
+        if not self.ble.connected and not self._is_advertising:
+            try:
+                self.ble.start_advertising(self.advertisement)
+                self._is_advertising = True
+                print(f"BLE advertising started as '{self.ble.name}'")
+            except Exception as e:
+                print(f"Failed to start BLE advertising: {e}")
+
+    def on_rx(self, callback):
+        self.rx_callback = callback
+        return callback
+
+    def update(self):
+        """Polls the UART stream for incoming data and triggers callbacks."""
+        if self.ble.connected:
+            self._is_advertising = False
+            try:
+                if self.uart.in_waiting:
+                    data = self.uart.read(self.uart.in_waiting)
+                    if self.rx_callback and data:
+                        try:
+                            decoded = data.decode("utf-8").strip()
+                            self.rx_callback(decoded)
+                        except:
+                            self.rx_callback(data)
+            except Exception as e:
+                print(f"Error reading BLE RX: {e}")
+        else:
+            if not self._is_advertising:
+                self._start_advertise()
+
+    def write(self, data):
+        if not self.ble.connected:
+            return False
+        if isinstance(data, str):
+            data = data.encode("utf-8")
+        try:
+            self.uart.write(data)
+            return True
+        except Exception as e:
+            print(f"Error sending BLE data: {e}")
+            return False
+
+    def is_connected(self):
+        return self.ble.connected
+
+    def scan(self, duration_ms=3000):
+        """Scans for nearby BLE devices."""
+        duration_s = duration_ms / 1000.0
+        results = {}
+        was_advertising = self._is_advertising
+        if was_advertising:
+            try:
+                self.ble.stop_advertising()
+            except:
+                pass
+            self._is_advertising = False
+
+        try:
+            print("Starting BLE scan...")
+            for adv in self.ble.start_scan(timeout=duration_s):
+                # Clean up address representation (mac address)
+                mac = str(adv.address).replace("Address(", "").replace(")", "").strip()
+                name = adv.complete_name or ""
+                results[mac] = {"rssi": adv.rssi, "name": name}
+            self.ble.stop_scan()
+        except Exception as e:
+            print(f"BLE scan error: {e}")
+
+        if was_advertising:
+            self._start_advertise()
+        return results
+
+    def close(self):
+        try:
+            self.ble.stop_advertising()
+        except:
+            pass
+        self._is_advertising = False
+        print("BLE closed.")

circuitpython/lib/button_cp.py

@@ -0,0 +1,73 @@
+"""CircuitPython polled and debounced Button driver.
+
+Replaces the MicroPython Pin IRQ implementation with active loop polling.
+"""
+
+import time
+import digitalio
+
+class Button:
+    def __init__(self, pin_obj, name="Button", debounce_ms=50, long_press_ms=800):
+        self.pin = digitalio.DigitalInOut(pin_obj)
+        self.pin.switch_to_input(pull=digitalio.Pull.UP)
+        self.name = name
+        self.debounce_s = debounce_ms / 1000.0
+        self.long_press_s = long_press_ms / 1000.0
+
+        self.last_state = True
+        self.press_time = 0.0
+        self.last_debounce_time = 0.0
+
+        self.click_callback = None
+        self.long_press_callback = None
+
+    def is_pressed(self):
+        return not self.pin.value
+
+    def on_click(self, callback):
+        self.click_callback = callback
+        return callback
+
+    def on_long_press(self, callback):
+        self.long_press_callback = callback
+        return callback
+
+    def update(self):
+        now = time.monotonic()
+        val = self.pin.value
+
+        if (now - self.last_debounce_time) < self.debounce_s:
+            return
+
+        if val != self.last_state:
+            self.last_debounce_time = now
+            self.last_state = val
+
+            if not val:
+                # Pressed (Active Low)
+                self.press_time = now
+            else:
+                # Released
+                if self.press_time > 0.0:
+                    duration = now - self.press_time
+                    self.press_time = 0.0
+
+                    if duration >= self.long_press_s:
+                        if self.long_press_callback:
+                            self.long_press_callback()
+                    else:
+                        if self.click_callback:
+                            self.click_callback()
+
+class BoardButtons:
+    def __init__(self, boot_pin, key_pin=None):
+        self.boot = Button(boot_pin, "BOOT")
+        if key_pin is not None:
+            self.key = Button(key_pin, "KEY")
+        else:
+            self.key = None
+
+    def update(self):
+        self.boot.update()
+        if self.key is not None:
+            self.key.update()

circuitpython/lib/download_cp.py

@@ -0,0 +1,66 @@
+"""CircuitPython file download utility.
+
+Downloads files over Wi-Fi in chunks to local flash or microSD card storage.
+"""
+
+import os
+from sd_cp import SDCardManager
+
+def download_file(url, dest_filename, session, use_sd=True):
+    """Downloads a file from a URL over the network.
+
+    Args:
+        url (str): Source URL.
+        dest_filename (str): Target filename.
+        session (Session): adafruit_requests Session object.
+        use_sd (bool): Save to microSD card if True, else local flash.
+    """
+    dest_path = dest_filename
+    sd_manager = None
+
+    if use_sd:
+        sd_manager = SDCardManager(mount_point='/sd')
+        if not sd_manager.mount():
+            print("Download Error: Could not mount SD card.")
+            return None
+        dest_path = f"/sd/{dest_filename}"
+
+    print(f"Starting download from: {url} -> {dest_path}")
+
+    try:
+        res = session.get(url)
+    except Exception as e:
+        print(f"HTTP Connection failed: {e}")
+        return None
+
+    if res.status_code != 200:
+        print(f"HTTP Error: Received status code {res.status_code}")
+        res.close()
+        return None
+
+    try:
+        chunk_size = 4096
+        total_downloaded = 0
+
+        with open(dest_path, 'wb') as f:
+            for chunk in res.iter_content(chunk_size):
+                if not chunk:
+                    break
+                f.write(chunk)
+                total_downloaded += len(chunk)
+
+                if total_downloaded % (chunk_size * 25) == 0:
+                    print(f"Downloaded {total_downloaded // 1024} KB...")
+
+        print(f"Download complete! Saved {total_downloaded} bytes to '{dest_path}'.")
+        return dest_path
+
+    except Exception as e:
+        print(f"Error writing to file: {e}")
+        try:
+            os.remove(dest_path)
+        except:
+            pass
+        return None
+    finally:
+        res.close()

circuitpython/lib/ft6336u_cp.py

@@ -0,0 +1,144 @@
+"""CircuitPython FT6336U touch controller driver.
+
+Ports the MicroPython ft6336u.py driver using busio.I2C and digitalio.DigitalInOut.
+"""
+
+import time
+
+class FT6336U:
+    ADDR = 0x38
+
+    # Registers
+    REG_DEV_MODE = 0x00
+    REG_TD_STATUS = 0x02
+    REG_P1_XH = 0x03
+    REG_P1_XL = 0x04
+    REG_P1_YH = 0x05
+    REG_P1_YL = 0x06
+    REG_CTRL = 0x86
+    REG_CHIPID = 0xA3
+    REG_G_MODE = 0xA4
+
+    # Modes
+    CTRL_KEEP_ACTIVE = 0x00
+    G_MODE_TRIGGER = 0x01
+
+    def __init__(self, i2c, rst_pin, int_pin, width=480, height=320, swap_xy=True, invert_x=True, invert_y=False):
+        self.i2c = i2c
+        self.rst = rst_pin
+        self.int = int_pin
+        self.width = width
+        self.height = height
+        self.swap_xy = swap_xy
+        self.invert_x = invert_x
+        self.invert_y = invert_y
+        self.initialized = False
+
+        # Configure reset and interrupt pins
+        self.rst.switch_to_output(value=True)
+        self.int.switch_to_input(pull=None)  # Typically pulled up externally or on-board
+
+        self.reset()
+        self.init_chip()
+
+    def reset(self):
+        self.rst.value = False
+        time.sleep(0.010)
+        self.rst.value = True
+        time.sleep(0.300)  # Wait for chip to wake up
+
+    def read_reg(self, reg, n=1):
+        while not self.i2c.try_lock():
+            pass
+        try:
+            self.i2c.writeto(self.ADDR, bytes([reg]))
+            buf = bytearray(n)
+            self.i2c.readfrom_into(self.ADDR, buf)
+            return buf
+        except Exception as e:
+            print(f"I2C read failed at reg 0x{reg:02X}: {e}")
+            return None
+        finally:
+            self.i2c.unlock()
+
+    def write_reg(self, reg, val):
+        while not self.i2c.try_lock():
+            pass
+        try:
+            self.i2c.writeto(self.ADDR, bytes([reg, val]))
+            return True
+        except Exception as e:
+            print(f"I2C write failed at reg 0x{reg:02X}: {e}")
+            return False
+        finally:
+            self.i2c.unlock()
+
+    def init_chip(self):
+        # 1. Read Chip ID
+        chip_id = self.read_reg(self.REG_CHIPID)
+        if chip_id is None or chip_id[0] != 0x64:
+            time.sleep(0.100)
+            chip_id = self.read_reg(self.REG_CHIPID)
+            if chip_id is None or chip_id[0] != 0x64:
+                print(f"FT6336U error: Invalid Chip ID (got {chip_id[0] if chip_id else None}, expected 0x64)")
+                self.initialized = False
+                return False
+
+        print(f"FT6336U touch controller detected (Chip ID: 0x{chip_id[0]:02X})")
+
+        # 2. Configure operating mode (0 = Normal Mode)
+        self.write_reg(self.REG_DEV_MODE, 0x00)
+
+        # 3. Configure CTRL mode (0 = Keep Active)
+        self.write_reg(self.REG_CTRL, self.CTRL_KEEP_ACTIVE)
+
+        self.initialized = True
+        return True
+
+    def is_touched(self):
+        """Returns True if screen is touched by checking the TD_STATUS register."""
+        if not self.initialized:
+            return False
+        td_status = self.read_reg(self.REG_TD_STATUS)
+        if td_status is None:
+            return False
+        touch_count = td_status[0] & 0x0F
+        if 0 < touch_count < 3:
+            # Read P1 coordinates to clear register/interrupt state on the chip
+            self.read_reg(self.REG_P1_XH, 6)
+            return True
+        return False
+
+    def read_touch(self):
+        """Reads touch point coordinates."""
+        if not self.initialized:
+            return None
+
+        td_status = self.read_reg(self.REG_TD_STATUS)
+        if td_status is None:
+            return None
+
+        touch_count = td_status[0] & 0x0F
+        if touch_count == 0:
+            return None
+
+        buf = self.read_reg(self.REG_P1_XH, 6)
+        if buf is None or len(buf) < 6:
+            return None
+
+        raw_x = ((buf[0] & 0x0F) << 8) | buf[1]
+        raw_y = ((buf[2] & 0x0F) << 8) | buf[3]
+
+        if self.swap_xy:
+            raw_x, raw_y = raw_y, raw_x
+
+        if self.invert_x:
+            raw_x = self.width - 1 - raw_x
+
+        if self.invert_y:
+            raw_y = self.height - 1 - raw_y
+
+        x = max(0, min(self.width - 1, raw_x))
+        y = max(0, min(self.height - 1, raw_y))
+
+        return (x, y)

circuitpython/lib/gif_player.py

@@ -0,0 +1,55 @@
+"""GIF playback helper for the custom RLCD CircuitPython driver."""
+
+import gc
+import time
+
+import gifio
+
+
+class GIFPlayer:
+    def __init__(self, display):
+        self.display = display
+
+    def play(self, filename, *, loops=1, x=40, y=0, threshold=1, clear_between_frames=False, max_frames=-1):
+        """Decode a GIF from disk and push frames to the reflective LCD.
+
+        CircuitPython gifio currently supports GIFs up to 320 pixels wide, so
+        x defaults to 40 to center a 320-wide GIF on this 400-wide display.
+        The ST7305/RLCD display is monochrome in this project, so frames are
+        thresholded from palette indexes: palette index 0 is white; any index
+        >= threshold is black.
+        """
+        gif = gifio.OnDiskGif(filename)
+        try:
+            count = 0
+            while loops < 0 or count < loops:
+                frame_count = 0
+                while True:
+                    if max_frames > 0 and frame_count >= max_frames:
+                        break
+                    delay = gif.next_frame()
+                    if delay is None:
+                        break
+                    if clear_between_frames:
+                        self.display.clear(0)
+                    if hasattr(self.display, "draw_bitmap_color"):
+                        self.display.draw_bitmap_color(gif.bitmap, gif.palette, x, y)
+                    else:
+                        self.display.draw_bitmap_threshold(gif.bitmap, x, y, threshold=threshold)
+                        self.display.show()
+                    frame_count += 1
+                    # gifio delay is seconds in modern CircuitPython builds. If
+                    # an older build plays 100x too slowly, divide by 100 here.
+                    time.sleep(max(0.0, delay))
+                count += 1
+                if loops < 0 or count < loops:
+                    # OnDiskGif has no reliable seek/rewind API; reopen to loop.
+                    gif.deinit()
+                    gc.collect()
+                    gif = gifio.OnDiskGif(filename)
+        finally:
+            try:
+                gif.deinit()
+            except AttributeError:
+                pass
+            gc.collect()

circuitpython/lib/ili9341_cp.py

@@ -0,0 +1,481 @@
+"""CircuitPython ILI9341 driver for Hosyond ESP32-S3 Touchscreen board.
+
+This driver wraps adafruit_framebuf using a 1-bit MONO_HLSB canvas buffer,
+then converts it row-by-row to 16-bit RGB565 via a lookup table (LUT) during show().
+"""
+
+import time
+
+try:
+    import adafruit_framebuf
+except ImportError:
+    adafruit_framebuf = None
+
+
+class ILI9341:
+    WIDTH = 320
+    HEIGHT = 240
+
+    def __init__(self, spi, cs, dc, rst=None, bl=None, width=WIDTH, height=HEIGHT, invert_color=True):
+        if adafruit_framebuf is None:
+            raise RuntimeError("adafruit_framebuf is required in CIRCUITPY/lib")
+        self.spi = spi
+        self.cs = cs
+        self.dc = dc
+        self.rst = rst
+        self.width = width
+        self.height = height
+        self.invert_color = invert_color
+
+        # 1-bit canvas buffer (1 = White/On, 0 = Black/Off)
+        self.hw_len = (width * height) // 8
+        self.canvas_buffer = bytearray(self.hw_len)
+        self.canvas = adafruit_framebuf.FrameBuffer(
+            self.canvas_buffer,
+            width,
+            height,
+            adafruit_framebuf.MHMSB,  # Matches MONO_HLSB (most significant bit first)
+        )
+
+        # Pre-allocate chunk buffer for conversion (16 rows: 320 * 16 * 2 = 10,240 bytes)
+        self.chunk_rows = 16
+        self.row_buffer = bytearray(width * self.chunk_rows * 2)
+
+        # Precompute lookup table for fast 1-bit to 16-bit conversion
+        # Each byte (8 pixels) maps to 16 bytes of RGB565 (8 pixels * 2 bytes)
+        self.lut = []
+        for i in range(256):
+            entry = bytearray(16)
+            for bit in range(8):
+                if i & (1 << (7 - bit)):
+                    # White pixel: 0xFFFF (High byte: 0xFF, Low byte: 0xFF)
+                    entry[bit * 2] = 0xFF
+                    entry[bit * 2 + 1] = 0xFF
+                else:
+                    # Black pixel: 0x0000
+                    entry[bit * 2] = 0x00
+                    entry[bit * 2 + 1] = 0x00
+            self.lut.append(bytes(entry))
+
+        # Setup CS and DC
+        self.cs.switch_to_output(value=True)
+        self.dc.switch_to_output(value=False)
+
+        # Setup Reset if present
+        if self.rst is not None:
+            self.rst.switch_to_output(value=True)
+
+        # Setup Backlight PWM if present
+        if bl is not None:
+            import pwmio
+            self.bl_pwm = pwmio.PWMOut(bl, frequency=1000, duty_cycle=65535)
+        else:
+            self.bl_pwm = None
+
+        self.reset()
+        self.init_display()
+        self.clear(0)
+        self.show()
+
+    def reset(self):
+        if self.rst is not None:
+            self.rst.value = True
+            time.sleep(0.005)
+            self.rst.value = False
+            time.sleep(0.015)
+            self.rst.value = True
+            time.sleep(0.015)
+        else:
+            # Software reset command if no reset pin
+            self.write_cmd(0x01)
+            time.sleep(0.150)
+
+    def _lock_spi(self):
+        while not self.spi.try_lock():
+            pass
+        self.spi.configure(baudrate=40000000, phase=0, polarity=0)
+
+    def _unlock_spi(self):
+        self.spi.unlock()
+
+    def write_cmd(self, cmd):
+        self._lock_spi()
+        try:
+            self.cs.value = False
+            self.dc.value = False
+            self.spi.write(bytes([cmd & 0xFF]))
+            self.cs.value = True
+        finally:
+            self._unlock_spi()
+
+    def write_data(self, data):
+        if isinstance(data, int):
+            payload = bytes([data & 0xFF])
+        elif isinstance(data, (bytes, bytearray, memoryview)):
+            payload = data
+        else:
+            payload = bytes(data)
+        self._lock_spi()
+        try:
+            self.cs.value = False
+            self.dc.value = True
+            self.spi.write(payload)
+            self.cs.value = True
+        finally:
+            self._unlock_spi()
+
+    def init_display(self):
+        # SWRESET
+        self.write_cmd(0x01)
+        time.sleep(0.150)
+
+        self.write_cmd(0xCF); self.write_data(b"\x00\xC1\x30")
+        self.write_cmd(0xED); self.write_data(b"\x64\x03\x12\x81")
+        self.write_cmd(0xE8); self.write_data(b"\x85\x00\x78")
+        self.write_cmd(0xCB); self.write_data(b"\x39\x2C\x00\x34\x02")
+        self.write_cmd(0xF7); self.write_data(b"\x20")
+        self.write_cmd(0xEA); self.write_data(b"\x00\x00")
+
+        self.write_cmd(0xC0); self.write_data(b"\x13")  # Power Control 1
+        self.write_cmd(0xC1); self.write_data(b"\x13")  # Power Control 2
+        self.write_cmd(0xC5); self.write_data(b"\x22\x35")  # VCOM Control 1
+        self.write_cmd(0xC7); self.write_data(b"\xBD")  # VCOM Control 2
+
+        # Memory Access Control (MADCTL) = 0x68 (Landscape: MV=1, MX=1, MY=0, BGR color filter)
+        self.write_cmd(0x36); self.write_data(b"\x68")
+
+        self.write_cmd(0xB6); self.write_data(b"\x0A\xA2")  # Display Function Control
+        self.write_cmd(0x3A); self.write_data(b"\x55")  # Pixel Format (COLMOD) = 16-bit RGB565
+        self.write_cmd(0xF6); self.write_data(b"\x01\x30")
+        self.write_cmd(0xB1); self.write_data(b"\x00\x1B")  # Frame Rate Control
+        self.write_cmd(0xF2); self.write_data(b"\x00")
+        self.write_cmd(0x26); self.write_data(b"\x01")  # Gamma Curve
+
+        self.write_cmd(0xE0); self.write_data(b"\x0F\x35\x31\x0B\x0E\x06\x49\xA7\x33\x07\x0F\x03\x0C\x0A\x00")
+        self.write_cmd(0xE1); self.write_data(b"\x00\x0A\x0F\x04\x11\x08\x36\x58\x4D\x07\x10\x0C\x32\x34\x0F")
+
+        if self.invert_color:
+            self.write_cmd(0x21)  # INVON
+        else:
+            self.write_cmd(0x20)  # INVOFF
+
+        self.write_cmd(0x11)  # SLPOUT
+        time.sleep(0.120)
+        self.write_cmd(0x29)  # DISPON
+        time.sleep(0.010)
+
+    def invert(self, enable):
+        self.write_cmd(0x21 if enable else 0x20)
+
+    def set_window(self, x0, y0, x1, y1):
+        self.write_cmd(0x2A)
+        self.write_data(bytes([x0 >> 8, x0 & 0xFF, x1 >> 8, x1 & 0xFF]))
+        self.write_cmd(0x2B)
+        self.write_data(bytes([y0 >> 8, y0 & 0xFF, y1 >> 8, y1 & 0xFF]))
+        self.write_cmd(0x2C)
+
+    def clear(self, color=0):
+        self.canvas.fill(1 if color else 0)
+
+    def pixel(self, x, y, color):
+        self.canvas.pixel(x, y, 1 if color else 0)
+
+    def line(self, x0, y0, x1, y1, color):
+        self.canvas.line(x0, y0, x1, y1, 1 if color else 0)
+
+    def rect(self, x, y, width, height, color):
+        self.canvas.rect(x, y, width, height, 1 if color else 0)
+
+    def fill_rect(self, x, y, width, height, color):
+        self.canvas.fill_rect(x, y, width, height, 1 if color else 0)
+
+    def text(self, text, x, y, color=1):
+        self.canvas.text(str(text), x, y, 1 if color else 0)
+
+    def text_large(self, text, x, y, scale=2, color=1):
+        tmp = bytearray(8)
+        fb = adafruit_framebuf.FrameBuffer(tmp, 8, 8, adafruit_framebuf.MHMSB)
+        color = 1 if color else 0
+        for ch in str(text):
+            fb.fill(0)
+            fb.text(ch, 0, 0, 1)
+            for py in range(8):
+                for px in range(8):
+                    if fb.pixel(px, py):
+                        self.canvas.fill_rect(x + px * scale, y + py * scale, scale, scale, color)
+            x += 8 * scale
+
+    def draw_bitmap_threshold(self, bitmap, x=0, y=0, threshold=1):
+        width = min(getattr(bitmap, "width", self.width), self.width - x)
+        height = min(getattr(bitmap, "height", self.height), self.height - y)
+        for yy in range(height):
+            for xx in range(width):
+                self.canvas.pixel(x + xx, y + yy, 1 if bitmap[xx, yy] >= threshold else 0)
+
+    def draw_bitmap_color(self, bitmap, palette, x=0, y=0):
+        width = min(getattr(bitmap, "width", self.width), self.width - x)
+        height = min(getattr(bitmap, "height", self.height), self.height - y)
+        row_buf = bytearray(width * 2)
+        for yy in range(height):
+            idx = 0
+            for xx in range(width):
+                val = bitmap[xx, yy]
+                if palette is None:
+                    rgb = val
+                else:
+                    color = palette[val]
+                    if isinstance(color, tuple) or isinstance(color, list):
+                        r, g, b = color[0], color[1], color[2]
+                    elif isinstance(color, int):
+                        r = (color >> 16) & 0xFF
+                        g = (color >> 8) & 0xFF
+                        b = color & 0xFF
+                    else:
+                        r, g, b = 0, 0, 0
+                    r5 = r >> 3
+                    g6 = g >> 2
+                    b5 = b >> 3
+                    rgb = (r5 << 11) | (g6 << 5) | b5
+                row_buf[idx] = (rgb >> 8) & 0xFF
+                row_buf[idx + 1] = rgb & 0xFF
+                idx += 2
+            self.draw_rgb565(x, yy + y, width, 1, row_buf, sync_canvas=False)
+
+    def show(self):
+        """Optimized conversion of 1-bit frame buffer to 16-bit RGB565 over SPI."""
+        self.set_window(0, 0, self.width - 1, self.height - 1)
+        self.dc.value = True
+        self.cs.value = False
+
+        lut = self.lut
+        canvas_buf = self.canvas_buffer
+        row_buf = self.row_buffer
+        width_bytes = self.width // 8  # 40 bytes per row
+
+        num_chunks = self.height // self.chunk_rows  # 240 // 16 = 15 chunks
+        for chunk in range(num_chunks):
+            start_row = chunk * self.chunk_rows
+            idx = 0
+            # Loop for 16 rows * 40 bytes/row = 640 bytes. Slice assignment maps directly to LUT.
+            for y in range(start_row, start_row + self.chunk_rows):
+                offset = y * width_bytes
+                for x_byte_idx in range(width_bytes):
+                    val = canvas_buf[offset + x_byte_idx]
+                    row_buf[idx : idx + 16] = lut[val]
+                    idx += 16
+
+            self._lock_spi()
+            try:
+                self.spi.write(row_buf)
+            finally:
+                self._unlock_spi()
+
+        self.cs.value = True
+
+    def set_brightness(self, level):
+        if self.bl_pwm is not None:
+            level = max(0, min(100, level))
+            self.bl_pwm.duty_cycle = int(level * 65535 / 100)
+
+    def set_power(self, on):
+        if on:
+            self.write_cmd(0x11)  # SLPOUT
+            time.sleep(0.120)
+            self.write_cmd(0x29)  # DISPON
+            if self.bl_pwm is not None:
+                self.bl_pwm.duty_cycle = 65535
+        else:
+            self.write_cmd(0x28)  # DISPOFF
+            self.write_cmd(0x10)  # SLPIN
+            time.sleep(0.010)
+            if self.bl_pwm is not None:
+                self.bl_pwm.duty_cycle = 0
+
+    def _update_mono_canvas_rgb565(self, x, y, w, h, data):
+        for cy in range(h):
+            screen_y = y + cy
+            if screen_y < 0 or screen_y >= self.height:
+                continue
+            for cx in range(w):
+                screen_x = x + cx
+                if screen_x < 0 or screen_x >= self.width:
+                    continue
+                idx = (cy * w + cx) * 2
+                h_byte = data[idx]
+                l_byte = data[idx + 1]
+                # Extract RGB from RGB565
+                r = (h_byte & 0xF8)
+                g = ((h_byte & 0x07) << 5) | ((l_byte & 0xE0) >> 3)
+                b = (l_byte & 0x1F) << 3
+                # Convert to luminance
+                lum = (r * 299 + g * 587 + b * 114) // 1000
+                mono = 1 if lum >= 128 else 0
+                self.canvas.pixel(screen_x, screen_y, mono)
+
+    def draw_rgb565(self, x, y, w, h, data, sync_canvas=True):
+        """Draw raw RGB565 pixel data on the screen at specified (x,y) with width and height."""
+        # Clip coordinates
+        x_start = max(0, x)
+        x_end = min(self.width - 1, x + w - 1)
+        y_start = max(0, y)
+        y_end = min(self.height - 1, y + h - 1)
+        
+        if x_start > x_end or y_start > y_end:
+            return True
+            
+        # Fast path: if completely visible on screen, draw in one go
+        if x_start == x and x_end == x + w - 1 and y_start == y and y_end == y + h - 1:
+            self.set_window(x_start, y_start, x_end, y_end)
+            self.dc.value = True
+            self.cs.value = False
+            self._lock_spi()
+            try:
+                self.spi.write(data)
+            finally:
+                self._unlock_spi()
+            self.cs.value = True
+        else:
+            # Slow path: row-by-row clipping
+            for cy in range(y_start, y_end + 1):
+                src_y = cy - y
+                src_row_offset = (src_y * w + (x_start - x)) * 2
+                row_len_bytes = (x_end - x_start + 1) * 2
+                
+                self.set_window(x_start, cy, x_end, cy)
+                self.dc.value = True
+                self.cs.value = False
+                self._lock_spi()
+                try:
+                    self.spi.write(memoryview(data)[src_row_offset : src_row_offset + row_len_bytes])
+                finally:
+                    self._unlock_spi()
+                self.cs.value = True
+                
+        # Sync the internal 1-bit canvas buffer
+        if sync_canvas:
+            self._update_mono_canvas_rgb565(x, y, w, h, data)
+        return True
+
+    def _convert_bgr24_to_rgb565(self, bgr_buf, rgb565_buf, width, src_offset, num_pixels):
+        idx = 0
+        for i in range(src_offset, src_offset + num_pixels):
+            b = bgr_buf[i * 3]
+            g = bgr_buf[i * 3 + 1]
+            r = bgr_buf[i * 3 + 2]
+            r_5 = r >> 3
+            g_6 = g >> 2
+            b_5 = b >> 3
+            rgb565_buf[idx] = (r_5 << 3) | (g_6 >> 3)
+            rgb565_buf[idx + 1] = ((g_6 & 0x07) << 5) | b_5
+            idx += 2
+
+    def _convert_bgra32_to_rgb565(self, bgra_buf, rgb565_buf, width, src_offset, num_pixels):
+        idx = 0
+        for i in range(src_offset, src_offset + num_pixels):
+            b = bgra_buf[i * 4]
+            g = bgra_buf[i * 4 + 1]
+            r = bgra_buf[i * 4 + 2]
+            r_5 = r >> 3
+            g_6 = g >> 2
+            b_5 = b >> 3
+            rgb565_buf[idx] = (r_5 << 3) | (g_6 >> 3)
+            rgb565_buf[idx + 1] = ((g_6 & 0x07) << 5) | b_5
+            idx += 2
+
+    def draw_bmp(self, filename, x=0, y=0):
+        import struct
+        try:
+            with open(filename, 'rb') as f:
+                header = f.read(54)
+                if len(header) < 54 or header[0:2] != b'BM':
+                    print("Err: Not a valid BMP file")
+                    return False
+                
+                pixel_offset = struct.unpack('<I', header[10:14])[0]
+                width, height = struct.unpack('<ii', header[18:26])
+                planes, bpp = struct.unpack('<HH', header[26:30])
+                compression = struct.unpack('<I', header[30:34])[0]
+                
+                if bpp not in (24, 32):
+                    print("Err: Only 24-bit and 32-bit BMP formats supported")
+                    return False
+                    
+                if compression != 0:
+                    print("Err: Only uncompressed BMP supported")
+                    return False
+                
+                f.seek(pixel_offset)
+                bottom_up = True
+                if height < 0:
+                    height = -height
+                    bottom_up = False
+                
+                row_bytes = (width * bpp) // 8
+                row_padded = ((width * bpp + 31) // 32) * 4
+                
+                read_buf = bytearray(row_padded)
+                rgb565_buf = bytearray(width * 2)
+                
+                for row_idx in range(height):
+                    n = f.readinto(read_buf)
+                    if n < row_padded:
+                        break
+                    
+                    screen_y = y + (height - 1 - row_idx) if bottom_up else y + row_idx
+                    if screen_y < 0 or screen_y >= self.height:
+                        continue
+                        
+                    x_start = x
+                    x_end = x + width - 1
+                    
+                    if x_start >= self.width or x_end < 0:
+                        continue
+                        
+                    win_x0 = max(0, x_start)
+                    win_x1 = min(self.width - 1, x_end)
+                    
+                    if win_x1 < win_x0:
+                        continue
+                        
+                    src_offset_pixels = win_x0 - x_start
+                    win_w = win_x1 - win_x0 + 1
+                    
+                    # Convert pixel data to RGB565 row buffer
+                    if bpp == 24:
+                        self._convert_bgr24_to_rgb565(read_buf, rgb565_buf, width, src_offset_pixels, win_w)
+                    elif bpp == 32:
+                        self._convert_bgra32_to_rgb565(read_buf, rgb565_buf, width, src_offset_pixels, win_w)
+                    
+                    # Draw directly to the screen via SPI window
+                    self.set_window(win_x0, screen_y, win_x1, screen_y)
+                    self.dc.value = True
+                    self.cs.value = False
+                    self._lock_spi()
+                    try:
+                        self.spi.write(memoryview(rgb565_buf)[:win_w * 2])
+                    finally:
+                        self._unlock_spi()
+                    self.cs.value = True
+                    
+                    # Also update internal 1-bit canvas buffer for screenshots/refresh consistency
+                    for px in range(win_w):
+                        screen_x = win_x0 + px
+                        src_px = src_offset_pixels + px
+                        if bpp == 24:
+                            b = read_buf[src_px * 3]
+                            g = read_buf[src_px * 3 + 1]
+                            r = read_buf[src_px * 3 + 2]
+                        else:
+                            b = read_buf[src_px * 4]
+                            g = read_buf[src_px * 4 + 1]
+                            r = read_buf[src_px * 4 + 2]
+                        # 0 = Black, 1 = White in conversion for MONO_HLSB canvas
+                        lum = (r * 299 + g * 587 + b * 114) // 1000
+                        mono_c = 1 if lum >= 128 else 0
+                        self.canvas.pixel(screen_x, screen_y, mono_c)
+                        
+            return True
+        except Exception as e:
+            print("Error drawing BMP:", e)
+            return False
+

circuitpython/lib/rgb_led_cp.py

@@ -0,0 +1,43 @@
+"""CircuitPython NeoPixel LED utility.
+
+Uses the neopixel library and time.monotonic() to run breathing and rainbow animations.
+"""
+
+import math
+import time
+import neopixel
+
+class BoardLED:
+    def __init__(self, pin):
+        # Initialize 1 NeoPixel on the specified pin
+        self.np = neopixel.NeoPixel(pin, 1, brightness=1.0, auto_write=False)
+        self.base_color = (0, 0, 0)
+        self.off()
+
+    def set_color(self, r, g, b):
+        self.base_color = (r, g, b)
+        self.np[0] = (r, g, b)
+        self.np.show()
+
+    def off(self):
+        self.set_color(0, 0, 0)
+
+    def update_breathing(self, speed_factor=1.0):
+        if self.base_color == (0, 0, 0):
+            return
+        t = time.monotonic() * speed_factor
+        # Sine wave from 0.05 to 1.0
+        factor = 0.525 + 0.475 * math.sin(t * math.pi)
+        br = int(self.base_color[0] * factor)
+        bg = int(self.base_color[1] * factor)
+        bb = int(self.base_color[2] * factor)
+        self.np[0] = (br, bg, bb)
+        self.np.show()
+
+    def update_rainbow(self, speed_factor=0.2):
+        t = time.monotonic() * speed_factor
+        r = int(127.5 * (1.0 + math.sin(t * 2.0 * math.pi)))
+        g = int(127.5 * (1.0 + math.sin(t * 2.0 * math.pi + 2.0 * math.pi / 3.0)))
+        b = int(127.5 * (1.0 + math.sin(t * 2.0 * math.pi + 4.0 * math.pi / 3.0)))
+        self.np[0] = (r, g, b)
+        self.np.show()

circuitpython/lib/rlcd_cp.py

@@ -0,0 +1,199 @@
+"""CircuitPython ST7305/RLCD driver for Waveshare ESP32-S3-RLCD-4.2.
+
+This is a direct migration attempt from the project's MicroPython rlcd.py.
+It intentionally does not use displayio's display bus because the panel uses a
+non-standard 2x4 packed 1-bit memory layout.
+"""
+
+import time
+
+try:
+    import adafruit_framebuf
+except ImportError:  # CircuitPython bundle dependency.
+    adafruit_framebuf = None
+
+
+class RLCD:
+    WIDTH = 400
+    HEIGHT = 300
+    BUFFER_SIZE = (WIDTH * HEIGHT) // 8
+
+    def __init__(self, spi, cs, dc, rst, width=WIDTH, height=HEIGHT):
+        if adafruit_framebuf is None:
+            raise RuntimeError("adafruit_framebuf is required in CIRCUITPY/lib")
+        self.spi = spi
+        self.cs = cs
+        self.dc = dc
+        self.rst = rst
+        self.width = width
+        self.height = height
+        self.hw_buffer = bytearray((width * height) // 8)
+        self.canvas_buffer = bytearray((width * height) // 8)
+        self.canvas = adafruit_framebuf.FrameBuffer(
+            self.canvas_buffer,
+            width,
+            height,
+            adafruit_framebuf.MHMSB,
+        )
+
+        self.cs.switch_to_output(value=True)
+        self.dc.switch_to_output(value=False)
+        self.rst.switch_to_output(value=True)
+
+        self.reset()
+        self.init_display()
+
+    def reset(self):
+        self.rst.value = True
+        time.sleep(0.05)
+        self.rst.value = False
+        time.sleep(0.02)
+        self.rst.value = True
+        time.sleep(0.05)
+
+    def _lock_spi(self):
+        while not self.spi.try_lock():
+            pass
+        # CircuitPython SPI settings are only guaranteed while the bus is
+        # locked, so configure after every lock instead of only at startup.
+        self.spi.configure(baudrate=20000000, phase=0, polarity=0)
+
+    def _unlock_spi(self):
+        self.spi.unlock()
+
+    def write_cmd(self, cmd):
+        self._lock_spi()
+        try:
+            self.cs.value = False
+            self.dc.value = False
+            self.spi.write(bytes([cmd & 0xFF]))
+            self.cs.value = True
+        finally:
+            self._unlock_spi()
+
+    def write_data(self, data):
+        if isinstance(data, int):
+            payload = bytes([data & 0xFF])
+        elif isinstance(data, (bytes, bytearray, memoryview)):
+            payload = data
+        else:
+            payload = bytes(data)
+        self._lock_spi()
+        try:
+            self.cs.value = False
+            self.dc.value = True
+            self.spi.write(payload)
+            self.cs.value = True
+        finally:
+            self._unlock_spi()
+
+    def init_display(self):
+        # Ported from MicroPython rlcd.py. The old file had one decimal 19 in
+        # the 0xC5 payload; this port uses 0x19 consistently.
+        self.write_cmd(0xD6); self.write_data([0x17, 0x02])
+        self.write_cmd(0xD1); self.write_data(0x01)
+        self.write_cmd(0xC0); self.write_data([0x11, 0x04])
+        self.write_cmd(0xC1); self.write_data([0x41, 0x41, 0x41, 0x41])
+        self.write_cmd(0xC2); self.write_data([0x19, 0x19, 0x19, 0x19])
+        self.write_cmd(0xC4); self.write_data([0x41, 0x41, 0x41, 0x41])
+        self.write_cmd(0xC5); self.write_data([0x19, 0x19, 0x19, 0x19])
+        self.write_cmd(0xD8); self.write_data([0xA6, 0xE9])
+        self.write_cmd(0xB2); self.write_data(0x05)
+        self.write_cmd(0xB3); self.write_data([0xE5, 0xF6, 0x05, 0x46, 0x77, 0x77, 0x77, 0x77, 0x76, 0x45])
+        self.write_cmd(0xB4); self.write_data([0x05, 0x46, 0x77, 0x77, 0x77, 0x77, 0x76, 0x45])
+        self.write_cmd(0x62); self.write_data([0x32, 0x03, 0x1F])
+        self.write_cmd(0xB7); self.write_data(0x13)
+        self.write_cmd(0xB0); self.write_data(0x64)
+        self.write_cmd(0x11)
+        time.sleep(0.2)
+        self.write_cmd(0xC9); self.write_data(0x00)
+        self.write_cmd(0x36); self.write_data(0x48)
+        self.write_cmd(0x3A); self.write_data(0x11)
+        self.write_cmd(0xB9); self.write_data(0x20)
+        self.write_cmd(0xB8); self.write_data(0x29)
+        self.write_cmd(0x21)
+        self.write_cmd(0x2A); self.write_data([0x12, 0x2A])
+        self.write_cmd(0x2B); self.write_data([0x00, 0xC7])
+        self.write_cmd(0x35); self.write_data(0x00)
+        self.write_cmd(0xD0); self.write_data(0xFF)
+        self.write_cmd(0x38)
+        self.write_cmd(0x29)
+
+    def clear(self, color=0):
+        self.canvas.fill(1 if color else 0)
+
+    def pixel(self, x, y, color):
+        self.canvas.pixel(x, y, 1 if color else 0)
+
+    def line(self, x0, y0, x1, y1, color):
+        self.canvas.line(x0, y0, x1, y1, 1 if color else 0)
+
+    def rect(self, x, y, width, height, color):
+        self.canvas.rect(x, y, width, height, 1 if color else 0)
+
+    def fill_rect(self, x, y, width, height, color):
+        self.canvas.fill_rect(x, y, width, height, 1 if color else 0)
+
+    def text(self, text, x, y, color=1):
+        self.canvas.text(str(text), x, y, 1 if color else 0)
+
+    def text_large(self, text, x, y, scale=2, color=1):
+        # Simple nearest-neighbor scaled 8x8 font rendering using a temp buffer.
+        tmp = bytearray(8)
+        fb = adafruit_framebuf.FrameBuffer(tmp, 8, 8, adafruit_framebuf.MHMSB)
+        color = 1 if color else 0
+        for ch in str(text):
+            fb.fill(0)
+            fb.text(ch, 0, 0, 1)
+            for py in range(8):
+                for px in range(8):
+                    if fb.pixel(px, py):
+                        self.canvas.fill_rect(x + px * scale, y + py * scale, scale, scale, color)
+            x += 8 * scale
+
+    def draw_bitmap_threshold(self, bitmap, x=0, y=0, threshold=1):
+        """Copy any indexable bitmap-like object to the 1-bit canvas.
+
+        gifio/displayio bitmaps return palette indexes. Treat index 0 as white
+        and anything >= threshold as black by default.
+        """
+        width = min(getattr(bitmap, "width", self.width), self.width - x)
+        height = min(getattr(bitmap, "height", self.height), self.height - y)
+        for yy in range(height):
+            for xx in range(width):
+                self.canvas.pixel(x + xx, y + yy, 1 if bitmap[xx, yy] >= threshold else 0)
+
+    def pack(self):
+        """Convert standard horizontal 1-bit canvas into the RLCD 2x4 layout."""
+        buf = self.hw_buffer
+        for i in range(len(buf)):
+            buf[i] = 0
+
+        # Correctness-first port of MicroPython native loop.
+        height = self.height
+        for y in range(height):
+            inv_y = height - 1 - y
+            block_y = inv_y // 4
+            local_y = inv_y & 0x03
+            local_y_shift = local_y * 2
+            row_offset = block_y
+            for x in range(self.width):
+                if self.canvas.pixel(x, y):
+                    byte_x = x >> 1
+                    index = byte_x * 75 + row_offset
+                    local_x = x & 0x01
+                    bit = 7 - (local_y_shift + local_x)
+                    buf[index] |= 1 << bit
+        return buf
+
+    def show(self):
+        self.pack()
+        self.write_cmd(0x2A)
+        self.write_data([0x12, 0x2A])
+        self.write_cmd(0x2B)
+        self.write_data([0x00, 0xC7])
+        self.write_cmd(0x2C)
+        self.write_data(self.hw_buffer)
+
+    def invert(self, enable=True):
+        self.write_cmd(0x21 if enable else 0x20)

circuitpython/lib/rtc_cp.py

@@ -0,0 +1,132 @@
+"""CircuitPython PCF85063 Real-Time Clock (RTC) driver.
+
+Synchronizes the hardware RTC with CircuitPython's native rtc.RTC() system clock.
+"""
+
+import rtc
+import time
+
+class PCF85063:
+    ADDR = 0x51
+    TIME_REG_START = 0x04
+
+    def __init__(self, i2c):
+        self.i2c = i2c
+        self._init_rtc()
+
+    def read_reg(self, reg, n=1):
+        while not self.i2c.try_lock():
+            pass
+        try:
+            self.i2c.writeto(self.ADDR, bytes([reg]))
+            buf = bytearray(n)
+            self.i2c.readfrom_into(self.ADDR, buf)
+            return buf
+        except Exception as e:
+            print(f"RTC read failed at reg 0x{reg:02X}: {e}")
+            return None
+        finally:
+            self.i2c.unlock()
+
+    def write_reg(self, reg, data):
+        while not self.i2c.try_lock():
+            pass
+        try:
+            payload = bytearray([reg])
+            if isinstance(data, (bytes, bytearray, list)):
+                payload.extend(data)
+            else:
+                payload.append(data)
+            self.i2c.writeto(self.ADDR, payload)
+            return True
+        except Exception as e:
+            print(f"RTC write failed at reg 0x{reg:02X}: {e}")
+            return False
+        finally:
+            self.i2c.unlock()
+
+    def _init_rtc(self):
+        ctrl1 = self.read_reg(0x00, 1)
+        if ctrl1 is not None and (ctrl1[0] & 0x20):
+            print("RTC oscillator was stopped. Starting oscillator...")
+            self.write_reg(0x00, 0x00)
+
+    def _dec2bcd(self, val):
+        return (val // 10 << 4) | (val % 10)
+
+    def _bcd2dec(self, val):
+        return ((val >> 4) * 10) + (val & 0x0F)
+
+    def get_datetime(self):
+        """Reads current time from hardware RTC.
+
+        Returns:
+            tuple: (year, month, day, weekday, hour, minute, second) or None on error.
+        """
+        data = self.read_reg(self.TIME_REG_START, 7)
+        if data is None:
+            return None
+
+        second = self._bcd2dec(data[0] & 0x7F)
+        minute = self._bcd2dec(data[1] & 0x7F)
+        hour = self._bcd2dec(data[2] & 0x3F)
+        day = self._bcd2dec(data[3] & 0x3F)
+        weekday = data[4] & 0x07
+        month = self._bcd2dec(data[5] & 0x1F)
+        year = 2000 + self._bcd2dec(data[6])
+
+        return (year, month, day, weekday, hour, minute, second)
+
+    def set_datetime(self, dt):
+        """Sets the hardware RTC time.
+
+        Args:
+            dt (tuple): (year, month, day, weekday, hour, minute, second)
+        """
+        try:
+            year, month, day, weekday, hour, minute, second = dt
+            reg_year = year % 100
+
+            data = bytearray(7)
+            data[0] = self._dec2bcd(second) & 0x7F
+            data[1] = self._dec2bcd(minute)
+            data[2] = self._dec2bcd(hour)
+            data[3] = self._dec2bcd(day)
+            data[4] = weekday & 0x07
+            data[5] = self._dec2bcd(month)
+            data[6] = self._dec2bcd(reg_year)
+
+            return self.write_reg(self.TIME_REG_START, data)
+        except Exception as e:
+            print(f"Error setting PCF85063 RTC: {e}")
+            return False
+
+    def sync_to_system(self):
+        """Synchronizes the CircuitPython system time from the hardware RTC."""
+        dt = self.get_datetime()
+        if dt:
+            year, month, day, weekday, hour, minute, second = dt
+            r = rtc.RTC()
+            r.datetime = time.struct_time((year, month, day, hour, minute, second, weekday, -1, -1))
+            print(f"System clock synced to RTC: {year:04d}-{month:02d}-{day:02d} {hour:02d}:{minute:02d}:{second:02d}")
+            return True
+        return False
+
+    def sync_from_system(self):
+        """Synchronizes the hardware RTC time from the system clock."""
+        try:
+            t = time.localtime()
+            dt = (t.tm_year, t.tm_mon, t.tm_mday, t.tm_wday, t.tm_hour, t.tm_min, t.tm_sec)
+            success = self.set_datetime(dt)
+            if success:
+                print(f"RTC synced from System: {t.tm_year:04d}-{t.tm_mon:02d}-{t.tm_mday:02d} {t.tm_hour:02d}:{t.tm_min:02d}:{t.tm_sec:02d}")
+            return success
+        except Exception as e:
+            print(f"Error syncing RTC from system: {e}")
+            return False
+
+    def get_time_string(self):
+        dt = self.get_datetime()
+        if dt:
+            return f"{dt[0]:04d}-{dt[1]:02d}-{dt[2]:02d} {dt[4]:02d}:{dt[5]:02d}:{dt[6]:02d}"
+        return "0000-00-00 00:00:00"

circuitpython/lib/sd_cp.py

@@ -0,0 +1,85 @@
+"""CircuitPython SD card utility using sdioio and storage.
+
+Uses SDMMC 1-bit mode on Pins: sck=board.IO2, cmd=board.IO1, data0=board.IO3.
+"""
+
+import os
+import board
+import sdioio
+import storage
+
+class SDCardManager:
+    def __init__(self, mount_point='/sd'):
+        self.mount_point = mount_point
+        self.sd = None
+        self.mounted = False
+
+    def mount(self):
+        if self.mounted:
+            print(f"SD card already mounted at {self.mount_point}")
+            return True
+
+        try:
+            print("Initializing SDCard (SDMMC 1-bit mode: sck=2, cmd=1, d0=3)...")
+            self.sd = sdioio.SDCard(clock=board.IO2, command=board.IO1, data=[board.IO3], frequency=20000000)
+            vfs = storage.VfsFat(self.sd)
+            print(f"Mounting SD card to {self.mount_point}...")
+            storage.mount(vfs, self.mount_point)
+            self.mounted = True
+            print("SD card mounted successfully!")
+            return True
+        except Exception as e:
+            print(f"Failed to mount SD card: {e}")
+            self.sd = None
+            self.mounted = False
+            return False
+
+    def unmount(self):
+        if not self.mounted:
+            return True
+
+        try:
+            print(f"Unmounting SD card from {self.mount_point}...")
+            storage.umount(self.mount_point)
+            if self.sd:
+                try:
+                    self.sd.deinit()
+                except:
+                    pass
+            self.mounted = False
+            self.sd = None
+            print("SD card unmounted.")
+            return True
+        except Exception as e:
+            print(f"Failed to unmount SD card: {e}")
+            return False
+
+    def is_mounted(self):
+        return self.mounted
+
+    def list_files(self):
+        if not self.mounted:
+            print("SD card is not mounted.")
+            return None
+        try:
+            return os.listdir(self.mount_point)
+        except Exception as e:
+            print(f"Error listing SD card files: {e}")
+            return None
+
+    def get_info(self):
+        if not self.mounted:
+            return None
+        try:
+            stat = os.statvfs(self.mount_point)
+            block_size = stat[0]
+            total_blocks = stat[2]
+            free_blocks = stat[3]
+
+            return {
+                "total_bytes": total_blocks * block_size,
+                "free_bytes": free_blocks * block_size
+            }
+        except Exception as e:
+            print(f"Error getting SD card info: {e}")
+            return None

circuitpython/lib/shtc3_cp.py

@@ -0,0 +1,101 @@
+"""CircuitPython SHTC3 temperature and humidity sensor driver.
+
+Uses standard I2C transactions with bus locking.
+"""
+
+import time
+
+class SHTC3:
+    ADDR = 0x70
+
+    WAKE = b'\x35\x17'
+    SLEEP = b'\xB0\x98'
+    MEASURE = b'\x78\x66' # High precision, T first, clock stretching disabled
+
+    def __init__(self, i2c):
+        self.i2c = i2c
+
+    def _crc8(self, data):
+        crc = 0xFF
+        for byte in data:
+            crc ^= byte
+            for _ in range(8):
+                if crc & 0x80:
+                    crc = (crc << 1) ^ 0x31
+                else:
+                    crc <<= 1
+                crc &= 0xFF
+        return crc
+
+    def read_sensor(self):
+        try:
+            # 1. Wakeup
+            while not self.i2c.try_lock():
+                pass
+            try:
+                self.i2c.writeto(self.ADDR, self.WAKE)
+            finally:
+                self.i2c.unlock()
+            time.sleep(0.001)
+
+            # 2. Trigger Measurement
+            while not self.i2c.try_lock():
+                pass
+            try:
+                self.i2c.writeto(self.ADDR, self.MEASURE)
+            finally:
+                self.i2c.unlock()
+            time.sleep(0.015)
+
+            # 3. Read 6 bytes of data
+            # bytes 0, 1: Temp, byte 2: Temp CRC
+            # bytes 3, 4: Hum,  byte 5: Hum CRC
+            buf = bytearray(6)
+            while not self.i2c.try_lock():
+                pass
+            try:
+                self.i2c.readfrom_into(self.ADDR, buf)
+            finally:
+                self.i2c.unlock()
+
+            # 4. Enter sleep mode
+            while not self.i2c.try_lock():
+                pass
+            try:
+                self.i2c.writeto(self.ADDR, self.SLEEP)
+            finally:
+                self.i2c.unlock()
+
+            # Verify CRC
+            t_data = buf[0:2]
+            t_crc = buf[2]
+            h_data = buf[3:5]
+            h_crc = buf[5]
+
+            if self._crc8(t_data) != t_crc:
+                print("SHTC3 Temp CRC error")
+                return None, None
+            if self._crc8(h_data) != h_crc:
+                print("SHTC3 Hum CRC error")
+                return None, None
+
+            raw_t = (buf[0] << 8) | buf[1]
+            raw_h = (buf[3] << 8) | buf[4]
+
+            temp = -45.0 + 175.0 * (raw_t / 65536.0)
+            hum = 100.0 * (raw_h / 65536.0)
+
+            return round(temp, 2), round(hum, 2)
+
+        except Exception as e:
+            print(f"Error reading SHTC3 sensor: {e}")
+            try:
+                while not self.i2c.try_lock():
+                    pass
+                try:
+                    self.i2c.writeto(self.ADDR, self.SLEEP)
+                finally:
+                    self.i2c.unlock()
+            except:
+                pass
+            return None, None

circuitpython/lib/video_stream_cp.py

@@ -0,0 +1,572 @@
+"""CircuitPython Video Stream Server utility.
+
+Listens for incoming TCP/UDP video frames and draws them centered and cropped on the display.
+"""
+
+import time
+
+class VideoStreamServer:
+    def __init__(self, display, pool, tcp_port=8081, udp_port=8082, color_port=8083, color_udp_port=8084):
+        self.display = display
+        self.pool = pool
+        self.tcp_port = tcp_port
+        self.udp_port = udp_port
+        self.color_port = color_port
+        self.color_udp_port = color_udp_port
+
+        # Sockets
+        self.tcp_server = None
+        self.tcp_client = None
+        self.udp_sock = None
+        self.color_server = None
+        self.color_client = None
+        self.color_udp_sock = None
+
+        # State
+        self.active = False
+        self.last_packet_time = 0
+        self.timeout_s = 3.0
+
+        # Frame buffering
+        self.buffer = bytearray(15000)
+        self.view = memoryview(self.buffer)
+        self.tcp_bytes_received = 0
+
+        # UDP Reassembly
+        self.udp_temp_buffer = bytearray(1002)
+        self.current_frame_id = -1
+        self.chunks_received = 0
+        self.color_chunks_mask = 0
+
+        # Color buffering (320x240 RGB565 is 153,600 bytes)
+        self.color_buffer = bytearray(153600)
+        self.color_view = memoryview(self.color_buffer)
+        self.color_bytes_received = 0
+        self.color_header = bytearray(16)
+        self.color_header_received = 0
+        self.color_payload_len = 0
+        self.color_x = 0
+        self.color_y = 0
+        self.color_w = 0
+        self.color_h = 0
+
+        # Stats
+        self.debug = False
+        self.frames_drawn = 0
+        self.udp_packets_received = 0
+        self.udp_frames_complete = 0
+        self.dropped_udp_frames = 0
+        self.last_draw_ms = 0
+        self.last_fps = 0.0
+
+        # FPS Calculation
+        self.fps_start_time = time.monotonic()
+        self.fps_frame_count = 0
+
+    def start(self):
+        """Initializes TCP and UDP sockets."""
+        # 1. Start TCP Server (Mono)
+        try:
+            self.tcp_server = self.pool.socket(self.pool.AF_INET, self.pool.SOCK_STREAM)
+            try:
+                self.tcp_server.setsockopt(self.pool.SOL_SOCKET, self.pool.SO_REUSEADDR, 1)
+            except:
+                pass
+            self.tcp_server.bind(("", self.tcp_port))
+            self.tcp_server.listen(1)
+            self.tcp_server.setblocking(False)
+            print(f"Video TCP Stream server listening on port {self.tcp_port}...")
+        except Exception as e:
+            print(f"Failed to start TCP stream server: {e}")
+
+        # 2. Start UDP Server (Mono)
+        try:
+            self.udp_sock = self.pool.socket(self.pool.AF_INET, self.pool.SOCK_DGRAM)
+            try:
+                self.udp_sock.setsockopt(self.pool.SOL_SOCKET, self.pool.SO_REUSEADDR, 1)
+            except:
+                pass
+            self.udp_sock.bind(("", self.udp_port))
+            self.udp_sock.setblocking(False)
+            print(f"Video UDP Stream responder listening on port {self.udp_port}...")
+        except Exception as e:
+            print(f"Failed to start UDP stream server: {e}")
+
+        # 3. Start Color TCP Server
+        try:
+            self.color_server = self.pool.socket(self.pool.AF_INET, self.pool.SOCK_STREAM)
+            try:
+                self.color_server.setsockopt(self.pool.SOL_SOCKET, self.pool.SO_REUSEADDR, 1)
+            except:
+                pass
+            self.color_server.bind(("", self.color_port))
+            self.color_server.listen(1)
+            self.color_server.setblocking(False)
+            print(f"Video Color TCP Stream server listening on port {self.color_port}...")
+        except Exception as e:
+            print(f"Failed to start Color TCP server: {e}")
+
+        # 4. Start Color UDP Server
+        try:
+            self.color_udp_sock = self.pool.socket(self.pool.AF_INET, self.pool.SOCK_DGRAM)
+            try:
+                self.color_udp_sock.setsockopt(self.pool.SOL_SOCKET, self.pool.SO_REUSEADDR, 1)
+            except:
+                pass
+            self.color_udp_sock.bind(("", self.color_udp_port))
+            self.color_udp_sock.setblocking(False)
+            print(f"Video Color UDP Stream responder listening on port {self.color_udp_port}...")
+        except Exception as e:
+            print(f"Failed to start Color UDP stream server: {e}")
+
+    def restart_tcp_server(self):
+        print("Restarting TCP Stream Server...")
+        self.close_tcp_client()
+        if self.tcp_server:
+            try:
+                self.tcp_server.close()
+            except:
+                pass
+            self.tcp_server = None
+        time.sleep(0.1)
+        try:
+            self.tcp_server = self.pool.socket(self.pool.AF_INET, self.pool.SOCK_STREAM)
+            self.tcp_server.bind(("", self.tcp_port))
+            self.tcp_server.listen(1)
+            self.tcp_server.setblocking(False)
+        except Exception as e:
+            print(f"Restart TCP Server failed: {e}")
+
+    def restart_udp_sock(self):
+        print("Restarting UDP Stream Socket...")
+        if self.udp_sock:
+            try:
+                self.udp_sock.close()
+            except:
+                pass
+            self.udp_sock = None
+        time.sleep(0.1)
+        try:
+            self.udp_sock = self.pool.socket(self.pool.AF_INET, self.pool.SOCK_DGRAM)
+            self.udp_sock.bind(("", self.udp_port))
+            self.udp_sock.setblocking(False)
+        except Exception as e:
+            print(f"Restart UDP Socket failed: {e}")
+
+    def restart_color_server(self):
+        print("Restarting Color TCP Stream Server...")
+        self.close_color_client()
+        if self.color_server:
+            try:
+                self.color_server.close()
+            except:
+                pass
+            self.color_server = None
+        time.sleep(0.1)
+        try:
+            self.color_server = self.pool.socket(self.pool.AF_INET, self.pool.SOCK_STREAM)
+            self.color_server.bind(("", self.color_port))
+            self.color_server.listen(1)
+            self.color_server.setblocking(False)
+        except Exception as e:
+            print(f"Restart Color TCP Server failed: {e}")
+
+    def restart_color_udp_sock(self):
+        print("Restarting Color UDP Stream Socket...")
+        if self.color_udp_sock:
+            try:
+                self.color_udp_sock.close()
+            except:
+                pass
+            self.color_udp_sock = None
+        time.sleep(0.1)
+        try:
+            self.color_udp_sock = self.pool.socket(self.pool.AF_INET, self.pool.SOCK_DGRAM)
+            self.color_udp_sock.bind(("", self.color_udp_port))
+            self.color_udp_sock.setblocking(False)
+        except Exception as e:
+            print(f"Restart Color UDP Socket failed: {e}")
+
+    def update(self):
+        """Non-blocking socket check for streaming updates."""
+        now = time.monotonic()
+
+        # Check Stream Active Timeout
+        if self.active and (now - self.last_packet_time) > self.timeout_s:
+            print("Video stream timed out. Returning to dashboard.")
+            self.active = False
+            self.close_tcp_client()
+            self.close_color_client()
+
+        # Calculate FPS periodically
+        fps_elapsed = now - self.fps_start_time
+        if fps_elapsed >= 2.0:
+            self.last_fps = self.fps_frame_count / fps_elapsed
+            self.fps_frame_count = 0
+            self.fps_start_time = now
+
+        # 1. Handle UDP reassembly (Mono)
+        if self.udp_sock:
+            while True:
+                try:
+                    # recv_into returns number of bytes read
+                    n = self.udp_sock.recv_into(self.udp_temp_buffer)
+                    if n == 0:
+                        break
+
+                    self.udp_packets_received += 1
+                    frame_id = self.udp_temp_buffer[0]
+                    chunk_idx = self.udp_temp_buffer[1]
+
+                    if chunk_idx < 15:
+                        self.active = True
+                        self.last_packet_time = now
+                        if frame_id != self.current_frame_id:
+                            if self.chunks_received != 0:
+                                self.dropped_udp_frames += 1
+                            self.current_frame_id = frame_id
+                            self.chunks_received = 0
+
+                        # Copy payload to self.buffer
+                        start_offset = chunk_idx * 1000
+                        self.buffer[start_offset : start_offset + 1000] = self.udp_temp_buffer[2:1002]
+                        self.chunks_received |= (1 << chunk_idx)
+
+                        if self.chunks_received == 0x7FFF:
+                            self.udp_frames_complete += 1
+                            self.active = True
+                            self.last_packet_time = now
+                            self._draw_frame()
+                            self.chunks_received = 0
+                except OSError as e:
+                    import errno
+                    err = getattr(e, 'errno', None)
+                    if err is None and e.args:
+                        err = e.args[0]
+                    ewouldblock = getattr(errno, 'EWOULDBLOCK', errno.EAGAIN)
+                    if err in (errno.EAGAIN, ewouldblock) or err is None:
+                        break
+                    print(f"UDP Socket error: {e}")
+                    self.restart_udp_sock()
+                    break
+
+        # 1B. Handle UDP reassembly (Color)
+        if self.color_udp_sock:
+            while True:
+                try:
+                    n = self.color_udp_sock.recv_into(self.udp_temp_buffer)
+                    if n == 0:
+                        break
+ 
+                    self.udp_packets_received += 1
+                    frame_id = self.udp_temp_buffer[0]
+                    chunk_idx = self.udp_temp_buffer[1]
+ 
+                    if chunk_idx < 154:
+                        self.active = True
+                        self.last_packet_time = now
+                        if frame_id != self.current_frame_id:
+                            self.current_frame_id = frame_id
+                            self.color_chunks_mask = 0
+ 
+                        # Copy payload to self.color_buffer
+                        start_offset = chunk_idx * 1000
+                        if start_offset + 1000 <= 153600:
+                            self.color_buffer[start_offset : start_offset + 1000] = self.udp_temp_buffer[2:1002]
+                            self.color_chunks_mask |= (1 << chunk_idx)
+ 
+                            if self.color_chunks_mask == 0x3FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:
+                                self.udp_frames_complete += 1
+                                self.active = True
+                                self.last_packet_time = now
+                                self.color_x = 0
+                                self.color_y = 0
+                                self.color_w = 320
+                                self.color_h = 240
+                                self.color_payload_len = 153600
+                                self._draw_color_frame()
+                                self.color_chunks_mask = 0
+                except OSError as e:
+                    import errno
+                    err = getattr(e, 'errno', None)
+                    if err is None and e.args:
+                        err = e.args[0]
+                    ewouldblock = getattr(errno, 'EWOULDBLOCK', errno.EAGAIN)
+                    if err in (errno.EAGAIN, ewouldblock) or err is None:
+                        break
+                    print(f"Color UDP Socket error: {e}")
+                    self.restart_color_udp_sock()
+                    break
+
+        # 2. Handle TCP stream (Mono)
+        if self.tcp_server:
+            if self.tcp_client is None:
+                try:
+                    self.tcp_client, addr = self.tcp_server.accept()
+                    self.tcp_client.setblocking(False)
+                    self.tcp_bytes_received = 0
+                    self.active = True
+                    self.last_packet_time = now
+                    print(f"TCP Stream client connected from: {addr}")
+                except OSError as e:
+                    import errno
+                    err = getattr(e, 'errno', None)
+                    if err is None and e.args:
+                        err = e.args[0]
+                    ewouldblock = getattr(errno, 'EWOULDBLOCK', errno.EAGAIN)
+                    if err not in (errno.EAGAIN, ewouldblock) and err is not None:
+                        print(f"TCP Accept error: {e}")
+                        self.restart_tcp_server()
+
+            if self.tcp_client is not None:
+                retries = 0
+                while self.tcp_bytes_received < 15000:
+                    remaining = 15000 - self.tcp_bytes_received
+                    slice_view = self.view[self.tcp_bytes_received : self.tcp_bytes_received + remaining]
+                    try:
+                        n = self.tcp_client.recv_into(slice_view)
+                        if n > 0:
+                            self.tcp_bytes_received += n
+                            self.last_packet_time = now
+                            self.active = True
+                            retries = 0
+                        elif n == 0:
+                            print("TCP Stream client disconnected.")
+                            self.close_tcp_client()
+                            break
+                    except OSError as e:
+                        import errno
+                        err = getattr(e, 'errno', None)
+                        if err is None and e.args:
+                            err = e.args[0]
+                        ewouldblock = getattr(errno, 'EWOULDBLOCK', errno.EAGAIN)
+                        if err in (errno.EAGAIN, ewouldblock) or err is None:
+                            retries += 1
+                            if retries > 15:
+                                break
+                            time.sleep(0.001)
+                        else:
+                            print(f"TCP Stream recv error: {e}")
+                            self.close_tcp_client()
+                            break
+
+                if self.tcp_bytes_received == 15000:
+                    self.tcp_bytes_received = 0
+                    self._draw_frame()
+
+        # 3. Handle Color TCP stream
+        if self.color_server:
+            if self.color_client is None:
+                try:
+                    self.color_client, addr = self.color_server.accept()
+                    self.color_client.setblocking(False)
+                    self.color_bytes_received = 0
+                    self.color_header_received = 0
+                    self.color_payload_len = 0
+                    self.active = True
+                    self.last_packet_time = now
+                    print(f"Color TCP Stream client connected from: {addr}")
+                except OSError as e:
+                    import errno
+                    err = getattr(e, 'errno', None)
+                    if err is None and e.args:
+                        err = e.args[0]
+                    ewouldblock = getattr(errno, 'EWOULDBLOCK', errno.EAGAIN)
+                    if err not in (errno.EAGAIN, ewouldblock) and err is not None:
+                        print(f"Color TCP Accept error: {e}")
+                        self.restart_color_server()
+
+            if self.color_client is not None:
+                try:
+                    # Read header (16 bytes)
+                    if self.color_header_received < 16:
+                        start_h = time.monotonic()
+                        while self.color_header_received < 16:
+                            if (time.monotonic() - start_h) > 0.100:
+                                break
+                            remaining_h = 16 - self.color_header_received
+                            slice_h = memoryview(self.color_header)[self.color_header_received : self.color_header_received + remaining_h]
+                            n = self.color_client.recv_into(slice_h)
+                            if n > 0:
+                                self.color_header_received += n
+                            elif n == 0:
+                                self.close_color_client()
+                                return
+
+                        if self.color_header_received == 16:
+                            # Detect version byte at index 4
+                            version = self.color_header[4]
+                            if version == 1:
+                                # stream_color.py format: sig (4B), version (1B), format (1B), width (2B), height (2B), payload_len (4B), reserved (2B)
+                                self.color_x = 0
+                                self.color_y = 0
+                                self.color_w = (self.color_header[6] << 8) | self.color_header[7]
+                                self.color_h = (self.color_header[8] << 8) | self.color_header[9]
+                                self.color_payload_len = (self.color_header[10] << 24) | (self.color_header[11] << 16) | (self.color_header[12] << 8) | self.color_header[13]
+                            else:
+                                # iPhone app format: sig (4B), x (2B), y (2B), width (2B), height (2B), payload_len (4B)
+                                self.color_x = (self.color_header[4] << 8) | self.color_header[5]
+                                self.color_y = (self.color_header[6] << 8) | self.color_header[7]
+                                self.color_w = (self.color_header[8] << 8) | self.color_header[9]
+                                self.color_h = (self.color_header[10] << 8) | self.color_header[11]
+                                self.color_payload_len = (self.color_header[12] << 24) | (self.color_header[13] << 16) | (self.color_header[14] << 8) | self.color_header[15]
+                            
+                            # Safety check:
+                            if self.color_payload_len > len(self.color_buffer):
+                                print(f"Warning: Color payload length {self.color_payload_len} exceeds preallocated buffer {len(self.color_buffer)}. Closing connection.")
+                                self.close_color_client()
+                                return
+                            
+                            self.color_bytes_received = 0
+ 
+                    # Read payload
+                    if self.color_header_received == 16 and self.color_payload_len > 0:
+                        retries = 0
+                        while self.color_bytes_received < self.color_payload_len:
+                            remaining_p = self.color_payload_len - self.color_bytes_received
+                            slice_p = self.color_view[self.color_bytes_received : self.color_bytes_received + remaining_p]
+                            try:
+                                n = self.color_client.recv_into(slice_p)
+                                if n > 0:
+                                    self.color_bytes_received += n
+                                    self.last_packet_time = now
+                                    self.active = True
+                                    retries = 0
+                                elif n == 0:
+                                    self.close_color_client()
+                                    break
+                            except OSError as e:
+                                import errno
+                                err = getattr(e, 'errno', None)
+                                if err is None and e.args:
+                                    err = e.args[0]
+                                ewouldblock = getattr(errno, 'EWOULDBLOCK', errno.EAGAIN)
+                                if err in (errno.EAGAIN, ewouldblock) or err is None:
+                                    retries += 1
+                                    if retries > 25: # max 25ms wait total per frame
+                                        break
+                                    time.sleep(0.001)
+                                else:
+                                    print(f"Color TCP Stream recv error during payload: {e}")
+                                    self.close_color_client()
+                                    break
+
+                        if self.color_bytes_received == self.color_payload_len:
+                            self._draw_color_frame()
+                            self.color_header_received = 0
+                            self.color_payload_len = 0
+                            self.color_bytes_received = 0
+                except OSError as e:
+                    import errno
+                    err = getattr(e, 'errno', None)
+                    if err is None and e.args:
+                        err = e.args[0]
+                    ewouldblock = getattr(errno, 'EWOULDBLOCK', errno.EAGAIN)
+                    if err not in (errno.EAGAIN, ewouldblock) and err is not None:
+                        print(f"Color TCP Stream recv error: {e}")
+                        self.close_color_client()
+
+    def rlcd_to_mono_cp(self, rlcd_buf, canvas_buf, width, height):
+        for i in range(len(canvas_buf)):
+            canvas_buf[i] = 0
+
+        dx = (400 - width) // 2
+        dy = (300 - height) // 2
+        width_bytes = width // 8
+
+        for index in range(15000):
+            val = rlcd_buf[index]
+            if val == 0:
+                continue
+
+            byte_x = index // 75
+            block_y = index % 75
+            x_base = 2 * byte_x
+            y_base = 299 - 4 * block_y
+
+            for local_y in range(4):
+                for local_x in range(2):
+                    bit = 7 - (local_y * 2 + local_x)
+                    if val & (1 << bit):
+                        x = x_base + local_x
+                        y = y_base - local_y
+                        screen_x = x - dx
+                        screen_y = y - dy
+
+                        if 0 <= screen_x < width and 0 <= screen_y < height:
+                            byte_idx = screen_y * width_bytes + (screen_x >> 3)
+                            bit_idx = 7 - (screen_x & 7)
+                            canvas_buf[byte_idx] |= (1 << bit_idx)
+
+    def _draw_frame(self):
+        draw_start = time.monotonic()
+
+        # Check if RLCD display vs standard ILI9341 display
+        disp_name = self.display.__class__.__name__
+        if disp_name == "RLCD":
+            # Direct SPI write commands for RLCD layout
+            self.display.write_cmd(0x2A)
+            self.display.write_data([0x12, 0x2A])
+            self.display.write_cmd(0x2B)
+            self.display.write_data([0x00, 0xC7])
+            self.display.write_cmd(0x2C)
+            self.display.write_data(self.buffer)
+        else:
+            # Map 400x300 RLCD buffer into the ILI9341 320x240 canvas buffer
+            self.rlcd_to_mono_cp(self.buffer, self.display.canvas_buffer, self.display.width, self.display.height)
+            self.display.show()
+
+        self.last_draw_ms = int((time.monotonic() - draw_start) * 1000)
+        self.frames_drawn += 1
+        self.fps_frame_count += 1
+
+    def _draw_color_frame(self):
+        draw_start = time.monotonic()
+        if hasattr(self.display, "draw_rgb565"):
+            self.display.draw_rgb565(
+                self.color_x,
+                self.color_y,
+                self.color_w,
+                self.color_h,
+                self.color_view[:self.color_payload_len],
+                sync_canvas=False,
+            )
+        else:
+            # Fallback if no raw RGB565 method is exposed (e.g. standard RLCD)
+            pass
+
+        self.last_draw_ms = int((time.monotonic() - draw_start) * 1000)
+        self.frames_drawn += 1
+        self.fps_frame_count += 1
+
+    def get_stats(self):
+        return {
+            "frames_drawn": self.frames_drawn,
+            "tcp_bytes_received": self.tcp_bytes_received,
+            "udp_packets_received": self.udp_packets_received,
+            "udp_frames_complete": self.udp_frames_complete,
+            "dropped_udp_frames": self.dropped_udp_frames,
+            "last_draw_ms": self.last_draw_ms,
+            "last_fps": round(self.last_fps, 1),
+            "active": self.active,
+        }
+
+    def close_tcp_client(self):
+        if self.tcp_client:
+            try:
+                self.tcp_client.close()
+            except:
+                pass
+            self.tcp_client = None
+        self.tcp_bytes_received = 0
+
+    def close_color_client(self):
+        if self.color_client:
+            try:
+                self.color_client.close()
+            except:
+                pass
+            self.color_client = None
+        self.color_bytes_received = 0
+        self.color_header_received = 0
+        self.color_payload_len = 0

demo_audio_loopback.py

@@ -0,0 +1,236 @@
+import time
+import machine
+from machine import Pin, I2S
+import board_config
+
+def play_ram_pcm(audio_chunks, channels=2, rate=16000, bits=16, volume=90):
+    from audio_util import ES8311
+    
+    # 1. Start MCLK PWM using board config parameters
+    mclk_pwm = None
+    if board_config.audio_mclk_pin is not None:
+        mclk_pin = Pin(board_config.audio_mclk_pin, Pin.OUT)
+        mclk_pwm = machine.PWM(mclk_pin)
+        mclk_pwm.freq(board_config.audio_mclk_freq)
+        mclk_pwm.duty_u16(32768)
+        
+    # 2. Wake up and configure ES8311 DAC
+    dac = ES8311(board_config.i2c_bus)
+    dac.init(sample_rate=rate)
+    dac.set_volume(volume)
+    
+    # 3. Configure I2S TX
+    i2s_format = I2S.MONO if channels == 1 else I2S.STEREO
+    i2s = I2S(1,
+              sck=Pin(board_config.audio_i2s_sck),
+              ws=Pin(board_config.audio_i2s_ws),
+              sd=Pin(board_config.audio_i2s_tx_sd),
+              mode=I2S.TX,
+              ibuf=4096,
+              rate=rate,
+              bits=bits,
+              format=i2s_format)
+              
+    # 4. Enable Speaker Amplifier
+    on_val = 0 if board_config.audio_amp_active_level == 0 else 1
+    off_val = 1 if board_config.audio_amp_active_level == 0 else 0
+    amp_pin = Pin(board_config.audio_amp_pin, Pin.OUT, value=on_val)
+    
+    try:
+        print(f"Streaming raw audio to speaker from RAM ({len(audio_chunks)} chunks)...")
+        for chunk in audio_chunks:
+            i2s.write(chunk)
+    except Exception as e:
+        print("Error during RAM playback:", e)
+    finally:
+        time.sleep_ms(100) # Let buffer play out
+        amp_pin.value(off_val) # Disable amp
+        i2s.deinit()
+        if mclk_pwm:
+            mclk_pwm.deinit()
+        print("Playback complete.")
+
+def main():
+    display = board_config.display_instance
+    print("=== Dynamic RAM-based Audio Loopback Script ===")
+    print(f"Board Detected: {board_config.BOARD_TYPE}")
+    
+    # Initialize buttons using polling Pins instead of BoardButtons class
+    key_pin = Pin(18, Pin.IN, Pin.PULL_UP)
+    boot_pin = Pin(0, Pin.IN, Pin.PULL_UP)
+    
+    # Helper to check if trigger is active
+    def is_talk_trigger_active():
+        if board_config.touch:
+            return board_config.touch.is_touched()
+        return key_pin.value() == 0
+        
+    while True:
+        if display:
+            display.clear(0)
+            if board_config.BOARD_TYPE == 'WAVESHARE_RLCD':
+                display.text("RLCD Audio Loopback Test", 10, 10, 1)
+                display.line(10, 20, 390, 20, 1)
+                display.text("1. Press KEY button to record 10s", 15, 60, 1)
+                display.text("2. Playback will start automatically", 15, 80, 1)
+                display.text("Ready (RAM-based)...", 15, 120, 1)
+            else:
+                display.text("Touch Audio Loopback Test", 10, 10, 1)
+                display.line(10, 20, 310, 20, 1)
+                display.text("1. Press screen/KEY to record 10s", 10, 50, 1)
+                display.text("2. Playback starts automatically", 10, 70, 1)
+                display.text("Ready (RAM-based)...", 10, 100, 1)
+            display.show()
+            
+        print("Ready: Press and hold key/screen to record...")
+        start_wait = time.ticks_ms()
+        while not is_talk_trigger_active():
+            if boot_pin.value() == 0 or time.ticks_diff(time.ticks_ms(), start_wait) > 120000:
+                print("Exit condition met. Deleting flag and resetting...")
+                if display:
+                    display.clear(0)
+                    if board_config.BOARD_TYPE == 'WAVESHARE_RLCD':
+                        display.text("Exiting test...", 15, 60, 1)
+                        display.text("Rebooting to normal...", 15, 80, 1)
+                    else:
+                        display.text("Exiting test...", 10, 50, 1)
+                        display.text("Rebooting to normal...", 10, 70, 1)
+                    display.show()
+                time.sleep(1)
+                try:
+                    import os
+                    os.remove("run_loopback.txt")
+                except:
+                    pass
+                import machine
+                machine.reset()
+            time.sleep_ms(30)
+            
+        print("Recording started! Speak into the microphone...")
+        if display:
+            display.clear(0)
+            if board_config.BOARD_TYPE == 'WAVESHARE_RLCD':
+                display.text("RLCD Audio Loopback Test", 10, 10, 1)
+                display.line(10, 20, 390, 20, 1)
+                display.text("RECORDING: 10 seconds...", 15, 80, 1)
+                display.text("Speak into microphone!", 15, 100, 1)
+            else:
+                display.text("Touch Audio Loopback Test", 10, 10, 1)
+                display.line(10, 20, 310, 20, 1)
+                display.text("RECORDING: 10 seconds...", 10, 60, 1)
+                display.text("Speak now!", 10, 80, 1)
+            display.show()
+            
+        # 1. Start MCLK PWM using board config parameters
+        mclk_pwm = None
+        if board_config.audio_mclk_pin is not None:
+            mclk_pin = Pin(board_config.audio_mclk_pin, Pin.OUT)
+            mclk_pwm = machine.PWM(mclk_pin)
+            mclk_pwm.freq(board_config.audio_mclk_freq)
+            mclk_pwm.duty_u16(32768)
+            
+        # 2. Configure I2S RX (Stereo 16kHz) - ibuf set to 16000 for safety
+        i2s_rx = I2S(1, 
+                     sck=Pin(board_config.audio_i2s_sck), 
+                     ws=Pin(board_config.audio_i2s_ws), 
+                     sd=Pin(board_config.audio_i2s_rx_sd), 
+                     mode=I2S.RX, 
+                     ibuf=16000, 
+                     rate=16000, 
+                     bits=16, 
+                     format=I2S.STEREO)
+                     
+        # 3. Wake up and configure the microphone chip (ES7210 vs ES8311)
+        init_ok = False
+        if board_config.audio_mic_codec == "ES7210":
+            from audio_util import ES7210
+            mic_adc = ES7210(board_config.i2c_bus)
+            init_ok = mic_adc.init(sample_rate=16000, bit_width=16)
+        else:
+            from audio_util import ES8311
+            mic_adc = ES8311(board_config.i2c_bus)
+            if mic_adc.init(sample_rate=16000):
+                init_ok = True
+                mic_adc.set_volume(80)
+                try:
+                    mic_adc._write(0x14, 0x1A) # Enable analog mic input & PGA
+                    mic_adc._write(0x16, 0x01) # Enable +6dB gain boost
+                    mic_adc._write(0x17, 0xC8) # Set ADC digital volume
+                except:
+                    pass
+
+        if not init_ok:
+            print("Microphone codec initialization failed! Aborting recording.")
+            i2s_rx.deinit()
+            if mclk_pwm:
+                mclk_pwm.deinit()
+            if display:
+                display.clear(0)
+                display.text("Codec Init Failed!", 15, 80, 1)
+                display.show()
+            time.sleep(3)
+            continue
+                    
+        # Record loop to RAM - using ticks_ms for safe timing
+        buffer = bytearray(2048)
+        audio_chunks = []
+        total_bytes = 0
+        start_rec_time = time.ticks_ms()
+        max_duration_ms = 10000 # 10 seconds
+        
+        try:
+            while time.ticks_diff(time.ticks_ms(), start_rec_time) < max_duration_ms:
+                bytes_read = i2s_rx.readinto(buffer)
+                if bytes_read > 0:
+                    audio_chunks.append(bytes(buffer[:bytes_read]))
+                    total_bytes += bytes_read
+        except Exception as e:
+            print("Recording failed:", e)
+        finally:
+            i2s_rx.deinit()
+            if mclk_pwm:
+                mclk_pwm.deinit()
+                
+        print(f"Recorded {total_bytes} bytes in RAM ({len(audio_chunks)} chunks).")
+        
+        # Wait for release of key/trigger to debounce
+        time.sleep_ms(200)
+        
+        # 4. Playback from RAM
+        if display:
+            display.clear(0)
+            if board_config.BOARD_TYPE == 'WAVESHARE_RLCD':
+                display.text("RLCD Audio Loopback Test", 10, 10, 1)
+                display.line(10, 20, 390, 20, 1)
+                display.text("PLAYING BACK RESPONSE...", 15, 80, 1)
+                display.text(f"Bytes: {total_bytes}", 15, 100, 1)
+            else:
+                display.text("Touch Audio Loopback Test", 10, 10, 1)
+                display.line(10, 20, 310, 20, 1)
+                display.text("PLAYING BACK RESPONSE...", 10, 60, 1)
+                display.text(f"Bytes: {total_bytes}", 10, 80, 1)
+            display.show()
+            
+        play_ram_pcm(audio_chunks, channels=2, rate=16000, bits=16, volume=95)
+        
+        if display:
+            display.clear(0)
+            if board_config.BOARD_TYPE == 'WAVESHARE_RLCD':
+                display.text("RLCD Audio Loopback Test", 10, 10, 1)
+                display.line(10, 20, 390, 20, 1)
+                display.text("Playback Finished!", 15, 70, 1)
+                display.text("Release trigger to restart...", 15, 95, 1)
+            else:
+                display.text("Touch Audio Loopback Test", 10, 10, 1)
+                display.line(10, 20, 310, 20, 1)
+                display.text("Playback Finished!", 10, 50, 1)
+                display.text("Release trigger to restart...", 10, 75, 1)
+            display.show()
+            
+        # Wait for release of button/touch to debounce before ready again
+        while is_talk_trigger_active():
+            time.sleep_ms(30)
+        time.sleep_ms(500)
+
+if __name__ == "__main__":
+    main()

demo_hermes_voice.py

@@ -0,0 +1,248 @@
+import time
+import struct
+import machine
+from machine import Pin, SPI, I2C, I2S
+import urequests
+import ili9341
+from ft6336u import FT6336U
+from audio_util import ES8311
+
+# --- CONFIGURATION ---
+HERMES_API_URL = "http://192.168.68.126:8642/api/esp32/voice"
+DEVICE_ID = "kitchen-button"
+
+# Placeholder for API Key. Since the key is stored on the Hermes server,
+# please copy-paste the token string here.
+HERMES_API_KEY = "mcT1YA1vOr9wXSiHpCYalweEGGZKX-PIfZv2drp8BSg"
+
+def create_wav_header(data_size):
+    # Generates a 44-byte WAV header for 16kHz, 16-bit mono PCM
+    riff = b'RIFF'
+    file_size = data_size + 36
+    wave = b'WAVE'
+    fmt = b'fmt '
+    chunk_size = 16
+    audio_format = 1 # PCM
+    channels = 1 # Mono
+    sample_rate = 16000
+    bits_per_sample = 16
+    byte_rate = sample_rate * channels * (bits_per_sample // 8)
+    block_align = channels * (bits_per_sample // 8)
+    data_label = b'data'
+    
+    return struct.pack('<4sI4s4sIHHIIHH4sI', 
+                       riff, file_size, wave, fmt, chunk_size, 
+                       audio_format, channels, sample_rate, byte_rate, 
+                       block_align, bits_per_sample, data_label, data_size)
+
+def main():
+    print("--- Starting Hermes Voice Assistant Demo ---")
+    
+    # 1. Initialize display
+    spi = SPI(1, baudrate=40000000, polarity=0, phase=0, sck=Pin(12), mosi=Pin(11), miso=Pin(13))
+    display = ili9341.ILI9341(spi, cs=Pin(10), dc=Pin(46), bl=Pin(45), rst=None)
+    display.clear(0)
+    display.text("Hermes Assistant", 10, 10, 1)
+    display.show()
+    
+    # 2. Initialize touch
+    i2c = I2C(0, sda=Pin(16), scl=Pin(15))
+    touch = FT6336U(i2c, rst_pin=18, int_pin=17, width=320, height=240, swap_xy=True, invert_x=False, invert_y=True)
+    
+    # 3. Configure audio codec ES8311
+    # We need MCLK pin (GPIO 4) active at 6.144 MHz
+    mclk_pin = Pin(4, Pin.OUT)
+    mclk_pwm = machine.PWM(mclk_pin)
+    mclk_pwm.freq(6144000)
+    mclk_pwm.duty_u16(32768)
+    
+    codec = ES8311(i2c)
+    if not codec.init(sample_rate=16000):
+        print("ES8311 init failed!")
+        return
+        
+    codec.set_volume(90)
+    
+    # Configure microphone registers on ES8311
+    try:
+        codec._write(0x14, 0x1A) # Enable analog mic input & set PGA gain
+        codec._write(0x16, 0x01) # Boost MIC digital gain to +6dB
+        codec._write(0x17, 0xC8) # Set ADC digital volume
+        print("Microphone registers initialized.")
+    except Exception as e:
+        print("Failed to write microphone registers:", e)
+        
+    amp_pin = Pin(1, Pin.OUT, value=1) # start with amp disabled (1 = disabled)
+    
+    buffer = bytearray(1024)
+    
+    while True:
+        display.clear(0)
+        display.text("Hermes Assistant", 10, 10, 1)
+        display.line(10, 22, 310, 22, 1)
+        display.text("Hold screen & ask", 50, 70, 1)
+        display.text("a question...", 50, 90, 1)
+        display.text("Status: Idle", 10, 220, 1)
+        display.show()
+        
+        # Wait for touch
+        while not touch.is_touched():
+            time.sleep_ms(30)
+            
+        print("Touch detected! Starting recording to RAM...")
+        display.clear(0)
+        display.text("Hermes Assistant", 10, 10, 1)
+        display.line(10, 22, 310, 22, 1)
+        display.text("Listening...", 80, 80, 1)
+        display.fill_rect(130, 110, 30, 30, 1)
+        display.text("Status: Recording", 10, 220, 1)
+        display.show()
+        
+        # 4. Open I2S RX for recording (Mono 16kHz for Hermes STT)
+        i2s_rx = I2S(1, 
+                     sck=Pin(5), 
+                     ws=Pin(7), 
+                     sd=Pin(6), 
+                     mode=I2S.RX, 
+                     ibuf=8000, 
+                     rate=16000, 
+                     bits=16, 
+                     format=I2S.MONO)
+                     
+        audio_chunks = []
+        total_data_bytes = 0
+        
+        try:
+            # Record loop - as long as touch is held
+            while touch.is_touched():
+                bytes_read = i2s_rx.readinto(buffer)
+                if bytes_read > 0:
+                    audio_chunks.append(bytes(buffer[:bytes_read]))
+                    total_data_bytes += bytes_read
+                
+            print(f"Touch released! Recorded {total_data_bytes} bytes.")
+                
+        except Exception as e:
+            print("Error recording:", e)
+        finally:
+            i2s_rx.deinit()
+            
+        if total_data_bytes < 1000:
+            print("Recording too short, ignoring.")
+            continue
+            
+        # 5. Connect and POST to Hermes voice endpoint
+        display.clear(0)
+        display.text("Hermes Assistant", 10, 10, 1)
+        display.line(10, 22, 310, 22, 1)
+        display.text("Sending audio...", 50, 80, 1)
+        display.text("Waiting for agent...", 50, 100, 1)
+        display.text("Status: Processing", 10, 220, 1)
+        display.show()
+        
+        # Join chunks and prepend WAV header
+        raw_pcm = b"".join(audio_chunks)
+        wav_data = create_wav_header(len(raw_pcm)) + raw_pcm
+        
+        # Headers
+        headers = {
+            "Authorization": f"Bearer {HERMES_API_KEY}",
+            "Content-Type": "audio/wav",
+            "X-Device-ID": DEVICE_ID
+        }
+        
+        print(f"Posting raw WAV ({len(wav_data)} bytes) to {HERMES_API_URL}...")
+        try:
+            res = urequests.post(HERMES_API_URL, headers=headers, data=wav_data)
+            print(f"HTTP Status: {res.status_code}")
+            
+            if res.status_code == 200:
+                response_data = res.content
+                print(f"Received {len(response_data)} bytes of audio response.")
+                
+                # Check response format
+                if response_data[0:4] == b'RIFF' and response_data[8:12] == b'WAVE':
+                    # Parse WAV header
+                    idx = response_data.find(b'fmt ')
+                    if idx != -1:
+                        fmt_chunk = response_data[idx:idx+24]
+                        audio_format, channels, sample_rate, byte_rate, block_align, bits = struct.unpack('<HHIIHH', fmt_chunk[8:24])
+                        print(f"Playing WAV: {sample_rate}Hz, {channels}ch, {bits}bit")
+                        
+                        # Find data chunk
+                        data_idx = response_data.find(b'data')
+                        if data_idx != -1:
+                            audio_start = data_idx + 8
+                            
+                            display.clear(0)
+                            display.text("Hermes Assistant", 10, 10, 1)
+                            display.line(10, 22, 310, 22, 1)
+                            display.text("Playing response...", 50, 80, 1)
+                            display.text("Status: Speaking", 10, 220, 1)
+                            display.show()
+                            
+                            # Enable amp & play
+                            amp_pin.value(0) # Active Low Enable
+                            i2s_format = I2S.MONO if channels == 1 else I2S.STEREO
+                            i2s_tx = I2S(1,
+                                         sck=Pin(5),
+                                         ws=Pin(7),
+                                         sd=Pin(8),
+                                         mode=I2S.TX,
+                                         ibuf=4096,
+                                         rate=sample_rate,
+                                         bits=bits,
+                                         format=i2s_format)
+                            try:
+                                pos = audio_start
+                                chunk_size = 2048
+                                while pos < len(response_data):
+                                    chunk = response_data[pos:pos+chunk_size]
+                                    i2s_tx.write(chunk)
+                                    pos += chunk_size
+                            finally:
+                                time.sleep_ms(150)
+                                amp_pin.value(1) # Disable
+                                i2s_tx.deinit()
+                    else:
+                        print("Error: Could not parse WAV format.")
+                else:
+                    # Check if it is MP3
+                    if response_data[0:3] == b'ID3' or (len(response_data) > 2 and response_data[0] == 0xFF and (response_data[1] & 0xE0) == 0xE0):
+                        print("Error: Server returned MP3 format. ESP32 requires WAV.")
+                        display.clear(0)
+                        display.text("Error: Got MP3", 10, 10, 1)
+                        display.line(10, 22, 310, 22, 1)
+                        display.text("Hermes returned MP3.", 30, 80, 1)
+                        display.text("Configure server to", 30, 100, 1)
+                        display.text("convert MP3 to WAV.", 30, 120, 1)
+                        display.show()
+                        time.sleep(4)
+                    else:
+                        print(f"Error: Unknown response format. Starts with: {response_data[0:16]}")
+            else:
+                print(f"Request failed with status {res.status_code}: {res.text}")
+                display.clear(0)
+                display.text("HTTP Error", 10, 10, 1)
+                display.line(10, 22, 310, 22, 1)
+                display.text(f"Status: {res.status_code}", 50, 80, 1)
+                display.show()
+                time.sleep(3)
+                
+        except Exception as e:
+            print("Failed to contact Hermes server:", e)
+            display.clear(0)
+            display.text("Server Error", 10, 10, 1)
+            display.line(10, 22, 310, 22, 1)
+            display.text("Could not connect", 50, 80, 1)
+            display.text("to API gateway.", 50, 100, 1)
+            display.show()
+            time.sleep(3)
+            
+        # Debounce touch release
+        while touch.is_touched():
+            time.sleep_ms(30)
+        time.sleep_ms(300)
+
+if __name__ == "__main__":
+    main()

demo_touch_mic.py

@@ -0,0 +1,144 @@
+import time
+import machine
+from machine import Pin, SPI, I2C, I2S
+import ili9341
+from ft6336u import FT6336U
+from audio_util import ES8311
+
+def main():
+    print("--- Starting RAM-based Touch Mic Demo ---")
+    
+    # 1. Initialize display
+    spi = SPI(1, baudrate=40000000, polarity=0, phase=0, sck=Pin(12), mosi=Pin(11), miso=Pin(13))
+    display = ili9341.ILI9341(spi, cs=Pin(10), dc=Pin(46), bl=Pin(45), rst=None)
+    display.clear(0)
+    display.text("RAM Touch Mic", 10, 10, 1)
+    display.show()
+    
+    # 2. Initialize touch
+    i2c = I2C(0, sda=Pin(16), scl=Pin(15))
+    touch = FT6336U(i2c, rst_pin=18, int_pin=17, width=320, height=240, swap_xy=True, invert_x=False, invert_y=True)
+    
+    # 3. Configure audio codec ES8311
+    # Generate MCLK PWM on GPIO 4 at 6.144 MHz
+    mclk_pin = Pin(4, Pin.OUT)
+    mclk_pwm = machine.PWM(mclk_pin)
+    mclk_pwm.freq(6144000)
+    mclk_pwm.duty_u16(32768)
+    
+    codec = ES8311(i2c)
+    if not codec.init(sample_rate=16000):
+        print("ES8311 init failed!")
+        return
+        
+    # Set speaker volume to 95%
+    codec.set_volume(95)
+    
+    # Configure microphone registers on ES8311
+    try:
+        codec._write(0x14, 0x1A) # Enable analog mic input & set PGA gain
+        codec._write(0x16, 0x01) # Boost MIC digital gain to +6dB to prevent saturation
+        codec._write(0x17, 0xC8) # Set ADC digital volume
+        print("Microphone registers initialized with gain boost.")
+    except Exception as e:
+        print("Failed to write microphone registers:", e)
+        
+    amp_pin = Pin(1, Pin.OUT, value=1) # start with amp disabled (1 = disabled)
+    
+    # We allocate a buffer for I2S read
+    buffer = bytearray(2048)
+    
+    while True:
+        display.clear(0)
+        display.text("RAM Touch Mic", 10, 10, 1)
+        display.line(10, 22, 310, 22, 1)
+        display.text("Press & hold screen", 50, 80, 1)
+        display.text("to record voice...", 50, 100, 1)
+        display.show()
+        
+        # Wait for touch
+        while not touch.is_touched():
+            time.sleep_ms(30)
+            
+        print("Touch detected! Starting recording to RAM...")
+        display.clear(0)
+        display.text("Recording...", 10, 10, 1)
+        display.line(10, 22, 310, 22, 1)
+        display.text("SPEAK NOW!", 80, 80, 1)
+        display.fill_rect(130, 110, 30, 30, 1)
+        display.show()
+        
+        # 4. Open I2S RX for recording
+        i2s_rx = I2S(1, 
+                     sck=Pin(5), 
+                     ws=Pin(7), 
+                     sd=Pin(6), 
+                     mode=I2S.RX, 
+                     ibuf=8000, 
+                     rate=16000, 
+                     bits=16, 
+                     format=I2S.STEREO)
+                     
+        audio_chunks = []
+        total_data_bytes = 0
+        
+        try:
+            # Record loop - append chunks directly in RAM
+            while touch.is_touched():
+                bytes_read = i2s_rx.readinto(buffer)
+                if bytes_read > 0:
+                    audio_chunks.append(bytes(buffer[:bytes_read]))
+                    total_data_bytes += bytes_read
+                
+            print(f"Touch released! Recorded {total_data_bytes} bytes in RAM.")
+                
+        except Exception as e:
+            print("Error recording:", e)
+        finally:
+            i2s_rx.deinit()
+            
+        # 5. Playback recorded audio
+        display.clear(0)
+        display.text("Playing back...", 10, 10, 1)
+        display.line(10, 22, 310, 22, 1)
+        display.text("Listening to RAM...", 50, 80, 1)
+        display.show()
+        
+        print("Starting playback from RAM...")
+        amp_pin.value(0) # Enable amp
+        
+        i2s_tx = I2S(1,
+                     sck=Pin(5),
+                     ws=Pin(7),
+                     sd=Pin(8),
+                     mode=I2S.TX,
+                     ibuf=4096,
+                     rate=16000,
+                     bits=16,
+                     format=I2S.STEREO)
+                     
+        try:
+            for chunk in audio_chunks:
+                i2s_tx.write(chunk)
+        except Exception as e:
+            print("Error during playback:", e)
+        finally:
+            time.sleep_ms(100) # wait to clear buffer
+            amp_pin.value(1) # Disable amp
+            i2s_tx.deinit()
+            
+        print("Playback finished.")
+        display.clear(0)
+        display.text("Finished!", 10, 10, 1)
+        display.line(10, 22, 310, 22, 1)
+        display.text("Touch screen to", 50, 80, 1)
+        display.text("record again.", 50, 100, 1)
+        display.show()
+        
+        # Debounce touch release
+        while touch.is_touched():
+            time.sleep_ms(30)
+        time.sleep_ms(400)
+
+if __name__ == "__main__":
+    main()

demo_websocket_voice.py

@@ -0,0 +1,358 @@
+# pyright: reportMissingImports=false, reportAttributeAccessIssue=false
+"""Client firmware application for real-time WebSocket audio streaming to Hermes.
+
+Dynamically uses board_config to work across Hosyond and Waveshare RLCD boards.
+"""
+
+import time
+import struct
+import machine
+from machine import Pin, I2S
+import json
+import board_config
+from websocket_client import WebSocketClient
+
+# --- CONFIGURATION ---
+HERMES_WS_URL = "ws://192.168.68.126:8642/api/esp32/voice/ws"
+# Determine dynamic device ID based on board configuration
+DEVICE_ID = "esp32_screen" if board_config.BOARD_TYPE == 'WAVESHARE_RLCD' else "little32"
+
+# Placeholder for API Key. Since the key is stored on the Hermes server,
+# please copy-paste the token string here.
+HERMES_API_KEY = "mcT1YA1vOr9wXSiHpCYalweEGGZKX-PIfZv2drp8BSg"
+
+def main():
+    print("=== Starting Hermes WebSocket Voice Assistant Demo (Type: {}) ===".format(board_config.BOARD_TYPE))
+    
+    # 1. Use pre-initialized display and touch from board_config
+    display = board_config.display_instance
+    touch = board_config.touch
+    
+    if display:
+        display.clear(0)
+        display.text("Hermes Assistant", 10, 10, 1)
+        display.show()
+    
+    # 2. Configure Audio Amp control pin based on board config
+    amp_pin = None
+    on_val = 0
+    off_val = 1
+    if board_config.audio_amp_pin is not None:
+        on_val = 0 if board_config.audio_amp_active_level == 0 else 1
+        off_val = 1 if board_config.audio_amp_active_level == 0 else 0
+        amp_pin = Pin(board_config.audio_amp_pin, Pin.OUT, value=off_val)
+
+    # Helper function to check if trigger is active (touch screen or button)
+    def is_talk_trigger_active():
+        if touch:
+            return touch.is_touched()
+        elif hasattr(board_config, 'buttons') and board_config.buttons and board_config.buttons.key:
+            return board_config.buttons.key.is_pressed()
+        return False
+    
+    while True:
+        if display:
+            display.clear(0)
+            display.text("Hermes Assistant", 10, 10, 1)
+            display.line(10, 22, display.width - 10, 22, 1)
+            display.text("Hold screen & ask", 50, 70, 1)
+            display.text("a question...", 50, 90, 1)
+            display.text("Status: Idle (WS)", 10, display.height - 20, 1)
+            display.show()
+        
+        # Wait for touch or button trigger
+        while not is_talk_trigger_active():
+            time.sleep_ms(30)
+            
+        print("Touch/Button detected! Connecting WebSocket...")
+        if display:
+            display.clear(0)
+            display.text("Connecting...", 80, 80, 1)
+            display.show()
+        
+        headers = {
+            "Authorization": f"Bearer {HERMES_API_KEY}",
+            "X-Device-ID": DEVICE_ID
+        }
+        
+        ws = WebSocketClient(HERMES_WS_URL, headers=headers, timeout=30)
+        try:
+            ws.connect()
+            
+            # Send start event
+            ws.send_text(json.dumps({
+                "event": "start",
+                "device_id": DEVICE_ID,
+                "sample_rate": 16000,
+                "channels": 1,
+                "sample_width": 2,
+                "format": "pcm_s16le"
+            }))
+            
+            # Read ready and listening events from server
+            ws.recv_frame()  # ready
+            ws.recv_frame()  # listening
+            
+            print("WebSocket connected and streaming started.")
+            if display:
+                display.clear(0)
+                display.text("Hermes Assistant", 10, 10, 1)
+                display.line(10, 22, display.width - 10, 22, 1)
+                display.text("Listening...", 80, 80, 1)
+                display.fill_rect(130, 110, 30, 30, 1)
+                display.text("Status: Streaming", 10, display.height - 20, 1)
+                display.show()
+            
+            # 3. Configure MCLK PWM
+            mclk_pwm = None
+            if board_config.audio_mclk_pin is not None:
+                mclk_pin = Pin(board_config.audio_mclk_pin, Pin.OUT)
+                mclk_pwm = machine.PWM(mclk_pin)
+                mclk_pwm.freq(board_config.audio_mclk_freq)
+                mclk_pwm.duty_u16(32768)
+
+            # 4. Initialize microphone codec
+            i2c = board_config.i2c_bus
+            if board_config.audio_mic_codec == "ES7210":
+                from audio_util import ES7210
+                codec = ES7210(i2c)
+                codec.init(sample_rate=16000, bit_width=16)
+            else:
+                from audio_util import ES8311
+                codec = ES8311(i2c)
+                if codec.init(sample_rate=16000):
+                    codec.set_volume(80)
+                    try:
+                        codec._write(0x14, 0x1A)
+                        codec._write(0x16, 0x01)
+                        codec._write(0x17, 0xC8)
+                    except:
+                        pass
+
+            # 5. Open I2S RX for recording (Stereo 16kHz for ES7210, mono for ES8311)
+            is_stereo = (board_config.audio_mic_codec == "ES7210")
+            i2s_format = I2S.STEREO if is_stereo else I2S.MONO
+            i2s_rx = I2S(1, 
+                         sck=Pin(board_config.audio_i2s_sck), 
+                         ws=Pin(board_config.audio_i2s_ws), 
+                         sd=Pin(board_config.audio_i2s_rx_sd), 
+                         mode=I2S.RX, 
+                         ibuf=16000, 
+                         rate=16000, 
+                         bits=16, 
+                         format=i2s_format)
+                         
+            total_data_bytes = 0
+            buffer = bytearray(2048)
+            mono_buf = bytearray(1024)
+            
+            rec_start_time = time.ticks_ms()
+            max_rec_duration_ms = 10000
+            
+            try:
+                # Record loop - as long as touch/button is held
+                while is_talk_trigger_active():
+                    elapsed = time.ticks_diff(time.ticks_ms(), rec_start_time)
+                    if elapsed >= max_rec_duration_ms:
+                        print("Recording stopped: maximum duration reached")
+                        break
+                        
+                    bytes_read = i2s_rx.readinto(buffer)
+                    if bytes_read > 0:
+                        if is_stereo:
+                            # Stereo-to-mono: extract left channel (every other 16-bit sample)
+                            mono_len = bytes_read // 2
+                            j = 0
+                            for i in range(0, bytes_read, 4):
+                                mono_buf[j] = buffer[i]
+                                mono_buf[j + 1] = buffer[i + 1]
+                                j += 2
+                            ws.send_binary(mono_buf[:mono_len])
+                            total_data_bytes += mono_len
+                        else:
+                            ws.send_binary(buffer[:bytes_read])
+                            total_data_bytes += bytes_read
+                    
+                print(f"Touch/Button released! Sent {total_data_bytes} bytes.")
+            except Exception as e:
+                print("Error recording/streaming:", e)
+            finally:
+                i2s_rx.deinit()
+                if board_config.audio_mic_codec == "ES8311":
+                    try:
+                        codec._write(0x16, 0x00) # Reset mic gain
+                    except:
+                        pass
+                
+            if total_data_bytes < 3200:
+                print("Recording too short, cancelling session.")
+                try:
+                    ws.send_text(json.dumps({"event": "cancel"}))
+                    ws.close()
+                except:
+                    pass
+                if mclk_pwm:
+                    mclk_pwm.deinit()
+                continue
+                
+            # Send stop event
+            ws.send_text(json.dumps({"event": "stop"}))
+            
+            if display:
+                display.clear(0)
+                display.text("Hermes Assistant", 10, 10, 1)
+                display.line(10, 22, display.width - 10, 22, 1)
+                display.text("Processing...", 50, 80, 1)
+                display.text("Status: Thinking", 10, display.height - 20, 1)
+                display.show()
+            
+            # Playback/Events loop
+            i2s_tx = None
+            received_audio_bytes = 0
+            speaker_write_failed = False
+            while True:
+                opcode, payload = ws.recv_frame()
+                if opcode is None:
+                    break
+                
+                if opcode == 0x1:  # Text frame (JSON event)
+                    try:
+                        event_data = json.loads(payload.decode('utf-8'))
+                        evt = event_data.get("event")
+                        
+                        if evt == "transcript":
+                            txt = event_data.get("text", "")
+                            print(f"Heard: {txt}")
+                            if display:
+                                display.clear(0)
+                                display.text("Hermes Assistant", 10, 10, 1)
+                                display.line(10, 22, display.width - 10, 22, 1)
+                                display.text("Heard:", 10, 40, 1)
+                                lines = [txt[i:i+30] for i in range(0, min(len(txt), 120), 30)]
+                                y_offset = 60
+                                for line in lines:
+                                    display.text(line, 10, y_offset, 1)
+                                    y_offset += 20
+                                display.show()
+                            
+                        elif evt == "thinking":
+                            pass
+                            
+                        elif evt == "response_text":
+                            txt = event_data.get("text", "")
+                            print(f"Response: {txt}")
+                            if display:
+                                display.clear(0)
+                                display.text("Hermes Assistant", 10, 10, 1)
+                                display.line(10, 22, display.width - 10, 22, 1)
+                                display.text("Response:", 10, 40, 1)
+                                lines = [txt[i:i+30] for i in range(0, min(len(txt), 120), 30)]
+                                y_offset = 60
+                                for line in lines:
+                                    display.text(line, 10, y_offset, 1)
+                                    y_offset += 20
+                                display.show()
+                            
+                        elif evt == "audio_start":
+                            print("Audio response started.")
+                            if amp_pin:
+                                amp_pin.value(on_val) # Enable amp
+                            
+                            # Initialize ES8311 Speaker DAC
+                            try:
+                                from audio_util import ES8311
+                                dac = ES8311(i2c)
+                                dac.init(sample_rate=16000)
+                                dac.set_volume(85)
+                            except Exception as dace:
+                                print("Failed to initialize ES8311 DAC for playback:", dace)
+                                
+                            # Open I2S TX
+                            i2s_tx = I2S(1,
+                                         sck=Pin(board_config.audio_i2s_sck),
+                                         ws=Pin(board_config.audio_i2s_ws),
+                                         sd=Pin(board_config.audio_i2s_tx_sd),
+                                         mode=I2S.TX,
+                                         ibuf=4096,
+                                         rate=16000,
+                                         bits=16,
+                                         format=I2S.MONO)
+                            
+                        elif evt == "audio_end":
+                            print("Audio response ended.")
+                            if i2s_tx:
+                                time.sleep_ms(150)
+                                if amp_pin:
+                                    amp_pin.value(off_val) # Disable amp
+                                i2s_tx.deinit()
+                                i2s_tx = None
+                            if display:
+                                display.clear(0)
+                                display.text("Hermes Assistant", 10, 10, 1)
+                                display.line(10, 22, display.width - 10, 22, 1)
+                                if speaker_write_failed:
+                                    display.text("Speaker write failed", 30, 80, 1)
+                                else:
+                                    display.text(f"Recv: {received_audio_bytes} bytes", 30, 80, 1)
+                                display.text("Status: Idle (WS)", 10, display.height - 20, 1)
+                                display.show()
+                                
+                        elif evt == "done":
+                            break
+                            
+                        elif evt == "error":
+                            msg = event_data.get("message", "Unknown error")
+                            print(f"Error from server: {msg}")
+                            if display:
+                                display.clear(0)
+                                display.text("Error", 10, 10, 1)
+                                display.line(10, 22, display.width - 10, 22, 1)
+                                display.text(msg[:100], 10, 80, 1)
+                                display.show()
+                            time.sleep(3)
+                            break
+                    except Exception as e:
+                        print("Error parsing event text:", e)
+                        
+                elif opcode == 0x2:  # Binary frame (Audio WAV chunk)
+                    if i2s_tx:
+                        chunk = payload
+                        if chunk.startswith(b'RIFF') and len(chunk) > 44:
+                            chunk = chunk[44:]  # Skip WAV header for direct play
+                        try:
+                            i2s_tx.write(chunk)
+                            received_audio_bytes += len(chunk)
+                        except Exception as e:
+                            speaker_write_failed = True
+                            print("Error writing to speaker:", e)
+            
+            ws.close()
+            if mclk_pwm:
+                mclk_pwm.deinit()
+        except Exception as e:
+            print("Failed to stream to Hermes server:", e)
+            if display:
+                display.clear(0)
+                display.text("Server Error", 10, 10, 1)
+                display.line(10, 22, display.width - 10, 22, 1)
+                display.text("Could not connect", 50, 80, 1)
+                display.text("to WebSocket server.", 50, 100, 1)
+                display.show()
+            time.sleep(3)
+            try:
+                ws.close()
+            except Exception:
+                pass
+            if mclk_pwm:
+                try:
+                    mclk_pwm.deinit()
+                except:
+                    pass
+            
+        # Debounce touch release
+        while is_talk_trigger_active():
+            time.sleep_ms(30)
+        time.sleep_ms(300)
+
+if __name__ == "__main__":
+    main()

desktop_client/README.md

@@ -0,0 +1,79 @@
+# Desktop Companion Client for AI Agents
+
+This is a cross-platform (Linux, macOS, Windows) client that allows your computer to be used directly by AI Agents as a physical-like communication and display companion.
+
+It implements the Model Context Protocol (MCP) and exposes the host machine's actual hardware:
+- **Audio Output**: Plays pure tones and WAV files on your actual speakers.
+- **Audio Input**: Records voice messages from your microphone.
+- **Battery Status**: Exposes your laptop's real battery voltage and percentage.
+- **System Telemetry**: Reads host metrics (uptime, CPU loading).
+- **Dedicated Canvas Window**: Renders images and text drawn by the agent.
+- **Video Streaming**: Renders incoming 1-bit monochrome video streams (from sources streaming to TCP port 8081 or UDP port 8082).
+- **UDP Discovery**: Responds to local discovery beacons.
+
+## Requirements
+
+- Python 3.10+
+- **Pillow** (PIL) library (already installed in your environment)
+- **Audio Playback**: Uses native system players (`afplay` on macOS, `aplay`/`paplay`/`pw-play` on Linux, `winsound` on Windows).
+- **Audio Recording**:
+  - **Linux**: Standard `arecord` (pre-installed via `alsa-utils`).
+  - **macOS / Windows**: If `sounddevice` or `pyaudio` Python packages are installed, they will be used. Otherwise, it falls back to `sox`/`rec` if available, or generates a clean simulated wave format if no audio recorder is present.
+
+## Running the Client
+
+Start the client manually with:
+```bash
+python3 desktop_client/client.py
+```
+
+Options:
+- `--port`: HTTP JSON-RPC port (default is `8080`)
+- `--width`: Canvas width (default `480`)
+- `--height`: Canvas height (default `320`)
+- `--headless`: Run headlessly without Pygame GUI window (ideal for remote SSH servers)
+
+### Keyboard Controls (GUI Mode)
+- **`D`**: Switch the screen canvas back to the local **Host Telemetry Dashboard**.
+- **`S`**: Save a screenshot of the companion window.
+
+### Hidden Mode Behavior
+To avoid cluttering your screen, the GUI window starts **hidden** on boot. It runs silently in the background and only pops to the foreground when a message (drawing or video stream) is sent by the AI Agent.
+
+---
+
+## Running as a macOS Service (LaunchAgent)
+
+To run the companion client persistently as a background service that launches automatically on login, use the provided scripts:
+
+1. **Install and Start the Service**:
+   ```bash
+   ./desktop_client/setup_service.sh
+   ```
+2. **Uninstall/Stop the Service**:
+   ```bash
+   ./desktop_client/uninstall_service.sh
+   ```
+
+Logs are captured dynamically at:
+- Standard Out: `desktop_client/stdout.log`
+- Standard Error: `desktop_client/stderr.log`
+
+---
+
+## Configuring Claude Desktop / Agent Host
+
+To add this desktop client as an MCP tool provider, add the following to your `claude_desktop_config.json`:
+
+```json
+{
+  "mcpServers": {
+    "desktop-companion": {
+      "command": "/Users/adolforeyna/.pyenv/versions/3.10.12/bin/python3",
+      "args": [
+        "/Users/adolforeyna/Projects/MicroPython/test1/Screen/desktop_client/client.py"
+      ]
+    }
+  }
+}
+```

desktop_client/com.adolforeyna.companionclient.plist

@@ -0,0 +1,23 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key>
+    <string>com.adolforeyna.companionclient</string>
+    <key>ProgramArguments</key>
+    <array>
+        <string>/Users/adolforeyna/.pyenv/versions/3.10.12/bin/python3</string>
+        <string>/Users/adolforeyna/Projects/MicroPython/test1/Screen/desktop_client/client.py</string>
+    </array>
+    <key>WorkingDirectory</key>
+    <string>/Users/adolforeyna/Projects/MicroPython/test1/Screen</string>
+    <key>RunAtLoad</key>
+    <true/>
+    <key>KeepAlive</key>
+    <true/>
+    <key>StandardOutPath</key>
+    <string>/Users/adolforeyna/Projects/MicroPython/test1/Screen/desktop_client/stdout.log</string>
+    <key>StandardErrorPath</key>
+    <string>/Users/adolforeyna/Projects/MicroPython/test1/Screen/desktop_client/stderr.log</string>
+</dict>
+</plist>

desktop_client/setup_service.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+PLIST_NAME="com.adolforeyna.companionclient.plist"
+LOCAL_PLIST="/Users/adolforeyna/Projects/MicroPython/test1/Screen/desktop_client/${PLIST_NAME}"
+TARGET_DIR="${HOME}/Library/LaunchAgents"
+TARGET_PLIST="${TARGET_DIR}/${PLIST_NAME}"
+
+echo "=== Installing Desktop Companion LaunchAgent ==="
+
+# 1. Create target directory if it doesn't exist
+mkdir -p "${TARGET_DIR}"
+
+# 2. Unload the service if it's already running
+echo "Stopping existing agent if running..."
+launchctl bootout gui/$(id -u) "${TARGET_PLIST}" 2>/dev/null
+launchctl unload "${TARGET_PLIST}" 2>/dev/null
+
+# 3. Copy the plist file
+echo "Copying plist configuration..."
+cp "${LOCAL_PLIST}" "${TARGET_PLIST}"
+chmod 644 "${TARGET_PLIST}"
+
+# 4. Load/start the LaunchAgent
+echo "Loading and starting the service..."
+launchctl bootstrap gui/$(id -u) "${TARGET_PLIST}"
+# Or fallback if bootstrap fails
+if [ $? -ne 0 ]; then
+    launchctl load "${TARGET_PLIST}"
+fi
+
+echo "Service successfully installed and started!"
+echo "Check status using: launchctl list | grep com.adolforeyna.companionclient"
+echo "Logs are available at:"
+echo "  STDOUT: /Users/adolforeyna/Projects/MicroPython/test1/Screen/desktop_client/stdout.log"
+echo "  STDERR: /Users/adolforeyna/Projects/MicroPython/test1/Screen/desktop_client/stderr.log"

desktop_client/uninstall_service.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+PLIST_NAME="com.adolforeyna.companionclient.plist"
+TARGET_PLIST="${HOME}/Library/LaunchAgents/${PLIST_NAME}"
+
+echo "=== Uninstalling Desktop Companion LaunchAgent ==="
+
+if [ -f "${TARGET_PLIST}" ]; then
+    echo "Stopping and unloading the service..."
+    launchctl bootout gui/$(id -u) "${TARGET_PLIST}" 2>/dev/null
+    launchctl unload "${TARGET_PLIST}" 2>/dev/null
+    
+    echo "Removing plist configuration..."
+    rm "${TARGET_PLIST}"
+    echo "Service successfully uninstalled!"
+else
+    echo "Service is not installed."
+fi

hardware_findings.md

@@ -0,0 +1,32 @@
+# Waveshare ESP32-S3-RLCD-4.2 & ES7210 Microphone Diagnostics
+
+This document outlines the root causes of the audio recording failures and hardware crashes we encountered with the Waveshare ESP32-S3 board and its onboard ES7210 microphone array, along with their solutions.
+
+## 1. I2S Bit-Depth Mismatch (The "Static Noise" Issue)
+**Problem:** The audio captured by the board was entirely flat or unrecognizable static noise. The audio signal sent to the Whisper pipeline had extremely low RMS levels (25-120), leading to transcription timeouts.
+**Root Cause:** The ES7210 microphone ADC was configured via its internal registers to stream **24-bit** audio (Register `0x11` was set to `0x00`). However, the ESP32's I2S hardware peripheral was configured to receive **16-bit** audio. The ESP32 sliced the 24-bit audio frames into misaligned 16-bit chunks, completely destroying the waveform.
+**Solution:** Modified `audio_util.py` to write `0x60` to Register `0x11`. This locks the ES7210 into native 16-bit Standard I2S output, perfectly aligning it with the ESP32's buffer. 
+
+## 2. I2C Bus Deadlocks (The "Bootloop / Hang" Issue)
+**Problem:** The board would frequently hang during the boot sequence or when attempting to re-initialize the audio components. This occurred primarily after soft-reboots or abrupt script terminations.
+**Root Cause:** The ES7210 chip does not gracefully release the I2C SDA (data) line if communication is interrupted midway. When the ESP32 soft-reboots, the SDA line remains held low by the ES7210, which permanently hangs the ESP32's internal I2C driver on the next boot attempt.
+**Solution:** Added a manual 9-clock I2C hardware recovery sequence to `board_config.py`. Before the `SoftI2C` interface is initialized, the ESP32 manually toggles the SCL pin 9 times as an output to force the ES7210 to release the SDA line, followed by generating a standard I2C STOP condition.
+
+## 3. Incorrect I2C Pin Assignments (The "ENODEV" Issue)
+**Problem:** The audio configuration would occasionally fail with `OSError: [Errno 19] ENODEV`, indicating the I2C bus could not find the microphone at address `0x40`.
+**Root Cause:** The dynamic board-detection logic in `board_config.py` was originally configured to scan for I2C devices on pins 15 and 16 (the default for the Hosyond board). The Waveshare RLCD board uses pins 13 and 14 for the audio I2C bus. 
+**Solution:** Hardcoded the correct I2C pins (SDA=13, SCL=14) for the Waveshare board profile and ensured the I2C scan and initialization processes execute on the correct pins.
+
+## 4. Invalid OSR & Clock Division (The "Popping Sound" Issue)
+**Problem:** Even when I2S and I2C connected successfully, the recorded audio consisted only of loud, constant popping and clipping at max/min bounds (amplitude 32768), with no recognizable voice signal.
+**Root Cause:** The ES7210 microphone ADC was initialized with an incorrect clock and oversampling configuration:
+1. Register `0x07` was written with `0x40` to select an oversampling ratio (OSR) of 64. However, the register allocation for `ADC_OSR` is only 6 bits (`bits 5:0`), meaning `0x40` overflowed and set the OSR value to `0`, causing the internal modulator state machines to malfunction.
+2. Register `0x02` was configured as a flat division of 12 (`0x0C`), which failed to route and clock the delta-sigma modulators properly.
+**Solution:** We analyzed the official C++ implementation of the ES7210 driver in the ESPHome repository (`es7210.cpp` and `es7210_const.h`). By cross-referencing its clock coefficient lookup table for a 12.288MHz Master Clock and 16kHz sample rate, we retrieved the correct register values. We modified `audio_util.py` to match this C++ clock configuration:
+*   Set OSR configuration register `0x07` to `0x20` (OSR = 32).
+*   Set main clock control register `0x02` to `0xC3` (enables clock doubler, sets multiply by 2 via bits 7:6 = `11`, and sets division to 3 via bits 4:0 = `0x03`).
+*   Configured LRCK divider registers `0x04`/`0x05` to `0x03` and `0x00` (division factor of 768).
+*   Gated unused clocks by writing `0x34` to Register `0x01` (keeps only active ADC12 channels and master MCLK active).
+*   Corrected the power sequence by initially clearing all MIC bias and PGA settings (`0xFF` to `0x4B`/`0x4C`) before enabling MIC1 and MIC2.
+
+

hermes_voice_gateway/README.md

@@ -0,0 +1,90 @@
+# Hermes ESP32 Voice Gateway
+
+This folder captures the Hermes API-server implementation used by the ESP32 screen voice demo.
+
+It is intentionally kept next to the screen firmware because the ESP32 client in `demo_hermes_voice.py` talks to this gateway route:
+
+`POST /api/esp32/voice`
+
+## Flow
+
+1. ESP32 records push-to-talk audio as 16 kHz mono 16-bit PCM WAV.
+2. ESP32 posts the raw WAV bytes to Hermes with an Authorization header containing the configured API server token.
+3. Hermes saves the upload to its audio cache and transcribes it with the configured Hermes STT provider.
+4. Hermes sends the transcript through the normal agent session using a persistent conversation key: `esp32:<device_id>`.
+5. Hermes turns the final answer into speech with the configured Hermes TTS provider.
+6. Hermes normalizes the TTS output into an ESP32-friendly WAV: mono, 16 kHz, signed 16-bit little-endian PCM.
+7. Hermes returns the WAV bytes to the ESP32 for playback.
+
+## Files
+
+- `api_server_endpoint.py` — the route/mixin implementation for Hermes' `gateway/platforms/api_server.py`.
+- `smoke_test.py` — local helper tests for MIME mapping, safe headers, WAV normalization, and fallback ACK audio generation.
+
+## Client request contract
+
+Required:
+
+- Method: `POST`
+- Path: `/api/esp32/voice`
+- Header: Authorization containing the configured API server token
+- Header: `Content-Type: audio/wav`
+- Header: `X-Device-ID: kitchen-button` or another stable device id
+- Body: raw audio bytes, preferably 16 kHz mono 16-bit PCM WAV
+
+Optional headers:
+
+- `X-Hermes-Instructions` — extra system instructions for this turn.
+- `X-Hermes-Reply-Mode: ack` — return a quick ACK WAV immediately, then finish the agent turn in the background.
+- `X-Hermes-Screen-Url` — JSON-RPC MCP endpoint for the initiating screen, e.g. `http://192.168.68.123/api/mcp`.
+- `X-Hermes-Screen-Device` — local device registry alias to resolve to a screen URL.
+
+Sync-mode response:
+
+- `Content-Type: audio/wav`
+- WAV body playable by the ESP32
+- Diagnostic headers:
+  - `X-Hermes-Transcript`
+  - `X-Hermes-Text-Response`
+  - `X-Hermes-Response-Id`
+  - `X-Hermes-Conversation`
+  - `X-Hermes-TTS-Source-Type`
+
+ACK-mode response:
+
+- `Content-Type: audio/wav`
+- short acknowledgement WAV immediately
+- the full answer is displayed/spoken on the initiating MCP screen when a screen URL can be resolved
+
+## Hermes integration notes
+
+In Hermes Agent, wire this into `gateway/platforms/api_server.py` by adding:
+
+```python
+self._app.router.add_post("/api/esp32/voice", self._handle_esp32_voice)
+```
+
+after the API server app is created.
+
+The mixin expects the API server class to already provide these Hermes internals:
+
+- `_check_auth(request)`
+- `_run_agent(...)`
+- `_build_response_conversation_history(...)`
+- `_response_messages_turn_start_index(...)`
+- `_extract_output_items(...)`
+- `_response_store`
+- `_model_name`
+- `_background_tasks`
+
+Do not commit API keys or device tokens here. The ESP32 should use the live `API_SERVER_KEY` configured on the Hermes host.
+
+## Smoke test
+
+From this repo:
+
+```bash
+python3 hermes_voice_gateway/smoke_test.py
+```
+
+If `ffmpeg` is installed, the smoke test also verifies WAV normalization through the same decode/rewrite path used for TTS output.

hermes_voice_gateway/api_server_endpoint.py

@@ -0,0 +1,911 @@
+# pyright: reportMissingImports=false, reportAttributeAccessIssue=false, reportArgumentType=false
+"""Hermes API-server ESP32 voice gateway implementation.
+
+This module is a self-contained reference implementation for the Hermes
+`POST /api/esp32/voice` route used by the ESP32 screen voice demo.
+
+It is written as a mixin so the implementation can live beside this screen
+repo while still documenting the exact methods that belong on Hermes'
+`APIServerPlatform` class.  The host class must provide the normal Hermes API
+server internals listed in README.md.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import math
+import re
+import subprocess
+import time
+import uuid
+import wave
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+from urllib.parse import urlparse
+
+try:  # aiohttp is present in Hermes gateway runtime.
+    from aiohttp import web
+except Exception:  # pragma: no cover - allows helper smoke tests without aiohttp.
+    web = None  # type: ignore[assignment]
+
+logger = logging.getLogger(__name__)
+
+ESP32_AUDIO_MAX_BYTES = 8_000_000  # Enough for push-to-talk WAV tests.
+
+ESP32_AUDIO_EXTENSIONS = {
+    "audio/wav": ".wav",
+    "audio/wave": ".wav",
+    "audio/x-wav": ".wav",
+    "audio/mpeg": ".mp3",
+    "audio/mp3": ".mp3",
+    "audio/ogg": ".ogg",
+    "audio/opus": ".ogg",
+    "audio/webm": ".webm",
+    "audio/mp4": ".m4a",
+    "audio/x-m4a": ".m4a",
+    "application/octet-stream": ".wav",
+}
+
+AUDIO_RESPONSE_MIME_BY_SUFFIX = {
+    ".mp3": "audio/mpeg",
+    ".wav": "audio/wav",
+    ".ogg": "audio/ogg",
+    ".opus": "audio/ogg",
+    ".m4a": "audio/mp4",
+    ".mp4": "audio/mp4",
+    ".flac": "audio/flac",
+}
+
+
+def openai_error(message: str, *, code: str = "esp32_voice_error") -> Dict[str, Any]:
+    """Small OpenAI-shaped error body for this endpoint."""
+    return {"error": {"message": message, "type": "invalid_request_error", "code": code}}
+
+
+def audio_extension_for_content_type(content_type: str) -> str:
+    """Map inbound audio Content-Type to a safe cache-file suffix."""
+    media_type = (content_type or "").split(";", 1)[0].strip().lower()
+    return ESP32_AUDIO_EXTENSIONS.get(media_type, ".wav")
+
+
+def audio_response_mime(path: str) -> str:
+    """Return an HTTP Content-Type for a generated audio file path."""
+    suffix = Path(path).suffix.lower()
+    return AUDIO_RESPONSE_MIME_BY_SUFFIX.get(suffix, "application/octet-stream")
+
+
+def safe_audio_header(value: Any, max_length: int) -> str:
+    """Make diagnostic text safe for HTTP response headers."""
+    text = str(value or "")[:max_length]
+    return text.replace("\r", " ").replace("\n", " ").replace("\x00", " ")
+
+
+def convert_audio_to_esp32_wav(input_path: str, *, output_dir: Path, device_id: str) -> Path:
+    """Normalize generated TTS audio to ESP32-friendly WAV.
+
+    Hermes TTS providers commonly emit MP3 or OGG.  The ESP32 client expects a
+    simple RIFF/WAVE file, so decode with ffmpeg to raw PCM and rewrite the WAV
+    container with Python's `wave` module.  The result is mono, 16 kHz,
+    signed 16-bit little-endian PCM with a minimal `fmt` + `data` layout.
+    """
+    safe_device = re.sub(r"[^A-Za-z0-9_.-]+", "_", device_id)[:80] or "default"
+    output_dir.mkdir(parents=True, exist_ok=True)
+    raw_path = output_dir / f"{safe_device}_{int(time.time() * 1000)}_{uuid.uuid4().hex[:8]}.s16le"
+    wav_path = raw_path.with_suffix(".wav")
+    cmd = [
+        "ffmpeg",
+        "-y",
+        "-hide_banner",
+        "-loglevel",
+        "error",
+        "-i",
+        input_path,
+        "-ac",
+        "1",
+        "-ar",
+        "16000",
+        "-f",
+        "s16le",
+        str(raw_path),
+    ]
+    try:
+        subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        raw = raw_path.read_bytes()
+        with wave.open(str(wav_path), "wb") as wav:
+            wav.setnchannels(1)
+            wav.setsampwidth(2)
+            wav.setframerate(16000)
+            wav.writeframes(raw)
+        return wav_path
+    finally:
+        raw_path.unlink(missing_ok=True)
+
+
+def make_ack_wav(path: Path, *, frequency_hz: int = 880, duration_s: float = 0.18) -> Path:
+    """Create a tiny valid mono 16 kHz WAV tone used as last-resort ACK audio."""
+    sample_rate = 16000
+    frames = bytearray()
+    for i in range(int(sample_rate * duration_s)):
+        value = int(9000 * math.sin(2 * math.pi * frequency_hz * (i / sample_rate)))
+        frames.extend(value.to_bytes(2, "little", signed=True))
+    path.parent.mkdir(parents=True, exist_ok=True)
+    with wave.open(str(path), "wb") as wav:
+        wav.setnchannels(1)
+        wav.setsampwidth(2)
+        wav.setframerate(sample_rate)
+        wav.writeframes(bytes(frames))
+    return path
+
+
+class HermesESP32VoiceGatewayMixin:
+    """Mixin for Hermes' API-server adapter.
+
+    The consuming class is expected to provide the Hermes-specific internals:
+    auth, agent execution, Responses API storage, and background task tracking.
+    """
+
+    @staticmethod
+    def _audio_extension_for_content_type(content_type: str) -> str:
+        return audio_extension_for_content_type(content_type)
+
+    @staticmethod
+    def _audio_response_mime(path: str) -> str:
+        return audio_response_mime(path)
+
+    @staticmethod
+    def _safe_audio_header(value: Any, max_length: int) -> str:
+        return safe_audio_header(value, max_length)
+
+    @staticmethod
+    def _convert_audio_to_esp32_wav(input_path: str, *, output_dir: Path, device_id: str) -> Path:
+        return convert_audio_to_esp32_wav(input_path, output_dir=output_dir, device_id=device_id)
+
+    @staticmethod
+    def _esp32_reply_mode(request: Any) -> str:
+        value = (
+            request.headers.get("X-Hermes-Reply-Mode")
+            or request.query.get("reply_mode")
+            or request.query.get("mode")
+            or "sync"
+        )
+        value = str(value).strip().lower()
+        if value in {"ack", "async", "background", "quick_ack", "quick-ack"}:
+            return "ack"
+        return "sync"
+
+    @staticmethod
+    def _looks_like_http_url(value: str) -> bool:
+        try:
+            parsed = urlparse(value)
+            return parsed.scheme in {"http", "https"} and bool(parsed.netloc)
+        except Exception:
+            return False
+
+    @staticmethod
+    def _load_local_device_url(device_name: str) -> Optional[str]:
+        """Best-effort local MCP screen lookup from ~/.hermes/local_devices.yaml."""
+        if not device_name:
+            return None
+        try:
+            import yaml
+
+            registry = Path.home() / ".hermes" / "local_devices.yaml"
+            data = yaml.safe_load(registry.read_text()) if registry.exists() else {}
+            aliases = data.get("aliases") if isinstance(data, dict) else None
+            devices = data.get("devices") if isinstance(data, dict) else None
+            names = [str(device_name).strip(), str(device_name).strip().replace("-", "_")]
+            seen: set[str] = set()
+            while names:
+                name = names.pop(0)
+                if not name or name in seen:
+                    continue
+                seen.add(name)
+                if isinstance(aliases, dict):
+                    alias = aliases.get(name)
+                    if isinstance(alias, str) and alias:
+                        if alias.startswith("http"):
+                            return alias
+                        names.append(alias)
+                info = devices.get(name) if isinstance(devices, dict) else None
+                if isinstance(info, dict):
+                    return str(info.get("fallback_url") or info.get("url") or "") or None
+        except Exception:
+            return None
+        return None
+
+    def _resolve_esp32_screen_url(self, request: Any, device_id: str) -> Optional[str]:
+        explicit = (
+            request.headers.get("X-Hermes-Screen-Url")
+            or request.headers.get("X-Screen-MCP-Url")
+            or request.query.get("screen_url")
+        )
+        if explicit and self._looks_like_http_url(explicit.strip()):
+            return explicit.strip()
+        screen_device = (
+            request.headers.get("X-Hermes-Screen-Device")
+            or request.query.get("screen_device")
+            or device_id
+        )
+        for candidate in [screen_device, "esp32_screen"]:
+            url = self._load_local_device_url(str(candidate).strip())
+            if url and self._looks_like_http_url(url):
+                return url
+        return None
+
+    async def _call_screen_mcp(self, screen_url: Optional[str], tool_name: str, arguments: Dict[str, Any]) -> bool:
+        """Call a simple HTTP JSON-RPC MCP screen tool directly."""
+        if not screen_url:
+            return False
+        try:
+            from aiohttp import ClientSession
+
+            payload = {
+                "jsonrpc": "2.0",
+                "id": uuid.uuid4().hex[:12],
+                "method": "tools/call",
+                "params": {"name": tool_name, "arguments": arguments},
+            }
+            async with ClientSession() as session:
+                async with session.post(screen_url, json=payload, timeout=8) as resp:
+                    if resp.status >= 400:
+                        logger.warning("screen MCP %s failed: HTTP %s", tool_name, resp.status)
+                        return False
+                    body = await resp.text()
+                    try:
+                        parsed = json.loads(body)
+                        if isinstance(parsed, dict) and parsed.get("error"):
+                            logger.warning("screen MCP %s JSON-RPC error: %s", tool_name, parsed["error"])
+                            return False
+                    except Exception:
+                        pass
+                    return True
+        except Exception as exc:
+            logger.warning("screen MCP %s failed: %s", tool_name, exc)
+            return False
+
+    async def _send_screen_text(self, screen_url: Optional[str], text: str, *, clear: bool = False) -> None:
+        if not screen_url:
+            return
+        if clear:
+            await self._call_screen_mcp(screen_url, "clear_screen", {"color": 0})
+        await self._call_screen_mcp(screen_url, "draw_text", {"x": 8, "y": 8, "text": text[:900]})
+
+    async def _send_screen_audio_response(
+        self,
+        screen_url: Optional[str],
+        text: str,
+        *,
+        audio_dir: Path,
+        device_id: str,
+        volume: int = 80,
+    ) -> bool:
+        """Best-effort spoken follow-up for ACK-mode ESP32 voice turns."""
+        if not screen_url or not text.strip():
+            return False
+        try:
+            import base64
+            from tools.tts_tool import text_to_speech_tool
+
+            tts_raw = await asyncio.to_thread(text_to_speech_tool, text)
+            tts_result = json.loads(tts_raw)
+            src = str(tts_result.get("file_path") or "")
+            if not tts_result.get("success") or not src or not Path(src).exists():
+                logger.warning("screen audio TTS failed: %s", tts_result.get("error") or "missing TTS output")
+                return False
+            wav_path = await asyncio.to_thread(
+                self._convert_audio_to_esp32_wav,
+                src,
+                output_dir=audio_dir,
+                device_id=f"{device_id}_response",
+            )
+            wav_base64 = base64.b64encode(wav_path.read_bytes()).decode("ascii")
+            return await self._call_screen_mcp(
+                screen_url,
+                "play_audio_base64",
+                {"wav_base64": wav_base64, "volume": volume},
+            )
+        except Exception:
+            logger.exception("failed to play ESP32 screen audio response")
+            return False
+
+    @staticmethod
+    def _short_voice_summary(text: str, limit: int = 220) -> str:
+        clean = " ".join(str(text or "").split())
+        if len(clean) <= limit:
+            return clean
+        return clean[: max(0, limit - 1)].rstrip() + "…"
+
+    async def _esp32_ack_wav(self, *, audio_dir: Path, device_id: str) -> Path:
+        """Return a reusable quick acknowledgement WAV for low-latency voice UX."""
+        ack_path = audio_dir / "quick_ack.wav"
+        if ack_path.exists():
+            return ack_path
+        try:
+            from tools.tts_tool import text_to_speech_tool
+
+            tts_raw = await asyncio.to_thread(text_to_speech_tool, "Got it. I’m working on that.")
+            tts_result = json.loads(tts_raw)
+            src = str(tts_result.get("file_path") or "")
+            if tts_result.get("success") and src and Path(src).exists():
+                converted = self._convert_audio_to_esp32_wav(src, output_dir=audio_dir, device_id=f"{device_id}_ack")
+                converted.replace(ack_path)
+                return ack_path
+        except Exception:
+            logger.exception("failed to generate spoken ESP32 ack; falling back to tone")
+        return make_ack_wav(ack_path)
+
+    def _append_esp32_voice_audit(self, audio_dir: Path, event: Dict[str, Any]) -> None:
+        try:
+            log_path = audio_dir / "voice_requests.jsonl"
+            with log_path.open("a", encoding="utf-8") as handle:
+                handle.write(json.dumps(event, default=str) + "\n")
+        except Exception:
+            logger.debug("failed to append ESP32 voice audit log", exc_info=True)
+
+    async def _run_esp32_voice_background(
+        self,
+        *,
+        transcript: str,
+        device_id: str,
+        instructions: Optional[str],
+        audio_dir: Path,
+        screen_url: Optional[str],
+    ) -> None:
+        """Run the slow ESP32 voice turn after the device already heard an ACK."""
+        try:
+            await self._send_screen_text(
+                screen_url,
+                f"Heard: {self._short_voice_summary(transcript, 180)}\n\nWorking…",
+                clear=True,
+            )
+            screen_hint = (
+                "\n\nThis request came from an ESP32 voice/screen device. "
+                "Keep the spoken follow-up short. The gateway will display the full answer on the initiating screen when possible."
+            )
+            merged_instructions = (instructions or "") + screen_hint
+            response_data, final_text = await self._run_esp32_voice_turn(
+                transcript=transcript,
+                device_id=device_id,
+                instructions=merged_instructions,
+            )
+            if not final_text.strip():
+                final_text = self._extract_response_text(response_data) or "I finished, but I do not have a text answer."
+            display_text = final_text
+            if len(display_text) > 1200:
+                display_text = self._short_voice_summary(display_text, 900) + "\n\nFull answer is long; WhatsApp delivery is the next wiring step."
+            await self._send_screen_text(screen_url, display_text, clear=True)
+            await self._send_screen_audio_response(screen_url, final_text, audio_dir=audio_dir, device_id=device_id)
+        except Exception:
+            logger.exception("ESP32 background voice turn failed")
+            await self._send_screen_text(screen_url, "Hermes hit an error while preparing the full answer.", clear=True)
+
+    @staticmethod
+    def _extract_response_text(response_data: Dict[str, Any]) -> str:
+        chunks: List[str] = []
+        for item in response_data.get("output") or []:
+            if not isinstance(item, dict) or item.get("type") != "message":
+                continue
+            for part in item.get("content") or []:
+                if isinstance(part, dict) and part.get("type") in {"output_text", "text"}:
+                    text = part.get("text")
+                    if isinstance(text, str) and text:
+                        chunks.append(text)
+        return "\n".join(chunks).strip()
+
+    async def _run_esp32_voice_turn(
+        self,
+        *,
+        transcript: str,
+        device_id: str,
+        instructions: Optional[str] = None,
+    ) -> Tuple[Dict[str, Any], str]:
+        """Run one persistent Responses-style turn for an ESP32 voice device."""
+        conversation = f"esp32:{device_id}"
+        previous_response_id = self._response_store.get_conversation(conversation)
+        conversation_history: List[Dict[str, Any]] = []
+        stored_session_id = None
+        stored_instructions = None
+
+        if previous_response_id:
+            stored = self._response_store.get(previous_response_id)
+            if stored is not None:
+                conversation_history = list(stored.get("conversation_history", []))
+                stored_session_id = stored.get("session_id")
+                stored_instructions = stored.get("instructions")
+
+        if instructions is None:
+            instructions = stored_instructions
+
+        session_id = stored_session_id or str(uuid.uuid4())
+        user_message = f'[Voice input from ESP32 device "{device_id}"]\n{transcript}'
+
+        result, usage = await self._run_agent(
+            user_message=user_message,
+            conversation_history=conversation_history,
+            ephemeral_system_prompt=instructions,
+            session_id=session_id,
+            gateway_session_key=conversation,
+        )
+
+        final_response = result.get("final_response", "") or result.get("error", "") or "(No response generated)"
+        response_id = f"resp_{uuid.uuid4().hex[:28]}"
+        created_at = int(time.time())
+        full_history = self._build_response_conversation_history(conversation_history, user_message, result, final_response)
+        output_start_index = self._response_messages_turn_start_index(conversation_history, user_message, result)
+        output_items = self._extract_output_items(result, start_index=output_start_index)
+        response_data = {
+            "id": response_id,
+            "object": "response",
+            "status": "completed",
+            "created_at": created_at,
+            "model": self._model_name,
+            "output": output_items,
+            "usage": {
+                "input_tokens": usage.get("input_tokens", 0),
+                "output_tokens": usage.get("output_tokens", 0),
+                "total_tokens": usage.get("total_tokens", 0),
+            },
+        }
+        self._response_store.put(response_id, {
+            "response": response_data,
+            "conversation_history": full_history,
+            "instructions": instructions,
+            "session_id": session_id,
+        })
+        self._response_store.set_conversation(conversation, response_id)
+        return response_data, final_response
+
+    async def _handle_esp32_voice(self, request: Any) -> Any:
+        """POST /api/esp32/voice — receive audio, run Hermes, return TTS audio."""
+        if web is None:
+            raise RuntimeError("aiohttp is required for the ESP32 voice endpoint")
+
+        auth_err = self._check_auth(request)
+        if auth_err:
+            return auth_err
+
+        device_id = (request.headers.get("X-Device-ID") or request.query.get("device_id") or "default").strip()[:80] or "default"
+        try:
+            audio_bytes = await request.read()
+        except Exception as exc:
+            return web.json_response(openai_error(f"Failed to read audio body: {exc}"), status=400)
+
+        if not audio_bytes:
+            return web.json_response(openai_error("Empty audio body", code="empty_audio"), status=400)
+        if len(audio_bytes) > ESP32_AUDIO_MAX_BYTES:
+            return web.json_response(
+                openai_error(f"Audio body is too large ({len(audio_bytes)} bytes); max is {ESP32_AUDIO_MAX_BYTES}", code="audio_too_large"),
+                status=413,
+            )
+
+        suffix = self._audio_extension_for_content_type(request.headers.get("Content-Type", ""))
+        try:
+            from hermes_constants import get_hermes_dir
+
+            audio_dir = get_hermes_dir("cache/audio", "audio_cache") / "esp32"
+        except Exception:
+            audio_dir = Path.home() / ".hermes" / "cache" / "audio" / "esp32"
+        audio_dir.mkdir(parents=True, exist_ok=True)
+        safe_device = re.sub(r"[^A-Za-z0-9_.-]+", "_", device_id)[:80] or "default"
+        audio_path = audio_dir / f"{safe_device}_{int(time.time() * 1000)}_{uuid.uuid4().hex[:8]}{suffix}"
+        audio_path.write_bytes(audio_bytes)
+
+        audit_event: Dict[str, Any] = {
+            "timestamp": time.time(),
+            "device_id": device_id,
+            "source_ip": getattr(request, "remote", "") or "",
+            "forwarded_for": request.headers.get("X-Forwarded-For", ""),
+            "real_ip": request.headers.get("X-Real-IP", ""),
+            "user_agent": request.headers.get("User-Agent", ""),
+            "reply_mode": self._esp32_reply_mode(request),
+            "screen_device": request.headers.get("X-Hermes-Screen-Device") or request.query.get("screen_device") or "",
+            "screen_url": request.headers.get("X-Hermes-Screen-Url") or request.headers.get("X-Screen-MCP-Url") or request.query.get("screen_url") or "",
+            "content_type": request.headers.get("Content-Type", ""),
+            "audio_bytes": len(audio_bytes),
+            "audio_path": str(audio_path),
+            "status": "received",
+        }
+        self._append_esp32_voice_audit(audio_dir, audit_event)
+
+        if self._esp32_reply_mode(request) == "ack":
+            screen_url = self._resolve_esp32_screen_url(request, device_id)
+
+            async def _ack_background() -> None:
+                try:
+                    from tools.transcription_tools import transcribe_audio
+
+                    stt_result_bg = await asyncio.wait_for(asyncio.to_thread(transcribe_audio, str(audio_path)), timeout=45)
+                    if not stt_result_bg.get("success"):
+                        failed_event = dict(audit_event)
+                        failed_event.update({"status": "transcription_failed", "error": stt_result_bg.get("error") or "Transcription failed", "timestamp": time.time()})
+                        self._append_esp32_voice_audit(audio_dir, failed_event)
+                        await self._send_screen_text(screen_url, "I heard audio, but transcription failed. Try again.", clear=True)
+                        return
+                    transcript_bg = str(stt_result_bg.get("transcript") or "").strip()
+                    if not transcript_bg:
+                        empty_event = dict(audit_event)
+                        empty_event.update({"status": "empty_transcript", "transcript": "", "timestamp": time.time()})
+                        self._append_esp32_voice_audit(audio_dir, empty_event)
+                        await self._send_screen_text(screen_url, "I heard audio, but got no words. Try again.", clear=True)
+                        return
+                    transcript_event = dict(audit_event)
+                    transcript_event.update({"status": "transcribed", "transcript": transcript_bg, "timestamp": time.time()})
+                    self._append_esp32_voice_audit(audio_dir, transcript_event)
+                    await self._run_esp32_voice_background(
+                        transcript=transcript_bg,
+                        device_id=device_id,
+                        instructions=request.headers.get("X-Hermes-Instructions") or None,
+                        audio_dir=audio_dir,
+                        screen_url=screen_url,
+                    )
+                except asyncio.TimeoutError:
+                    timeout_event = dict(audit_event)
+                    timeout_event.update({"status": "transcription_timeout", "error": "Transcription exceeded 45s", "timestamp": time.time()})
+                    self._append_esp32_voice_audit(audio_dir, timeout_event)
+                except Exception as exc:
+                    logger.exception("ESP32 ack-mode background processing failed")
+                    failed_event = dict(audit_event)
+                    failed_event.update({"status": "background_exception", "error": str(exc), "timestamp": time.time()})
+                    self._append_esp32_voice_audit(audio_dir, failed_event)
+
+            task = asyncio.create_task(_ack_background())
+            if hasattr(self, "_background_tasks"):
+                self._background_tasks.add(task)
+                task.add_done_callback(self._background_tasks.discard)
+            ack_path = await self._esp32_ack_wav(audio_dir=audio_dir, device_id=device_id)
+            return web.FileResponse(
+                ack_path,
+                headers={
+                    "X-Hermes-Conversation": f"esp32:{device_id}",
+                    "X-Hermes-Reply-Mode": "ack",
+                    "X-Hermes-Screen-Url": screen_url or "",
+                    "Content-Type": "audio/wav",
+                },
+            )
+
+        try:
+            from tools.transcription_tools import transcribe_audio
+
+            stt_result = await asyncio.to_thread(transcribe_audio, str(audio_path))
+        except Exception as exc:
+            logger.exception("ESP32 transcription failed")
+            failed_event = dict(audit_event)
+            failed_event.update({"status": "transcription_exception", "error": str(exc), "timestamp": time.time()})
+            self._append_esp32_voice_audit(audio_dir, failed_event)
+            return web.json_response(openai_error(f"Transcription failed: {exc}"), status=500)
+
+        if not stt_result.get("success"):
+            failed_event = dict(audit_event)
+            failed_event.update({"status": "transcription_failed", "error": stt_result.get("error") or "Transcription failed", "timestamp": time.time()})
+            self._append_esp32_voice_audit(audio_dir, failed_event)
+            return web.json_response(openai_error(stt_result.get("error") or "Transcription failed", code="transcription_failed"), status=502)
+
+        transcript = str(stt_result.get("transcript") or "").strip()
+        if not transcript:
+            empty_event = dict(audit_event)
+            empty_event.update({"status": "empty_transcript", "transcript": "", "timestamp": time.time()})
+            self._append_esp32_voice_audit(audio_dir, empty_event)
+            return web.json_response(openai_error("Transcription was empty", code="empty_transcript"), status=422)
+
+        transcript_event = dict(audit_event)
+        transcript_event.update({"status": "transcribed", "transcript": transcript, "timestamp": time.time()})
+        self._append_esp32_voice_audit(audio_dir, transcript_event)
+
+        try:
+            response_data, final_text = await self._run_esp32_voice_turn(
+                transcript=transcript,
+                device_id=device_id,
+                instructions=request.headers.get("X-Hermes-Instructions") or None,
+            )
+        except Exception as exc:
+            logger.exception("ESP32 agent turn failed")
+            return web.json_response(openai_error(f"Agent turn failed: {exc}"), status=500)
+
+        if not final_text.strip():
+            final_text = self._extract_response_text(response_data) or "I heard you, but I do not have a spoken answer."
+
+        try:
+            from tools.tts_tool import text_to_speech_tool
+
+            tts_raw = await asyncio.to_thread(text_to_speech_tool, final_text)
+            tts_result = json.loads(tts_raw)
+        except Exception as exc:
+            logger.exception("ESP32 TTS failed")
+            return web.json_response(
+                openai_error(f"TTS failed after response was generated: {exc}", code="tts_failed"),
+                status=500,
+                headers={"X-Hermes-Transcript": self._safe_audio_header(transcript, 1000), "X-Hermes-Text-Response": self._safe_audio_header(final_text, 2000)},
+            )
+
+        if not tts_result.get("success"):
+            return web.json_response(
+                openai_error(tts_result.get("error") or "TTS failed", code="tts_failed"),
+                status=502,
+                headers={"X-Hermes-Transcript": self._safe_audio_header(transcript, 1000), "X-Hermes-Text-Response": self._safe_audio_header(final_text, 2000)},
+            )
+
+        tts_path = str(tts_result.get("file_path") or "")
+        if not tts_path or not Path(tts_path).exists():
+            return web.json_response(openai_error("TTS succeeded but no audio file was produced", code="tts_file_missing"), status=500)
+
+        try:
+            response_audio_path = self._convert_audio_to_esp32_wav(tts_path, output_dir=audio_dir, device_id=device_id)
+        except Exception as exc:
+            logger.exception("ESP32 WAV normalization failed")
+            return web.json_response(
+                openai_error(f"Could not convert TTS audio to ESP32 WAV: {exc}", code="wav_conversion_failed"),
+                status=500,
+                headers={"X-Hermes-Transcript": self._safe_audio_header(transcript, 1000), "X-Hermes-Text-Response": self._safe_audio_header(final_text, 2000)},
+            )
+
+        return web.FileResponse(
+            response_audio_path,
+            headers={
+                "X-Hermes-Transcript": self._safe_audio_header(transcript, 1000),
+                "X-Hermes-Text-Response": self._safe_audio_header(final_text, 2000),
+                "X-Hermes-Response-Id": str(response_data.get("id") or ""),
+                "X-Hermes-Conversation": f"esp32:{device_id}",
+                "X-Hermes-TTS-Source-Type": self._audio_response_mime(tts_path),
+                "Content-Type": "audio/wav",
+            },
+        )
+
+    async def _handle_esp32_voice_websocket(self, request: Any) -> Any:
+        """GET /api/esp32/voice/ws — real-time WebSocket audio streaming and status route."""
+        if web is None:
+            raise RuntimeError("aiohttp is required for the ESP32 voice websocket endpoint")
+
+        # 1. Check authentication first (HTTP headers)
+        auth_err = self._check_auth(request)
+        if auth_err:
+            return auth_err
+
+        # Prepare websocket response
+        ws = web.WebSocketResponse(heartbeat=None)
+        await ws.prepare(request)
+
+        device_id = (request.headers.get("X-Device-ID") or request.query.get("device_id") or "default").strip()[:80] or "default"
+
+        # 2. Receive JSON start frame
+        try:
+            msg = await asyncio.wait_for(ws.receive(), timeout=10)
+            if msg.type != web.WSMsgType.TEXT:
+                await ws.send_json({"event": "error", "message": "Expected text start frame", "code": "protocol_error"})
+                await ws.close()
+                return ws
+            
+            data = json.loads(msg.data)
+            if data.get("event") != "start":
+                await ws.send_json({"event": "error", "message": "Expected start event", "code": "protocol_error"})
+                await ws.close()
+                return ws
+                
+            if "device_id" in data:
+                device_id = str(data["device_id"]).strip()[:80] or device_id
+        except asyncio.TimeoutError:
+            await ws.close()
+            return ws
+        except Exception as exc:
+            try:
+                await ws.send_json({"event": "error", "message": f"Invalid start payload: {exc}", "code": "protocol_error"})
+            except Exception:
+                pass
+            await ws.close()
+            return ws
+
+        # Send ready status
+        await ws.send_json({"event": "ready"})
+        await ws.send_json({"event": "listening"})
+
+        try:
+            from hermes_constants import get_hermes_dir
+            audio_dir = get_hermes_dir("cache/audio", "audio_cache") / "esp32"
+        except Exception:
+            audio_dir = Path.home() / ".hermes" / "cache" / "audio" / "esp32"
+        audio_dir.mkdir(parents=True, exist_ok=True)
+
+        safe_device = re.sub(r"[^A-Za-z0-9_.-]+", "_", device_id)[:80] or "default"
+        raw_path = audio_dir / f"ws_{safe_device}_{int(time.time() * 1000)}_{uuid.uuid4().hex[:8]}.s16le"
+        
+        total_bytes = 0
+        start_time = time.time()
+        last_frame_time = time.time()
+        cancelled = False
+        timings = {"ws_accepted": start_time}
+
+        try:
+            with open(raw_path, "wb") as f:
+                while True:
+                    now = time.time()
+                    elapsed = now - last_frame_time
+                    if elapsed > 10.0:
+                        await ws.send_json({"event": "error", "message": "Idle timeout exceeded", "code": "timeout"})
+                        break
+
+                    if now - start_time > 60.0:
+                        await ws.send_json({"event": "error", "message": "Max session duration exceeded", "code": "duration_limit"})
+                        break
+
+                    timeout_left = min(10.0 - elapsed, 60.0 - (now - start_time))
+                    if timeout_left <= 0:
+                        break
+
+                    try:
+                        msg = await asyncio.wait_for(ws.receive(), timeout=timeout_left)
+                    except asyncio.TimeoutError:
+                        await ws.send_json({"event": "error", "message": "Idle timeout exceeded", "code": "timeout"})
+                        break
+
+                    last_frame_time = time.time()
+
+                    if msg.type == web.WSMsgType.BINARY:
+                        if total_bytes + len(msg.data) > 8_000_000:
+                            await ws.send_json({"event": "error", "message": "Max audio size exceeded", "code": "size_limit"})
+                            break
+                        f.write(msg.data)
+                        total_bytes += len(msg.data)
+                        if "first_audio" not in timings:
+                            timings["first_audio"] = last_frame_time
+
+                    elif msg.type == web.WSMsgType.TEXT:
+                        try:
+                            data = json.loads(msg.data)
+                            event = data.get("event")
+                            if event == "stop":
+                                timings["stop_received"] = last_frame_time
+                                break
+                            elif event == "cancel":
+                                cancelled = True
+                                break
+                        except Exception:
+                            pass
+
+                    elif msg.type in (web.WSMsgType.CLOSE, web.WSMsgType.CLOSING):
+                        break
+                    elif msg.type == web.WSMsgType.ERROR:
+                        break
+        except Exception as exc:
+            logger.exception("Error in websocket audio loop")
+            try:
+                await ws.send_json({"event": "error", "message": f"Internal server error: {exc}", "code": "server_error"})
+            except Exception:
+                pass
+            raw_path.unlink(missing_ok=True)
+            await ws.close()
+            return ws
+
+        if cancelled or ws.closed or total_bytes < 3200 or "stop_received" not in timings:
+            raw_path.unlink(missing_ok=True)
+            if not ws.closed:
+                if total_bytes < 3200:
+                    await ws.send_json({"event": "error", "message": "Audio recording too short", "code": "audio_too_short"})
+                else:
+                    await ws.send_json({"event": "error", "message": "Session cancelled", "code": "cancelled"})
+                await ws.close()
+            return ws
+
+        wav_path = raw_path.with_suffix(".wav")
+        try:
+            with wave.open(str(wav_path), "wb") as wav:
+                wav.setnchannels(1)
+                wav.setsampwidth(2)
+                wav.setframerate(16000)
+                wav.writeframes(raw_path.read_bytes())
+        finally:
+            raw_path.unlink(missing_ok=True)
+
+        timings["wav_finalized"] = time.time()
+
+        try:
+            from tools.transcription_tools import transcribe_audio
+            stt_result = await asyncio.wait_for(
+                asyncio.to_thread(transcribe_audio, str(wav_path)),
+                timeout=45
+            )
+            timings["stt_done"] = time.time()
+
+            if not stt_result.get("success"):
+                await ws.send_json({"event": "error", "message": stt_result.get("error") or "Transcription failed", "code": "stt_failed"})
+                await ws.close()
+                return ws
+
+            transcript = str(stt_result.get("transcript") or "").strip()
+            if not transcript:
+                await ws.send_json({"event": "error", "message": "Transcription empty", "code": "empty_transcript"})
+                await ws.close()
+                return ws
+
+            await ws.send_json({"event": "transcript", "text": transcript})
+            await ws.send_json({"event": "thinking"})
+
+            response_data, final_text = await self._run_esp32_voice_turn(
+                transcript=transcript,
+                device_id=device_id,
+                instructions=request.headers.get("X-Hermes-Instructions") or None,
+            )
+            timings["agent_done"] = time.time()
+
+            await ws.send_json({"event": "response_text", "text": final_text})
+
+            from tools.tts_tool import text_to_speech_tool
+            tts_raw = await asyncio.to_thread(text_to_speech_tool, final_text)
+            tts_result = json.loads(tts_raw)
+            timings["tts_done"] = time.time()
+
+            if not tts_result.get("success"):
+                await ws.send_json({"event": "error", "message": tts_result.get("error") or "TTS generation failed", "code": "tts_failed"})
+                await ws.close()
+                return ws
+
+            tts_path = str(tts_result.get("file_path") or "")
+            if not tts_path or not Path(tts_path).exists():
+                await ws.send_json({"event": "error", "message": "TTS file missing", "code": "tts_file_missing"})
+                await ws.close()
+                return ws
+
+            response_audio_path = self._convert_audio_to_esp32_wav(
+                tts_path, 
+                output_dir=audio_dir, 
+                device_id=device_id
+            )
+
+            wav_bytes = response_audio_path.read_bytes()
+            await ws.send_json({
+                "event": "audio_start",
+                "content_type": "audio/wav",
+                "bytes": len(wav_bytes)
+            })
+
+            pos = 0
+            chunk_size = 4096
+            while pos < len(wav_bytes):
+                chunk = wav_bytes[pos : pos + chunk_size]
+                await ws.send_bytes(chunk)
+                pos += chunk_size
+
+            await ws.send_json({"event": "audio_end"})
+            await ws.send_json({"event": "done"})
+            timings["audio_sent"] = time.time()
+
+        except Exception as exc:
+            logger.exception("Error processing voice session")
+            try:
+                await ws.send_json({"event": "error", "message": f"Processing error: {exc}", "code": "processing_failed"})
+            except Exception:
+                pass
+        finally:
+            wav_path.unlink(missing_ok=True)
+            try:
+                await ws.close()
+            except Exception:
+                pass
+
+            if "audio_sent" in timings:
+                duration = timings["audio_sent"] - timings["ws_accepted"]
+                status = "completed"
+            else:
+                duration = time.time() - timings["ws_accepted"]
+                status = "failed"
+
+            logger.info("ESP32 Voice WebSocket Session Completed. Timings: %s", timings)
+            audit_event = {
+                "timestamp": time.time(),
+                "device_id": device_id,
+                "source_ip": getattr(request, "remote", "") or "",
+                "duration": duration,
+                "byte_count": total_bytes,
+                "timings": timings,
+                "status": status
+            }
+            self._append_esp32_voice_audit(audio_dir, audit_event)
+
+        return ws
+
+
+def register_esp32_voice_route(app: Any, api_server: HermesESP32VoiceGatewayMixin) -> None:
+    """Register the endpoint on an aiohttp app."""
+    app.router.add_post("/api/esp32/voice", api_server._handle_esp32_voice)
+    app.router.add_get("/api/esp32/voice/ws", api_server._handle_esp32_voice_websocket)
+

hermes_voice_gateway/smoke_test.py

@@ -0,0 +1,63 @@
+#!/usr/bin/env python3
+"""Smoke tests for hermes_voice_gateway helper code.
+
+These tests avoid live Hermes/STT/TTS calls. They verify the pieces that are
+safe to exercise locally: content-type mapping, response MIME mapping, header
+sanitization, fallback ACK WAV generation, and ffmpeg WAV normalization when
+ffmpeg is available.
+"""
+
+from __future__ import annotations
+
+import shutil
+import tempfile
+import wave
+from pathlib import Path
+
+from api_server_endpoint import (
+    audio_extension_for_content_type,
+    audio_response_mime,
+    convert_audio_to_esp32_wav,
+    make_ack_wav,
+    safe_audio_header,
+)
+
+
+def _assert_wav_16k_mono_s16(path: Path) -> None:
+    with wave.open(str(path), "rb") as wav:
+        assert wav.getnchannels() == 1, wav.getnchannels()
+        assert wav.getsampwidth() == 2, wav.getsampwidth()
+        assert wav.getframerate() == 16000, wav.getframerate()
+        assert wav.getnframes() > 0, wav.getnframes()
+
+
+def main() -> None:
+    assert audio_extension_for_content_type("audio/wav") == ".wav"
+    assert audio_extension_for_content_type("audio/wav; charset=binary") == ".wav"
+    assert audio_extension_for_content_type("audio/webm") == ".webm"
+    assert audio_extension_for_content_type("application/unknown") == ".wav"
+
+    assert audio_response_mime("reply.wav") == "audio/wav"
+    assert audio_response_mime("reply.mp3") == "audio/mpeg"
+    assert audio_response_mime("reply.bin") == "application/octet-stream"
+
+    assert safe_audio_header("hello\nworld\r\x00!", 100) == "hello world  !"
+    assert safe_audio_header("abcdef", 3) == "abc"
+
+    with tempfile.TemporaryDirectory() as tmp:
+        tmpdir = Path(tmp)
+        ack = make_ack_wav(tmpdir / "ack.wav")
+        _assert_wav_16k_mono_s16(ack)
+
+        if shutil.which("ffmpeg"):
+            converted = convert_audio_to_esp32_wav(str(ack), output_dir=tmpdir, device_id="kitchen/button")
+            _assert_wav_16k_mono_s16(converted)
+            assert "kitchen_button" in converted.name
+        else:
+            print("ffmpeg not found; skipped conversion path")
+
+    print("hermes_voice_gateway smoke tests passed")
+
+
+if __name__ == "__main__":
+    main()

hermes_voice_gateway/test_websocket.py

@@ -0,0 +1,230 @@
+# pyright: reportMissingImports=false, reportAttributeAccessIssue=false
+"""Integration and unit tests for the ESP32 Voice WebSocket gateway endpoint."""
+
+import asyncio
+import json
+import logging
+import sys
+import unittest
+from pathlib import Path
+from unittest.mock import MagicMock
+
+# 1. Mock external tools modules before importing the API endpoint
+mock_transcription = MagicMock()
+mock_transcription.transcribe_audio.return_value = {"success": True, "transcript": "Hello world"}
+sys.modules["tools.transcription_tools"] = mock_transcription
+
+mock_tts = MagicMock()
+mock_tts.text_to_speech_tool.return_value = json.dumps({"success": True, "file_path": __file__})
+sys.modules["tools.tts_tool"] = mock_tts
+
+# Mock aiohttp
+from aiohttp import web, ClientSession
+
+# Import gateway mixin and register function
+sys.path.append(str(Path(__file__).parent))
+from api_server_endpoint import HermesESP32VoiceGatewayMixin, register_esp32_voice_route
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+class MockHermesServer(HermesESP32VoiceGatewayMixin):
+    def __init__(self):
+        self.auth_token = "valid_secret_key"
+        self._background_tasks = set()
+
+    def _check_auth(self, request):
+        auth_header = request.headers.get("Authorization")
+        if not auth_header or auth_header != f"Bearer {self.auth_token}":
+            return web.json_response({"error": "Unauthorized"}, status=401)
+        return None
+
+    async def _run_esp32_voice_turn(self, transcript, device_id, instructions=None):
+        return {"id": "resp_test123"}, f"Echo: {transcript}"
+
+    @staticmethod
+    def _convert_audio_to_esp32_wav(input_path, *, output_dir, device_id):
+        out_path = Path(output_dir) / "mock_response.wav"
+        import wave
+        with wave.open(str(out_path), "wb") as wav:
+            wav.setnchannels(1)
+            wav.setsampwidth(2)
+            wav.setframerate(16000)
+            wav.writeframes(b"\x00" * 8000)  # Dummy PCM data
+        return out_path
+
+
+class TestWebSocketGateway(unittest.IsolatedAsyncioTestCase):
+    async def asyncSetUp(self):
+        self.server = MockHermesServer()
+        self.app = web.Application()
+        register_esp32_voice_route(self.app, self.server)
+        
+        self.runner = web.AppRunner(self.app)
+        await self.runner.setup()
+        self.site = web.TCPSite(self.runner, "127.0.0.1", 0)
+        await self.site.start()
+        
+        self.port = self.runner.addresses[0][1]
+        self.base_url = f"http://127.0.0.1:{self.port}"
+        self.ws_url = f"ws://127.0.0.1:{self.port}/api/esp32/voice/ws"
+
+    async def asyncTearDown(self):
+        await self.runner.cleanup()
+
+    async def test_auth_failure_missing_token(self):
+        """Verify authentication failure (missing token) returns 401."""
+        async with ClientSession() as session:
+            try:
+                async with session.ws_connect(self.ws_url) as ws:
+                    pass
+                self.fail("Connection should have been rejected with 401")
+            except Exception as e:
+                # HTTP status code should be checked
+                pass
+
+            # Test HTTP request directly to assert 401
+            async with session.get(self.ws_url) as resp:
+                self.assertEqual(resp.status, 401)
+                body = await resp.json()
+                self.assertEqual(body["error"], "Unauthorized")
+
+    async def test_auth_failure_bad_token(self):
+        """Verify authentication failure (bad token) returns 401."""
+        headers = {"Authorization": "Bearer bad_token"}
+        async with ClientSession() as session:
+            async with session.get(self.ws_url, headers=headers) as resp:
+                self.assertEqual(resp.status, 401)
+
+    async def test_successful_streaming_flow(self):
+        """Verify a normal WebSocket start, stream, stop, and response flow."""
+        headers = {
+            "Authorization": "Bearer valid_secret_key",
+            "X-Device-ID": "test-device"
+        }
+        
+        async with ClientSession() as session:
+            async with session.ws_connect(self.ws_url, headers=headers) as ws:
+                # 1. Send start event
+                await ws.send_str(json.dumps({
+                    "event": "start",
+                    "device_id": "test-device",
+                    "sample_rate": 16000
+                }))
+
+                # Read events
+                ready_msg = await ws.receive_json()
+                self.assertEqual(ready_msg["event"], "ready")
+
+                listening_msg = await ws.receive_json()
+                self.assertEqual(listening_msg["event"], "listening")
+
+                # 2. Send simulated binary PCM audio chunks (Total: 4000 bytes, > 3200 byte limit)
+                await ws.send_bytes(b"\x00" * 2000)
+                await ws.send_bytes(b"\x00" * 2000)
+
+                # 3. Send stop event
+                await ws.send_str(json.dumps({"event": "stop"}))
+
+                # 4. Check transcription event
+                transcript_msg = await ws.receive_json()
+                self.assertEqual(transcript_msg["event"], "transcript")
+                self.assertEqual(transcript_msg["text"], "Hello world")
+
+                # 5. Check thinking event
+                thinking_msg = await ws.receive_json()
+                self.assertEqual(thinking_msg["event"], "thinking")
+
+                # 6. Check response_text event
+                resp_text_msg = await ws.receive_json()
+                self.assertEqual(resp_text_msg["event"], "response_text")
+                self.assertEqual(resp_text_msg["text"], "Echo: Hello world")
+
+                # 7. Check audio_start event
+                audio_start_msg = await ws.receive_json()
+                self.assertEqual(audio_start_msg["event"], "audio_start")
+                self.assertEqual(audio_start_msg["content_type"], "audio/wav")
+                audio_bytes_len = audio_start_msg["bytes"]
+
+                # 8. Check binary audio chunks
+                received_audio = b""
+                while True:
+                    msg = await ws.receive()
+                    if msg.type == web.WSMsgType.BINARY:
+                        received_audio += msg.data
+                    elif msg.type == web.WSMsgType.TEXT:
+                        data = json.loads(msg.data)
+                        if data["event"] == "audio_end":
+                            break
+                    else:
+                        self.fail(f"Unexpected msg type: {msg.type}")
+
+                self.assertEqual(len(received_audio), audio_bytes_len)
+
+                # 9. Check done event
+                done_msg = await ws.receive_json()
+                self.assertEqual(done_msg["event"], "done")
+
+    async def test_disconnect_cleanup(self):
+        """Verify that raw files are cleaned up if client disconnects abruptly."""
+        headers = {
+            "Authorization": "Bearer valid_secret_key",
+            "X-Device-ID": "disconnect-device"
+        }
+        
+        # We need to find the file created during this session.
+        # Let's verify files in ~/.hermes/cache/audio/esp32 or /tmp/esp32
+        audio_dir = Path.home() / ".hermes" / "cache" / "audio" / "esp32"
+        existing_files_before = set(audio_dir.glob("ws_disconnect_device_*.s16le")) if audio_dir.exists() else set()
+
+        async with ClientSession() as session:
+            async with session.ws_connect(self.ws_url, headers=headers) as ws:
+                await ws.send_str(json.dumps({
+                    "event": "start",
+                    "device_id": "disconnect-device"
+                }))
+                
+                await ws.receive_json() # ready
+                await ws.receive_json() # listening
+                
+                # Send some bytes
+                await ws.send_bytes(b"\x00" * 1000)
+                
+                # Abrupt close/exit without 'stop'
+                await ws.close()
+
+        # Allow loop to process close
+        await asyncio.sleep(0.5)
+
+        existing_files_after = set(audio_dir.glob("ws_disconnect_device_*.s16le")) if audio_dir.exists() else set()
+        new_files = existing_files_after - existing_files_before
+        
+        # Verify no orphaned raw files remaining
+        self.assertEqual(len(new_files), 0, f"Found orphaned temp files: {new_files}")
+
+    async def test_too_short_audio(self):
+        """Verify session is rejected with audio_too_short error if total PCM is below 3200 bytes."""
+        headers = {
+            "Authorization": "Bearer valid_secret_key",
+            "X-Device-ID": "short-device"
+        }
+        async with ClientSession() as session:
+            async with session.ws_connect(self.ws_url, headers=headers) as ws:
+                await ws.send_str(json.dumps({
+                    "event": "start",
+                    "device_id": "short-device"
+                }))
+                await ws.receive_json() # ready
+                await ws.receive_json() # listening
+                
+                # Send only 1000 bytes (less than 3200 bytes limit)
+                await ws.send_bytes(b"\x00" * 1000)
+                await ws.send_str(json.dumps({"event": "stop"}))
+                
+                error_msg = await ws.receive_json()
+                self.assertEqual(error_msg["event"], "error")
+                self.assertEqual(error_msg["code"], "audio_too_short")
+
+
+if __name__ == "__main__":
+    unittest.main()

iphone_app/IMCAppDelegate.h

@@ -0,0 +1,5 @@
+#import <UIKit/UIKit.h>
+
+@interface IMCAppDelegate : UIResponder <UIApplicationDelegate>
+@property (nonatomic, strong) UIWindow *window;
+@end

iphone_app/IMCAppDelegate.m

@@ -0,0 +1,23 @@
+#import "IMCAppDelegate.h"
+#import "IMCViewController.h"
+
+@implementation IMCAppDelegate
+
+- (BOOL)application:(UIApplication *)application
+        didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
+    application.idleTimerDisabled = YES;
+    self.window = [[UIWindow alloc] initWithFrame:[UIScreen mainScreen].bounds];
+    self.window.rootViewController = [[IMCViewController alloc] init];
+    [self.window makeKeyAndVisible];
+    return YES;
+}
+
+- (void)applicationDidBecomeActive:(UIApplication *)application {
+    application.idleTimerDisabled = YES;
+}
+
+- (void)applicationWillTerminate:(UIApplication *)application {
+    application.idleTimerDisabled = NO;
+}
+
+@end

iphone_app/IMCCanvasView.h

@@ -0,0 +1,10 @@
+#import <UIKit/UIKit.h>
+
+@interface IMCCanvasView : UIView
+- (void)clearWithColor:(NSInteger)color;
+- (void)drawText:(NSString *)text x:(CGFloat)x y:(CGFloat)y size:(NSInteger)size;
+- (void)drawImageData:(NSData *)data x:(CGFloat)x y:(CGFloat)y;
+- (void)displayMonochromeFrameBuffer:(NSData *)frameData;
+- (BOOL)displayRGB565FrameBuffer:(NSData *)frameData width:(NSUInteger)width height:(NSUInteger)height;
+- (NSString *)snapshotPNGBase64;
+@end

iphone_app/IMCCanvasView.m

@@ -0,0 +1,218 @@
+#import "IMCCanvasView.h"
+
+@interface IMCCanvasView ()
+@property (nonatomic, assign) NSInteger clearColor;
+@property (nonatomic, strong) NSMutableArray *commands;
+@property (nonatomic, strong) UIImage *streamImage;
+@end
+
+@implementation IMCCanvasView
+
+- (instancetype)initWithFrame:(CGRect)frame {
+    self = [super initWithFrame:frame];
+    if (self) {
+        _clearColor = 0;
+        _commands = [NSMutableArray array];
+        self.opaque = YES;
+    }
+    return self;
+}
+
+- (void)clearWithColor:(NSInteger)color {
+    self.clearColor = color == 1 ? 1 : 0;
+    self.streamImage = nil;
+    [self.commands removeAllObjects];
+    [self setNeedsDisplay];
+}
+
+- (void)drawText:(NSString *)text x:(CGFloat)x y:(CGFloat)y size:(NSInteger)size {
+    if (!text) {
+        text = @"";
+    }
+    self.streamImage = nil;
+    [self.commands addObject:@{
+        @"type": @"text",
+        @"text": text,
+        @"x": @(x),
+        @"y": @(y),
+        @"size": @(size == 2 ? 2 : 1)
+    }];
+    [self setNeedsDisplay];
+}
+
+- (void)drawImageData:(NSData *)data x:(CGFloat)x y:(CGFloat)y {
+    UIImage *image = [UIImage imageWithData:data];
+    if (!image) {
+        return;
+    }
+    self.streamImage = nil;
+    [self.commands addObject:@{
+        @"type": @"image",
+        @"image": image,
+        @"x": @(x),
+        @"y": @(y)
+    }];
+    [self setNeedsDisplay];
+}
+
+- (void)displayMonochromeFrameBuffer:(NSData *)frameData {
+    if (frameData.length < 15000) {
+        return;
+    }
+
+    const NSUInteger width = 400;
+    const NSUInteger height = 300;
+    NSMutableData *rgba = [NSMutableData dataWithLength:width * height * 4];
+    const uint8_t *source = frameData.bytes;
+    uint8_t *pixels = rgba.mutableBytes;
+
+    for (NSUInteger y = 0; y < height; y++) {
+        NSUInteger invY = height - 1 - y;
+        for (NSUInteger x = 0; x < width; x++) {
+            NSUInteger bx = x / 2;
+            NSUInteger by = invY / 4;
+            NSUInteger index = bx * 75 + by;
+            NSUInteger bit = 7 - ((invY % 4) * 2 + (x % 2));
+            BOOL on = (source[index] & (1 << bit)) != 0;
+            uint8_t value = on ? 255 : 0;
+            NSUInteger pixelIndex = (y * width + x) * 4;
+            pixels[pixelIndex + 0] = value;
+            pixels[pixelIndex + 1] = value;
+            pixels[pixelIndex + 2] = value;
+            pixels[pixelIndex + 3] = 255;
+        }
+    }
+
+    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
+    CGDataProviderRef provider = CGDataProviderCreateWithCFData((__bridge CFDataRef)rgba);
+    CGImageRef cgImage = CGImageCreate(width,
+                                       height,
+                                       8,
+                                       32,
+                                       width * 4,
+                                       colorSpace,
+                                       kCGImageAlphaLast | kCGBitmapByteOrderDefault,
+                                       provider,
+                                       NULL,
+                                       false,
+                                       kCGRenderingIntentDefault);
+    if (cgImage) {
+        self.streamImage = [UIImage imageWithCGImage:cgImage];
+        [self.commands removeAllObjects];
+        [self setNeedsDisplay];
+        CGImageRelease(cgImage);
+    }
+    CGDataProviderRelease(provider);
+    CGColorSpaceRelease(colorSpace);
+}
+
+- (BOOL)displayRGB565FrameBuffer:(NSData *)frameData width:(NSUInteger)width height:(NSUInteger)height {
+    if (width == 0 || height == 0 || width > 2048 || height > 2048 ||
+        frameData.length != width * height * 2) {
+        return NO;
+    }
+
+    const uint8_t *source = frameData.bytes;
+    NSMutableData *rgba = [NSMutableData dataWithLength:width * height * 4];
+    uint8_t *pixels = rgba.mutableBytes;
+    for (NSUInteger i = 0; i < width * height; i++) {
+        // Network color frames use big-endian RGB565.
+        uint16_t value = ((uint16_t)source[i * 2] << 8) | source[i * 2 + 1];
+        uint8_t red5 = (value >> 11) & 0x1f;
+        uint8_t green6 = (value >> 5) & 0x3f;
+        uint8_t blue5 = value & 0x1f;
+        pixels[i * 4 + 0] = (red5 << 3) | (red5 >> 2);
+        pixels[i * 4 + 1] = (green6 << 2) | (green6 >> 4);
+        pixels[i * 4 + 2] = (blue5 << 3) | (blue5 >> 2);
+        pixels[i * 4 + 3] = 255;
+    }
+
+    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
+    CGDataProviderRef provider = CGDataProviderCreateWithCFData((__bridge CFDataRef)rgba);
+    CGImageRef cgImage = CGImageCreate(width,
+                                       height,
+                                       8,
+                                       32,
+                                       width * 4,
+                                       colorSpace,
+                                       kCGImageAlphaLast | kCGBitmapByteOrderDefault,
+                                       provider,
+                                       NULL,
+                                       false,
+                                       kCGRenderingIntentDefault);
+    if (cgImage) {
+        self.streamImage = [UIImage imageWithCGImage:cgImage];
+        [self.commands removeAllObjects];
+        [self setNeedsDisplay];
+        CGImageRelease(cgImage);
+    }
+    CGDataProviderRelease(provider);
+    CGColorSpaceRelease(colorSpace);
+    return cgImage != NULL;
+}
+
+- (NSString *)snapshotPNGBase64 {
+    UIGraphicsBeginImageContextWithOptions(self.bounds.size, YES, 0.0);
+    [self drawViewHierarchyInRect:self.bounds afterScreenUpdates:YES];
+    UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
+    UIGraphicsEndImageContext();
+
+    NSData *pngData = UIImagePNGRepresentation(image);
+    return [pngData base64EncodedStringWithOptions:0];
+}
+
+- (CGPoint)pointForDeviceX:(CGFloat)x y:(CGFloat)y {
+    CGFloat scaleX = self.bounds.size.width / 400.0;
+    CGFloat scaleY = self.bounds.size.height / 300.0;
+    return CGPointMake(x * scaleX, y * scaleY);
+}
+
+- (void)drawRect:(CGRect)rect {
+    UIColor *background = self.clearColor == 1 ? [UIColor blackColor] : [UIColor whiteColor];
+    UIColor *foreground = self.clearColor == 1 ? [UIColor whiteColor] : [UIColor blackColor];
+
+    [background setFill];
+    UIRectFill(self.bounds);
+
+    if (self.streamImage) {
+        [self.streamImage drawInRect:self.bounds];
+        return;
+    }
+
+    CGContextRef context = UIGraphicsGetCurrentContext();
+    CGContextSetStrokeColorWithColor(context, foreground.CGColor);
+    CGContextSetLineWidth(context, 2.0);
+    CGContextStrokeRect(context, CGRectInset(self.bounds, 1.0, 1.0));
+
+    for (NSDictionary *command in self.commands) {
+        NSString *type = command[@"type"];
+        CGFloat x = [command[@"x"] doubleValue];
+        CGFloat y = [command[@"y"] doubleValue];
+        CGPoint point = [self pointForDeviceX:x y:y];
+
+        if ([type isEqualToString:@"text"]) {
+            NSInteger size = [command[@"size"] integerValue];
+            UIFont *font = [UIFont boldSystemFontOfSize:size == 2 ? 26.0 : 15.0];
+            NSDictionary *attrs = @{
+                NSFontAttributeName: font,
+                NSForegroundColorAttributeName: foreground
+            };
+            [command[@"text"] drawAtPoint:point withAttributes:attrs];
+        } else if ([type isEqualToString:@"image"]) {
+            UIImage *image = command[@"image"];
+            CGFloat maxWidth = self.bounds.size.width - point.x;
+            CGFloat maxHeight = self.bounds.size.height - point.y;
+            CGSize imageSize = image.size;
+            CGFloat scale = MIN(maxWidth / MAX(imageSize.width, 1.0),
+                                maxHeight / MAX(imageSize.height, 1.0));
+            scale = MIN(scale, 1.0);
+            CGRect imageRect = CGRectMake(point.x,
+                                          point.y,
+                                          imageSize.width * scale,
+                                          imageSize.height * scale);
+            [image drawInRect:imageRect];
+        }
+    }
+}
+
+@end

iphone_app/IMCMCPServer.h

@@ -0,0 +1,8 @@
+#import <Foundation/Foundation.h>
+
+@class IMCViewController;
+
+@interface IMCMCPServer : NSObject
+- (instancetype)initWithViewController:(IMCViewController *)viewController port:(NSInteger)port;
+- (void)start;
+@end

iphone_app/IMCMCPServer.m

@@ -0,0 +1,163 @@
+#import "IMCMCPServer.h"
+#import "IMCViewController.h"
+#import <arpa/inet.h>
+#import <netinet/in.h>
+#import <sys/socket.h>
+#import <unistd.h>
+
+@interface IMCMCPServer ()
+@property (nonatomic, weak) IMCViewController *viewController;
+@property (nonatomic, assign) NSInteger port;
+@property (nonatomic, assign) BOOL running;
+@end
+
+@implementation IMCMCPServer
+
+- (instancetype)initWithViewController:(IMCViewController *)viewController port:(NSInteger)port {
+    self = [super init];
+    if (self) {
+        _viewController = viewController;
+        _port = port;
+    }
+    return self;
+}
+
+- (void)start {
+    if (self.running) {
+        return;
+    }
+    self.running = YES;
+    [NSThread detachNewThreadSelector:@selector(serverLoop) toTarget:self withObject:nil];
+}
+
+- (void)serverLoop {
+    @autoreleasepool {
+        int serverSocket = socket(AF_INET, SOCK_STREAM, 0);
+        if (serverSocket < 0) {
+            return;
+        }
+
+        int yes = 1;
+        setsockopt(serverSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
+
+        struct sockaddr_in address;
+        memset(&address, 0, sizeof(address));
+        address.sin_family = AF_INET;
+        address.sin_addr.s_addr = INADDR_ANY;
+        address.sin_port = htons((uint16_t)self.port);
+
+        if (bind(serverSocket, (struct sockaddr *)&address, sizeof(address)) < 0) {
+            close(serverSocket);
+            return;
+        }
+
+        if (listen(serverSocket, 8) < 0) {
+            close(serverSocket);
+            return;
+        }
+
+        while (self.running) {
+            @autoreleasepool {
+                struct sockaddr_in clientAddress;
+                socklen_t clientLength = sizeof(clientAddress);
+                int clientSocket = accept(serverSocket,
+                                          (struct sockaddr *)&clientAddress,
+                                          &clientLength);
+                if (clientSocket >= 0) {
+                    [self handleClientSocket:clientSocket];
+                    close(clientSocket);
+                }
+            }
+        }
+
+        close(serverSocket);
+    }
+}
+
+- (void)handleClientSocket:(int)clientSocket {
+    NSMutableData *requestData = [NSMutableData data];
+    char buffer[2048];
+
+    while (requestData.length < 1024 * 1024) {
+        ssize_t count = recv(clientSocket, buffer, sizeof(buffer), 0);
+        if (count <= 0) {
+            break;
+        }
+        [requestData appendBytes:buffer length:(NSUInteger)count];
+
+        NSString *partial = [[NSString alloc] initWithData:requestData encoding:NSUTF8StringEncoding];
+        NSRange separator = [partial rangeOfString:@"\r\n\r\n"];
+        if (separator.location == NSNotFound) {
+            continue;
+        }
+
+        NSInteger contentLength = [self contentLengthFromHeader:partial];
+        NSUInteger bodyStart = separator.location + separator.length;
+        if (requestData.length >= bodyStart + MAX(contentLength, 0)) {
+            break;
+        }
+    }
+
+    NSString *request = [[NSString alloc] initWithData:requestData encoding:NSUTF8StringEncoding];
+    NSDictionary *responseObject = [self responseObjectForHTTPRequest:request];
+    NSData *responseJSON = [NSJSONSerialization dataWithJSONObject:responseObject options:0 error:nil];
+    if (!responseJSON) {
+        responseJSON = [@"{}" dataUsingEncoding:NSUTF8StringEncoding];
+    }
+
+    NSString *status = responseObject[@"httpStatus"] ?: @"200 OK";
+    NSMutableDictionary *bodyObject = [responseObject mutableCopy];
+    [bodyObject removeObjectForKey:@"httpStatus"];
+    responseJSON = [NSJSONSerialization dataWithJSONObject:bodyObject options:0 error:nil];
+
+    NSMutableData *responseData = [NSMutableData data];
+    NSString *header = [NSString stringWithFormat:
+                        @"HTTP/1.1 %@\r\nContent-Type: application/json\r\nContent-Length: %lu\r\nConnection: close\r\n\r\n",
+                        status,
+                        (unsigned long)responseJSON.length];
+    [responseData appendData:[header dataUsingEncoding:NSUTF8StringEncoding]];
+    [responseData appendData:responseJSON];
+    send(clientSocket, responseData.bytes, responseData.length, 0);
+}
+
+- (NSDictionary *)responseObjectForHTTPRequest:(NSString *)request {
+    if (![request hasPrefix:@"POST /api/mcp "]) {
+        return @{@"httpStatus": @"404 Not Found",
+                 @"jsonrpc": @"2.0",
+                 @"error": @{@"code": @-32601, @"message": @"POST /api/mcp required"},
+                 @"id": [NSNull null]};
+    }
+
+    NSRange separator = [request rangeOfString:@"\r\n\r\n"];
+    if (separator.location == NSNotFound) {
+        return [self parseError:@"Missing HTTP body."];
+    }
+
+    NSString *body = [request substringFromIndex:separator.location + separator.length];
+    NSData *bodyData = [body dataUsingEncoding:NSUTF8StringEncoding];
+    NSError *jsonError = nil;
+    NSDictionary *rpcRequest = [NSJSONSerialization JSONObjectWithData:bodyData options:0 error:&jsonError];
+    if (jsonError || ![rpcRequest isKindOfClass:NSDictionary.class]) {
+        return [self parseError:@"Invalid JSON-RPC object."];
+    }
+
+    return [self.viewController handleRPCRequest:rpcRequest];
+}
+
+- (NSDictionary *)parseError:(NSString *)message {
+    return @{@"jsonrpc": @"2.0",
+             @"error": @{@"code": @-32700, @"message": message ?: @"Parse error"},
+             @"id": [NSNull null]};
+}
+
+- (NSInteger)contentLengthFromHeader:(NSString *)request {
+    NSArray *lines = [request componentsSeparatedByString:@"\r\n"];
+    for (NSString *line in lines) {
+        if ([[line lowercaseString] hasPrefix:@"content-length:"]) {
+            return [[line substringFromIndex:15] integerValue];
+        }
+    }
+    return 0;
+}
+
+@end

iphone_app/IMCVideoStreamServer.h

@@ -0,0 +1,11 @@
+#import <Foundation/Foundation.h>
+
+@class IMCCanvasView;
+
+@interface IMCVideoStreamServer : NSObject
+- (instancetype)initWithCanvasView:(IMCCanvasView *)canvasView;
+- (void)start;
+- (NSDictionary *)streamStats;
+- (void)resetStreamStats;
+- (void)setDebugEnabled:(BOOL)enabled;
+@end

iphone_app/IMCVideoStreamServer.m

@@ -0,0 +1,447 @@
+#import "IMCVideoStreamServer.h"
+#import "IMCCanvasView.h"
+#import <arpa/inet.h>
+#import <netinet/in.h>
+#import <sys/socket.h>
+#import <string.h>
+#import <unistd.h>
+
+static const NSInteger IMCFrameSize = 15000;
+static const NSInteger IMCUDPChunkSize = 1000;
+static const NSInteger IMCUDPChunkCount = 15;
+static const NSInteger IMCColorStreamPort = 8083;
+static const NSUInteger IMCColorHeaderSize = 16;
+
+@interface IMCVideoStreamServer ()
+@property (nonatomic, weak) IMCCanvasView *canvasView;
+@property (nonatomic, assign) BOOL running;
+@property (nonatomic, assign) BOOL debugEnabled;
+@property (nonatomic, strong) NSDate *startedAt;
+@property (nonatomic, strong) NSDate *lastFrameAt;
+@property (nonatomic, copy) NSString *lastProtocol;
+@property (nonatomic, assign) uint64_t tcpConnections;
+@property (nonatomic, assign) uint64_t tcpFrames;
+@property (nonatomic, assign) uint64_t tcpBytes;
+@property (nonatomic, assign) uint64_t udpFrames;
+@property (nonatomic, assign) uint64_t udpPackets;
+@property (nonatomic, assign) uint64_t udpBytes;
+@property (nonatomic, assign) uint64_t invalidUDPPackets;
+@property (nonatomic, assign) uint64_t discoveryRequests;
+@property (nonatomic, assign) uint64_t colorConnections;
+@property (nonatomic, assign) uint64_t colorFrames;
+@property (nonatomic, assign) uint64_t colorBytes;
+@property (nonatomic, assign) NSUInteger lastColorWidth;
+@property (nonatomic, assign) NSUInteger lastColorHeight;
+@end
+
+@implementation IMCVideoStreamServer
+
+- (instancetype)initWithCanvasView:(IMCCanvasView *)canvasView {
+    self = [super init];
+    if (self) {
+        _canvasView = canvasView;
+        _startedAt = [NSDate date];
+        _lastProtocol = @"none";
+    }
+    return self;
+}
+
+- (void)start {
+    if (self.running) {
+        return;
+    }
+    self.running = YES;
+    [NSThread detachNewThreadSelector:@selector(tcpLoop) toTarget:self withObject:nil];
+    [NSThread detachNewThreadSelector:@selector(udpLoop) toTarget:self withObject:nil];
+    [NSThread detachNewThreadSelector:@selector(discoveryLoop) toTarget:self withObject:nil];
+    [NSThread detachNewThreadSelector:@selector(colorTCPLoop) toTarget:self withObject:nil];
+}
+
+- (BOOL)readExactly:(NSUInteger)length fromSocket:(int)socketFD intoData:(NSMutableData *)data {
+    [data setLength:length];
+    NSUInteger received = 0;
+    while (self.running && received < length) {
+        ssize_t count = recv(socketFD,
+                             (uint8_t *)data.mutableBytes + received,
+                             length - received,
+                             0);
+        if (count <= 0) {
+            return NO;
+        }
+        received += (NSUInteger)count;
+    }
+    return received == length;
+}
+
+- (void)colorTCPLoop {
+    @autoreleasepool {
+        int serverSocket = socket(AF_INET, SOCK_STREAM, 0);
+        if (serverSocket < 0) {
+            return;
+        }
+
+        int yes = 1;
+        setsockopt(serverSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
+        struct sockaddr_in address;
+        memset(&address, 0, sizeof(address));
+        address.sin_family = AF_INET;
+        address.sin_addr.s_addr = INADDR_ANY;
+        address.sin_port = htons((uint16_t)IMCColorStreamPort);
+        if (bind(serverSocket, (struct sockaddr *)&address, sizeof(address)) < 0 ||
+            listen(serverSocket, 1) < 0) {
+            close(serverSocket);
+            return;
+        }
+
+        while (self.running) {
+            @autoreleasepool {
+                int clientSocket = accept(serverSocket, NULL, NULL);
+                if (clientSocket < 0) {
+                    continue;
+                }
+                @synchronized (self) {
+                    self.colorConnections += 1;
+                }
+
+                NSMutableData *header = [NSMutableData data];
+                while ([self readExactly:IMCColorHeaderSize
+                              fromSocket:clientSocket
+                                intoData:header]) {
+                    const uint8_t *bytes = header.bytes;
+                    if (memcmp(bytes, "IMCR", 4) != 0 || bytes[4] != 1 || bytes[5] != 1) {
+                        break;
+                    }
+                    NSUInteger width = ((NSUInteger)bytes[6] << 8) | bytes[7];
+                    NSUInteger height = ((NSUInteger)bytes[8] << 8) | bytes[9];
+                    uint32_t payloadLength = ((uint32_t)bytes[10] << 24) |
+                                             ((uint32_t)bytes[11] << 16) |
+                                             ((uint32_t)bytes[12] << 8) |
+                                             bytes[13];
+                    if (width == 0 || height == 0 || width > 2048 || height > 2048 ||
+                        payloadLength != width * height * 2 || payloadLength > 8 * 1024 * 1024) {
+                        break;
+                    }
+
+                    NSMutableData *payload = [NSMutableData data];
+                    if (![self readExactly:payloadLength
+                                fromSocket:clientSocket
+                                  intoData:payload]) {
+                        break;
+                    }
+                    NSData *snapshot = [payload copy];
+                    @synchronized (self) {
+                        self.colorFrames += 1;
+                        self.colorBytes += payloadLength;
+                        self.lastColorWidth = width;
+                        self.lastColorHeight = height;
+                        self.lastProtocol = @"color_tcp";
+                        self.lastFrameAt = [NSDate date];
+                    }
+                    dispatch_async(dispatch_get_main_queue(), ^{
+                        [self.canvasView displayRGB565FrameBuffer:snapshot width:width height:height];
+                    });
+                }
+                close(clientSocket);
+            }
+        }
+        close(serverSocket);
+    }
+}
+
+- (void)tcpLoop {
+    @autoreleasepool {
+        int serverSocket = socket(AF_INET, SOCK_STREAM, 0);
+        if (serverSocket < 0) {
+            return;
+        }
+
+        int yes = 1;
+        setsockopt(serverSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
+
+        struct sockaddr_in address;
+        memset(&address, 0, sizeof(address));
+        address.sin_family = AF_INET;
+        address.sin_addr.s_addr = INADDR_ANY;
+        address.sin_port = htons(8081);
+
+        if (bind(serverSocket, (struct sockaddr *)&address, sizeof(address)) < 0) {
+            close(serverSocket);
+            return;
+        }
+
+        if (listen(serverSocket, 1) < 0) {
+            close(serverSocket);
+            return;
+        }
+
+        while (self.running) {
+            @autoreleasepool {
+                int clientSocket = accept(serverSocket, NULL, NULL);
+                if (clientSocket < 0) {
+                    continue;
+                }
+                [self noteTCPConnection];
+
+                NSMutableData *frame = [NSMutableData dataWithLength:IMCFrameSize];
+                NSUInteger received = 0;
+                while (self.running) {
+                    ssize_t count = recv(clientSocket,
+                                         (uint8_t *)frame.mutableBytes + received,
+                                         IMCFrameSize - received,
+                                         0);
+                    if (count <= 0) {
+                        break;
+                    }
+
+                    received += (NSUInteger)count;
+                    [self noteTCPBytes:(NSUInteger)count];
+                    if (received == IMCFrameSize) {
+                        [self displayFrame:frame protocol:@"tcp"];
+                        received = 0;
+                    }
+                }
+
+                close(clientSocket);
+            }
+        }
+
+        close(serverSocket);
+    }
+}
+
+- (void)udpLoop {
+    @autoreleasepool {
+        int udpSocket = socket(AF_INET, SOCK_DGRAM, 0);
+        if (udpSocket < 0) {
+            return;
+        }
+
+        int yes = 1;
+        setsockopt(udpSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
+
+        struct sockaddr_in address;
+        memset(&address, 0, sizeof(address));
+        address.sin_family = AF_INET;
+        address.sin_addr.s_addr = INADDR_ANY;
+        address.sin_port = htons(8082);
+
+        if (bind(udpSocket, (struct sockaddr *)&address, sizeof(address)) < 0) {
+            close(udpSocket);
+            return;
+        }
+
+        NSMutableData *frame = [NSMutableData dataWithLength:IMCFrameSize];
+        uint8_t currentFrameID = 0;
+        uint16_t chunksReceived = 0;
+        BOOL hasFrame = NO;
+        uint8_t packet[1002];
+
+        while (self.running) {
+            ssize_t count = recvfrom(udpSocket, packet, sizeof(packet), 0, NULL, NULL);
+            if (count < 2) {
+                [self noteInvalidUDPPacket];
+                continue;
+            }
+
+            uint8_t frameID = packet[0];
+            uint8_t chunkIndex = packet[1];
+            if (chunkIndex >= IMCUDPChunkCount || count < 2 + IMCUDPChunkSize) {
+                [self noteInvalidUDPPacket];
+                continue;
+            }
+            [self noteUDPPacketBytes:(NSUInteger)count];
+
+            if (!hasFrame || frameID != currentFrameID) {
+                currentFrameID = frameID;
+                chunksReceived = 0;
+                hasFrame = YES;
+            }
+
+            memcpy((uint8_t *)frame.mutableBytes + chunkIndex * IMCUDPChunkSize,
+                   packet + 2,
+                   IMCUDPChunkSize);
+            chunksReceived |= (uint16_t)(1 << chunkIndex);
+
+            if (chunksReceived == 0x7fff) {
+                [self displayFrame:frame protocol:@"udp"];
+                chunksReceived = 0;
+            }
+        }
+
+        close(udpSocket);
+    }
+}
+
+- (void)discoveryLoop {
+    @autoreleasepool {
+        int udpSocket = socket(AF_INET, SOCK_DGRAM, 0);
+        if (udpSocket < 0) {
+            return;
+        }
+
+        int yes = 1;
+        setsockopt(udpSocket, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));
+        setsockopt(udpSocket, SOL_SOCKET, SO_BROADCAST, &yes, sizeof(yes));
+
+        struct sockaddr_in address;
+        memset(&address, 0, sizeof(address));
+        address.sin_family = AF_INET;
+        address.sin_addr.s_addr = INADDR_ANY;
+        address.sin_port = htons(5000);
+
+        if (bind(udpSocket, (struct sockaddr *)&address, sizeof(address)) < 0) {
+            close(udpSocket);
+            return;
+        }
+
+        char buffer[64];
+        while (self.running) {
+            struct sockaddr_in clientAddress;
+            socklen_t clientLength = sizeof(clientAddress);
+            ssize_t count = recvfrom(udpSocket,
+                                     buffer,
+                                     sizeof(buffer),
+                                     0,
+                                     (struct sockaddr *)&clientAddress,
+                                     &clientLength);
+            if (count == 15 && memcmp(buffer, "DISCOVER_SCREEN", 15) == 0) {
+                [self noteDiscoveryRequest];
+                const char *reply = "SCREEN_IP_8080";
+                sendto(udpSocket,
+                       reply,
+                       strlen(reply),
+                       0,
+                       (struct sockaddr *)&clientAddress,
+                       clientLength);
+            }
+        }
+
+        close(udpSocket);
+    }
+}
+
+- (void)displayFrame:(NSData *)frame protocol:(NSString *)protocol {
+    [self noteFrameForProtocol:protocol];
+    NSData *snapshot = [frame copy];
+    dispatch_async(dispatch_get_main_queue(), ^{
+        [self.canvasView displayMonochromeFrameBuffer:snapshot];
+    });
+}
+
+- (NSDictionary *)streamStats {
+    @synchronized (self) {
+        NSTimeInterval uptime = [[NSDate date] timeIntervalSinceDate:self.startedAt ?: [NSDate date]];
+        NSTimeInterval secondsSinceLastFrame = self.lastFrameAt ? [[NSDate date] timeIntervalSinceDate:self.lastFrameAt] : -1.0;
+        return @{
+            @"debug_enabled": @(self.debugEnabled),
+            @"running": @(self.running),
+            @"tcp_port": @8081,
+            @"udp_port": @8082,
+            @"color_tcp_port": @(IMCColorStreamPort),
+            @"discovery_port": @5000,
+            @"frame_size_bytes": @(IMCFrameSize),
+            @"udp_chunk_size_bytes": @(IMCUDPChunkSize),
+            @"udp_chunks_per_frame": @(IMCUDPChunkCount),
+            @"uptime_sec": @((NSInteger)uptime),
+            @"tcp_connections": @(self.tcpConnections),
+            @"tcp_frames": @(self.tcpFrames),
+            @"tcp_bytes": @(self.tcpBytes),
+            @"udp_frames": @(self.udpFrames),
+            @"udp_packets": @(self.udpPackets),
+            @"udp_bytes": @(self.udpBytes),
+            @"invalid_udp_packets": @(self.invalidUDPPackets),
+            @"discovery_requests": @(self.discoveryRequests),
+            @"color_connections": @(self.colorConnections),
+            @"color_frames": @(self.colorFrames),
+            @"color_bytes": @(self.colorBytes),
+            @"last_color_width": @(self.lastColorWidth),
+            @"last_color_height": @(self.lastColorHeight),
+            @"last_protocol": self.lastProtocol ?: @"none",
+            @"seconds_since_last_frame": @(secondsSinceLastFrame)
+        };
+    }
+}
+
+- (void)resetStreamStats {
+    @synchronized (self) {
+        self.startedAt = [NSDate date];
+        self.lastFrameAt = nil;
+        self.lastProtocol = @"none";
+        self.tcpConnections = 0;
+        self.tcpFrames = 0;
+        self.tcpBytes = 0;
+        self.udpFrames = 0;
+        self.udpPackets = 0;
+        self.udpBytes = 0;
+        self.invalidUDPPackets = 0;
+        self.discoveryRequests = 0;
+        self.colorConnections = 0;
+        self.colorFrames = 0;
+        self.colorBytes = 0;
+        self.lastColorWidth = 0;
+        self.lastColorHeight = 0;
+    }
+}
+
+- (void)setDebugEnabled:(BOOL)enabled {
+    @synchronized (self) {
+        _debugEnabled = enabled;
+    }
+}
+
+- (void)noteTCPConnection {
+    @synchronized (self) {
+        self.tcpConnections += 1;
+        if (self.debugEnabled) {
+            NSLog(@"IPhoneMCP stream: TCP client connected");
+        }
+    }
+}
+
+- (void)noteTCPBytes:(NSUInteger)count {
+    @synchronized (self) {
+        self.tcpBytes += count;
+    }
+}
+
+- (void)noteUDPPacketBytes:(NSUInteger)count {
+    @synchronized (self) {
+        self.udpPackets += 1;
+        self.udpBytes += count;
+    }
+}
+
+- (void)noteInvalidUDPPacket {
+    @synchronized (self) {
+        self.invalidUDPPackets += 1;
+        if (self.debugEnabled) {
+            NSLog(@"IPhoneMCP stream: invalid UDP packet");
+        }
+    }
+}
+
+- (void)noteDiscoveryRequest {
+    @synchronized (self) {
+        self.discoveryRequests += 1;
+        if (self.debugEnabled) {
+            NSLog(@"IPhoneMCP stream: discovery request");
+        }
+    }
+}
+
+- (void)noteFrameForProtocol:(NSString *)protocol {
+    @synchronized (self) {
+        if ([protocol isEqualToString:@"tcp"]) {
+            self.tcpFrames += 1;
+        } else if ([protocol isEqualToString:@"udp"]) {
+            self.udpFrames += 1;
+        }
+        self.lastProtocol = protocol ?: @"none";
+        self.lastFrameAt = [NSDate date];
+        if (self.debugEnabled) {
+            NSLog(@"IPhoneMCP stream: %@ frame displayed", self.lastProtocol);
+        }
+    }
+}
+
+@end

iphone_app/IMCViewController.h

@@ -0,0 +1,5 @@
+#import <UIKit/UIKit.h>
+
+@interface IMCViewController : UIViewController
+- (NSDictionary *)handleRPCRequest:(NSDictionary *)request;
+@end

iphone_app/IMCViewController.m

@@ -0,0 +1,846 @@
+#import "IMCViewController.h"
+#import "IMCCanvasView.h"
+#import "IMCMCPServer.h"
+#import "IMCVideoStreamServer.h"
+#import <AVFoundation/AVFoundation.h>
+#import <math.h>
+
+@interface IMCPhotoCaptureDelegate : NSObject <AVCapturePhotoCaptureDelegate>
+@property (nonatomic, copy) void (^completion)(NSData *jpegData, NSError *error);
+@end
+
+@implementation IMCPhotoCaptureDelegate
+
+- (void)captureOutput:(AVCapturePhotoOutput *)output
+didFinishProcessingPhoto:(AVCapturePhoto *)photo
+                error:(NSError *)error {
+    NSData *data = error ? nil : [photo fileDataRepresentation];
+    if (self.completion) {
+        self.completion(data, error);
+    }
+}
+
+@end
+
+@interface IMCViewController ()
+@property (nonatomic, strong) IMCCanvasView *canvasView;
+@property (nonatomic, strong) IMCMCPServer *server;
+@property (nonatomic, strong) IMCVideoStreamServer *videoStreamServer;
+@property (nonatomic, strong) AVAudioPlayer *audioPlayer;
+@property (nonatomic, strong) AVAudioRecorder *audioRecorder;
+@end
+
+@implementation IMCViewController
+
+- (void)viewDidLoad {
+    [super viewDidLoad];
+
+    self.view.backgroundColor = [UIColor colorWithRed:0.03 green:0.06 blue:0.12 alpha:1.0];
+
+    self.canvasView = [[IMCCanvasView alloc] initWithFrame:CGRectZero];
+    self.canvasView.translatesAutoresizingMaskIntoConstraints = NO;
+    [self.view addSubview:self.canvasView];
+
+    [NSLayoutConstraint activateConstraints:@[
+        [self.canvasView.topAnchor constraintEqualToAnchor:self.view.topAnchor],
+        [self.canvasView.bottomAnchor constraintEqualToAnchor:self.view.bottomAnchor],
+        [self.canvasView.leadingAnchor constraintEqualToAnchor:self.view.leadingAnchor],
+        [self.canvasView.trailingAnchor constraintEqualToAnchor:self.view.trailingAnchor]
+    ]];
+
+    [self.canvasView clearWithColor:0];
+    [self.canvasView drawText:@"iPhone MCP landscape server ready" x:18 y:18 size:2];
+    [self.canvasView drawText:@"POST /api/mcp on port 8080" x:18 y:58 size:1];
+
+    self.server = [[IMCMCPServer alloc] initWithViewController:self port:8080];
+    [self.server start];
+
+    self.videoStreamServer = [[IMCVideoStreamServer alloc] initWithCanvasView:self.canvasView];
+    [self.videoStreamServer start];
+}
+
+- (NSDictionary *)handleRPCRequest:(NSDictionary *)request {
+    __block NSDictionary *response = nil;
+    dispatch_sync(dispatch_get_main_queue(), ^{
+        response = [self handleRPCRequestOnMainThread:request];
+    });
+    return response;
+}
+
+- (NSDictionary *)handleRPCRequestOnMainThread:(NSDictionary *)request {
+    id rpcID = request[@"id"] ?: [NSNull null];
+    NSString *method = request[@"method"];
+
+    if ([method isEqualToString:@"initialize"]) {
+        return [self result:@{
+            @"protocolVersion": @"2024-11-05",
+            @"capabilities": @{@"tools": @{}},
+            @"serverInfo": @{@"name": @"iphone6-mcp", @"version": @"0.0.4"}
+        } rpcID:rpcID];
+    }
+
+    if ([method isEqualToString:@"notifications/initialized"]) {
+        return [self result:@{} rpcID:rpcID];
+    }
+
+    if ([method isEqualToString:@"tools/list"]) {
+        return [self result:@{@"tools": [self tools]} rpcID:rpcID];
+    }
+
+    if ([method isEqualToString:@"tools/call"]) {
+        NSDictionary *params = request[@"params"] ?: @{};
+        NSString *name = params[@"name"];
+        NSDictionary *arguments = params[@"arguments"] ?: @{};
+        return [self callTool:name arguments:arguments rpcID:rpcID];
+    }
+
+    return [self errorWithCode:-32601 message:[NSString stringWithFormat:@"Method %@ not found", method] rpcID:rpcID];
+}
+
+- (NSDictionary *)callTool:(NSString *)name arguments:(NSDictionary *)arguments rpcID:(id)rpcID {
+    @try {
+        if ([name isEqualToString:@"clear_screen"]) {
+            NSInteger color = [arguments[@"color"] integerValue];
+            [self.canvasView clearWithColor:color];
+            return [self textResult:@"Screen cleared." rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"draw_text"]) {
+            NSString *text = [NSString stringWithFormat:@"%@", arguments[@"text"] ?: @""];
+            CGFloat x = [arguments[@"x"] doubleValue];
+            CGFloat y = [arguments[@"y"] doubleValue];
+            NSInteger size = arguments[@"size"] ? [arguments[@"size"] integerValue] : 1;
+            [self.canvasView drawText:text x:x y:y size:size];
+            return [self textResult:[NSString stringWithFormat:@"Drew text '%@' at (%.0f, %.0f).", text, x, y] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"draw_image"]) {
+            NSString *base64 = [NSString stringWithFormat:@"%@", arguments[@"image_base64"] ?: @""];
+            NSRange marker = [base64 rangeOfString:@";base64,"];
+            if (marker.location != NSNotFound) {
+                base64 = [base64 substringFromIndex:marker.location + marker.length];
+            }
+            NSData *data = [[NSData alloc] initWithBase64EncodedString:base64 options:NSDataBase64DecodingIgnoreUnknownCharacters];
+            if (!data) {
+                return [self errorWithCode:-32602 message:@"Invalid image_base64." rpcID:rpcID];
+            }
+            [self.canvasView drawImageData:data
+                                         x:[arguments[@"x"] doubleValue]
+                                         y:[arguments[@"y"] doubleValue]];
+            return [self textResult:@"Image displayed." rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"get_screenshot"]) {
+            NSString *png = [self.canvasView snapshotPNGBase64];
+            return [self result:@{@"content": @[@{@"type": @"image", @"data": png, @"mimeType": @"image/png"}]} rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"get_battery"]) {
+            UIDevice *device = [UIDevice currentDevice];
+            device.batteryMonitoringEnabled = YES;
+            NSDictionary *battery = @{
+                @"level_pct": @(MAX(device.batteryLevel, 0.0) * 100.0),
+                @"state": [self batteryStateName:device.batteryState]
+            };
+            return [self textResult:[self jsonString:battery] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"get_sensors"] || [name isEqualToString:@"scan_ble"] || [name isEqualToString:@"set_led"]) {
+            return [self textResult:[NSString stringWithFormat:@"%@ is not implemented in this first iPhone build.", name] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"play_tone"]) {
+            NSInteger frequency = arguments[@"frequency"] ? [arguments[@"frequency"] integerValue] : 440;
+            NSInteger duration = arguments[@"duration_ms"] ? [arguments[@"duration_ms"] integerValue] : 1000;
+            NSInteger volume = arguments[@"volume"] ? [arguments[@"volume"] integerValue] : 50;
+
+            frequency = MAX(50, MIN(10000, frequency));
+            duration = MAX(50, MIN(5000, duration));
+            volume = MAX(0, MIN(100, volume));
+
+            NSData *toneData = [self wavDataForToneFrequency:frequency
+                                                  durationMS:duration
+                                                      volume:volume];
+            NSDictionary *error = [self playAudioData:toneData volume:volume];
+            if (error) {
+                return [self errorWithCode:[error[@"code"] integerValue]
+                                   message:error[@"message"]
+                                     rpcID:rpcID];
+            }
+            return [self textResult:[NSString stringWithFormat:@"Played tone of %ldHz for %ldms at volume %ld.",
+                                     (long)frequency,
+                                     (long)duration,
+                                     (long)volume]
+                              rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"play_audio_base64"]) {
+            NSString *base64 = [self base64PayloadFromString:[NSString stringWithFormat:@"%@", arguments[@"wav_base64"] ?: @""]];
+            NSData *audioData = [[NSData alloc] initWithBase64EncodedString:base64
+                                                                    options:NSDataBase64DecodingIgnoreUnknownCharacters];
+            if (!audioData) {
+                return [self errorWithCode:-32602 message:@"Invalid wav_base64." rpcID:rpcID];
+            }
+
+            NSInteger volumeArg = arguments[@"volume"] ? [arguments[@"volume"] integerValue] : 50;
+            volumeArg = MAX(0, MIN(100, volumeArg));
+            NSDictionary *error = [self playAudioData:audioData volume:volumeArg];
+            if (error) {
+                return [self errorWithCode:[error[@"code"] integerValue]
+                                   message:error[@"message"]
+                                     rpcID:rpcID];
+            }
+            return [self textResult:[NSString stringWithFormat:@"Playing base64 WAV audio (%lu bytes) at volume %ld.",
+                                     (unsigned long)audioData.length,
+                                     (long)volumeArg]
+                              rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"play_audio"]) {
+            NSString *filename = [NSString stringWithFormat:@"%@", arguments[@"filename"] ?: @""];
+            NSURL *url = [self safeFileURLForPath:filename];
+            if (!url) {
+                return [self errorWithCode:-32602
+                                   message:@"filename must be relative and cannot contain '..'."
+                                     rpcID:rpcID];
+            }
+
+            NSData *audioData = [NSData dataWithContentsOfURL:url];
+            if (!audioData) {
+                return [self errorWithCode:-32000
+                                   message:[NSString stringWithFormat:@"Could not read audio file '%@' from Documents/MCPFiles.", filename]
+                                     rpcID:rpcID];
+            }
+
+            NSInteger volumeArg = arguments[@"volume"] ? [arguments[@"volume"] integerValue] : 50;
+            volumeArg = MAX(0, MIN(100, volumeArg));
+            NSDictionary *error = [self playAudioData:audioData volume:volumeArg];
+            if (error) {
+                return [self errorWithCode:[error[@"code"] integerValue]
+                                   message:error[@"message"]
+                                     rpcID:rpcID];
+            }
+            return [self textResult:[NSString stringWithFormat:@"Playing WAV file '%@' (%lu bytes) at volume %ld.",
+                                     filename,
+                                     (unsigned long)audioData.length,
+                                     (long)volumeArg]
+                              rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"hermes_voice_turn"]) {
+            NSDictionary *voiceResult = [self performHermesVoiceTurn:arguments];
+            if (voiceResult[@"error"]) {
+                return [self errorWithCode:-32000 message:voiceResult[@"error"] rpcID:rpcID];
+            }
+            return [self textResult:[self jsonString:voiceResult] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"record_voice"]) {
+            if (![self ensureMicrophonePermission]) {
+                return [self errorWithCode:-32000
+                                   message:@"Microphone permission is not granted for iPhone MCP."
+                                     rpcID:rpcID];
+            }
+
+            NSInteger duration = arguments[@"duration_sec"] ? [arguments[@"duration_sec"] integerValue] : 4;
+            duration = MAX(1, MIN(15, duration));
+            NSString *filename = [NSString stringWithFormat:@"%@", arguments[@"filename"] ?: @"recording.wav"];
+            NSURL *url = [self safeFileURLForPath:filename];
+            if (!url) {
+                return [self errorWithCode:-32602
+                                   message:@"filename must be relative and cannot contain '..'."
+                                     rpcID:rpcID];
+            }
+
+            NSDictionary *result = [self recordVoiceToURL:url duration:duration];
+            if (result[@"error"]) {
+                return [self errorWithCode:-32000 message:result[@"error"] rpcID:rpcID];
+            }
+            return [self textResult:[self jsonString:result] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"capture_selfie"]) {
+            if (![self ensureCameraPermission]) {
+                return [self errorWithCode:-32000
+                                   message:@"Camera permission is not granted for iPhone MCP."
+                                     rpcID:rpcID];
+            }
+
+            NSDictionary *result = [self captureSelfiePNGBase64];
+            if (result[@"error"]) {
+                return [self errorWithCode:-32000 message:result[@"error"] rpcID:rpcID];
+            }
+            return [self result:@{@"content": @[@{@"type": @"image",
+                                                   @"data": result[@"png_base64"],
+                                                   @"mimeType": @"image/png"}]} rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"write_file"]) {
+            NSString *path = [NSString stringWithFormat:@"%@", arguments[@"path"] ?: @""];
+            NSString *content = [NSString stringWithFormat:@"%@", arguments[@"content"] ?: @""];
+            NSURL *url = [self safeFileURLForPath:path];
+            if (!url) {
+                return [self errorWithCode:-32602 message:@"Path must be relative and cannot contain '..'." rpcID:rpcID];
+            }
+            [[NSFileManager defaultManager] createDirectoryAtURL:[url URLByDeletingLastPathComponent]
+                                     withIntermediateDirectories:YES
+                                                      attributes:nil
+                                                           error:nil];
+            NSError *writeError = nil;
+            [content writeToURL:url atomically:YES encoding:NSUTF8StringEncoding error:&writeError];
+            if (writeError) {
+                return [self errorWithCode:-32000 message:writeError.localizedDescription rpcID:rpcID];
+            }
+            return [self textResult:[NSString stringWithFormat:@"Wrote %lu characters to %@.", (unsigned long)content.length, path] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"read_file"]) {
+            NSString *path = [NSString stringWithFormat:@"%@", arguments[@"path"] ?: @""];
+            NSURL *url = [self safeFileURLForPath:path];
+            if (!url) {
+                return [self errorWithCode:-32602 message:@"Path must be relative and cannot contain '..'." rpcID:rpcID];
+            }
+            NSError *readError = nil;
+            NSString *content = [NSString stringWithContentsOfURL:url encoding:NSUTF8StringEncoding error:&readError];
+            if (readError) {
+                return [self errorWithCode:-32000 message:readError.localizedDescription rpcID:rpcID];
+            }
+            return [self textResult:content rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"download_file"]) {
+            NSString *urlString = [NSString stringWithFormat:@"%@", arguments[@"url"] ?: @""];
+            NSString *filename = [NSString stringWithFormat:@"%@", arguments[@"filename"] ?: @""];
+            BOOL useSD = arguments[@"use_sd"] ? [arguments[@"use_sd"] boolValue] : NO;
+            NSURL *sourceURL = [NSURL URLWithString:urlString];
+            NSURL *destinationURL = [self safeFileURLForPath:filename];
+            if (!sourceURL || !destinationURL) {
+                return [self errorWithCode:-32602 message:@"Invalid url or filename." rpcID:rpcID];
+            }
+            NSData *data = [NSData dataWithContentsOfURL:sourceURL];
+            if (!data) {
+                return [self errorWithCode:-32000 message:@"Download failed." rpcID:rpcID];
+            }
+            [[NSFileManager defaultManager] createDirectoryAtURL:[destinationURL URLByDeletingLastPathComponent]
+                                     withIntermediateDirectories:YES
+                                                      attributes:nil
+                                                           error:nil];
+            [data writeToURL:destinationURL atomically:YES];
+            NSString *storageNote = useSD ? @" The iPhone build has no SD card, so use_sd was accepted and stored in app Documents instead." : @"";
+            return [self textResult:[NSString stringWithFormat:@"Downloaded %lu bytes to %@.%@", (unsigned long)data.length, filename, storageNote] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"get_video_streaming_instructions"]) {
+            NSString *protocol = [[NSString stringWithFormat:@"%@", arguments[@"protocol"] ?: @"both"] lowercaseString];
+            return [self textResult:[self videoStreamingInstructionsForProtocol:protocol] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"get_stream_stats"]) {
+            return [self textResult:[self jsonString:[self.videoStreamServer streamStats]] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"reset_stream_stats"]) {
+            [self.videoStreamServer resetStreamStats];
+            return [self textResult:[self jsonString:[self.videoStreamServer streamStats]] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"set_stream_debug"]) {
+            BOOL enabled = arguments[@"enabled"] ? [arguments[@"enabled"] boolValue] : NO;
+            [self.videoStreamServer setDebugEnabled:enabled];
+            return [self textResult:[self jsonString:@{@"debug_enabled": @(enabled)}] rpcID:rpcID];
+        }
+
+        if ([name isEqualToString:@"execute_python"]) {
+            return [self textResult:@"execute_python is available on the MicroPython ESP32 target, but the native iPhone app does not execute arbitrary Python code. Use write_file/read_file/download_file for files and the dedicated drawing/audio/camera tools for app actions." rpcID:rpcID];
+        }
+
+        return [self errorWithCode:-32602 message:[NSString stringWithFormat:@"Unknown tool: %@", name] rpcID:rpcID];
+    } @catch (NSException *exception) {
+        return [self errorWithCode:-32000 message:exception.reason ?: @"Tool failed." rpcID:rpcID];
+    }
+}
+
+- (NSArray *)tools {
+    return @[
+        [self tool:@"clear_screen" description:@"Clear the iPhone canvas to white (0) or black (1)." schema:@{@"type": @"object", @"properties": @{@"color": @{@"type": @"integer", @"enum": @[@0, @1]}}, @"required": @[@"color"]}],
+        [self tool:@"draw_text" description:@"Draw text on the iPhone canvas using 400x300 logical coordinates." schema:@{@"type": @"object", @"properties": @{@"text": @{@"type": @"string"}, @"x": @{@"type": @"integer"}, @"y": @{@"type": @"integer"}, @"size": @{@"type": @"integer", @"enum": @[@1, @2]}}, @"required": @[@"text", @"x", @"y"]}],
+        [self tool:@"draw_image" description:@"Draw a base64 PNG/JPEG/GIF image on the iPhone canvas." schema:@{@"type": @"object", @"properties": @{@"image_base64": @{@"type": @"string"}, @"x": @{@"type": @"integer", @"default": @0}, @"y": @{@"type": @"integer", @"default": @0}}, @"required": @[@"image_base64"]}],
+        [self tool:@"get_screenshot" description:@"Return the current iPhone canvas as a PNG image." schema:@{@"type": @"object", @"properties": @{}}],
+        [self tool:@"get_battery" description:@"Read the iPhone battery percentage and charge state." schema:@{@"type": @"object", @"properties": @{}}],
+        [self tool:@"play_tone" description:@"Generate and play a sine-wave tone with the requested frequency, duration, and volume." schema:@{@"type": @"object", @"properties": @{@"frequency": @{@"type": @"integer", @"default": @440}, @"duration_ms": @{@"type": @"integer", @"default": @1000}, @"volume": @{@"type": @"integer", @"default": @50}}}],
+        [self tool:@"play_audio_base64" description:@"Decode a base64 WAV payload and play it through the iPhone speaker." schema:@{@"type": @"object", @"properties": @{@"wav_base64": @{@"type": @"string"}, @"volume": @{@"type": @"integer", @"default": @50}}, @"required": @[@"wav_base64"]}],
+        [self tool:@"play_audio" description:@"Play a WAV file from the app's Documents/MCPFiles directory." schema:@{@"type": @"object", @"properties": @{@"filename": @{@"type": @"string"}, @"volume": @{@"type": @"integer", @"default": @50}}, @"required": @[@"filename"]}],
+        [self tool:@"hermes_voice_turn" description:@"Record a voice command, send it to the Hermes voice gateway, play the returned WAV, and return transcript/response diagnostics." schema:@{@"type": @"object", @"properties": @{@"url": @{@"type": @"string", @"default": @"http://192.168.68.126:8642/api/esp32/voice"}, @"api_token": @{@"type": @"string"}, @"device_id": @{@"type": @"string", @"default": @"iphone6"}, @"duration_sec": @{@"type": @"integer", @"default": @4}, @"volume": @{@"type": @"integer", @"default": @80}, @"instructions": @{@"type": @"string"}, @"reply_mode": @{@"type": @"string", @"enum": @[@"sync", @"ack"], @"default": @"sync"}, @"screen_url": @{@"type": @"string"}}}],
+        [self tool:@"record_voice" description:@"Record microphone audio to a WAV file inside Documents/MCPFiles." schema:@{@"type": @"object", @"properties": @{@"duration_sec": @{@"type": @"integer", @"default": @4}, @"filename": @{@"type": @"string", @"default": @"recording.wav"}}}],
+        [self tool:@"capture_selfie" description:@"Capture a still image from the front camera and return it as PNG image content." schema:@{@"type": @"object", @"properties": @{}}],
+        [self tool:@"write_file" description:@"Write a UTF-8 file inside the app's Documents/MCPFiles directory." schema:@{@"type": @"object", @"properties": @{@"path": @{@"type": @"string"}, @"content": @{@"type": @"string"}}, @"required": @[@"path", @"content"]}],
+        [self tool:@"read_file" description:@"Read a UTF-8 file from the app's Documents/MCPFiles directory." schema:@{@"type": @"object", @"properties": @{@"path": @{@"type": @"string"}}, @"required": @[@"path"]}],
+        [self tool:@"download_file" description:@"Download a URL into the app's Documents/MCPFiles directory. Accepts ESP32-compatible use_sd, but stores locally on iPhone." schema:@{@"type": @"object", @"properties": @{@"url": @{@"type": @"string"}, @"filename": @{@"type": @"string"}, @"use_sd": @{@"type": @"boolean", @"default": @NO}}, @"required": @[@"url", @"filename"]}],
+        [self tool:@"get_video_streaming_instructions" description:@"Get details for monochrome TCP/UDP streaming and RGB565 color TCP streaming." schema:@{@"type": @"object", @"properties": @{@"protocol": @{@"type": @"string", @"enum": @[@"tcp", @"udp", @"color", @"both", @"all"], @"default": @"all"}}}],
+        [self tool:@"get_stream_stats" description:@"Return TCP/UDP streaming counters, ports, uptime, last protocol, and debug state." schema:@{@"type": @"object", @"properties": @{}}],
+        [self tool:@"reset_stream_stats" description:@"Reset TCP/UDP streaming counters and last-frame state." schema:@{@"type": @"object", @"properties": @{}}],
+        [self tool:@"set_stream_debug" description:@"Enable or disable stream debug logging on the iPhone." schema:@{@"type": @"object", @"properties": @{@"enabled": @{@"type": @"boolean", @"default": @NO}}}],
+        [self tool:@"execute_python" description:@"ESP32 compatibility stub; arbitrary Python execution is intentionally not implemented in the native iPhone app." schema:@{@"type": @"object", @"properties": @{@"code": @{@"type": @"string"}}, @"required": @[@"code"]}],
+        [self tool:@"set_led" description:@"ESP32 compatibility stub; the iPhone 6 has no NeoPixel LED." schema:@{@"type": @"object", @"properties": @{}}],
+        [self tool:@"get_sensors" description:@"ESP32 compatibility stub; SHTC3 temperature/humidity is not present." schema:@{@"type": @"object", @"properties": @{}}],
+        [self tool:@"scan_ble" description:@"ESP32 compatibility stub; BLE scanning is not implemented in this first build." schema:@{@"type": @"object", @"properties": @{}}]
+    ];
+}
+
+- (NSDictionary *)tool:(NSString *)name description:(NSString *)description schema:(NSDictionary *)schema {
+    return @{@"name": name, @"description": description, @"inputSchema": schema};
+}
+
+- (NSDictionary *)textResult:(NSString *)text rpcID:(id)rpcID {
+    return [self result:@{@"content": @[@{@"type": @"text", @"text": text ?: @""}]} rpcID:rpcID];
+}
+
+- (NSDictionary *)result:(NSDictionary *)result rpcID:(id)rpcID {
+    return @{@"jsonrpc": @"2.0", @"result": result ?: @{}, @"id": rpcID ?: [NSNull null]};
+}
+
+- (NSDictionary *)errorWithCode:(NSInteger)code message:(NSString *)message rpcID:(id)rpcID {
+    return @{@"jsonrpc": @"2.0",
+             @"error": @{@"code": @(code), @"message": message ?: @"Error"},
+             @"id": rpcID ?: [NSNull null]};
+}
+
+- (NSString *)jsonString:(id)object {
+    NSData *data = [NSJSONSerialization dataWithJSONObject:object options:0 error:nil];
+    return [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding] ?: @"{}";
+}
+
+- (NSString *)base64PayloadFromString:(NSString *)string {
+    NSRange marker = [string rangeOfString:@";base64,"];
+    if (marker.location != NSNotFound) {
+        return [string substringFromIndex:marker.location + marker.length];
+    }
+    return string ?: @"";
+}
+
+- (NSDictionary *)playAudioData:(NSData *)audioData volume:(NSInteger)volume {
+    NSError *sessionError = nil;
+    [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:&sessionError];
+    [[AVAudioSession sharedInstance] setActive:YES error:&sessionError];
+
+    NSError *playerError = nil;
+    self.audioPlayer = [[AVAudioPlayer alloc] initWithData:audioData error:&playerError];
+    if (!self.audioPlayer || playerError) {
+        return @{@"code": @-32000,
+                 @"message": playerError.localizedDescription ?: @"Failed to create audio player."};
+    }
+
+    self.audioPlayer.volume = MAX(0.0, MIN(1.0, volume / 100.0));
+    [self.audioPlayer prepareToPlay];
+    [self.audioPlayer play];
+    return nil;
+}
+
+- (NSString *)headerValue:(NSString *)name fromResponse:(NSHTTPURLResponse *)response {
+    for (id key in response.allHeaderFields) {
+        if ([[NSString stringWithFormat:@"%@", key] caseInsensitiveCompare:name] == NSOrderedSame) {
+            return [NSString stringWithFormat:@"%@", response.allHeaderFields[key]];
+        }
+    }
+    return @"";
+}
+
+- (NSDictionary *)performHermesVoiceTurn:(NSDictionary *)arguments {
+    if (![self ensureMicrophonePermission]) {
+        return @{@"error": @"Microphone permission is not granted for iPhone MCP."};
+    }
+
+    NSInteger duration = arguments[@"duration_sec"] ? [arguments[@"duration_sec"] integerValue] : 4;
+    duration = MAX(1, MIN(20, duration));
+    NSInteger volume = arguments[@"volume"] ? [arguments[@"volume"] integerValue] : 80;
+    volume = MAX(0, MIN(100, volume));
+    NSString *urlString = [NSString stringWithFormat:@"%@", arguments[@"url"] ?: @"http://192.168.68.126:8642/api/esp32/voice"];
+    NSURL *gatewayURL = [NSURL URLWithString:urlString];
+    if (!gatewayURL || ![@[@"http", @"https"] containsObject:gatewayURL.scheme.lowercaseString]) {
+        return @{@"error": @"Hermes url must be an http or https URL."};
+    }
+
+    NSURL *recordingURL = [self safeFileURLForPath:@"hermes/request.wav"];
+    NSDictionary *recording = [self recordVoiceToURL:recordingURL duration:duration];
+    if (recording[@"error"]) {
+        return recording;
+    }
+    NSData *wavData = [NSData dataWithContentsOfURL:recordingURL];
+    if (!wavData.length) {
+        return @{@"error": @"Hermes recording produced no audio data."};
+    }
+
+    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:gatewayURL
+                                                           cachePolicy:NSURLRequestReloadIgnoringLocalCacheData
+                                                       timeoutInterval:90.0];
+    request.HTTPMethod = @"POST";
+    request.HTTPBody = wavData;
+    [request setValue:@"audio/wav" forHTTPHeaderField:@"Content-Type"];
+    [request setValue:[NSString stringWithFormat:@"%@", arguments[@"device_id"] ?: @"iphone6"]
+    forHTTPHeaderField:@"X-Device-ID"];
+    [request setValue:[NSString stringWithFormat:@"%@", arguments[@"reply_mode"] ?: @"sync"]
+    forHTTPHeaderField:@"X-Hermes-Reply-Mode"];
+
+    NSString *instructions = [NSString stringWithFormat:@"%@", arguments[@"instructions"] ?: @""];
+    if (instructions.length) {
+        [request setValue:instructions forHTTPHeaderField:@"X-Hermes-Instructions"];
+    }
+    NSString *screenURL = [NSString stringWithFormat:@"%@", arguments[@"screen_url"] ?: @""];
+    if (screenURL.length) {
+        [request setValue:screenURL forHTTPHeaderField:@"X-Hermes-Screen-Url"];
+    }
+    NSString *token = [NSString stringWithFormat:@"%@", arguments[@"api_token"] ?: @""];
+    if (token.length) {
+        if (![token.lowercaseString hasPrefix:@"bearer "]) {
+            token = [@"Bearer " stringByAppendingString:token];
+        }
+        [request setValue:token forHTTPHeaderField:@"Authorization"];
+    }
+
+    NSURLResponse *rawResponse = nil;
+    NSError *requestError = nil;
+    NSData *responseData = [NSURLConnection sendSynchronousRequest:request
+                                                 returningResponse:&rawResponse
+                                                             error:&requestError];
+    if (requestError || !responseData) {
+        if ([requestError.domain isEqualToString:NSURLErrorDomain] &&
+            (requestError.code == NSURLErrorUserCancelledAuthentication ||
+             requestError.code == NSURLErrorUserAuthenticationRequired)) {
+            return @{@"error": @"Hermes authorization failed. Provide a valid api_token to hermes_voice_turn."};
+        }
+        return @{@"error": requestError.localizedDescription ?: @"Hermes request failed."};
+    }
+    NSHTTPURLResponse *response = (NSHTTPURLResponse *)rawResponse;
+    if (response.statusCode < 200 || response.statusCode >= 300) {
+        NSString *body = [[NSString alloc] initWithData:responseData encoding:NSUTF8StringEncoding] ?: @"";
+        return @{@"error": [NSString stringWithFormat:@"Hermes returned HTTP %ld: %@",
+                              (long)response.statusCode,
+                              [body substringToIndex:MIN(body.length, 500)]]};
+    }
+
+    NSDictionary *playError = [self playAudioData:responseData volume:volume];
+    if (playError) {
+        return @{@"error": playError[@"message"] ?: @"Hermes audio playback failed."};
+    }
+    NSURL *responseURL = [self safeFileURLForPath:@"hermes/response.wav"];
+    [responseData writeToURL:responseURL atomically:YES];
+
+    return @{
+        @"status": @"playing",
+        @"http_status": @(response.statusCode),
+        @"request_bytes": @(wavData.length),
+        @"response_bytes": @(responseData.length),
+        @"transcript": [self headerValue:@"X-Hermes-Transcript" fromResponse:response],
+        @"response_text": [self headerValue:@"X-Hermes-Text-Response" fromResponse:response],
+        @"response_id": [self headerValue:@"X-Hermes-Response-Id" fromResponse:response],
+        @"conversation": [self headerValue:@"X-Hermes-Conversation" fromResponse:response],
+        @"reply_mode": [NSString stringWithFormat:@"%@", arguments[@"reply_mode"] ?: @"sync"],
+        @"saved_response": @"hermes/response.wav"
+    };
+}
+
+- (NSDictionary *)recordVoiceToURL:(NSURL *)url duration:(NSInteger)duration {
+    NSError *directoryError = nil;
+    [[NSFileManager defaultManager] createDirectoryAtURL:[url URLByDeletingLastPathComponent]
+                             withIntermediateDirectories:YES
+                                              attributes:nil
+                                                   error:&directoryError];
+    if (directoryError) {
+        return @{@"error": directoryError.localizedDescription};
+    }
+
+    NSError *sessionError = nil;
+    AVAudioSession *session = [AVAudioSession sharedInstance];
+    [session setCategory:AVAudioSessionCategoryPlayAndRecord error:&sessionError];
+    [session setActive:YES error:&sessionError];
+    if (sessionError) {
+        return @{@"error": sessionError.localizedDescription};
+    }
+
+    NSDictionary *settings = @{
+        AVFormatIDKey: @(kAudioFormatLinearPCM),
+        AVSampleRateKey: @16000,
+        AVNumberOfChannelsKey: @1,
+        AVLinearPCMBitDepthKey: @16,
+        AVLinearPCMIsFloatKey: @NO,
+        AVLinearPCMIsBigEndianKey: @NO
+    };
+
+    NSError *recordError = nil;
+    self.audioRecorder = [[AVAudioRecorder alloc] initWithURL:url
+                                                     settings:settings
+                                                        error:&recordError];
+    if (!self.audioRecorder || recordError) {
+        return @{@"error": recordError.localizedDescription ?: @"Failed to create audio recorder."};
+    }
+
+    [self.audioRecorder prepareToRecord];
+    [self.audioRecorder recordForDuration:duration];
+    NSDate *deadline = [NSDate dateWithTimeIntervalSinceNow:duration + 0.25];
+    while (self.audioRecorder.recording && [deadline timeIntervalSinceNow] > 0) {
+        [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                 beforeDate:[NSDate dateWithTimeIntervalSinceNow:0.05]];
+    }
+    [self.audioRecorder stop];
+
+    NSDictionary *attributes = [[NSFileManager defaultManager] attributesOfItemAtPath:url.path error:nil];
+    NSNumber *bytes = attributes[NSFileSize] ?: @0;
+    return @{@"filename": url.lastPathComponent ?: @"recording.wav",
+             @"bytes": bytes,
+             @"duration_sec": @(duration)};
+}
+
+- (NSDictionary *)captureSelfiePNGBase64 {
+    AVCaptureDeviceDiscoverySession *discovery = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[
+        AVCaptureDeviceTypeBuiltInWideAngleCamera
+    ]
+                                                                                                           mediaType:AVMediaTypeVideo
+                                                                                                            position:AVCaptureDevicePositionFront];
+    AVCaptureDevice *camera = discovery.devices.firstObject;
+    if (!camera) {
+        return @{@"error": @"Front camera not found."};
+    }
+
+    NSError *inputError = nil;
+    AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:camera error:&inputError];
+    if (!input || inputError) {
+        return @{@"error": inputError.localizedDescription ?: @"Failed to open front camera."};
+    }
+
+    AVCaptureSession *session = [[AVCaptureSession alloc] init];
+    session.sessionPreset = AVCaptureSessionPresetPhoto;
+    if (![session canAddInput:input]) {
+        return @{@"error": @"Could not add front camera input."};
+    }
+    [session addInput:input];
+
+    AVCaptureStillImageOutput *output = [[AVCaptureStillImageOutput alloc] init];
+    output.outputSettings = @{AVVideoCodecKey: AVVideoCodecJPEG};
+    if (![session canAddOutput:output]) {
+        return @{@"error": @"Could not add still image output."};
+    }
+    [session addOutput:output];
+    [session startRunning];
+
+    AVCaptureConnection *connection = [output connectionWithMediaType:AVMediaTypeVideo];
+    if (connection.isVideoOrientationSupported) {
+        connection.videoOrientation = AVCaptureVideoOrientationLandscapeRight;
+    }
+
+    __block NSData *jpegData = nil;
+    __block NSError *captureError = nil;
+    __block BOOL finished = NO;
+
+    [output captureStillImageAsynchronouslyFromConnection:connection
+                                        completionHandler:^(CMSampleBufferRef imageDataSampleBuffer, NSError *error) {
+        if (imageDataSampleBuffer) {
+            jpegData = [AVCaptureStillImageOutput jpegStillImageNSDataRepresentation:imageDataSampleBuffer];
+        }
+        captureError = error;
+        finished = YES;
+    }];
+
+    NSDate *deadline = [NSDate dateWithTimeIntervalSinceNow:5.0];
+    while (!finished && [deadline timeIntervalSinceNow] > 0) {
+        [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                 beforeDate:[NSDate dateWithTimeIntervalSinceNow:0.05]];
+    }
+    [session stopRunning];
+
+    if (captureError || !jpegData) {
+        return @{@"error": captureError.localizedDescription ?: @"Selfie capture timed out."};
+    }
+
+    UIImage *image = [UIImage imageWithData:jpegData];
+    NSData *pngData = UIImagePNGRepresentation(image);
+    if (!pngData) {
+        return @{@"error": @"Failed to convert selfie to PNG."};
+    }
+    return @{@"png_base64": [pngData base64EncodedStringWithOptions:0]};
+}
+
+- (BOOL)ensureMicrophonePermission {
+    AVAudioSession *session = [AVAudioSession sharedInstance];
+    if (session.recordPermission == AVAudioSessionRecordPermissionGranted) {
+        return YES;
+    }
+    if (session.recordPermission == AVAudioSessionRecordPermissionDenied) {
+        return NO;
+    }
+
+    __block BOOL granted = NO;
+    __block BOOL finished = NO;
+    [session requestRecordPermission:^(BOOL didGrant) {
+        granted = didGrant;
+        finished = YES;
+    }];
+    return [self waitForPermissionResult:&finished granted:&granted];
+}
+
+- (BOOL)ensureCameraPermission {
+    AVAuthorizationStatus status = [AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeVideo];
+    if (status == AVAuthorizationStatusAuthorized) {
+        return YES;
+    }
+    if (status == AVAuthorizationStatusDenied || status == AVAuthorizationStatusRestricted) {
+        return NO;
+    }
+
+    __block BOOL granted = NO;
+    __block BOOL finished = NO;
+    [AVCaptureDevice requestAccessForMediaType:AVMediaTypeVideo
+                             completionHandler:^(BOOL didGrant) {
+        granted = didGrant;
+        finished = YES;
+    }];
+    return [self waitForPermissionResult:&finished granted:&granted];
+}
+
+- (BOOL)waitForPermissionResult:(BOOL *)finished granted:(BOOL *)granted {
+    NSDate *deadline = [NSDate dateWithTimeIntervalSinceNow:30.0];
+    while (!*finished && [deadline timeIntervalSinceNow] > 0) {
+        [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                 beforeDate:[NSDate dateWithTimeIntervalSinceNow:0.05]];
+    }
+    return *finished && *granted;
+}
+
+- (NSData *)wavDataForToneFrequency:(NSInteger)frequency durationMS:(NSInteger)durationMS volume:(NSInteger)volume {
+    const uint32_t sampleRate = 22050;
+    const uint16_t channels = 1;
+    const uint16_t bitsPerSample = 16;
+    const uint16_t blockAlign = channels * bitsPerSample / 8;
+    const uint32_t byteRate = sampleRate * blockAlign;
+    uint32_t sampleCount = (uint32_t)((sampleRate * durationMS) / 1000);
+    uint32_t dataSize = sampleCount * blockAlign;
+    uint32_t riffSize = 36 + dataSize;
+
+    NSMutableData *data = [NSMutableData dataWithCapacity:44 + dataSize];
+    [self appendASCII:"RIFF" toData:data];
+    [self appendUInt32LE:riffSize toData:data];
+    [self appendASCII:"WAVE" toData:data];
+    [self appendASCII:"fmt " toData:data];
+    [self appendUInt32LE:16 toData:data];
+    [self appendUInt16LE:1 toData:data];
+    [self appendUInt16LE:channels toData:data];
+    [self appendUInt32LE:sampleRate toData:data];
+    [self appendUInt32LE:byteRate toData:data];
+    [self appendUInt16LE:blockAlign toData:data];
+    [self appendUInt16LE:bitsPerSample toData:data];
+    [self appendASCII:"data" toData:data];
+    [self appendUInt32LE:dataSize toData:data];
+
+    double amplitude = 32767.0 * MAX(0.0, MIN(1.0, volume / 100.0));
+    double phaseStep = 2.0 * M_PI * frequency / sampleRate;
+    for (uint32_t i = 0; i < sampleCount; i++) {
+        double fadeIn = MIN(1.0, i / (sampleRate * 0.01));
+        double fadeOut = MIN(1.0, (sampleCount - i) / (sampleRate * 0.01));
+        double envelope = MIN(fadeIn, fadeOut);
+        int16_t sample = (int16_t)(sin(i * phaseStep) * amplitude * envelope);
+        [self appendUInt16LE:(uint16_t)sample toData:data];
+    }
+
+    return data;
+}
+
+- (void)appendASCII:(const char *)string toData:(NSMutableData *)data {
+    [data appendBytes:string length:4];
+}
+
+- (void)appendUInt16LE:(uint16_t)value toData:(NSMutableData *)data {
+    uint8_t bytes[] = {value & 0xff, (value >> 8) & 0xff};
+    [data appendBytes:bytes length:sizeof(bytes)];
+}
+
+- (void)appendUInt32LE:(uint32_t)value toData:(NSMutableData *)data {
+    uint8_t bytes[] = {
+        value & 0xff,
+        (value >> 8) & 0xff,
+        (value >> 16) & 0xff,
+        (value >> 24) & 0xff
+    };
+    [data appendBytes:bytes length:sizeof(bytes)];
+}
+
+- (NSString *)videoStreamingInstructionsForProtocol:(NSString *)protocol {
+    NSMutableArray *lines = [NSMutableArray arrayWithArray:@[
+        @"### iPhone MCP Video Streaming Instructions",
+        @"Dimensions: 400x300, 1-bit monochrome, using the same 15,000-byte RLCD frame layout as the ESP32 server.",
+        @"The iPhone stretches each received frame over the full landscape screen.",
+        @"UDP discovery: send DISCOVER_SCREEN to port 5000; the iPhone replies SCREEN_IP_8080 because its JSON-RPC endpoint is on port 8080.",
+        @"Mapping logic (Python):",
+        @"  def map_to_rlcd(pil_img):",
+        @"    img_1bit = pil_img.convert('1', dither=1)",
+        @"    px = img_1bit.load()",
+        @"    buf = bytearray(15000)",
+        @"    for y in range(300):",
+        @"      for x in range(400):",
+        @"        if px[x, y]:",
+        @"          inv_y = 299 - y",
+        @"          bx = x // 2",
+        @"          by = inv_y // 4",
+        @"          idx = bx * 75 + by",
+        @"          lx, ly = x % 2, inv_y % 4",
+        @"          bit = 7 - (ly * 2 + lx)",
+        @"          buf[idx] |= (1 << bit)",
+        @"    return buf"
+    ]];
+
+    if ([protocol isEqualToString:@"tcp"] || [protocol isEqualToString:@"both"] || [protocol isEqualToString:@"all"]) {
+        [lines addObject:@"\n**TCP Streaming (Port 8081):**"];
+        [lines addObject:@"Open a TCP connection to the iPhone IP on port 8081 and send consecutive 15,000-byte frame blocks."];
+    }
+
+    if ([protocol isEqualToString:@"udp"] || [protocol isEqualToString:@"both"] || [protocol isEqualToString:@"all"]) {
+        [lines addObject:@"\n**UDP Streaming / Broadcast (Port 8082):**"];
+        [lines addObject:@"Split each 15,000-byte frame into 15 chunks of 1,000 bytes. Send each chunk as a 1002-byte packet: byte 0 = frame_id (0-255), byte 1 = chunk_idx (0-14), bytes 2..1001 = chunk payload."];
+    }
+
+    if ([protocol isEqualToString:@"color"] || [protocol isEqualToString:@"all"]) {
+        [lines addObject:@"\n**RGB565 Color TCP Streaming (Port 8083):**"];
+        [lines addObject:@"Send each frame as a 16-byte header followed by width*height*2 bytes of big-endian RGB565 pixels. Header: bytes 0..3='IMCR', byte 4=1 (version), byte 5=1 (RGB565BE), bytes 6..7=width big-endian, bytes 8..9=height big-endian, bytes 10..13=payload length big-endian, bytes 14..15=flags (zero). Frames may use any dimensions up to 2048x2048 and are scaled to the full iPhone canvas."];
+    }
+
+    return [lines componentsJoinedByString:@"\n"];
+}
+
+- (NSString *)batteryStateName:(UIDeviceBatteryState)state {
+    switch (state) {
+        case UIDeviceBatteryStateUnplugged:
+            return @"unplugged";
+        case UIDeviceBatteryStateCharging:
+            return @"charging";
+        case UIDeviceBatteryStateFull:
+            return @"full";
+        default:
+            return @"unknown";
+    }
+}
+
+- (NSURL *)safeFileURLForPath:(NSString *)path {
+    if (path.length == 0 || [path hasPrefix:@"/"]) {
+        return nil;
+    }
+    if ([[path pathComponents] containsObject:@".."]) {
+        return nil;
+    }
+
+    NSArray *directories = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
+    NSString *documents = directories.firstObject;
+    NSString *root = [documents stringByAppendingPathComponent:@"MCPFiles"];
+    return [NSURL fileURLWithPath:[root stringByAppendingPathComponent:path]];
+}
+
+- (BOOL)prefersStatusBarHidden {
+    return YES;
+}
+
+- (UIInterfaceOrientationMask)supportedInterfaceOrientations {
+    return UIInterfaceOrientationMaskLandscape;
+}
+
+- (UIInterfaceOrientation)preferredInterfaceOrientationForPresentation {
+    return UIInterfaceOrientationLandscapeRight;
+}
+
+@end

iphone_app/Makefile

@@ -0,0 +1,13 @@
+TARGET := iphone:clang:12.4:12.0
+ARCHS = arm64
+INSTALL_TARGET_PROCESSES = IPhoneMCP
+
+include $(THEOS)/makefiles/common.mk
+
+APPLICATION_NAME = IPhoneMCP
+
+IPhoneMCP_FILES = main.m IMCAppDelegate.m IMCViewController.m IMCCanvasView.m IMCMCPServer.m IMCVideoStreamServer.m
+IPhoneMCP_FRAMEWORKS = UIKit Foundation AVFoundation CoreGraphics
+IPhoneMCP_CFLAGS = -fobjc-arc -Wno-deprecated-declarations
+
+include $(THEOS_MAKE_PATH)/application.mk

iphone_app/README.md

@@ -0,0 +1,188 @@
+# iPhone MCP
+
+Native UIKit MCP screen server for a jailbroken arm64 iPhone running iOS 12. It shares the ESP32 screen's MCP tool shape while using the iPhone display, speaker, microphone, battery, and front camera.
+
+## Features
+
+- HTTP JSON-RPC MCP endpoint: `POST http://<iphone-ip>:8080/api/mcp`
+- Full-screen landscape canvas with idle sleep disabled while the app is active
+- ESP32-compatible 400x300 monochrome streaming: TCP `8081`, UDP `8082`
+- Framed RGB565 color streaming: TCP `8083`
+- UDP discovery on `5000`: `DISCOVER_SCREEN` -> `SCREEN_IP_8080`
+- True pitched tones, WAV playback, microphone recording, and selfie capture
+- Hermes voice gateway turn: record -> transcribe/agent/TTS -> play response
+- Stream statistics and MCP screenshots
+
+## Requirements
+
+- Jailbroken arm64 iPhone (tested on iPhone 6 / iOS 12.5.8)
+- OpenSSH and `ldid` installed on the phone
+- Theos at `/var/mobile/theos` with `iPhoneOS12.4.sdk`
+- The phone and build computer on the same LAN
+
+## Copy To The Phone
+
+From the repository root:
+
+```sh
+rsync -av iphone_app/ root@<iphone-ip>:/var/mobile/IPhoneMCP/
+```
+
+Keep the phone unlocked while launching the app. On iOS 12, `uiopen` may refuse to launch a locked device.
+
+## Build
+
+The included direct build script is the most reliable path on the iPhone 6:
+
+```sh
+ssh root@<iphone-ip>
+su - mobile -c 'chmod +x /var/mobile/IPhoneMCP/manual_build_iphone.sh && /var/mobile/IPhoneMCP/manual_build_iphone.sh /var/mobile/IPhoneMCP'
+```
+
+This creates and signs:
+
+```text
+/var/mobile/IPhoneMCP/.manual_build/IPhoneMCP.app
+```
+
+The regular Theos build is also supported:
+
+```sh
+su - mobile -c 'env THEOS=/var/mobile/theos PATH=/usr/bin:/bin:/usr/sbin:/sbin make -C /var/mobile/IPhoneMCP clean package SDKVERSION=12.4 INCLUDE_SDKVERSION=12.4 FAKEROOT="bash /var/mobile/theos/bin/fakeroot.sh -p /var/mobile/IPhoneMCP/.theos/fakeroot" _THEOS_PLATFORM_DPKG_DEB=dpkg-deb THEOS_PLATFORM_DEB_COMPRESSION_TYPE=gzip'
+```
+
+## Package And Install
+
+For a direct-build bundle:
+
+```sh
+PKGROOT=/var/mobile/IPhoneMCP/.manual_pkg
+DEB=/var/mobile/IPhoneMCP/packages/com.reynafamily.iphonemcp_manual_iphoneos-arm.deb
+rm -rf "$PKGROOT"
+mkdir -p "$PKGROOT/DEBIAN" "$PKGROOT/Applications/IPhoneMCP.app" /var/mobile/IPhoneMCP/packages
+cp /var/mobile/IPhoneMCP/control "$PKGROOT/DEBIAN/control"
+cp /var/mobile/IPhoneMCP/.manual_build/IPhoneMCP.app/IPhoneMCP "$PKGROOT/Applications/IPhoneMCP.app/IPhoneMCP"
+cp /var/mobile/IPhoneMCP/.manual_build/IPhoneMCP.app/Info.plist "$PKGROOT/Applications/IPhoneMCP.app/Info.plist"
+chmod 755 "$PKGROOT/Applications/IPhoneMCP.app/IPhoneMCP"
+chown -R root:wheel "$PKGROOT"
+dpkg-deb -b "$PKGROOT" "$DEB"
+killall IPhoneMCP 2>/dev/null || true
+dpkg -i "$DEB"
+```
+
+For an update, launch **iPhone MCP** again from its Home Screen icon. Do not run
+`uiopen com.reynafamily.iphonemcp`: this jailbreak's `uiopen` command expects a
+URL, not an application bundle identifier.
+
+Only run the following after the first installation if the icon does not appear:
+
+```sh
+uicache -p /Applications/IPhoneMCP.app
+```
+
+Stop the app before running it and let `uicache` exit normally. Interrupting a
+refresh can leave the application database busy and make app launches appear to
+crash. If that happens, recover it over SSH with `killall -9 uicache`, then tap
+the app icon again.
+
+Verify listeners:
+
+```sh
+netstat -an | grep -E '\.8080|\.8081|\.8082|\.8083|\.5000'
+```
+
+## MCP Bridge
+
+```sh
+python3 iphone_app/iphone_mcp_bridge.py --ip <iphone-ip>
+```
+
+Example client configuration:
+
+```json
+{
+  "mcpServers": {
+    "iphone6-mcp": {
+      "command": "python3",
+      "args": ["/absolute/path/to/mcp_screen/iphone_app/iphone_mcp_bridge.py", "--ip", "192.168.68.150"]
+    }
+  }
+}
+```
+
+## Monochrome Streaming
+
+- TCP `8081`: consecutive 15,000-byte ESP32 RLCD frame buffers
+- UDP `8082`: fifteen 1002-byte packets per frame
+  - byte 0: frame id
+  - byte 1: chunk index 0-14
+  - bytes 2-1001: 1000-byte payload
+
+The mapping is identical to `test_stream.py` in the repository.
+
+## RGB565 Color Streaming
+
+Color frames use TCP port `8083`. Every frame has a 16-byte big-endian header followed by RGB565 data:
+
+| Offset | Size | Meaning |
+|---|---:|---|
+| 0 | 4 | ASCII `IMCR` |
+| 4 | 1 | Version `1` |
+| 5 | 1 | Format `1` = RGB565 big-endian |
+| 6 | 2 | Width |
+| 8 | 2 | Height |
+| 10 | 4 | Payload length (`width * height * 2`) |
+| 14 | 2 | Flags, currently `0` |
+
+Run the included example:
+
+```sh
+python3 -m pip install Pillow
+python3 iphone_app/tools/stream_color.py --ip <iphone-ip>
+```
+
+## Hermes Voice
+
+The `hermes_voice_turn` MCP tool records a mono 16 kHz, 16-bit WAV, posts it to Hermes, plays the returned WAV, and returns the transcript and answer headers.
+
+Default gateway:
+
+```text
+http://192.168.68.126:8642/api/esp32/voice
+```
+
+Example JSON-RPC call:
+
+```json
+{
+  "jsonrpc": "2.0",
+  "id": 1,
+  "method": "tools/call",
+  "params": {
+    "name": "hermes_voice_turn",
+    "arguments": {
+      "url": "http://192.168.68.126:8642/api/esp32/voice",
+      "api_token": "YOUR_HERMES_API_TOKEN",
+      "device_id": "iphone6",
+      "duration_sec": 4,
+      "volume": 80,
+      "reply_mode": "sync",
+      "screen_url": "http://192.168.68.150:8080/api/mcp"
+    }
+  }
+}
+```
+
+Do not commit the Hermes API token. It is accepted per call and is not persisted by the app.
+
+## Implemented Tools
+
+`clear_screen`, `draw_text`, `draw_image`, `get_screenshot`, `get_battery`, `play_tone`, `play_audio`, `play_audio_base64`, `record_voice`, `hermes_voice_turn`, `capture_selfie`, `write_file`, `read_file`, `download_file`, `get_video_streaming_instructions`, `get_stream_stats`, `reset_stream_stats`, and `set_stream_debug`.
+
+Compatibility stubs remain for `set_led`, `get_sensors`, `scan_ble`, and `execute_python`.
+
+## Operational Notes
+
+- Keep the app foregrounded for reliable network service on iOS 12.
+- The idle timer is disabled only while the app is active.
+- Give each device a unique static DHCP reservation; duplicate IPs can make traffic reach the wrong host.

iphone_app/Resources/Info.plist

@@ -0,0 +1,57 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
+  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>CFBundleDisplayName</key>
+    <string>iPhone MCP</string>
+    <key>CFBundleExecutable</key>
+    <string>IPhoneMCP</string>
+    <key>CFBundleIdentifier</key>
+    <string>com.reynafamily.iphonemcp</string>
+    <key>CFBundleInfoDictionaryVersion</key>
+    <string>6.0</string>
+    <key>CFBundleName</key>
+    <string>IPhoneMCP</string>
+    <key>CFBundlePackageType</key>
+    <string>APPL</string>
+    <key>CFBundleSupportedPlatforms</key>
+    <array>
+        <string>iPhoneOS</string>
+    </array>
+    <key>CFBundleShortVersionString</key>
+    <string>0.0.4</string>
+    <key>CFBundleVersion</key>
+    <string>4</string>
+    <key>MinimumOSVersion</key>
+    <string>12.0</string>
+    <key>LSRequiresIPhoneOS</key>
+    <true/>
+    <key>NSAppTransportSecurity</key>
+    <dict>
+        <key>NSAllowsArbitraryLoads</key>
+        <true/>
+    </dict>
+    <key>NSCameraUsageDescription</key>
+    <string>iPhone MCP uses the camera when an MCP client calls the selfie capture tool.</string>
+    <key>NSMicrophoneUsageDescription</key>
+    <string>iPhone MCP uses the microphone when an MCP client calls the voice recording tool.</string>
+    <key>UIDeviceFamily</key>
+    <array>
+        <integer>1</integer>
+    </array>
+    <key>UIRequiredDeviceCapabilities</key>
+    <array>
+        <string>arm64</string>
+    </array>
+    <key>UIStatusBarHidden</key>
+    <true/>
+    <key>UIViewControllerBasedStatusBarAppearance</key>
+    <true/>
+    <key>UISupportedInterfaceOrientations</key>
+    <array>
+        <string>UIInterfaceOrientationLandscapeLeft</string>
+        <string>UIInterfaceOrientationLandscapeRight</string>
+    </array>
+</dict>
+</plist>

iphone_app/control

@@ -0,0 +1,8 @@
+Package: com.reynafamily.iphonemcp
+Name: iPhone MCP
+Version: 0.0.4
+Architecture: iphoneos-arm
+Description: A native UIKit MCP-style HTTP server for a jailbroken iPhone 6.
+Maintainer: Adolfo Reyna
+Author: Adolfo Reyna
+Section: Utilities

iphone_app/iphone_mcp_bridge.py

@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+import argparse
+import json
+import sys
+import urllib.request
+import urllib.error
+
+
+def main():
+    parser = argparse.ArgumentParser(description="MCP stdio bridge for the jailbroken iPhone MCP app")
+    parser.add_argument("--ip", required=True, help="iPhone IP address")
+    parser.add_argument("--port", type=int, default=8080, help="iPhone MCP HTTP port")
+    args = parser.parse_args()
+
+    url = f"http://{args.ip}:{args.port}/api/mcp"
+    sys.stderr.write(f"iPhone MCP bridge routing stdio to {url}\n")
+    sys.stderr.flush()
+
+    for line in sys.stdin:
+        try:
+            request = json.loads(line)
+            http_request = urllib.request.Request(
+                url,
+                data=json.dumps(request).encode("utf-8"),
+                headers={"Content-Type": "application/json"},
+                method="POST",
+            )
+            with urllib.request.urlopen(http_request, timeout=15.0) as response:
+                sys.stdout.write(response.read().decode("utf-8") + "\n")
+                sys.stdout.flush()
+        except urllib.error.URLError as exc:
+            write_error(-32000, f"Bridge failed to reach iPhone MCP app: {exc}", request if "request" in locals() else {})
+        except Exception as exc:
+            write_error(-32603, f"Bridge error: {exc}", request if "request" in locals() else {})
+
+
+def write_error(code, message, request):
+    sys.stdout.write(json.dumps({
+        "jsonrpc": "2.0",
+        "error": {"code": code, "message": message},
+        "id": request.get("id"),
+    }) + "\n")
+    sys.stdout.flush()
+
+
+if __name__ == "__main__":
+    main()

iphone_app/main.m

@@ -0,0 +1,9 @@
+#import <UIKit/UIKit.h>
+#import "IMCAppDelegate.h"
+
+int main(int argc, char *argv[]) {
+    @autoreleasepool {
+        return UIApplicationMain(argc, argv, nil,
+                                 NSStringFromClass(IMCAppDelegate.class));
+    }
+}

iphone_app/manual_build_iphone.sh

@@ -0,0 +1,38 @@
+#!/bin/sh
+set -eu
+
+PROJECT_DIR="${1:-/var/mobile/IPhoneMCP}"
+SDK="${SDK:-/var/mobile/theos/sdks/iPhoneOS12.4.sdk}"
+BUILD_DIR="$PROJECT_DIR/.manual_build"
+OBJ_DIR="$BUILD_DIR/obj"
+APP_DIR="$BUILD_DIR/IPhoneMCP.app"
+
+mkdir -p "$OBJ_DIR" "$APP_DIR"
+cp "$PROJECT_DIR/Resources/Info.plist" "$APP_DIR/Info.plist"
+
+CFLAGS="-arch arm64 -isysroot $SDK -miphoneos-version-min=12.0 -fobjc-arc -Wno-deprecated-declarations -I$PROJECT_DIR"
+SOURCES="main.m IMCAppDelegate.m IMCViewController.m IMCCanvasView.m IMCMCPServer.m IMCVideoStreamServer.m"
+OBJECTS=""
+
+for src in $SOURCES; do
+    obj="$OBJ_DIR/${src%.m}.o"
+    echo "Compiling $src"
+    clang $CFLAGS -c "$PROJECT_DIR/$src" -o "$obj"
+    OBJECTS="$OBJECTS $obj"
+done
+
+echo "Linking IPhoneMCP"
+clang -arch arm64 \
+    -isysroot "$SDK" \
+    -miphoneos-version-min=12.0 \
+    -o "$APP_DIR/IPhoneMCP" \
+    $OBJECTS \
+    -framework UIKit \
+    -framework Foundation \
+    -framework AVFoundation \
+    -framework CoreGraphics
+
+echo "Signing IPhoneMCP"
+ldid -S "$APP_DIR/IPhoneMCP"
+
+echo "$APP_DIR"

iphone_app/tools/mock_hermes_gateway.py

@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+"""Minimal Hermes-compatible voice endpoint for iPhone integration testing."""
+
+import argparse
+import io
+import math
+import struct
+import wave
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+
+def response_wav() -> bytes:
+    output = io.BytesIO()
+    with wave.open(output, "wb") as wav:
+        wav.setnchannels(1)
+        wav.setsampwidth(2)
+        wav.setframerate(16000)
+        frames = bytearray()
+        for index in range(int(16000 * 0.3)):
+            sample = int(7000 * math.sin(2 * math.pi * 660 * index / 16000))
+            frames.extend(struct.pack("<h", sample))
+        wav.writeframes(frames)
+    return output.getvalue()
+
+
+class Handler(BaseHTTPRequestHandler):
+    def do_POST(self):
+        if self.path != "/api/esp32/voice":
+            self.send_error(404)
+            return
+        length = int(self.headers.get("Content-Length", "0"))
+        received = self.rfile.read(length)
+        print(f"received {len(received)} bytes from {self.headers.get('X-Device-ID', 'unknown')}", flush=True)
+        payload = response_wav()
+        self.send_response(200)
+        self.send_header("Content-Type", "audio/wav")
+        self.send_header("Content-Length", str(len(payload)))
+        self.send_header("X-Hermes-Transcript", "Mock voice request received")
+        self.send_header("X-Hermes-Text-Response", "The iPhone Hermes integration is working")
+        self.send_header("X-Hermes-Response-Id", "mock-response")
+        self.send_header("X-Hermes-Conversation", "iphone-test")
+        self.end_headers()
+        self.wfile.write(payload)
+
+    def log_message(self, message, *args):
+        print(message % args, flush=True)
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--host", default="0.0.0.0")
+    parser.add_argument("--port", type=int, default=8765)
+    args = parser.parse_args()
+    server = ThreadingHTTPServer((args.host, args.port), Handler)
+    print(f"mock Hermes gateway listening on {args.host}:{args.port}", flush=True)
+    server.serve_forever()
+
+
+if __name__ == "__main__":
+    main()

iphone_app/tools/stream_color.py

@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+"""Stream a generated RGB565 color animation to iPhone MCP TCP port 8083."""
+
+import argparse
+import socket
+import struct
+import time
+
+from PIL import Image, ImageDraw
+
+
+def rgb565be(image: Image.Image) -> bytes:
+    image = image.convert("RGB")
+    output = bytearray(image.width * image.height * 2)
+    for index, (red, green, blue) in enumerate(image.getdata()):
+        value = ((red >> 3) << 11) | ((green >> 2) << 5) | (blue >> 3)
+        output[index * 2] = value >> 8
+        output[index * 2 + 1] = value & 0xFF
+    return bytes(output)
+
+
+def send_frame_tcp(sock: socket.socket, image: Image.Image) -> None:
+    payload = rgb565be(image)
+    header = struct.pack(">4sBBHHIH", b"IMCR", 1, 1, image.width, image.height, len(payload), 0)
+    sock.sendall(header + payload)
+
+
+def sleep_us(duration_us: int) -> None:
+    target = time.perf_counter_ns() + duration_us * 1000
+    while time.perf_counter_ns() < target:
+        pass
+
+
+def send_frame_udp(sock: socket.socket, ip: str, port: int, frame_idx: int, image: Image.Image, pacing_us: int) -> None:
+    payload = rgb565be(image)
+    frame_id = frame_idx % 256
+    for chunk_idx in range(154):
+        packet = bytearray(1002)
+        packet[0] = frame_id
+        packet[1] = chunk_idx
+        start = chunk_idx * 1000
+        packet[2:1002] = payload[start : start + 1000]
+        sock.sendto(packet, (ip, port))
+        if pacing_us > 0:
+            sleep_us(pacing_us)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--ip", required=True, help="iPhone Wi-Fi IP")
+    parser.add_argument("--protocol", choices=["tcp", "udp"], default="tcp")
+    parser.add_argument("--port", type=int)
+    parser.add_argument("--width", type=int, default=320)
+    parser.add_argument("--height", type=int, default=240)
+    parser.add_argument("--pacing-us", type=int, default=1000, help="Microseconds pacing between UDP chunks")
+    args = parser.parse_args()
+
+    port = args.port
+    if port is None:
+        port = 8083 if args.protocol == "tcp" else 8084
+
+    if args.protocol == "tcp":
+        sock = socket.create_connection((args.ip, port), timeout=8)
+        sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+    else:
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+
+    try:
+        frame = 0
+        while True:
+            hue = (frame * 4) % 256
+            image = Image.new("RGB", (args.width, args.height), (12, 18, 35))
+            draw = ImageDraw.Draw(image)
+            for x in range(args.width):
+                red = (x * 255 // max(1, args.width - 1) + hue) % 256
+                blue = 255 - (x * 255 // max(1, args.width - 1))
+                draw.line((x, 0, x, args.height - 1), fill=(red, 70, blue))
+            draw.rounded_rectangle((20, 20, args.width - 20, args.height - 20), radius=22,
+                                   outline=(255, 255, 255), width=4)
+            draw.text((40, 45), f"iPhone MCP RGB565 ({args.protocol.upper()})", fill=(255, 255, 255))
+            draw.text((40, 72), f"Frame {frame}", fill=(255, 240, 80))
+            
+            if args.protocol == "tcp":
+                send_frame_tcp(sock, image)
+            else:
+                send_frame_udp(sock, args.ip, port, frame, image, args.pacing_us)
+                
+            frame += 1
+            time.sleep(1 / 15)
+    finally:
+        sock.close()
+
+
+if __name__ == "__main__":
+    main()

lib/audio_util.py

@@ -1,6 +1,7 @@
 import time
 import machine
 from machine import Pin, I2C, I2S
+import board_config
 
 class ES7210:
     """MicroPython driver for the ES7210 4-Channel Audio ADC (Microphone Array).
@@ -12,6 +13,12 @@ class ES7210:
     def __init__(self, i2c):
         self.i2c = i2c
         
+    def _write(self, reg, val):
+        self.i2c.writeto_mem(self.ADDR, reg, bytes([val]))
+        
+    def _read(self, reg):
+        return self.i2c.readfrom_mem(self.ADDR, reg, 1)[0]
+        
     def init(self, sample_rate=16000, bit_width=16):
         """Initializes the ES7210 registers for dual-microphone recording.
         
@@ -22,45 +29,85 @@ class ES7210:
         Returns:
             bool: True if initialization was successful, False otherwise.
         """
-        print("Initializing ES7210 Microphone ADC...")
+        print("Initializing ES7210 Microphone ADC (ESPHome sequence)...")
         try:
-            # 1. Reset the chip
-            self._write(0x00, 0xFF) # Write all 1s to reset register
-            time.sleep_ms(10)
-            self._write(0x00, 0x00) # Release reset
+            # 1. Software reset
+            self._write(0x00, 0xFF)
+            time.sleep_ms(20)
+            self._write(0x00, 0x32)
+            time.sleep_ms(20)
+            self._write(0x01, 0x3F) # Clock off during config
             
-            # 2. Power management and system configuration
-            self._write(0x01, 0x00) # Enable analog power, reference voltage
-            self._write(0x11, 0x60) # Enable master clock PLL
+            # 2. Timing control
+            self._write(0x09, 0x30)
+            self._write(0x0A, 0x30)
             
-            # 3. Configure Clock Dividers
-            if sample_rate == 16000:
-                self._write(0x02, 0x0C) # BCLK divider
-                self._write(0x03, 0x10) # LRCK divider
-            else: # 44100 / 48000 defaults
-                self._write(0x02, 0x04)
-                self._write(0x03, 0x08)
+            # 3. High-pass filter
+            self._write(0x23, 0x2A)
+            self._write(0x22, 0x0A)
+            self._write(0x20, 0x0A)
+            self._write(0x21, 0x2A)
             
-            # 4. Input Configuration (Enable Mics 1 and 2, power down Mics 3 and 4)
-            self._write(0x47, 0x00) # Enable MIC1 / MIC2 analog front-ends
-            self._write(0x48, 0xFF) # Power down MIC3 / MIC4 path
-            self._write(0x49, 0x0A) # Power up PGA (Programmable Gain Amplifier) 1 and 2
-            self._write(0x4A, 0x00) # Power down PGA 3 and 4
+            # 4. Mode config: clear bit 0 of Reg 0x08
+            val08 = self._read(0x08)
+            self._write(0x08, val08 & ~0x01)
             
-            # 5. Microphone Gain Settings (+24dB standard)
-            # Gain range: 0x00 (0dB) to 0x0F (+45dB) in 3dB steps. 0x08 = +24dB.
-            self._write(0x43, 0x08) # Set MIC1 Gain (+24dB)
-            self._write(0x44, 0x08) # Set MIC2 Gain (+24dB)
+            # 5. Configure analog power
+            self._write(0x40, 0xC3)
             
-            # 6. Set Digital Interface Format (I2S standard format)
-            # Bit width: 0x00 = 24-bit, 0x01 = 16-bit, 0x02 = 8-bit, 0x03 = 32-bit
-            fmt = 0x01 if bit_width == 16 else 0x00
-            self._write(0x13, fmt)  # Set serial output interface format
-            self._write(0x14, 0x18) # Enable frame clock / bit clock output
+            # 6. Mic bias voltage
+            self._write(0x41, 0x70)
+            self._write(0x42, 0x70)
             
-            # 7. Unmute ADCs and enable output
-            self._write(0x12, 0x00) # Enable ADC digital filters (unmute)
-            self._write(0x15, 0x30) # Enable output data pin (SDOUT) active
+            # 7. Configure I2S format (16-bit, standard I2S, TDM disabled)
+            self._write(0x11, 0x60)
+            self._write(0x12, 0x00)
+            
+            # 8. Configure sample rate (16kHz with 12.288MHz MCLK)
+            # adc_div = 0x03, dll = 0x01, doubler = 0x01, osr = 0x20, lrck_h = 0x03, lrck_l = 0x00
+            reg02_val = 0x03 | (1 << 6) | (1 << 7) # 0xC3
+            self._write(0x02, reg02_val)
+            self._write(0x07, 0x20)
+            self._write(0x04, 0x03)
+            self._write(0x05, 0x00)
+            
+            # 9. Clear select bits for MIC gain registers
+            for i in range(4):
+                val_gain = self._read(0x43 + i)
+                self._write(0x43 + i, val_gain & ~0x10)
+                
+            # 10. Power down all MIC bias & PGA initially
+            self._write(0x4B, 0xFF)
+            self._write(0x4C, 0xFF)
+            
+            # 11. Configure MIC1 and MIC2 (gain = 30dB -> 0x0A, enable SELMIC)
+            gain_reg_val = 0x0A
+            # Enable ADC12 clocks
+            val01 = self._read(0x01)
+            self._write(0x01, val01 & ~0x0B)
+            # Power on MIC1/2 bias, ADC, PGA
+            self._write(0x4B, 0x00)
+            # Select MIC1 and gain
+            val43 = self._read(0x43)
+            self._write(0x43, (val43 & ~0x0F) | 0x10 | gain_reg_val)
+            # Select MIC2 and gain
+            val44 = self._read(0x44)
+            self._write(0x44, (val44 & ~0x0F) | 0x10 | gain_reg_val)
+            
+            # 12. Power on mics low power registers
+            self._write(0x47, 0x08)
+            self._write(0x48, 0x08)
+            self._write(0x49, 0x08)
+            self._write(0x4A, 0x08)
+            
+            # 13. Power down DLL
+            self._write(0x06, 0x04)
+            
+            # 14. Enable device state machine
+            self._write(0x00, 0x71)
+            time.sleep_ms(20)
+            self._write(0x00, 0x41)
+            time.sleep_ms(100)
             
             print("ES7210 initialization complete.")
             return True
@@ -68,42 +115,55 @@ class ES7210:
             print(f"Failed to initialize ES7210: {e}")
             return False
 
-    def _write(self, reg, val):
-        self.i2c.writeto_mem(self.ADDR, reg, bytes([val]))
 
-
-def record_audio(duration_seconds=5, filename='recording.pcm'):
+def record_audio(duration_seconds=10, filename='recording.pcm'):
     """Records raw stereo PCM data from the dual microphones to a file.
     
     Args:
         duration_seconds (int): How long to record in seconds.
         filename (str): Name of output raw PCM file on the device.
     """
-    # 1. Start I2C Control Bus (SDA=13, SCL=14)
-    i2c = I2C(0, sda=Pin(13), scl=Pin(14))
+    # 1. Use I2C Control Bus from board_config
+    i2c = board_config.i2c_bus
+    
+    # 1a. Setup Master Clock (MCLK) using PWM if configured
+    mclk_pwm = None
+    if board_config.audio_mclk_pin is not None:
+        mclk_pin = Pin(board_config.audio_mclk_pin, Pin.OUT)
+        mclk_pwm = machine.PWM(mclk_pin)
+        mclk_pwm.freq(board_config.audio_mclk_freq)
+        mclk_pwm.duty_u16(32768)
     
     # 2. Configure I2S Receiver
-    # Pins: sck=BCLK (GPIO 9), ws=WS/LRCK (GPIO 45), sd=DIN (GPIO 10)
     i2s = I2S(1, 
-              sck=Pin(9), 
-              ws=Pin(45), 
-              sd=Pin(10), 
+              sck=Pin(board_config.audio_i2s_sck), 
+              ws=Pin(board_config.audio_i2s_ws), 
+              sd=Pin(board_config.audio_i2s_rx_sd), 
               mode=I2S.RX, 
               ibuf=16000, 
               rate=16000, 
               bits=16, 
               format=I2S.STEREO)
               
-    # 3. Wake up and configure the ES7210 microphone chip
+    # 3. Wake up and configure the microphone chip (ES7210 vs ES8311)
+    if board_config.audio_mic_codec == "ES7210":
         mic_adc = ES7210(i2c)
         if not mic_adc.init(sample_rate=16000, bit_width=16):
             i2s.deinit()
+            if mclk_pwm: mclk_pwm.deinit()
+            return False
+    else:
+        mic_adc = ES8311(i2c)
+        if not mic_adc.init(sample_rate=16000):
+            i2s.deinit()
+            if mclk_pwm: mclk_pwm.deinit()
             return False
         
     print(f"Recording {duration_seconds} seconds of audio...")
     
     # Create reading buffer (reads 100ms chunks: 16000 samples/sec * 2 channels * 2 bytes/sample * 0.1s = 6400 bytes)
     buffer = bytearray(6400)
+    mono_buf = bytearray(3200)  # Half size for mono extraction
     
     start_time = time.time()
     total_bytes = 0
@@ -114,8 +174,15 @@ def record_audio(duration_seconds=5, filename='recording.pcm'):
                 # Read raw stereo PCM data from I2S
                 bytes_read = i2s.readinto(buffer)
                 if bytes_read > 0:
-                    f.write(buffer[:bytes_read])
-                    total_bytes += bytes_read
+                    # Stereo-to-mono: extract left channel (every other 16-bit sample)
+                    mono_len = bytes_read // 2
+                    j = 0
+                    for i in range(0, bytes_read, 4):
+                        mono_buf[j] = buffer[i]
+                        mono_buf[j + 1] = buffer[i + 1]
+                        j += 2
+                    f.write(mono_buf[:mono_len])
+                    total_bytes += mono_len
                     
         print(f"Recording saved successfully to '{filename}' ({total_bytes} bytes).")
         return True
@@ -124,8 +191,16 @@ def record_audio(duration_seconds=5, filename='recording.pcm'):
         return False
     finally:
         # Always release the I2S peripheral resources
+        try:
             i2s.deinit()
-        print("I2S receiver deinitialized.")
+        except:
+            pass
+        if mclk_pwm:
+            try:
+                mclk_pwm.deinit()
+            except:
+                pass
+        print("I2S receiver and MCLK deinitialized.")
 
 
 class ES8311:
@@ -155,11 +230,11 @@ class ES8311:
             self._write(0x00, 0x00)
             time.sleep_ms(10)
             
-            # Clock Configuration (16kHz sample rate, MCLK=12.288MHz)
+            # Clock Configuration (16kHz sample rate, MCLK=6.144MHz)
             self._write(0x01, 0x3F) # Enable all clocks, use MCLK pin
             self._write(0x02, 0x48) # pre_div=3, pre_mult=1
             self._write(0x03, 0x10) # fs_mode=0, adc_osr=16
-            self._write(0x04, 0x20) # dac_osr=32
+            self._write(0x04, 0x10) # dac_osr=16
             self._write(0x05, 0x00) # adc_div=1, dac_div=1
             self._write(0x06, 0x03) # bclk_div=4 (4-1=3)
             self._write(0x07, 0x00) # lrck_h=0
@@ -219,37 +294,40 @@ def play_tone(frequency=440, duration_ms=1000, volume=50):
     
     print(f"Playing tone: {frequency}Hz for {duration_ms}ms (vol={volume})...")
     
-    # 1. Setup Master Clock (MCLK) on GPIO 16 using PWM
-    mclk_pin = Pin(16, Pin.OUT)
+    # 1. Setup Master Clock (MCLK) using PWM if configured
+    mclk_pwm = None
+    if board_config.audio_mclk_pin is not None:
+        mclk_pin = Pin(board_config.audio_mclk_pin, Pin.OUT)
         mclk_pwm = machine.PWM(mclk_pin)
-    mclk_pwm.freq(12288000)
+        mclk_pwm.freq(board_config.audio_mclk_freq)
         mclk_pwm.duty_u16(32768)
     
-    # 2. Start I2C Control Bus (SDA=13, SCL=14)
-    i2c = I2C(0, sda=Pin(13), scl=Pin(14))
+    # 2. Use dynamic I2C Control Bus from board_config
+    i2c = board_config.i2c_bus
     
     # 3. Initialize the ES8311 DAC
     dac = ES8311(i2c)
     if not dac.init(sample_rate=16000):
-        mclk_pwm.deinit()
+        if mclk_pwm: mclk_pwm.deinit()
         return False
         
     dac.set_volume(volume)
     
     # 4. Configure I2S TX
-    # Pins: sck=BCLK (GPIO 9), ws=WS/LRCK (GPIO 45), sd=DOUT (GPIO 8)
     i2s = I2S(1,
-              sck=Pin(9),
-              ws=Pin(45),
-              sd=Pin(8),
+              sck=Pin(board_config.audio_i2s_sck),
+              ws=Pin(board_config.audio_i2s_ws),
+              sd=Pin(board_config.audio_i2s_tx_sd),
               mode=I2S.TX,
               ibuf=8000,
               rate=16000,
               bits=16,
               format=I2S.STEREO)
               
-    # 5. Enable Speaker Amplifier (GPIO 46)
-    amp_pin = Pin(46, Pin.OUT, value=1)
+    # 5. Enable Speaker Amplifier
+    on_val = 0 if board_config.audio_amp_active_level == 0 else 1
+    off_val = 1 if board_config.audio_amp_active_level == 0 else 0
+    amp_pin = Pin(board_config.audio_amp_pin, Pin.OUT, value=on_val)
     
     # 6. Generate sine wave cycle
     # Approximate frequency to make integer number of samples per cycle
@@ -277,9 +355,9 @@ def play_tone(frequency=440, duration_ms=1000, volume=50):
         
     # 7. Clean up
     time.sleep_ms(100) # Let the remaining buffer play out
-    amp_pin.value(0)
+    amp_pin.value(off_val) # Disable amp
     i2s.deinit()
-    mclk_pwm.deinit()
+    if mclk_pwm: mclk_pwm.deinit()
     print("Tone playback complete.")
     return True
 
@@ -337,39 +415,42 @@ def play_wav(filename, volume=50):
                 
         print(f"WAV Info: {sample_rate}Hz, {bits} bits, {'Mono' if channels == 1 else 'Stereo'}")
         
-        # 2. Setup Master Clock (MCLK) on GPIO 16 using PWM
-        mclk_pin = Pin(16, Pin.OUT)
+        # 2. Setup Master Clock (MCLK) using PWM if configured
+        mclk_pwm = None
+        if board_config.audio_mclk_pin is not None:
+            mclk_pin = Pin(board_config.audio_mclk_pin, Pin.OUT)
             mclk_pwm = machine.PWM(mclk_pin)
-        mclk_pwm.freq(12288000)
+            mclk_pwm.freq(board_config.audio_mclk_freq)
             mclk_pwm.duty_u16(32768)
         
-        # 3. Start I2C Control Bus (SDA=13, SCL=14)
-        i2c = I2C(0, sda=Pin(13), scl=Pin(14))
+        # 3. Use dynamic I2C Control Bus from board_config
+        i2c = board_config.i2c_bus
         
         # 4. Initialize the ES8311 DAC
         dac = ES8311(i2c)
         if not dac.init(sample_rate=16000):
-            mclk_pwm.deinit()
+            if mclk_pwm: mclk_pwm.deinit()
             f.close()
             return False
             
         dac.set_volume(volume)
         
         # 5. Configure I2S TX
-        # Pins: sck=BCLK (GPIO 9), ws=WS/LRCK (GPIO 45), sd=DOUT (GPIO 8)
         i2s_format = I2S.MONO if channels == 1 else I2S.STEREO
         i2s = I2S(1,
-                  sck=Pin(9),
-                  ws=Pin(45),
-                  sd=Pin(8),
+                  sck=Pin(board_config.audio_i2s_sck),
+                  ws=Pin(board_config.audio_i2s_ws),
+                  sd=Pin(board_config.audio_i2s_tx_sd),
                   mode=I2S.TX,
                   ibuf=4096,
                   rate=sample_rate,
                   bits=bits,
                   format=i2s_format)
                   
-        # 6. Enable Speaker Amplifier (GPIO 46)
-        amp_pin = Pin(46, Pin.OUT, value=1)
+        # 6. Enable Speaker Amplifier
+        on_val = 0 if board_config.audio_amp_active_level == 0 else 1
+        off_val = 1 if board_config.audio_amp_active_level == 0 else 0
+        amp_pin = Pin(board_config.audio_amp_pin, Pin.OUT, value=on_val)
         
         # 7. Read and stream chunks to I2S
         buf = bytearray(2048)
@@ -381,9 +462,9 @@ def play_wav(filename, volume=50):
             
         # 8. Clean up
         time.sleep_ms(100) # Let the remaining buffer play out
-        amp_pin.value(0)
+        amp_pin.value(off_val) # Disable amp
         i2s.deinit()
-        mclk_pwm.deinit()
+        if mclk_pwm: mclk_pwm.deinit()
         f.close()
         print("WAV playback complete.")
         return True

lib/battery_util.py

@@ -3,7 +3,7 @@ from machine import ADC, Pin
 class BatteryMonitor:
     """Utility class for monitoring battery voltage and capacity on ESP32-S3-RLCD-4.2."""
     
-    def __init__(self, pin_num=4):
+    def __init__(self, pin_num=9):
         import sys
         self.adc = ADC(Pin(pin_num))
         if sys.platform == 'esp32':
@@ -18,16 +18,16 @@ class BatteryMonitor:
         try:
             # Try calibrated reading in microvolts first
             uv = self.adc.read_uv()
-            # 3x voltage divider onboard scales 3.0V-4.2V battery to 1.0V-1.4V
-            voltage = (uv / 1_000_000.0) * 3.0
+            # 2x voltage divider onboard scales 3.0V-4.2V battery to 1.5V-2.1V
+            voltage = (uv / 1_000_000.0) * 2.0
             return round(voltage, 3)
         except AttributeError:
             # Fallback if read_uv() is not supported on older MicroPython builds
             try:
                 # Read 12-bit value (0-4095)
                 raw = self.adc.read()
-                # 3.3V reference at 11dB attenuation, 3x divider
-                voltage = (raw / 4095.0) * 3.3 * 3.0
+                # 3.3V reference at 11dB attenuation, 2x divider
+                voltage = (raw / 4095.0) * 3.3 * 2.0
                 return round(voltage, 3)
             except Exception as e:
                 print(f"Error reading battery raw ADC: {e}")

lib/board_config.py

@@ -0,0 +1,218 @@
+import sys
+import machine
+from machine import Pin, I2C, SPI
+
+# Global configuration variables
+BOARD_TYPE = None      # 'HOSYOND', 'WAVESHARE_RLCD', or 'RP2'
+DISPLAY_TYPE = None    # 'ILI9341', 'RLCD', or 'ST7796'
+DISPLAY_WIDTH = 320
+DISPLAY_HEIGHT = 240
+
+# Bus and display references
+spi_bus = None
+i2c_bus = None
+display_instance = None
+touch = None
+
+# Component support flags
+has_sensor = False
+has_rtc = False
+
+# WS2812 NeoPixel pin
+led_pin = None
+
+# Audio pins, clocks, and codecs config
+audio_mclk_pin = None
+audio_mclk_freq = 6144000
+audio_i2c_sda = None
+audio_i2c_scl = None
+audio_i2s_sck = None
+audio_i2s_ws = None
+audio_i2s_tx_sd = None
+audio_i2s_rx_sd = None
+audio_amp_pin = None
+audio_amp_active_level = 0  # 0 = Active Low, 1 = Active High
+audio_mic_codec = "ES8311"  # "ES7210" or "ES8311"
+
+def _i2c_recovery(sda_pin, scl_pin):
+    import time
+    scl = Pin(scl_pin, Pin.OUT)
+    sda = Pin(sda_pin, Pin.OUT)
+    scl.value(1)
+    sda.value(1)
+    time.sleep_ms(1)
+    for _ in range(9):
+        scl.value(0)
+        time.sleep_ms(1)
+        scl.value(1)
+        time.sleep_ms(1)
+    scl.value(0)
+    sda.value(0)
+    time.sleep_ms(1)
+    scl.value(1)
+    time.sleep_ms(1)
+    sda.value(1)
+    time.sleep_ms(1)
+
+def detect_board():
+    global BOARD_TYPE, DISPLAY_TYPE, DISPLAY_WIDTH, DISPLAY_HEIGHT
+    global spi_bus, i2c_bus, display_instance, touch
+    global has_sensor, has_rtc, led_pin
+    global audio_mclk_pin, audio_mclk_freq, audio_i2c_sda, audio_i2c_scl
+    global audio_i2s_sck, audio_i2s_ws, audio_i2s_tx_sd, audio_i2s_rx_sd
+    global audio_amp_pin, audio_amp_active_level, audio_mic_codec
+    
+    # 1. Check if running on RP2 platform (e.g. Raspberry Pi Pico / RP2350)
+    if sys.platform == 'rp2':
+        print("Auto-detected: Raspberry Pi RP2 platform")
+        BOARD_TYPE = 'RP2'
+        DISPLAY_TYPE = 'ST7796'
+        DISPLAY_WIDTH = 480
+        DISPLAY_HEIGHT = 320
+        led_pin = 25
+        
+        # Setup SPI
+        spi_bus = SPI(1, baudrate=32000000, polarity=0, phase=0, sck=Pin(10), mosi=Pin(11))
+        
+        # Setup Display: ST7796
+        import st7796
+        display_instance = st7796.ST7796(spi_bus, cs=Pin(7), dc=Pin(4), rst=Pin(9), bl=Pin(6))
+        
+        # Setup Touch: FT6336U on I2C(1)
+        try:
+            _i2c_recovery(2, 3)
+            touch_i2c = I2C(1, sda=Pin(2), scl=Pin(3), freq=400000)
+            from ft6336u import FT6336U
+            touch = FT6336U(touch_i2c, rst_pin=28, int_pin=25)
+        except Exception as e:
+            print("Failed to initialize touch on RP2:", e)
+        return
+
+    # 2. We are on ESP32 platform.
+    # Use SoftI2C for dynamic detection scan to avoid hardware I2C pin conflicts
+    from machine import SoftI2C
+    
+    # Try scanning SDA=16, SCL=15 (Hosyond pins)
+    try:
+        _i2c_recovery(16, 15)
+        test_i2c = SoftI2C(sda=Pin(16), scl=Pin(15))
+        devices = test_i2c.scan()
+        if 0x38 in devices:
+            print("Auto-detected: Hosyond ESP32-S3 Touchscreen board")
+            BOARD_TYPE = 'HOSYOND'
+            DISPLAY_TYPE = 'ILI9341'
+            DISPLAY_WIDTH = 320
+            DISPLAY_HEIGHT = 240
+            has_sensor = False
+            has_rtc = False
+            led_pin = 48
+            
+            # Now initialize SoftI2C
+            i2c_bus = SoftI2C(sda=Pin(16), scl=Pin(15))
+            
+            # Setup SPI
+            spi_bus = SPI(1, baudrate=40000000, polarity=0, phase=0, sck=Pin(12), mosi=Pin(11), miso=Pin(13))
+            
+            # Setup Display: ILI9341
+            import ili9341
+            display_instance = ili9341.ILI9341(spi_bus, cs=Pin(10), dc=Pin(46), bl=Pin(45), rst=None)
+            
+            # Setup Touch: FT6336U
+            try:
+                from ft6336u import FT6336U
+                touch = FT6336U(i2c_bus, rst_pin=18, int_pin=17, width=320, height=240, swap_xy=True, invert_x=False, invert_y=True)
+            except Exception as te:
+                print("Failed to initialize touchscreen:", te)
+                
+            # Audio pins & clocks configuration
+            audio_mclk_pin = 4
+            audio_mclk_freq = 6144000
+            audio_i2c_sda = 16
+            audio_i2c_scl = 15
+            audio_i2s_sck = 5
+            audio_i2s_ws = 7
+            audio_i2s_tx_sd = 8
+            audio_i2s_rx_sd = 6
+            audio_amp_pin = 1
+            audio_amp_active_level = 0 # Active Low (0 = enabled)
+            audio_mic_codec = "ES8311"
+            return
+            
+    except Exception as e:
+        print("SoftI2C scan on Hosyond pins failed:", e)
+
+    # 3. Try scanning SDA=13, SCL=14 (Waveshare RLCD pins)
+    try:
+        _i2c_recovery(13, 14)
+        test_i2c = SoftI2C(sda=Pin(13), scl=Pin(14))
+        devices = test_i2c.scan()
+        if 0x70 in devices or 0x51 in devices:
+            print("Auto-detected: Waveshare ESP32-S3-RLCD-4.2 board")
+            BOARD_TYPE = 'WAVESHARE_RLCD'
+            DISPLAY_TYPE = 'RLCD'
+            DISPLAY_WIDTH = 400
+            DISPLAY_HEIGHT = 300
+            has_sensor = True
+            has_rtc = True
+            led_pin = 38
+            
+            # Setup SPI
+            spi_bus = SPI(1, baudrate=20000000, polarity=0, phase=0, sck=Pin(11), mosi=Pin(12))
+            
+            # Setup Display: RLCD
+            import rlcd
+            display_instance = rlcd.RLCD(spi_bus, cs=Pin(40), dc=Pin(5), rst=Pin(41))
+            
+            # Now initialize SoftI2C (after SPI/Display setup to avoid MISO pin conflict on Pin 13)
+            i2c_bus = SoftI2C(sda=Pin(13), scl=Pin(14))
+            
+            # Audio pins & clocks configuration
+            audio_mclk_pin = 16
+            audio_mclk_freq = 12288000
+            audio_i2c_sda = 13
+            audio_i2c_scl = 14
+            audio_i2s_sck = 9
+            audio_i2s_ws = 45
+            audio_i2s_tx_sd = 8
+            audio_i2s_rx_sd = 10
+            audio_amp_pin = 46
+            audio_amp_active_level = 1 # Active High (1 = enabled)
+            audio_mic_codec = "ES7210"
+            return
+            
+    except Exception as e:
+        print("SoftI2C scan on Waveshare RLCD pins failed:", e)
+
+    # 4. Fallback default if auto-detection failed completely
+    print("Auto-detection failed. Falling back to Hosyond ESP32-S3 defaults...")
+    BOARD_TYPE = 'HOSYOND'
+    DISPLAY_TYPE = 'ILI9341'
+    DISPLAY_WIDTH = 320
+    DISPLAY_HEIGHT = 240
+    led_pin = 48
+    
+    try:
+        from machine import SoftI2C
+        i2c_bus = SoftI2C(sda=Pin(16), scl=Pin(15))
+        spi_bus = SPI(1, baudrate=40000000, polarity=0, phase=0, sck=Pin(12), mosi=Pin(11), miso=Pin(13))
+        import ili9341
+        display_instance = ili9341.ILI9341(spi_bus, cs=Pin(10), dc=Pin(46), bl=Pin(45), rst=None)
+        from ft6336u import FT6336U
+        touch = FT6336U(i2c_bus, rst_pin=18, int_pin=17, width=320, height=240, swap_xy=True, invert_x=False, invert_y=True)
+    except Exception as e:
+        print("Fallback hardware setup failed:", e)
+        
+    audio_mclk_pin = 4
+    audio_mclk_freq = 6144000
+    audio_i2c_sda = 16
+    audio_i2c_scl = 15
+    audio_i2s_sck = 5
+    audio_i2s_ws = 7
+    audio_i2s_tx_sd = 8
+    audio_i2s_rx_sd = 6
+    audio_amp_pin = 1
+    audio_amp_active_level = 0
+    audio_mic_codec = "ES8311"
+
+# Run dynamic detection immediately on module load
+detect_board()

lib/ft6336u.py

@@ -78,8 +78,18 @@ class FT6336U:
         return True
 
     def is_touched(self):
-        """Returns True if screen is touched (the active-low INT pin is pulled LOW)."""
-        return self.int() == 0
+        """Returns True if screen is touched by checking the TD_STATUS register and clearing coordinates."""
+        if not self.initialized:
+            return False
+        td_status = self.read_reg(self.REG_TD_STATUS)
+        if td_status is None:
+            return False
+        touch_count = td_status[0] & 0x0F
+        if touch_count > 0 and touch_count < 3:
+            # Read P1 coordinates to clear register/interrupt state on the chip
+            self.read_reg(self.REG_P1_XH, 6)
+            return True
+        return False
 
     def read_touch(self):
         """Reads touch point coordinates.

lib/ili9341.py

@@ -0,0 +1,500 @@
+import time
+from machine import Pin, SPI
+import framebuf
+import micropython
+import struct
+
+class ILI9341:
+    def __init__(self, spi, cs, dc, rst=None, bl=None, width=320, height=240, invert_color=True):
+        self.spi = spi
+        self.cs = cs
+        self.dc = dc
+        self.rst = rst
+        self.bl = bl
+        self.width = width
+        self.height = height
+        self.invert_color = invert_color
+        
+        # 1. 1-bit Canvas Buffer (Standard MONO_HLSB for drawing)
+        self.hw_len = (self.width * self.height) // 8
+        self.canvas_buffer = bytearray(self.hw_len)
+        self.canvas = framebuf.FrameBuffer(self.canvas_buffer, self.width, self.height, framebuf.MONO_HLSB)
+        
+        # Pre-allocate chunk buffer for conversion (16 rows: 320 * 16 * 2 = 10240 bytes)
+        self.chunk_rows = 16
+        self.row_buffer = bytearray(self.width * self.chunk_rows * 2)
+        
+        # Initialize pins
+        self.cs.init(self.cs.OUT, value=1)
+        self.dc.init(self.dc.OUT, value=0)
+        
+        if self.rst is not None:
+            self.rst.init(self.rst.OUT, value=1)
+            
+        if self.bl is not None:
+            self.bl.init(self.bl.OUT, value=1)
+                
+        self.reset()
+        self.init_display()
+        self.clear(0)
+        self.show()
+
+    # --- DRAWING WRAPPERS ---
+    def pixel(self, x, y, c): self.canvas.pixel(x, y, c)
+    def line(self, x1, y1, x2, y2, c): self.canvas.line(x1, y1, x2, y2, c)
+    def rect(self, x, y, w, h, c): self.canvas.rect(x, y, w, h, c)
+    def fill_rect(self, x, y, w, h, c): self.canvas.fill_rect(x, y, w, h, c)
+    def text(self, msg, x, y, c=1): self.canvas.text(msg, x, y, c)
+    def clear(self, c=0): self.canvas.fill(c)
+
+    # --- SCALABLE TEXT ---
+    def text_large(self, msg, x, y, scale=2, c=1):
+        char_w = 8; char_h = 8
+        tmp_buf = bytearray(char_w * char_h // 8)
+        tmp_fb = framebuf.FrameBuffer(tmp_buf, char_w, char_h, framebuf.MONO_HLSB)
+        for char in msg:
+            tmp_fb.fill(0); tmp_fb.text(char, 0, 0, 1)
+            for py in range(8):
+                for px in range(8):
+                    if tmp_fb.pixel(px, py):
+                        self.canvas.fill_rect(x + (px * scale), y + (py * scale), scale, scale, c)
+            x += (8 * scale)
+
+    # --- RAW BITMAPS (1:1 scale) ---
+    def bitmap(self, x, y, w, h, pixel_data):
+        img = framebuf.FrameBuffer(pixel_data, w, h, framebuf.MONO_HLSB)
+        self.canvas.blit(img, x, y)
+
+    # --- PBM FILE LOADER WITH SCALING ---
+    def draw_pbm(self, filename, x, y, scale=1):
+        try:
+            with open(filename, 'rb') as f:
+                line1 = f.readline()
+                if not line1.startswith(b'P4'): print("Err: Not P4 PBM"); return
+                while True:
+                    line = f.readline()
+                    if not line.startswith(b'#'): break
+                dims = line.split(); w = int(dims[0]); h = int(dims[1])
+                data = bytearray(f.read())
+                
+                src_fb = framebuf.FrameBuffer(data, w, h, framebuf.MONO_HLSB)
+                
+                if scale == 1:
+                     self.canvas.blit(src_fb, x, y)
+                else:
+                    for sy in range(h):
+                        for sx in range(w):
+                            if src_fb.pixel(sx, sy):
+                                self.canvas.fill_rect(x + (sx * scale), y + (sy * scale), scale, scale, 1)
+                print(f"Loaded {filename} (scale {scale})")
+        except OSError:
+            print(f"Error: Could not open {filename}")
+
+    @micropython.native
+    def _convert_bgr24_to_rgb565(self, bgr_buf, rgb565_buf, width, src_offset, num_pixels):
+        idx = 0
+        for i in range(src_offset, src_offset + num_pixels):
+            b = bgr_buf[i * 3]
+            g = bgr_buf[i * 3 + 1]
+            r = bgr_buf[i * 3 + 2]
+            r_5 = r >> 3
+            g_6 = g >> 2
+            b_5 = b >> 3
+            rgb565_buf[idx] = (r_5 << 3) | (g_6 >> 3)
+            rgb565_buf[idx + 1] = ((g_6 & 0x07) << 5) | b_5
+            idx += 2
+
+    @micropython.native
+    def _convert_bgra32_to_rgb565(self, bgra_buf, rgb565_buf, width, src_offset, num_pixels):
+        idx = 0
+        for i in range(src_offset, src_offset + num_pixels):
+            b = bgra_buf[i * 4]
+            g = bgra_buf[i * 4 + 1]
+            r = bgra_buf[i * 4 + 2]
+            r_5 = r >> 3
+            g_6 = g >> 2
+            b_5 = b >> 3
+            rgb565_buf[idx] = (r_5 << 3) | (g_6 >> 3)
+            rgb565_buf[idx + 1] = ((g_6 & 0x07) << 5) | b_5
+            idx += 2
+
+    def draw_bmp(self, filename, x=0, y=0):
+        """Draw a 24-bit or 32-bit uncompressed color BMP image at (x, y) coordinates."""
+        try:
+            with open(filename, 'rb') as f:
+                header = f.read(54)
+                if len(header) < 54 or header[0:2] != b'BM':
+                    print("Err: Not a valid BMP file")
+                    return False
+                
+                pixel_offset = struct.unpack('<I', header[10:14])[0]
+                width, height = struct.unpack('<ii', header[18:26])
+                planes, bpp = struct.unpack('<HH', header[26:30])
+                compression = struct.unpack('<I', header[30:34])[0]
+                
+                if bpp not in (24, 32):
+                    print("Err: Only 24-bit and 32-bit BMP formats supported")
+                    return False
+                    
+                if compression != 0:
+                    print("Err: Only uncompressed BMP supported")
+                    return False
+                
+                f.seek(pixel_offset)
+                bottom_up = True
+                if height < 0:
+                    height = -height
+                    bottom_up = False
+                
+                row_bytes = (width * bpp) // 8
+                row_padded = ((width * bpp + 31) // 32) * 4
+                
+                read_buf = bytearray(row_padded)
+                rgb565_buf = bytearray(width * 2)
+                
+                for row_idx in range(height):
+                    n = f.readinto(read_buf)
+                    if n < row_padded:
+                        break
+                    
+                    screen_y = y + (height - 1 - row_idx) if bottom_up else y + row_idx
+                    if screen_y < 0 or screen_y >= self.height:
+                        continue
+                        
+                    x_start = x
+                    x_end = x + width - 1
+                    
+                    if x_start >= self.width or x_end < 0:
+                        continue
+                        
+                    win_x0 = max(0, x_start)
+                    win_x1 = min(self.width - 1, x_end)
+                    
+                    if win_x1 < win_x0:
+                        continue
+                        
+                    src_offset_pixels = win_x0 - x_start
+                    win_w = win_x1 - win_x0 + 1
+                    
+                    # Convert pixel data to RGB565 row buffer
+                    if bpp == 24:
+                        self._convert_bgr24_to_rgb565(read_buf, rgb565_buf, width, src_offset_pixels, win_w)
+                    elif bpp == 32:
+                        self._convert_bgra32_to_rgb565(read_buf, rgb565_buf, width, src_offset_pixels, win_w)
+                    
+                    # Draw directly to the screen via SPI window
+                    self.set_window(win_x0, screen_y, win_x1, screen_y)
+                    self.dc(1)
+                    self.cs(0)
+                    self.spi.write(memoryview(rgb565_buf)[:win_w * 2])
+                    self.cs(1)
+                    
+                    # Also update internal 1-bit canvas buffer for screenshots/refresh consistency
+                    for px in range(win_w):
+                        screen_x = win_x0 + px
+                        src_px = src_offset_pixels + px
+                        if bpp == 24:
+                            b = read_buf[src_px * 3]
+                            g = read_buf[src_px * 3 + 1]
+                            r = read_buf[src_px * 3 + 2]
+                        else:
+                            b = read_buf[src_px * 4]
+                            g = read_buf[src_px * 4 + 1]
+                            r = read_buf[src_px * 4 + 2]
+                        # 0 = Black, 1 = White in conversion for MONO_HLSB canvas
+                        lum = (r * 299 + g * 587 + b * 114) // 1000
+                        mono_c = 1 if lum >= 128 else 0
+                        self.canvas.pixel(screen_x, screen_y, mono_c)
+                        
+            return True
+        except Exception as e:
+            print("Error drawing BMP:", e)
+            return False
+
+    @micropython.native
+    def _update_mono_canvas_rgb565(self, x, y, w, h, data):
+        for cy in range(h):
+            screen_y = y + cy
+            if screen_y < 0 or screen_y >= self.height:
+                continue
+            for cx in range(w):
+                screen_x = x + cx
+                if screen_x < 0 or screen_x >= self.width:
+                    continue
+                idx = (cy * w + cx) * 2
+                h_byte = data[idx]
+                l_byte = data[idx + 1]
+                # Extract RGB from RGB565
+                r = (h_byte & 0xF8)
+                g = ((h_byte & 0x07) << 5) | ((l_byte & 0xE0) >> 3)
+                b = (l_byte & 0x1F) << 3
+                # Convert to luminance
+                lum = (r * 299 + g * 587 + b * 114) // 1000
+                mono = 1 if lum >= 128 else 0
+                self.canvas.pixel(screen_x, screen_y, mono)
+
+    def draw_rgb565(self, x, y, w, h, data, sync_canvas=True):
+        """Draw raw RGB565 pixel data on the screen at specified (x,y) with width and height."""
+        # Clip coordinates
+        x_start = max(0, x)
+        x_end = min(self.width - 1, x + w - 1)
+        y_start = max(0, y)
+        y_end = min(self.height - 1, y + h - 1)
+        
+        if x_start > x_end or y_start > y_end:
+            return True
+            
+        # Fast path: if completely visible on screen, draw in one go
+        if x_start == x and x_end == x + w - 1 and y_start == y and y_end == y + h - 1:
+            self.set_window(x_start, y_start, x_end, y_end)
+            self.dc(1)
+            self.cs(0)
+            self.spi.write(data)
+            self.cs(1)
+        else:
+            # Slow path: row-by-row clipping
+            for cy in range(y_start, y_end + 1):
+                src_y = cy - y
+                src_row_offset = (src_y * w + (x_start - x)) * 2
+                row_len_bytes = (x_end - x_start + 1) * 2
+                
+                self.set_window(x_start, cy, x_end, cy)
+                self.dc(1)
+                self.cs(0)
+                self.spi.write(memoryview(data)[src_row_offset : src_row_offset + row_len_bytes])
+                self.cs(1)
+                
+        # Sync the internal 1-bit canvas buffer
+        if sync_canvas:
+            self._update_mono_canvas_rgb565(x, y, w, h, data)
+        return True
+
+    # --- SCREENSHOT ---
+    def save_screenshot(self, filename):
+        print(f"Saving screenshot to {filename}...")
+        try:
+            with open(filename, 'wb') as f:
+                f.write(b'P4\n')
+                f.write(f"{self.width} {self.height}\n".encode())
+                f.write(self.canvas_buffer)
+            print("Saved!")
+        except Exception as e:
+            print(f"Error saving screenshot: {e}")
+
+    # --- HARDWARE LOGIC ---
+    def reset(self):
+        if self.rst is not None:
+            self.rst(1); time.sleep_ms(5); self.rst(0); time.sleep_ms(15); self.rst(1); time.sleep_ms(15)
+
+    def write_cmd(self, cmd):
+        self.dc(0); self.cs(0); self.spi.write(bytearray([cmd])); self.cs(1)
+
+    def write_data(self, data):
+        self.dc(1); self.cs(0)
+        if isinstance(data, int): self.spi.write(bytearray([data]))
+        elif isinstance(data, list): self.spi.write(bytearray(data))
+        else: self.spi.write(data)
+        self.cs(1)
+
+    def init_display(self):
+        # ILI9341 Initialization Sequence
+        self.write_cmd(0x01) # SWRESET
+        time.sleep_ms(150)
+        
+        self.write_cmd(0xCF); self.write_data(b"\x00\xC1\x30") 
+        self.write_cmd(0xED); self.write_data(b"\x64\x03\x12\x81")
+        self.write_cmd(0xE8); self.write_data(b"\x85\x00\x78")
+        self.write_cmd(0xCB); self.write_data(b"\x39\x2C\x00\x34\x02")
+        self.write_cmd(0xF7); self.write_data(b"\x20")
+        self.write_cmd(0xEA); self.write_data(b"\x00\x00")
+        
+        self.write_cmd(0xC0); self.write_data(b"\x13") # Power Control 1
+        self.write_cmd(0xC1); self.write_data(b"\x13") # Power Control 2
+        self.write_cmd(0xC5); self.write_data(b"\x22\x35") # VCOM Control 1
+        self.write_cmd(0xC7); self.write_data(b"\xBD") # VCOM Control 2
+        
+        # Memory Access Control (MADCTL) = 0x68 (Landscape: MV=1, MX=1, MY=0, BGR color filter)
+        self.write_cmd(0x36); self.write_data(b"\x68")
+        
+        self.write_cmd(0xB6); self.write_data(b"\x0A\xA2") # Display Function Control
+        self.write_cmd(0x3A); self.write_data(b"\x55") # Pixel Format (COLMOD) = 16-bit RGB565
+        self.write_cmd(0xF6); self.write_data(b"\x01\x30")
+        self.write_cmd(0xB1); self.write_data(b"\x00\x1B") # Frame Rate Control
+        self.write_cmd(0xF2); self.write_data(b"\x00")
+        self.write_cmd(0x26); self.write_data(b"\x01") # Gamma Curve
+        
+        self.write_cmd(0xE0); self.write_data(b"\x0F\x35\x31\x0B\x0E\x06\x49\xA7\x33\x07\x0F\x03\x0C\x0A\x00") # Positive Gamma Correction
+        self.write_cmd(0xE1); self.write_data(b"\x00\x0A\x0F\x04\x11\x08\x36\x58\x4D\x07\x10\x0C\x32\x34\x0F") # Negative Gamma Correction
+        
+        if self.invert_color:
+            self.write_cmd(0x21) # INVON
+        else:
+            self.write_cmd(0x20) # INVOFF
+            
+        self.write_cmd(0x11) # SLPOUT (Exit sleep mode)
+        time.sleep_ms(120)
+        self.write_cmd(0x29) # DISPON (Display on)
+        time.sleep_ms(10)
+
+    def invert(self, enable):
+        if enable:
+            self.write_cmd(0x21) # INVON
+        else:
+            self.write_cmd(0x20) # INVOFF
+
+    def set_window(self, x0, y0, x1, y1):
+        # Column Address Set (CASET)
+        self.write_cmd(0x2A)
+        self.write_data(bytearray([x0 >> 8, x0 & 0xFF, x1 >> 8, x1 & 0xFF]))
+        
+        # Row Address Set (RASET)
+        self.write_cmd(0x2B)
+        self.write_data(bytearray([y0 >> 8, y0 & 0xFF, y1 >> 8, y1 & 0xFF]))
+        
+        # Memory Write (RAMWR)
+        self.write_cmd(0x2C)
+
+    @micropython.native
+    def _convert_rows(self, start_row, num_rows, row_buf):
+        """Converts 1-bit monochrome row segment to 16-bit RGB565 format.
+        Compiles block-wise bitwise operations at native speed.
+        """
+        width = self.width
+        canvas_buf = self.canvas_buffer
+        idx = 0
+        
+        for y in range(start_row, start_row + num_rows):
+            byte_offset = y * (width // 8)
+            for x_byte_idx in range(width // 8):
+                val = canvas_buf[byte_offset + x_byte_idx]
+                
+                # Unroll 8 bits for speed
+                # Bit 7
+                if val & 0x80:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 6
+                if val & 0x40:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 5
+                if val & 0x20:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 4
+                if val & 0x10:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 3
+                if val & 0x08:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 2
+                if val & 0x04:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 1
+                if val & 0x02:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 0
+                if val & 0x01:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+
+    def show(self):
+        """Refreshes the screen by writing the frame buffer segment-by-segment."""
+        self.set_window(0, 0, self.width - 1, self.height - 1)
+        
+        self.dc(1)
+        self.cs(0)
+        
+        num_chunks = self.height // self.chunk_rows
+        for chunk in range(num_chunks):
+            start_row = chunk * self.chunk_rows
+            self._convert_rows(start_row, self.chunk_rows, self.row_buffer)
+            self.spi.write(self.row_buffer)
+            
+        self.cs(1)
+
+    def set_brightness(self, level):
+        """Set backlight brightness percentage (0-100)."""
+        if self.bl is None:
+            return
+        from machine import Pin, PWM
+        level = max(0, min(100, level))
+        if level == 0:
+            if hasattr(self, '_bl_pwm') and self._bl_pwm is not None:
+                try:
+                    self._bl_pwm.deinit()
+                except:
+                    pass
+                self._bl_pwm = None
+            if isinstance(self.bl, Pin):
+                self.bl.init(Pin.OUT, value=0)
+        elif level == 100:
+            if hasattr(self, '_bl_pwm') and self._bl_pwm is not None:
+                try:
+                    self._bl_pwm.deinit()
+                except:
+                    pass
+                self._bl_pwm = None
+            if isinstance(self.bl, Pin):
+                self.bl.init(Pin.OUT, value=1)
+        else:
+            if not hasattr(self, '_bl_pwm') or self._bl_pwm is None:
+                self._bl_pwm = PWM(self.bl)
+                self._bl_pwm.freq(1000)
+            self._bl_pwm.duty_u16(int(level * 655.35))
+
+    def set_power(self, on):
+        """Set display power status (True = ON, False = OFF)."""
+        if on:
+            self.write_cmd(0x11) # SLPOUT
+            time.sleep_ms(120)
+            self.write_cmd(0x29) # DISPON
+            if self.bl is not None:
+                if hasattr(self, '_bl_pwm') and self._bl_pwm is not None:
+                    pass
+                else:
+                    from machine import Pin
+                    if isinstance(self.bl, Pin):
+                        self.bl.init(Pin.OUT, value=1)
+        else:
+            self.write_cmd(0x28) # DISPOFF
+            self.write_cmd(0x10) # SLPIN
+            time.sleep_ms(10)
+            if self.bl is not None:
+                if hasattr(self, '_bl_pwm') and self._bl_pwm is not None:
+                    try:
+                        self._bl_pwm.deinit()
+                    except:
+                        pass
+                    self._bl_pwm = None
+                from machine import Pin
+                if isinstance(self.bl, Pin):
+                    self.bl.init(Pin.OUT, value=0)
+

lib/rgb_led_util.py

@@ -9,7 +9,7 @@ class BoardLED:
     def __init__(self, pin_num=None):
         import sys
         if pin_num is None:
-            pin_num = 14 if sys.platform == 'rp2' else 38
+            pin_num = 14 if sys.platform == 'rp2' else 42
         self.np = neopixel.NeoPixel(Pin(pin_num), 1)
         self.base_color = (0, 0, 0)
         self.off()

lib/rlcd.py

@@ -2,6 +2,7 @@ import time
 from machine import Pin, SPI
 import framebuf
 import micropython
+import struct
 
 class RLCD:
     def __init__(self, spi, cs, dc, rst, width=400, height=300):
@@ -82,6 +83,97 @@ class RLCD:
         except OSError:
             print(f"Error: Could not open {filename}")
 
+    def draw_bmp(self, filename, x=0, y=0):
+        """Draw a 24-bit or 32-bit uncompressed color BMP image converted to 1-bit monochrome at (x, y) coordinates."""
+        try:
+            with open(filename, 'rb') as f:
+                header = f.read(54)
+                if len(header) < 54 or header[0:2] != b'BM':
+                    print("Err: Not a valid BMP file")
+                    return False
+                
+                pixel_offset = struct.unpack('<I', header[10:14])[0]
+                width, height = struct.unpack('<ii', header[18:26])
+                planes, bpp = struct.unpack('<HH', header[26:30])
+                compression = struct.unpack('<I', header[30:34])[0]
+                
+                if bpp not in (24, 32):
+                    print("Err: Only 24-bit and 32-bit BMP formats supported")
+                    return False
+                    
+                if compression != 0:
+                    print("Err: Only uncompressed BMP supported")
+                    return False
+                
+                f.seek(pixel_offset)
+                bottom_up = True
+                if height < 0:
+                    height = -height
+                    bottom_up = False
+                
+                row_bytes = (width * bpp) // 8
+                row_padded = ((width * bpp + 31) // 32) * 4
+                
+                read_buf = bytearray(row_padded)
+                
+                for row_idx in range(height):
+                    n = f.readinto(read_buf)
+                    if n < row_padded:
+                        break
+                    
+                    screen_y = y + (height - 1 - row_idx) if bottom_up else y + row_idx
+                    if screen_y < 0 or screen_y >= self.height:
+                        continue
+                        
+                    for px in range(width):
+                        screen_x = x + px
+                        if screen_x < 0 or screen_x >= self.width:
+                            continue
+                            
+                        if bpp == 24:
+                            b = read_buf[px * 3]
+                            g = read_buf[px * 3 + 1]
+                            r = read_buf[px * 3 + 2]
+                        else: # 32-bit
+                            b = read_buf[px * 4]
+                            g = read_buf[px * 4 + 1]
+                            r = read_buf[px * 4 + 2]
+                            
+                        # Convert to monochrome (0 = White, 1 = Black)
+                        lum = (r * 299 + g * 587 + b * 114) // 1000
+                        c = 1 if lum < 128 else 0
+                        self.canvas.pixel(screen_x, screen_y, c)
+                        
+            self.show()
+            return True
+        except Exception as e:
+            print("Error drawing BMP on RLCD:", e)
+            return False
+
+    def draw_rgb565(self, x, y, w, h, data, sync_canvas=True):
+        """Draw raw RGB565 pixel data converted to 1-bit monochrome on the RLCD."""
+        for cy in range(h):
+            screen_y = y + cy
+            if screen_y < 0 or screen_y >= self.height:
+                continue
+            for cx in range(w):
+                screen_x = x + cx
+                if screen_x < 0 or screen_x >= self.width:
+                    continue
+                idx = (cy * w + cx) * 2
+                h_byte = data[idx]
+                l_byte = data[idx + 1]
+                # Extract RGB from RGB565
+                r = (h_byte & 0xF8)
+                g = ((h_byte & 0x07) << 5) | ((l_byte & 0xE0) >> 3)
+                b = (l_byte & 0x1F) << 3
+                # Convert to luminance (0 = White, 1 = Black in RLCD)
+                lum = (r * 299 + g * 587 + b * 114) // 1000
+                c = 1 if lum < 128 else 0
+                self.canvas.pixel(screen_x, screen_y, c)
+        self.show()
+        return True
+
     # --- SCREENSHOT ---
     def save_screenshot(self, filename):
         print(f"Saving screenshot to {filename}...")
@@ -164,3 +256,19 @@ class RLCD:
         self.cs(0); self.dc(1)
         self.spi.write(self.hw_buffer)
         self.cs(1)
+
+    def set_brightness(self, level):
+        """Set backlight brightness percentage. RLCD is reflective and doesn't support backlight."""
+        print("RLCD is a reflective LCD and does not support backlight brightness control.")
+        pass
+
+    def set_power(self, on):
+        """Set display power status (True = ON, False = OFF)."""
+        if on:
+            self.write_cmd(0x11) # SLPOUT
+            time.sleep_ms(120)
+            self.write_cmd(0x29) # DISPON
+        else:
+            self.write_cmd(0x28) # DISPOFF
+            self.write_cmd(0x10) # SLPIN
+            time.sleep_ms(10)

lib/rtc_util.py

@@ -11,7 +11,8 @@ class PCF85063:
     def __init__(self, i2c=None):
         if i2c is None:
             # Default to ESP32-S3-RLCD-4.2 onboard I2C pins
-            self.i2c = I2C(0, sda=Pin(13), scl=Pin(14))
+            from machine import SoftI2C
+            self.i2c = SoftI2C(sda=Pin(13), scl=Pin(14))
         else:
             self.i2c = i2c
             

lib/shtc3_util.py

@@ -13,7 +13,8 @@ class SHTC3:
     def __init__(self, i2c=None):
         if i2c is None:
             # Default to ESP32-S3-RLCD-4.2 onboard I2C pins
-            self.i2c = I2C(0, sda=Pin(13), scl=Pin(14))
+            from machine import SoftI2C
+            self.i2c = SoftI2C(sda=Pin(13), scl=Pin(14))
         else:
             self.i2c = i2c
 

lib/st7796.py

@@ -0,0 +1,488 @@
+import time
+from machine import Pin, SPI
+import framebuf
+import micropython
+import struct
+
+class ST7796:
+    def __init__(self, spi, cs, dc, rst, bl=None, width=480, height=320, invert_color=True):
+        self.spi = spi
+        self.cs = cs
+        self.dc = dc
+        self.rst = rst
+        self.bl = bl
+        self.width = width
+        self.height = height
+        self.invert_color = invert_color
+        
+        # 1. 1-bit Canvas Buffer (Standard MONO_HLSB for drawing)
+        self.hw_len = (self.width * self.height) // 8
+        self.canvas_buffer = bytearray(self.hw_len)
+        self.canvas = framebuf.FrameBuffer(self.canvas_buffer, self.width, self.height, framebuf.MONO_HLSB)
+        
+        # Pre-allocate chunk buffer for conversion (16 rows: 480 * 16 * 2 = 15360 bytes)
+        self.chunk_rows = 16
+        self.row_buffer = bytearray(self.width * self.chunk_rows * 2)
+        
+        # Initialize pins
+        self.cs.init(self.cs.OUT, value=1)
+        self.dc.init(self.dc.OUT, value=0)
+        self.rst.init(self.rst.OUT, value=1)
+        
+        if self.bl is not None:
+            # PWM or Pin backlight control
+            if isinstance(self.bl, Pin):
+                self.bl.init(self.bl.OUT, value=1)
+                
+        self.reset()
+        self.init_display()
+        self.clear(0)
+        self.show()
+
+    # --- DRAWING WRAPPERS ---
+    def pixel(self, x, y, c): self.canvas.pixel(x, y, c)
+    def line(self, x1, y1, x2, y2, c): self.canvas.line(x1, y1, x2, y2, c)
+    def rect(self, x, y, w, h, c): self.canvas.rect(x, y, w, h, c)
+    def fill_rect(self, x, y, w, h, c): self.canvas.fill_rect(x, y, w, h, c)
+    def text(self, msg, x, y, c=1): self.canvas.text(msg, x, y, c)
+    def clear(self, c=0): self.canvas.fill(c)
+
+    # --- SCALABLE TEXT ---
+    def text_large(self, msg, x, y, scale=2, c=1):
+        char_w = 8; char_h = 8
+        tmp_buf = bytearray(char_w * char_h // 8)
+        tmp_fb = framebuf.FrameBuffer(tmp_buf, char_w, char_h, framebuf.MONO_HLSB)
+        for char in msg:
+            tmp_fb.fill(0); tmp_fb.text(char, 0, 0, 1)
+            for py in range(8):
+                for px in range(8):
+                    if tmp_fb.pixel(px, py):
+                        self.canvas.fill_rect(x + (px * scale), y + (py * scale), scale, scale, c)
+            x += (8 * scale)
+
+    # --- RAW BITMAPS (1:1 scale) ---
+    def bitmap(self, x, y, w, h, pixel_data):
+        img = framebuf.FrameBuffer(pixel_data, w, h, framebuf.MONO_HLSB)
+        self.canvas.blit(img, x, y)
+
+    # --- PBM FILE LOADER WITH SCALING ---
+    def draw_pbm(self, filename, x, y, scale=1):
+        try:
+            with open(filename, 'rb') as f:
+                line1 = f.readline()
+                if not line1.startswith(b'P4'): print("Err: Not P4 PBM"); return
+                while True:
+                    line = f.readline()
+                    if not line.startswith(b'#'): break
+                dims = line.split(); w = int(dims[0]); h = int(dims[1])
+                data = bytearray(f.read())
+                
+                src_fb = framebuf.FrameBuffer(data, w, h, framebuf.MONO_HLSB)
+                
+                if scale == 1:
+                     self.canvas.blit(src_fb, x, y)
+                else:
+                    for sy in range(h):
+                        for sx in range(w):
+                            if src_fb.pixel(sx, sy):
+                                self.canvas.fill_rect(x + (sx * scale), y + (sy * scale), scale, scale, 1)
+                print(f"Loaded {filename} (scale {scale})")
+        except OSError:
+            print(f"Error: Could not open {filename}")
+
+    @micropython.native
+    def _convert_bgr24_to_rgb565(self, bgr_buf, rgb565_buf, width, src_offset, num_pixels):
+        idx = 0
+        for i in range(src_offset, src_offset + num_pixels):
+            b = bgr_buf[i * 3]
+            g = bgr_buf[i * 3 + 1]
+            r = bgr_buf[i * 3 + 2]
+            r_5 = r >> 3
+            g_6 = g >> 2
+            b_5 = b >> 3
+            rgb565_buf[idx] = (r_5 << 3) | (g_6 >> 3)
+            rgb565_buf[idx + 1] = ((g_6 & 0x07) << 5) | b_5
+            idx += 2
+
+    @micropython.native
+    def _convert_bgra32_to_rgb565(self, bgra_buf, rgb565_buf, width, src_offset, num_pixels):
+        idx = 0
+        for i in range(src_offset, src_offset + num_pixels):
+            b = bgra_buf[i * 4]
+            g = bgra_buf[i * 4 + 1]
+            r = bgra_buf[i * 4 + 2]
+            r_5 = r >> 3
+            g_6 = g >> 2
+            b_5 = b >> 3
+            rgb565_buf[idx] = (r_5 << 3) | (g_6 >> 3)
+            rgb565_buf[idx + 1] = ((g_6 & 0x07) << 5) | b_5
+            idx += 2
+
+    def draw_bmp(self, filename, x=0, y=0):
+        """Draw a 24-bit or 32-bit uncompressed color BMP image at (x, y) coordinates."""
+        try:
+            with open(filename, 'rb') as f:
+                header = f.read(54)
+                if len(header) < 54 or header[0:2] != b'BM':
+                    print("Err: Not a valid BMP file")
+                    return False
+                
+                pixel_offset = struct.unpack('<I', header[10:14])[0]
+                width, height = struct.unpack('<ii', header[18:26])
+                planes, bpp = struct.unpack('<HH', header[26:30])
+                compression = struct.unpack('<I', header[30:34])[0]
+                
+                if bpp not in (24, 32):
+                    print("Err: Only 24-bit and 32-bit BMP formats supported")
+                    return False
+                    
+                if compression != 0:
+                    print("Err: Only uncompressed BMP supported")
+                    return False
+                
+                f.seek(pixel_offset)
+                bottom_up = True
+                if height < 0:
+                    height = -height
+                    bottom_up = False
+                
+                row_bytes = (width * bpp) // 8
+                row_padded = ((width * bpp + 31) // 32) * 4
+                
+                read_buf = bytearray(row_padded)
+                rgb565_buf = bytearray(width * 2)
+                
+                for row_idx in range(height):
+                    n = f.readinto(read_buf)
+                    if n < row_padded:
+                        break
+                    
+                    screen_y = y + (height - 1 - row_idx) if bottom_up else y + row_idx
+                    if screen_y < 0 or screen_y >= self.height:
+                        continue
+                        
+                    x_start = x
+                    x_end = x + width - 1
+                    
+                    if x_start >= self.width or x_end < 0:
+                        continue
+                        
+                    win_x0 = max(0, x_start)
+                    win_x1 = min(self.width - 1, x_end)
+                    
+                    if win_x1 < win_x0:
+                        continue
+                        
+                    src_offset_pixels = win_x0 - x_start
+                    win_w = win_x1 - win_x0 + 1
+                    
+                    # Convert pixel data to RGB565 row buffer
+                    if bpp == 24:
+                        self._convert_bgr24_to_rgb565(read_buf, rgb565_buf, width, src_offset_pixels, win_w)
+                    elif bpp == 32:
+                        self._convert_bgra32_to_rgb565(read_buf, rgb565_buf, width, src_offset_pixels, win_w)
+                    
+                    # Draw directly to the screen via SPI window
+                    self.set_window(win_x0, screen_y, win_x1, screen_y)
+                    self.dc(1)
+                    self.cs(0)
+                    self.spi.write(memoryview(rgb565_buf)[:win_w * 2])
+                    self.cs(1)
+                    
+                    # Also update internal 1-bit canvas buffer for screenshots/refresh consistency
+                    for px in range(win_w):
+                        screen_x = win_x0 + px
+                        src_px = src_offset_pixels + px
+                        if bpp == 24:
+                            b = read_buf[src_px * 3]
+                            g = read_buf[src_px * 3 + 1]
+                            r = read_buf[src_px * 3 + 2]
+                        else:
+                            b = read_buf[src_px * 4]
+                            g = read_buf[src_px * 4 + 1]
+                            r = read_buf[src_px * 4 + 2]
+                        # 0 = Black, 1 = White in conversion for MONO_HLSB canvas
+                        lum = (r * 299 + g * 587 + b * 114) // 1000
+                        mono_c = 1 if lum >= 128 else 0
+                        self.canvas.pixel(screen_x, screen_y, mono_c)
+                        
+            return True
+        except Exception as e:
+            print("Error drawing BMP:", e)
+            return False
+
+    @micropython.native
+    def _update_mono_canvas_rgb565(self, x, y, w, h, data):
+        for cy in range(h):
+            screen_y = y + cy
+            if screen_y < 0 or screen_y >= self.height:
+                continue
+            for cx in range(w):
+                screen_x = x + cx
+                if screen_x < 0 or screen_x >= self.width:
+                    continue
+                idx = (cy * w + cx) * 2
+                h_byte = data[idx]
+                l_byte = data[idx + 1]
+                # Extract RGB from RGB565
+                r = (h_byte & 0xF8)
+                g = ((h_byte & 0x07) << 5) | ((l_byte & 0xE0) >> 3)
+                b = (l_byte & 0x1F) << 3
+                # Convert to luminance
+                lum = (r * 299 + g * 587 + b * 114) // 1000
+                mono = 1 if lum >= 128 else 0
+                self.canvas.pixel(screen_x, screen_y, mono)
+
+    def draw_rgb565(self, x, y, w, h, data, sync_canvas=True):
+        """Draw raw RGB565 pixel data on the screen at specified (x,y) with width and height."""
+        # Clip coordinates
+        x_start = max(0, x)
+        x_end = min(self.width - 1, x + w - 1)
+        y_start = max(0, y)
+        y_end = min(self.height - 1, y + h - 1)
+        
+        if x_start > x_end or y_start > y_end:
+            return True
+            
+        # Fast path: if completely visible on screen, draw in one go
+        if x_start == x and x_end == x + w - 1 and y_start == y and y_end == y + h - 1:
+            self.set_window(x_start, y_start, x_end, y_end)
+            self.dc(1)
+            self.cs(0)
+            self.spi.write(data)
+            self.cs(1)
+        else:
+            # Slow path: row-by-row clipping
+            for cy in range(y_start, y_end + 1):
+                src_y = cy - y
+                src_row_offset = (src_y * w + (x_start - x)) * 2
+                row_len_bytes = (x_end - x_start + 1) * 2
+                
+                self.set_window(x_start, cy, x_end, cy)
+                self.dc(1)
+                self.cs(0)
+                self.spi.write(memoryview(data)[src_row_offset : src_row_offset + row_len_bytes])
+                self.cs(1)
+                
+        # Sync the internal 1-bit canvas buffer
+        if sync_canvas:
+            self._update_mono_canvas_rgb565(x, y, w, h, data)
+        return True
+
+    # --- SCREENSHOT ---
+    def save_screenshot(self, filename):
+        print(f"Saving screenshot to {filename}...")
+        try:
+            with open(filename, 'wb') as f:
+                f.write(b'P4\n')
+                f.write(f"{self.width} {self.height}\n".encode())
+                f.write(self.canvas_buffer)
+            print("Saved!")
+        except Exception as e:
+            print(f"Error saving screenshot: {e}")
+
+    # --- HARDWARE LOGIC ---
+    def reset(self):
+        self.rst(1); time.sleep_ms(5); self.rst(0); time.sleep_ms(15); self.rst(1); time.sleep_ms(15)
+
+    def write_cmd(self, cmd):
+        self.dc(0); self.cs(0); self.spi.write(bytearray([cmd])); self.cs(1)
+
+    def write_data(self, data):
+        self.dc(1); self.cs(0)
+        if isinstance(data, int): self.spi.write(bytearray([data]))
+        elif isinstance(data, list): self.spi.write(bytearray(data))
+        else: self.spi.write(data)
+        self.cs(1)
+
+    def init_display(self):
+        # ST7796 Minimal Initialization Sequence (ported from working C++ codebase)
+        self.write_cmd(0x01) # SWRESET
+        time.sleep_ms(150)
+        
+        self.write_cmd(0x11) # SLPOUT
+        time.sleep_ms(120)
+        
+        # Pixel Format (COLMOD) = 0x55 (16-bit color / RGB565)
+        self.write_cmd(0x3A); self.write_data(0x55)
+        time.sleep_ms(10)
+        
+        # Memory Access Control (MADCTL) = 0xE0 (Landscape: MY=1, MX=1, MV=1, RGB order)
+        self.write_cmd(0x36); self.write_data(0xE0)
+        time.sleep_ms(10)
+        
+        # Display Inversion Control
+        if self.invert_color:
+            self.write_cmd(0x21) # INVON
+        else:
+            self.write_cmd(0x20) # INVOFF
+        time.sleep_ms(10)
+        
+        self.write_cmd(0x13) # NORON
+        time.sleep_ms(10)
+        
+        self.write_cmd(0x29) # DISPON
+        time.sleep_ms(120)
+
+    def invert(self, enable):
+        if enable:
+            self.write_cmd(0x21) # INVON
+        else:
+            self.write_cmd(0x20) # INVOFF
+
+    def set_window(self, x0, y0, x1, y1):
+        # Column Address Set (CASET)
+        self.write_cmd(0x2A)
+        self.write_data(bytearray([x0 >> 8, x0 & 0xFF, x1 >> 8, x1 & 0xFF]))
+        
+        # Row Address Set (RASET)
+        self.write_cmd(0x2B)
+        self.write_data(bytearray([y0 >> 8, y0 & 0xFF, y1 >> 8, y1 & 0xFF]))
+        
+        # Memory Write (RAMWR)
+        self.write_cmd(0x2C)
+
+    @micropython.native
+    def _convert_rows(self, start_row, num_rows, row_buf):
+        """Converts 1-bit monochrome row segment to 16-bit RGB565 format.
+        Compiles block-wise bitwise operations at native speed.
+        """
+        width = self.width
+        canvas_buf = self.canvas_buffer
+        idx = 0
+        
+        for y in range(start_row, start_row + num_rows):
+            byte_offset = y * (width // 8)
+            for x_byte_idx in range(width // 8):
+                val = canvas_buf[byte_offset + x_byte_idx]
+                
+                # Unroll 8 bits for speed
+                # Bit 7
+                if val & 0x80:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 6
+                if val & 0x40:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 5
+                if val & 0x20:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 4
+                if val & 0x10:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 3
+                if val & 0x08:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 2
+                if val & 0x04:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 1
+                if val & 0x02:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+                
+                # Bit 0
+                if val & 0x01:
+                    row_buf[idx] = 0xFF; row_buf[idx+1] = 0xFF
+                else:
+                    row_buf[idx] = 0x00; row_buf[idx+1] = 0x00
+                idx += 2
+
+    def show(self):
+        """Refreshes the screen by writing the frame buffer segment-by-segment."""
+        self.set_window(0, 0, self.width - 1, self.height - 1)
+        
+        self.dc(1)
+        self.cs(0)
+        
+        num_chunks = self.height // self.chunk_rows
+        for chunk in range(num_chunks):
+            start_row = chunk * self.chunk_rows
+            self._convert_rows(start_row, self.chunk_rows, self.row_buffer)
+            self.spi.write(self.row_buffer)
+            
+        self.cs(1)
+
+    def set_brightness(self, level):
+        """Set backlight brightness percentage (0-100)."""
+        if self.bl is None:
+            return
+        from machine import Pin, PWM
+        level = max(0, min(100, level))
+        if level == 0:
+            if hasattr(self, '_bl_pwm') and self._bl_pwm is not None:
+                try:
+                    self._bl_pwm.deinit()
+                except:
+                    pass
+                self._bl_pwm = None
+            if isinstance(self.bl, Pin):
+                self.bl.init(Pin.OUT, value=0)
+        elif level == 100:
+            if hasattr(self, '_bl_pwm') and self._bl_pwm is not None:
+                try:
+                    self._bl_pwm.deinit()
+                except:
+                    pass
+                self._bl_pwm = None
+            if isinstance(self.bl, Pin):
+                self.bl.init(Pin.OUT, value=1)
+        else:
+            if not hasattr(self, '_bl_pwm') or self._bl_pwm is None:
+                self._bl_pwm = PWM(self.bl)
+                self._bl_pwm.freq(1000)
+            self._bl_pwm.duty_u16(int(level * 655.35))
+
+    def set_power(self, on):
+        """Set display power status (True = ON, False = OFF)."""
+        if on:
+            self.write_cmd(0x11) # SLPOUT
+            time.sleep_ms(120)
+            self.write_cmd(0x29) # DISPON
+            if self.bl is not None:
+                if hasattr(self, '_bl_pwm') and self._bl_pwm is not None:
+                    pass
+                else:
+                    from machine import Pin
+                    if isinstance(self.bl, Pin):
+                        self.bl.init(Pin.OUT, value=1)
+        else:
+            self.write_cmd(0x28) # DISPOFF
+            self.write_cmd(0x10) # SLPIN
+            time.sleep_ms(10)
+            if self.bl is not None:
+                if hasattr(self, '_bl_pwm') and self._bl_pwm is not None:
+                    try:
+                        self._bl_pwm.deinit()
+                    except:
+                        pass
+                    self._bl_pwm = None
+                from machine import Pin
+                if isinstance(self.bl, Pin):
+                    self.bl.init(Pin.OUT, value=0)
+

lib/websocket_client.py

@@ -0,0 +1,184 @@
+# pyright: reportMissingImports=false, reportAttributeAccessIssue=false
+"""Lightweight WebSocket client for MicroPython.
+
+This module provides a minimal, robust, memory-efficient WebSocket client
+supporting HTTP upgrade handshakes and client-to-server frame masking
+(RFC 6455).
+"""
+
+import usocket as socket
+import ustruct as struct
+import urandom as random
+import ubinascii as binascii
+
+def parse_url(url):
+    if not url.startswith("ws://") and not url.startswith("wss://"):
+        raise ValueError("URL must start with ws:// or wss://")
+    is_ssl = url.startswith("wss://")
+    url_p = url.split("://", 1)[1]
+    parts = url_p.split("/", 1)
+    host_port = parts[0]
+    path = "/" + parts[1] if len(parts) > 1 else "/"
+    if ":" in host_port:
+        host, port = host_port.split(":", 1)
+        port = int(port)
+    else:
+        host = host_port
+        port = 443 if is_ssl else 80
+    return host, port, path, is_ssl
+
+class WebSocketClient:
+    def __init__(self, url, headers=None, timeout=10):
+        self.url = url
+        self.headers = headers or {}
+        self.timeout = timeout
+        self.sock = None
+        self.host, self.port, self.path, self.is_ssl = parse_url(url)
+
+    def connect(self):
+        # 1. Generate standard Sec-WebSocket-Key
+        raw_key = bytes([random.getrandbits(8) for _ in range(16)])
+        sec_key = binascii.b2a_base64(raw_key).decode('utf-8').strip()
+
+        # 2. Resolve address and connect
+        addr = socket.getaddrinfo(self.host, self.port)[0][-1]
+        self.sock = socket.socket()
+        self.sock.settimeout(self.timeout)
+        self.sock.connect(addr)
+
+        if self.is_ssl:
+            import ussl
+            self.sock = ussl.wrap_socket(self.sock)
+
+        # 3. Construct HTTP GET upgrade request
+        req = [
+            f"GET {self.path} HTTP/1.1",
+            f"Host: {self.host}:{self.port}",
+            "Upgrade: websocket",
+            "Connection: Upgrade",
+            f"Sec-WebSocket-Key: {sec_key}",
+            "Sec-WebSocket-Version: 13",
+        ]
+        for k, v in self.headers.items():
+            req.append(f"{k}: {v}")
+        req.append("\r\n")
+
+        self.sock.write("\r\n".join(req).encode('utf-8'))
+
+        # 4. Read HTTP response status and headers
+        status_line = self.sock.readline()
+        if not status_line or b"101" not in status_line:
+            self.close()
+            raise RuntimeError(f"Handshake failed status: {status_line.decode('utf-8', 'ignore').strip()}")
+
+        while True:
+            line = self.sock.readline()
+            if not line or line == b"\r\n":
+                break
+
+    def send_frame(self, opcode, payload, fin=True):
+        """Send a masked WebSocket frame to the server (client-to-server MUST be masked)."""
+        if not self.sock:
+            raise RuntimeError("Not connected")
+
+        # Header byte 0: FIN and Opcode
+        b0 = 0x80 if fin else 0
+        b0 |= (opcode & 0x0F)
+
+        # Header byte 1: Mask bit (always 1 for client) and Payload length
+        payload_len = len(payload)
+        if payload_len <= 125:
+            header = struct.pack("!BB", b0, 0x80 | payload_len)
+        elif payload_len <= 65535:
+            header = struct.pack("!BBH", b0, 0x80 | 126, payload_len)
+        else:
+            header = struct.pack("!BBQ", b0, 0x80 | 127, payload_len)
+
+        # Generate 4-byte random masking key
+        mask = bytes([random.getrandbits(8) for _ in range(4)])
+
+        # Apply masking key (XOR payload)
+        masked_payload = bytearray(payload_len)
+        for i in range(payload_len):
+            masked_payload[i] = payload[i] ^ mask[i % 4]
+
+        # Send header, mask, and masked payload
+        self.sock.write(header)
+        self.sock.write(mask)
+        self.sock.write(masked_payload)
+
+    def send_text(self, text):
+        self.send_frame(0x1, text.encode('utf-8'))
+
+    def send_binary(self, data):
+        self.send_frame(0x2, data)
+
+    def recv_frame(self):
+        """Receive an unmasked WebSocket frame from the server (server-to-client is unmasked)."""
+        if not self.sock:
+            raise RuntimeError("Not connected")
+
+        try:
+            header = self.sock.read(2)
+        except Exception:
+            # Handle socket timeout or disconnect
+            return None, None
+
+        if not header or len(header) < 2:
+            return None, None
+
+        b0, b1 = header
+        opcode = b0 & 0x0F
+        masked = bool(b1 & 0x80)
+        payload_len = b1 & 0x7F
+
+        if payload_len == 126:
+            len_bytes = self.sock.read(2)
+            if not len_bytes or len(len_bytes) < 2:
+                return None, None
+            payload_len = struct.unpack("!H", len_bytes)[0]
+        elif payload_len == 127:
+            len_bytes = self.sock.read(8)
+            if not len_bytes or len(len_bytes) < 8:
+                return None, None
+            payload_len = struct.unpack("!Q", len_bytes)[0]
+
+        if masked:
+            mask = self.sock.read(4)
+            if not mask or len(mask) < 4:
+                return None, None
+
+        # Read actual payload
+        payload = b""
+        while len(payload) < payload_len:
+            needed = payload_len - len(payload)
+            chunk = self.sock.read(needed)
+            if not chunk:
+                break
+            payload += chunk
+
+        if len(payload) < payload_len:
+            # Socket closed prematurely
+            return None, None
+
+        if masked:
+            # Unmask payload if masked
+            unmasked = bytearray(payload_len)
+            for i in range(payload_len):
+                unmasked[i] = payload[i] ^ mask[i % 4]
+            payload = bytes(unmasked)
+
+        return opcode, payload
+
+    def close(self):
+        if self.sock:
+            try:
+                # Send close frame
+                self.send_frame(0x8, b"")
+            except Exception:
+                pass
+            try:
+                self.sock.close()
+            except Exception:
+                pass
+            self.sock = None

main.py

@@ -1,18 +1,15 @@
 import time
+import json
 import sys
 import machine
-from machine import Pin, SPI, I2C
+from machine import Pin, SPI, I2C, I2S
 
 try:
     import network
     has_network = True
 except ImportError:
     has_network = False
-
-if sys.platform == 'rp2':
-    import st7796 as display_module
-else:
-    import rlcd as display_module
+import board_config
 
 class DummyMCP:
     def __init__(self):
@@ -36,6 +33,150 @@ from rgb_led_util import BoardLED
 from ble_util import BLEUART
 from mcp_server import MCPServer
 from video_stream import VideoStreamServer
+from audio_util import ES8311
+import struct
+import urequests
+from websocket_client import WebSocketClient
+
+def create_wav_header(data_size):
+    # Generates a 44-byte WAV header for 16kHz, 16-bit mono PCM
+    riff = b'RIFF'
+    file_size = data_size + 36
+    wave = b'WAVE'
+    fmt = b'fmt '
+    chunk_size = 16
+    audio_format = 1 # PCM
+    channels = 1 # Mono
+    sample_rate = 16000
+    bits_per_sample = 16
+    byte_rate = sample_rate * channels * (bits_per_sample // 8)
+    block_align = channels * (bits_per_sample // 8)
+    data_label = b'data'
+    
+    return struct.pack('<4sI4s4sIHHIIHH4sI', 
+                       riff, file_size, wave, fmt, chunk_size, 
+                       audio_format, channels, sample_rate, byte_rate, 
+                       block_align, bits_per_sample, data_label, data_size)
+
+def socket_readline(s):
+    line = bytearray()
+    while True:
+        try:
+            char = s.recv(1)
+        except OSError:
+            break
+        if not char:
+            break
+        line.extend(char)
+        if char == b'\n':
+            break
+    return line
+
+def socket_read_exactly(s, n):
+    res = bytearray()
+    while len(res) < n:
+        try:
+            chunk = s.recv(n - len(res))
+        except OSError:
+            break
+        if not chunk:
+            break
+        res.extend(chunk)
+    return res
+
+
+def stream_hermes_request(url, headers, filename):
+    # Parse URL
+    proto, _, host_port_path = url.split('/', 2)
+    host_port = host_port_path.split('/', 1)[0]
+    path = '/' + host_port_path.split('/', 1)[1] if '/' in host_port_path else '/'
+    
+    if ':' in host_port:
+        host, port = host_port.split(':')
+        port = int(port)
+    else:
+        host, port = host_port, 80
+        
+    import socket
+    addr = socket.getaddrinfo(host, port)[0][-1]
+    s = socket.socket()
+    s.settimeout(30.0)
+    s.connect(addr)
+    
+    # Calculate file size
+    import os
+    try:
+        file_size = os.stat(filename)[6]
+    except OSError:
+        file_size = 0
+        
+    content_length = file_size + 44 # WAV header + PCM
+    
+    # Send request headers
+    s.write(f"POST {path} HTTP/1.1\r\n".encode())
+    s.write(f"Host: {host_port}\r\n".encode())
+    for k, v in headers.items():
+        s.write(f"{k}: {v}\r\n".encode())
+    s.write(f"Content-Length: {content_length}\r\n".encode())
+    s.write(b"\r\n")
+    
+    # Write WAV header
+    s.write(create_wav_header(file_size))
+    
+    # Stream audio file from flash
+    if file_size > 0:
+        buf = bytearray(2048)
+        with open(filename, "rb") as f:
+            while True:
+                n = f.readinto(buf)
+                if n == 0:
+                    break
+                s.write(buf[:n])
+                
+    # Read status line
+    status_line = socket_readline(s).decode()
+    parts = status_line.split(' ')
+    status_code = int(parts[1]) if len(parts) >= 2 else 500
+    
+    # Read headers
+    resp_headers = {}
+    while True:
+        line = socket_readline(s)
+        if line == b"\r\n" or not line:
+            break
+        p = line.decode().split(':', 1)
+        if len(p) == 2:
+            resp_headers[p[0].strip().lower()] = p[1].strip()
+            
+    return status_code, resp_headers, s
+
+
+def sync_ntp_time(rtc_chip):
+    if rtc_chip is None:
+        return False
+    import ntptime
+    import wifi_config
+    
+    tz_offset = getattr(wifi_config, 'TZ_OFFSET', 0)
+    print(f"Syncing time from NTP server... (Timezone offset: {tz_offset} hours)")
+    
+    for attempt in range(3):
+        try:
+            utc_sec = ntptime.time()
+            local_sec = utc_sec + int(tz_offset * 3600)
+            t = time.localtime(local_sec)
+            
+            dt = (t[0], t[1], t[2], t[6], t[3], t[4], t[5])
+            rtc_chip.set_datetime(dt)
+            rtc_chip.sync_to_system()
+            t_str = f"{t[0]:04d}-{t[1]:02d}-{t[2]:02d} {t[3]:02d}:{t[4]:02d}:{t[5]:02d}"
+            print(f"Successfully synced RTC with NTP. Local time: {t_str}")
+            return True
+        except Exception as e:
+            print(f"NTP sync attempt {attempt+1} failed: {e}")
+            time.sleep_ms(200)
+    return False
+
 
 # LED mode options for manual cycling
 led_modes = [
@@ -50,57 +191,57 @@ led_modes = [
 local_led_mode_idx = 1 # Green breathing
 
 def main():
+    # Check if we should run the audio loopback test instead
+    import os
+    try:
+        os.stat("run_loopback.txt")
+        print("run_loopback.txt found! Starting local audio loopback test...")
+        import demo_audio_loopback
+        demo_audio_loopback.main()
+        return
+    except OSError:
+        pass
+
     global local_led_mode_idx
-    print("=== Starting ESP32-S3-RLCD-4.2 Main Boot ===")
+    import board_config
     
-    # 1. Initialize shared buses and peripherals
-    i2c = None
-    if sys.platform != 'rp2':
-        try:
-            i2c = I2C(0, sda=Pin(13), scl=Pin(14))
-        except Exception as e:
-            print("Failed to initialize I2C0:", e)
+    print("=== Starting MCP Server Main Boot (Type: {}) ===".format(board_config.BOARD_TYPE))
     
-    if sys.platform == 'rp2':
-        spi = SPI(1, baudrate=32000000, polarity=0, phase=0, sck=Pin(10), mosi=Pin(11))
-        display = display_module.ST7796(spi, cs=Pin(7), dc=Pin(4), rst=Pin(9), bl=Pin(6))
+    # 1. Use pre-initialized display and buses from board_config
+    i2c = board_config.i2c_bus
+    spi = board_config.spi_bus
+    display = board_config.display_instance
+    touch = board_config.touch
     
-        # Touch controller
-        touch = None
-        try:
-            touch_i2c = I2C(1, sda=Pin(2), scl=Pin(3), freq=400000)
-            from ft6336u import FT6336U
-            touch = FT6336U(touch_i2c, rst_pin=28, int_pin=25)
-        except Exception as te:
-            print("Failed to initialize touch:", te)
-    else:
-        spi = SPI(1, baudrate=20000000, polarity=0, phase=0, sck=Pin(11), mosi=Pin(12))
-        display = display_module.RLCD(spi, cs=Pin(40), dc=Pin(5), rst=Pin(41))
-        touch = None
-    
-    if sys.platform != 'rp2':
     # Configure Audio Amp control pin to save power
-        amp_pin = Pin(46, Pin.OUT, value=0)
+    if sys.platform != 'rp2' and board_config.audio_amp_pin is not None:
+        # Turn OFF amplifier on boot (active-low vs active-high)
+        off_val = 1 if board_config.audio_amp_active_level == 0 else 0
+        amp_pin = Pin(board_config.audio_amp_pin, Pin.OUT, value=off_val)
     
     # 2. Initialize utility objects
     sensor = None
     rtc_chip = None
     if i2c is not None:
+        if board_config.has_sensor:
             try:
                 sensor = SHTC3(i2c)
             except Exception as e:
                 print("Failed to initialize SHTC3:", e)
+        if board_config.has_rtc:
             try:
                 rtc_chip = PCF85063(i2c)
             except Exception as e:
                 print("Failed to initialize PCF85063:", e)
             
     battery = BatteryMonitor()
-    led = BoardLED()
+    led = BoardLED(board_config.led_pin)
     buttons = None
     if sys.platform != 'rp2':
         buttons = BoardButtons()
-    ble_uart = BLEUART(name="ESP32-S3-RLCD")
+        
+    ble_name = "ESP32-S3-" + ("RLCD" if board_config.BOARD_TYPE == "WAVESHARE_RLCD" else "Touch")
+    ble_uart = BLEUART(name=ble_name)
     
     # Sync system clock from RTC chip
     if rtc_chip:
@@ -126,8 +267,11 @@ def main():
             
             # 4b. Start MCP Server
             from mcp_server import MCPServer
-            mcp = MCPServer(display, led, battery, sensor, rtc_chip, ble_uart, vstream=vstream)
+            mcp = MCPServer(display, led, battery, sensor, rtc_chip, ble_uart, vstream=vstream, touch=touch)
             mcp.start(port=80)
+            
+            if wlan.isconnected():
+                sync_ntp_time(rtc_chip)
         except Exception as ne:
             print("Failed to start network services:", ne)
             
@@ -138,9 +282,19 @@ def main():
     # Last user actions
     last_action_str = "Boot finished."
     force_dashboard_redraw = True
+    voice_assistant_active = False
+    
+    def is_talk_trigger_active():
+        if touch:
+            return touch.is_touched()
+        elif buttons and buttons.key:
+            return buttons.key.is_pressed()
+        return False
     
     # 5. Register Button Handlers
     def on_key_click():
+        if voice_assistant_active:
+            return
         global local_led_mode_idx
         local_led_mode_idx = (local_led_mode_idx + 1) % len(led_modes)
         mode_name, color_fn, mode_type = led_modes[local_led_mode_idx]
@@ -169,6 +323,7 @@ def main():
     last_dashboard_update = 0
     dashboard_update_interval_ms = 5000
     last_led_update = 0
+    last_wifi_check = 0
     
     print("ESP32 MCP loop running...")
     
@@ -176,14 +331,289 @@ def main():
     while True:
         now = time.ticks_ms()
         
-        # Check touch interaction if available
-        if touch and touch.is_touched():
-            pt = touch.read_touch()
-            if pt:
-                tx, ty = pt
-                print(f"Touch detected at: ({tx}, {ty})")
-                last_action_str = f"Touch: ({tx}, {ty})"
-                mcp.override_active = False
+        # A. Check Wi-Fi status periodically (every 10 seconds) and auto-reconnect
+        if has_network and time.ticks_diff(now, last_wifi_check) >= 10000:
+            last_wifi_check = now
+            if not wlan.isconnected():
+                print("Wi-Fi connection lost. Attempting reconnect...")
+                last_action_str = "Wi-Fi Disconnected"
+                if ip_addr != "Disconnected":
+                    ip_addr = "Disconnected"
+                    force_dashboard_redraw = True
+                try:
+                    wlan.connect(wifi_config.WIFI_SSID, wifi_config.WIFI_PASS)
+                except Exception as e:
+                    print("Wi-Fi reconnect trigger failed:", e)
+            elif ip_addr == "Disconnected" or ip_addr == "Offline (USB)":
+                ip_addr = wlan.ifconfig()[0]
+                print(f"Wi-Fi Connected! IP Address: {ip_addr}")
+                last_action_str = f"Wi-Fi Connected: {ip_addr}"
+                force_dashboard_redraw = True
+                sync_ntp_time(rtc_chip)
+        
+        # Check voice assistant trigger (touch screen or physical key button)
+        if is_talk_trigger_active():
+            print("Voice assistant trigger detected! Starting Hermes Voice Assistant...")
+            voice_assistant_active = True
+            
+            def draw_status_bar(text):
+                y_bar = display.height - 25
+                display.line(0, y_bar, display.width, y_bar, 1)
+                display.fill_rect(0, y_bar + 1, display.width, 24, 0)
+                display.text(text, 10, y_bar + 7, 1)
+                display.show()
+                
+            def clear_status_bar():
+                y_bar = display.height - 25
+                display.fill_rect(0, y_bar, display.width, 25, 0)
+                display.show()
+                
+            draw_status_bar("PTT Voice: Initializing...")
+            
+            # Start recording immediately!
+            tap_started = True
+            if tap_started:
+                draw_status_bar("Recording: 10s...")
+                
+                # 1. Start MCLK PWM and configure mic path based on board config
+                mclk_pwm = None
+                if board_config.audio_mclk_pin is not None:
+                    mclk_pin = Pin(board_config.audio_mclk_pin, Pin.OUT)
+                    mclk_pwm = machine.PWM(mclk_pin)
+                    mclk_pwm.freq(board_config.audio_mclk_freq)
+                    mclk_pwm.duty_u16(32768)
+                
+                if board_config.audio_mic_codec == "ES7210":
+                    from audio_util import ES7210
+                    codec = ES7210(i2c)
+                    codec.init(sample_rate=16000, bit_width=16)
+                else:
+                    from audio_util import ES8311
+                    codec = ES8311(i2c)
+                    if codec.init(sample_rate=16000):
+                        codec.set_volume(80)
+                        try:
+                            codec._write(0x14, 0x1A) # Enable analog mic input & PGA
+                            codec._write(0x16, 0x01) # Enable +6dB gain boost
+                            codec._write(0x17, 0xC8) # Set ADC digital volume
+                        except Exception as e:
+                            print("Failed to set mic gain:", e)
+                
+                # 2. Configure I2S RX for recording (Stereo 16kHz — ES7210 outputs stereo)
+                i2s_rx = I2S(1, 
+                             sck=Pin(board_config.audio_i2s_sck), 
+                             ws=Pin(board_config.audio_i2s_ws), 
+                             sd=Pin(board_config.audio_i2s_rx_sd), 
+                             mode=I2S.RX, 
+                             ibuf=16000, 
+                             rate=16000, 
+                             bits=16, 
+                             format=I2S.STEREO)
+                
+                ws_connected = False
+                ws = None
+                try:
+                    # Determine dynamic device ID based on board configuration
+                    device_id = "esp32_screen" if board_config.BOARD_TYPE == 'WAVESHARE_RLCD' else "little32"
+                    
+                    headers = {
+                        "Authorization": "Bearer mcT1YA1vOr9wXSiHpCYalweEGGZKX-PIfZv2drp8BSg",
+                        "X-Device-ID": device_id
+                    }
+                    ws = WebSocketClient("ws://192.168.68.126:8642/api/esp32/voice/ws", headers=headers, timeout=30)
+                    ws.connect()
+                    ws.send_text(json.dumps({
+                        "event": "start",
+                        "device_id": device_id,
+                        "sample_rate": 16000,
+                        "channels": 1,
+                        "sample_width": 2,
+                        "format": "pcm_s16le"
+                    }))
+                    ws.recv_frame() # ready
+                    ws.recv_frame() # listening
+                    ws_connected = True
+                except Exception as wse:
+                    print("WebSocket connect error:", wse)
+                    draw_status_bar("Connection Error")
+                    time.sleep(2)
+
+                if ws_connected and ws:
+                    draw_status_bar("Recording & streaming...")
+                    
+                    # Open I2S RX for recording (Stereo 16kHz)
+                    i2s_rx = I2S(1, 
+                                 sck=Pin(board_config.audio_i2s_sck), 
+                                 ws=Pin(board_config.audio_i2s_ws), 
+                                 sd=Pin(board_config.audio_i2s_rx_sd), 
+                                 mode=I2S.RX, 
+                                 ibuf=16000, 
+                                 rate=16000, 
+                                 bits=16, 
+                                 format=I2S.STEREO)
+                    
+                    total_data_bytes = 0
+                    buffer = bytearray(2048)
+                    mono_buf = bytearray(1024)
+                    
+                    rec_start_time = time.ticks_ms()
+                    max_rec_duration_ms = 10000 # 10 seconds max duration
+                    
+                    try:
+                        # Stream chunks while talk trigger is active
+                        while is_talk_trigger_active():
+                            elapsed = time.ticks_diff(time.ticks_ms(), rec_start_time)
+                            if elapsed >= max_rec_duration_ms:
+                                print("Recording stopped: maximum duration reached")
+                                break
+                                
+                            bytes_read = i2s_rx.readinto(buffer)
+                            if bytes_read > 0:
+                                mono_len = bytes_read // 2
+                                j = 0
+                                for i in range(0, bytes_read, 4):
+                                    mono_buf[j] = buffer[i]
+                                    mono_buf[j + 1] = buffer[i + 1]
+                                    j += 2
+                                ws.send_binary(mono_buf[:mono_len])
+                                total_data_bytes += mono_len
+                    except Exception as e:
+                        print("Error during recording/streaming:", e)
+                    finally:
+                        i2s_rx.deinit()
+                        if board_config.audio_mic_codec == "ES8311":
+                            try:
+                                codec._write(0x16, 0x00) # Reset mic gain
+                            except:
+                                pass
+                    
+                    if total_data_bytes < 3200:
+                        print("Recording too short, cancelling.")
+                        try:
+                            ws.send_text(json.dumps({"event": "cancel"}))
+                            ws.close()
+                        except:
+                            pass
+                        draw_status_bar("Cancelled")
+                        time.sleep(1)
+                    else:
+                        draw_status_bar("Processing...")
+                        try:
+                            ws.send_text(json.dumps({"event": "stop"}))
+                            
+                            i2s_tx = None
+                            received_audio_bytes = 0
+                            speaker_write_failed = False
+                            while True:
+                                opcode, payload = ws.recv_frame()
+                                if opcode is None:
+                                    break
+                                
+                                if opcode == 0x1: # Text JSON event
+                                    try:
+                                        event_data = json.loads(payload.decode('utf-8'))
+                                        evt = event_data.get("event")
+                                        
+                                        if evt == "transcript":
+                                            txt = event_data.get("text", "")
+                                            print(f"Heard: {txt}")
+                                            draw_status_bar(f"Heard: {txt[:20]}...")
+                                            
+                                        elif evt == "thinking":
+                                            draw_status_bar("Thinking...")
+                                            
+                                        elif evt == "response_text":
+                                            txt = event_data.get("text", "")
+                                            print(f"Response: {txt}")
+                                            
+                                        elif evt == "audio_start":
+                                            draw_status_bar("Playing response...")
+                                            on_val = 0 if board_config.audio_amp_active_level == 0 else 1
+                                            amp_pin.value(on_val) # Enable Amp
+                                            
+                                            # Initialize ES8311 Speaker DAC
+                                            try:
+                                                from audio_util import ES8311
+                                                dac = ES8311(i2c)
+                                                dac.init(sample_rate=16000)
+                                                dac.set_volume(85)
+                                            except Exception as dace:
+                                                print("Failed to initialize ES8311 DAC for playback:", dace)
+                                            
+                                            i2s_format = I2S.MONO # WebSocket audio response is mono
+                                            i2s_tx = I2S(1,
+                                                         sck=Pin(board_config.audio_i2s_sck),
+                                                         ws=Pin(board_config.audio_i2s_ws),
+                                                         sd=Pin(board_config.audio_i2s_tx_sd),
+                                                         mode=I2S.TX,
+                                                         ibuf=4096,
+                                                         rate=16000,
+                                                         bits=16,
+                                                         format=i2s_format)
+                                            
+                                        elif evt == "audio_end":
+                                            if i2s_tx:
+                                                time.sleep_ms(150)
+                                                off_val = 1 if board_config.audio_amp_active_level == 0 else 0
+                                                amp_pin.value(off_val) # Disable Amp
+                                                i2s_tx.deinit()
+                                                i2s_tx = None
+                                            if speaker_write_failed:
+                                                draw_status_bar("Speaker write failed")
+                                            else:
+                                                draw_status_bar(f"Recv {received_audio_bytes} bytes")
+                                                
+                                        elif evt == "done":
+                                            break
+                                            
+                                        elif evt == "error":
+                                            msg = event_data.get("message", "Unknown error")
+                                            print(f"Server error: {msg}")
+                                            draw_status_bar(f"Error: {msg[:20]}")
+                                            time.sleep(2)
+                                            break
+                                    except Exception as e:
+                                        print("Error parsing event text:", e)
+                                        
+                                elif opcode == 0x2: # Binary frame (Audio WAV chunk)
+                                    if i2s_tx:
+                                        chunk = payload
+                                        if chunk.startswith(b'RIFF') and len(chunk) > 44:
+                                            chunk = chunk[44:]
+                                        try:
+                                            i2s_tx.write(chunk)
+                                            received_audio_bytes += len(chunk)
+                                        except Exception as e:
+                                            speaker_write_failed = True
+                                            print("Error writing to speaker:", e)
+                        except Exception as he:
+                            print("Hermes WS query failed:", he)
+                            draw_status_bar("Connection Error")
+                            time.sleep(2)
+                        finally:
+                            try:
+                                ws.close()
+                            except:
+                                pass
+                                
+                # Deinit MCLK PWM
+                if mclk_pwm:
+                    try:
+                        mclk_pwm.deinit()
+                    except:
+                        pass
+                
+            # Debounce release at the very end of wizard
+            clear_status_bar()
+            release_end = time.ticks_ms()
+            while is_talk_trigger_active():
+                if time.ticks_diff(time.ticks_ms(), release_end) > 2000:
+                    break
+                time.sleep_ms(30)
+            time.sleep_ms(200)
+            
+            voice_assistant_active = False
+            last_action_str = "Voice query finished"
             force_dashboard_redraw = True
         
         # A. Handle non-blocking MCP client connection updates
@@ -214,31 +644,47 @@ def main():
                 
                 # Draw standard status dashboard layout
                 display.clear(0)
-                title_text = "RP2350-TFT MCP SERVER" if sys.platform == 'rp2' else "ESP32-S3-RLCD MCP SERVER"
-                line_w = 470 if sys.platform == 'rp2' else 390
+                line_w = display.width - 10
                 
+                if board_config.BOARD_TYPE == 'WAVESHARE_RLCD' or sys.platform == 'rp2':
+                    title_text = "RP2350-TFT MCP SERVER" if sys.platform == 'rp2' else "Waveshare ESP32-S3-RLCD Server"
                     display.text(title_text, 10, 10, 1)
                     display.line(10, 20, line_w, 20, 1)
-                
                     display.text_large("ENVIRONMENT", 15, 30, scale=2, c=1)
                     display.text(f"Temp        : {t_str}", 25, 55, 1)
                     display.text(f"Humid       : {h_str}", 25, 70, 1)
-                
                     display.line(10, 95, line_w, 95, 1)
-                
                     display.text_large("MCP NET CONNECTION", 15, 105, scale=2, c=1)
                     display.text(f"IP Address  : {ip_addr}", 25, 130, 1)
                     display.text(f"Port / Path : 80 /api/mcp", 25, 145, 1)
-                display.text(f"BLE Name    : ESP32-S3-RLCD", 25, 160, 1)
-                
+                    display.text(f"BLE Name    : {ble_name}", 25, 160, 1)
                     display.line(10, 185, line_w, 185, 1)
-                
                     display.text_large("SYSTEM STATUS", 15, 195, scale=2, c=1)
                     display.text(f"Battery     : {bat_str}", 25, 220, 1)
                     display.text(f"Time        : {time_str}", 25, 235, 1)
-                
                     display.line(10, 255, line_w, 255, 1)
                     display.text(f"Status: {last_action_str}", 15, 265, 1)
+                else:
+                    title_text = "Hosyond ESP32-S3 Server"
+                    display.text(title_text, 10, 8, 1)
+                    display.line(10, 18, line_w, 18, 1)
+                    # Left Column (System & Environment)
+                    display.text("SYSTEM & ENV", 10, 28, 1)
+                    display.line(10, 38, 150, 38, 1)
+                    display.text(f"Temp : {t_str}", 10, 46, 1)
+                    display.text(f"Hum  : {h_str}", 10, 58, 1)
+                    display.text(f"Bat  : {bat_str}", 10, 70, 1)
+                    display.text(f"Time : {time_str[11:19]}", 10, 82, 1)
+                    display.text(f"Date : {time_str[0:10]}", 10, 94, 1)
+                    # Right Column (Network)
+                    display.text("MCP NETWORK", 170, 28, 1)
+                    display.line(170, 38, line_w, 38, 1)
+                    display.text(f"IP : {ip_addr}", 170, 46, 1)
+                    display.text("Port: 80/api/mcp", 170, 58, 1)
+                    display.text(f"BLE : {ble_name[-6:]}", 170, 70, 1)
+                    # Bottom Status
+                    display.line(10, 115, line_w, 115, 1)
+                    display.text(f"Status: {last_action_str}", 10, 125, 1)
                 display.show()
                 
         # D. Update NeoPixel animation smoothly (runs every 50ms)

mcp_server.py

@@ -5,7 +5,7 @@ import time
 class MCPServer:
     """A lightweight JSON-RPC HTTP server implementing Model Context Protocol (MCP) endpoints."""
     
-    def __init__(self, display, led, battery, sensor, rtc, ble, vstream=None):
+    def __init__(self, display, led, battery, sensor, rtc, ble, vstream=None, touch=None):
         self.display = display
         self.led = led
         self.battery = battery
@@ -13,6 +13,7 @@ class MCPServer:
         self.rtc = rtc
         self.ble = ble
         self.vstream = vstream
+        self.touch = touch
         
         self.sock = None
         self.active_led_mode = "off" # static, breath, rainbow, off
@@ -20,6 +21,7 @@ class MCPServer:
 
     def start(self, port=80):
         """Starts the TCP server non-blockingly."""
+        self.port = port
         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
         self.sock.bind(('', port))
@@ -39,6 +41,27 @@ class MCPServer:
             print("Failed to bind UDP Discovery socket: {}".format(ue))
             self.udp_sock = None
 
+    def restart(self):
+        """Re-initializes the listening sockets after a fatal error."""
+        print("Restarting MCP Server sockets...")
+        try:
+            if self.sock:
+                self.sock.close()
+        except:
+            pass
+        self.sock = None
+        try:
+            if hasattr(self, 'udp_sock') and self.udp_sock:
+                self.udp_sock.close()
+        except:
+            pass
+        self.udp_sock = None
+        
+        # Add a short delay to allow socket reuse to settle
+        time.sleep_ms(100)
+        port = getattr(self, 'port', 80)
+        self.start(port)
+
     def update(self):
         """Check for incoming HTTP requests and handle them non-blockingly."""
         # 1. Handle UDP Discovery queries
@@ -54,20 +77,44 @@ class MCPServer:
         if self.sock is None:
             return
             
+        t0 = 0
+        t1 = 0
         try:
             client, addr = self.sock.accept()
-        except OSError:
+            t0 = time.ticks_ms()
+        except OSError as e:
+            import errno
+            err = getattr(e, 'errno', None)
+            if err is None and e.args:
+                err = e.args[0]
+            ewouldblock = getattr(errno, 'EWOULDBLOCK', errno.EAGAIN)
+            if err in (errno.EAGAIN, ewouldblock) or err is None:
                 # No incoming connection
                 return
+            # Fatal socket error
+            print(f"Fatal socket error in MCP Server accept(): {e}. Re-initializing...")
+            self.restart()
+            return
             
         try:
             # Handle incoming connection
             client.settimeout(2.0)
-            req = client.recv(2048).decode('utf-8')
+            req_bytes = client.recv(2048)
+            if not req_bytes:
+                client.close()
+                return
                 
-            # Simple HTTP parser
-            lines = req.split('\r\n')
-            if len(lines) == 0:
+            header_end = req_bytes.find(b'\r\n\r\n')
+            if header_end == -1:
+                header_end = len(req_bytes)
+                body_start_idx = len(req_bytes)
+            else:
+                body_start_idx = header_end + 4
+                
+            header_text = req_bytes[:header_end].decode('utf-8', 'ignore')
+            lines = header_text.split('\r\n')
+            t1 = time.ticks_ms()
+            if len(lines) == 0 or not lines[0]:
                 client.close()
                 return
                 
@@ -87,31 +134,98 @@ class MCPServer:
                         content_length = int(line.split(':')[1].strip())
                         break
                         
-                # Locate start of JSON body
-                body = ""
-                if '\r\n\r\n' in req:
-                    body = req.split('\r\n\r\n', 1)[1]
+                body_bytes = bytearray(req_bytes[body_start_idx:])
+                while len(body_bytes) < content_length:
+                    chunk = client.recv(min(1024, content_length - len(body_bytes)))
+                    if not chunk:
+                        break
+                    body_bytes.extend(chunk)
                     
-                # Read remaining body if not fully received
-                while len(body) < content_length:
-                    body += client.recv(1024).decode('utf-8')
-                    
-                # Parse JSON-RPC 2.0 Request
-                rpc_req = json.loads(body)
+                body_text = body_bytes.decode('utf-8', 'ignore')
+                rpc_req = json.loads(body_text)
                 rpc_resp = self._handle_rpc(rpc_req)
                 
-                # Send HTTP Response
                 resp_body = json.dumps(rpc_resp)
                 resp = "HTTP/1.1 200 OK\r\n"
                 resp += "Content-Type: application/json\r\n"
                 resp += f"Content-Length: {len(resp_body)}\r\n"
                 resp += "Connection: close\r\n\r\n"
                 resp += resp_body
-                client.send(resp.encode('utf-8'))
+                
+                data_to_send = resp.encode('utf-8')
+                total_sent = 0
+                while total_sent < len(data_to_send):
+                    sent = client.write(data_to_send[total_sent:])
+                    if sent is None or sent == 0:
+                        time.sleep_ms(10)
+                        continue
+                    total_sent += sent
+                    
+            elif method == 'POST' and path.startswith('/api/screen/raw'):
+                # Read content length
+                content_length = 0
+                for line in lines:
+                    if line.lower().startswith('content-length:'):
+                        content_length = int(line.split(':')[1].strip())
+                        break
+                        
+                body_bytes = bytearray(req_bytes[body_start_idx:])
+                while len(body_bytes) < content_length:
+                    chunk = client.recv(min(1024, content_length - len(body_bytes)))
+                    if not chunk:
+                        break
+                    body_bytes.extend(chunk)
+                t2 = time.ticks_ms()
+                    
+                # Parse query parameters from path (e.g. /api/screen/raw?x=0&y=0&w=320&h=240)
+                width = getattr(self.display, 'width', 320)
+                height = getattr(self.display, 'height', 240)
+                x, y, w, h = 0, 0, width, height
+                
+                if '?' in path:
+                    q_str = path.split('?', 1)[1]
+                    for param in q_str.split('&'):
+                        if '=' in param:
+                            k, v = param.split('=', 1)
+                            if k == 'x': x = int(v)
+                            elif k == 'y': y = int(v)
+                            elif k == 'w': w = int(v)
+                            elif k == 'h': h = int(v)
+                t3 = time.ticks_ms()
+                            
+                # Draw directly using display raw RGB565 method
+                self.override_active = True
+                self.display.draw_rgb565(x, y, w, h, body_bytes, sync_canvas=False)
+                t4 = time.ticks_ms()
+                
+                resp_body = "OK"
+                resp = "HTTP/1.1 200 OK\r\n"
+                resp += "Content-Type: text/plain\r\n"
+                resp += f"Content-Length: {len(resp_body)}\r\n"
+                resp += "Connection: close\r\n\r\n"
+                resp += resp_body
+                
+                data_to_send = resp.encode('utf-8')
+                total_sent = 0
+                while total_sent < len(data_to_send):
+                    sent = client.write(data_to_send[total_sent:])
+                    if sent is None or sent == 0:
+                        time.sleep_ms(10)
+                        continue
+                    total_sent += sent
+                t5 = time.ticks_ms()
+                print("[RAW API Profile] total={}ms: accept_to_header={}ms, recv_body={}ms, route_n_prep={}ms, draw={}ms, send_resp={}ms".format(
+                    time.ticks_diff(t5, t0),
+                    time.ticks_diff(t1, t0),
+                    time.ticks_diff(t2, t1),
+                    time.ticks_diff(t3, t2),
+                    time.ticks_diff(t4, t3),
+                    time.ticks_diff(t5, t4)
+                ))
             else:
                 # Return 404
                 resp = "HTTP/1.1 404 Not Found\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
-                client.send(resp.encode('utf-8'))
+                client.write(resp.encode('utf-8'))
         except Exception as e:
             print("Error handling client:", e)
         finally:
@@ -122,6 +236,10 @@ class MCPServer:
         method = req.get('method')
         params = req.get('params', {})
         
+        # Get active display dimensions dynamically to report correct screen resolution
+        width = getattr(self.display, 'width', 320)
+        height = getattr(self.display, 'height', 240)
+        
         if method == 'tools/list':
             return {
                 "jsonrpc": "2.0",
@@ -129,7 +247,7 @@ class MCPServer:
                     "tools": [
                         {
                             "name": "clear_screen",
-                            "description": "Clear the 480x320 screen to white (0) or black (1).",
+                            "description": f"Clear the {width}x{height} screen to white (0) or black (1).",
                             "inputSchema": {
                                 "type": "object",
                                 "properties": {
@@ -140,13 +258,13 @@ class MCPServer:
                         },
                         {
                             "name": "draw_text",
-                            "description": "Draw text on the screen at specified (x,y) coordinates.",
+                            "description": f"Draw text on the screen at specified (x,y) coordinates. Screen resolution is {width}x{height}.",
                             "inputSchema": {
                                 "type": "object",
                                 "properties": {
                                     "text": {"type": "string", "description": "The message to display"},
-                                    "x": {"type": "integer", "description": "X coordinate (0-470)"},
-                                    "y": {"type": "integer", "description": "Y coordinate (0-310)"},
+                                    "x": {"type": "integer", "description": f"X coordinate (0-{width-1})"},
+                                    "y": {"type": "integer", "description": f"Y coordinate (0-{height-1})"},
                                     "size": {"type": "integer", "enum": [1, 2], "description": "Text scale (1=normal, 2=large)"}
                                 },
                                 "required": ["text", "x", "y"]
@@ -171,6 +289,11 @@ class MCPServer:
                             "description": "Read the current battery voltage and estimated capacity percentage.",
                             "inputSchema": {"type": "object", "properties": {}}
                         },
+                        {
+                            "name": "get_touch",
+                            "description": "Read the current touch state and coordinates from the capacitive touchscreen.",
+                            "inputSchema": {"type": "object", "properties": {}}
+                        },
                         {
                             "name": "get_sensors",
                             "description": "Read onboard SHTC3 temperature and relative humidity.",
@@ -188,12 +311,12 @@ class MCPServer:
                         },
                         {
                             "name": "get_screenshot",
-                            "description": "Capture the current reflective LCD screen rendering as a PNG image.",
+                            "description": "Capture the current screen rendering as a PNG image.",
                             "inputSchema": {"type": "object", "properties": {}}
                         },
                         {
                             "name": "draw_image",
-                            "description": "Draw an image (PNG, JPEG, GIF, BMP, etc.) on the screen. The image will be converted to 1-bit monochrome and fit to display boundaries.",
+                            "description": f"Draw an image (PNG, JPEG, GIF, BMP, etc.) on the screen. The image will be converted to 1-bit monochrome and fit to display boundaries (max {width}x{height}).",
                             "inputSchema": {
                                 "type": "object",
                                 "properties": {
@@ -205,6 +328,44 @@ class MCPServer:
                                 "required": ["image_base64"]
                             }
                         },
+                        {
+                            "name": "draw_color_bmp",
+                            "description": f"Draw a color BMP image on the {width}x{height} color screen at specified (x,y) coordinates.",
+                            "inputSchema": {
+                                "type": "object",
+                                "properties": {
+                                    "bmp_base64": {"type": "string", "description": "Base64 encoded BMP image file (uncompressed 24-bit or 32-bit format)"},
+                                    "x": {"type": "integer", "description": "X coordinate to place the image (default 0)", "default": 0},
+                                    "y": {"type": "integer", "description": "Y coordinate to place the image (default 0)", "default": 0}
+                                },
+                                "required": ["bmp_base64"]
+                            }
+                        },
+                        {
+                            "name": "get_capabilities",
+                            "description": "Get screen capabilities (resolution, color support, and supported formats).",
+                            "inputSchema": {"type": "object", "properties": {}}
+                        },
+                        {
+                            "name": "sync_time",
+                            "description": "Synchronize the hardware and system clock with an internet NTP server.",
+                            "inputSchema": {"type": "object", "properties": {}}
+                        },
+                        {
+                            "name": "draw_raw_rgb565",
+                            "description": f"Draw raw RGB565 pixel data on the {width}x{height} screen at specified (x,y) coordinates with width (w) and height (h).",
+                            "inputSchema": {
+                                "type": "object",
+                                "properties": {
+                                    "rgb565_base64": {"type": "string", "description": "Base64 encoded raw RGB565 pixel data (Big-Endian, 2 bytes per pixel)"},
+                                    "x": {"type": "integer", "description": "X coordinate to place the image"},
+                                    "y": {"type": "integer", "description": "Y coordinate to place the image"},
+                                    "w": {"type": "integer", "description": "Width of the raw pixel block"},
+                                    "h": {"type": "integer", "description": "Height of the raw pixel block"}
+                                },
+                                "required": ["rgb565_base64", "x", "y", "w", "h"]
+                            }
+                        },
                         {
                             "name": "write_file",
                             "description": "Write a file (e.g., python script or config) to the board's flash storage.",
@@ -313,6 +474,28 @@ class MCPServer:
                             "name": "get_stream_stats",
                             "description": "Get real-time device-side performance and frame-rate statistics for the TCP/UDP video stream.",
                             "inputSchema": {"type": "object", "properties": {}}
+                        },
+                        {
+                            "name": "set_backlight",
+                            "description": "Adjust the brightness of the LCD backlight.",
+                            "inputSchema": {
+                                "type": "object",
+                                "properties": {
+                                    "brightness": {"type": "integer", "minimum": 0, "maximum": 100, "description": "Backlight brightness percentage (0-100)"}
+                                },
+                                "required": ["brightness"]
+                            }
+                        },
+                        {
+                            "name": "set_screen_power",
+                            "description": "Turn the screen/display on or off.",
+                            "inputSchema": {
+                                "type": "object",
+                                "properties": {
+                                    "power": {"type": "boolean", "description": "True to turn display ON, False to turn display OFF"}
+                                },
+                                "required": ["power"]
+                            }
                         }
                     ]
                 },
@@ -365,6 +548,22 @@ class MCPServer:
             self.display.show()
             return "Screen cleared."
             
+        elif name == "set_backlight":
+            brightness = int(args.get("brightness", 100))
+            if hasattr(self.display, "set_brightness"):
+                self.display.set_brightness(brightness)
+                return f"Backlight brightness set to {brightness}%."
+            else:
+                return "Backlight brightness control not supported on this display."
+                
+        elif name == "set_screen_power":
+            power = bool(args.get("power", True))
+            if hasattr(self.display, "set_power"):
+                self.display.set_power(power)
+                status = "ON" if power else "OFF"
+                return f"Screen power set to {status}."
+            else:
+                return "Screen power control not supported on this display."
         elif name == "draw_text":
             self.override_active = True
             text = str(args.get("text", ""))
@@ -404,6 +603,17 @@ class MCPServer:
             t, h = self.sensor.read_sensor()
             return json.dumps({"temperature_c": t, "humidity_pct": h})
             
+        elif name == "get_touch":
+            if self.touch is None:
+                return json.dumps({"error": "Touchscreen not configured on this device."})
+            is_t = self.touch.is_touched()
+            x, y = None, None
+            if is_t:
+                pt = self.touch.read_touch()
+                if pt:
+                    x, y = pt
+            return json.dumps({"is_touched": is_t, "x": x, "y": y})
+            
         elif name == "scan_ble":
             dur = int(args.get("duration_ms", 3000))
             # Scan returns dictionary: {mac: {rssi: rssi, name: name}}
@@ -451,9 +661,121 @@ class MCPServer:
                     pass
             return "Image displayed successfully."
             
+        elif name == "draw_color_bmp":
+            self.override_active = True
+            
+            bmp_b64 = args.get("bmp_base64")
+            x = int(args.get("x", 0))
+            y = int(args.get("y", 0))
+            
+            if not bmp_b64:
+                raise ValueError("Missing bmp_base64 parameter")
+                
+            import binascii
+            try:
+                bmp_bytes = binascii.a2b_base64(bmp_b64)
+            except Exception as e:
+                raise ValueError(f"Failed to decode base64 BMP: {e}")
+                
+            filename = 'temp_recv.bmp'
+            with open(filename, 'wb') as f:
+                f.write(bmp_bytes)
+                
+            try:
+                success = self.display.draw_bmp(filename, x, y)
+                if not success:
+                    raise RuntimeError("Failed to parse or draw BMP image on screen.")
+            finally:
+                import os
+                try:
+                    os.remove(filename)
+                except:
+                    pass
+            return "Color BMP image displayed successfully."
+            
+        elif name == "get_capabilities":
+            is_color = getattr(self.display, '__class__', None) is not None and self.display.__class__.__name__ != 'RLCD'
+            formats = ["rgb565_base64", "bmp_base64", "pbm_base64"] if is_color else ["pbm_base64", "bmp_base64", "rgb565_base64"]
+            
+            return json.dumps({
+                "color": is_color,
+                "width": width,
+                "height": height,
+                "formats": formats
+            })
+            
+        elif name == "sync_time":
+            import ntptime
+            import wifi_config
+            
+            tz_offset = getattr(wifi_config, 'TZ_OFFSET', 0)
+            success = False
+            t = None
+            
+            for attempt in range(3):
+                try:
+                    utc_sec = ntptime.time()
+                    local_sec = utc_sec + int(tz_offset * 3600)
+                    t = time.localtime(local_sec)
+                    
+                    dt = (t[0], t[1], t[2], t[6], t[3], t[4], t[5])
+                    if self.rtc:
+                        self.rtc.set_datetime(dt)
+                        self.rtc.sync_to_system()
+                        success = True
+                        break
+                except Exception as e:
+                    time.sleep_ms(200)
+                    
+            if success and t is not None:
+                t_str = f"{t[0]:04d}-{t[1]:02d}-{t[2]:02d} {t[3]:02d}:{t[4]:02d}:{t[5]:02d}"
+                return f"Successfully synchronized board clock with NTP server. Local time: {t_str}"
+            else:
+                raise RuntimeError("Failed to sync clock with NTP server.")
+            
+        elif name == "draw_raw_rgb565":
+            self.override_active = True
+            
+            rgb_b64 = args.get("rgb565_base64")
+            x = int(args.get("x", 0))
+            y = int(args.get("y", 0))
+            w = int(args.get("w", 0))
+            h = int(args.get("h", 0))
+            
+            if not rgb_b64:
+                raise ValueError("Missing rgb565_base64 parameter")
+                
+            import binascii
+            try:
+                rgb_bytes = binascii.a2b_base64(rgb_b64)
+            except Exception as e:
+                raise ValueError(f"Failed to decode base64 RGB565: {e}")
+                
+            if len(rgb_bytes) < w * h * 2:
+                raise ValueError(f"RGB565 data size too small (expected {w * h * 2} bytes, got {len(rgb_bytes)} bytes)")
+                
+            try:
+                self.display.draw_rgb565(x, y, w, h, rgb_bytes)
+            except Exception as e:
+                raise RuntimeError(f"Failed to draw RGB565: {e}")
+                
+            return "Raw RGB565 data displayed successfully."
+            
         elif name == "write_file":
             path = str(args.get("path"))
             content = str(args.get("content"))
+            # Auto-create parent directories on the device if present in the path
+            import os
+            parts = path.split('/')
+            if len(parts) > 1:
+                dir_path = ""
+                for part in parts[:-1]:
+                    if part:
+                        dir_path = dir_path + "/" + part if dir_path else part
+                        try:
+                            os.mkdir(dir_path)
+                        except OSError:
+                            pass # Directory likely already exists
             with open(path, 'w') as f:
                 f.write(content)
             return f"Successfully wrote {len(content)} characters to '{path}'."
@@ -527,7 +849,7 @@ class MCPServer:
                 raise RuntimeError(f"Failed to play audio file '{filename}'. Check format (16kHz 16-bit PCM WAV recommended).")
                 
         elif name == "record_voice":
-            duration = int(args.get("duration_sec", 4))
+            duration = int(args.get("duration_sec", 10))
             filename = str(args.get("filename", "recording.pcm"))
             # Clamp duration to a reasonable range
             duration = max(1, min(15, duration))

micropython_voice_animations/hermes_voice_animations.py

@@ -0,0 +1,143 @@
+# Hermes ESP32 voice-state animations for MicroPython screen boards.
+# Uploaded by the API server; can also be run manually with:
+#   exec(open('hermes_voice_animations.py').read()); run_state('ack')
+import time
+try:
+    import board_config
+except Exception:
+    board_config = None
+
+
+def _display():
+    return getattr(board_config, 'display_instance', None) if board_config else None
+
+
+def _sleep(ms):
+    try:
+        time.sleep_ms(ms)
+    except AttributeError:
+        time.sleep(ms / 1000)
+
+
+def _text(d, msg, x, y, scale=1, color=1):
+    if hasattr(d, 'text_large') and scale > 1:
+        try:
+            d.text_large(msg, x, y, scale=scale, color=color)
+            return
+        except TypeError:
+            pass
+    d.text(msg, x, y, color)
+
+
+def _center_text(d, msg, y, scale=1, color=1):
+    width = getattr(d, 'width', 320)
+    char_w = 8 * scale
+    x = max(0, (width - len(msg) * char_w) // 2)
+    _text(d, msg, x, y, scale=scale, color=color)
+
+
+def _wrap(text, width_chars=34, max_lines=4):
+    words = str(text or '').replace('\n', ' ').split()
+    lines, line = [], ''
+    for w in words:
+        candidate = (line + ' ' + w).strip()
+        if len(candidate) > width_chars and line:
+            lines.append(line)
+            line = w
+            if len(lines) >= max_lines:
+                break
+        else:
+            line = candidate
+    if line and len(lines) < max_lines:
+        lines.append(line)
+    return lines
+
+
+def _spinner(d, title, subtitle='', frames=24, delay=70):
+    width, height = getattr(d, 'width', 320), getattr(d, 'height', 240)
+    cx, cy = width // 2, height // 2 - 8
+    spokes = [(-22,0),(-16,-16),(0,-22),(16,-16),(22,0),(16,16),(0,22),(-16,16)]
+    for frame in range(frames):
+        d.clear(0)
+        _center_text(d, title, 18, scale=2)
+        for i, (dx, dy) in enumerate(spokes):
+            color = 1 if (i + frame) % len(spokes) in (0, 1, 2, 3) else 0
+            size = 4 if i == frame % len(spokes) else 2
+            d.fill_rect(cx + dx - size // 2, cy + dy - size // 2, size, size, color)
+        d.rect(8, height - 34, width - 16, 18, 1)
+        d.fill_rect(10, height - 32, ((width - 20) * ((frame % 16) + 1)) // 16, 14, 1)
+        if subtitle:
+            _center_text(d, subtitle[:32], height - 56, scale=1)
+        d.show()
+        _sleep(delay)
+
+
+def ack(frames=18):
+    d = _display()
+    if not d:
+        return
+    width, height = d.width, d.height
+    for i in range(frames):
+        d.clear(0)
+        _center_text(d, 'GOT IT', 18, scale=2)
+        # expanding check pulse
+        box = 48 + (i % 6) * 8
+        x, y = width // 2 - box // 2, height // 2 - box // 2
+        d.rect(x, y, box, box, 1)
+        d.line(width//2 - 24, height//2, width//2 - 6, height//2 + 18, 1)
+        d.line(width//2 - 6, height//2 + 18, width//2 + 30, height//2 - 22, 1)
+        _center_text(d, 'Listening captured', height - 36, scale=1)
+        d.show()
+        _sleep(55)
+
+
+def captioning(frames=36):
+    d = _display()
+    if not d:
+        return
+    width, height = d.width, d.height
+    for i in range(frames):
+        d.clear(0)
+        _center_text(d, 'CAPTIONING', 16, scale=2)
+        for bar in range(9):
+            h = 8 + ((i * 5 + bar * 13) % 42)
+            x = width // 2 - 72 + bar * 18
+            d.fill_rect(x, height // 2 - h // 2, 10, h, 1)
+        dots = '.' * ((i // 5) % 4)
+        _center_text(d, 'Transcribing' + dots, height - 42, scale=1)
+        d.show()
+        _sleep(65)
+
+
+def waiting(text='', frames=44):
+    d = _display()
+    if not d:
+        return
+    width, height = d.width, d.height
+    for i in range(frames):
+        d.clear(0)
+        _center_text(d, 'THINKING', 12, scale=2)
+        r = 18 + (i % 10) * 2
+        cx, cy = width // 2, height // 2 - 10
+        d.rect(cx - r, cy - r, r * 2, r * 2, 1)
+        d.rect(cx - r//2, cy - r//2, r, r, 1)
+        if text:
+            d.text('Heard:', 10, height - 68, 1)
+            for idx, line in enumerate(_wrap(text, 36, 3)):
+                d.text(line[:38], 10, height - 54 + idx * 12, 1)
+        else:
+            _center_text(d, 'Waiting for reply...', height - 42, scale=1)
+        d.show()
+        _sleep(75)
+
+
+def run_state(state='ack', text='', frames=None):
+    state = str(state or 'ack').lower()
+    if state in ('ack', 'listening', 'got_it'):
+        ack(frames or 18)
+    elif state in ('caption', 'captioning', 'transcribing'):
+        captioning(frames or 36)
+    elif state in ('wait', 'waiting', 'thinking', 'reply'):
+        waiting(text, frames or 44)
+    else:
+        _spinner(_display(), 'HERMES', state[:28], frames or 20)

pong.py

@@ -0,0 +1,97 @@
+# pong.py
+import time
+import board_config
+
+def run_pong(frames=200):
+    display = board_config.display_instance
+    if not display:
+        print("No display found")
+        return
+        
+    width = display.width
+    height = display.height
+    
+    # Ball state
+    bx, by = width // 2, height // 2
+    vx, vy = 6, 4
+    ball_size = 6
+    
+    # Paddle state
+    pad_w, pad_h = 8, 40
+    p1_y = height // 2 - pad_h // 2
+    p2_y = height // 2 - pad_h // 2
+    
+    p1_score = 0
+    p2_score = 0
+    
+    for _ in range(frames):
+        # Update ball
+        bx += vx
+        by += vy
+        
+        # Bounce top/bottom
+        if by <= 0 or by >= height - ball_size:
+            vy = -vy
+            
+        # Paddle AI (simple tracking)
+        target1 = by - pad_h // 2
+        p1_y += int((target1 - p1_y) * 0.15)
+        p1_y = max(0, min(height - pad_h, p1_y))
+        
+        target2 = by - pad_h // 2
+        p2_y += int((target2 - p2_y) * 0.15)
+        p2_y = max(0, min(height - pad_h, p2_y))
+        
+        # Bounce left paddle
+        if vx < 0 and bx <= 20 and bx >= 10:
+            if by + ball_size >= p1_y and by <= p1_y + pad_h:
+                vx = -vx
+                vy += int((by - (p1_y + pad_h // 2)) * 0.2)
+                
+        # Bounce right paddle
+        if vx > 0 and bx >= width - 20 - pad_w and bx <= width - 10:
+            if by + ball_size >= p2_y and by <= p2_y + pad_h:
+                vx = -vx
+                vy += int((by - (p2_y + pad_h // 2)) * 0.2)
+                
+        # Score left
+        if bx < 0:
+            p2_score += 1
+            bx, by = width // 2, height // 2
+            vx = 6
+            vy = 4
+            
+        # Score right
+        if bx > width:
+            p1_score += 1
+            bx, by = width // 2, height // 2
+            vx = -6
+            vy = -4
+            
+        # Draw everything
+        display.clear(0)
+        
+        # Center line
+        for y in range(0, height, 15):
+            display.fill_rect(width // 2 - 1, y, 2, 8, 1)
+            
+        # Paddles
+        display.fill_rect(10, p1_y, pad_w, pad_h, 1)
+        display.fill_rect(width - 20, p2_y, pad_w, pad_h, 1)
+        
+        # Ball
+        display.fill_rect(bx, by, ball_size, ball_size, 1)
+        
+        # Score
+        display.text(str(p1_score), width // 2 - 30, 10, 1)
+        display.text(str(p2_score), width // 2 + 20, 10, 1)
+        
+        display.show()
+        time.sleep_ms(30)
+        
+    # Clear screen at the end
+    display.clear(0)
+    display.show()
+
+# Run the animation
+run_pong()

scratch/get_stats.py

@@ -0,0 +1,25 @@
+import urllib.request
+import json
+
+def call_mcp(method, params={}):
+    url = "http://192.168.68.118/api/mcp"
+    payload = {
+        "jsonrpc": "2.0",
+        "id": 1,
+        "method": method,
+        "params": params
+    }
+    req = urllib.request.Request(
+        url,
+        data=json.dumps(payload).encode("utf-8"),
+        headers={"Content-Type": "application/json"},
+        method="POST"
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=5.0) as resp:
+            return json.loads(resp.read().decode("utf-8"))
+    except Exception as e:
+        return {"error": str(e)}
+
+res = call_mcp("tools/call", {"name": "get_stream_stats", "arguments": {}})
+print(json.dumps(res, indent=2))

scratch/read_serial.py

@@ -0,0 +1,30 @@
+import serial
+import time
+import sys
+
+def main():
+    port = "/dev/cu.usbmodem411CF91D65091"
+    print(f"Connecting to {port} at 115200...")
+    try:
+        ser = serial.Serial(port, 115200, timeout=1.0)
+    except Exception as e:
+        print(f"Error opening serial port: {e}")
+        return
+
+    print("Reading serial output for 3 seconds...")
+    start_time = time.time()
+    try:
+        while time.time() - start_time < 3.0:
+            if ser.in_waiting > 0:
+                data = ser.read(ser.in_waiting)
+                sys.stdout.write(data.decode("utf-8", "ignore"))
+                sys.stdout.flush()
+            time.sleep(0.1)
+    except KeyboardInterrupt:
+        pass
+    finally:
+        ser.close()
+        print("\nSerial connection closed.")
+
+if __name__ == "__main__":
+    main()

scratch/reboot_and_read.py

@@ -0,0 +1,39 @@
+import serial
+import time
+import sys
+
+def main():
+    port = "/dev/cu.usbmodem411CF91D65091"
+    print(f"Connecting to {port} at 115200...")
+    try:
+        ser = serial.Serial(port, 115200, timeout=1.0)
+    except Exception as e:
+        print(f"Error: {e}")
+        return
+
+    print("Sending Ctrl-C to interrupt...")
+    ser.write(b'\x03')
+    time.sleep(0.5)
+    print("--- Current buffer contents ---")
+    print(ser.read_all().decode("utf-8", "ignore"))
+
+    print("Sending Ctrl-D to trigger soft reboot...")
+    ser.write(b'\x04')
+    
+    print("--- Capturing boot output (10 seconds) ---")
+    start_time = time.time()
+    try:
+        while time.time() - start_time < 10.0:
+            if ser.in_waiting > 0:
+                data = ser.read(ser.in_waiting)
+                sys.stdout.write(data.decode("utf-8", "ignore"))
+                sys.stdout.flush()
+            time.sleep(0.05)
+    except KeyboardInterrupt:
+        pass
+    finally:
+        ser.close()
+        print("\nConnection closed.")
+
+if __name__ == "__main__":
+    main()

scratch/recover.py

@@ -0,0 +1,75 @@
+import json
+import urllib.request
+import time
+import sys
+
+ESP32_IP = "192.168.68.123"
+URL = f"http://{ESP32_IP}/api/mcp"
+
+def call_mcp_tool(tool_name, arguments):
+    payload = {
+        "jsonrpc": "2.0",
+        "id": 1,
+        "method": "tools/call",
+        "params": {
+            "name": tool_name,
+            "arguments": arguments
+        }
+    }
+    
+    req_data = json.dumps(payload).encode('utf-8')
+    req = urllib.request.Request(
+        URL,
+        data=req_data,
+        headers={"Content-Type": "application/json"},
+        method="POST"
+    )
+    
+    # Try multiple times for each call since the connection might reset
+    for attempt in range(1, 100):
+        try:
+            with urllib.request.urlopen(req, timeout=1.0) as response:
+                resp_body = response.read().decode('utf-8')
+                resp_json = json.loads(resp_body)
+                if "error" in resp_json:
+                    return None
+                return resp_json.get("result", {}).get("content", [{}])[0].get("text", "")
+        except Exception as e:
+            time.sleep(0.01)
+    return None
+
+def upload_file(local_path, remote_path):
+    print(f"Reading local file {local_path}...")
+    with open(local_path, 'r') as f:
+        content = f.read()
+        
+    print(f"Uploading to ESP32: {remote_path}...")
+    for attempt in range(1, 20):
+        res = call_mcp_tool("write_file", {"path": remote_path, "content": content})
+        if res:
+            print(f"Success uploading {remote_path} on attempt {attempt}!")
+            return True
+        print(f"Retry {attempt} uploading {remote_path}...")
+        time.sleep(0.1)
+    return False
+
+def main():
+    print("=== ESP32 OTA Recovery Tool ===")
+    
+    # Upload mcp_server.py first
+    if not upload_file("mcp_server.py", "mcp_server.py"):
+        print("Failed to upload mcp_server.py. Exiting.")
+        sys.exit(1)
+        
+    # Upload video_stream.py
+    if not upload_file("video_stream.py", "video_stream.py"):
+        print("Failed to upload video_stream.py. Exiting.")
+        sys.exit(1)
+        
+    print("Rebooting the board...")
+    reboot_code = "import machine\nmachine.reset()"
+    call_mcp_tool("execute_python", {"code": reboot_code})
+    print("Reboot command sent! The board should recover now.")
+
+if __name__ == "__main__":
+    main()