Add auto-discovery support (UDP discovery responder & subnet scanner) and implement virtual screen MCP server

2026-06-03 14:29:49 -04:00
parent 81e7b15b61
commit 070390355e
8 changed files with 2115 additions and 2 deletions
@@ -0,0 +1,101 @@
+# Virtual Screen & Speakers MCP Companion
+
+This directory implements a python-based Model Context Protocol (MCP) server that hosts a premium, futuristic web dashboard. You can open this web interface on **any local network device** (like a phone, tablet, or secondary monitor) to act as a virtual screen and speakers for the LLM.
+
+```
+┌─────────────────────────────────┐
+│     Local LLM Client / Cursor   │
+└────────────────┬────────────────┘
+                 │ (JSON-RPC over Stdio)
+                 ▼
+┌─────────────────────────────────┐
+│           server.py             │ <── (Maintains PIL canvas state in memory)
+└────────────────┬────────────────┘
+                 │ (JSON-RPC over WebSockets)
+                 ▼
+┌─────────────────────────────────┐
+│       Browser Web Client        │ (Render canvas, play Audio API tones/WAV,
+│       (index.html UI)           │  capture mic and upload WAV)
+└─────────────────────────────────┘
+```
+
+---
+
+## 1. Prerequisites
+
+Ensure you have Python 3.10+ installed along with the required libraries:
+```bash
+pip install pillow tornado
+```
+
+---
+
+## 2. Launching the Server
+
+Start the companion server from this folder:
+```bash
+python3 server.py --port 8080
+```
+Upon launching, it will print out the connection URLs, such as:
+```text
+--------------------------------------------------
+Virtual Screen & Speaker MCP Server Initialized.
+Connect local devices in your browser to:
+==> http://192.168.1.15:8080/
+==> http://localhost:8080/
+--------------------------------------------------
+```
+Open either URL on your laptop or enter the local IP version (`http://192.168.1.15:8080/`) into the browser of your phone or tablet.
+
+---
+
+## 3. Claude Desktop Integration
+
+To register this server in Claude Desktop, open your configuration file:
+*   **macOS**: `/Users/<username>/Library/Application Support/Claude/claude_desktop_config.json`
+*   **Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
+
+Add the server to the list:
+```json
+{
+  "mcpServers": {
+    "virtual-companion": {
+      "command": "python3",
+      "args": [
+        "/Users/adolforeyna/Projects/MicroPython/test1/Screen/virtual_screen_mcp/server.py",
+        "--port",
+        "8080"
+      ]
+    }
+  }
+}
+```
+Restart Claude Desktop to load the companion tools.
+
+---
+
+## 4. Features & Tools
+
+The server exposes the following MCP tools to the LLM:
+
+| Tool Name | Parameters | Description |
+|---|---|---|
+| `clear_screen` | `color` (optional CSS color, e.g., `#1e1e2e` or legacy 0/1) | Clears the web canvas. |
+| `draw_text` | `text`, `x`, `y`, `size`, `color` | Draws text with custom fonts and colors on the display. |
+| `draw_shape` | `shape` (rect/circle/line), `x`, `y`, `width`, `height`, `color`, `fill` | Draws vector shapes on the canvas. |
+| `draw_image` | `image_base64`, `x`, `y`, `width`, `height` | Draws base64 encoded images. |
+| `get_screenshot` | *(None)* | Captures the virtual display buffer as a PNG image for the LLM. |
+| `set_led` | `r`, `g`, `b`, `mode` (static/breath/rainbow/off) | Controls the CSS WS2812 NeoPixel ring. |
+| `get_sensors` | *(None)* | Reads telemetry values adjusted by the dashboard sliders. |
+| `play_tone` | `frequency`, `duration_ms`, `volume` | Synthesizes pure tones on the browser speakers. |
+| `play_audio` | `audio_base64_or_url`, `volume` | Plays an audio track on the browser speakers. |
+| `record_voice` | `duration_sec` | Captures microphone input from the browser, encodes it as a mono 16-bit WAV, and saves it. |
+
+---
+
+## 5. Web Client Telemetry
+
+The dashboard provides interactive elements:
+*   **Virtual Sensor Sliders**: Move the sliders for Temperature, Humidity, and Light to feed custom telemetry data to the LLM. When the LLM calls `get_sensors`, it receives these live values.
+*   **Microphone Recorder**: Shows active microphone status, handles media constraints, and encodes 16-bit WAV files locally on the browser side.
+*   **LED NeoPixel ring**: Visualizes breath, rainbow, and static LED modes in full glowing CSS.