# Virtual Screen & Speakers MCP Companion

This directory implements a python-based Model Context Protocol (MCP) server that hosts a premium, futuristic web dashboard. You can open this web interface on **any local network device** (like a phone, tablet, or secondary monitor) to act as a virtual screen and speakers for the LLM.

```
┌─────────────────────────────────┐
│     Local LLM Client / Cursor   │
└────────────────┬────────────────┘
                 │ (JSON-RPC over Stdio)
                 ▼
┌─────────────────────────────────┐
│           server.py             │ <── (Maintains PIL canvas state in memory)
└────────────────┬────────────────┘
                 │ (JSON-RPC over WebSockets)
                 ▼
┌─────────────────────────────────┐
│       Browser Web Client        │ (Render canvas, play Audio API tones/WAV,
│       (index.html UI)           │  capture mic and upload WAV)
└─────────────────────────────────┘
```

---

## 1. Prerequisites

Ensure you have Python 3.10+ installed along with the required libraries:
```bash
pip install pillow tornado
```

---

## 2. Launching the Server

Start the companion server from this folder:
```bash
python3 server.py --port 8080
```
Upon launching, it will print out the connection URLs, such as:
```text
--------------------------------------------------
Virtual Screen & Speaker MCP Server Initialized.
Connect local devices in your browser to:
==> http://192.168.1.15:8080/
==> http://localhost:8080/
--------------------------------------------------
```
Open either URL on your laptop or enter the local IP version (`http://192.168.1.15:8080/`) into the browser of your phone or tablet.

---

## 3. Claude Desktop Integration

To register this server in Claude Desktop, open your configuration file:
*   **macOS**: `/Users/<username>/Library/Application Support/Claude/claude_desktop_config.json`
*   **Windows**: `%APPDATA%\Claude\claude_desktop_config.json`

Add the server to the list:
```json
{
  "mcpServers": {
    "virtual-companion": {
      "command": "python3",
      "args": [
        "/Users/adolforeyna/Projects/MicroPython/test1/Screen/virtual_screen_mcp/server.py",
        "--port",
        "8080"
      ]
    }
  }
}
```
Restart Claude Desktop to load the companion tools.

---

## 4. Features & Tools

The server exposes the following MCP tools to the LLM:

| Tool Name | Parameters | Description |
|---|---|---|
| `clear_screen` | `color` (optional CSS color, e.g., `#1e1e2e` or legacy 0/1) | Clears the web canvas. |
| `draw_text` | `text`, `x`, `y`, `size`, `color` | Draws text with custom fonts and colors on the display. |
| `draw_shape` | `shape` (rect/circle/line), `x`, `y`, `width`, `height`, `color`, `fill` | Draws vector shapes on the canvas. |
| `draw_image` | `image_base64`, `x`, `y`, `width`, `height` | Draws base64 encoded images. |
| `get_screenshot` | *(None)* | Captures the virtual display buffer as a PNG image for the LLM. |
| `set_led` | `r`, `g`, `b`, `mode` (static/breath/rainbow/off) | Controls the CSS WS2812 NeoPixel ring. |
| `get_sensors` | *(None)* | Reads telemetry values adjusted by the dashboard sliders. |
| `play_tone` | `frequency`, `duration_ms`, `volume` | Synthesizes pure tones on the browser speakers. |
| `play_audio` | `audio_base64_or_url`, `volume` | Plays an audio track on the browser speakers. |
| `record_voice` | `duration_sec` | Captures microphone input from the browser, encodes it as a mono 16-bit WAV, and saves it. |

---

## 5. Web Client Telemetry

The dashboard provides interactive elements:
*   **Virtual Sensor Sliders**: Move the sliders for Temperature, Humidity, and Light to feed custom telemetry data to the LLM. When the LLM calls `get_sensors`, it receives these live values.
*   **Microphone Recorder**: Shows active microphone status, handles media constraints, and encodes 16-bit WAV files locally on the browser side.
*   **LED NeoPixel ring**: Visualizes breath, rainbow, and static LED modes in full glowing CSS.