mcp_screen/hermes_voice_gateway

Hermes ESP32 Voice Gateway

This folder captures the Hermes API-server implementation used by the ESP32 screen voice demo.

It is intentionally kept next to the screen firmware because the ESP32 client in demo_hermes_voice.py talks to this gateway route:

POST /api/esp32/voice

Flow

1. ESP32 records push-to-talk audio as 16 kHz mono 16-bit PCM WAV.
2. ESP32 posts the raw WAV bytes to Hermes with an Authorization header containing the configured API server token.
3. Hermes saves the upload to its audio cache and transcribes it with the configured Hermes STT provider.
4. Hermes sends the transcript through the normal agent session using a persistent conversation key: esp32:<device_id>.
5. Hermes turns the final answer into speech with the configured Hermes TTS provider.
6. Hermes normalizes the TTS output into an ESP32-friendly WAV: mono, 16 kHz, signed 16-bit little-endian PCM.
7. Hermes returns the WAV bytes to the ESP32 for playback.

Files

• api_server_endpoint.py — the route/mixin implementation for Hermes' gateway/platforms/api_server.py.
• smoke_test.py — local helper tests for MIME mapping, safe headers, WAV normalization, and fallback ACK audio generation.

Client request contract

Required:

• Method: POST
• Path: /api/esp32/voice
• Header: Authorization containing the configured API server token
• Header: Content-Type: audio/wav
• Header: X-Device-ID: kitchen-button or another stable device id
• Body: raw audio bytes, preferably 16 kHz mono 16-bit PCM WAV

Optional headers:

• X-Hermes-Instructions — extra system instructions for this turn.
• X-Hermes-Reply-Mode: ack — return a quick ACK WAV immediately, then finish the agent turn in the background.
• X-Hermes-Screen-Url — JSON-RPC MCP endpoint for the initiating screen, e.g. http://192.168.68.123/api/mcp.
• X-Hermes-Screen-Device — local device registry alias to resolve to a screen URL.

Sync-mode response:

• Content-Type: audio/wav
• WAV body playable by the ESP32
• Diagnostic headers:
  • X-Hermes-Transcript
  • X-Hermes-Text-Response
  • X-Hermes-Response-Id
  • X-Hermes-Conversation
  • X-Hermes-TTS-Source-Type

ACK-mode response:

• Content-Type: audio/wav
• short acknowledgement WAV immediately
• the full answer is displayed/spoken on the initiating MCP screen when a screen URL can be resolved

Hermes integration notes

In Hermes Agent, wire this into gateway/platforms/api_server.py by adding:

self._app.router.add_post("/api/esp32/voice", self._handle_esp32_voice)

after the API server app is created.

The mixin expects the API server class to already provide these Hermes internals:

• _check_auth(request)
• _run_agent(...)
• _build_response_conversation_history(...)
• _response_messages_turn_start_index(...)
• _extract_output_items(...)
• _response_store
• _model_name
• _background_tasks

Do not commit API keys or device tokens here. The ESP32 should use the live API_SERVER_KEY configured on the Hermes host.

Smoke test

From this repo:

python3 hermes_voice_gateway/smoke_test.py

If ffmpeg is installed, the smoke test also verifies WAV normalization through the same decode/rewrite path used for TTS output.