aeroreyna
91 lines
3.2 KiB
Markdown
91 lines
3.2 KiB
Markdown
# Hermes ESP32 Voice Gateway
|
|
|
|
This folder captures the Hermes API-server implementation used by the ESP32 screen voice demo.
|
|
|
|
It is intentionally kept next to the screen firmware because the ESP32 client in `demo_hermes_voice.py` talks to this gateway route:
|
|
|
|
`POST /api/esp32/voice`
|
|
|
|
## Flow
|
|
|
|
1. ESP32 records push-to-talk audio as 16 kHz mono 16-bit PCM WAV.
|
|
2. ESP32 posts the raw WAV bytes to Hermes with an Authorization header containing the configured API server token.
|
|
3. Hermes saves the upload to its audio cache and transcribes it with the configured Hermes STT provider.
|
|
4. Hermes sends the transcript through the normal agent session using a persistent conversation key: `esp32:<device_id>`.
|
|
5. Hermes turns the final answer into speech with the configured Hermes TTS provider.
|
|
6. Hermes normalizes the TTS output into an ESP32-friendly WAV: mono, 16 kHz, signed 16-bit little-endian PCM.
|
|
7. Hermes returns the WAV bytes to the ESP32 for playback.
|
|
|
|
## Files
|
|
|
|
- `api_server_endpoint.py` — the route/mixin implementation for Hermes' `gateway/platforms/api_server.py`.
|
|
- `smoke_test.py` — local helper tests for MIME mapping, safe headers, WAV normalization, and fallback ACK audio generation.
|
|
|
|
## Client request contract
|
|
|
|
Required:
|
|
|
|
- Method: `POST`
|
|
- Path: `/api/esp32/voice`
|
|
- Header: Authorization containing the configured API server token
|
|
- Header: `Content-Type: audio/wav`
|
|
- Header: `X-Device-ID: kitchen-button` or another stable device id
|
|
- Body: raw audio bytes, preferably 16 kHz mono 16-bit PCM WAV
|
|
|
|
Optional headers:
|
|
|
|
- `X-Hermes-Instructions` — extra system instructions for this turn.
|
|
- `X-Hermes-Reply-Mode: ack` — return a quick ACK WAV immediately, then finish the agent turn in the background.
|
|
- `X-Hermes-Screen-Url` — JSON-RPC MCP endpoint for the initiating screen, e.g. `http://192.168.68.123/api/mcp`.
|
|
- `X-Hermes-Screen-Device` — local device registry alias to resolve to a screen URL.
|
|
|
|
Sync-mode response:
|
|
|
|
- `Content-Type: audio/wav`
|
|
- WAV body playable by the ESP32
|
|
- Diagnostic headers:
|
|
- `X-Hermes-Transcript`
|
|
- `X-Hermes-Text-Response`
|
|
- `X-Hermes-Response-Id`
|
|
- `X-Hermes-Conversation`
|
|
- `X-Hermes-TTS-Source-Type`
|
|
|
|
ACK-mode response:
|
|
|
|
- `Content-Type: audio/wav`
|
|
- short acknowledgement WAV immediately
|
|
- the full answer is displayed/spoken on the initiating MCP screen when a screen URL can be resolved
|
|
|
|
## Hermes integration notes
|
|
|
|
In Hermes Agent, wire this into `gateway/platforms/api_server.py` by adding:
|
|
|
|
```python
|
|
self._app.router.add_post("/api/esp32/voice", self._handle_esp32_voice)
|
|
```
|
|
|
|
after the API server app is created.
|
|
|
|
The mixin expects the API server class to already provide these Hermes internals:
|
|
|
|
- `_check_auth(request)`
|
|
- `_run_agent(...)`
|
|
- `_build_response_conversation_history(...)`
|
|
- `_response_messages_turn_start_index(...)`
|
|
- `_extract_output_items(...)`
|
|
- `_response_store`
|
|
- `_model_name`
|
|
- `_background_tasks`
|
|
|
|
Do not commit API keys or device tokens here. The ESP32 should use the live `API_SERVER_KEY` configured on the Hermes host.
|
|
|
|
## Smoke test
|
|
|
|
From this repo:
|
|
|
|
```bash
|
|
python3 hermes_voice_gateway/smoke_test.py
|
|
```
|
|
|
|
If `ffmpeg` is installed, the smoke test also verifies WAV normalization through the same decode/rewrite path used for TTS output.
|