Add AGENTS.md documenting project architecture, Hermes integration learnings, and build steps

This commit is contained in:
Adolfo Reyna
2026-08-09 22:53:51 -04:00
parent 84e20f8d81
commit 583131221c
2 changed files with 82 additions and 0 deletions
+1
View File
@@ -11,4 +11,5 @@ dist/
journal.jsonl journal.jsonl
model_settings.json model_settings.json
voice_settings.json voice_settings.json
.hermes-voice-session.json
+81
View File
@@ -0,0 +1,81 @@
# VoiceAgent Architecture & Developer Guide (AGENTS.md)
This document contains architectural guidelines, operational patterns, and learnings for AI agents and developers working on the `VoiceAgent` codebase.
---
## 1. Project Overview & Architecture
`VoiceAgent` is a high-performance, real-time voice conversation system built on Pipecat and optimized for macOS.
- **Audio Pipeline**:
- **Sample Rates**: STT / VAD operates natively at **16 kHz**; Kokoro TTS synthesizes at **24 kHz**.
- **VAD / Push-to-Talk**: Silero VAD (`SileroVADAnalyzer`) and Push-To-Talk (`push_to_talk.py`, `global_hotkey.py`).
- **STT**: Apple `SpeechAnalyzer` (macOS 26+) or Whisper MLX / CPU (`speech_analyzer_stt.py`, `apple_stt.py`).
- **TTS**: Kokoro Neural TTS (`KokoroTTSService`) or macOS System Speech (`apple_tts.py`).
- **LLM Slot**:
- Primary Harness: **Hermes** (`hermes_llm.py` / `HermesLLM`).
- Additional Engines: Claude Code (`claude_llm.py`), local macOS MLX (`apple_llm.py`).
---
## 2. Hermes Integration & Session Handling
- **CLI Subcommand & Mode**:
- Uses `hermes chat -q "<query>" -Q --source voice`.
- The `-Q` (quiet mode) flag suppresses decorative headers, spinners, and preview output so only cleaned text reaches TTS.
- **Stderr Session Tracking**:
- Hermes outputs session headers (e.g. `session_id: 20260809_...`) on **`stderr`**.
- `_remember_session_id()` in `hermes_llm.py` extracts the session ID from stderr and persists it.
- **Per-Workspace Session Persistence**:
- Session state is saved per workspace in `.hermes-voice-session.json` (inside `cwd`), keeping project histories isolated.
- On restart, `HermesLLM` loads the active session ID and resumes with `hermes chat ... -r <session_id>`.
- **Display Renaming**:
- Automatically renames the active session to `"Voice Agent"` via `hermes sessions rename <session_id> "Voice Agent"` on the first turn.
- **Native Persona & Memory**:
- Hermes manages persona, preferences, and long-term memory natively in `~/.hermes`.
- Avoid injecting redundant system prompt wrappers (`Brain` memories or `AGENTS.md` personality prompts) into Hermes turns.
- **Model Selection**:
- Default model setting is `"default"`, letting Hermes use its configured model in `~/.hermes/config.yaml`. Avoid passing `-m` unless explicitly overriding the model.
---
## 3. App Directory Storage (No `~/Workspace` Dependency)
The project operates entirely from the app directory without depending on `~/Workspace`:
- **Vocabulary & Repairs**: `vocabulary.txt` and `corrections.txt` live directly in the app root folder.
- **App Settings**: `model_settings.json`, `voice_settings.json`, and `journal.jsonl` are saved in the app folder.
- **CLI Helper Scripts**:
- `bin/voice_tool.py`: List and switch TTS voices (`python bin/voice_tool.py list`, `python bin/voice_tool.py set <voice>`).
- `bin/model_tool.py`: List and switch LLM models (`python bin/model_tool.py list`, `python bin/model_tool.py set <model>`).
- `bin/web_tool.py`: Inspect files or URLs in the Companion Web UI drawer (`http://localhost:8888`).
---
## 4. Building & Packaging `/Applications/VoiceAgent.app`
Whenever python or Swift sources are modified, update the standalone macOS application bundle:
```bash
bash build_app.sh
```
**Build Workflow**:
1. Builds Swift binaries (`swift/build.sh` -> `speech-helper`, `llm-helper`, `VoiceAgentLauncher`).
2. Bundles Python source files, `bin/` tools, and assets into `dist/VoiceAgent.app`.
3. Signs the app bundle (`codesign -s - --deep --force`).
4. Replaces `/Applications/VoiceAgent.app`.
---
## 5. Verification & Testing
Before committing changes, execute the test suite:
```bash
.venv/bin/python3 test_model_manager.py
.venv/bin/python3 test_spoken_text.py
.venv/bin/python3 test_journal.py
.venv/bin/python3 test_working_phrase.py
```