Add AGENTS.md documenting project architecture, Hermes integration learnings, and build steps

2026-08-09 22:53:51 -04:00
parent 84e20f8d81
commit 583131221c
2 changed files with 82 additions and 0 deletions
@@ -11,4 +11,5 @@ dist/
 journal.jsonl
 model_settings.json
 voice_settings.json
+.hermes-voice-session.json

@@ -0,0 +1,81 @@
+# VoiceAgent Architecture & Developer Guide (AGENTS.md)
+
+This document contains architectural guidelines, operational patterns, and learnings for AI agents and developers working on the `VoiceAgent` codebase.
+
+---
+
+## 1. Project Overview & Architecture
+
+`VoiceAgent` is a high-performance, real-time voice conversation system built on Pipecat and optimized for macOS.
+
+- **Audio Pipeline**:
+  - **Sample Rates**: STT / VAD operates natively at **16 kHz**; Kokoro TTS synthesizes at **24 kHz**.
+  - **VAD / Push-to-Talk**: Silero VAD (`SileroVADAnalyzer`) and Push-To-Talk (`push_to_talk.py`, `global_hotkey.py`).
+  - **STT**: Apple `SpeechAnalyzer` (macOS 26+) or Whisper MLX / CPU (`speech_analyzer_stt.py`, `apple_stt.py`).
+  - **TTS**: Kokoro Neural TTS (`KokoroTTSService`) or macOS System Speech (`apple_tts.py`).
+- **LLM Slot**:
+  - Primary Harness: **Hermes** (`hermes_llm.py` / `HermesLLM`).
+  - Additional Engines: Claude Code (`claude_llm.py`), local macOS MLX (`apple_llm.py`).
+
+---
+
+## 2. Hermes Integration & Session Handling
+
+- **CLI Subcommand & Mode**:
+  - Uses `hermes chat -q "<query>" -Q --source voice`.
+  - The `-Q` (quiet mode) flag suppresses decorative headers, spinners, and preview output so only cleaned text reaches TTS.
+- **Stderr Session Tracking**:
+  - Hermes outputs session headers (e.g. `session_id: 20260809_...`) on **`stderr`**.
+  - `_remember_session_id()` in `hermes_llm.py` extracts the session ID from stderr and persists it.
+- **Per-Workspace Session Persistence**:
+  - Session state is saved per workspace in `.hermes-voice-session.json` (inside `cwd`), keeping project histories isolated.
+  - On restart, `HermesLLM` loads the active session ID and resumes with `hermes chat ... -r <session_id>`.
+- **Display Renaming**:
+  - Automatically renames the active session to `"Voice Agent"` via `hermes sessions rename <session_id> "Voice Agent"` on the first turn.
+- **Native Persona & Memory**:
+  - Hermes manages persona, preferences, and long-term memory natively in `~/.hermes`.
+  - Avoid injecting redundant system prompt wrappers (`Brain` memories or `AGENTS.md` personality prompts) into Hermes turns.
+- **Model Selection**:
+  - Default model setting is `"default"`, letting Hermes use its configured model in `~/.hermes/config.yaml`. Avoid passing `-m` unless explicitly overriding the model.
+
+---
+
+## 3. App Directory Storage (No `~/Workspace` Dependency)
+
+The project operates entirely from the app directory without depending on `~/Workspace`:
+
+- **Vocabulary & Repairs**: `vocabulary.txt` and `corrections.txt` live directly in the app root folder.
+- **App Settings**: `model_settings.json`, `voice_settings.json`, and `journal.jsonl` are saved in the app folder.
+- **CLI Helper Scripts**:
+  - `bin/voice_tool.py`: List and switch TTS voices (`python bin/voice_tool.py list`, `python bin/voice_tool.py set <voice>`).
+  - `bin/model_tool.py`: List and switch LLM models (`python bin/model_tool.py list`, `python bin/model_tool.py set <model>`).
+  - `bin/web_tool.py`: Inspect files or URLs in the Companion Web UI drawer (`http://localhost:8888`).
+
+---
+
+## 4. Building & Packaging `/Applications/VoiceAgent.app`
+
+Whenever python or Swift sources are modified, update the standalone macOS application bundle:
+
+```bash
+bash build_app.sh
+```
+
+**Build Workflow**:
+1. Builds Swift binaries (`swift/build.sh` -> `speech-helper`, `llm-helper`, `VoiceAgentLauncher`).
+2. Bundles Python source files, `bin/` tools, and assets into `dist/VoiceAgent.app`.
+3. Signs the app bundle (`codesign -s - --deep --force`).
+4. Replaces `/Applications/VoiceAgent.app`.
+
+---
+
+## 5. Verification & Testing
+
+Before committing changes, execute the test suite:
+
+```bash
+.venv/bin/python3 test_model_manager.py
+.venv/bin/python3 test_spoken_text.py
+.venv/bin/python3 test_journal.py
+.venv/bin/python3 test_working_phrase.py
+```