diff --git a/.gitignore b/.gitignore index a838a11..941b2b5 100644 --- a/.gitignore +++ b/.gitignore @@ -11,4 +11,5 @@ dist/ journal.jsonl model_settings.json voice_settings.json +.hermes-voice-session.json diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..5309b15 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,81 @@ +# VoiceAgent Architecture & Developer Guide (AGENTS.md) + +This document contains architectural guidelines, operational patterns, and learnings for AI agents and developers working on the `VoiceAgent` codebase. + +--- + +## 1. Project Overview & Architecture + +`VoiceAgent` is a high-performance, real-time voice conversation system built on Pipecat and optimized for macOS. + +- **Audio Pipeline**: + - **Sample Rates**: STT / VAD operates natively at **16 kHz**; Kokoro TTS synthesizes at **24 kHz**. + - **VAD / Push-to-Talk**: Silero VAD (`SileroVADAnalyzer`) and Push-To-Talk (`push_to_talk.py`, `global_hotkey.py`). + - **STT**: Apple `SpeechAnalyzer` (macOS 26+) or Whisper MLX / CPU (`speech_analyzer_stt.py`, `apple_stt.py`). + - **TTS**: Kokoro Neural TTS (`KokoroTTSService`) or macOS System Speech (`apple_tts.py`). +- **LLM Slot**: + - Primary Harness: **Hermes** (`hermes_llm.py` / `HermesLLM`). + - Additional Engines: Claude Code (`claude_llm.py`), local macOS MLX (`apple_llm.py`). + +--- + +## 2. Hermes Integration & Session Handling + +- **CLI Subcommand & Mode**: + - Uses `hermes chat -q "" -Q --source voice`. + - The `-Q` (quiet mode) flag suppresses decorative headers, spinners, and preview output so only cleaned text reaches TTS. +- **Stderr Session Tracking**: + - Hermes outputs session headers (e.g. `session_id: 20260809_...`) on **`stderr`**. + - `_remember_session_id()` in `hermes_llm.py` extracts the session ID from stderr and persists it. +- **Per-Workspace Session Persistence**: + - Session state is saved per workspace in `.hermes-voice-session.json` (inside `cwd`), keeping project histories isolated. + - On restart, `HermesLLM` loads the active session ID and resumes with `hermes chat ... -r `. +- **Display Renaming**: + - Automatically renames the active session to `"Voice Agent"` via `hermes sessions rename "Voice Agent"` on the first turn. +- **Native Persona & Memory**: + - Hermes manages persona, preferences, and long-term memory natively in `~/.hermes`. + - Avoid injecting redundant system prompt wrappers (`Brain` memories or `AGENTS.md` personality prompts) into Hermes turns. +- **Model Selection**: + - Default model setting is `"default"`, letting Hermes use its configured model in `~/.hermes/config.yaml`. Avoid passing `-m` unless explicitly overriding the model. + +--- + +## 3. App Directory Storage (No `~/Workspace` Dependency) + +The project operates entirely from the app directory without depending on `~/Workspace`: + +- **Vocabulary & Repairs**: `vocabulary.txt` and `corrections.txt` live directly in the app root folder. +- **App Settings**: `model_settings.json`, `voice_settings.json`, and `journal.jsonl` are saved in the app folder. +- **CLI Helper Scripts**: + - `bin/voice_tool.py`: List and switch TTS voices (`python bin/voice_tool.py list`, `python bin/voice_tool.py set `). + - `bin/model_tool.py`: List and switch LLM models (`python bin/model_tool.py list`, `python bin/model_tool.py set `). + - `bin/web_tool.py`: Inspect files or URLs in the Companion Web UI drawer (`http://localhost:8888`). + +--- + +## 4. Building & Packaging `/Applications/VoiceAgent.app` + +Whenever python or Swift sources are modified, update the standalone macOS application bundle: + +```bash +bash build_app.sh +``` + +**Build Workflow**: +1. Builds Swift binaries (`swift/build.sh` -> `speech-helper`, `llm-helper`, `VoiceAgentLauncher`). +2. Bundles Python source files, `bin/` tools, and assets into `dist/VoiceAgent.app`. +3. Signs the app bundle (`codesign -s - --deep --force`). +4. Replaces `/Applications/VoiceAgent.app`. + +--- + +## 5. Verification & Testing + +Before committing changes, execute the test suite: + +```bash +.venv/bin/python3 test_model_manager.py +.venv/bin/python3 test_spoken_text.py +.venv/bin/python3 test_journal.py +.venv/bin/python3 test_working_phrase.py +```