# VoiceAgent Architecture & Developer Guide (AGENTS.md)

This document contains architectural guidelines, operational patterns, and learnings for AI agents and developers working on the `VoiceAgent` codebase.

---

## 1. Project Overview & Architecture

`VoiceAgent` is a high-performance, real-time voice conversation system built on Pipecat and optimized for macOS.

- **Audio Pipeline**:
  - **Sample Rates**: STT / VAD operates natively at **16 kHz**; Kokoro TTS synthesizes at **24 kHz**.
  - **VAD / Push-to-Talk**: Silero VAD (`SileroVADAnalyzer`) and Push-To-Talk (`push_to_talk.py`, `global_hotkey.py`).
  - **STT**: Apple `SpeechAnalyzer` (macOS 26+) or Whisper MLX / CPU (`speech_analyzer_stt.py`, `apple_stt.py`).
  - **TTS**: Kokoro Neural TTS (`KokoroTTSService`) or macOS System Speech (`apple_tts.py`).
- **LLM Slot**:
  - Primary Harness: **Hermes** (`hermes_llm.py` / `HermesLLM`).
  - Additional Engines: Claude Code (`claude_llm.py`), local macOS MLX (`apple_llm.py`).

---

## 2. Hermes Integration & Session Handling

- **CLI Subcommand & Mode**:
  - Uses `hermes chat -q "<query>" -Q --source voice`.
  - The `-Q` (quiet mode) flag suppresses decorative headers, spinners, and preview output so only cleaned text reaches TTS.
- **Stderr Session Tracking**:
  - Hermes outputs session headers (e.g. `session_id: 20260809_...`) on **`stderr`**.
  - `_remember_session_id()` in `hermes_llm.py` extracts the session ID from stderr and persists it.
- **Per-Workspace Session Persistence**:
  - Session state is saved per workspace in `.hermes-voice-session.json` (inside `cwd`), keeping project histories isolated.
  - On restart, `HermesLLM` loads the active session ID and resumes with `hermes chat ... -r <session_id>`.
- **Display Renaming**:
  - Automatically renames the active session to `"Voice Agent"` via `hermes sessions rename <session_id> "Voice Agent"` on the first turn.
- **Native Persona & Memory**:
  - Hermes manages persona, preferences, and long-term memory natively in `~/.hermes`.
  - Avoid injecting redundant system prompt wrappers (`Brain` memories or `AGENTS.md` personality prompts) into Hermes turns.
- **Model Selection**:
  - Default model setting is `"default"`, letting Hermes use its configured model in `~/.hermes/config.yaml`. Avoid passing `-m` unless explicitly overriding the model.

---

## 3. App Directory Storage (No `~/Workspace` Dependency)

The project operates entirely from the app directory without depending on `~/Workspace`:

- **Vocabulary & Repairs**: `vocabulary.txt` and `corrections.txt` live directly in the app root folder.
- **App Settings**: `model_settings.json`, `voice_settings.json`, and `journal.jsonl` are saved in the app folder.
- **CLI Helper Scripts**:
  - `bin/voice_tool.py`: List and switch TTS voices (`python bin/voice_tool.py list`, `python bin/voice_tool.py set <voice>`).
  - `bin/model_tool.py`: List and switch LLM models (`python bin/model_tool.py list`, `python bin/model_tool.py set <model>`).
  - `bin/session_tool.py`: Inspect or reset active Hermes session (`python bin/session_tool.py get`, `python bin/session_tool.py reset`).
  - `bin/profile_tool.py`: List and switch Hermes agent profiles (`python bin/profile_tool.py list`, `python bin/profile_tool.py set <profile>`).
  - `bin/web_tool.py`: Inspect files or URLs in the Companion Web UI drawer (`http://localhost:8888`).

---

## 4. Building & Packaging `/Applications/VoiceAgent.app`

`VoiceAgentLauncher` runs Python scripts dynamically from the project workspace (configured in `~/.voiceagent.env` or `VOICEAGENT_DIR`). 

- **Python code edits take effect immediately** without recompiling or re-signing the macOS app bundle.
- **Rebuilding binaries** (`bash build_app.sh`) is only required when modifying Swift sources (`swift/*.swift`), helper binaries, or `Info.plist` entitlements.
- `build_app.sh` automatically updates `~/.voiceagent.env` and skips binary re-signing if Swift sources are unchanged, preserving macOS privacy permissions (Microphone, Accessibility, Speech Recognition). Use `bash build_app.sh -f` to force a full rebuild.

```bash
bash build_app.sh
```

**Build Workflow**:
1. Configures `~/.voiceagent.env` with `VOICEAGENT_DIR` and `VOICEAGENT_PYTHON`.
2. Checks if Swift binaries (`speech-helper`, `llm-helper`, `VoiceAgentLauncher`) need recompiling.
3. If up to date, copies Python resources without re-signing the app bundle (preserving macOS TCC permissions).
4. If modified, compiles Swift binaries, creates `dist/VoiceAgent.app`, signs, and installs to `/Applications/VoiceAgent.app`.

---

## 5. Verification & Testing

Before committing changes, execute the test suite:

```bash
.venv/bin/python3 test_profile_tool.py
.venv/bin/python3 test_session_tool.py
.venv/bin/python3 test_model_manager.py
.venv/bin/python3 test_spoken_text.py
.venv/bin/python3 test_journal.py
.venv/bin/python3 test_working_phrase.py
```