diff --git a/AGENTS.md b/AGENTS.md index 5309b15..45e3c0a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -49,23 +49,29 @@ The project operates entirely from the app directory without depending on `~/Wor - **CLI Helper Scripts**: - `bin/voice_tool.py`: List and switch TTS voices (`python bin/voice_tool.py list`, `python bin/voice_tool.py set `). - `bin/model_tool.py`: List and switch LLM models (`python bin/model_tool.py list`, `python bin/model_tool.py set `). + - `bin/session_tool.py`: Inspect or reset active Hermes session (`python bin/session_tool.py get`, `python bin/session_tool.py reset`). + - `bin/profile_tool.py`: List and switch Hermes agent profiles (`python bin/profile_tool.py list`, `python bin/profile_tool.py set `). - `bin/web_tool.py`: Inspect files or URLs in the Companion Web UI drawer (`http://localhost:8888`). --- ## 4. Building & Packaging `/Applications/VoiceAgent.app` -Whenever python or Swift sources are modified, update the standalone macOS application bundle: +`VoiceAgentLauncher` runs Python scripts dynamically from the project workspace (configured in `~/.voiceagent.env` or `VOICEAGENT_DIR`). + +- **Python code edits take effect immediately** without recompiling or re-signing the macOS app bundle. +- **Rebuilding binaries** (`bash build_app.sh`) is only required when modifying Swift sources (`swift/*.swift`), helper binaries, or `Info.plist` entitlements. +- `build_app.sh` automatically updates `~/.voiceagent.env` and skips binary re-signing if Swift sources are unchanged, preserving macOS privacy permissions (Microphone, Accessibility, Speech Recognition). Use `bash build_app.sh -f` to force a full rebuild. ```bash bash build_app.sh ``` **Build Workflow**: -1. Builds Swift binaries (`swift/build.sh` -> `speech-helper`, `llm-helper`, `VoiceAgentLauncher`). -2. Bundles Python source files, `bin/` tools, and assets into `dist/VoiceAgent.app`. -3. Signs the app bundle (`codesign -s - --deep --force`). -4. Replaces `/Applications/VoiceAgent.app`. +1. Configures `~/.voiceagent.env` with `VOICEAGENT_DIR` and `VOICEAGENT_PYTHON`. +2. Checks if Swift binaries (`speech-helper`, `llm-helper`, `VoiceAgentLauncher`) need recompiling. +3. If up to date, copies Python resources without re-signing the app bundle (preserving macOS TCC permissions). +4. If modified, compiles Swift binaries, creates `dist/VoiceAgent.app`, signs, and installs to `/Applications/VoiceAgent.app`. --- @@ -74,6 +80,8 @@ bash build_app.sh Before committing changes, execute the test suite: ```bash +.venv/bin/python3 test_profile_tool.py +.venv/bin/python3 test_session_tool.py .venv/bin/python3 test_model_manager.py .venv/bin/python3 test_spoken_text.py .venv/bin/python3 test_journal.py diff --git a/README.md b/README.md index dfe0434..777c42d 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,41 @@ terminal under System Settings → Privacy & Security → Microphone. The speech model downloads itself the first time a locale is used. +## macOS AirPods route switching checklist + +When no audio device is pinned, VoiceAgent follows macOS's current default +microphone and speaker while a conversation is running. Use this checklist when +validating AirPods or another Bluetooth headset: + +1. Connect AirPods before starting `./talk`; confirm that both microphone and + speaker audio use them. +2. Start a conversation on the built-in microphone and speakers, then connect + AirPods. Confirm that each available direction moves independently to the + new macOS default without ending the conversation. +3. Disconnect AirPods during a conversation. Confirm that the available input + and output return to the macOS defaults and that the conversation remains + usable. +4. In Control Center or Sound settings, explicitly switch back to the built-in + microphone and speakers while AirPods remain connected. Confirm both routes + follow those defaults. +5. Repeat connect, disconnect, and manual default changes several times to + catch delayed Bluetooth profile changes or a stale route listener. + +Run this validation without `--input-device` or `--output-device`: omitted +values intentionally follow macOS defaults. Supplying either flag pins only that +direction (for example, a pinned USB microphone still allows an unpinned output +to follow AirPods); pinning both directions disables automatic route following. +Use `./talk --list-devices` to identify a device by index or name substring. + +This behavior depends on macOS Core Audio notifications and the device profile +currently exposed by Bluetooth. A headset may briefly expose only output, or +reject the negotiated sample rate while switching profiles; VoiceAgent keeps the +previous working route when a replacement cannot open. The terminal/app still +needs macOS Microphone permission, and the global hold-key needs Input +Monitoring. A sandboxed packaged app also needs the appropriate microphone usage +description and audio-input entitlement; those platform permissions cannot be +granted by route switching code. + ## Turn taking, and why it's push-to-talk Detecting the end of a turn by listening for silence is both slow and wrong diff --git a/audio_device_monitor.py b/audio_device_monitor.py new file mode 100644 index 0000000..81f633d --- /dev/null +++ b/audio_device_monitor.py @@ -0,0 +1,357 @@ +"""macOS default audio-device monitoring. + +The monitor deliberately depends on a small backend protocol. Core Audio invokes +listeners on an arbitrary thread, while VoiceAgent consumes snapshots on its +asyncio loop; :class:`AudioDeviceMonitor` is the lifecycle and thread boundary +between those two worlds. The backend can be replaced by a deterministic fake +in unit tests. +""" + +from __future__ import annotations + +import asyncio +import ctypes +import ctypes.util +import inspect +import sys +from dataclasses import dataclass, field +from enum import Enum +from typing import Callable, Protocol + + +class DeviceChangeReason(str, Enum): + INITIAL = "initial" + DEFAULT_CHANGED = "default_changed" + DEVICE_ADDED = "device_added" + DEVICE_REMOVED = "device_removed" + DEVICE_RECONFIGURED = "device_reconfigured" + PROFILE_CHANGED = "profile_changed" + + +@dataclass(frozen=True) +class AudioDevice: + uid: str + name: str + can_input: bool + can_output: bool + transport: str = "unknown" + alive: bool = True + + +@dataclass(frozen=True) +class AudioDeviceSnapshot: + generation: int + default_input_uid: str | None + default_output_uid: str | None + devices: dict[str, AudioDevice] = field(default_factory=dict) + reason: DeviceChangeReason = DeviceChangeReason.INITIAL + + +Listener = Callable[[], None] + + +class AudioDeviceBackend(Protocol): + """The minimal Core Audio surface required by the monitor.""" + + def enumerate_devices(self) -> list[AudioDevice]: ... + def default_input_uid(self) -> str | None: ... + def default_output_uid(self) -> str | None: ... + def add_listener(self, callback: Listener) -> object: ... + def remove_listener(self, token: object) -> None: ... + + +class AudioDeviceMonitor: + """Coalesces Core Audio notifications and dispatches snapshots on one loop.""" + + def __init__(self, backend: AudioDeviceBackend, *, debounce_seconds: float = 0.05, + loop: asyncio.AbstractEventLoop | None = None): + self._backend = backend + self._debounce_seconds = debounce_seconds + self._loop = loop + self._callback: Callable[[AudioDeviceSnapshot], object] | None = None + self._snapshot = AudioDeviceSnapshot(0, None, None) + self._generation = 0 + self._listener_tokens: list[object] = [] + self._debounce_handle: asyncio.TimerHandle | None = None + self._running = False + self._refresh_scheduled = False + self._refresh_task: asyncio.Task[None] | None = None + + async def start(self, on_change: Callable[[AudioDeviceSnapshot], object]) -> None: + """Register listeners once and publish an initial snapshot. + + ``on_change`` may be synchronous or return an awaitable. It is always + called on the loop used by this monitor, never on a Core Audio thread. + """ + if self._running: + return + self._loop = self._loop or asyncio.get_running_loop() + self._callback = on_change + self._running = True + try: + # Register all listeners before the initial read so a concurrent + # device change cannot be missed. + self._listener_tokens = [self._backend.add_listener(self._on_backend_event) for _ in range(1)] + await self._refresh(DeviceChangeReason.INITIAL) + except Exception: + await self.stop() + raise + + async def stop(self) -> None: + """Stop idempotently and ensure no callback can be scheduled afterward.""" + self._running = False + if self._debounce_handle: + self._debounce_handle.cancel() + self._debounce_handle = None + current_task = asyncio.current_task() + if (self._refresh_task and not self._refresh_task.done() + and self._refresh_task is not current_task): + self._refresh_task.cancel() + try: + await self._refresh_task + except asyncio.CancelledError: + pass + self._refresh_task = None + tokens, self._listener_tokens = self._listener_tokens, [] + for token in tokens: + self._backend.remove_listener(token) + self._callback = None + self._refresh_scheduled = False + + def snapshot(self) -> AudioDeviceSnapshot: + return self._snapshot + + def _on_backend_event(self) -> None: + """Core Audio callback entry point; safe to call from any thread.""" + if not self._running or not self._loop: + return + self._loop.call_soon_threadsafe(self._schedule_refresh) + + def _schedule_refresh(self) -> None: + if not self._running or self._refresh_scheduled: + return + self._refresh_scheduled = True + if self._debounce_handle: + self._debounce_handle.cancel() + self._debounce_handle = self._loop.call_later(self._debounce_seconds, self._start_refresh) + + def _start_refresh(self) -> None: + self._debounce_handle = None + self._refresh_scheduled = False + if self._running: + self._refresh_task = self._loop.create_task(self._refresh(self._infer_reason())) + self._refresh_task.add_done_callback(self._consume_refresh_failure) + + @staticmethod + def _consume_refresh_failure(task: asyncio.Task[None]) -> None: + """Retrieve scheduled refresh failures after cleanup has completed.""" + if not task.cancelled(): + task.exception() + + def _infer_reason(self) -> DeviceChangeReason: + # The backend intentionally keeps the callback payload-free. The + # resulting snapshot is authoritative; callers can inspect UID/device + # differences. A generic device reconfiguration is safest here. + return DeviceChangeReason.DEFAULT_CHANGED + + async def _refresh(self, reason: DeviceChangeReason) -> None: + if not self._running: + return + devices = {d.uid: d for d in self._backend.enumerate_devices() if d.uid and d.alive} + new_input = self._backend.default_input_uid() + new_output = self._backend.default_output_uid() + # An unavailable default is represented as None rather than a stale UID. + if new_input not in devices or not devices[new_input].can_input: + new_input = None + if new_output not in devices or not devices[new_output].can_output: + new_output = None + old = self._snapshot + if (old.default_input_uid == new_input and old.default_output_uid == new_output + and old.devices == devices and old.generation != 0): + return + if old.generation and reason != DeviceChangeReason.INITIAL: + old_uids, new_uids = set(old.devices), set(devices) + if old.default_input_uid != new_input or old.default_output_uid != new_output: + reason = DeviceChangeReason.DEFAULT_CHANGED + elif new_uids - old_uids: + reason = DeviceChangeReason.DEVICE_ADDED + elif old_uids - new_uids: + reason = DeviceChangeReason.DEVICE_REMOVED + elif any(old.devices[uid] != device for uid, device in devices.items() + if uid in old.devices): + reason = DeviceChangeReason.PROFILE_CHANGED + self._generation += 1 + self._snapshot = AudioDeviceSnapshot(self._generation, new_input, new_output, devices, reason) + callback = self._callback + if callback and self._running: + try: + result = callback(self._snapshot) + if inspect.isawaitable(result): + await result + except Exception: + # A failed consumer must not leave a native listener active with + # an unusable callback. ``stop`` handles the current refresh + # task specially so this cleanup is safe from inside _refresh. + await self.stop() + raise + + +class MacOSCoreAudioBackend: + """Core Audio backend hook. + + PyObjC's CoreAudio listener ABI differs between macOS releases. Keeping the + native adapter behind this class lets packaging provide the matching adapter + without exposing it to the async monitor or its tests. + """ + + def __init__(self, adapter): + self._adapter = adapter + + def enumerate_devices(self) -> list[AudioDevice]: + return list(self._adapter.enumerate_devices()) + + def default_input_uid(self) -> str | None: + return self._adapter.default_input_uid() + + def default_output_uid(self) -> str | None: + return self._adapter.default_output_uid() + + def add_listener(self, callback: Listener) -> object: + return self._adapter.add_device_listener(callback) + + def remove_listener(self, token: object) -> None: + self._adapter.remove_device_listener(token) + + +class NativeMacOSCoreAudioAdapter: + """Native Core Audio adapter using stable device UIDs, never PortAudio IDs.""" + + _SYSTEM_OBJECT = 1 + _GLOBAL = int.from_bytes(b"glob", "big") + _INPUT_SCOPE = int.from_bytes(b"inpt", "big") + _OUTPUT_SCOPE = int.from_bytes(b"outp", "big") + _DEFAULT_INPUT = int.from_bytes(b"dIn ", "big") + _DEFAULT_OUTPUT = int.from_bytes(b"dOut", "big") + _DEVICES = int.from_bytes(b"dev#", "big") + _UID = int.from_bytes(b"uid ", "big") + _NAME = int.from_bytes(b"lnam", "big") + _ALIVE = int.from_bytes(b"livn", "big") + _TRANSPORT = int.from_bytes(b"tran", "big") + _STREAMS = int.from_bytes(b"stm#", "big") + + class _Address(ctypes.Structure): + _fields_ = [("selector", ctypes.c_uint32), ("scope", ctypes.c_uint32), ("element", ctypes.c_uint32)] + + def __init__(self): + if sys.platform != "darwin": + raise RuntimeError("Core Audio is only available on macOS") + core_audio = ctypes.util.find_library("CoreAudio") + core_foundation = ctypes.util.find_library("CoreFoundation") + if not core_audio or not core_foundation: + raise RuntimeError("CoreAudio.framework is unavailable") + self._lib = ctypes.CDLL(core_audio) + self._cf = ctypes.CDLL(core_foundation) + self._listener_type = ctypes.CFUNCTYPE(ctypes.c_int32, ctypes.c_uint32, ctypes.c_uint32, + ctypes.POINTER(self._Address), ctypes.c_void_p) + self._lib.AudioObjectGetPropertyData.argtypes = [ctypes.c_uint32, ctypes.POINTER(self._Address), ctypes.c_uint32, ctypes.c_void_p, ctypes.POINTER(ctypes.c_uint32), ctypes.c_void_p] + self._lib.AudioObjectGetPropertyData.restype = ctypes.c_int32 + self._lib.AudioObjectGetPropertyDataSize.argtypes = [ctypes.c_uint32, ctypes.POINTER(self._Address), ctypes.c_uint32, ctypes.c_void_p, ctypes.POINTER(ctypes.c_uint32)] + self._lib.AudioObjectGetPropertyDataSize.restype = ctypes.c_int32 + self._lib.AudioObjectAddPropertyListener.argtypes = [ctypes.c_uint32, ctypes.POINTER(self._Address), self._listener_type, ctypes.c_void_p] + self._lib.AudioObjectRemovePropertyListener.argtypes = [ctypes.c_uint32, ctypes.POINTER(self._Address), self._listener_type, ctypes.c_void_p] + self._cf.CFStringGetCString.argtypes = [ctypes.c_void_p, ctypes.c_char_p, ctypes.c_long, ctypes.c_uint32] + self._callbacks = {} + + def _address(self, selector, scope=None): + return self._Address(selector, self._GLOBAL if scope is None else scope, 0) + + def _value(self, object_id, selector, value_type, scope=None): + value, size = value_type(), ctypes.c_uint32(ctypes.sizeof(value_type)) + status = self._lib.AudioObjectGetPropertyData(object_id, ctypes.byref(self._address(selector, scope)), 0, None, ctypes.byref(size), ctypes.byref(value)) + if status: + raise OSError(f"AudioObjectGetPropertyData({selector}) failed: {status}") + return value.value + + def _string(self, object_id, selector): + ref = self._value(object_id, selector, ctypes.c_void_p) + if not ref: + return "" + buffer = ctypes.create_string_buffer(1024) + if not self._cf.CFStringGetCString(ref, buffer, len(buffer), 0x08000100): + return "" + return buffer.value.decode("utf-8", "replace") + + def _device_ids(self): + address, size = self._address(self._DEVICES), ctypes.c_uint32() + status = self._lib.AudioObjectGetPropertyDataSize(self._SYSTEM_OBJECT, ctypes.byref(address), 0, None, ctypes.byref(size)) + if status: + raise OSError(f"Audio device enumeration failed: {status}") + devices = (ctypes.c_uint32 * (size.value // ctypes.sizeof(ctypes.c_uint32)))() + status = self._lib.AudioObjectGetPropertyData(self._SYSTEM_OBJECT, ctypes.byref(address), 0, None, ctypes.byref(size), devices) + if status: + raise OSError(f"Audio device enumeration failed: {status}") + return list(devices) + + def _has_streams(self, device_id, scope): + address, size = self._address(self._STREAMS, scope), ctypes.c_uint32() + status = self._lib.AudioObjectGetPropertyDataSize(device_id, ctypes.byref(address), 0, None, ctypes.byref(size)) + return not status and bool(size.value) + + def enumerate_devices(self): + devices = [] + for device_id in self._device_ids(): + try: + uid = self._string(device_id, self._UID) + if not uid: + continue + devices.append(AudioDevice(uid, self._string(device_id, self._NAME) or uid, + self._has_streams(device_id, self._INPUT_SCOPE), self._has_streams(device_id, self._OUTPUT_SCOPE), + self._fourcc(self._value(device_id, self._TRANSPORT, ctypes.c_uint32)), bool(self._value(device_id, self._ALIVE, ctypes.c_uint32)))) + except OSError: + continue + return devices + + @staticmethod + def _fourcc(value): + return value.to_bytes(4, "big").decode("ascii", "replace").strip() or "unknown" + + def default_input_uid(self): + return self._uid_for_id(self._value(self._SYSTEM_OBJECT, self._DEFAULT_INPUT, ctypes.c_uint32)) + + def default_output_uid(self): + return self._uid_for_id(self._value(self._SYSTEM_OBJECT, self._DEFAULT_OUTPUT, ctypes.c_uint32)) + + def _uid_for_id(self, device_id): + return self._string(device_id, self._UID) if device_id else None + + def add_device_listener(self, callback): + addresses = [(self._SYSTEM_OBJECT, self._address(selector)) for selector in (self._DEFAULT_INPUT, self._DEFAULT_OUTPUT, self._DEVICES)] + addresses += [(device_id, self._address(selector, scope)) for device_id in self._device_ids() for selector, scope in ((self._ALIVE, None), (self._STREAMS, self._INPUT_SCOPE), (self._STREAMS, self._OUTPUT_SCOPE))] + native_callback = self._listener_type(lambda *_: (callback(), 0)[1]) + registered = [] + try: + for object_id, address in addresses: + status = self._lib.AudioObjectAddPropertyListener(object_id, ctypes.byref(address), native_callback, None) + if status: + raise OSError(f"AudioObjectAddPropertyListener failed: {status}") + registered.append((object_id, address)) + except Exception: + for object_id, address in registered: + self._lib.AudioObjectRemovePropertyListener(object_id, ctypes.byref(address), native_callback, None) + raise + token = (native_callback, registered) + self._callbacks[id(token)] = token + return token + + def remove_device_listener(self, token): + native_callback, addresses = token + for object_id, address in addresses: + self._lib.AudioObjectRemovePropertyListener(object_id, ctypes.byref(address), native_callback, None) + self._callbacks.pop(id(token), None) + + +def create_macos_audio_monitor(*, adapter=None, **kwargs) -> AudioDeviceMonitor: + """Create a native Core Audio monitor (or a supplied test adapter).""" + if adapter is None: + adapter = NativeMacOSCoreAudioAdapter() + return AudioDeviceMonitor(MacOSCoreAudioBackend(adapter), **kwargs) diff --git a/bin/audio_tool.py b/bin/audio_tool.py new file mode 100644 index 0000000..a754723 --- /dev/null +++ b/bin/audio_tool.py @@ -0,0 +1,56 @@ +#!/usr/bin/env python3 +"""Inspect or switch VoiceAgent's live input/output device through its local UI API.""" + +import argparse +import json +import sys +from urllib.error import HTTPError, URLError +from urllib.request import Request, urlopen + +BASE_URL = "http://127.0.0.1:8888" + + +def request(path, method="GET", payload=None): + data = json.dumps(payload).encode() if payload is not None else None + req = Request(BASE_URL + path, data=data, method=method) + if data is not None: + req.add_header("Content-Type", "application/json") + try: + with urlopen(req, timeout=5) as response: + return json.load(response) + except HTTPError as exc: + try: + message = json.load(exc) + except Exception: + message = {"error": exc.read().decode("utf-8", "replace")} + raise RuntimeError(message.get("error", str(exc))) from exc + except URLError as exc: + raise RuntimeError(f"VoiceAgent is not reachable at {BASE_URL}: {exc.reason}") from exc + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + commands = parser.add_subparsers(dest="command", required=True) + commands.add_parser("list", help="List selectable input/output devices.") + select = commands.add_parser("set", help="Switch one live route or follow the macOS default.") + select.add_argument("direction", choices=("input", "output")) + select.add_argument("device", help="Unique device-name substring, PortAudio index, or 'default'.") + args = parser.parse_args() + try: + if args.command == "list": + result = request("/api/audio-devices") + for device in result["devices"]: + kinds = "/".join(kind for kind in ("input" if device["input"] else "", "output" if device["output"] else "") if kind) + print(f"[{device['id']}] {device['name']} ({kinds})") + else: + result = request("/api/audio-device", "POST", {"direction": args.direction, "device": args.device}) + mode = "following macOS default" if result["following_default"] else f"pinned to [{result['device']}]" + print(f"{args.direction} changed to {result['name']} ({mode})") + return 0 + except (KeyError, RuntimeError) as exc: + print(f"audio-tool: {exc}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/bin/profile_tool.py b/bin/profile_tool.py new file mode 100644 index 0000000..f62de5c --- /dev/null +++ b/bin/profile_tool.py @@ -0,0 +1,89 @@ +#!/usr/bin/env python3 +"""CLI helper to list, get, and switch Hermes agent profiles for VoiceAgent.""" + +import os +import re +import shutil +import subprocess +import sys +from pathlib import Path + +# Add VoiceAgent1 project root to sys.path +project_root = Path(__file__).resolve().parent.parent +if str(project_root) not in sys.path: + sys.path.insert(0, str(project_root)) + +from hermes_llm import find_hermes_cli + + +def _get_hermes_cli() -> str: + cli = find_hermes_cli() + if not cli: + raise RuntimeError("Hermes CLI binary not found in PATH or standard paths") + return cli + + +def list_profiles() -> str: + cli = _get_hermes_cli() + res = subprocess.run([cli, "profile", "list"], capture_output=True, text=True) + if res.returncode != 0: + return f"Error listing profiles: {res.stderr.strip()}" + return res.stdout.strip() + + +def get_current_profile() -> str: + cli = _get_hermes_cli() + res = subprocess.run([cli, "profile", "list"], capture_output=True, text=True) + if res.returncode == 0: + for line in res.stdout.splitlines(): + line_clean = line.strip() + if line_clean.startswith("◆") or line_clean.startswith("*"): + parts = line_clean.lstrip("◆* ").split() + if parts: + return parts[0] + return "default" + + +def set_profile(profile_name: str) -> tuple[bool, str]: + target = profile_name.strip().lstrip("◆* ") + if not target: + return False, "No profile name specified." + + cli = _get_hermes_cli() + res = subprocess.run([cli, "profile", "use", target], capture_output=True, text=True) + if res.returncode == 0: + output_msg = res.stdout.strip() or f"Switched to Hermes profile '{target}'." + try: + import web_server + web_server.broadcast_event("status_change", {"profile": target}) + except Exception: + pass + return True, output_msg + else: + err_msg = res.stderr.strip() or res.stdout.strip() or f"Failed to set profile '{target}'." + return False, err_msg + + +def main(): + if len(sys.argv) < 2 or sys.argv[1] in ("list", "ls", "--list"): + print(list_profiles()) + return + + action = sys.argv[1].lower() + if action in ("get", "current", "show"): + print(f"Active Hermes Profile: {get_current_profile()}") + return + + if action in ("set", "use", "change") and len(sys.argv) >= 3: + target = sys.argv[2] + ok, msg = set_profile(target) + print(msg) + else: + # Treat single argument as target profile + target = sys.argv[1] + ok, msg = set_profile(target) + print(msg) + + +if __name__ == "__main__": + main() diff --git a/bin/session_tool.py b/bin/session_tool.py new file mode 100644 index 0000000..4f1b19e --- /dev/null +++ b/bin/session_tool.py @@ -0,0 +1,78 @@ +#!/usr/bin/env python3 +"""CLI helper to inspect and reset Hermes voice agent sessions.""" + +import json +import sys +from pathlib import Path + +# Add VoiceAgent1 project root to sys.path +project_root = Path(__file__).resolve().parent.parent +if str(project_root) not in sys.path: + sys.path.insert(0, str(project_root)) + +SESSION_FILE_NAME = ".hermes-voice-session.json" + + +def get_session_file(app_dir: Path | None = None) -> Path: + base_dir = app_dir or project_root + return base_dir / SESSION_FILE_NAME + + +def get_active_session_id(app_dir: Path | None = None) -> str | None: + session_file = get_session_file(app_dir) + if session_file.exists(): + try: + data = json.loads(session_file.read_text()) + sid = data.get("session_id") + if sid and isinstance(sid, str): + return sid.strip() + except Exception: + pass + return None + + +def reset_session(app_dir: Path | None = None) -> tuple[bool, str]: + session_file = get_session_file(app_dir) + deleted = False + if session_file.exists(): + try: + session_file.unlink() + deleted = True + except Exception as e: + return False, f"Could not remove session file: {e}" + + msg = "Session reset successfully. A fresh Hermes session will start on the next turn." + if not deleted: + msg = "No active session file found. Next turn will start with a fresh session." + + try: + import web_server + web_server.broadcast_event("session_reset", {"message": msg}) + except Exception: + pass + + return True, msg + + +def main(): + app_dir = project_root + + if len(sys.argv) < 2 or sys.argv[1] in ("get", "info", "current", "show"): + sid = get_active_session_id(app_dir) + if sid: + print(f"Active Session ID: {sid}") + else: + print("No active Hermes session (a new session will start on the next turn).") + return + + action = sys.argv[1].lower() + if action in ("reset", "new", "clear"): + ok, msg = reset_session(app_dir) + print(msg) + else: + print(f"Unknown action: {sys.argv[1]}. Usage: python bin/session_tool.py [get|reset]") + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/bot.py b/bot.py index ddd3998..96e71be 100644 --- a/bot.py +++ b/bot.py @@ -82,6 +82,9 @@ say them: - Use your available tools (listing directories, searching, reading files, shell execution) whenever the user asks about files, commands, CLI tools (such as Paseo), or workspace tasks. - You can change your own voice! If the user asks to list available voices or switch voice, run `python bin/voice_tool.py list` or `python bin/voice_tool.py set ` (voices: af_heart, af_bella, am_michael, am_fenrir, am_puck, bf_emma, bm_george, Moira, Daniel). - You can change your AI model on the fly! If the user asks to list available models or change model, run `python bin/model_tool.py list` or `python bin/model_tool.py set ` (models: luna, gemma, deepseek, gpt-oss, sonnet, etc.). +- You can reset or start a fresh conversation session! If the user asks to start a fresh session, reset the conversation, or clear session context, run `python bin/session_tool.py reset`. +- You can switch Hermes agent profiles! If the user asks to list Hermes profiles or switch profile, run `python bin/profile_tool.py list` or `python bin/profile_tool.py set `. +- You can change the running microphone and speakers independently. For requests such as “use AirPods”, “switch to Mac speakers”, or “use the Mac default mic”, run `python bin/audio_tool.py list` then `python bin/audio_tool.py set input|output `. Report the command result plainly; do not claim a device changed if the tool says it is unavailable. - You can open files visually for the user in the Companion Web UI drawer! Run `python bin/web_tool.py show `. - You can open links or the Companion Web UI in the default browser! Run `python bin/web_tool.py open `. - Keep implementation details and tool activity silent in the spoken channel. The user can see technical progress in the logs or Companion Web UI; only speak the useful conversational response. @@ -571,6 +574,21 @@ async def main() -> int: logger.info(f"Created {workspace}") logger.info(f"Workspace: {workspace}") + async def on_audio_device_event(snapshot) -> None: + """Keep native route changes observable without touching conversation state.""" + logger.info( + "Audio device event " + f"generation={snapshot.generation} reason={snapshot.reason.value} " + f"input={snapshot.default_input_uid!r} output={snapshot.default_output_uid!r}" + ) + if not getattr(args, "no_web", False): + web_server.broadcast_event("audio_device", { + "generation": snapshot.generation, + "reason": snapshot.reason.value, + "input_uid": snapshot.default_input_uid, + "output_uid": snapshot.default_output_uid, + }) + transport = SoundDeviceTransport( SoundDeviceTransportParams( audio_in_enabled=True, @@ -579,7 +597,8 @@ async def main() -> int: audio_out_sample_rate=TTS_SAMPLE_RATE, input_device=as_device(args.input_device), output_device=as_device(args.output_device), - ) + ), + device_event_sink=on_audio_device_event, ) brain = None @@ -603,7 +622,7 @@ async def main() -> int: model_manager = ModelManager(workspace) if not getattr(args, "no_web", False): await web_server.start_server(workspace, port=getattr(args, "web_port", 8888)) - web_server.set_managers(workspace, model_manager, voice_manager) + web_server.set_managers(workspace, model_manager, voice_manager, audio_controller=transport) llm = build_llm( args, @@ -674,7 +693,7 @@ async def main() -> int: await worker.queue_frames(frames) if not getattr(args, "no_web", False): - web_server.set_managers(workspace, model_manager, voice_manager, input_callback=on_web_input) + web_server.set_managers(workspace, model_manager, voice_manager, input_callback=on_web_input, audio_controller=transport) if args.greeting: await worker.queue_frames([TTSSpeakFrame(args.greeting)]) diff --git a/build_app.sh b/build_app.sh index 9cffcea..29f9d6b 100644 --- a/build_app.sh +++ b/build_app.sh @@ -5,17 +5,84 @@ set -euo pipefail HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" cd "$HERE" -echo "=== 1. Building Swift Helper & Launcher Binaries ===" -bash swift/build.sh +FORCE=0 +for arg in "$@"; do + if [ "$arg" = "-f" ] || [ "$arg" = "--force" ]; then + FORCE=1 + fi +done APP_NAME="VoiceAgent" DIST_DIR="$HERE/dist" APP_BUNDLE="$DIST_DIR/$APP_NAME.app" +INSTALLED_APP="/Applications/VoiceAgent.app" CONTENTS_DIR="$APP_BUNDLE/Contents" MACOS_DIR="$CONTENTS_DIR/MacOS" RESOURCES_DIR="$CONTENTS_DIR/Resources" SRC_DIR="$RESOURCES_DIR/src" +ENV_FILE="$HOME/.voiceagent.env" +echo "=== 0. Updating Local Environment Config ($ENV_FILE) ===" +mkdir -p "$(dirname "$ENV_FILE")" +touch "$ENV_FILE" + +# Helper to update or append key=val in .env file +set_env_var() { + local key="$1" + local val="$2" + if grep -q "^${key}=" "$ENV_FILE" 2>/dev/null; then + # Replace existing line using python helper for clean string replacement + python3 -c " +import sys, re +path, k, v = sys.argv[1], sys.argv[2], sys.argv[3] +with open(path, 'r') as f: content = f.read() +new_content = re.sub(r'^' + re.escape(k) + r'=.*$', f'{k}={v}', content, flags=re.MULTILINE) +with open(path, 'w') as f: f.write(new_content) +" "$ENV_FILE" "$key" "$val" + else + echo "${key}=${val}" >> "$ENV_FILE" + fi +} + +set_env_var "VOICEAGENT_DIR" "$HERE" +set_env_var "VOICEAGENT_PYTHON" "$HERE/.venv/bin/python" +echo "Configured VOICEAGENT_DIR=$HERE in $ENV_FILE" + +echo "=== 1. Building Swift Helper Binaries ===" +bash swift/build.sh "$@" + +INSTALLED_LAUNCHER="$INSTALLED_APP/Contents/MacOS/VoiceAgent" +NEED_REBUILD=0 + +if [ "$FORCE" -eq 1 ] || [ ! -f "$INSTALLED_LAUNCHER" ]; then + NEED_REBUILD=1 +elif [ "$HERE/swift/VoiceAgentLauncher.swift" -nt "$INSTALLED_LAUNCHER" ]; then + NEED_REBUILD=1 +elif [ "$HERE/swift/SpeechHelper.swift" -nt "$INSTALLED_LAUNCHER" ]; then + NEED_REBUILD=1 +fi + +if [ "$NEED_REBUILD" -eq 0 ]; then + echo "=== [SKIPPED BINARY REBUILD & CODESIGN] ===" + echo "Native launcher binary is up to date." + echo "Copying updated python resources without modifying code signature..." + + mkdir -p "$SRC_DIR" + cp "$HERE"/*.py "$SRC_DIR/" 2>/dev/null || true + cp -R "$HERE/bin" "$SRC_DIR/" 2>/dev/null || true + + if [ -d "$INSTALLED_APP/Contents/Resources/src" ]; then + cp "$HERE"/*.py "$INSTALLED_APP/Contents/Resources/src/" 2>/dev/null || true + cp -R "$HERE/bin" "$INSTALLED_APP/Contents/Resources/src/" 2>/dev/null || true + fi + + echo "==========================================================" + echo "Python changes deployed cleanly! App binary signature unchanged." + echo "macOS permissions preserved for: $INSTALLED_APP" + echo "==========================================================" + exit 0 +fi + echo "=== 2. Creating macOS App Bundle Structure ===" rm -rf "$APP_BUNDLE" mkdir -p "$MACOS_DIR" @@ -77,12 +144,12 @@ echo "=== 6. Code-signing App Bundle ===" codesign -s - --deep --force "$APP_BUNDLE" echo "=== 7. Installing to /Applications ===" -rm -rf /Applications/VoiceAgent.app +rm -rf "$INSTALLED_APP" cp -R "$APP_BUNDLE" /Applications/ -codesign -s - --deep --force /Applications/VoiceAgent.app +codesign -s - --deep --force "$INSTALLED_APP" echo "==========================================================" echo "Successfully built and installed VoiceAgent.app to:" -echo "1. /Applications/VoiceAgent.app" +echo "1. $INSTALLED_APP" echo "2. $APP_BUNDLE" echo "==========================================================" diff --git a/hermes_llm.py b/hermes_llm.py index 257a8eb..e888074 100644 --- a/hermes_llm.py +++ b/hermes_llm.py @@ -95,16 +95,24 @@ def find_hermes_cli() -> str | None: return shutil.which("hermes") -async def ensure_hermes_server(port: int = 8642) -> tuple[bool, str]: - """Ensure Hermes gateway server daemon is available on port.""" +async def check_hermes_server_active(port: int = 8642) -> tuple[bool, str]: + """Check if Hermes gateway server daemon is responding to health requests.""" url = f"http://localhost:{port}/api/health" try: - async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=2.0)) as session: + async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=1.5)) as session: async with session.get(url) as resp: if resp.status == 200: return True, f"Hermes server active on http://localhost:{port}" except Exception: pass + return False, "Hermes server daemon not active" + + +async def ensure_hermes_server(port: int = 8642) -> tuple[bool, str]: + """Ensure Hermes gateway server daemon or CLI binary is available.""" + active, msg = await check_hermes_server_active(port) + if active: + return True, msg cli = find_hermes_cli() if not cli: @@ -146,6 +154,7 @@ class HermesLLM(FrameProcessor): self._cli_path = find_hermes_cli() or "hermes" self._use_server = use_server self._session_renamed = False + self._http_session: aiohttp.ClientSession | None = None # Keep the conversation lineage with the workspace. A single global # session file can make two voice-agent workspaces resume each other's @@ -157,6 +166,37 @@ class HermesLLM(FrameProcessor): if self._session_id: logger.info(f"Loaded existing Hermes session ID: {self._session_id}") + def reset_session(self): + """Reset active session so a fresh session starts on the next turn.""" + logger.info("Resetting active Hermes session state...") + self._session_id = None + self._session_renamed = False + self._history.clear() + try: + if self._session_state_file.exists(): + self._session_state_file.unlink() + except Exception as e: + logger.warning(f"Could not remove session file during reset: {e}") + + def _sync_disk_session(self): + """Sync in-memory session ID with disk file state prior to each turn.""" + disk_sid = self._load_session_id() + if disk_sid != self._session_id: + logger.info(f"Hermes session state updated from disk: {self._session_id} -> {disk_sid}") + self._session_id = disk_sid + self._session_renamed = False + self._history.clear() + + async def _get_http_session(self) -> aiohttp.ClientSession: + if self._http_session is None or self._http_session.closed: + self._http_session = aiohttp.ClientSession() + return self._http_session + + async def _close_http_session(self): + if self._http_session and not self._http_session.closed: + await self._http_session.close() + self._http_session = None + def _load_session_id(self) -> str | None: if self._session_state_file.exists(): try: @@ -219,6 +259,7 @@ class HermesLLM(FrameProcessor): logger.info(f"Hermes LLM engine initialized: {reason}") elif isinstance(frame, (EndFrame, CancelFrame)): await self._cancel_turn() + await self._close_http_session() await self.push_frame(frame, direction) elif isinstance(frame, InterruptionFrame): await self._cancel_turn() @@ -289,6 +330,7 @@ class HermesLLM(FrameProcessor): async def _run_turn(self, utterance: str): self._sync_disk_model() + self._sync_disk_session() self._history.append({"role": "user", "content": utterance}) await self.push_frame(LLMFullResponseStartFrame()) @@ -313,7 +355,7 @@ class HermesLLM(FrameProcessor): async def _run_turn_server(self, utterance: str, chunks: list[str]): """Run turn via Hermes Server / Gateway HTTP API if available.""" try: - ok, _ = await ensure_hermes_server(self._port) + ok, _ = await check_hermes_server_active(self._port) if not ok: raise RuntimeError("Hermes server daemon unavailable") @@ -325,8 +367,8 @@ class HermesLLM(FrameProcessor): if self._model and self._model.lower() not in ("default", "none", ""): payload["model"] = self._model - async with aiohttp.ClientSession() as session: - async with session.post(url, json=payload) as resp: + session = await self._get_http_session() + async with session.post(url, json=payload) as resp: if resp.status == 200: data = await resp.json() text_val = data.get("reply") or data.get("text") or data.get("content", "") diff --git a/sounddevice_transport.py b/sounddevice_transport.py index 0f77e76..3639c8d 100644 --- a/sounddevice_transport.py +++ b/sounddevice_transport.py @@ -8,6 +8,8 @@ identical to the upstream transport. """ import asyncio +import inspect +import sys from concurrent.futures import ThreadPoolExecutor import sounddevice as sd @@ -18,6 +20,7 @@ from pipecat.processors.frame_processor import FrameProcessor from pipecat.transports.base_input import BaseInputTransport from pipecat.transports.base_output import BaseOutputTransport from pipecat.transports.base_transport import BaseTransport, TransportParams +from audio_device_monitor import AudioDeviceSnapshot, create_macos_audio_monitor class SoundDeviceTransportParams(TransportParams): @@ -36,11 +39,14 @@ class SoundDeviceInputTransport(BaseInputTransport): """Captures microphone audio and pushes it into the pipeline.""" _params: SoundDeviceTransportParams + _transport: "SoundDeviceTransport" def __init__(self, params: SoundDeviceTransportParams): super().__init__(params) self._in_stream: sd.RawInputStream | None = None self._sample_rate = 0 + self._stream_generation = 0 + self._stream_lock = asyncio.Lock() async def start(self, frame: StartFrame): await super().start(frame) @@ -51,29 +57,74 @@ class SoundDeviceInputTransport(BaseInputTransport): self._sample_rate = self._params.audio_in_sample_rate or frame.audio_in_sample_rate blocksize = int(self._sample_rate / 100) * 2 # 20ms - self._in_stream = sd.RawInputStream( - samplerate=self._sample_rate, - blocksize=blocksize, - device=self._params.input_device, - channels=self._params.audio_in_channels, - dtype="int16", - callback=self._audio_in_callback, - ) - self._in_stream.start() - - device_name = sd.query_devices(self._in_stream.device, "input")["name"] - logger.info(f"Microphone: {device_name} @ {self._sample_rate} Hz") + await self._open_stream() await self.set_transport_ready(frame) + if hasattr(self, "_transport"): + await self._transport.start_device_monitor(self) + + async def reopen(self, *, device=None): + """Recreate an unset-device stream so PortAudio resolves the new default.""" + if not self._in_stream: + return + async with self._stream_lock: + old_stream = self._in_stream + old_device = old_stream.device + # Fail before disrupting an otherwise healthy conversation whenever + # PortAudio can already tell us that the new default is unavailable. + sd.check_input_settings( + device=self._params.input_device if device is None else device, samplerate=self._sample_rate, + channels=self._params.audio_in_channels, dtype="int16", + ) + self._stream_generation += 1 # makes callbacks from the old stream inert + try: + old_stream.stop() + old_stream.close() + self._in_stream = None + await self._open_stream(device=device) + except Exception as exc: + logger.warning(f"Audio input route change failed; restoring prior stream: {type(exc).__name__}") + try: + self._in_stream = None + await self._open_stream(device=old_device) + except Exception as restore_exc: + logger.error(f"Audio input fallback unavailable: {type(restore_exc).__name__}") + raise + + async def _open_stream(self, *, device=None): + blocksize = int(self._sample_rate / 100) * 2 + self._stream_generation += 1 + generation = self._stream_generation + stream = sd.RawInputStream( + samplerate=self._sample_rate, blocksize=blocksize, + device=self._params.input_device if device is None else device, + channels=self._params.audio_in_channels, dtype="int16", + callback=lambda *args: self._audio_in_callback(generation, *args), + ) + try: + stream.start() + except Exception: + stream.close() + raise + self._in_stream = stream + device_name = sd.query_devices(self._in_stream.device, "input")["name"] + logger.info(f"Microphone: {device_name} @ {self._sample_rate} Hz") + async def cleanup(self): await super().cleanup() - if self._in_stream: - self._in_stream.stop() - self._in_stream.close() - self._in_stream = None + async with self._stream_lock: + self._stream_generation += 1 + if self._in_stream: + self._in_stream.stop() + self._in_stream.close() + self._in_stream = None + if hasattr(self, "_transport"): + await self._transport.stop_device_monitor(self) - def _audio_in_callback(self, indata, frame_count, time_info, status): + def _audio_in_callback(self, generation, indata, frame_count, time_info, status): + if generation != self._stream_generation: + return if status: logger.trace(f"Audio input status: {status}") @@ -83,7 +134,16 @@ class SoundDeviceInputTransport(BaseInputTransport): num_channels=self._params.audio_in_channels, ) - asyncio.run_coroutine_threadsafe(self.push_audio_frame(frame), self.get_event_loop()) + asyncio.run_coroutine_threadsafe( + self._push_audio_frame_if_current(generation, frame), self.get_event_loop() + ) + + async def _push_audio_frame_if_current(self, generation, frame): + """Serialize frame delivery with replacement so a closed route cannot leak audio.""" + async with self._stream_lock: + if generation != self._stream_generation: + return + await self.push_audio_frame(frame) class SoundDeviceOutputTransport(BaseOutputTransport): @@ -97,6 +157,7 @@ class SoundDeviceOutputTransport(BaseOutputTransport): self._sample_rate = 0 # Writes are serialized by the pipeline, so one worker is enough. self._executor = ThreadPoolExecutor(max_workers=1) + self._stream_lock = asyncio.Lock() async def start(self, frame: StartFrame): await super().start(frame) @@ -105,53 +166,258 @@ class SoundDeviceOutputTransport(BaseOutputTransport): return self._sample_rate = self._params.audio_out_sample_rate or frame.audio_out_sample_rate + await self._open_stream() - self._out_stream = sd.RawOutputStream( + await self.set_transport_ready(frame) + if hasattr(self, "_transport"): + await self._transport.start_device_monitor(self) + + async def reopen(self, *, device=None): + """Recreate an unset-device stream so PortAudio resolves the new default.""" + if not self._out_stream: + return + async with self._stream_lock: + old_stream = self._out_stream + old_device = old_stream.device + # Keep the current output route intact when the selected default + # cannot satisfy this stream's negotiated format. + sd.check_output_settings( + device=self._params.output_device if device is None else device, samplerate=self._sample_rate, + channels=self._params.audio_out_channels, dtype="int16", + ) + try: + old_stream.stop() + old_stream.close() + self._out_stream = None + await self._open_stream(device=device) + except Exception as exc: + logger.warning(f"Audio output route change failed; restoring prior stream: {type(exc).__name__}") + try: + self._out_stream = None + await self._open_stream(device=old_device) + except Exception as restore_exc: + logger.error(f"Audio output fallback unavailable: {type(restore_exc).__name__}") + raise + + async def _open_stream(self, *, device=None): + stream = sd.RawOutputStream( samplerate=self._sample_rate, - device=self._params.output_device, - channels=self._params.audio_out_channels, - dtype="int16", + device=self._params.output_device if device is None else device, + channels=self._params.audio_out_channels, dtype="int16", ) - self._out_stream.start() - + try: + stream.start() + except Exception: + stream.close() + raise + self._out_stream = stream device_name = sd.query_devices(self._out_stream.device, "output")["name"] logger.info(f"Speaker: {device_name} @ {self._sample_rate} Hz") - await self.set_transport_ready(frame) - async def cleanup(self): await super().cleanup() - if self._out_stream: - self._out_stream.stop() - self._out_stream.close() - self._out_stream = None + async with self._stream_lock: + if self._out_stream: + self._out_stream.stop() + self._out_stream.close() + self._out_stream = None + if hasattr(self, "_transport"): + await self._transport.stop_device_monitor(self) async def write_audio_frame(self, frame: OutputAudioRawFrame) -> bool: - if not self._out_stream: - return False - await self.get_event_loop().run_in_executor( - self._executor, self._out_stream.write, frame.audio - ) + async with self._stream_lock: + if not self._out_stream: + return False + await self.get_event_loop().run_in_executor( + self._executor, self._out_stream.write, frame.audio + ) return True class SoundDeviceTransport(BaseTransport): """Local microphone + speaker transport.""" - def __init__(self, params: SoundDeviceTransportParams): + def __init__(self, params: SoundDeviceTransportParams, *, device_monitor=None, + device_event_sink=None): super().__init__() self._params = params self._input: SoundDeviceInputTransport | None = None self._output: SoundDeviceOutputTransport | None = None + self._device_monitor = device_monitor + self._device_event_sink = device_event_sink + self._last_snapshot: AudioDeviceSnapshot | None = None + self._restart_lock = asyncio.Lock() + self._monitor_started = False + self._monitor_owners: set[object] = set() + self._monitor_lock = asyncio.Lock() + + def _monitor_if_needed(self): + if self._device_monitor is not None: + return self._device_monitor + if sys.platform != "darwin": + return None + if self._params.input_device is None or self._params.output_device is None: + try: + self._device_monitor = create_macos_audio_monitor() + except Exception as exc: + logger.warning(f"Audio default monitoring unavailable: {exc}") + return self._device_monitor + + async def start_device_monitor(self, owner=None): + """Keep the shared monitor running while any transport side is active.""" + owner = self if owner is None else owner + async with self._monitor_lock: + self._monitor_owners.add(owner) + if self._monitor_started: + return + monitor = self._monitor_if_needed() + if monitor: + await monitor.start(self._on_device_change) + self._monitor_started = True + + async def stop_device_monitor(self, owner=None): + """Release one transport side; stop only after the final release.""" + owner = self if owner is None else owner + async with self._monitor_lock: + self._monitor_owners.discard(owner) + if self._monitor_owners or not self._device_monitor or not self._monitor_started: + return + await self._device_monitor.stop() + self._monitor_started = False + + async def _on_device_change(self, snapshot: AudioDeviceSnapshot): + if self._last_snapshot and snapshot.generation <= self._last_snapshot.generation: + logger.debug(f"Ignoring stale audio route event generation={snapshot.generation}") + return + if self._device_event_sink: + try: + result = self._device_event_sink(snapshot) + if inspect.isawaitable(result): + await result + except Exception as exc: + logger.warning(f"Audio device event sink failed: {type(exc).__name__}") + old = self._last_snapshot + self._last_snapshot = snapshot + if old is None: + return + input_changed = old.default_input_uid != snapshot.default_input_uid + output_changed = old.default_output_uid != snapshot.default_output_uid + if not (input_changed or output_changed): + return + async with self._restart_lock: + if input_changed and self._params.input_device is None and self._input: + try: + await self._reopen_default(self._input, snapshot, snapshot.default_input_uid, "input") + except Exception as exc: + logger.warning(f"Audio input route refresh failed: {type(exc).__name__}") + if output_changed and self._params.output_device is None and self._output: + try: + await self._reopen_default(self._output, snapshot, snapshot.default_output_uid, "output") + except Exception as exc: + logger.warning(f"Audio output route refresh failed: {type(exc).__name__}") + + async def _reopen_default(self, stream, snapshot: AudioDeviceSnapshot, uid: str | None, direction: str) -> None: + # Empty snapshots are supported for legacy/injected monitors. Native + # snapshots always carry devices and therefore get an explicit index. + if not snapshot.devices: + await stream.reopen() + return + await stream.reopen(device=self._portaudio_device(snapshot, uid, direction)) + + @staticmethod + def _portaudio_device(snapshot: AudioDeviceSnapshot, uid: str | None, direction: str) -> int: + """Map Core Audio's current default to an explicit PortAudio index. + + ``device=None`` in a long-lived sounddevice process retains PortAudio's + startup default. Reopening with the current index is what makes a + default-device event actually move the live stream. + """ + if not uid or uid not in snapshot.devices: + raise RuntimeError(f"No available Core Audio default {direction} device") + native = snapshot.devices[uid] + capability = "max_input_channels" if direction == "input" else "max_output_channels" + matches = [index for index, candidate in enumerate(sd.query_devices()) + if candidate["name"] == native.name and candidate[capability] > 0] + if len(matches) != 1: + raise RuntimeError(f"No unique PortAudio {direction} device for {native.name!r}: {matches}") + return matches[0] + + @staticmethod + def available_devices() -> list[dict]: + return [ + {"id": index, "name": device["name"], "input": bool(device["max_input_channels"]), + "output": bool(device["max_output_channels"])} + for index, device in enumerate(sd.query_devices()) + if device["max_input_channels"] or device["max_output_channels"] + ] + + @classmethod + def _select_device(cls, request: int | str, direction: str) -> int: + capability = "input" if direction == "input" else "output" + devices = cls.available_devices() + if isinstance(request, int) or (isinstance(request, str) and request.isdecimal()): + index = int(request) + if any(device["id"] == index and device[capability] for device in devices): + return index + else: + needle = str(request).casefold().strip() + matches = [device["id"] for device in devices + if device[capability] and needle in device["name"].casefold()] + if len(matches) == 1: + return matches[0] + raise ValueError(f"No unique available {direction} device matches {request!r}") + + async def set_runtime_device(self, direction: str, request: int | str | None) -> dict: + """Pin one route live, or pass ``default``/None to follow macOS again.""" + if direction not in {"input", "output"}: + raise ValueError("direction must be input or output") + following_default = request is None or str(request).casefold().strip() in {"default", "mac default", "system default"} + selected = None if following_default else self._select_device(request, direction) # type: ignore[arg-type] + previous = self._params.input_device if direction == "input" else self._params.output_device + async with self._restart_lock: + stream = self._input if direction == "input" else self._output + try: + # Temporarily clear this pin so an all-pinned transport can + # create its native monitor and take a fresh default snapshot. + if following_default: + if direction == "input": + self._params.input_device = None + else: + self._params.output_device = None + await self.start_device_monitor() + if stream: + if following_default: + snapshot = self._last_snapshot + if not snapshot: + raise RuntimeError(f"No macOS default-{direction} snapshot is available yet") + uid = snapshot.default_input_uid if direction == "input" else snapshot.default_output_uid + await self._reopen_default(stream, snapshot, uid, direction) + else: + await stream.reopen(device=selected) + if direction == "input": + self._params.input_device = selected + else: + self._params.output_device = selected + except Exception: + if direction == "input": + self._params.input_device = previous + else: + self._params.output_device = previous + raise + name = "macOS default" if following_default else next(device["name"] for device in self.available_devices() if device["id"] == selected) + logger.info(f"Runtime {direction} device changed to {name}") + return {"direction": direction, "device": selected, "name": name, "following_default": following_default} def input(self) -> FrameProcessor: if not self._input: self._input = SoundDeviceInputTransport(self._params) + self._input._transport = self return self._input def output(self) -> FrameProcessor: if not self._output: self._output = SoundDeviceOutputTransport(self._params) + self._output._transport = self return self._output diff --git a/swift/VoiceAgentLauncher.swift b/swift/VoiceAgentLauncher.swift index 91e39ce..52477dd 100644 --- a/swift/VoiceAgentLauncher.swift +++ b/swift/VoiceAgentLauncher.swift @@ -4,15 +4,53 @@ import Foundation struct VoiceAgentLauncher { static func main() { let fileManager = FileManager.default - let projDir = "/Users/adolforeyna/Projects/VoiceAgent1" + let homeDir = fileManager.homeDirectoryForCurrentUser.path + + // Load configuration from .env files if present + var envConfig = [String: String]() + let envPaths = [ + "\(homeDir)/.voiceagent.env", + "\(homeDir)/.config/voiceagent/env", + "\(homeDir)/.env" + ] + + for envPath in envPaths { + if let content = try? String(contentsOfFile: envPath, encoding: .utf8) { + for line in content.components(separatedBy: .newlines) { + let trimmed = line.trimmingCharacters(in: .whitespaces) + if trimmed.isEmpty || trimmed.hasPrefix("#") { continue } + let parts = trimmed.split(separator: "=", maxSplits: 1).map { String($0).trimmingCharacters(in: .whitespacesAndNewlines) } + if parts.count == 2 { + let key = parts[0] + var val = parts[1] + if (val.hasPrefix("\"") && val.hasSuffix("\"")) || (val.hasPrefix("'") && val.hasSuffix("'")) { + val = String(val.dropFirst().dropLast()) + } + envConfig[key] = val + } + } + } + } + + let defaultProjDir = "/Users/adolforeyna/Projects/VoiceAgent1" + let configuredProjDir = ProcessInfo.processInfo.environment["VOICEAGENT_DIR"] + ?? envConfig["VOICEAGENT_DIR"] + ?? envConfig["WORKSPACE_DIR"] + ?? defaultProjDir let bundleResPath = Bundle.main.resourcePath ?? "" let bundledSrcPath = "\(bundleResPath)/src" - let workDir = fileManager.fileExists(atPath: projDir) ? projDir : bundledSrcPath - let pythonBin = "\(projDir)/.venv/bin/python" + let workDir = fileManager.fileExists(atPath: configuredProjDir) ? configuredProjDir : bundledSrcPath + + let defaultPythonBin = "\(workDir)/.venv/bin/python" + let configuredPython = ProcessInfo.processInfo.environment["VOICEAGENT_PYTHON"] + ?? envConfig["VOICEAGENT_PYTHON"] + ?? envConfig["PYTHON_PATH"] + ?? defaultPythonBin + let fallbackPython = "/usr/bin/python3" - let targetPython = fileManager.fileExists(atPath: pythonBin) ? pythonBin : fallbackPython + let targetPython = fileManager.fileExists(atPath: configuredPython) ? configuredPython : fallbackPython let targetScript = "\(workDir)/app_main.py" setenv("SSL_CERT_FILE", "/etc/ssl/cert.pem", 1) diff --git a/swift/build.sh b/swift/build.sh index f6fc79d..eb8be29 100755 --- a/swift/build.sh +++ b/swift/build.sh @@ -9,8 +9,19 @@ set -euo pipefail HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" cd "$HERE" -swiftc -O -parse-as-library SpeechHelper.swift -o speech-helper -echo "built $HERE/speech-helper" +FORCE=0 +for arg in "$@"; do + if [ "$arg" = "-f" ] || [ "$arg" = "--force" ]; then + FORCE=1 + fi +done + +if [ "$FORCE" -eq 1 ] || [ ! -f speech-helper ] || [ SpeechHelper.swift -nt speech-helper ]; then + swiftc -O -parse-as-library SpeechHelper.swift -o speech-helper + echo "built $HERE/speech-helper" +else + echo "speech-helper is up to date — skipping recompile" +fi if ./speech-helper --check >/dev/null 2>&1; then echo "speech-helper runs — ./speech-helper --check for details" @@ -24,12 +35,16 @@ else fi fi -if swiftc -O -parse-as-library -target arm64-apple-macosx26.0 LLMHelper.swift -o llm-helper 2>/dev/null; then - echo "built $HERE/llm-helper" - if ./llm-helper --check >/dev/null 2>&1; then - echo "llm-helper runs — ./llm-helper --check for details" +if [ "$FORCE" -eq 1 ] || [ ! -f llm-helper ] || [ LLMHelper.swift -nt llm-helper ]; then + if swiftc -O -parse-as-library -target arm64-apple-macosx26.0 LLMHelper.swift -o llm-helper 2>/dev/null; then + echo "built $HERE/llm-helper" + if ./llm-helper --check >/dev/null 2>&1; then + echo "llm-helper runs — ./llm-helper --check for details" + fi + else + echo "could not build llm-helper with FoundationModels; fallback to Python MLX bridge will be available" fi else - echo "could not build llm-helper with FoundationModels; fallback to Python MLX bridge will be available" + echo "llm-helper is up to date — skipping recompile" fi diff --git a/test_audio_device_monitor.py b/test_audio_device_monitor.py new file mode 100644 index 0000000..47a1509 --- /dev/null +++ b/test_audio_device_monitor.py @@ -0,0 +1,244 @@ +import asyncio +import sys +import threading +import unittest + + +from audio_device_monitor import AudioDevice, AudioDeviceMonitor, DeviceChangeReason + + +class FakeBackend: + def __init__(self): + self.devices = {} + self.input = None + self.output = None + self.listeners = [] + self.removed = [] + + def enumerate_devices(self): + return list(self.devices.values()) + + def default_input_uid(self): + return self.input + + def default_output_uid(self): + return self.output + + def add_listener(self, callback): + self.listeners.append(callback) + return callback + + def remove_listener(self, token): + self.removed.append(token) + self.listeners.remove(token) + + def notify(self): + # Emulate Core Audio's arbitrary callback thread. + threads = [threading.Thread(target=callback) for callback in list(self.listeners)] + for thread in threads: + thread.start() + for thread in threads: + thread.join() + + +class AudioDeviceMonitorTests(unittest.IsolatedAsyncioTestCase): + def setUp(self): + self.backend = FakeBackend() + self.mic = AudioDevice("mic", "Microphone", True, False, "built-in") + self.speaker = AudioDevice("speaker", "Speaker", False, True, "built-in") + self.airpods = AudioDevice("airpods", "AirPods", True, True, "bluetooth") + self.backend.devices = {x.uid: x for x in (self.mic, self.speaker)} + self.backend.input, self.backend.output = "mic", "speaker" + + async def test_initial_and_independent_default_change(self): + events = [] + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.01) + await monitor.start(events.append) + self.assertEqual(events[-1].default_input_uid, "mic") + + self.backend.devices["airpods"] = self.airpods + self.backend.output = "airpods" + self.backend.notify() + await asyncio.sleep(.04) + self.assertEqual(len(events), 2) + self.assertEqual(events[-1].default_output_uid, "airpods") + self.assertEqual(events[-1].default_input_uid, "mic") + self.assertEqual(events[-1].reason, DeviceChangeReason.DEFAULT_CHANGED) + await monitor.stop() + + async def test_duplicate_burst_is_coalesced(self): + events = [] + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.03) + await monitor.start(events.append) + for _ in range(10): + self.backend.notify() + await asyncio.sleep(.08) + self.assertEqual(len(events), 1) # no actual state change + await monitor.stop() + + async def test_unavailable_default_is_none(self): + events = [] + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.01) + await monitor.start(events.append) + self.backend.input = "gone" + self.backend.notify() + await asyncio.sleep(.04) + self.assertIsNone(events[-1].default_input_uid) + self.assertEqual(events[-1].default_output_uid, "speaker") + await monitor.stop() + + async def test_stop_removes_listener_and_blocks_late_callbacks(self): + events = [] + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.01) + await monitor.start(events.append) + await monitor.stop() + self.assertEqual(len(self.backend.removed), 1) + self.backend.input = "gone" + self.backend.notify() + await asyncio.sleep(.04) + self.assertEqual(len(events), 1) + await monitor.stop() # idempotent + + @unittest.skipUnless(sys.platform == "darwin", "Core Audio is macOS-only") + def test_native_adapter_uses_coreaudio_uids_not_portaudio_indices(self): + """Stable Core Audio identity must survive PortAudio index renumbering.""" + from audio_device_monitor import NativeMacOSCoreAudioAdapter + + adapter = NativeMacOSCoreAudioAdapter() + devices = adapter.enumerate_devices() + self.assertTrue(devices) + self.assertTrue(all(not device.uid.isdecimal() for device in devices)) + + + async def test_connect_disconnect_replacement_and_profile_events_are_classified(self): + events = [] + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.001) + await monitor.start(events.append) + + self.backend.devices["airpods"] = self.airpods + self.backend.notify() + await asyncio.sleep(.01) + self.assertEqual(events[-1].reason, DeviceChangeReason.DEVICE_ADDED) + + del self.backend.devices["speaker"] + self.backend.output = "airpods" + self.backend.notify() + await asyncio.sleep(.01) + self.assertEqual(events[-1].reason, DeviceChangeReason.DEFAULT_CHANGED) + + self.backend.devices["airpods"] = AudioDevice( + "airpods", "AirPods Hands-Free", True, True, "bluetooth" + ) + self.backend.notify() + await asyncio.sleep(.01) + self.assertEqual(events[-1].reason, DeviceChangeReason.PROFILE_CHANGED) + + del self.backend.devices["airpods"] + self.backend.output = None + self.backend.notify() + await asyncio.sleep(.01) + self.assertEqual(events[-1].reason, DeviceChangeReason.DEFAULT_CHANGED) + self.backend.devices["usb"] = AudioDevice("usb", "USB headset", False, True) + self.backend.notify() + await asyncio.sleep(.01) + del self.backend.devices["usb"] + self.backend.notify() + await asyncio.sleep(.01) + self.assertEqual(events[-1].reason, DeviceChangeReason.DEVICE_REMOVED) + self.assertIsNone(events[-1].default_output_uid) + await monitor.stop() + + async def test_backend_callback_is_dispatched_on_monitor_event_loop(self): + callback_threads = [] + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.001) + + def record(snapshot): + callback_threads.append((snapshot, threading.get_ident())) + + await monitor.start(record) + loop_thread = threading.get_ident() + self.backend.output = "airpods" + self.backend.devices["airpods"] = self.airpods + self.backend.notify() + await asyncio.sleep(.01) + + self.assertEqual(callback_threads[-1][1], loop_thread) + await monitor.stop() + + async def test_listener_registration_failure_cleans_up_registered_tokens(self): + class FailingBackend(FakeBackend): + def add_listener(self, callback): + raise RuntimeError("listener registration failed") + + backend = FailingBackend() + backend.devices = self.backend.devices + backend.input, backend.output = self.backend.input, self.backend.output + monitor = AudioDeviceMonitor(backend) + with self.assertRaises(RuntimeError): + await monitor.start(lambda _snapshot: None) + self.assertFalse(monitor._running) + self.assertEqual(len(backend.listeners), 0) + + async def test_repeated_start_is_idempotent_and_does_not_duplicate_listener(self): + first_events = [] + second_events = [] + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.001) + + await monitor.start(first_events.append) + await monitor.start(second_events.append) + + self.assertEqual(len(self.backend.listeners), 1) + self.assertEqual(len(first_events), 1) + self.assertEqual(second_events, []) + await monitor.stop() + + async def test_callback_failure_stops_monitor_and_removes_listener(self): + callback_started = asyncio.Event() + + async def failing_callback(_snapshot): + callback_started.set() + raise RuntimeError("consumer failed") + + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.001) + with self.assertRaises(RuntimeError): + await monitor.start(failing_callback) + self.assertFalse(monitor._running) + self.assertEqual(len(self.backend.listeners), 0) + self.assertEqual(len(self.backend.removed), 1) + self.assertTrue(callback_started.is_set()) + + async def test_callback_failure_after_start_cleans_up_listener(self): + callback_started = asyncio.Event() + calls = 0 + + def failing_after_initial(snapshot): + nonlocal calls + calls += 1 + if calls == 2: + callback_started.set() + raise RuntimeError("consumer failed after notification") + + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.001) + await monitor.start(failing_after_initial) + self.backend.output = None + self.backend.notify() + await asyncio.wait_for(callback_started.wait(), timeout=.2) + await asyncio.sleep(.01) + + self.assertFalse(monitor._running) + self.assertEqual(len(self.backend.listeners), 0) + self.assertEqual(len(self.backend.removed), 1) + + async def test_notifications_after_stop_are_ignored_even_if_callback_was_queued(self): + events = [] + monitor = AudioDeviceMonitor(self.backend, debounce_seconds=.05) + await monitor.start(events.append) + self.backend.notify() + await monitor.stop() + await asyncio.sleep(.06) + self.assertEqual(len(events), 1) + self.assertEqual(len(self.backend.listeners), 0) + + +if __name__ == "__main__": + unittest.main() diff --git a/test_profile_tool.py b/test_profile_tool.py new file mode 100644 index 0000000..cb5ed4d --- /dev/null +++ b/test_profile_tool.py @@ -0,0 +1,26 @@ +#!/usr/bin/env python3 +"""Tests for bin/profile_tool.py.""" + +from bin.profile_tool import get_current_profile, list_profiles, set_profile + + +def test_profile_tool(): + # Test profile listing + profiles_text = list_profiles() + assert len(profiles_text) > 0 + + # Test current profile getter + cur_profile = get_current_profile() + assert isinstance(cur_profile, str) + assert len(cur_profile) > 0 + + # Test setting valid profile (switch back to current profile to be idempotent) + ok, msg = set_profile(cur_profile) + assert ok + assert cur_profile in msg or "Switched" in msg + print(f"PASS: test_profile_tool verified (active profile: '{cur_profile}')") + + +if __name__ == "__main__": + test_profile_tool() + print("\nAll profile tool tests passed successfully!") diff --git a/test_session_tool.py b/test_session_tool.py new file mode 100644 index 0000000..c16d282 --- /dev/null +++ b/test_session_tool.py @@ -0,0 +1,69 @@ +#!/usr/bin/env python3 +"""Tests for bin/session_tool.py and Hermes session resetting in hermes_llm.py.""" + +import json +import tempfile +from pathlib import Path +from bin.session_tool import get_active_session_id, reset_session, SESSION_FILE_NAME +from hermes_llm import HermesLLM + + +def test_session_tool_api(): + with tempfile.TemporaryDirectory() as tmpdir: + workspace = Path(tmpdir) + session_file = workspace / SESSION_FILE_NAME + + # Initially no session + assert get_active_session_id(workspace) is None + + # Write mock session file + session_file.write_text(json.dumps({"session_id": "test_session_123"}) + "\n") + assert get_active_session_id(workspace) == "test_session_123" + + # Reset session + ok, msg = reset_session(workspace) + assert ok + assert "reset successfully" in msg.lower() or "no active session" in msg.lower() + assert get_active_session_id(workspace) is None + assert not session_file.exists() + + # Double reset safely handles non-existent file + ok, msg = reset_session(workspace) + assert ok + print("PASS: test_session_tool_api verified") + + +def test_hermes_llm_session_sync(): + with tempfile.TemporaryDirectory() as tmpdir: + workspace = Path(tmpdir) + session_file = workspace / SESSION_FILE_NAME + + # Write mock session ID + session_file.write_text(json.dumps({"session_id": "test_session_456"}) + "\n") + + llm = HermesLLM(cwd=workspace) + assert llm._session_id == "test_session_456" + + # External reset via tool + reset_session(workspace) + assert get_active_session_id(workspace) is None + + # Sync disk state in HermesLLM + llm._sync_disk_session() + assert llm._session_id is None + + # Call reset_session directly on instance + session_file.write_text(json.dumps({"session_id": "test_session_789"}) + "\n") + llm._sync_disk_session() + assert llm._session_id == "test_session_789" + + llm.reset_session() + assert llm._session_id is None + assert not session_file.exists() + print("PASS: test_hermes_llm_session_sync verified") + + +if __name__ == "__main__": + test_session_tool_api() + test_hermes_llm_session_sync() + print("\nAll session tool tests passed successfully!") diff --git a/test_sounddevice_transport.py b/test_sounddevice_transport.py new file mode 100644 index 0000000..f4b3135 --- /dev/null +++ b/test_sounddevice_transport.py @@ -0,0 +1,350 @@ +import unittest +from unittest.mock import AsyncMock, patch + +from audio_device_monitor import AudioDeviceSnapshot +from pipecat.frames.frames import StartFrame +from sounddevice_transport import ( + SoundDeviceInputTransport, + SoundDeviceOutputTransport, + SoundDeviceTransport, + SoundDeviceTransportParams, +) + + +class FakeMonitor: + def __init__(self): + self.start = AsyncMock() + self.stop = AsyncMock() + + +class SoundDeviceTransportSwitchTests(unittest.IsolatedAsyncioTestCase): + def _transport(self, *, input_device=None, output_device=None): + transport = SoundDeviceTransport( + SoundDeviceTransportParams(input_device=input_device, output_device=output_device) + ) + transport._input = type("Input", (), {"reopen": AsyncMock()})() + transport._output = type("Output", (), {"reopen": AsyncMock()})() + return transport + + async def test_default_changes_reopen_only_unset_sides(self): + transport = self._transport(input_device=None, output_device="My Speakers") + + await transport._on_device_change(AudioDeviceSnapshot(1, "mic", "speakers")) + await transport._on_device_change(AudioDeviceSnapshot(2, "airpods", "airpods")) + + transport._input.reopen.assert_awaited_once_with() + transport._output.reopen.assert_not_awaited() + + async def test_default_changes_preserve_each_override_for_all_override_combinations(self): + devices = { + "old-mic": type("Device", (), {"name": "Built-in Mic", "can_input": True, "can_output": False})(), + "old-speaker": type("Device", (), {"name": "Built-in Speaker", "can_input": False, "can_output": True})(), + "new-mic": type("Device", (), {"name": "USB Mic", "can_input": True, "can_output": False})(), + "new-speaker": type("Device", (), {"name": "USB Speaker", "can_input": False, "can_output": True})(), + } + portaudio_devices = [ + {"name": "Built-in Mic", "max_input_channels": 1, "max_output_channels": 0}, + {"name": "Built-in Speaker", "max_input_channels": 0, "max_output_channels": 2}, + {"name": "USB Mic", "max_input_channels": 1, "max_output_channels": 0}, + {"name": "USB Speaker", "max_input_channels": 0, "max_output_channels": 2}, + ] + initial = AudioDeviceSnapshot(1, "old-mic", "old-speaker", devices=devices) + changed = AudioDeviceSnapshot(2, "new-mic", "new-speaker", devices=devices) + + with patch("sounddevice_transport.sd.query_devices", return_value=portaudio_devices): + for input_override, output_override in ( + (None, None), + ("Pinned Mic", None), + (None, "Pinned Speaker"), + ("Pinned Mic", "Pinned Speaker"), + ): + with self.subTest(input_override=input_override, output_override=output_override): + transport = self._transport( + input_device=input_override, + output_device=output_override, + ) + await transport._on_device_change(initial) + await transport._on_device_change(changed) + + if input_override is None: + transport._input.reopen.assert_awaited_once_with(device=2) + else: + transport._input.reopen.assert_not_awaited() + if output_override is None: + transport._output.reopen.assert_awaited_once_with(device=3) + else: + transport._output.reopen.assert_not_awaited() + + async def test_unavailable_default_does_not_reopen_that_side_but_reopens_other_side(self): + transport = self._transport() + devices = { + "mic": type("Device", (), {"name": "Mic", "can_input": True, "can_output": False})(), + "speaker": type("Device", (), {"name": "Speaker", "can_input": False, "can_output": True})(), + "headphones": type("Device", (), {"name": "Headphones", "can_input": False, "can_output": True})(), + } + initial = AudioDeviceSnapshot(1, "mic", "speaker", devices=devices) + unavailable_input = AudioDeviceSnapshot(2, None, "headphones", devices=devices) + with patch("sounddevice_transport.sd.query_devices", return_value=[ + {"name": "Headphones", "max_input_channels": 0, "max_output_channels": 2}, + ]): + await transport._on_device_change(initial) + await transport._on_device_change(unavailable_input) + + transport._input.reopen.assert_not_awaited() + transport._output.reopen.assert_awaited_once_with(device=0) + + async def test_input_only_output_only_and_simultaneous_changes_route_independently(self): + transport = self._transport() + await transport._on_device_change(AudioDeviceSnapshot(1, "mic", "speaker")) + + await transport._on_device_change(AudioDeviceSnapshot(2, "airpods-mic", "speaker")) + transport._input.reopen.assert_awaited_once_with() + transport._output.reopen.assert_not_awaited() + + await transport._on_device_change(AudioDeviceSnapshot(3, "airpods-mic", "airpods-speaker")) + transport._input.reopen.assert_awaited_once_with() + transport._output.reopen.assert_awaited_once_with() + + await transport._on_device_change(AudioDeviceSnapshot(4, "mac-mic", "mac-speaker")) + self.assertEqual(transport._input.reopen.await_count, 2) + self.assertEqual(transport._output.reopen.await_count, 2) + + async def test_default_change_uses_current_portaudio_device_not_process_startup_default(self): + transport = self._transport() + initial = AudioDeviceSnapshot(1, "built-in-mic", "built-in-speaker") + switched = AudioDeviceSnapshot(2, "airpods-input", "airpods-output", devices={ + "airpods-input": type("Device", (), {"name": "AirPods", "can_input": True, "can_output": False})(), + "airpods-output": type("Device", (), {"name": "AirPods", "can_input": False, "can_output": True})(), + }) + with patch("sounddevice_transport.sd.query_devices", return_value=[ + {"name": "MacBook Air Speakers", "max_input_channels": 0, "max_output_channels": 2}, + {"name": "AirPods", "max_input_channels": 1, "max_output_channels": 0}, + {"name": "AirPods", "max_input_channels": 0, "max_output_channels": 2}, + ]): + await transport._on_device_change(initial) + await transport._on_device_change(switched) + + transport._input.reopen.assert_awaited_once_with(device=1) + transport._output.reopen.assert_awaited_once_with(device=2) + + async def test_input_override_does_not_follow_default_but_output_does(self): + params = SoundDeviceTransportParams(input_device="USB Mic", output_device=None) + transport = SoundDeviceTransport(params) + transport._input = type("Input", (), {"reopen": AsyncMock()})() + transport._output = type("Output", (), {"reopen": AsyncMock()})() + + await transport._on_device_change(AudioDeviceSnapshot(1, "mic", "speakers")) + await transport._on_device_change(AudioDeviceSnapshot(2, "airpods", "headphones")) + + transport._input.reopen.assert_not_awaited() + transport._output.reopen.assert_awaited_once_with() + + async def test_runtime_event_sink_receives_native_device_snapshot(self): + sink = AsyncMock() + transport = SoundDeviceTransport(SoundDeviceTransportParams(), device_event_sink=sink) + + snapshot = AudioDeviceSnapshot(1, "mic", "speaker") + await transport._on_device_change(snapshot) + + sink.assert_awaited_once_with(snapshot) + + async def test_stale_snapshot_cannot_reopen_a_replaced_stream(self): + transport = self._transport() + + await transport._on_device_change(AudioDeviceSnapshot(1, "mic", "speaker")) + await transport._on_device_change(AudioDeviceSnapshot(3, "airpods", "airpods")) + await transport._on_device_change(AudioDeviceSnapshot(2, "mic", "speaker")) + + transport._input.reopen.assert_awaited_once_with() + transport._output.reopen.assert_awaited_once_with() + + async def test_failed_input_reopen_keeps_output_route_change_alive(self): + transport = self._transport() + transport._input.reopen.side_effect = OSError("device unavailable") + + await transport._on_device_change(AudioDeviceSnapshot(1, "mic", "speaker")) + await transport._on_device_change(AudioDeviceSnapshot(2, "airpods", "airpods")) + + transport._input.reopen.assert_awaited_once_with() + transport._output.reopen.assert_awaited_once_with() + + async def test_transport_starts_and_stops_injected_monitor_once(self): + monitor = FakeMonitor() + transport = SoundDeviceTransport(SoundDeviceTransportParams(), device_monitor=monitor) + + await transport.start_device_monitor() + await transport.start_device_monitor() + await transport.stop_device_monitor() + await transport.stop_device_monitor() + + monitor.start.assert_awaited_once_with(transport._on_device_change) + monitor.stop.assert_awaited_once_with() + + async def test_production_transport_installs_macos_monitor_when_defaults_are_unset(self): + monitor = FakeMonitor() + with ( + patch("sounddevice_transport.sys.platform", "darwin"), + patch("sounddevice_transport.create_macos_audio_monitor", return_value=monitor) as factory, + ): + transport = SoundDeviceTransport(SoundDeviceTransportParams()) + await transport.start_device_monitor() + + factory.assert_called_once_with() + monitor.start.assert_awaited_once_with(transport._on_device_change) + + async def test_production_input_start_installs_and_cleanup_releases_device_monitor(self): + monitor = FakeMonitor() + transport = SoundDeviceTransport( + SoundDeviceTransportParams(), device_monitor=monitor + ) + input_transport = transport.input() + + class FakeInputStream: + device = 0 + + def __init__(self, **_kwargs): + pass + + def start(self): + pass + + def stop(self): + pass + + def close(self): + pass + + with ( + patch("sounddevice_transport.sd.RawInputStream", FakeInputStream), + patch("sounddevice_transport.sd.query_devices", return_value={"name": "Fake Mic"}), + ): + await input_transport.start(StartFrame(audio_in_sample_rate=16000)) + await input_transport.cleanup() + + monitor.start.assert_awaited_once_with(transport._on_device_change) + monitor.stop.assert_awaited_once_with() + + async def test_production_input_start_skips_unavailable_monitor_off_macos(self): + transport = SoundDeviceTransport(SoundDeviceTransportParams()) + input_transport = transport.input() + + class FakeInputStream: + device = 0 + + def __init__(self, **_kwargs): + pass + + def start(self): + pass + + def stop(self): + pass + + def close(self): + pass + + with ( + patch("sounddevice_transport.sys.platform", "linux"), + patch("sounddevice_transport.create_macos_audio_monitor") as factory, + patch("sounddevice_transport.sd.RawInputStream", FakeInputStream), + patch("sounddevice_transport.sd.query_devices", return_value={"name": "Fake Mic"}), + ): + await input_transport.start(StartFrame(audio_in_sample_rate=16000)) + await input_transport.cleanup() + + factory.assert_not_called() + + async def test_cleaning_one_side_keeps_monitor_until_last_side_stops(self): + monitor = FakeMonitor() + transport = SoundDeviceTransport(SoundDeviceTransportParams(), device_monitor=monitor) + input_transport = transport.input() + output_transport = transport.output() + + await transport.start_device_monitor(input_transport) + await transport.start_device_monitor(output_transport) + await input_transport.cleanup() + + monitor.stop.assert_not_awaited() + await output_transport.cleanup() + + monitor.start.assert_awaited_once_with(transport._on_device_change) + monitor.stop.assert_awaited_once_with() + + async def test_failed_runtime_selection_keeps_previous_pin(self): + transport = self._transport(output_device=5) + transport._output.reopen.side_effect = OSError("unavailable") + with patch("sounddevice_transport.sd.query_devices", return_value=[ + {"name": "MacBook Air Speakers", "max_input_channels": 0, "max_output_channels": 2}, + {"name": "AirPods", "max_input_channels": 0, "max_output_channels": 2}, + ]): + with self.assertRaises(OSError): + await transport.set_runtime_device("output", "airpods") + + self.assertEqual(transport._params.output_device, 5) + + async def test_runtime_selection_pins_only_requested_direction(self): + transport = self._transport() + with patch("sounddevice_transport.sd.query_devices", return_value=[ + {"name": "MacBook Air Speakers", "max_input_channels": 0, "max_output_channels": 2}, + {"name": "AirPods", "max_input_channels": 1, "max_output_channels": 0}, + {"name": "AirPods", "max_input_channels": 0, "max_output_channels": 2}, + ]): + result = await transport.set_runtime_device("output", "airpods") + + self.assertEqual(result["device"], 2) + self.assertEqual(transport._params.output_device, 2) + self.assertIsNone(transport._params.input_device) + transport._output.reopen.assert_awaited_once_with(device=2) + transport._input.reopen.assert_not_awaited() + + async def test_switching_a_pinned_side_to_default_starts_monitor_for_snapshot(self): + monitor = FakeMonitor() + + async def publish_initial_snapshot(callback): + await callback(AudioDeviceSnapshot(1, "mic", "speaker")) + + monitor.start.side_effect = publish_initial_snapshot + transport = SoundDeviceTransport( + SoundDeviceTransportParams(input_device="USB Mic", output_device="USB Speakers"), + device_monitor=monitor, + ) + transport._input = type("Input", (), {"reopen": AsyncMock()})() + + await transport.set_runtime_device("input", None) + + monitor.start.assert_awaited_once_with(transport._on_device_change) + transport._input.reopen.assert_awaited_once_with() + + async def test_old_input_callback_cannot_deliver_after_replacement(self): + input_transport = SoundDeviceInputTransport(SoundDeviceTransportParams()) + input_transport.push_audio_frame = AsyncMock() + input_transport._stream_generation = 2 + + await input_transport._push_audio_frame_if_current(1, object()) + + input_transport.push_audio_frame.assert_not_awaited() + + async def test_failed_output_stream_start_closes_partial_stream(self): + class FailingStream: + closed = False + + def __init__(self, **_kwargs): + pass + + def start(self): + raise OSError("unavailable") + + def close(self): + self.closed = True + + output = SoundDeviceOutputTransport(SoundDeviceTransportParams()) + output._sample_rate = 24000 + with patch("sounddevice_transport.sd.RawOutputStream", FailingStream): + with self.assertRaises(OSError): + await output._open_stream() + + self.assertIsNone(output._out_stream) + + +if __name__ == "__main__": + unittest.main() diff --git a/web_server.py b/web_server.py index 5808c68..3287ded 100644 --- a/web_server.py +++ b/web_server.py @@ -25,18 +25,21 @@ _sse_clients: Set[asyncio.Queue] = set() _workspace_dir: Path = Path(__file__).parent.resolve() _model_manager = None _voice_manager = None +_audio_controller = None _input_callback: Callable = None _recent_events: list = [] _MAX_HISTORY_EVENTS = 200 -def set_managers(workspace: Path, model_mgr=None, voice_mgr=None, input_callback=None): - global _workspace_dir, _model_manager, _voice_manager, _input_callback +def set_managers(workspace: Path, model_mgr=None, voice_mgr=None, input_callback=None, audio_controller=None): + global _workspace_dir, _model_manager, _voice_manager, _input_callback, _audio_controller _workspace_dir = Path(workspace) _model_manager = model_mgr _voice_manager = voice_mgr if input_callback is not None: _input_callback = input_callback + if audio_controller is not None: + _audio_controller = audio_controller def broadcast_event(event_type: str, payload: dict): @@ -185,6 +188,43 @@ async def handle_set_voice(request): return web.json_response({"error": str(e)}, status=500) +async def handle_reset_session(request): + try: + from bin.session_tool import reset_session + ok, msg = reset_session(_workspace_dir) + if ok: + broadcast_event("session_reset", {"message": msg}) + return web.json_response({"success": True, "message": msg}) + return web.json_response({"error": msg}, status=500) + except Exception as e: + return web.json_response({"error": str(e)}, status=500) + + +async def handle_get_audio_devices(request): + if not _audio_controller: + return web.json_response({"error": "Audio controller not active"}, status=503) + return web.json_response({"devices": _audio_controller.available_devices()}) + + +async def handle_set_audio_device(request): + try: + if not _audio_controller: + return web.json_response({"error": "Audio controller not active"}, status=503) + body = await request.json() + direction, device = body.get("direction"), body.get("device") + if direction not in {"input", "output"}: + return web.json_response({"error": "direction must be input or output"}, status=400) + result = await _audio_controller.set_runtime_device(direction, device) + broadcast_event("audio_device_selected", result) + return web.json_response({"success": True, **result}) + except (ValueError, RuntimeError) as exc: + logger.warning(f"Runtime audio-device selection failed: {exc}") + return web.json_response({"error": str(exc)}, status=400) + except Exception as exc: + logger.exception("Runtime audio-device selection failed") + return web.json_response({"error": str(exc)}, status=500) + + async def handle_show_file_api(request): try: body = await request.json() @@ -249,9 +289,12 @@ async def start_server(workspace: Path, host: str = "127.0.0.1", port: int = 888 app.router.add_get("/api/models", handle_get_models) app.router.add_post("/api/model", handle_set_model) app.router.add_post("/api/voice", handle_set_voice) + app.router.add_get("/api/audio-devices", handle_get_audio_devices) + app.router.add_post("/api/audio-device", handle_set_audio_device) app.router.add_post("/api/show_file", handle_show_file_api) app.router.add_post("/api/open_browser", handle_open_browser_api) app.router.add_post("/api/send", handle_send_message_api) + app.router.add_post("/api/session/reset", handle_reset_session) runner = web.AppRunner(app) await runner.setup() @@ -692,6 +735,9 @@ HTML_INDEX = """
Voice: Loading...
+
+ 🔄 Reset Session +
@@ -843,6 +889,21 @@ HTML_INDEX = """ .catch(err => console.error('Failed to send message:', err)); } + async function resetSession() { + if (!confirm('Start a fresh Hermes conversation session?')) return; + try { + const res = await fetch('/api/session/reset', { method: 'POST' }); + const data = await res.json(); + if (data.success) { + appendToolStep('System', data.message || 'Session reset.'); + } else { + alert('Failed to reset session: ' + (data.error || 'Unknown error')); + } + } catch (err) { + alert('Error resetting session: ' + err.message); + } + } + function handleKeyDown(e) { if (e.key === 'Enter') { sendMessage();