chore: archive mac mini automation baseline
This commit is contained in:
Adolfo Reyna
@@ -0,0 +1,7 @@
|
||||
node_modules/
|
||||
.venv/
|
||||
.env
|
||||
*.log
|
||||
.logs/
|
||||
.DS_Store
|
||||
coverage/
|
||||
@@ -0,0 +1,233 @@
|
||||
# MacMini MCP
|
||||
|
||||
A local Model Context Protocol server that exposes selected macOS app actions to
|
||||
an AI harness. It uses Apple's scripting interfaces through `/usr/bin/osascript`
|
||||
and stays on the local machine.
|
||||
|
||||
## Available tools
|
||||
|
||||
| Tool | Action |
|
||||
| --- | --- |
|
||||
| `notes_list` | Search notes; returns titles and metadata unless previews are explicitly requested |
|
||||
| `notes_read` | Read a note by the ID returned by `notes_list` |
|
||||
| `notes_create` | Create a plaintext-backed note |
|
||||
| `mail_accounts` | List configured Apple Mail accounts (read-only) |
|
||||
| `mail_list_mailboxes` | List Apple Mail's top-level mailboxes (read-only) |
|
||||
| `mail_list_messages` | List metadata from one selected account and mailbox (read-only) |
|
||||
| `mail_read_message` | Read a selected Apple Mail message (read-only) |
|
||||
| `calendar_list_calendars` | List calendar indexes, names, and write capability |
|
||||
| `calendar_list_events` | List events in an ISO-8601 time window from the focused `Home` calendar |
|
||||
| `calendar_create_event` | Create an event in the focused `Home` calendar |
|
||||
| `reminders_list_lists` | List reminder lists with account context and assignment metadata availability |
|
||||
| `reminders_list` | List reminders, including assignment details or assignment hints for shared-list reminders |
|
||||
| `reminders_create` | Create a reminder |
|
||||
| `contacts_search` | Search contact names and organizations without disclosing contact methods |
|
||||
| `contacts_read` | Read phone and email details for one selected contact |
|
||||
| `contacts_create` | Create a contact with optional email and phone details |
|
||||
| `deco_get_config_status` | Show Deco connection config without revealing the password |
|
||||
| `deco_get_overview` | Read TP-Link Deco overview stats and firmware |
|
||||
| `deco_list_clients` | List online Deco clients, current traffic speeds, and linked mesh node when available |
|
||||
| `deco_get_ipv4_status` | Read WAN/LAN IPv4 status |
|
||||
| `deco_get_firmware` | Read Deco model and firmware version |
|
||||
| `system_get_info` | Get macOS version (sw_vers), hardware model, and SpeechAnalyzer availability |
|
||||
| `system_speech_api_status` | Full check for Apple SpeechAnalyzer/SpeechTranscriber (macOS 26+) |
|
||||
| `speech_kokoro_status` | Check the warm `ksay` Kokoro TTS daemon |
|
||||
| `speech_kokoro_synthesize` | Generate fast Kokoro speech to a local WAV file |
|
||||
| `speech_kokoro_synthesize_base64` | Generate fast Kokoro speech and return WAV base64 |
|
||||
| `codex_image_get_config_status` | Show local Codex CLI image generation config |
|
||||
| `codex_image_generate` | Generate an image with this Mac's Codex CLI and save it locally |
|
||||
| `gemini_image_get_config_status` | Show Gemini image generation config without revealing the API key |
|
||||
| `gemini_image_generate` | Generate an image with the Gemini API and save it locally |
|
||||
| `gemini_chrome_prompt_get_config_status` | Show config for the Codex Chrome-skill Gemini image prompt builder |
|
||||
| `gemini_chrome_prompt_build` | Build a ready-to-run Codex prompt for Gemini web-app image generation |
|
||||
|
||||
There are no destructive tools in the initial server.
|
||||
|
||||
Calendar names can repeat across accounts. This server is focused on the
|
||||
event-rich `Home` calendar discovered during setup (`calendarIndex: 2`) and
|
||||
verifies the selected index is still named `Home` before operating on it.
|
||||
Calendar selector parameters remain available as advanced overrides.
|
||||
|
||||
Reminder assignment data is exposed on `reminders_list` as an `assignment`
|
||||
object. Apple Reminders automation does not currently publish shared-list
|
||||
participant metadata directly, so the tool first checks for any native assignee
|
||||
field macOS exposes and then falls back to assignment hints embedded in the
|
||||
reminder title or notes, such as `(Alicia)` or `Captured 2026-05-18, Alicia;`.
|
||||
|
||||
**macOS 26 note:** This Mac is on macOS 26.5.2 (Mac16,10 M4) — so `system_speech_api_status`
|
||||
confirms SpeechAnalyzer/SpeechTranscriber is available. Apple's new engine beats Whisper
|
||||
Small 2.12% vs 3.74% WER per Inscribe benchmark (2026-07-13).
|
||||
|
||||
## Setup
|
||||
|
||||
Requires macOS and Node.js 20 or newer.
|
||||
|
||||
```sh
|
||||
npm install
|
||||
npm run python:install
|
||||
npm run check
|
||||
npm run service:install
|
||||
npm run ksay:install
|
||||
```
|
||||
|
||||
The service defaults to a same-Mac endpoint:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:7331/mcp
|
||||
```
|
||||
|
||||
Health check:
|
||||
|
||||
```sh
|
||||
curl -s http://127.0.0.1:7331/health
|
||||
```
|
||||
|
||||
`launchd` runs `node --watch src/http.js`, so edits to the server or imported
|
||||
modules cause it to restart automatically while the agent remains installed.
|
||||
After changing installed dependencies or service configuration, run:
|
||||
|
||||
```sh
|
||||
npm install
|
||||
npm run service:install
|
||||
```
|
||||
|
||||
Operational commands:
|
||||
|
||||
```sh
|
||||
npm run service:status
|
||||
npm run service:restart
|
||||
npm run service:uninstall
|
||||
npm run ksay:restart
|
||||
```
|
||||
|
||||
Service logs are stored in `.logs/`.
|
||||
|
||||
## Fast Kokoro speech with `ksay`
|
||||
|
||||
This repo includes a warm Kokoro TTS daemon backed by `mlx-audio`, plus
|
||||
`bin/ksay`, a console command intended as a neural replacement for macOS `say`.
|
||||
The launchd service preloads the model and keeps it resident, so normal calls
|
||||
only pay generation and playback time.
|
||||
|
||||
Install Python dependencies and the warm service:
|
||||
|
||||
```sh
|
||||
npm run python:install
|
||||
npm run ksay:install
|
||||
```
|
||||
|
||||
Add the repo's `bin` directory to your shell path:
|
||||
|
||||
```sh
|
||||
export PATH="/Users/adolforeyna/Projects/MacMiniMCP/bin:$PATH"
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```sh
|
||||
ksay "Hello from Kokoro."
|
||||
ksay -v af_bella --speed 1.15 "Fast, warm speech."
|
||||
echo "Piped text works too." | ksay
|
||||
ksay --no-play -o /tmp/hello.wav "Write a wav without playback."
|
||||
ksay --status
|
||||
```
|
||||
|
||||
Defaults can be overridden with environment variables:
|
||||
|
||||
```text
|
||||
KSAY_MODEL=mlx-community/Kokoro-82M-8bit
|
||||
KSAY_VOICE=af_heart
|
||||
KSAY_LANG_CODE=a
|
||||
KSAY_PORT=7332
|
||||
KSAY_OUTPUT_DIR=/Users/adolforeyna/Projects/MacMiniMCP/generated-audio
|
||||
```
|
||||
|
||||
Useful Kokoro voices include `af_heart`, `af_bella`, `af_nova`, `af_sky`,
|
||||
`am_adam`, `am_echo`, `bf_alice`, `bf_emma`, `bm_daniel`, and `bm_george`.
|
||||
Use language code `a` for American English and `b` for British English.
|
||||
|
||||
## Harness configuration
|
||||
|
||||
For a harness that supports Streamable HTTP, configure the local MCP URL as
|
||||
`http://127.0.0.1:7331/mcp`.
|
||||
|
||||
For a trusted local-network harness such as a Raspberry Pi, set
|
||||
`MACMINI_MCP_HOST` in `.env` to the Mac's LAN IP and set a strong
|
||||
`MACMINI_MCP_TOKEN`. Then configure the remote MCP client with:
|
||||
|
||||
```text
|
||||
URL: http://<mac-lan-ip>:7331/mcp
|
||||
Authorization: Bearer <MACMI...KEN>
|
||||
```
|
||||
|
||||
Restart after changing `.env`:
|
||||
|
||||
```sh
|
||||
npm run service:restart
|
||||
```
|
||||
|
||||
For TP-Link Deco tools, install Python dependencies with `npm run
|
||||
python:install`, then set `DECO_HOST`, `DECO_USERNAME=admin`, `DECO_PASSWORD`,
|
||||
and optionally `DECO_VERIFY_SSL=false` in `.env`.
|
||||
|
||||
For Codex image generation, make sure the Mac is logged in with `codex login`.
|
||||
Generated files are saved to `generated-images/` by default; override this with
|
||||
`CODEX_IMAGE_OUTPUT_DIR`. Set `CODEX_IMAGE_MODEL` or `CODEX_IMAGE_TIMEOUT_MS` or
|
||||
`CODEX_CLI_PATH` to the absolute `codex` path.
|
||||
|
||||
For Gemini image generation, set `GEMINI_API_KEY` in `.env`. Generated files are
|
||||
saved to `generated-images/` by default; override this with
|
||||
`GEMINI_IMAGE_OUTPUT_DIR`. The Gemini image tool calls the Gemini API directly
|
||||
and does not expose browser navigation, page inspection, or screenshot tools.
|
||||
|
||||
For Gemini image generation through the signed-in Chrome web app, use
|
||||
`gemini_chrome_prompt_build` to create a ready-to-run Codex prompt. The MCP
|
||||
service does not control Chrome directly; the Chrome skill is only available
|
||||
inside an active Codex session. The generated prompt verifies the Chrome profile
|
||||
name is `ReynaFamilyBot`, opens only `https://gemini.google.com/app`, submits the
|
||||
image prompt, downloads the generated image, and copies it to
|
||||
`generated-images/`. Override the expected Chrome profile with
|
||||
`GEMINI_CHROME_PROFILE_NAME` and the output directory with
|
||||
`GEMINI_CHROME_IMAGE_OUTPUT_DIR`.
|
||||
|
||||
For a harness that launches stdio servers, use:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"macmini": {
|
||||
"command": "/Users/adolforeyna/.nvm/versions/node/v22.22.0/bin/node",
|
||||
"args": ["/Users/adolforeyna/Projects/MacMiniMCP/src/stdio.js"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Permissions and security
|
||||
|
||||
On first use of a Notes, Calendar, Reminders, or Contacts tool, macOS may ask for
|
||||
Automation access for Node. Permit only the applications you want the server
|
||||
to control under **System Settings > Privacy & Security > Automation**.
|
||||
|
||||
The HTTP service binds to `127.0.0.1` by default. When configured to bind to a
|
||||
LAN address, it refuses to start without `MACMINI_MCP_TOKEN`; clients must send
|
||||
`Authorization: Bearer *** This is HTTP bearer authentication on your
|
||||
local network, not encrypted transport. Use it only on a trusted LAN or put it
|
||||
behind a private encrypted network such as a VPN.
|
||||
|
||||
## Development
|
||||
|
||||
```sh
|
||||
npm run check
|
||||
npm run dev
|
||||
```
|
||||
|
||||
The MCP transport follows the official TypeScript SDK Streamable HTTP server
|
||||
approach: [Model Context Protocol TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk).
|
||||
|
||||
## Image generation
|
||||
|
||||
This server exposes Codex-backed and Gemini-backed image generation tools.
|
||||
Gemini image generation uses the Gemini API directly, not browser automation.
|
||||
Image Playground was tried and removed because the macOS app does not expose a
|
||||
scriptable prompt-to-file action through AppleScript or Shortcuts/App Intents.
|
||||
+127
@@ -0,0 +1,127 @@
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
|
||||
enum AudioCaptureState: Equatable {
|
||||
case idle
|
||||
case starting
|
||||
case recording
|
||||
case failed(String)
|
||||
}
|
||||
|
||||
@MainActor
|
||||
final class AudioCaptureManager: ObservableObject {
|
||||
@Published private(set) var state: AudioCaptureState = .idle
|
||||
|
||||
var onPCMChunk: ((Data) -> Void)?
|
||||
|
||||
private let engine = AVAudioEngine()
|
||||
private var converter: AVAudioConverter?
|
||||
private var inputFormat: AVAudioFormat?
|
||||
|
||||
private let targetFormat = AVAudioFormat(
|
||||
commonFormat: .pcmFormatInt16,
|
||||
sampleRate: 16_000,
|
||||
channels: 1,
|
||||
interleaved: true
|
||||
)!
|
||||
|
||||
func requestPermission() async -> Bool {
|
||||
await withCheckedContinuation { continuation in
|
||||
AVAudioSession.sharedInstance().requestRecordPermission { granted in
|
||||
continuation.resume(returning: granted)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func startRecording() async {
|
||||
guard state != .recording && state != .starting else { return }
|
||||
|
||||
state = .starting
|
||||
|
||||
guard await requestPermission() else {
|
||||
state = .failed("Microphone permission was denied.")
|
||||
return
|
||||
}
|
||||
|
||||
do {
|
||||
try configureSession()
|
||||
try configureEngine()
|
||||
try engine.start()
|
||||
state = .recording
|
||||
} catch {
|
||||
stopRecording()
|
||||
state = .failed(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
|
||||
func stopRecording() {
|
||||
guard state == .recording || state == .starting else { return }
|
||||
|
||||
engine.inputNode.removeTap(onBus: 0)
|
||||
engine.stop()
|
||||
converter = nil
|
||||
inputFormat = nil
|
||||
state = .idle
|
||||
}
|
||||
|
||||
private func configureSession() throws {
|
||||
let session = AVAudioSession.sharedInstance()
|
||||
try session.setCategory(.playAndRecord, mode: .voiceChat, options: [.defaultToSpeaker])
|
||||
try session.setPreferredSampleRate(16_000)
|
||||
try session.setPreferredIOBufferDuration(0.02)
|
||||
try session.setActive(true)
|
||||
}
|
||||
|
||||
private func configureEngine() throws {
|
||||
let inputNode = engine.inputNode
|
||||
let hardwareFormat = inputNode.outputFormat(forBus: 0)
|
||||
inputFormat = hardwareFormat
|
||||
converter = AVAudioConverter(from: hardwareFormat, to: targetFormat)
|
||||
|
||||
inputNode.removeTap(onBus: 0)
|
||||
inputNode.installTap(onBus: 0, bufferSize: 1024, format: hardwareFormat) { [weak self] buffer, _ in
|
||||
Task { @MainActor in
|
||||
guard let self else { return }
|
||||
guard let data = self.convertToTargetPCM(buffer) else { return }
|
||||
self.onPCMChunk?(data)
|
||||
}
|
||||
}
|
||||
|
||||
engine.prepare()
|
||||
}
|
||||
|
||||
private func convertToTargetPCM(_ inputBuffer: AVAudioPCMBuffer) -> Data? {
|
||||
guard let converter else { return nil }
|
||||
|
||||
let ratio = targetFormat.sampleRate / inputBuffer.format.sampleRate
|
||||
let targetFrameCapacity = AVAudioFrameCount(Double(inputBuffer.frameLength) * ratio) + 1
|
||||
guard let outputBuffer = AVAudioPCMBuffer(pcmFormat: targetFormat, frameCapacity: targetFrameCapacity) else {
|
||||
return nil
|
||||
}
|
||||
|
||||
var didProvideInput = false
|
||||
var conversionError: NSError?
|
||||
|
||||
converter.convert(to: outputBuffer, error: &conversionError) { _, status in
|
||||
if didProvideInput {
|
||||
status.pointee = .noDataNow
|
||||
return nil
|
||||
}
|
||||
|
||||
didProvideInput = true
|
||||
status.pointee = .haveData
|
||||
return inputBuffer
|
||||
}
|
||||
|
||||
guard conversionError == nil else { return nil }
|
||||
return outputBuffer.interleavedPCMData()
|
||||
}
|
||||
}
|
||||
|
||||
private extension AVAudioPCMBuffer {
|
||||
func interleavedPCMData() -> Data? {
|
||||
let audioBuffer = audioBufferList.pointee.mBuffers
|
||||
guard let source = audioBuffer.mData else { return nil }
|
||||
return Data(bytes: source, count: Int(audioBuffer.mDataByteSize))
|
||||
}
|
||||
}
|
||||
+202
@@ -0,0 +1,202 @@
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
|
||||
enum HermesConnectionState: Equatable {
|
||||
case disconnected
|
||||
case connecting
|
||||
case connected
|
||||
case failed(String)
|
||||
}
|
||||
|
||||
enum HermesPlaybackState: Equatable {
|
||||
case idle
|
||||
case playing
|
||||
case failed(String)
|
||||
}
|
||||
|
||||
@MainActor
|
||||
final class AudioStreamManager: NSObject, ObservableObject {
|
||||
@Published private(set) var connectionState: HermesConnectionState = .disconnected
|
||||
@Published private(set) var playbackState: HermesPlaybackState = .idle
|
||||
|
||||
private let endpoint: URL
|
||||
private let session: URLSession
|
||||
private var task: URLSessionWebSocketTask?
|
||||
private var avPlayer: AVAudioPlayer?
|
||||
|
||||
private let rawPlaybackEngine = AVAudioEngine()
|
||||
private let rawPlaybackNode = AVAudioPlayerNode()
|
||||
private let rawPlaybackFormat = AVAudioFormat(
|
||||
commonFormat: .pcmFormatInt16,
|
||||
sampleRate: 16_000,
|
||||
channels: 1,
|
||||
interleaved: true
|
||||
)!
|
||||
|
||||
init(endpoint: URL) {
|
||||
self.endpoint = endpoint
|
||||
self.session = URLSession(configuration: .default)
|
||||
super.init()
|
||||
configureRawPlaybackEngine()
|
||||
}
|
||||
|
||||
func connect() {
|
||||
guard task == nil else { return }
|
||||
|
||||
connectionState = .connecting
|
||||
let task = session.webSocketTask(with: endpoint)
|
||||
self.task = task
|
||||
task.resume()
|
||||
connectionState = .connected
|
||||
receiveLoop()
|
||||
}
|
||||
|
||||
func disconnect() {
|
||||
task?.cancel(with: .goingAway, reason: nil)
|
||||
task = nil
|
||||
connectionState = .disconnected
|
||||
}
|
||||
|
||||
func sendPCMChunk(_ data: Data) {
|
||||
guard let task else { return }
|
||||
|
||||
task.send(.data(data)) { [weak self] error in
|
||||
guard let error else { return }
|
||||
Task { @MainActor in
|
||||
self?.connectionState = .failed(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func sendText(_ text: String) {
|
||||
guard let task else { return }
|
||||
|
||||
task.send(.string(text)) { [weak self] error in
|
||||
guard let error else { return }
|
||||
Task { @MainActor in
|
||||
self?.connectionState = .failed(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Optional protocol hint for servers that distinguish press/release.
|
||||
func sendControlEvent(_ name: String) {
|
||||
sendText(#"{"type":"\#(name)"}"#)
|
||||
}
|
||||
|
||||
private func receiveLoop() {
|
||||
task?.receive { [weak self] result in
|
||||
Task { @MainActor in
|
||||
guard let self else { return }
|
||||
|
||||
switch result {
|
||||
case .success(let message):
|
||||
self.handle(message)
|
||||
self.receiveLoop()
|
||||
case .failure(let error):
|
||||
self.task = nil
|
||||
self.connectionState = .failed(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func handle(_ message: URLSessionWebSocketTask.Message) {
|
||||
switch message {
|
||||
case .data(let data):
|
||||
if data.isWAV {
|
||||
playWAV(data)
|
||||
} else {
|
||||
playRawPCMChunk(data)
|
||||
}
|
||||
case .string(let text):
|
||||
handleControlMessage(text)
|
||||
@unknown default:
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
private func handleControlMessage(_ text: String) {
|
||||
if text == "done" || text.contains(#""type":"done""#) {
|
||||
playbackState = .idle
|
||||
}
|
||||
}
|
||||
|
||||
private func playWAV(_ data: Data) {
|
||||
do {
|
||||
try configurePlaybackSession()
|
||||
avPlayer = try AVAudioPlayer(data: data)
|
||||
avPlayer?.delegate = self
|
||||
avPlayer?.prepareToPlay()
|
||||
avPlayer?.play()
|
||||
playbackState = .playing
|
||||
} catch {
|
||||
playbackState = .failed(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
|
||||
private func configureRawPlaybackEngine() {
|
||||
rawPlaybackEngine.attach(rawPlaybackNode)
|
||||
rawPlaybackEngine.connect(rawPlaybackNode, to: rawPlaybackEngine.mainMixerNode, format: rawPlaybackFormat)
|
||||
}
|
||||
|
||||
private func playRawPCMChunk(_ data: Data) {
|
||||
do {
|
||||
try configurePlaybackSession()
|
||||
|
||||
if !rawPlaybackEngine.isRunning {
|
||||
try rawPlaybackEngine.start()
|
||||
}
|
||||
if !rawPlaybackNode.isPlaying {
|
||||
rawPlaybackNode.play()
|
||||
}
|
||||
|
||||
guard let buffer = data.makePCMBuffer(format: rawPlaybackFormat) else { return }
|
||||
rawPlaybackNode.scheduleBuffer(buffer, completionHandler: nil)
|
||||
playbackState = .playing
|
||||
} catch {
|
||||
playbackState = .failed(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
|
||||
private func configurePlaybackSession() throws {
|
||||
let session = AVAudioSession.sharedInstance()
|
||||
try session.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker])
|
||||
try session.setActive(true)
|
||||
}
|
||||
}
|
||||
|
||||
extension AudioStreamManager: AVAudioPlayerDelegate {
|
||||
nonisolated func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool) {
|
||||
Task { @MainActor in
|
||||
self.playbackState = .idle
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private extension Data {
|
||||
var isWAV: Bool {
|
||||
count >= 12 &&
|
||||
self[0] == 0x52 && self[1] == 0x49 && self[2] == 0x46 && self[3] == 0x46 &&
|
||||
self[8] == 0x57 && self[9] == 0x41 && self[10] == 0x56 && self[11] == 0x45
|
||||
}
|
||||
|
||||
func makePCMBuffer(format: AVAudioFormat) -> AVAudioPCMBuffer? {
|
||||
let bytesPerFrame = Int(format.streamDescription.pointee.mBytesPerFrame)
|
||||
guard bytesPerFrame > 0 else { return nil }
|
||||
|
||||
let frameCount = AVAudioFrameCount(count / bytesPerFrame)
|
||||
guard let buffer = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: frameCount) else { return nil }
|
||||
buffer.frameLength = frameCount
|
||||
|
||||
let audioBuffer = buffer.audioBufferList.pointee.mBuffers
|
||||
guard let destination = audioBuffer.mData else { return nil }
|
||||
|
||||
withUnsafeBytes { rawBuffer in
|
||||
guard let source = rawBuffer.baseAddress else { return }
|
||||
destination.copyMemory(from: source, byteCount: Int(audioBuffer.mDataByteSize))
|
||||
}
|
||||
|
||||
return buffer
|
||||
}
|
||||
}
|
||||
+135
@@ -0,0 +1,135 @@
|
||||
import SwiftUI
|
||||
|
||||
struct ContentView: View {
|
||||
@StateObject private var streamManager: AudioStreamManager
|
||||
@StateObject private var captureManager = AudioCaptureManager()
|
||||
|
||||
@State private var isPressing = false
|
||||
|
||||
init() {
|
||||
let manager = AudioStreamManager(endpoint: URL(string: "ws://192.168.68.126:8642/stream")!)
|
||||
_streamManager = StateObject(wrappedValue: manager)
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: 14) {
|
||||
Text("Hermes")
|
||||
.font(.headline)
|
||||
|
||||
Text(statusText)
|
||||
.font(.caption2)
|
||||
.foregroundStyle(statusColor)
|
||||
.lineLimit(2)
|
||||
.multilineTextAlignment(.center)
|
||||
.frame(minHeight: 28)
|
||||
|
||||
Circle()
|
||||
.fill(buttonFill)
|
||||
.overlay {
|
||||
Image(systemName: isPressing ? "waveform" : "mic.fill")
|
||||
.font(.system(size: 36, weight: .semibold))
|
||||
.foregroundStyle(.white)
|
||||
}
|
||||
.overlay {
|
||||
Circle()
|
||||
.stroke(.white.opacity(isPressing ? 0.9 : 0.25), lineWidth: 3)
|
||||
}
|
||||
.frame(width: 112, height: 112)
|
||||
.scaleEffect(isPressing ? 0.94 : 1.0)
|
||||
.animation(.spring(response: 0.2, dampingFraction: 0.75), value: isPressing)
|
||||
.gesture(
|
||||
DragGesture(minimumDistance: 0)
|
||||
.onChanged { _ in
|
||||
guard !isPressing else { return }
|
||||
isPressing = true
|
||||
startPTT()
|
||||
}
|
||||
.onEnded { _ in
|
||||
isPressing = false
|
||||
stopPTT()
|
||||
}
|
||||
)
|
||||
|
||||
Text(isPressing ? "Release to send" : "Hold to talk")
|
||||
.font(.caption2)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
.padding()
|
||||
.onAppear {
|
||||
streamManager.connect()
|
||||
captureManager.onPCMChunk = { [streamManager] chunk in
|
||||
streamManager.sendPCMChunk(chunk)
|
||||
}
|
||||
}
|
||||
.onDisappear {
|
||||
captureManager.stopRecording()
|
||||
streamManager.disconnect()
|
||||
}
|
||||
}
|
||||
|
||||
private var statusText: String {
|
||||
switch captureManager.state {
|
||||
case .recording:
|
||||
return "Listening"
|
||||
case .starting:
|
||||
return "Starting mic"
|
||||
case .failed(let message):
|
||||
return message
|
||||
case .idle:
|
||||
break
|
||||
}
|
||||
|
||||
switch streamManager.playbackState {
|
||||
case .playing:
|
||||
return "Playing response"
|
||||
case .failed(let message):
|
||||
return message
|
||||
case .idle:
|
||||
break
|
||||
}
|
||||
|
||||
switch streamManager.connectionState {
|
||||
case .connected:
|
||||
return "Ready"
|
||||
case .connecting:
|
||||
return "Connecting"
|
||||
case .disconnected:
|
||||
return "Disconnected"
|
||||
case .failed(let message):
|
||||
return message
|
||||
}
|
||||
}
|
||||
|
||||
private var statusColor: Color {
|
||||
if case .failed = captureManager.state { return .red }
|
||||
if case .failed = streamManager.connectionState { return .red }
|
||||
if case .failed = streamManager.playbackState { return .red }
|
||||
if captureManager.state == .recording { return .green }
|
||||
if streamManager.playbackState == .playing { return .blue }
|
||||
return .secondary
|
||||
}
|
||||
|
||||
private var buttonFill: Color {
|
||||
if captureManager.state == .recording { return .red }
|
||||
if streamManager.playbackState == .playing { return .blue }
|
||||
return .accentColor
|
||||
}
|
||||
|
||||
private func startPTT() {
|
||||
streamManager.connect()
|
||||
streamManager.sendControlEvent("start")
|
||||
|
||||
Task {
|
||||
await captureManager.startRecording()
|
||||
}
|
||||
}
|
||||
|
||||
private func stopPTT() {
|
||||
captureManager.stopRecording()
|
||||
streamManager.sendControlEvent("stop")
|
||||
}
|
||||
}
|
||||
|
||||
#Preview {
|
||||
ContentView()
|
||||
}
|
||||
+9
@@ -0,0 +1,9 @@
|
||||
<!-- Add these entries inside the top-level <dict> of the watchOS target Info.plist. -->
|
||||
<key>NSMicrophoneUsageDescription</key>
|
||||
<string>Hermes needs microphone access so you can hold the talk button and speak to your assistant.</string>
|
||||
<key>NSLocalNetworkUsageDescription</key>
|
||||
<string>Hermes connects to your local assistant server over your Wi-Fi network.</string>
|
||||
<key>UIBackgroundModes</key>
|
||||
<array>
|
||||
<string>audio</string>
|
||||
</array>
|
||||
@@ -0,0 +1,18 @@
|
||||
# Hermes Watch PTT Foundation
|
||||
|
||||
These files are intended to be added to a standalone watchOS SwiftUI app target:
|
||||
|
||||
- `AudioCaptureManager.swift`: captures microphone audio with `AVAudioEngine`, converts it to 16 kHz, 16-bit, mono PCM, and emits raw `Data` chunks.
|
||||
- `AudioStreamManager.swift`: opens a `URLSessionWebSocketTask`, sends binary PCM chunks, receives binary response audio, and plays either complete WAV blobs or raw 16 kHz PCM chunks.
|
||||
- `ContentView.swift`: minimal push-to-talk UI using `DragGesture(minimumDistance: 0)` for press/release.
|
||||
- `InfoPlistAdditions.xml`: exact plist entries for microphone, LAN access, and background audio.
|
||||
|
||||
The referenced ESP32 demo posts a complete WAV file to `http://192.168.68.126:8642/api/esp32/voice` with 16 kHz, 16-bit, mono PCM and receives WAV audio back. The watch implementation here follows your requested WebSocket model: it streams raw PCM chunks while the button is held, sends simple `{"type":"start"}` and `{"type":"stop"}` text control messages, and accepts either WAV or raw PCM binary responses.
|
||||
|
||||
Update the endpoint in `ContentView.init()`:
|
||||
|
||||
```swift
|
||||
AudioStreamManager(endpoint: URL(string: "ws://YOUR_LOCAL_IP:PORT/stream")!)
|
||||
```
|
||||
|
||||
If your Hermes WebSocket endpoint expects authentication headers, create a `URLRequest`, set the headers, and pass that request into `session.webSocketTask(with:)` in `AudioStreamManager.connect()`.
|
||||
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
|
||||
DEFAULT_URL = os.environ.get("KSAY_URL", "http://127.0.0.1:7332")
|
||||
LAUNCHD_LABEL = os.environ.get("KSAY_LAUNCHD_LABEL", "com.local.ksay-kokoro")
|
||||
|
||||
|
||||
def read_stdin_if_needed(text_parts: list[str]) -> str:
|
||||
if text_parts:
|
||||
return " ".join(text_parts)
|
||||
if not sys.stdin.isatty():
|
||||
return sys.stdin.read()
|
||||
return ""
|
||||
|
||||
|
||||
def request_json(method: str, path: str, payload: dict | None = None) -> dict:
|
||||
data = None
|
||||
headers = {}
|
||||
if payload is not None:
|
||||
data = json.dumps(payload).encode("utf-8")
|
||||
headers["content-type"] = "application/json"
|
||||
req = urllib.request.Request(
|
||||
f"{DEFAULT_URL}{path}",
|
||||
data=data,
|
||||
headers=headers,
|
||||
method=method,
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=120) as res:
|
||||
return json.loads(res.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as exc:
|
||||
body = exc.read().decode("utf-8", errors="replace")
|
||||
try:
|
||||
return json.loads(body)
|
||||
except json.JSONDecodeError:
|
||||
raise RuntimeError(f"ksay server returned HTTP {exc.code}: {body}") from exc
|
||||
|
||||
|
||||
def try_wake_service() -> None:
|
||||
domain = f"gui/{os.getuid()}"
|
||||
subprocess.run(
|
||||
["launchctl", "kickstart", "-k", f"{domain}/{LAUNCHD_LABEL}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
def post_say(payload: dict) -> dict:
|
||||
try:
|
||||
return request_json("POST", "/say", payload)
|
||||
except (
|
||||
ConnectionError,
|
||||
ConnectionResetError,
|
||||
http.client.RemoteDisconnected,
|
||||
urllib.error.URLError,
|
||||
):
|
||||
try_wake_service()
|
||||
time.sleep(1.0)
|
||||
return request_json("POST", "/say", payload)
|
||||
|
||||
|
||||
def play_audio(path: str) -> None:
|
||||
subprocess.run(["/usr/bin/afplay", path], check=True)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="ksay",
|
||||
description="Fast warm Kokoro TTS, intended as a neural replacement for macOS say.",
|
||||
)
|
||||
parser.add_argument("text", nargs="*", help="Text to speak. Reads stdin when omitted.")
|
||||
parser.add_argument("-v", "--voice", default=os.environ.get("KSAY_VOICE", "af_heart"))
|
||||
parser.add_argument("-r", "--rate", type=float, default=None, help="Compatibility alias; maps words/minute-ish values to speed.")
|
||||
parser.add_argument("--speed", type=float, default=None, help="Kokoro speed multiplier.")
|
||||
parser.add_argument("--lang-code", default=os.environ.get("KSAY_LANG_CODE", "a"))
|
||||
parser.add_argument("-o", "--output", help="Write WAV to this path.")
|
||||
parser.add_argument("--no-play", action="store_true", help="Generate the file without playing it.")
|
||||
parser.add_argument("--json", action="store_true", help="Print the server response as JSON.")
|
||||
parser.add_argument("--status", action="store_true", help="Show warm server health.")
|
||||
parser.add_argument("--start", action="store_true", help="Kick the launchd service and wait for health.")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.status:
|
||||
print(json.dumps(request_json("GET", "/health"), indent=2))
|
||||
return 0
|
||||
|
||||
if args.start:
|
||||
try_wake_service()
|
||||
for _ in range(60):
|
||||
try:
|
||||
print(json.dumps(request_json("GET", "/health"), indent=2))
|
||||
return 0
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
print("ksay service did not become healthy within 60s.", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
text = read_stdin_if_needed(args.text).strip()
|
||||
if not text:
|
||||
parser.error("text is required, or pipe text on stdin")
|
||||
|
||||
speed = args.speed
|
||||
if speed is None:
|
||||
speed = 1.0
|
||||
if args.rate:
|
||||
speed = max(0.5, min(2.0, args.rate / 180.0))
|
||||
|
||||
result = post_say(
|
||||
{
|
||||
"text": text,
|
||||
"voice": args.voice,
|
||||
"speed": speed,
|
||||
"langCode": args.lang_code,
|
||||
"output": args.output,
|
||||
}
|
||||
)
|
||||
if not result.get("ok"):
|
||||
print(result.get("error", "ksay failed"), file=sys.stderr)
|
||||
return 1
|
||||
|
||||
if args.json:
|
||||
print(json.dumps(result, indent=2))
|
||||
else:
|
||||
print(result["filePath"])
|
||||
|
||||
if not args.no_play:
|
||||
play_audio(result["filePath"])
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
+1160
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"name": "macmini-mcp",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"description": "Local MCP server for approved macOS app automation.",
|
||||
"type": "module",
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
},
|
||||
"scripts": {
|
||||
"start": "node src/http.js",
|
||||
"start:stdio": "node src/stdio.js",
|
||||
"dev": "node --watch src/http.js",
|
||||
"check": "node --check src/*.js && node --check src/integrations/*.js && node --test",
|
||||
"python:install": "./scripts/setup-python.sh",
|
||||
"service:install": "./scripts/install-service.sh",
|
||||
"service:restart": "./scripts/restart-service.sh",
|
||||
"service:status": "./scripts/status-service.sh",
|
||||
"service:uninstall": "./scripts/uninstall-service.sh",
|
||||
"ksay:install": "./scripts/install-ksay-service.sh",
|
||||
"ksay:restart": "./scripts/restart-ksay-service.sh"
|
||||
},
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "^1.0.0",
|
||||
"zod": "^3.25.0"
|
||||
},
|
||||
"devDependencies": {}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
tplinkrouterc6u==5.21.0
|
||||
mlx-audio
|
||||
misaki
|
||||
num2words
|
||||
spacy
|
||||
phonemizer
|
||||
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl
|
||||
+156
@@ -0,0 +1,156 @@
|
||||
#!/usr/bin/env python3
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from tplinkrouterc6u import TPLinkDecoClient
|
||||
|
||||
|
||||
def default_gateway():
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["/sbin/route", "-n", "get", "default"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
for line in result.stdout.splitlines():
|
||||
key, _, value = line.partition(":")
|
||||
if key.strip() == "gateway":
|
||||
return value.strip()
|
||||
return None
|
||||
|
||||
|
||||
def config():
|
||||
host = os.environ.get("DECO_HOST") or default_gateway()
|
||||
password = os.environ.get("DECO_PASSWORD")
|
||||
if not password:
|
||||
raise RuntimeError("DECO_PASSWORD must be set in .env.")
|
||||
if not host:
|
||||
raise RuntimeError("DECO_HOST must be set in .env; default gateway detection failed.")
|
||||
|
||||
verify_ssl = os.environ.get("DECO_VERIFY_SSL", "true").lower() not in {
|
||||
"0",
|
||||
"false",
|
||||
"no",
|
||||
}
|
||||
timeout = int(os.environ.get("DECO_TIMEOUT", "10"))
|
||||
return {
|
||||
"host": host,
|
||||
"password": password,
|
||||
"username": os.environ.get("DECO_USERNAME", "admin"),
|
||||
"verify_ssl": verify_ssl,
|
||||
"timeout": timeout,
|
||||
}
|
||||
|
||||
|
||||
def client():
|
||||
return TPLinkDecoClient(**config())
|
||||
|
||||
|
||||
def device_to_dict(device):
|
||||
return {
|
||||
"hostname": device.hostname,
|
||||
"mac": device.macaddr,
|
||||
"ip": device.ipaddr,
|
||||
"connection": getattr(device.type, "value", str(device.type)),
|
||||
"upSpeed": device.up_speed,
|
||||
"downSpeed": device.down_speed,
|
||||
"active": device.active,
|
||||
}
|
||||
|
||||
|
||||
def firmware_to_dict(firmware):
|
||||
return {
|
||||
"model": firmware.model,
|
||||
"hardwareVersion": firmware.hardware_version,
|
||||
"firmwareVersion": firmware.firmware_version,
|
||||
}
|
||||
|
||||
|
||||
def ipv4_to_dict(status):
|
||||
return {
|
||||
"wanMac": status.wan_macaddr,
|
||||
"wanIp": status.wan_ipv4_ipaddr,
|
||||
"wanGateway": status.wan_ipv4_gateway,
|
||||
"wanConnectionType": status.wan_ipv4_conntype,
|
||||
"wanNetmask": status.wan_ipv4_netmask,
|
||||
"wanPrimaryDns": status.wan_ipv4_pridns,
|
||||
"wanSecondaryDns": status.wan_ipv4_snddns,
|
||||
"lanMac": status.lan_macaddr,
|
||||
"lanIp": status.lan_ipv4_ipaddr,
|
||||
"lanNetmask": status.lan_ipv4_netmask,
|
||||
}
|
||||
|
||||
|
||||
def status_to_dict(status, include_clients=True):
|
||||
data = {
|
||||
"wanMac": status.wan_macaddr,
|
||||
"lanMac": status.lan_macaddr,
|
||||
"wanIp": status.wan_ipv4_addr,
|
||||
"lanIp": status.lan_ipv4_addr,
|
||||
"wanGateway": status.wan_ipv4_gateway,
|
||||
"connectionType": status.conn_type,
|
||||
"cpuUsage": status.cpu_usage,
|
||||
"memoryUsage": status.mem_usage,
|
||||
"clientsTotal": status.clients_total,
|
||||
"wiredClientsTotal": status.wired_total,
|
||||
"wifiClientsTotal": status.wifi_clients_total,
|
||||
"guestClientsTotal": status.guest_clients_total,
|
||||
"iotClientsTotal": status.iot_clients_total,
|
||||
"wifi": {
|
||||
"host2g": status.wifi_2g_enable,
|
||||
"host5g": status.wifi_5g_enable,
|
||||
"host6g": status.wifi_6g_enable,
|
||||
"guest2g": status.guest_2g_enable,
|
||||
"guest5g": status.guest_5g_enable,
|
||||
"guest6g": status.guest_6g_enable,
|
||||
},
|
||||
}
|
||||
if include_clients:
|
||||
data["clients"] = [device_to_dict(device) for device in status.devices]
|
||||
return data
|
||||
|
||||
|
||||
def run(action):
|
||||
deco = client()
|
||||
try:
|
||||
if action == "overview":
|
||||
status = deco.get_status()
|
||||
firmware = deco.get_firmware()
|
||||
return {
|
||||
"status": status_to_dict(status, include_clients=False),
|
||||
"firmware": firmware_to_dict(firmware),
|
||||
}
|
||||
if action == "clients":
|
||||
status = deco.get_status()
|
||||
return {"clients": [device_to_dict(device) for device in status.devices]}
|
||||
if action == "ipv4":
|
||||
return ipv4_to_dict(deco.get_ipv4_status())
|
||||
if action == "firmware":
|
||||
return firmware_to_dict(deco.get_firmware())
|
||||
raise RuntimeError(f"Unknown action: {action}")
|
||||
finally:
|
||||
try:
|
||||
deco.logout()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 2:
|
||||
raise RuntimeError("Usage: deco_bridge.py <overview|clients|ipv4|firmware>")
|
||||
print(json.dumps(run(sys.argv[1]), indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except Exception as err:
|
||||
print(json.dumps({"error": str(err)}), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
@@ -0,0 +1,383 @@
|
||||
#!/usr/bin/env python3
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import ssl
|
||||
import subprocess
|
||||
import sys
|
||||
from urllib.parse import quote_plus
|
||||
|
||||
import requests
|
||||
from Crypto.Cipher import AES, PKCS1_v1_5
|
||||
from Crypto.PublicKey import RSA
|
||||
from Crypto.Util.Padding import pad, unpad
|
||||
from requests import RequestException
|
||||
|
||||
|
||||
AES_KEY_BYTES = 16
|
||||
MIN_AES_KEY = 10 ** (AES_KEY_BYTES - 1)
|
||||
MAX_AES_KEY = (10**AES_KEY_BYTES) - 1
|
||||
PKCS1_V1_5_HEADER_BYTES = 11
|
||||
|
||||
|
||||
def load_env_file():
|
||||
path = os.path.join(os.path.dirname(os.path.dirname(__file__)), ".env")
|
||||
try:
|
||||
with open(path, "r", encoding="utf-8") as env_file:
|
||||
for line in env_file:
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, value = line.split("=", 1)
|
||||
os.environ.setdefault(key, value.strip().strip("\"'"))
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
|
||||
|
||||
def default_gateway():
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["/sbin/route", "-n", "get", "default"],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
for line in result.stdout.splitlines():
|
||||
key, _, value = line.partition(":")
|
||||
if key.strip() == "gateway":
|
||||
return value.strip()
|
||||
return None
|
||||
|
||||
|
||||
def config():
|
||||
load_env_file()
|
||||
host = os.environ.get("DECO_HOST") or default_gateway()
|
||||
password = os.environ.get("DECO_PASSWORD")
|
||||
if not password:
|
||||
raise RuntimeError("DECO_PASSWORD must be set in .env.")
|
||||
if not host:
|
||||
raise RuntimeError("DECO_HOST must be set in .env; default gateway detection failed.")
|
||||
if not host.startswith(("http://", "https://")):
|
||||
host = f"http://{host}"
|
||||
verify_ssl = os.environ.get("DECO_VERIFY_SSL", "true").lower() not in {
|
||||
"0",
|
||||
"false",
|
||||
"no",
|
||||
}
|
||||
return {
|
||||
"host": host.rstrip("/"),
|
||||
"username": os.environ.get("DECO_USERNAME", "admin"),
|
||||
"password": password,
|
||||
"verify_ssl": verify_ssl,
|
||||
"timeout": int(os.environ.get("DECO_TIMEOUT", "10")),
|
||||
}
|
||||
|
||||
|
||||
def byte_len(n):
|
||||
return (int(math.log2(n)) + 8) >> 3
|
||||
|
||||
|
||||
def rsa_encrypt(n, e, plaintext):
|
||||
public_key = RSA.construct((n, e)).publickey()
|
||||
encryptor = PKCS1_v1_5.new(public_key)
|
||||
block_size = byte_len(n)
|
||||
bytes_per_block = block_size - PKCS1_V1_5_HEADER_BYTES
|
||||
encrypted_text = ""
|
||||
for index in range(0, len(plaintext), bytes_per_block):
|
||||
encrypted_text += encryptor.encrypt(plaintext[index:index + bytes_per_block]).hex()
|
||||
return encrypted_text
|
||||
|
||||
|
||||
def decode_name(value):
|
||||
if not value:
|
||||
return value
|
||||
try:
|
||||
return base64.b64decode(value).decode()
|
||||
except Exception:
|
||||
return value
|
||||
|
||||
|
||||
def title_from_snake(value):
|
||||
if not value:
|
||||
return value
|
||||
return " ".join(part.title() for part in value.split("_"))
|
||||
|
||||
|
||||
def deco_name(device):
|
||||
return (
|
||||
decode_name(device.get("custom_nickname"))
|
||||
or title_from_snake(device.get("nickname"))
|
||||
or device.get("device_model")
|
||||
or device.get("mac")
|
||||
)
|
||||
|
||||
|
||||
class DecoApi:
|
||||
def __init__(self, host, username, password, verify_ssl, timeout):
|
||||
self.host = host
|
||||
self.username = username
|
||||
self.password = password
|
||||
self.verify_ssl = verify_ssl
|
||||
self.timeout = timeout
|
||||
self.session = requests.Session()
|
||||
self.aes_key = None
|
||||
self.aes_iv = None
|
||||
self.password_rsa_n = None
|
||||
self.password_rsa_e = None
|
||||
self.sign_rsa_n = None
|
||||
self.sign_rsa_e = None
|
||||
self.seq = None
|
||||
self.stok = None
|
||||
self.cookie = None
|
||||
|
||||
def generate_aes_key_and_iv(self):
|
||||
self.aes_key = str(secrets.randbelow(MAX_AES_KEY - MIN_AES_KEY) + MIN_AES_KEY).encode()
|
||||
self.aes_iv = str(secrets.randbelow(MAX_AES_KEY - MIN_AES_KEY) + MIN_AES_KEY).encode()
|
||||
|
||||
def post(self, context, path, params, data):
|
||||
headers = {"Content-Type": "application/json"}
|
||||
cookies = {}
|
||||
if self.cookie:
|
||||
name, _, value = self.cookie.partition("=")
|
||||
if name and value:
|
||||
cookies[name] = value
|
||||
try:
|
||||
response = self.session.post(
|
||||
f"{self.host}{path}",
|
||||
params=params,
|
||||
data=data,
|
||||
headers=headers,
|
||||
cookies=cookies,
|
||||
verify=self.verify_ssl,
|
||||
timeout=self.timeout,
|
||||
)
|
||||
except RequestException:
|
||||
response = self.curl_post(path, params, data, cookies)
|
||||
if response.status_code == 403:
|
||||
self.clear_auth()
|
||||
raise RuntimeError(f"{context}: forbidden")
|
||||
response.raise_for_status()
|
||||
|
||||
for cookie in response.headers.get("Set-Cookie", "").split(","):
|
||||
match = re.search(r"(sysauth=[A-Za-z0-9]+)", cookie)
|
||||
if match:
|
||||
self.cookie = match.group(1)
|
||||
break
|
||||
|
||||
result = response.json()
|
||||
error_code = result.get("error_code")
|
||||
if error_code not in (None, "", 0):
|
||||
raise RuntimeError(f"{context}: response error_code={error_code}")
|
||||
return result
|
||||
|
||||
def curl_post(self, path, params, data, cookies):
|
||||
url = f"{self.host}{path}"
|
||||
if params:
|
||||
query = "&".join(f"{key}={quote_plus(str(value))}" for key, value in params.items())
|
||||
url = f"{url}?{query}"
|
||||
command = [
|
||||
"curl",
|
||||
"-s",
|
||||
"-i",
|
||||
"--connect-timeout",
|
||||
str(self.timeout),
|
||||
"-X",
|
||||
"POST",
|
||||
url,
|
||||
"-H",
|
||||
"Content-Type: application/json",
|
||||
"--data-raw",
|
||||
data,
|
||||
]
|
||||
if not self.verify_ssl:
|
||||
command.insert(2, "-k")
|
||||
if cookies:
|
||||
command.extend(["-H", "Cookie: " + "; ".join(f"{k}={v}" for k, v in cookies.items())])
|
||||
|
||||
result = subprocess.run(command, check=True, capture_output=True, text=True, timeout=self.timeout + 5)
|
||||
head, separator, body = result.stdout.rpartition("\r\n\r\n")
|
||||
if not separator:
|
||||
head, _, body = result.stdout.rpartition("\n\n")
|
||||
status_match = re.search(r"HTTP/\S+\s+(\d+)", head)
|
||||
status = int(status_match.group(1)) if status_match else 200
|
||||
response = requests.Response()
|
||||
response.status_code = status
|
||||
response._content = body.encode()
|
||||
response.url = url
|
||||
for line in head.splitlines():
|
||||
if ":" not in line:
|
||||
continue
|
||||
key, value = line.split(":", 1)
|
||||
response.headers[key.strip()] = value.strip()
|
||||
return response
|
||||
|
||||
def fetch_keys(self):
|
||||
response = self.post(
|
||||
"Fetch keys",
|
||||
"/cgi-bin/luci/;stok=/login",
|
||||
{"form": "keys"},
|
||||
json.dumps({"operation": "read"}),
|
||||
)
|
||||
keys = response["result"]["password"]
|
||||
self.password_rsa_n = int(keys[0], 16)
|
||||
self.password_rsa_e = int(keys[1], 16)
|
||||
|
||||
def fetch_auth(self):
|
||||
response = self.post(
|
||||
"Fetch auth",
|
||||
"/cgi-bin/luci/;stok=/login",
|
||||
{"form": "auth"},
|
||||
json.dumps({"operation": "read"}),
|
||||
)
|
||||
auth = response["result"]
|
||||
self.sign_rsa_n = int(auth["key"][0], 16)
|
||||
self.sign_rsa_e = int(auth["key"][1], 16)
|
||||
self.seq = auth["seq"]
|
||||
|
||||
def encode_payload(self, payload):
|
||||
payload_json = json.dumps(payload, separators=(",", ":")).encode()
|
||||
encrypted = AES.new(self.aes_key, AES.MODE_CBC, self.aes_iv).encrypt(
|
||||
pad(payload_json, AES.block_size)
|
||||
)
|
||||
data = base64.b64encode(encrypted).decode()
|
||||
sign = self.encode_sign(len(data))
|
||||
return f"sign={sign}&data={quote_plus(data)}"
|
||||
|
||||
def encode_sign(self, data_len):
|
||||
auth_hash = hashlib.md5(f"{self.username}{self.password}".encode()).hexdigest()
|
||||
sign_text = (
|
||||
f"k={self.aes_key.decode()}&i={self.aes_iv.decode()}&h={auth_hash}&s={self.seq + data_len}"
|
||||
)
|
||||
return rsa_encrypt(self.sign_rsa_n, self.sign_rsa_e, sign_text.encode())
|
||||
|
||||
def decrypt_data(self, context, data):
|
||||
if not data:
|
||||
self.clear_auth()
|
||||
raise RuntimeError(f"{context}: empty data")
|
||||
decrypted = AES.new(self.aes_key, AES.MODE_CBC, self.aes_iv).decrypt(
|
||||
base64.b64decode(data)
|
||||
)
|
||||
return json.loads(unpad(decrypted, AES.block_size).decode())
|
||||
|
||||
def login(self):
|
||||
if self.aes_key is None:
|
||||
self.generate_aes_key_and_iv()
|
||||
if self.password_rsa_n is None:
|
||||
self.fetch_keys()
|
||||
if self.seq is None:
|
||||
self.fetch_auth()
|
||||
encrypted_password = rsa_encrypt(
|
||||
self.password_rsa_n,
|
||||
self.password_rsa_e,
|
||||
self.password.encode(),
|
||||
)
|
||||
response = self.post(
|
||||
"Login",
|
||||
"/cgi-bin/luci/;stok=/login",
|
||||
{"form": "login"},
|
||||
self.encode_payload({
|
||||
"operation": "login",
|
||||
"params": {"password": encrypted_password},
|
||||
}),
|
||||
)
|
||||
data = self.decrypt_data("Login", response["data"])
|
||||
if data.get("error_code") != 0:
|
||||
result = data.get("result") or {}
|
||||
attempts = result.get("attemptsAllowed", "unknown")
|
||||
raise RuntimeError(f"Login failed: error_code={data.get('error_code')}; attempts={attempts}")
|
||||
self.stok = data["result"]["stok"]
|
||||
if not self.cookie:
|
||||
raise RuntimeError("Login succeeded but no sysauth cookie was returned.")
|
||||
|
||||
def clear_auth(self):
|
||||
self.seq = None
|
||||
self.stok = None
|
||||
self.cookie = None
|
||||
|
||||
def call(self, context, section, form, payload):
|
||||
if not self.stok or not self.cookie:
|
||||
self.login()
|
||||
response = self.post(
|
||||
context,
|
||||
f"/cgi-bin/luci/;stok={self.stok}/admin/{section}",
|
||||
{"form": form},
|
||||
self.encode_payload(payload),
|
||||
)
|
||||
data = self.decrypt_data(context, response["data"])
|
||||
error_code = data.get("error_code") or data.get("errorcode")
|
||||
if error_code:
|
||||
raise RuntimeError(f"{context}: decoded error_code={error_code}")
|
||||
return data["result"]
|
||||
|
||||
def list_decos(self):
|
||||
devices = self.call("List Devices", "device", "device_list", {"operation": "read"}).get("device_list", [])
|
||||
return [
|
||||
{
|
||||
"name": deco_name(device),
|
||||
"mac": device.get("mac"),
|
||||
"ip": device.get("device_ip"),
|
||||
"model": device.get("device_model"),
|
||||
"hardwareVersion": device.get("hardware_ver"),
|
||||
"firmwareVersion": device.get("software_ver"),
|
||||
"role": device.get("role"),
|
||||
"online": device.get("group_status") == "connected",
|
||||
"connectionType": device.get("connection_type"),
|
||||
}
|
||||
for device in devices
|
||||
]
|
||||
|
||||
def list_clients_for_deco(self, deco):
|
||||
clients = self.call(
|
||||
f"List Clients {deco['mac']}",
|
||||
"client",
|
||||
"client_list",
|
||||
{"operation": "read", "params": {"device_mac": deco["mac"]}},
|
||||
).get("client_list", [])
|
||||
return [
|
||||
{
|
||||
"hostname": decode_name(client.get("name")),
|
||||
"mac": client.get("mac"),
|
||||
"ip": client.get("ip"),
|
||||
"connection": client.get("connection_type"),
|
||||
"interface": client.get("interface"),
|
||||
"upSpeed": client.get("up_speed"),
|
||||
"downSpeed": client.get("down_speed"),
|
||||
"active": client.get("online"),
|
||||
"linkedDecoMac": deco.get("mac"),
|
||||
"linkedDecoName": deco.get("name"),
|
||||
"linkedDecoRole": deco.get("role"),
|
||||
}
|
||||
for client in clients
|
||||
if client.get("online")
|
||||
]
|
||||
|
||||
|
||||
def run():
|
||||
if not config()["verify_ssl"]:
|
||||
requests.packages.urllib3.disable_warnings()
|
||||
ssl._create_default_https_context = ssl._create_unverified_context
|
||||
deco = DecoApi(**config())
|
||||
decos = deco.list_decos()
|
||||
clients = {}
|
||||
for node in decos:
|
||||
if not node.get("mac"):
|
||||
continue
|
||||
for client in deco.list_clients_for_deco(node):
|
||||
clients[client["mac"]] = client
|
||||
return {"decos": decos, "clients": list(clients.values())}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
print(json.dumps(run(), indent=2))
|
||||
except Exception as err:
|
||||
print(json.dumps({"error": str(err)}), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
Executable
+64
@@ -0,0 +1,64 @@
|
||||
#!/bin/zsh
|
||||
set -euo pipefail
|
||||
|
||||
LABEL="com.local.ksay-kokoro"
|
||||
PROJECT_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
PYTHON_BIN="$PROJECT_DIR/.venv/bin/python"
|
||||
AGENT_DIR="$HOME/Library/LaunchAgents"
|
||||
PLIST="$AGENT_DIR/$LABEL.plist"
|
||||
LOG_DIR="$PROJECT_DIR/.logs"
|
||||
DOMAIN="gui/$(id -u)"
|
||||
CLI_LINK="/opt/homebrew/bin/ksay"
|
||||
|
||||
if [[ ! -x "$PYTHON_BIN" ]]; then
|
||||
echo "Missing $PYTHON_BIN. Run npm run python:install first." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mkdir -p "$AGENT_DIR" "$LOG_DIR"
|
||||
|
||||
cat > "$PLIST" <<PLIST
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>$LABEL</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>$PYTHON_BIN</string>
|
||||
<string>$PROJECT_DIR/scripts/ksay_server.py</string>
|
||||
</array>
|
||||
<key>WorkingDirectory</key>
|
||||
<string>$PROJECT_DIR</string>
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
<key>KeepAlive</key>
|
||||
<true/>
|
||||
<key>ThrottleInterval</key>
|
||||
<integer>2</integer>
|
||||
<key>StandardOutPath</key>
|
||||
<string>$LOG_DIR/ksay.out.log</string>
|
||||
<key>StandardErrorPath</key>
|
||||
<string>$LOG_DIR/ksay.err.log</string>
|
||||
</dict>
|
||||
</plist>
|
||||
PLIST
|
||||
|
||||
plutil -lint "$PLIST"
|
||||
launchctl bootout "$DOMAIN" "$PLIST" 2>/dev/null || true
|
||||
launchctl bootstrap "$DOMAIN" "$PLIST"
|
||||
launchctl enable "$DOMAIN/$LABEL"
|
||||
launchctl kickstart -k "$DOMAIN/$LABEL"
|
||||
|
||||
if [[ -d "$(dirname "$CLI_LINK")" && -w "$(dirname "$CLI_LINK")" ]]; then
|
||||
ln -sf "$PROJECT_DIR/bin/ksay" "$CLI_LINK"
|
||||
echo "Linked CLI: $CLI_LINK"
|
||||
else
|
||||
echo "Could not link $CLI_LINK. Add $PROJECT_DIR/bin to PATH or link bin/ksay manually." >&2
|
||||
fi
|
||||
|
||||
echo "Installed $LABEL"
|
||||
echo "Endpoint: http://127.0.0.1:7332"
|
||||
echo "Try: ksay --start"
|
||||
echo "Logs: $LOG_DIR/ksay.*.log"
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
#!/bin/zsh
|
||||
set -euo pipefail
|
||||
|
||||
LABEL="com.local.macmini-mcp"
|
||||
PROJECT_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
NODE_BIN="$(command -v node)"
|
||||
AGENT_DIR="$HOME/Library/LaunchAgents"
|
||||
PLIST="$AGENT_DIR/$LABEL.plist"
|
||||
LOG_DIR="$PROJECT_DIR/.logs"
|
||||
DOMAIN="gui/$(id -u)"
|
||||
|
||||
mkdir -p "$AGENT_DIR" "$LOG_DIR"
|
||||
|
||||
cat > "$PLIST" <<PLIST
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>$LABEL</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>$NODE_BIN</string>
|
||||
<string>--watch</string>
|
||||
<string>src/http.js</string>
|
||||
</array>
|
||||
<key>WorkingDirectory</key>
|
||||
<string>$PROJECT_DIR</string>
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
<key>KeepAlive</key>
|
||||
<true/>
|
||||
<key>ThrottleInterval</key>
|
||||
<integer>2</integer>
|
||||
<key>StandardOutPath</key>
|
||||
<string>$LOG_DIR/service.out.log</string>
|
||||
<key>StandardErrorPath</key>
|
||||
<string>$LOG_DIR/service.err.log</string>
|
||||
</dict>
|
||||
</plist>
|
||||
PLIST
|
||||
|
||||
plutil -lint "$PLIST"
|
||||
launchctl bootout "$DOMAIN" "$PLIST" 2>/dev/null || true
|
||||
launchctl bootstrap "$DOMAIN" "$PLIST"
|
||||
launchctl enable "$DOMAIN/$LABEL"
|
||||
launchctl kickstart -k "$DOMAIN/$LABEL"
|
||||
|
||||
echo "Installed $LABEL"
|
||||
echo "Endpoint: configured by .env (see the service log for the listening URL)"
|
||||
echo "Logs: $LOG_DIR"
|
||||
+210
@@ -0,0 +1,210 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
import traceback
|
||||
import uuid
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from threading import Lock
|
||||
from typing import Any
|
||||
|
||||
|
||||
PROJECT_DIR = Path(__file__).resolve().parents[1]
|
||||
OUTPUT_DIR = Path(os.environ.get("KSAY_OUTPUT_DIR", PROJECT_DIR / "generated-audio"))
|
||||
DEFAULT_HOST = os.environ.get("KSAY_HOST", "127.0.0.1")
|
||||
DEFAULT_PORT = int(os.environ.get("KSAY_PORT", "7332"))
|
||||
DEFAULT_MODEL = os.environ.get("KSAY_MODEL", "mlx-community/Kokoro-82M-8bit")
|
||||
DEFAULT_VOICE = os.environ.get("KSAY_VOICE", "af_heart")
|
||||
DEFAULT_LANG_CODE = os.environ.get("KSAY_LANG_CODE", "a")
|
||||
MAX_TEXT_CHARS = int(os.environ.get("KSAY_MAX_TEXT_CHARS", "8000"))
|
||||
|
||||
|
||||
class KokoroEngine:
|
||||
def __init__(self, model_name: str):
|
||||
self.model_name = model_name
|
||||
self.model = None
|
||||
self.loaded_at = None
|
||||
self.load_seconds = None
|
||||
self.lock = Lock()
|
||||
|
||||
def load(self) -> None:
|
||||
started = time.perf_counter()
|
||||
from mlx_audio.tts.utils import load_model
|
||||
|
||||
self.model = load_model(self.model_name)
|
||||
self.loaded_at = time.time()
|
||||
self.load_seconds = time.perf_counter() - started
|
||||
|
||||
def synthesize(
|
||||
self,
|
||||
*,
|
||||
text: str,
|
||||
voice: str,
|
||||
speed: float,
|
||||
lang_code: str,
|
||||
output: str | None,
|
||||
) -> dict[str, Any]:
|
||||
if self.model is None:
|
||||
raise RuntimeError("Kokoro model is not loaded.")
|
||||
|
||||
clean_text = text.strip()
|
||||
if not clean_text:
|
||||
raise ValueError("text is required.")
|
||||
clean_text = clean_text[:MAX_TEXT_CHARS]
|
||||
|
||||
output_path = resolve_output_path(output)
|
||||
started = time.perf_counter()
|
||||
|
||||
with self.lock:
|
||||
audio_chunks = []
|
||||
sample_rate = None
|
||||
for result in self.model.generate(
|
||||
text=clean_text,
|
||||
voice=voice,
|
||||
speed=speed,
|
||||
lang_code=lang_code,
|
||||
):
|
||||
audio_chunks.append(result.audio)
|
||||
sample_rate = result.sample_rate
|
||||
|
||||
if not audio_chunks:
|
||||
raise RuntimeError("Kokoro did not return audio.")
|
||||
|
||||
import mlx.core as mx
|
||||
import numpy as np
|
||||
from mlx_audio.audio_io import write as audio_write
|
||||
|
||||
audio = (
|
||||
mx.concatenate(audio_chunks, axis=0)
|
||||
if len(audio_chunks) > 1
|
||||
else audio_chunks[0]
|
||||
)
|
||||
audio_write(str(output_path), np.array(audio), sample_rate, format="wav")
|
||||
|
||||
elapsed = time.perf_counter() - started
|
||||
return {
|
||||
"ok": True,
|
||||
"filePath": str(output_path),
|
||||
"model": self.model_name,
|
||||
"voice": voice,
|
||||
"speed": speed,
|
||||
"langCode": lang_code,
|
||||
"sampleRate": sample_rate,
|
||||
"segments": len(audio_chunks),
|
||||
"seconds": round(elapsed, 3),
|
||||
"characters": len(clean_text),
|
||||
}
|
||||
|
||||
|
||||
def resolve_output_path(output: str | None) -> Path:
|
||||
if output:
|
||||
path = Path(output).expanduser()
|
||||
if path.suffix.lower() != ".wav":
|
||||
path = path.with_suffix(".wav")
|
||||
if not path.is_absolute():
|
||||
path = (Path.cwd() / path).resolve()
|
||||
else:
|
||||
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||
path = OUTPUT_DIR / f"ksay-{int(time.time() * 1000)}-{uuid.uuid4().hex[:6]}.wav"
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def parse_json_body(handler: BaseHTTPRequestHandler) -> dict[str, Any]:
|
||||
length = int(handler.headers.get("content-length", "0"))
|
||||
if length <= 0:
|
||||
return {}
|
||||
body = handler.rfile.read(length)
|
||||
return json.loads(body.decode("utf-8"))
|
||||
|
||||
|
||||
def make_handler(engine: KokoroEngine):
|
||||
class KsayHandler(BaseHTTPRequestHandler):
|
||||
server_version = "ksay-kokoro/0.1"
|
||||
|
||||
def log_message(self, fmt: str, *args: Any) -> None:
|
||||
sys.stderr.write("%s - %s\n" % (self.log_date_time_string(), fmt % args))
|
||||
|
||||
def write_json(self, status: int, value: dict[str, Any]) -> None:
|
||||
body = json.dumps(value).encode("utf-8")
|
||||
self.send_response(status)
|
||||
self.send_header("content-type", "application/json")
|
||||
self.send_header("content-length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_GET(self) -> None:
|
||||
if self.path == "/health":
|
||||
self.write_json(
|
||||
200,
|
||||
{
|
||||
"ok": True,
|
||||
"model": engine.model_name,
|
||||
"loaded": engine.model is not None,
|
||||
"loadedAt": engine.loaded_at,
|
||||
"loadSeconds": engine.load_seconds,
|
||||
"defaultVoice": DEFAULT_VOICE,
|
||||
"defaultLangCode": DEFAULT_LANG_CODE,
|
||||
},
|
||||
)
|
||||
return
|
||||
self.write_json(404, {"ok": False, "error": "not found"})
|
||||
|
||||
def do_POST(self) -> None:
|
||||
if self.path != "/say":
|
||||
self.write_json(404, {"ok": False, "error": "not found"})
|
||||
return
|
||||
|
||||
try:
|
||||
payload = parse_json_body(self)
|
||||
result = engine.synthesize(
|
||||
text=str(payload.get("text", "")),
|
||||
voice=str(payload.get("voice") or DEFAULT_VOICE),
|
||||
speed=float(payload.get("speed") or 1.0),
|
||||
lang_code=str(payload.get("langCode") or DEFAULT_LANG_CODE),
|
||||
output=payload.get("output"),
|
||||
)
|
||||
self.write_json(200, result)
|
||||
except Exception as exc:
|
||||
traceback.print_exc()
|
||||
self.write_json(500, {"ok": False, "error": str(exc)})
|
||||
|
||||
return KsayHandler
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Warm Kokoro TTS server for ksay.")
|
||||
parser.add_argument("--host", default=DEFAULT_HOST)
|
||||
parser.add_argument("--port", type=int, default=DEFAULT_PORT)
|
||||
parser.add_argument("--model", default=DEFAULT_MODEL)
|
||||
args = parser.parse_args()
|
||||
|
||||
engine = KokoroEngine(args.model)
|
||||
print(f"Loading {args.model}...", flush=True)
|
||||
engine.load()
|
||||
print(
|
||||
f"ksay Kokoro ready on http://{args.host}:{args.port} "
|
||||
f"after {engine.load_seconds:.2f}s",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
httpd = ThreadingHTTPServer((args.host, args.port), make_handler(engine))
|
||||
|
||||
def shutdown(_signum: int, _frame: Any) -> None:
|
||||
httpd.shutdown()
|
||||
|
||||
signal.signal(signal.SIGTERM, shutdown)
|
||||
signal.signal(signal.SIGINT, shutdown)
|
||||
httpd.serve_forever()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/bin/zsh
|
||||
set -euo pipefail
|
||||
|
||||
LABEL="com.local.ksay-kokoro"
|
||||
DOMAIN="gui/$(id -u)"
|
||||
|
||||
launchctl kickstart -k "$DOMAIN/$LABEL"
|
||||
echo "Restarted $LABEL"
|
||||
+8
@@ -0,0 +1,8 @@
|
||||
#!/bin/zsh
|
||||
set -euo pipefail
|
||||
|
||||
LABEL="com.local.macmini-mcp"
|
||||
DOMAIN="gui/$(id -u)"
|
||||
|
||||
launchctl kickstart -k "$DOMAIN/$LABEL"
|
||||
echo "Restarted $LABEL"
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
#!/bin/zsh
|
||||
set -euo pipefail
|
||||
|
||||
PROJECT_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$PROJECT_DIR"
|
||||
|
||||
if [[ ! -x .venv/bin/python ]]; then
|
||||
python3 -m venv .venv
|
||||
fi
|
||||
|
||||
.venv/bin/python -m pip install -r requirements.txt
|
||||
+7
@@ -0,0 +1,7 @@
|
||||
#!/bin/zsh
|
||||
set -euo pipefail
|
||||
|
||||
LABEL="com.local.macmini-mcp"
|
||||
DOMAIN="gui/$(id -u)"
|
||||
|
||||
launchctl print "$DOMAIN/$LABEL"
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
#!/bin/zsh
|
||||
set -euo pipefail
|
||||
|
||||
LABEL="com.local.macmini-mcp"
|
||||
PLIST="$HOME/Library/LaunchAgents/$LABEL.plist"
|
||||
DOMAIN="gui/$(id -u)"
|
||||
|
||||
launchctl bootout "$DOMAIN" "$PLIST" 2>/dev/null || true
|
||||
rm -f "$PLIST"
|
||||
echo "Uninstalled $LABEL"
|
||||
@@ -0,0 +1,49 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
export async function runJxa(script, input = {}) {
|
||||
try {
|
||||
const { stdout } = await execFileAsync(
|
||||
"/usr/bin/osascript",
|
||||
["-l", "JavaScript", "-e", script, "--", JSON.stringify(input)],
|
||||
{
|
||||
timeout: 30_000,
|
||||
maxBuffer: 2 * 1024 * 1024,
|
||||
},
|
||||
);
|
||||
|
||||
const text = stdout.trim();
|
||||
return text ? JSON.parse(text) : null;
|
||||
} catch (error) {
|
||||
const detail = error.stderr?.trim() || error.message;
|
||||
throw new Error(
|
||||
`macOS automation failed. Grant Automation access to the service if prompted. ${detail}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
export function dateFromInput(value, fieldName) {
|
||||
const date = new Date(value);
|
||||
if (Number.isNaN(date.valueOf())) {
|
||||
throw new Error(`${fieldName} must be a valid ISO-8601 date and time.`);
|
||||
}
|
||||
return date;
|
||||
}
|
||||
|
||||
export function plainTextToNoteHtml(title, body) {
|
||||
const escape = (text) =>
|
||||
text
|
||||
.replaceAll("&", "&")
|
||||
.replaceAll("<", "<")
|
||||
.replaceAll(">", ">")
|
||||
.replaceAll('"', """);
|
||||
|
||||
const paragraphs = body
|
||||
.split(/\n{2,}/)
|
||||
.map((paragraph) => `<div>${escape(paragraph).replaceAll("\n", "<br>")}</div>`)
|
||||
.join("");
|
||||
|
||||
return `<h1>${escape(title)}</h1>${paragraphs}`;
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import { loadEnvFile } from "node:process";
|
||||
|
||||
try {
|
||||
loadEnvFile();
|
||||
} catch (error) {
|
||||
if (error.code !== "ENOENT") {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function getConfig() {
|
||||
const port = Number.parseInt(process.env.MACMINI_MCP_PORT || "7331", 10);
|
||||
if (!Number.isInteger(port) || port < 1 || port > 65535) {
|
||||
throw new Error("MACMINI_MCP_PORT must be an integer between 1 and 65535.");
|
||||
}
|
||||
|
||||
const config = {
|
||||
host: process.env.MACMINI_MCP_HOST || "127.0.0.1",
|
||||
port,
|
||||
token: process.env.MACMINI_MCP_TOKEN || "",
|
||||
};
|
||||
|
||||
const isLoopback = ["127.0.0.1", "::1", "localhost"].includes(config.host);
|
||||
if (!isLoopback && !config.token) {
|
||||
throw new Error("MACMINI_MCP_TOKEN is required when listening beyond localhost.");
|
||||
}
|
||||
|
||||
return config;
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
import http from "node:http";
|
||||
import { timingSafeEqual } from "node:crypto";
|
||||
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
|
||||
import { getConfig } from "./config.js";
|
||||
import { createMacMiniMcpServer } from "./server.js";
|
||||
|
||||
const config = getConfig();
|
||||
|
||||
function authorized(request) {
|
||||
if (!config.token) {
|
||||
return true;
|
||||
}
|
||||
const authorization = request.headers.authorization || "";
|
||||
const expected = Buffer.from(`Bearer ${config.token}`);
|
||||
const provided = Buffer.from(authorization);
|
||||
return provided.length === expected.length && timingSafeEqual(provided, expected);
|
||||
}
|
||||
|
||||
const service = http.createServer(async (request, response) => {
|
||||
const pathname = new URL(request.url, `http://${request.headers.host || "localhost"}`).pathname;
|
||||
|
||||
if (pathname === "/health" && request.method === "GET") {
|
||||
response.writeHead(200, { "content-type": "application/json" });
|
||||
response.end(JSON.stringify({ ok: true, service: "macmini-mcp" }));
|
||||
return;
|
||||
}
|
||||
|
||||
if (pathname !== "/mcp") {
|
||||
response.writeHead(404).end("Not found");
|
||||
return;
|
||||
}
|
||||
|
||||
if (!authorized(request)) {
|
||||
response.writeHead(401, { "www-authenticate": "Bearer" }).end("Unauthorized");
|
||||
return;
|
||||
}
|
||||
|
||||
const server = createMacMiniMcpServer();
|
||||
const transport = new StreamableHTTPServerTransport({
|
||||
sessionIdGenerator: undefined,
|
||||
});
|
||||
|
||||
response.on("close", () => {
|
||||
transport.close();
|
||||
server.close();
|
||||
});
|
||||
|
||||
try {
|
||||
await server.connect(transport);
|
||||
await transport.handleRequest(request, response);
|
||||
} catch (error) {
|
||||
console.error("MCP HTTP request failed:", error);
|
||||
if (!response.headersSent) {
|
||||
response.writeHead(500).end("MCP request failed");
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
service.listen(config.port, config.host, () => {
|
||||
console.error(`macmini-mcp listening on http://${config.host}:${config.port}/mcp`);
|
||||
});
|
||||
+433
@@ -0,0 +1,433 @@
|
||||
import { execFile, spawn } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import { writeFile, rm, mkdtemp } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
function swiftSourceLLMPolish() {
|
||||
// From whisper-translation/apple_speech/Sources/AppleLLMPolish/main.swift
|
||||
// Added instant-reply mode + general conversation
|
||||
return `
|
||||
import Foundation
|
||||
import FoundationModels
|
||||
|
||||
struct InMsg: Decodable {
|
||||
var id: String?
|
||||
var mode: String // "line" | "paragraph" | "check" | "quick_reply" | "chat"
|
||||
var text: String?
|
||||
var prev1: String?
|
||||
var prev2: String?
|
||||
var context: String?
|
||||
var prevSource: String?
|
||||
var language: String?
|
||||
var instructions: String?
|
||||
var history: String? // JSON array [{"role":"user","text":".."},...]
|
||||
}
|
||||
struct OutMsg: Encodable {
|
||||
var id: String?
|
||||
var ok: Bool
|
||||
var text: String
|
||||
var error: String?
|
||||
var ms: Int?
|
||||
var mode: String?
|
||||
}
|
||||
func log(_ s: String) { fputs(s+"\\n", stderr) }
|
||||
|
||||
@main
|
||||
struct AppleLLMPolish {
|
||||
static func main() async {
|
||||
let args = CommandLine.arguments
|
||||
if args.contains("--help") || args.contains("-h") {
|
||||
fputs("Usage: apple-llm-polish [--check]\\nPipe JSONL in stdin, JSONL out\\nModes: line, paragraph, quick_reply, chat, check\\n", stderr); exit(0)
|
||||
}
|
||||
if args.contains("--check") { await runCheck(); return }
|
||||
await runPipe()
|
||||
}
|
||||
static func runCheck() async {
|
||||
let m = SystemLanguageModel.default
|
||||
var pingText = "unavailable"
|
||||
var ok = false
|
||||
if m.isAvailable {
|
||||
do {
|
||||
let session = LanguageModelSession(model: m, instructions: "You are concise.")
|
||||
let r = try await session.respond(to: "Say ok")
|
||||
pingText = r.content
|
||||
ok = true
|
||||
} catch { pingText = error.localizedDescription }
|
||||
}
|
||||
let out: [String: Any] = [
|
||||
"available": m.isAvailable,
|
||||
"availability": "\\(m.availability)",
|
||||
"ping": pingText,
|
||||
"ok": ok,
|
||||
"model": "SystemLanguageModel 3B ANE"
|
||||
]
|
||||
if let d = try? JSONSerialization.data(withJSONObject: out), let s = String(data: d, encoding: .utf8) { print(s) }
|
||||
}
|
||||
static func runPipe() async {
|
||||
let m = SystemLanguageModel.default
|
||||
guard m.isAvailable else {
|
||||
let reason = "\\(m.availability)"
|
||||
while let line = readLine() {
|
||||
if line.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { continue }
|
||||
var idv: String? = nil
|
||||
if let data = line.data(using: .utf8), let dict = try? JSONSerialization.jsonObject(with: data) as? [String:Any] { idv = dict["id"] as? String }
|
||||
let out = OutMsg(id: idv, ok: false, text: "", error: "model unavailable: \\(reason)", ms: nil, mode: "error")
|
||||
if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
|
||||
}
|
||||
return
|
||||
}
|
||||
// Keep sessions warm — separate useCases
|
||||
let lineSession = LanguageModelSession(model: SystemLanguageModel(useCase: .general, guardrails: .permissiveContentTransformations), instructions: lineSystemPrompt())
|
||||
let paraSession = LanguageModelSession(model: SystemLanguageModel(useCase: .general, guardrails: .permissiveContentTransformations), instructions: paraSystemPrompt())
|
||||
let quickSession = LanguageModelSession(model: SystemLanguageModel(useCase: .general, guardrails: .permissiveContentTransformations), instructions: quickReplySystemPrompt())
|
||||
let chatSession = LanguageModelSession(model: SystemLanguageModel(useCase: .general, guardrails: .permissiveContentTransformations), instructions: "You are Hermes, a concise helpful voice assistant for ESP32 devices. Keep replies under 40 words, warm and concrete, kid-safe.")
|
||||
|
||||
lineSession.prewarm()
|
||||
paraSession.prewarm()
|
||||
quickSession.prewarm()
|
||||
chatSession.prewarm()
|
||||
|
||||
log("[apple-llm] ready, ANE-backed 3B")
|
||||
|
||||
while let line = readLine() {
|
||||
let trimmed = line.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if trimmed.isEmpty { continue }
|
||||
guard let data = line.data(using: .utf8), let req = try? JSONDecoder().decode(InMsg.self, from: data) else {
|
||||
let out = OutMsg(id: nil, ok: false, text: "", error: "bad json", ms: nil, mode: "error")
|
||||
if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
|
||||
continue
|
||||
}
|
||||
if req.mode == "check" {
|
||||
let out = OutMsg(id: req.id, ok: m.isAvailable, text: "\\(m.availability)", error: nil, ms: 0, mode: "check")
|
||||
if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
|
||||
continue
|
||||
}
|
||||
let t0 = Date()
|
||||
do {
|
||||
let (session, prompt, temp): (LanguageModelSession, String, Double)
|
||||
switch req.mode {
|
||||
case "paragraph":
|
||||
session = paraSession
|
||||
prompt = buildParagraphPrompt(context: req.context ?? "", prevSource: req.prevSource ?? "", newText: req.text ?? "")
|
||||
temp = 0.2
|
||||
case "quick_reply":
|
||||
session = quickSession
|
||||
prompt = buildQuickReplyPrompt(draft: req.text ?? "", context: req.context, instructions: req.instructions)
|
||||
temp = 0.4
|
||||
case "chat":
|
||||
// For chat, rebuild prompt from history if provided, else use text directly
|
||||
session = chatSession
|
||||
if let hist = req.history, !hist.isEmpty {
|
||||
prompt = buildChatPrompt(historyJSON: hist, newText: req.text ?? "", instructions: req.instructions)
|
||||
} else {
|
||||
prompt = req.text ?? ""
|
||||
}
|
||||
temp = 0.5
|
||||
default: // line
|
||||
session = lineSession
|
||||
prompt = buildLinePrompt(text: req.text ?? "", prev1: req.prev1 ?? "", prev2: req.prev2 ?? "")
|
||||
temp = 0.1
|
||||
}
|
||||
var opts = GenerationOptions()
|
||||
opts.temperature = temp
|
||||
let resp = try await session.respond(to: prompt, options: opts)
|
||||
let ms = Int(Date().timeIntervalSince(t0)*1000)
|
||||
let cleaned = resp.content.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
let out = OutMsg(id: req.id, ok: true, text: cleaned.isEmpty ? (req.text ?? "") : cleaned, error: nil, ms: ms, mode: req.mode)
|
||||
if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
|
||||
} catch {
|
||||
let ms = Int(Date().timeIntervalSince(t0)*1000)
|
||||
let out = OutMsg(id: req.id, ok: false, text: req.text ?? "", error: error.localizedDescription, ms: ms, mode: req.mode)
|
||||
if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
|
||||
}
|
||||
}
|
||||
}
|
||||
static func lineSystemPrompt() -> String {
|
||||
return "You are a real-time caption polisher. Fix punctuation, casing, STT typos. Remove filler (uh, um). Keep meaning. Output one polished line only."
|
||||
}
|
||||
static func paraSystemPrompt() -> String {
|
||||
return "You are a careful live transcript editor. Goal: most faithful readable English. NEW SOURCE TEXT is primary. PREVIOUS CONTEXT only if helps continuity. English only. No meta commentary. Return revised transcript only."
|
||||
}
|
||||
static func quickReplySystemPrompt() -> String {
|
||||
return """
|
||||
You are Hermes instant-reply for ESP32 voice devices (iPhone, Watch, kids). You get LIVE draft transcript from user, possibly partial with typos.
|
||||
|
||||
Goal: produce a super-short, HIGHLY CONTEXTUAL reply preview (max 20 words) that shows you actually understood their specific request, not generic.
|
||||
|
||||
Rules:
|
||||
- Reference SPECIFIC keywords/entities from draft: names (Grace Priss/Rain), topics (Mac mini voice, weather, homework), intent.
|
||||
- Sound human, warm, playful for kids, concise.
|
||||
- If draft mentions Mac mini voice/boys voice/speech, acknowledge you'll use Mac mini voice.
|
||||
- If draft mentions a name, use it.
|
||||
- If draft asks something, hint at answer direction without fully answering (full answer comes next).
|
||||
- Never say "Thinking on full answer" verbatim — too generic. Instead vary: "Let me check...", "One sec, pulling that...", "Nice name! Love it..."
|
||||
- Under 20 words. Return ONLY reply text, no quotes.
|
||||
|
||||
Examples:
|
||||
Draft: "what's the weather today" -> "Checking weather now — one sec..."
|
||||
Draft: "Perfect. My first name is Grace Priss, and my other name is Grace Reign." -> "Wow, Grace Priss and Grace Reign — royal names! Love them!"
|
||||
Draft: "Why you're not answering with boys voice?" -> "Got it — you want boy voice, switching to Mac mini voice now..."
|
||||
Draft: "Can you use the Mac mini voice to generate answers?" -> "Yes! Using Mac mini voice for better audio, one sec..."
|
||||
Draft: "tell me a joke" -> "Joke coming up..."
|
||||
Draft: "Hey improvement I think now you should show quick response" -> "Nice! Quick response is live, working on full answer too..."
|
||||
"""
|
||||
}
|
||||
static func buildLinePrompt(text: String, prev1: String, prev2: String) -> String {
|
||||
var p = ""
|
||||
if !prev2.isEmpty { p += "Previous 2: \\(prev2)\\n" }
|
||||
if !prev1.isEmpty { p += "Previous 1: \\(prev1)\\n" }
|
||||
p += "Current: \\(text)\\nPolished:"
|
||||
return p
|
||||
}
|
||||
static func buildParagraphPrompt(context: String, prevSource: String, newText: String) -> String {
|
||||
return "Edit this transcript.\\n[PREVIOUS CONTEXT]\\n\\(context)\\n[PREVIOUS SOURCE]\\n\\(prevSource)\\n[NEW]\\n\\(newText)"
|
||||
}
|
||||
static func buildQuickReplyPrompt(draft: String, context: String?, instructions: String?) -> String {
|
||||
var p = "LIVE DRAFT from user speaking (may have typos, partial): \\\"\\(draft)\\\"\\n"
|
||||
if let c = context, !c.isEmpty { p += "Previous full transcript: \\(c)\\n" }
|
||||
if let i = instructions, !i.isEmpty { p += "Extra instructions: \\(i)\\n" }
|
||||
p += "\\nTask: produce contextual instant reply (max 20 words) that references SPECIFIC words from draft, not generic. If draft unclear, fall back to 'Heard you — working on full answer...'"
|
||||
return p
|
||||
}
|
||||
static func buildChatPrompt(historyJSON: String, newText: String, instructions: String?) -> String {
|
||||
// history is JSON array serialized
|
||||
return "Conversation history: \\(historyJSON)\\nUser says (draft/final): \\(newText)\\n\\(instructions != nil ? \"Instructions: \\(instructions!)\\n\" : \"\")Reply concisely for voice device (<40 words):"
|
||||
}
|
||||
}
|
||||
`;
|
||||
}
|
||||
|
||||
class AppleLLMSession {
|
||||
constructor() {
|
||||
this.proc = null;
|
||||
this.tmpDir = null;
|
||||
this.binFile = null;
|
||||
this.ready = false;
|
||||
this.reqId = 0;
|
||||
this.pending = new Map();
|
||||
this.lastActivity = Date.now();
|
||||
}
|
||||
|
||||
async ensureBuilt() {
|
||||
const tmpBase = path.join(os.tmpdir(), "apple-llm-");
|
||||
this.tmpDir = await mkdtemp(tmpBase);
|
||||
const swiftFile = path.join(this.tmpDir, "Main.swift");
|
||||
this.binFile = path.join(this.tmpDir, "apple-llm-polish");
|
||||
await writeFile(swiftFile, swiftSourceLLMPolish(), "utf8");
|
||||
try {
|
||||
await execFileAsync("/usr/bin/swiftc", ["-O", "-parse-as-library", swiftFile, "-o", this.binFile, "-framework", "Foundation", "-framework", "FoundationModels"], { timeout: 60000, maxBuffer: 20*1024*1024 });
|
||||
} catch (e) {
|
||||
throw new Error(`swiftc LLM build failed: ${e.stderr||e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async start() {
|
||||
if (this.proc && this.proc.exitCode === null && this.ready) return;
|
||||
if (!this.binFile) await this.ensureBuilt();
|
||||
return new Promise((resolve, reject) => {
|
||||
this.proc = spawn(this.binFile, [], { stdio: ["pipe", "pipe", "pipe"] });
|
||||
let stderrBuf = "";
|
||||
let stdoutBuf = "";
|
||||
this.proc.stderr.on("data", d => { stderrBuf += d.toString(); });
|
||||
this.proc.stdout.on("data", d => {
|
||||
const txt = d.toString();
|
||||
stdoutBuf += txt;
|
||||
// Parse lines for responses
|
||||
let lines = stdoutBuf.split("\n");
|
||||
stdoutBuf = lines.pop() || "";
|
||||
for (let line of lines) {
|
||||
line = line.trim();
|
||||
if (!line) continue;
|
||||
try {
|
||||
const obj = JSON.parse(line);
|
||||
const id = obj.id;
|
||||
if (id && this.pending.has(id)) {
|
||||
const {resolve} = this.pending.get(id);
|
||||
this.pending.delete(id);
|
||||
resolve(obj);
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
});
|
||||
|
||||
setTimeout(() => {
|
||||
if (this.proc.exitCode !== null) {
|
||||
reject(new Error(`apple-llm exited early code=${this.proc.exitCode} stderr=${stderrBuf.slice(0,2000)}`));
|
||||
} else {
|
||||
this.ready = true;
|
||||
// Capture remaining stdout buffering setup
|
||||
this._stdoutLeftover = "";
|
||||
resolve();
|
||||
}
|
||||
}, 1500);
|
||||
this.proc.on("error", reject);
|
||||
});
|
||||
}
|
||||
|
||||
// Ensure reader continues after initial start
|
||||
_ensureReader() {
|
||||
if (this._readerSetup) return;
|
||||
this._readerSetup = true;
|
||||
// Already setup in start() via stdout.on data - but need to handle leftover buffering for late responses
|
||||
let buf = "";
|
||||
if (this.proc) {
|
||||
// Additional listener for any missed
|
||||
this.proc.stdout.on("data", chunk => {
|
||||
buf += chunk.toString();
|
||||
let lines = buf.split("\n");
|
||||
buf = lines.pop() || "";
|
||||
for (let line of lines) {
|
||||
line = line.trim();
|
||||
if (!line) continue;
|
||||
try {
|
||||
const obj = JSON.parse(line);
|
||||
const id = obj.id;
|
||||
if (id && this.pending.has(id)) {
|
||||
const {resolve} = this.pending.get(id);
|
||||
this.pending.delete(id);
|
||||
resolve(obj);
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async call(args, timeoutMs = 10000) {
|
||||
await this.start();
|
||||
this._ensureReader();
|
||||
const id = String(this.reqId++);
|
||||
const payload = { id, ...args };
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
let timer = setTimeout(() => {
|
||||
if (this.pending.has(id)) {
|
||||
this.pending.delete(id);
|
||||
resolve({ ok: false, text: args.text || "", error: `timeout ${timeoutMs}ms`, id, ms: timeoutMs });
|
||||
}
|
||||
}, timeoutMs);
|
||||
|
||||
this.pending.set(id, {
|
||||
resolve: (obj) => {
|
||||
clearTimeout(timer);
|
||||
resolve(obj);
|
||||
}
|
||||
});
|
||||
|
||||
try {
|
||||
this.proc.stdin.write(JSON.stringify(payload) + "\n");
|
||||
} catch (e) {
|
||||
clearTimeout(timer);
|
||||
this.pending.delete(id);
|
||||
reject(e);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async close() {
|
||||
try {
|
||||
if (this.proc) {
|
||||
try { this.proc.stdin.end(); } catch {}
|
||||
await new Promise(r => { this.proc.on("close", r); setTimeout(r, 1500); });
|
||||
try { this.proc.kill(); } catch {}
|
||||
}
|
||||
} catch {}
|
||||
try { if (this.tmpDir) await rm(this.tmpDir, {recursive:true, force:true}); } catch {}
|
||||
this.proc = null;
|
||||
this.ready = false;
|
||||
}
|
||||
}
|
||||
|
||||
// Singleton global session for all LLM calls (fast, keeps KV cache warm)
|
||||
let globalSession = null;
|
||||
|
||||
async function getSession() {
|
||||
if (!globalSession) {
|
||||
globalSession = new AppleLLMSession();
|
||||
await globalSession.start();
|
||||
}
|
||||
globalSession.lastActivity = Date.now();
|
||||
return globalSession;
|
||||
}
|
||||
|
||||
export async function appleLLMCheck() {
|
||||
const s = await getSession();
|
||||
const res = await s.call({ mode: "check", text: "check" }, 10000);
|
||||
return {
|
||||
ok: res.ok,
|
||||
available: res.ok,
|
||||
text: res.text || res.error || "",
|
||||
ms: res.ms,
|
||||
engine: "Apple FoundationModels 3B ANE"
|
||||
};
|
||||
}
|
||||
|
||||
export async function appleLLMPolish({ text, prev1, prev2, mode } = {}) {
|
||||
if (!text) throw new Error("text required");
|
||||
const s = await getSession();
|
||||
const m = mode || "line";
|
||||
const res = await s.call({ mode: m, text, prev1: prev1||"", prev2: prev2||"" }, 8000);
|
||||
return {
|
||||
ok: res.ok,
|
||||
text: res.text || text,
|
||||
original: text,
|
||||
ms: res.ms,
|
||||
mode: m,
|
||||
error: res.error || undefined
|
||||
};
|
||||
}
|
||||
|
||||
export async function appleLLMQuickReply({ draft, context, instructions } = {}) {
|
||||
if (!draft) throw new Error("draft required");
|
||||
const s = await getSession();
|
||||
const res = await s.call({ mode: "quick_reply", text: draft, context: context||"", instructions: instructions||"" }, 3000);
|
||||
return {
|
||||
ok: res.ok,
|
||||
text: res.text || "Got it, working on it...",
|
||||
draft,
|
||||
ms: res.ms,
|
||||
engine: "Apple FoundationModels 3B instant"
|
||||
};
|
||||
}
|
||||
|
||||
export async function appleLLMChat({ text, history, instructions } = {}) {
|
||||
if (!text) throw new Error("text required");
|
||||
const s = await getSession();
|
||||
const histStr = history ? JSON.stringify(history) : "";
|
||||
const res = await s.call({ mode: "chat", text, history: histStr, instructions: instructions||"" }, 5000);
|
||||
return {
|
||||
ok: res.ok,
|
||||
text: res.text || "",
|
||||
input: text,
|
||||
ms: res.ms,
|
||||
engine: "Apple FoundationModels 3B voice"
|
||||
};
|
||||
}
|
||||
|
||||
export async function appleLLMClose() {
|
||||
if (globalSession) {
|
||||
await globalSession.close();
|
||||
globalSession = null;
|
||||
}
|
||||
return { ok: true, closed: true };
|
||||
}
|
||||
|
||||
export async function appleLLMStatus() {
|
||||
return {
|
||||
active: !!globalSession,
|
||||
ready: globalSession?.ready || false,
|
||||
lastActivity: globalSession ? new Date(globalSession.lastActivity).toISOString() : null,
|
||||
pid: globalSession?.proc?.pid || null
|
||||
};
|
||||
}
|
||||
|
||||
// Idle cleanup 2 min
|
||||
setInterval(async () => {
|
||||
if (globalSession && Date.now() - globalSession.lastActivity > 120000) {
|
||||
try { await globalSession.close(); } catch {}
|
||||
globalSession = null;
|
||||
}
|
||||
}, 20000);
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
import { dateFromInput, runJxa } from "../apple-events.js";
|
||||
|
||||
export const FOCUS_CALENDAR = Object.freeze({
|
||||
calendarIndex: 2,
|
||||
calendar: "Home",
|
||||
});
|
||||
|
||||
const CALENDAR_LIST_SCRIPT = String.raw`
|
||||
function run() {
|
||||
const app = Application("/System/Applications/Calendar.app");
|
||||
return JSON.stringify(app.calendars().map(function (calendar, index) {
|
||||
return {
|
||||
index: index,
|
||||
name: String(calendar.name()),
|
||||
writable: Boolean(calendar.writable())
|
||||
};
|
||||
}));
|
||||
}`;
|
||||
|
||||
const EVENTS_LIST_SCRIPT = String.raw`
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Calendar.app");
|
||||
const start = new Date(input.start);
|
||||
const end = new Date(input.end);
|
||||
const calendars = app.calendars();
|
||||
const found = [];
|
||||
|
||||
for (let c = 0; c < calendars.length; c++) {
|
||||
const calendar = calendars[c];
|
||||
const calendarName = String(calendar.name());
|
||||
if (input.calendarIndex !== null && c !== input.calendarIndex) continue;
|
||||
if (input.calendar && calendarName !== input.calendar) continue;
|
||||
// Calendar's JXA bridge treats multi-property date tests inconsistently.
|
||||
// Bound one indexed property here, then enforce overlap below.
|
||||
const events = calendar.events.whose({
|
||||
startDate: {_greaterThanEquals: start, _lessThanEquals: end}
|
||||
})();
|
||||
for (let e = 0; e < events.length; e++) {
|
||||
const event = events[e];
|
||||
const eventStart = event.startDate();
|
||||
const eventEnd = event.endDate();
|
||||
if (eventEnd < start || eventStart > end) continue;
|
||||
found.push({
|
||||
id: String(event.uid()),
|
||||
calendarIndex: c,
|
||||
calendar: calendarName,
|
||||
title: String(event.summary()),
|
||||
start: eventStart.toISOString(),
|
||||
end: eventEnd.toISOString(),
|
||||
allDay: Boolean(event.alldayEvent()),
|
||||
location: String(event.location() || "")
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
found.sort(function (a, b) { return a.start.localeCompare(b.start); });
|
||||
return JSON.stringify(found.slice(0, input.limit));
|
||||
}`;
|
||||
|
||||
const EVENT_CREATE_SCRIPT = String.raw`
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Calendar.app");
|
||||
const calendars = app.calendars();
|
||||
let destination = null;
|
||||
let destinationIndex = null;
|
||||
for (let c = 0; c < calendars.length; c++) {
|
||||
const indexMatches = input.calendarIndex !== null && c === input.calendarIndex &&
|
||||
(!input.calendar || String(calendars[c].name()) === input.calendar);
|
||||
const nameMatches = input.calendarIndex === null && String(calendars[c].name()) === input.calendar;
|
||||
if (indexMatches || nameMatches) {
|
||||
destination = calendars[c];
|
||||
destinationIndex = c;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!destination) throw new Error("Calendar not found");
|
||||
if (!destination.writable()) throw new Error("Calendar is read-only");
|
||||
const event = app.Event({
|
||||
summary: input.title,
|
||||
startDate: new Date(input.start),
|
||||
endDate: new Date(input.end),
|
||||
alldayEvent: input.allDay,
|
||||
description: input.notes || "",
|
||||
location: input.location || ""
|
||||
});
|
||||
destination.events.push(event);
|
||||
return JSON.stringify({
|
||||
id: String(event.uid()),
|
||||
calendarIndex: destinationIndex,
|
||||
calendar: String(destination.name()),
|
||||
title: String(event.summary()),
|
||||
start: event.startDate().toISOString(),
|
||||
end: event.endDate().toISOString()
|
||||
});
|
||||
}`;
|
||||
|
||||
export function listCalendars() {
|
||||
return runJxa(CALENDAR_LIST_SCRIPT);
|
||||
}
|
||||
|
||||
export function listEvents({ start, end, calendar, calendarIndex, limit }) {
|
||||
dateFromInput(start, "start");
|
||||
dateFromInput(end, "end");
|
||||
if (new Date(start) > new Date(end)) {
|
||||
throw new Error("start must occur before end.");
|
||||
}
|
||||
return runJxa(EVENTS_LIST_SCRIPT, {
|
||||
start,
|
||||
end,
|
||||
calendar,
|
||||
calendarIndex: calendarIndex ?? null,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
|
||||
export function createEvent(input) {
|
||||
const start = dateFromInput(input.start, "start");
|
||||
const end = dateFromInput(input.end, "end");
|
||||
if (start >= end) {
|
||||
throw new Error("start must occur before end.");
|
||||
}
|
||||
return runJxa(EVENT_CREATE_SCRIPT, input);
|
||||
}
|
||||
+175
@@ -0,0 +1,175 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { mkdir, stat } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
const DEFAULT_OUTPUT_DIR = new URL("../../generated-images", import.meta.url).pathname;
|
||||
const DEFAULT_CODEX_PATH = "codex";
|
||||
const DEFAULT_TIMEOUT_MS = 10 * 60 * 1000;
|
||||
|
||||
function getOutputDir() {
|
||||
return process.env.CODEX_IMAGE_OUTPUT_DIR || DEFAULT_OUTPUT_DIR;
|
||||
}
|
||||
|
||||
function getCodexPath() {
|
||||
return process.env.CODEX_CLI_PATH || DEFAULT_CODEX_PATH;
|
||||
}
|
||||
|
||||
function safeFilename(filename) {
|
||||
const fallback = `codex-${new Date().toISOString().replaceAll(/[:.]/g, "-")}.png`;
|
||||
const base = path.basename(filename || fallback).replaceAll(/[^a-zA-Z0-9._-]/g, "-");
|
||||
const trimmed = base.replaceAll(/-+/g, "-").replaceAll(/^\.+/g, "");
|
||||
return trimmed || fallback;
|
||||
}
|
||||
|
||||
function parseJsonFromOutput(output) {
|
||||
const trimmed = output.trim();
|
||||
if (!trimmed) {
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
return JSON.parse(trimmed);
|
||||
} catch {
|
||||
const match = trimmed.match(/\{[\s\S]*\}$/);
|
||||
if (!match) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return JSON.parse(match[0]);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function runCodex(codexPath, args, timeoutMs) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(codexPath, args, {
|
||||
env: process.env,
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
});
|
||||
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
let timedOut = false;
|
||||
const timeout = setTimeout(() => {
|
||||
timedOut = true;
|
||||
child.kill("SIGTERM");
|
||||
}, timeoutMs);
|
||||
|
||||
child.stdout.setEncoding("utf8");
|
||||
child.stderr.setEncoding("utf8");
|
||||
child.stdout.on("data", (chunk) => {
|
||||
stdout += chunk;
|
||||
});
|
||||
child.stderr.on("data", (chunk) => {
|
||||
stderr += chunk;
|
||||
});
|
||||
child.on("error", (error) => {
|
||||
clearTimeout(timeout);
|
||||
reject(error);
|
||||
});
|
||||
child.on("close", (code, signal) => {
|
||||
clearTimeout(timeout);
|
||||
if (timedOut) {
|
||||
reject(new Error(`timed out after ${timeoutMs}ms. ${stderr.trim()}`));
|
||||
return;
|
||||
}
|
||||
if (code !== 0) {
|
||||
reject(new Error(stderr.trim() || stdout.trim() || `exited with ${signal || code}`));
|
||||
return;
|
||||
}
|
||||
resolve({ stdout, stderr });
|
||||
});
|
||||
|
||||
child.stdin.end();
|
||||
});
|
||||
}
|
||||
|
||||
export function getCodexImageConfigStatus() {
|
||||
return {
|
||||
codexCliPath: getCodexPath(),
|
||||
outputDir: getOutputDir(),
|
||||
model: process.env.CODEX_IMAGE_MODEL || "(Codex CLI default)",
|
||||
timeoutMs: Number.parseInt(process.env.CODEX_IMAGE_TIMEOUT_MS || String(DEFAULT_TIMEOUT_MS), 10),
|
||||
};
|
||||
}
|
||||
|
||||
export async function generateCodexImage({
|
||||
prompt,
|
||||
filename,
|
||||
size,
|
||||
quality,
|
||||
style,
|
||||
referenceImage,
|
||||
}) {
|
||||
const outputDir = getOutputDir();
|
||||
await mkdir(outputDir, { recursive: true });
|
||||
|
||||
const outputFilename = safeFilename(filename);
|
||||
const outputPath = path.join(outputDir, outputFilename);
|
||||
const codexPath = getCodexPath();
|
||||
const timeoutMs = Number.parseInt(process.env.CODEX_IMAGE_TIMEOUT_MS || String(DEFAULT_TIMEOUT_MS), 10);
|
||||
|
||||
const details = [
|
||||
size ? `Requested size/aspect: ${size}` : null,
|
||||
quality ? `Requested quality: ${quality}` : null,
|
||||
style ? `Requested style: ${style}` : null,
|
||||
].filter(Boolean).join("\n");
|
||||
|
||||
const workerPrompt = [
|
||||
"Use $imagegen to generate exactly one raster image.",
|
||||
`Save the final image file at this exact absolute path: ${outputPath}`,
|
||||
"Do not modify any other files.",
|
||||
"After saving the file, respond only with JSON matching this shape:",
|
||||
`{"ok":true,"path":"${outputPath.replaceAll("\\", "\\\\")}","note":"short description"}`,
|
||||
details ? `Generation details:\n${details}` : null,
|
||||
`Image prompt:\n${prompt}`,
|
||||
].filter(Boolean).join("\n\n");
|
||||
|
||||
const args = [
|
||||
"exec",
|
||||
"--ephemeral",
|
||||
"--sandbox",
|
||||
"workspace-write",
|
||||
"--enable",
|
||||
"image_generation",
|
||||
"-C",
|
||||
new URL("../..", import.meta.url).pathname,
|
||||
];
|
||||
|
||||
if (process.env.CODEX_IMAGE_MODEL) {
|
||||
args.push("--model", process.env.CODEX_IMAGE_MODEL);
|
||||
}
|
||||
if (referenceImage) {
|
||||
args.push("--image", referenceImage);
|
||||
}
|
||||
|
||||
args.push(workerPrompt);
|
||||
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
try {
|
||||
const result = await runCodex(codexPath, args, timeoutMs);
|
||||
stdout = result.stdout;
|
||||
stderr = result.stderr;
|
||||
} catch (error) {
|
||||
throw new Error(`Codex image generation failed. ${error.message}`);
|
||||
}
|
||||
|
||||
try {
|
||||
const file = await stat(outputPath);
|
||||
const parsed = parseJsonFromOutput(stdout);
|
||||
return {
|
||||
ok: true,
|
||||
path: outputPath,
|
||||
filename: outputFilename,
|
||||
bytes: file.size,
|
||||
codexCliPath: codexPath,
|
||||
note: parsed?.note || null,
|
||||
stderr: stderr.trim() || null,
|
||||
};
|
||||
} catch {
|
||||
throw new Error(`Codex completed but did not create ${outputPath}. Output: ${stdout.trim() || "(empty)"}`);
|
||||
}
|
||||
}
|
||||
+102
@@ -0,0 +1,102 @@
|
||||
import { runJxa } from "../apple-events.js";
|
||||
|
||||
const CONTACTS_SEARCH_SCRIPT = String.raw`
|
||||
function text(value) {
|
||||
try { return value ? String(value) : ""; } catch (_) { return ""; }
|
||||
}
|
||||
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Contacts.app");
|
||||
const query = (input.query || "").toLowerCase();
|
||||
const people = app.people();
|
||||
const found = [];
|
||||
|
||||
for (let i = 0; i < people.length && found.length < input.limit; i++) {
|
||||
const person = people[i];
|
||||
const name = text(person.name());
|
||||
const organization = text(person.organization());
|
||||
if (query && (name + "\n" + organization).toLowerCase().indexOf(query) === -1) continue;
|
||||
found.push({
|
||||
id: String(person.id()),
|
||||
name: name,
|
||||
organization: organization,
|
||||
modifiedAt: person.modificationDate().toISOString()
|
||||
});
|
||||
}
|
||||
|
||||
return JSON.stringify(found);
|
||||
}`;
|
||||
|
||||
const CONTACTS_READ_SCRIPT = String.raw`
|
||||
function text(value) {
|
||||
try { return value ? String(value) : ""; } catch (_) { return ""; }
|
||||
}
|
||||
|
||||
function items(values) {
|
||||
return values.map(function (value) {
|
||||
return { label: text(value.label()), value: text(value.value()) };
|
||||
});
|
||||
}
|
||||
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Contacts.app");
|
||||
const people = app.people();
|
||||
for (let i = 0; i < people.length; i++) {
|
||||
const person = people[i];
|
||||
if (String(person.id()) !== input.id) continue;
|
||||
return JSON.stringify({
|
||||
id: String(person.id()),
|
||||
name: text(person.name()),
|
||||
firstName: text(person.firstName()),
|
||||
lastName: text(person.lastName()),
|
||||
organization: text(person.organization()),
|
||||
jobTitle: text(person.jobTitle()),
|
||||
emails: items(person.emails()),
|
||||
phones: items(person.phones()),
|
||||
modifiedAt: person.modificationDate().toISOString()
|
||||
});
|
||||
}
|
||||
throw new Error("Contact not found");
|
||||
}`;
|
||||
|
||||
const CONTACTS_CREATE_SCRIPT = String.raw`
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Contacts.app");
|
||||
const person = app.Person({
|
||||
firstName: input.firstName || "",
|
||||
lastName: input.lastName || "",
|
||||
organization: input.organization || "",
|
||||
jobTitle: input.jobTitle || "",
|
||||
note: input.note || ""
|
||||
});
|
||||
|
||||
app.people.push(person);
|
||||
if (input.email) {
|
||||
person.emails.push(app.Email({label: input.email.label, value: input.email.value}));
|
||||
}
|
||||
if (input.phone) {
|
||||
person.phones.push(app.Phone({label: input.phone.label, value: input.phone.value}));
|
||||
}
|
||||
app.save();
|
||||
|
||||
return JSON.stringify({
|
||||
id: String(person.id()),
|
||||
name: String(person.name()),
|
||||
organization: input.organization || ""
|
||||
});
|
||||
}`;
|
||||
|
||||
export function searchContacts(input) {
|
||||
return runJxa(CONTACTS_SEARCH_SCRIPT, input);
|
||||
}
|
||||
|
||||
export function readContact(id) {
|
||||
return runJxa(CONTACTS_READ_SCRIPT, { id });
|
||||
}
|
||||
|
||||
export function createContact(input) {
|
||||
return runJxa(CONTACTS_CREATE_SCRIPT, input);
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { loadEnvFile } from "node:process";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
try {
|
||||
loadEnvFile();
|
||||
} catch (error) {
|
||||
if (error.code !== "ENOENT") {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
const PYTHON = new URL("../../.venv/bin/python", import.meta.url).pathname;
|
||||
const BRIDGE = new URL("../../scripts/deco_bridge.py", import.meta.url).pathname;
|
||||
const HA_CLIENTS_BRIDGE = new URL("../../scripts/deco_ha_bridge.py", import.meta.url).pathname;
|
||||
|
||||
export async function getDecoStats(action) {
|
||||
try {
|
||||
const args = action === "clients" ? [HA_CLIENTS_BRIDGE] : [BRIDGE, action];
|
||||
const { stdout } = await execFileAsync(PYTHON, args, {
|
||||
timeout: 90_000,
|
||||
maxBuffer: 4 * 1024 * 1024,
|
||||
env: process.env,
|
||||
});
|
||||
return JSON.parse(stdout);
|
||||
} catch (error) {
|
||||
const detail = error.stderr?.trim() || error.message;
|
||||
throw new Error(`Deco stats request failed. ${detail}`);
|
||||
}
|
||||
}
|
||||
|
||||
export function getDecoConfigStatus() {
|
||||
return {
|
||||
host: process.env.DECO_HOST || "(default gateway)",
|
||||
username: process.env.DECO_USERNAME || "admin",
|
||||
passwordConfigured: Boolean(process.env.DECO_PASSWORD),
|
||||
passwordLength: process.env.DECO_PASSWORD?.length || 0,
|
||||
verifySsl: process.env.DECO_VERIFY_SSL ?? "true",
|
||||
timeout: process.env.DECO_TIMEOUT || "10",
|
||||
};
|
||||
}
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
import path from "node:path";
|
||||
|
||||
const DEFAULT_OUTPUT_DIR = new URL("../../generated-images", import.meta.url).pathname;
|
||||
const DEFAULT_PROFILE_NAME = "ReynaFamilyBot";
|
||||
|
||||
function getOutputDir() {
|
||||
return process.env.GEMINI_CHROME_IMAGE_OUTPUT_DIR || process.env.CODEX_IMAGE_OUTPUT_DIR || DEFAULT_OUTPUT_DIR;
|
||||
}
|
||||
|
||||
function getProfileName() {
|
||||
return process.env.GEMINI_CHROME_PROFILE_NAME || DEFAULT_PROFILE_NAME;
|
||||
}
|
||||
|
||||
function safeFilename(filename) {
|
||||
const fallback = `gemini-chrome-${new Date().toISOString().replaceAll(/[:.]/g, "-")}.png`;
|
||||
const base = path.basename(filename || fallback).replaceAll(/[^a-zA-Z0-9._-]/g, "-");
|
||||
const trimmed = base.replaceAll(/-+/g, "-").replaceAll(/^\.+/g, "");
|
||||
const name = trimmed || fallback;
|
||||
return name.endsWith(".png") ? name : `${name}.png`;
|
||||
}
|
||||
|
||||
function codexPrompt({ prompt, outputPath, profileName }) {
|
||||
return [
|
||||
"Use the Chrome skill, not Playwright and not MacMiniMCP browser tools.",
|
||||
"",
|
||||
"Goal: generate an image in Gemini using my Chrome profile named `" + profileName + "`, then save the downloaded image locally.",
|
||||
"",
|
||||
"Steps:",
|
||||
"1. Connect to Chrome through the Codex Chrome Extension.",
|
||||
"2. Verify the selected Chrome browser metadata has `profileName: \"" + profileName + "\"`. If not, stop and tell me.",
|
||||
"3. Open or create a Gemini tab at https://gemini.google.com/app.",
|
||||
"4. If Gemini shows the first-run notice, click `Got it`.",
|
||||
"5. Submit this image prompt:",
|
||||
"",
|
||||
prompt,
|
||||
"",
|
||||
"6. Wait until Gemini finishes and shows `Download full size image`.",
|
||||
"7. Click `Download full size image`.",
|
||||
"8. Find the newest `Gemini_Generated_Image_*.png` in `/Users/adolforeyna/Downloads`.",
|
||||
"9. Copy it to:",
|
||||
" `" + outputPath + "`",
|
||||
"10. Show me the saved path and render the image in the response.",
|
||||
"",
|
||||
"Do not expose browser control through MCP. Do not use arbitrary browsing. Only use Chrome for this Gemini image-generation task.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
export function getGeminiChromePromptConfigStatus() {
|
||||
return {
|
||||
outputDir: getOutputDir(),
|
||||
profileName: getProfileName(),
|
||||
note: "This MCP tool builds a Codex prompt. It does not control Chrome itself because the Chrome skill is only available inside an active Codex session.",
|
||||
};
|
||||
}
|
||||
|
||||
export function buildGeminiChromePrompt({ prompt, filename } = {}) {
|
||||
const outputFilename = safeFilename(filename);
|
||||
const outputPath = path.join(getOutputDir(), outputFilename);
|
||||
const profileName = getProfileName();
|
||||
|
||||
return {
|
||||
prompt: codexPrompt({ prompt, outputPath, profileName }),
|
||||
outputPath,
|
||||
filename: outputFilename,
|
||||
profileName,
|
||||
};
|
||||
}
|
||||
+99
@@ -0,0 +1,99 @@
|
||||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const projectRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../..");
|
||||
const defaultOutputDir = path.join(projectRoot, "generated-images");
|
||||
const defaultModel = "gemini-3.1-flash-image";
|
||||
const interactionsUrl = "https://generativelanguage.googleapis.com/v1beta/interactions";
|
||||
|
||||
function outputDir() {
|
||||
return process.env.GEMINI_IMAGE_OUTPUT_DIR || defaultOutputDir;
|
||||
}
|
||||
|
||||
function apiKey() {
|
||||
return process.env.GEMINI_API_KEY || "";
|
||||
}
|
||||
|
||||
function safeFilename(name) {
|
||||
const fallback = `gemini-${new Date().toISOString().replace(/[:.]/g, "-")}.png`;
|
||||
const base = path.basename(name || fallback).replace(/[^a-zA-Z0-9._-]/g, "-");
|
||||
if (!base) {
|
||||
return fallback;
|
||||
}
|
||||
return base.endsWith(".png") ? base : `${base}.png`;
|
||||
}
|
||||
|
||||
function buildResponseFormat({ aspectRatio, imageSize }) {
|
||||
if (!aspectRatio && !imageSize) {
|
||||
return undefined;
|
||||
}
|
||||
return {
|
||||
type: "image",
|
||||
mime_type: "image/png",
|
||||
...(aspectRatio ? { aspect_ratio: aspectRatio } : {}),
|
||||
...(imageSize ? { image_size: imageSize } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
export function getGeminiImageConfigStatus() {
|
||||
return {
|
||||
configured: Boolean(apiKey()),
|
||||
outputDir: outputDir(),
|
||||
defaultModel,
|
||||
};
|
||||
}
|
||||
|
||||
export async function generateGeminiImage({
|
||||
prompt,
|
||||
filename,
|
||||
model = defaultModel,
|
||||
aspectRatio,
|
||||
imageSize,
|
||||
useGoogleSearch = false,
|
||||
} = {}) {
|
||||
const key = apiKey();
|
||||
if (!key) {
|
||||
throw new Error("GEMINI_API_KEY is required for gemini_image_generate.");
|
||||
}
|
||||
|
||||
const responseFormat = buildResponseFormat({ aspectRatio, imageSize });
|
||||
const body = {
|
||||
model,
|
||||
input: [{ type: "text", text: prompt }],
|
||||
...(responseFormat ? { response_format: responseFormat } : {}),
|
||||
...(useGoogleSearch ? { tools: [{ type: "google_search" }] } : {}),
|
||||
};
|
||||
|
||||
const response = await fetch(interactionsUrl, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"content-type": "application/json",
|
||||
"x-goog-api-key": key,
|
||||
},
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
|
||||
const data = await response.json().catch(() => null);
|
||||
if (!response.ok) {
|
||||
const message = data?.error?.message || response.statusText || "Gemini image generation failed.";
|
||||
throw new Error(`Gemini API error ${response.status}: ${message}`);
|
||||
}
|
||||
|
||||
const image = data?.output_image;
|
||||
if (!image?.data) {
|
||||
throw new Error("Gemini API did not return output_image.data.");
|
||||
}
|
||||
|
||||
const destinationDir = outputDir();
|
||||
await fs.mkdir(destinationDir, { recursive: true });
|
||||
const destination = path.join(destinationDir, safeFilename(filename));
|
||||
await fs.writeFile(destination, Buffer.from(image.data, "base64"));
|
||||
|
||||
return {
|
||||
path: destination,
|
||||
model,
|
||||
mimeType: image.mime_type || "image/png",
|
||||
interactionId: data.id,
|
||||
};
|
||||
}
|
||||
+99
@@ -0,0 +1,99 @@
|
||||
import { readFile, stat } from "node:fs/promises";
|
||||
|
||||
const DEFAULT_KSAY_URL = process.env.KSAY_URL || "http://127.0.0.1:7332";
|
||||
|
||||
function cleanBaseUrl(url) {
|
||||
return String(url || DEFAULT_KSAY_URL).replace(/\/+$/, "");
|
||||
}
|
||||
|
||||
async function requestJson(path, { method = "GET", body } = {}) {
|
||||
const url = `${cleanBaseUrl()}${path}`;
|
||||
let response;
|
||||
try {
|
||||
response = await fetch(url, {
|
||||
method,
|
||||
headers: body ? { "content-type": "application/json" } : undefined,
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
});
|
||||
} catch (error) {
|
||||
throw new Error(`Kokoro ksay daemon is not reachable at ${cleanBaseUrl()}: ${error.message}`);
|
||||
}
|
||||
|
||||
const text = await response.text();
|
||||
let value;
|
||||
try {
|
||||
value = text ? JSON.parse(text) : {};
|
||||
} catch {
|
||||
throw new Error(`Kokoro ksay daemon returned non-JSON response: ${text.slice(0, 500)}`);
|
||||
}
|
||||
|
||||
if (!response.ok || value.ok === false) {
|
||||
throw new Error(value.error || `Kokoro ksay daemon returned HTTP ${response.status}`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function normalizeSpeed(speed) {
|
||||
if (speed === undefined || speed === null) return 1.0;
|
||||
const n = Number(speed);
|
||||
if (!Number.isFinite(n) || n < 0.5 || n > 2.0) {
|
||||
throw new Error("speed must be a number between 0.5 and 2.0.");
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
function normalizeVoice(voice) {
|
||||
return String(voice || process.env.KSAY_VOICE || "af_heart").trim().slice(0, 100);
|
||||
}
|
||||
|
||||
function normalizeLangCode(langCode) {
|
||||
return String(langCode || process.env.KSAY_LANG_CODE || "a").trim().slice(0, 8);
|
||||
}
|
||||
|
||||
export async function speechKokoroStatus() {
|
||||
return requestJson("/health");
|
||||
}
|
||||
|
||||
export async function speechKokoroSynthesize({ text, voice, speed, langCode, outputPath }) {
|
||||
if (!text || !String(text).trim()) throw new Error("text required");
|
||||
const cleanText = String(text).slice(0, 8000);
|
||||
const result = await requestJson("/say", {
|
||||
method: "POST",
|
||||
body: {
|
||||
text: cleanText,
|
||||
voice: normalizeVoice(voice),
|
||||
speed: normalizeSpeed(speed),
|
||||
langCode: normalizeLangCode(langCode),
|
||||
output: outputPath || undefined,
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
...result,
|
||||
text: cleanText,
|
||||
format: "wav 24kHz mono",
|
||||
fileSize: await stat(result.filePath).then((s) => s.size).catch(() => 0),
|
||||
note: "Uses the warm ksay Kokoro daemon. Use speech_kokoro_synthesize_base64 when the caller needs audio bytes.",
|
||||
};
|
||||
}
|
||||
|
||||
export async function speechKokoroSynthesizeBase64({ text, voice, speed, langCode }) {
|
||||
const result = await speechKokoroSynthesize({ text, voice, speed, langCode });
|
||||
const buf = await readFile(result.filePath);
|
||||
const wavBase64 = buf.toString("base64");
|
||||
return {
|
||||
ok: true,
|
||||
text: result.text,
|
||||
voice: result.voice,
|
||||
speed: result.speed,
|
||||
langCode: result.langCode,
|
||||
model: result.model,
|
||||
filePath: result.filePath,
|
||||
wavBase64,
|
||||
size: buf.length,
|
||||
base64Length: wavBase64.length,
|
||||
sampleRate: result.sampleRate,
|
||||
seconds: result.seconds,
|
||||
format: "wav 24kHz mono",
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
import { runJxa } from "../apple-events.js";
|
||||
|
||||
const MAX_MESSAGES = 50;
|
||||
|
||||
const ACCOUNTS_SCRIPT = String.raw`
|
||||
function stringList(value) {
|
||||
if (!value) return [];
|
||||
if (Array.isArray(value)) return value.map(String);
|
||||
return [String(value)];
|
||||
}
|
||||
|
||||
function run() {
|
||||
const app = Application("/System/Applications/Mail.app");
|
||||
return JSON.stringify(app.accounts().map(function (account) {
|
||||
let addresses = [];
|
||||
try { addresses = stringList(account.emailAddresses()); } catch (_) {}
|
||||
return { id: String(account.id()), name: String(account.name()), emailAddresses: addresses };
|
||||
}));
|
||||
}`;
|
||||
|
||||
const MAILBOXES_SCRIPT = String.raw`
|
||||
function addMailbox(found, role, mailbox) {
|
||||
try { found.push({ role: role, name: String(mailbox.name()) }); } catch (_) {}
|
||||
}
|
||||
|
||||
function run() {
|
||||
const app = Application("/System/Applications/Mail.app");
|
||||
const found = [];
|
||||
addMailbox(found, "inbox", app.inbox());
|
||||
addMailbox(found, "sent", app.sentMailbox());
|
||||
addMailbox(found, "drafts", app.draftsMailbox());
|
||||
addMailbox(found, "junk", app.junkMailbox());
|
||||
addMailbox(found, "trash", app.trashMailbox());
|
||||
return JSON.stringify(found);
|
||||
}`;
|
||||
|
||||
const LIST_MESSAGES_SCRIPT = String.raw`
|
||||
function isoDate(value) {
|
||||
try { return value ? value.toISOString() : null; } catch (_) { return null; }
|
||||
}
|
||||
|
||||
function globalMailbox(app, requestedName) {
|
||||
const candidates = [app.inbox(), app.sentMailbox(), app.draftsMailbox(), app.junkMailbox(), app.trashMailbox()];
|
||||
for (let i = 0; i < candidates.length; i++) {
|
||||
try { if (String(candidates[i].name()) === requestedName) return candidates[i]; } catch (_) {}
|
||||
}
|
||||
throw new Error("Mailbox not found; use mail_list_mailboxes first");
|
||||
}
|
||||
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Mail.app");
|
||||
const mailbox = globalMailbox(app, input.mailbox);
|
||||
const messages = mailbox.messages();
|
||||
const found = [];
|
||||
for (let i = 0; i < messages.length && found.length < input.limit; i++) {
|
||||
const message = messages[i];
|
||||
let account = null;
|
||||
try { account = message.mailbox().account(); } catch (_) { continue; }
|
||||
if (String(account.id()) !== input.accountId) continue;
|
||||
const read = Boolean(message.readStatus());
|
||||
if (input.unreadOnly && read) continue;
|
||||
found.push({
|
||||
id: String(message.id()),
|
||||
accountId: String(account.id()),
|
||||
account: String(account.name()),
|
||||
mailbox: String(mailbox.name()),
|
||||
subject: String(message.subject() || ""),
|
||||
sender: String(message.sender() || ""),
|
||||
dateSent: isoDate(message.dateSent()),
|
||||
read: read
|
||||
});
|
||||
}
|
||||
return JSON.stringify(found);
|
||||
}`;
|
||||
|
||||
const READ_MESSAGE_SCRIPT = String.raw`
|
||||
function isoDate(value) {
|
||||
try { return value ? value.toISOString() : null; } catch (_) { return null; }
|
||||
}
|
||||
|
||||
function globalMailbox(app, requestedName) {
|
||||
const candidates = [app.inbox(), app.sentMailbox(), app.draftsMailbox(), app.junkMailbox(), app.trashMailbox()];
|
||||
for (let i = 0; i < candidates.length; i++) {
|
||||
try { if (String(candidates[i].name()) === requestedName) return candidates[i]; } catch (_) {}
|
||||
}
|
||||
throw new Error("Mailbox not found; use mail_list_mailboxes first");
|
||||
}
|
||||
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Mail.app");
|
||||
const mailbox = globalMailbox(app, input.mailbox);
|
||||
let message;
|
||||
try { message = mailbox.messages.byId(Number(input.id)); } catch (_) { throw new Error("Message not found in the selected mailbox"); }
|
||||
let account;
|
||||
try { account = message.mailbox().account(); } catch (_) { throw new Error("Message not found in the selected mailbox"); }
|
||||
if (String(account.id()) !== input.accountId) throw new Error("Message does not belong to the selected account");
|
||||
return JSON.stringify({
|
||||
id: String(message.id()),
|
||||
accountId: String(account.id()),
|
||||
account: String(account.name()),
|
||||
mailbox: String(mailbox.name()),
|
||||
subject: String(message.subject() || ""),
|
||||
sender: String(message.sender() || ""),
|
||||
dateSent: isoDate(message.dateSent()),
|
||||
read: Boolean(message.readStatus()),
|
||||
body: String(message.content() || "")
|
||||
});
|
||||
}`;
|
||||
|
||||
function requireAccountId(value) {
|
||||
if (typeof value !== "string" || !value.trim()) throw new Error("accountId is required");
|
||||
return value;
|
||||
}
|
||||
|
||||
function requireMailbox(value) {
|
||||
if (typeof value !== "string" || !value.trim()) throw new Error("mailbox is required");
|
||||
return value;
|
||||
}
|
||||
|
||||
export function createMailClient(execute = runJxa) {
|
||||
return {
|
||||
accounts() {
|
||||
return execute(ACCOUNTS_SCRIPT, {});
|
||||
},
|
||||
mailboxes({ accountId }) {
|
||||
requireAccountId(accountId);
|
||||
return execute(MAILBOXES_SCRIPT, {});
|
||||
},
|
||||
listMessages({ accountId, mailbox, limit = 10, unreadOnly = false }) {
|
||||
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_MESSAGES) throw new Error(`limit must be between 1 and ${MAX_MESSAGES}`);
|
||||
return execute(LIST_MESSAGES_SCRIPT, { accountId: requireAccountId(accountId), mailbox: requireMailbox(mailbox), limit, unreadOnly: Boolean(unreadOnly) });
|
||||
},
|
||||
readMessage({ accountId, mailbox, id }) {
|
||||
if (typeof id !== "string" || !id.trim()) throw new Error("id is required");
|
||||
return execute(READ_MESSAGE_SCRIPT, { accountId: requireAccountId(accountId), mailbox: requireMailbox(mailbox), id });
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const mail = createMailClient();
|
||||
export const listMailAccounts = () => mail.accounts();
|
||||
export const listMailboxes = (input) => mail.mailboxes(input);
|
||||
export const listMailMessages = (input) => mail.listMessages(input);
|
||||
export const readMailMessage = (input) => mail.readMessage(input);
|
||||
@@ -0,0 +1,109 @@
|
||||
import { plainTextToNoteHtml, runJxa } from "../apple-events.js";
|
||||
|
||||
const NOTES_LIST_SCRIPT = String.raw`
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Notes.app");
|
||||
const query = (input.query || "").toLowerCase();
|
||||
const folderName = input.folder || "";
|
||||
const limit = input.limit;
|
||||
const found = [];
|
||||
const accounts = app.accounts();
|
||||
|
||||
for (let a = 0; a < accounts.length && found.length < limit; a++) {
|
||||
const folders = accounts[a].folders();
|
||||
for (let f = 0; f < folders.length && found.length < limit; f++) {
|
||||
const folder = folders[f];
|
||||
const currentFolder = String(folder.name());
|
||||
if (folderName && currentFolder !== folderName) continue;
|
||||
const notes = folder.notes();
|
||||
for (let n = 0; n < notes.length && found.length < limit; n++) {
|
||||
const note = notes[n];
|
||||
let title = "";
|
||||
let text = "";
|
||||
try { title = String(note.name()); } catch (_) {}
|
||||
try { text = String(note.plaintext()); } catch (_) {}
|
||||
if (query && (title + "\n" + text).toLowerCase().indexOf(query) === -1) continue;
|
||||
found.push({
|
||||
id: String(note.id()),
|
||||
title: title,
|
||||
folder: currentFolder,
|
||||
modifiedAt: note.modificationDate().toISOString(),
|
||||
preview: input.includePreview ? text.slice(0, 180) : undefined
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return JSON.stringify(found);
|
||||
}`;
|
||||
|
||||
const NOTES_READ_SCRIPT = String.raw`
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Notes.app");
|
||||
const accounts = app.accounts();
|
||||
for (let a = 0; a < accounts.length; a++) {
|
||||
const folders = accounts[a].folders();
|
||||
for (let f = 0; f < folders.length; f++) {
|
||||
const notes = folders[f].notes();
|
||||
for (let n = 0; n < notes.length; n++) {
|
||||
const note = notes[n];
|
||||
if (String(note.id()) === input.id) {
|
||||
return JSON.stringify({
|
||||
id: String(note.id()),
|
||||
title: String(note.name()),
|
||||
folder: String(folders[f].name()),
|
||||
bodyHtml: String(note.body()),
|
||||
plaintext: String(note.plaintext()),
|
||||
createdAt: note.creationDate().toISOString(),
|
||||
modifiedAt: note.modificationDate().toISOString()
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
throw new Error("Note not found");
|
||||
}`;
|
||||
|
||||
const NOTES_CREATE_SCRIPT = String.raw`
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Notes.app");
|
||||
const accounts = app.accounts();
|
||||
let destination = null;
|
||||
|
||||
for (let a = 0; a < accounts.length && !destination; a++) {
|
||||
const folders = accounts[a].folders();
|
||||
for (let f = 0; f < folders.length; f++) {
|
||||
if (!input.folder || String(folders[f].name()) === input.folder) {
|
||||
destination = folders[f];
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!destination) throw new Error("Notes destination folder not found");
|
||||
|
||||
const note = app.Note({body: input.html});
|
||||
destination.notes.push(note);
|
||||
return JSON.stringify({
|
||||
id: String(note.id()),
|
||||
title: String(note.name()),
|
||||
folder: String(destination.name())
|
||||
});
|
||||
}`;
|
||||
|
||||
export function listNotes(input) {
|
||||
return runJxa(NOTES_LIST_SCRIPT, input);
|
||||
}
|
||||
|
||||
export function readNote(id) {
|
||||
return runJxa(NOTES_READ_SCRIPT, { id });
|
||||
}
|
||||
|
||||
export function createNote({ title, body, folder }) {
|
||||
return runJxa(NOTES_CREATE_SCRIPT, {
|
||||
folder,
|
||||
html: plainTextToNoteHtml(title, body),
|
||||
});
|
||||
}
|
||||
+171
@@ -0,0 +1,171 @@
|
||||
import { dateFromInput, runJxa } from "../apple-events.js";
|
||||
|
||||
const LISTS_SCRIPT = String.raw`
|
||||
function run() {
|
||||
const app = Application("/System/Applications/Reminders.app");
|
||||
function stringValue(value) {
|
||||
try {
|
||||
return value === null || value === undefined ? null : String(value);
|
||||
} catch (_) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
function accountName(list) {
|
||||
try {
|
||||
const container = list.container();
|
||||
return stringValue(container.name ? container.name() : container);
|
||||
} catch (_) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return JSON.stringify(app.lists().map(function (list) {
|
||||
return {
|
||||
id: String(list.id()),
|
||||
name: String(list.name()),
|
||||
account: accountName(list),
|
||||
shared: null,
|
||||
assignmentMetadata: {
|
||||
available: null,
|
||||
note: "Apple Reminders automation does not expose shared-list participant metadata directly."
|
||||
}
|
||||
};
|
||||
}));
|
||||
}`;
|
||||
|
||||
const REMINDERS_LIST_SCRIPT = String.raw`
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Reminders.app");
|
||||
const lists = app.lists();
|
||||
const found = [];
|
||||
function stringValue(value) {
|
||||
try {
|
||||
return value === null || value === undefined ? null : String(value);
|
||||
} catch (_) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
function compact(value) {
|
||||
if (!value) return null;
|
||||
const text = String(value).trim();
|
||||
return text ? text : null;
|
||||
}
|
||||
function propertyValue(object, names) {
|
||||
for (let i = 0; i < names.length; i++) {
|
||||
try {
|
||||
const getter = object[names[i]];
|
||||
if (typeof getter !== "function") continue;
|
||||
const value = getter.call(object);
|
||||
const text = compact(stringValue(value));
|
||||
if (text && !text.startsWith("[object ")) return text;
|
||||
if (value && typeof value.name === "function") {
|
||||
const name = compact(stringValue(value.name()));
|
||||
if (name) return name;
|
||||
}
|
||||
} catch (_) {}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
function assignmentHint(title, notes) {
|
||||
const titleMatch = title.match(/\(([^()\n]{2,80})\)\s*$/);
|
||||
if (titleMatch) return { assignee: titleMatch[1].trim(), source: "title" };
|
||||
|
||||
const explicit = notes.match(/\b(?:assigned to|assignee)\s*:\s*([^.;,\n]{2,80})/i);
|
||||
if (explicit) return { assignee: explicit[1].trim(), source: "notes" };
|
||||
|
||||
const captured = notes.match(/\bCaptured\s+\d{4}-\d{2}-\d{2},\s*([^.;,\n]{2,80})\s*[.;]/i);
|
||||
if (captured) return { assignee: captured[1].trim(), source: "notes" };
|
||||
|
||||
return { assignee: null, source: null };
|
||||
}
|
||||
function assignmentFor(reminder, title, notes) {
|
||||
const nativeAssignee = propertyValue(reminder, [
|
||||
"assignedTo",
|
||||
"assignee",
|
||||
"assignment",
|
||||
"responsiblePerson",
|
||||
"principal"
|
||||
]);
|
||||
if (nativeAssignee) {
|
||||
return {
|
||||
assignee: nativeAssignee,
|
||||
source: "remindersAutomation",
|
||||
available: true
|
||||
};
|
||||
}
|
||||
|
||||
const hint = assignmentHint(title, notes);
|
||||
return {
|
||||
assignee: hint.assignee,
|
||||
source: hint.source,
|
||||
available: hint.assignee !== null
|
||||
};
|
||||
}
|
||||
for (let l = 0; l < lists.length && found.length < input.limit; l++) {
|
||||
const list = lists[l];
|
||||
const name = String(list.name());
|
||||
if (input.list && name !== input.list) continue;
|
||||
const reminders = list.reminders();
|
||||
for (let r = 0; r < reminders.length && found.length < input.limit; r++) {
|
||||
const reminder = reminders[r];
|
||||
const completed = Boolean(reminder.completed());
|
||||
if (input.completed !== null && completed !== input.completed) continue;
|
||||
let due = null;
|
||||
try {
|
||||
const value = reminder.dueDate();
|
||||
due = value ? value.toISOString() : null;
|
||||
} catch (_) {}
|
||||
const title = String(reminder.name());
|
||||
const notes = String(reminder.body() || "");
|
||||
found.push({
|
||||
id: String(reminder.id()),
|
||||
list: name,
|
||||
title: title,
|
||||
completed: completed,
|
||||
due: due,
|
||||
notes: notes,
|
||||
assignment: assignmentFor(reminder, title, notes)
|
||||
});
|
||||
}
|
||||
}
|
||||
return JSON.stringify(found);
|
||||
}`;
|
||||
|
||||
const REMINDER_CREATE_SCRIPT = String.raw`
|
||||
function run(argv) {
|
||||
const input = JSON.parse(argv[0]);
|
||||
const app = Application("/System/Applications/Reminders.app");
|
||||
const lists = app.lists();
|
||||
let destination = null;
|
||||
for (let l = 0; l < lists.length; l++) {
|
||||
if (!input.list || String(lists[l].name()) === input.list) {
|
||||
destination = lists[l];
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!destination) throw new Error("Reminders list not found");
|
||||
const properties = {name: input.title, body: input.notes || ""};
|
||||
if (input.due) properties.dueDate = new Date(input.due);
|
||||
const reminder = app.Reminder(properties);
|
||||
destination.reminders.push(reminder);
|
||||
return JSON.stringify({
|
||||
id: String(reminder.id()),
|
||||
list: String(destination.name()),
|
||||
title: String(reminder.name())
|
||||
});
|
||||
}`;
|
||||
|
||||
export function listReminderLists() {
|
||||
return runJxa(LISTS_SCRIPT);
|
||||
}
|
||||
|
||||
export function listReminders(input) {
|
||||
return runJxa(REMINDERS_LIST_SCRIPT, input);
|
||||
}
|
||||
|
||||
export function createReminder(input) {
|
||||
if (input.due) {
|
||||
dateFromInput(input.due, "due");
|
||||
}
|
||||
return runJxa(REMINDER_CREATE_SCRIPT, input);
|
||||
}
|
||||
+281
@@ -0,0 +1,281 @@
|
||||
import { execFile, spawn } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import { writeFile, readFile, stat, rm, mkdir, mkdtemp } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
function swiftSourcePipeTranscriber() {
|
||||
return `
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
import Speech
|
||||
|
||||
func parseArgs() -> (String, Bool) {
|
||||
var localeId = "en-US"
|
||||
var verbose = false
|
||||
var i = 1
|
||||
let raw = CommandLine.arguments
|
||||
while i < raw.count {
|
||||
let a = raw[i]
|
||||
if a == "--locale", i+1 < raw.count { localeId = raw[i+1]; i+=1 }
|
||||
else if a == "-v" || a == "--verbose" { verbose = true }
|
||||
i+=1
|
||||
}
|
||||
return (localeId, verbose)
|
||||
}
|
||||
func logv(_ msg: String, verbose: Bool) { if verbose { fputs("[apple-pipe] \\(msg)\\n", stderr) } }
|
||||
|
||||
@main
|
||||
struct ApplePipeCLI {
|
||||
static func main() async {
|
||||
let (localeId, verbose) = parseArgs()
|
||||
guard SpeechTranscriber.isAvailable else { fputs("Not available\\n", stderr); exit(1) }
|
||||
let reqLocale = Locale(identifier: localeId)
|
||||
let locale: Locale
|
||||
if let sup = await SpeechTranscriber.supportedLocale(equivalentTo: reqLocale) { locale = sup }
|
||||
else { locale = reqLocale }
|
||||
// warm asset check
|
||||
let warm = SpeechTranscriber(locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults], attributeOptions: [])
|
||||
let status = await AssetInventory.status(forModules: [warm])
|
||||
switch status {
|
||||
case .installed: logv("Assets installed", verbose: verbose)
|
||||
case .unsupported: fputs("Locale unsupported\\n", stderr); exit(2)
|
||||
case .supported:
|
||||
logv("Downloading assets...", verbose: true)
|
||||
do { if let req = try await AssetInventory.assetInstallationRequest(supporting: [warm]) { try await req.downloadAndInstall() } }
|
||||
catch { fputs("Asset download failed: \\(error)\\n", stderr); exit(3) }
|
||||
case .downloading:
|
||||
logv("Waiting assets...", verbose: true)
|
||||
for _ in 0..<30 { try? await Task.sleep(nanoseconds: 1_000_000_000); if await AssetInventory.status(forModules: [warm]) == .installed { break } }
|
||||
@unknown default: break
|
||||
}
|
||||
logv("Pipe ready locale=\\(locale.identifier)", verbose: true)
|
||||
let stdinH = FileHandle.standardInput
|
||||
var leftover = Data()
|
||||
var chunkIdx = 0
|
||||
func readExact(_ n: Int) -> Data? {
|
||||
var out = Data(); out.reserveCapacity(n)
|
||||
if leftover.count >= n { let d = leftover.prefix(n); leftover = leftover.dropFirst(n); return Data(d) }
|
||||
if leftover.count > 0 { out.append(leftover); leftover = Data() }
|
||||
while out.count < n {
|
||||
let d = stdinH.readData(ofLength: n - out.count)
|
||||
if d.isEmpty { if out.count==0 { return nil }; return nil }
|
||||
out.append(d)
|
||||
}
|
||||
return out
|
||||
}
|
||||
while true {
|
||||
guard let lenData = readExact(4) else { break }
|
||||
let length = lenData.withUnsafeBytes { $0.load(as: UInt32.self).bigEndian }
|
||||
if length == 0 { logv("Pipe EOF", verbose: true); break }
|
||||
if length > 20_000_000 { fputs("Chunk too large \\(length)\\n", stderr); break }
|
||||
guard let chunkData = readExact(Int(length)) else { fputs("Truncated expected \\(length)\\n", stderr); break }
|
||||
chunkIdx += 1
|
||||
let tmpURL = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("apple-pipe-\\(ProcessInfo.processInfo.processIdentifier)-\\(chunkIdx).wav")
|
||||
do { try chunkData.write(to: tmpURL) } catch { fputs("Write err: \\(error)\\n", stderr); continue }
|
||||
guard let audioFile = try? AVAudioFile(forReading: tmpURL) else { try? FileManager.default.removeItem(at: tmpURL); fputs("Open fail \\(chunkIdx)\\n", stderr); continue }
|
||||
let t = SpeechTranscriber(locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults], attributeOptions: [.audioTimeRange])
|
||||
guard let analyzer = try? await SpeechAnalyzer(inputAudioFile: audioFile, modules: [t], finishAfterFile: true) else { try? FileManager.default.removeItem(at: tmpURL); continue }
|
||||
do {
|
||||
for try await res in t.results {
|
||||
let txt = String(res.text.characters).trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if txt.isEmpty { continue }
|
||||
let d: [String: Any] = [
|
||||
"event": res.isFinal ? "final" : "draft",
|
||||
"text": txt, "isFinal": res.isFinal,
|
||||
"chunk": chunkIdx, "start": res.range.start.seconds,
|
||||
"duration": res.range.duration.seconds
|
||||
]
|
||||
if let jd = try? JSONSerialization.data(withJSONObject: d), let s = String(data: jd, encoding: .utf8) {
|
||||
print(s); fflush(stdout)
|
||||
}
|
||||
}
|
||||
} catch { fputs("Results err \\(chunkIdx): \\(error)\\n", stderr) }
|
||||
_ = analyzer
|
||||
try? FileManager.default.removeItem(at: tmpURL)
|
||||
}
|
||||
logv("Pipe done \\(chunkIdx)", verbose: true)
|
||||
}
|
||||
}
|
||||
`;
|
||||
}
|
||||
|
||||
class ApplePipeSession {
|
||||
constructor(locale) {
|
||||
this.locale = locale;
|
||||
this.proc = null;
|
||||
this.tmpDir = null;
|
||||
this.binFile = null;
|
||||
this.ready = false;
|
||||
this.chunkIdx = 0;
|
||||
this.lastActivity = Date.now();
|
||||
this._lineCallback = null;
|
||||
}
|
||||
async ensureBuilt() {
|
||||
const tmpBase = path.join(os.tmpdir(), "speech-live-");
|
||||
this.tmpDir = await mkdtemp(tmpBase);
|
||||
const swiftFile = path.join(this.tmpDir, "Main.swift");
|
||||
this.binFile = path.join(this.tmpDir, "apple-pipe-transcribe");
|
||||
await writeFile(swiftFile, swiftSourcePipeTranscriber(), "utf8");
|
||||
try {
|
||||
await execFileAsync("/usr/bin/swiftc", ["-O", "-parse-as-library", swiftFile, "-o", this.binFile, "-framework", "AVFoundation", "-framework", "Speech"], { timeout: 60000, maxBuffer: 20*1024*1024 });
|
||||
} catch (e) {
|
||||
throw new Error(`swiftc build failed: ${e.stderr||e.message}`);
|
||||
}
|
||||
}
|
||||
async start() {
|
||||
if (this.proc && this.proc.exitCode === null && this.ready) return;
|
||||
if (!this.binFile) await this.ensureBuilt();
|
||||
return new Promise((resolve, reject) => {
|
||||
const args = ["--locale", this.locale];
|
||||
this.proc = spawn(this.binFile, args, { stdio: ["pipe", "pipe", "pipe"] });
|
||||
let stderrBuf = "";
|
||||
this.proc.stderr.on("data", d => { stderrBuf += d.toString(); });
|
||||
setTimeout(() => {
|
||||
if (this.proc.exitCode !== null) {
|
||||
reject(new Error(`apple-pipe exited early code=${this.proc.exitCode} stderr=${stderrBuf.slice(0,2000)}`));
|
||||
} else {
|
||||
this.ready = true;
|
||||
this._setupReader();
|
||||
resolve();
|
||||
}
|
||||
}, 800);
|
||||
this.proc.on("error", reject);
|
||||
});
|
||||
}
|
||||
_setupReader() {
|
||||
let buf = "";
|
||||
this.proc.stdout.on("data", chunk => {
|
||||
buf += chunk.toString("utf8");
|
||||
let lines = buf.split("\n");
|
||||
buf = lines.pop() || "";
|
||||
for (let line of lines) {
|
||||
line = line.trim();
|
||||
if (!line) continue;
|
||||
try {
|
||||
const obj = JSON.parse(line);
|
||||
if (this._lineCallback) this._lineCallback(obj);
|
||||
} catch {}
|
||||
}
|
||||
});
|
||||
}
|
||||
async transcribeChunk(wavBytes, { timeoutMs = 6000 } = {}) {
|
||||
await this.start();
|
||||
this.chunkIdx++;
|
||||
const myIdx = this.chunkIdx;
|
||||
return new Promise((resolve, reject) => {
|
||||
let drafts = [];
|
||||
let finals = [];
|
||||
let timer = null;
|
||||
let done = false;
|
||||
const cleanup = () => { done = true; if (timer) clearTimeout(timer); this._lineCallback = null; };
|
||||
const finish = () => {
|
||||
if (done) return;
|
||||
cleanup();
|
||||
const fullFinal = finals.map(f=>f.text).join(" ").trim();
|
||||
const lastDraft = drafts.length ? drafts[drafts.length-1].text : "";
|
||||
const text = fullFinal || lastDraft || "";
|
||||
resolve({ text, finals, drafts, chunk: myIdx, isFinal: finals.length>0 });
|
||||
};
|
||||
this._lineCallback = (obj) => {
|
||||
if (obj.chunk !== myIdx) return;
|
||||
if (obj.isFinal || obj.event==="final") {
|
||||
finals.push(obj);
|
||||
if (timer) clearTimeout(timer);
|
||||
timer = setTimeout(finish, 350);
|
||||
} else {
|
||||
drafts.push(obj);
|
||||
if (timer) clearTimeout(timer);
|
||||
timer = setTimeout(finish, 700);
|
||||
}
|
||||
};
|
||||
timer = setTimeout(() => { if (!done) finish(); }, timeoutMs);
|
||||
try {
|
||||
const lenBuf = Buffer.alloc(4);
|
||||
lenBuf.writeUInt32BE(wavBytes.length, 0);
|
||||
this.proc.stdin.write(Buffer.concat([lenBuf, Buffer.from(wavBytes)]));
|
||||
} catch (e) {
|
||||
cleanup();
|
||||
reject(e);
|
||||
}
|
||||
});
|
||||
}
|
||||
async close() {
|
||||
try {
|
||||
if (this.proc && this.proc.stdin.writable) {
|
||||
const eof = Buffer.alloc(4); eof.writeUInt32BE(0,0);
|
||||
this.proc.stdin.write(eof);
|
||||
this.proc.stdin.end();
|
||||
}
|
||||
if (this.proc) {
|
||||
await new Promise(r => { this.proc.on("close", r); setTimeout(r, 1500); });
|
||||
try { this.proc.kill(); } catch {}
|
||||
}
|
||||
} catch {}
|
||||
try { if (this.tmpDir) await rm(this.tmpDir, {recursive:true, force:true}); } catch {}
|
||||
this.proc = null;
|
||||
this.ready = false;
|
||||
}
|
||||
}
|
||||
|
||||
const sessions = new Map();
|
||||
|
||||
async function getOrCreateSession(locale) {
|
||||
let s = sessions.get(locale);
|
||||
if (!s) {
|
||||
s = new ApplePipeSession(locale);
|
||||
sessions.set(locale, s);
|
||||
}
|
||||
await s.start();
|
||||
s.lastActivity = Date.now();
|
||||
return s;
|
||||
}
|
||||
|
||||
export async function speechLiveTranscribe({ audioBase64, locale } = {}) {
|
||||
if (!audioBase64) throw new Error("audioBase64 required (WAV 16k mono base64)");
|
||||
const wavBytes = Buffer.from(audioBase64, "base64");
|
||||
const loc = locale || "en-US";
|
||||
const session = await getOrCreateSession(loc);
|
||||
const res = await session.transcribeChunk(wavBytes, { timeoutMs: 8000 });
|
||||
return {
|
||||
ok: true,
|
||||
engine: "ApplePipeTranscriber/macOS26.5 --pipe volatile drafts",
|
||||
locale: loc,
|
||||
text: res.text,
|
||||
isFinal: res.isFinal,
|
||||
drafts: res.drafts,
|
||||
finals: res.finals,
|
||||
chunk: res.chunk,
|
||||
realtime: true
|
||||
};
|
||||
}
|
||||
|
||||
export async function speechLiveClose({ locale } = {}) {
|
||||
const loc = locale || "en-US";
|
||||
const s = sessions.get(loc);
|
||||
if (s) {
|
||||
await s.close();
|
||||
sessions.delete(loc);
|
||||
}
|
||||
return { ok: true, closed: loc };
|
||||
}
|
||||
|
||||
export async function speechLiveStatus() {
|
||||
const info = [];
|
||||
for (let [loc, s] of sessions.entries()) {
|
||||
info.push({ locale: loc, ready: s.ready, chunkIdx: s.chunkIdx, lastActivity: new Date(s.lastActivity).toISOString(), pid: s.proc?.pid || null });
|
||||
}
|
||||
return { sessions: info, count: info.length };
|
||||
}
|
||||
|
||||
setInterval(async () => {
|
||||
const now = Date.now();
|
||||
for (let [loc, s] of sessions.entries()) {
|
||||
if (now - s.lastActivity > 60000) {
|
||||
try { await s.close(); } catch {}
|
||||
sessions.delete(loc);
|
||||
}
|
||||
}
|
||||
}, 15000);
|
||||
+277
@@ -0,0 +1,277 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import { writeFile, readFile, stat, rm, mkdir } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const OUTPUT_BASE = path.join(os.homedir(), "Projects", "MacMiniMCP", "generated-transcriptions");
|
||||
|
||||
// Swift source for file transcription using the REAL macOS 26.5 API
|
||||
// From swiftinterface: SpeechAnalyzer has init(inputAudioFile:modules:) and analyzeSequence(from:)
|
||||
// No AssetInputSequenceProvider - it's direct AVAudioFile
|
||||
|
||||
function swiftSourceFileTranscriber() {
|
||||
return `import Speech
|
||||
import AVFoundation
|
||||
import Foundation
|
||||
|
||||
@main
|
||||
struct TranscribeCLI {
|
||||
static func main() async {
|
||||
let args = CommandLine.arguments
|
||||
let audioPath = args.count > 1 ? args[1] : ""
|
||||
let localeId = args.count > 2 ? args[2] : "en-US"
|
||||
let jsonOut = args.count > 3 ? args[3] : ""
|
||||
|
||||
if audioPath.isEmpty {
|
||||
fputs("Usage: transcriber <audioPath> [locale] [jsonOut]\\n", stderr)
|
||||
exit(1)
|
||||
}
|
||||
|
||||
let startTime = CFAbsoluteTimeGetCurrent()
|
||||
|
||||
guard FileManager.default.fileExists(atPath: audioPath) else {
|
||||
fputs("ERROR: File not found \\(audioPath)\\n", stderr)
|
||||
exit(3)
|
||||
}
|
||||
|
||||
let audioURL = URL(fileURLWithPath: audioPath)
|
||||
let requestedLocale = Locale(identifier: localeId)
|
||||
|
||||
let resolvedLocale: Locale
|
||||
if let l = await SpeechTranscriber.supportedLocale(equivalentTo: requestedLocale) {
|
||||
resolvedLocale = l
|
||||
} else if let fb = await SpeechTranscriber.supportedLocale(equivalentTo: Locale(identifier: "en-US")) {
|
||||
resolvedLocale = fb
|
||||
} else {
|
||||
fputs("ERROR: No supported locale for \\(localeId)\\n", stderr)
|
||||
exit(2)
|
||||
}
|
||||
|
||||
let isAvail = SpeechTranscriber.isAvailable
|
||||
fputs("Locale \\(resolvedLocale.identifier) isAvailable=\\(isAvail)\\n", stderr)
|
||||
|
||||
let transcriber = SpeechTranscriber(locale: resolvedLocale, preset: .transcription)
|
||||
|
||||
// Assets
|
||||
do {
|
||||
if let req = try await AssetInventory.assetInstallationRequest(supporting: [transcriber]) {
|
||||
fputs("Downloading assets for \\(resolvedLocale.identifier)...\\n", stderr)
|
||||
try await req.downloadAndInstall()
|
||||
fputs("Assets ready\\n", stderr)
|
||||
} else {
|
||||
fputs("No asset download needed\\n", stderr)
|
||||
}
|
||||
} catch {
|
||||
fputs("Asset note (continuing): \\(error)\\n", stderr)
|
||||
}
|
||||
|
||||
let avFile: AVAudioFile
|
||||
do {
|
||||
avFile = try AVAudioFile(forReading: audioURL)
|
||||
fputs("File: frames=\\(avFile.length) sr=\\(avFile.processingFormat.sampleRate) fmt=\\(avFile.fileFormat)\\n", stderr)
|
||||
} catch {
|
||||
fputs("ERROR opening file: \\(error)\\n", stderr)
|
||||
exit(4)
|
||||
}
|
||||
|
||||
var allSegments: [String] = []
|
||||
|
||||
do {
|
||||
// Use the new convenience: analyzer from audio file
|
||||
let analyzer = try await SpeechAnalyzer(inputAudioFile: avFile, modules: [transcriber], finishAfterFile: true)
|
||||
|
||||
// Collect results - must be concurrent
|
||||
let collector = Task {
|
||||
do {
|
||||
for try await r in transcriber.results {
|
||||
let plain = String(r.text.characters)
|
||||
if !plain.isEmpty {
|
||||
allSegments.append(plain)
|
||||
fputs("[result] \\(plain)\\n", stderr)
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
fputs("Results error: \\(error)\\n", stderr)
|
||||
}
|
||||
}
|
||||
|
||||
// analysis was already started by init with finishAfterFile=true, just wait
|
||||
// Alternatively use analyzeSequence(from:) pattern:
|
||||
// But since init already starts with file, we just wait for collector
|
||||
// The analyzer will finish automatically due to finishAfterFile:true
|
||||
|
||||
// Wait for collector - it finishes when analyzer finishes file and finalizes
|
||||
await collector.value
|
||||
|
||||
let elapsed = CFAbsoluteTimeGetCurrent() - startTime
|
||||
let full = allSegments.joined(separator: " ")
|
||||
let durationSec = avFile.length > 0 ? Double(avFile.length) / avFile.processingFormat.sampleRate : 0
|
||||
let installed = await SpeechTranscriber.installedLocales.map { $0.identifier }.sorted()
|
||||
|
||||
let payload: [String: Any] = [
|
||||
"ok": true,
|
||||
"engine": "SpeechAnalyzer+SpeechTranscriber/macOS26.5",
|
||||
"locale": resolvedLocale.identifier,
|
||||
"requestedLocale": localeId,
|
||||
"transcript": full,
|
||||
"segments": allSegments,
|
||||
"elapsedSeconds": elapsed,
|
||||
"audioPath": audioPath,
|
||||
"durationSeconds": durationSec,
|
||||
"realtimeFactor": durationSec > 0 ? elapsed / durationSec : 0,
|
||||
"rtfx": durationSec > 0 ? durationSec / elapsed : 0,
|
||||
"frames": Int(avFile.length),
|
||||
"sampleRate": avFile.processingFormat.sampleRate,
|
||||
"macOS": ProcessInfo.processInfo.operatingSystemVersionString,
|
||||
"isAvailable": isAvail,
|
||||
"installedLocales": installed
|
||||
]
|
||||
|
||||
let dataOut = try JSONSerialization.data(withJSONObject: payload, options: [.prettyPrinted, .sortedKeys])
|
||||
if !jsonOut.isEmpty {
|
||||
try dataOut.write(to: URL(fileURLWithPath: jsonOut))
|
||||
}
|
||||
FileHandle.standardOutput.write(dataOut)
|
||||
|
||||
} catch {
|
||||
fputs("Analysis failed: \\(error)\\n", stderr)
|
||||
// Dump chain
|
||||
var cur: Error? = error
|
||||
while let e = cur {
|
||||
fputs(" -> \\(e)\\n", stderr)
|
||||
cur = (e as NSError).userInfo[NSUnderlyingErrorKey] as? Error
|
||||
}
|
||||
exit(6)
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
}
|
||||
|
||||
async function ensureOutputDir() {
|
||||
await mkdir(OUTPUT_BASE, { recursive: true });
|
||||
}
|
||||
|
||||
async function cleanup(dir) {
|
||||
try { await rm(dir, { recursive: true, force: true }); } catch {}
|
||||
}
|
||||
|
||||
async function buildAndRun({ audioPath, locale = "en-US" }) {
|
||||
const tmpDir = await (await import("node:fs/promises")).mkdtemp.call(null, path.join(os.tmpdir(), "speech-t-"));
|
||||
// compatible mkdtemp
|
||||
const { mkdtemp } = await import("node:fs/promises");
|
||||
const dir = await mkdtemp(path.join(os.tmpdir(), "speech-t-"));
|
||||
const swiftFile = path.join(dir, "Main.swift");
|
||||
const binFile = path.join(dir, "transcriber");
|
||||
const jsonOut = path.join(dir, "result.json");
|
||||
|
||||
await writeFile(swiftFile, swiftSourceFileTranscriber(), "utf8");
|
||||
|
||||
try {
|
||||
await execFileAsync("/usr/bin/swiftc", ["-O", "-parse-as-library", swiftFile, "-o", binFile, "-framework", "AVFoundation", "-framework", "Speech"], { timeout: 60_000, maxBuffer: 20*1024*1024 });
|
||||
} catch (e) {
|
||||
await cleanup(dir);
|
||||
throw new Error(`swiftc compile failed:\n${e.stderr || e.message}\n${e.stdout||""}`);
|
||||
}
|
||||
|
||||
try { await stat(binFile); } catch { await cleanup(dir); throw new Error("Binary not built"); }
|
||||
|
||||
try {
|
||||
const { stdout, stderr } = await execFileAsync(binFile, [audioPath, locale, jsonOut], { timeout: 120_000, maxBuffer: 20*1024*1024 });
|
||||
if (stderr) { try { process.stderr.write(stderr.slice(0,4000)); } catch {} }
|
||||
let result;
|
||||
try { result = JSON.parse(await readFile(jsonOut, "utf8")); } catch { result = JSON.parse(stdout); }
|
||||
await ensureOutputDir();
|
||||
const persistPath = path.join(OUTPUT_BASE, `transcription-${Date.now()}.json`);
|
||||
try { await writeFile(persistPath, JSON.stringify(result, null, 2), "utf8"); result.persistedTo = persistPath; } catch {}
|
||||
await cleanup(dir);
|
||||
return result;
|
||||
} catch (e) {
|
||||
const out = e.stdout || "";
|
||||
const er = e.stderr || e.message || "";
|
||||
try {
|
||||
const partial = JSON.parse(out);
|
||||
if (partial && partial.ok) { await cleanup(dir); return partial; }
|
||||
} catch {}
|
||||
await cleanup(dir);
|
||||
throw new Error(`Transcribe failed\nSTDOUT:${out.slice(0,4000)}\nSTDERR:${er.slice(0,6000)}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function speechListLocales() {
|
||||
const { mkdtemp } = await import("node:fs/promises");
|
||||
const dir = await mkdtemp(path.join(os.tmpdir(), "speech-loc-"));
|
||||
const swiftFile = path.join(dir, "List.swift");
|
||||
const binFile = path.join(dir, "list");
|
||||
const swiftSrc = `import Speech
|
||||
import Foundation
|
||||
@main
|
||||
struct L {
|
||||
static func main() async {
|
||||
let isAvail = SpeechTranscriber.isAvailable
|
||||
let supported = await SpeechTranscriber.supportedLocales.map { $0.identifier }.sorted()
|
||||
let installed = await SpeechTranscriber.installedLocales.map { $0.identifier }.sorted()
|
||||
let reserved = await AssetInventory.reservedLocales.map { $0.identifier }
|
||||
let payload: [String: Any] = [
|
||||
"isAvailable": isAvail,
|
||||
"supported": supported,
|
||||
"installed": installed,
|
||||
"macOS": ProcessInfo.processInfo.operatingSystemVersionString,
|
||||
"maxReserved": AssetInventory.maximumReservedLocales,
|
||||
"reserved": reserved
|
||||
]
|
||||
let d = try! JSONSerialization.data(withJSONObject: payload, options: [.prettyPrinted, .sortedKeys])
|
||||
FileHandle.standardOutput.write(d)
|
||||
}
|
||||
}
|
||||
`;
|
||||
await writeFile(swiftFile, swiftSrc, "utf8");
|
||||
try {
|
||||
await execFileAsync("/usr/bin/swiftc", ["-O", "-parse-as-library", swiftFile, "-o", binFile, "-framework", "Speech"], { timeout: 30_000 });
|
||||
const { stdout } = await execFileAsync(binFile, [], { timeout: 15_000 });
|
||||
await cleanup(dir);
|
||||
return JSON.parse(stdout);
|
||||
} catch (e) {
|
||||
await cleanup(dir);
|
||||
throw new Error(`List locales failed: ${e.stderr||e.message}\n${e.stdout||""}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function speechTranscribeFile({ filePath, locale }) {
|
||||
if (!filePath) throw new Error("filePath required");
|
||||
const resolved = path.resolve(filePath.replace(/^~(?=$|\/)/, os.homedir()));
|
||||
try { await stat(resolved); } catch { throw new Error(`File not found: ${resolved}`); }
|
||||
return await buildAndRun({ audioPath: resolved, locale: locale || "en-US" });
|
||||
}
|
||||
|
||||
export async function speechQuickTest({ text, voice } = {}) {
|
||||
const testText = text || "Hello world this is a test of Apple SpeechAnalyzer on the Mac mini M four";
|
||||
const testVoice = voice || "Alex";
|
||||
const { mkdtemp } = await import("node:fs/promises");
|
||||
const tmp = await mkdtemp(path.join(os.tmpdir(), "speech-qtest-"));
|
||||
const aiffPath = path.join(tmp, "test.aiff");
|
||||
try {
|
||||
await execFileAsync("/usr/bin/say", ["-v", testVoice, "-o", aiffPath, testText], { timeout: 15_000 });
|
||||
} catch {
|
||||
try { await execFileAsync("/usr/bin/say", ["-o", aiffPath, testText], { timeout: 15_000 }); }
|
||||
catch (e2) { await cleanup(tmp); throw new Error(`say failed: ${e2.stderr||e2.message}`); }
|
||||
}
|
||||
try { await stat(aiffPath); } catch { await cleanup(tmp); throw new Error("Generated audio not found"); }
|
||||
|
||||
let result;
|
||||
try { result = await buildAndRun({ audioPath: aiffPath, locale: "en-US" }); }
|
||||
catch (e) { await cleanup(tmp); throw e; }
|
||||
|
||||
result.testInputText = testText;
|
||||
result.testVoice = testVoice;
|
||||
try {
|
||||
await ensureOutputDir();
|
||||
const dest = path.join(OUTPUT_BASE, `qtest-${Date.now()}.aiff`);
|
||||
await execFileAsync("/bin/cp", [aiffPath, dest], { timeout: 5_000 });
|
||||
result.persistedAudio = dest;
|
||||
} catch {}
|
||||
await cleanup(tmp);
|
||||
return result;
|
||||
}
|
||||
+149
@@ -0,0 +1,149 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
import { mkdir, stat, readFile, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const OUTPUT_BASE = path.join(os.homedir(), "Projects", "MacMiniMCP", "generated-audio");
|
||||
|
||||
async function ensureOutputDir() {
|
||||
await mkdir(OUTPUT_BASE, { recursive: true });
|
||||
}
|
||||
|
||||
function sanitizeVoice(v) {
|
||||
if (!v) return null;
|
||||
return String(v).trim().slice(0, 100) || null;
|
||||
}
|
||||
|
||||
function listVoicesParse(stdout) {
|
||||
// format from `say -v ?` : "Alex en_US # Most people recognize me by my voice."
|
||||
const lines = stdout.split("\n").map(l => l.trim()).filter(Boolean);
|
||||
const voices = [];
|
||||
for (const line of lines) {
|
||||
const match = line.match(/^(\S+)\s+([a-z]{2}_[A-Z]{2}(?:_[A-Z]+)?)\s+#?\s*(.*)$/);
|
||||
if (match) {
|
||||
voices.push({ name: match[1], locale: match[2], description: match[3] || "" });
|
||||
} else {
|
||||
const parts = line.split(/\s+/);
|
||||
if (parts.length >= 1 && parts[0]) {
|
||||
voices.push({ name: parts[0], locale: parts[1] || "", description: parts.slice(2).join(" ") });
|
||||
}
|
||||
}
|
||||
}
|
||||
return voices;
|
||||
}
|
||||
|
||||
export async function speechListVoices() {
|
||||
try {
|
||||
const { stdout } = await execFileAsync("/usr/bin/say", ["-v", "?"], { timeout: 10000, maxBuffer: 10 * 1024 * 1024 });
|
||||
const voices = listVoicesParse(stdout);
|
||||
return { ok: true, count: voices.length, voices: voices.slice(0, 150) };
|
||||
} catch (e) {
|
||||
throw new Error(`Failed to list voices: ${e.stderr || e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function speechSynthesize({ text, voice, rate, outputFormat }) {
|
||||
if (!text || !String(text).trim()) throw new Error("text required");
|
||||
const cleanText = String(text).slice(0, 5000);
|
||||
const v = sanitizeVoice(voice);
|
||||
await ensureOutputDir();
|
||||
|
||||
const ts = Date.now();
|
||||
const rand = Math.random().toString(16).slice(2, 8);
|
||||
const aiffPath = path.join(OUTPUT_BASE, `tts-${ts}-${rand}.aiff`);
|
||||
const wavPath = path.join(OUTPUT_BASE, `tts-${ts}-${rand}.wav`);
|
||||
const finalWav16k = path.join(OUTPUT_BASE, `tts-${ts}-${rand}-16k.wav`);
|
||||
|
||||
const sayArgs = [];
|
||||
if (v) sayArgs.push("-v", v);
|
||||
if (rate) {
|
||||
const r = parseInt(String(rate), 10);
|
||||
if (!isNaN(r) && r >= 80 && r <= 500) {
|
||||
sayArgs.push("-r", String(r));
|
||||
}
|
||||
}
|
||||
sayArgs.push("-o", aiffPath, cleanText);
|
||||
|
||||
try {
|
||||
await execFileAsync("/usr/bin/say", sayArgs, { timeout: 30000, maxBuffer: 20 * 1024 * 1024 });
|
||||
} catch (e) {
|
||||
throw new Error(`say failed: ${e.stderr || e.message}\nOUT:${e.stdout||""}`);
|
||||
}
|
||||
|
||||
try { await stat(aiffPath); } catch { throw new Error("say output not created"); }
|
||||
|
||||
// Prefer afconvert to make 16k mono wav compatible with iPhone play_audio_base64 (expects 16k PCM s16le mono)
|
||||
// afconvert -f WAVE -d LEI16@16000 -c 1 in.aiff out.wav
|
||||
// Fallback to ffmpeg if afconvert not available is not ideal on Mac, but we try afconvert first.
|
||||
try {
|
||||
try {
|
||||
await execFileAsync("/usr/bin/afconvert", ["-f", "WAVE", "-d", "LEI16@16000", "-c", "1", aiffPath, finalWav16k], { timeout: 15000 });
|
||||
await stat(finalWav16k);
|
||||
// Also create regular wav for compatibility
|
||||
await execFileAsync("/usr/bin/afconvert", ["-f", "WAVE", "-d", "LEI16", aiffPath, wavPath], { timeout: 10000 }).catch(()=>{});
|
||||
} catch {
|
||||
// Fallback: try format without @ rate, then use afinfo, or just keep aiff path
|
||||
await execFileAsync("/usr/bin/afconvert", ["-f", "WAVE", "-d", "LEI16", "-c", "1", aiffPath, finalWav16k], { timeout: 15000 });
|
||||
}
|
||||
} catch (e) {
|
||||
// If afconvert fails, keep aiff and tell caller
|
||||
// We'll still return aiff path
|
||||
}
|
||||
|
||||
let chosenPath = finalWav16k;
|
||||
try { await stat(chosenPath); } catch {
|
||||
try { await stat(wavPath); chosenPath = wavPath; } catch { chosenPath = aiffPath; }
|
||||
}
|
||||
|
||||
let wavBase64 = null;
|
||||
let base64Len = 0;
|
||||
// For fastest iPhone playback, provide base64 of 16k wav
|
||||
try {
|
||||
// try finalWav16k first
|
||||
let b64Target = finalWav16k;
|
||||
try { await stat(b64Target); } catch { b64Target = chosenPath; }
|
||||
const buf = await readFile(b64Target);
|
||||
// If file > 2MB, we still encode but warn - iPhone can handle ~500KB typical
|
||||
if (buf.length < 4 * 1024 * 1024) {
|
||||
wavBase64 = buf.toString("base64");
|
||||
base64Len = wavBase64.length;
|
||||
}
|
||||
} catch {}
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
text: cleanText,
|
||||
voice: v || "default",
|
||||
rate: rate || null,
|
||||
aiffPath,
|
||||
wavPath: finalWav16k,
|
||||
filePath: chosenPath,
|
||||
fallbackPath: aiffPath,
|
||||
wavBase64: wavBase64 ? wavBase64.slice(0, 50) + "...(truncated for display)" : null,
|
||||
wavBase64Full: wavBase64 ? "available" : null,
|
||||
base64Length: base64Len,
|
||||
fileSize: (await stat(chosenPath).then(s=>s.size).catch(()=>0)),
|
||||
note: "Use filePath on Mac. For iPhone play_audio_base64, use the base64 wav. Call speech_synthesize_file variant or read endpoint needs full base64."
|
||||
};
|
||||
}
|
||||
|
||||
export async function speechSynthesizeBase64({ text, voice, rate }) {
|
||||
if (!text) throw new Error("text required");
|
||||
const res = await speechSynthesize({ text, voice, rate });
|
||||
// read the 16k wav file full base64
|
||||
const target = res.wavPath || res.filePath;
|
||||
const buf = await readFile(target);
|
||||
const b64 = buf.toString("base64");
|
||||
return {
|
||||
ok: true,
|
||||
text: String(text).slice(0, 5000),
|
||||
voice: res.voice,
|
||||
filePath: target,
|
||||
wavBase64: b64,
|
||||
size: buf.length,
|
||||
base64Length: b64.length,
|
||||
format: "wav 16k mono s16le"
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
import { execFile, execFile as execFileCb } from "node:child_process";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
export async function getSystemInfo() {
|
||||
const results = {};
|
||||
// sw_vers for macOS version
|
||||
try {
|
||||
const { stdout } = await execFileAsync("/usr/bin/sw_vers", { timeout: 5000 });
|
||||
const info = {};
|
||||
for (const line of stdout.split("\n")) {
|
||||
const [k, ...rest] = line.split(":");
|
||||
if (!k) continue;
|
||||
const key = k.trim();
|
||||
if (!key) continue;
|
||||
info[key] = rest.join(":").trim();
|
||||
}
|
||||
results.sw_vers = info;
|
||||
results.macos_version = info.ProductVersion || info.productVersion || "";
|
||||
results.build = info.BuildVersion || info.buildVersion || "";
|
||||
} catch (e) {
|
||||
results.sw_vers_error = e.message;
|
||||
}
|
||||
|
||||
// uname -a
|
||||
try {
|
||||
const { stdout } = await execFileAsync("/usr/bin/uname", ["-a"], { timeout: 3000 });
|
||||
results.uname = stdout.trim();
|
||||
} catch (e) {
|
||||
results.uname_error = e.message;
|
||||
}
|
||||
|
||||
// Check for SpeechAnalyzer / SpeechTranscriber availability (macOS 26+)
|
||||
// These frameworks exist only on macOS 26+. We probe via swift/python check.
|
||||
try {
|
||||
const { stdout } = await execFileAsync("/usr/bin/python3", ["-c", `
|
||||
import os, glob, sys
|
||||
# Check if Speech framework contains new symbols (macOS 26)
|
||||
frameworks = glob.glob('/System/Library/Frameworks/Speech.framework/*') + glob.glob('/System/Library/PrivateFrameworks/*Speech*')
|
||||
# Simplest: check macOS version parse
|
||||
import platform
|
||||
print(platform.mac_ver()[0])
|
||||
`], { timeout: 5000 });
|
||||
results.python_mac_ver = stdout.trim();
|
||||
} catch (e) {
|
||||
results.python_mac_ver_error = e.message;
|
||||
}
|
||||
|
||||
// Try to detect SpeechAnalyzer via file existence / Swift availability
|
||||
try {
|
||||
// On macOS 26, Speech.framework/Versions should have newer build
|
||||
const { stdout } = await execFileAsync("/bin/ls", ["-la", "/System/Library/Frameworks/Speech.framework/"], { timeout: 3000 });
|
||||
results.speech_framework_ls = stdout.trim().slice(0, 2000);
|
||||
} catch (e) {
|
||||
results.speech_framework_error = e.message;
|
||||
}
|
||||
|
||||
// Check hardware model
|
||||
try {
|
||||
const { stdout } = await execFileAsync("/usr/sbin/sysctl", ["-n", "hw.model"], { timeout: 2000 });
|
||||
results.hw_model = stdout.trim();
|
||||
} catch {}
|
||||
try {
|
||||
const { stdout } = await execFileAsync("/usr/sbin/sysctl", ["-n", "machdep.cpu.brand_string"], { timeout: 2000 });
|
||||
results.cpu_brand = stdout.trim();
|
||||
} catch {}
|
||||
|
||||
// Is this macOS 26+ ?
|
||||
const versionToCheck = results.macos_version || results.python_mac_ver || "";
|
||||
if (versionToCheck) {
|
||||
const major = parseInt(versionToCheck.split(".")[0], 10);
|
||||
results.is_macos_26_plus = major >= 26;
|
||||
results.speech_analyzer_expected = major >= 26 ? "likely available (macOS 26+)" : "not available - requires macOS 26+";
|
||||
} else if (results.python_mac_ver) {
|
||||
const major = parseInt(results.python_mac_ver.split(".")[0], 10);
|
||||
results.macos_version = results.python_mac_ver;
|
||||
results.is_macos_26_plus = major >= 26;
|
||||
results.speech_analyzer_expected = major >= 26 ? "likely available (macOS 26+)" : "not available";
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
export async function getSpeechApiStatus() {
|
||||
const sysInfo = await getSystemInfo();
|
||||
// Try to run a tiny Swift snippet that imports Speech and checks for SpeechAnalyzer
|
||||
// Fallback to reporting version info if swiftc not available
|
||||
let swiftCheck = null;
|
||||
try {
|
||||
// Write temp swift file that checks API availability
|
||||
const { stdout: swiftPath } = await execFileAsync("/usr/bin/which", ["swift"], { timeout: 2000 });
|
||||
const swiftBin = swiftPath.trim();
|
||||
if (swiftBin) {
|
||||
// Create a small swift program to test SpeechAnalyzer availability
|
||||
const swiftCode = `
|
||||
import Speech
|
||||
import Foundation
|
||||
#if canImport(Speech)
|
||||
if #available(macOS 26.0, *) {
|
||||
print("SpeechAnalyzer: available")
|
||||
// Try to reference the type
|
||||
let _ = SpeechTranscriber.self
|
||||
print("SpeechTranscriber: available")
|
||||
} else {
|
||||
print("SpeechAnalyzer: requires macOS 26")
|
||||
}
|
||||
#else
|
||||
print("Speech framework not importable")
|
||||
#endif
|
||||
`;
|
||||
const tmpFile = `/tmp/speech_check_${Date.now()}.swift`;
|
||||
const { writeFile, unlink } = await import("node:fs/promises");
|
||||
await writeFile(tmpFile, swiftCode, "utf8");
|
||||
try {
|
||||
const { stdout, stderr } = await execFileAsync(swiftBin, [tmpFile], { timeout: 15000, maxBuffer: 2 * 1024 * 1024 });
|
||||
swiftCheck = { stdout: stdout.trim(), stderr: stderr.trim(), ok: true };
|
||||
} catch (e) {
|
||||
swiftCheck = { stdout: e.stdout?.toString().trim() || "", stderr: (e.stderr?.toString() || e.message).trim().slice(0, 2000), ok: false };
|
||||
} finally {
|
||||
try { await unlink(tmpFile); } catch {}
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
swiftCheck = { error: e.message };
|
||||
}
|
||||
|
||||
return {
|
||||
system: sysInfo,
|
||||
swift_availability: swiftCheck,
|
||||
conclusion: sysInfo.is_macos_26_plus
|
||||
? "macOS 26+ detected — SpeechAnalyzer/SpeechTranscriber should be available per Apple docs."
|
||||
: `macOS ${sysInfo.macos_version || "unknown"} detected — SpeechAnalyzer requires macOS 26+. ${sysInfo.macos_version ? `You are on ${sysInfo.macos_version}, need to upgrade to 26.` : ""}`,
|
||||
};
|
||||
}
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
import { mkdir, stat, readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
|
||||
const VOICEBOX_URL = process.env.VOICEBOX_URL || "http://127.0.0.1:17493";
|
||||
const OUTPUT_BASE = path.join(os.homedir(), "Projects", "MacMiniMCP", "generated-audio");
|
||||
|
||||
const KNOWN_PROFILES = {
|
||||
Aiden: "ff624ec6-5485-4173-a4f0-2ec2196efd39",
|
||||
Adolfo: "0e042c6b-ae52-4f28-835b-528381ed60b4",
|
||||
Nicole: "52330098-6fc3-4e9c-a30c-11164869636e",
|
||||
Jessica: "579c7444-3905-4aab-8067-eb10a0b3e76f",
|
||||
Dora: "1862f224-fd47-4791-9f37-76f76ca0450c",
|
||||
Alex: "a0cf179a-c033-47e9-92b5-f61596f68adc",
|
||||
};
|
||||
|
||||
async function fetchJSON(url, opts = {}) {
|
||||
const res = await fetch(url, opts);
|
||||
const txt = await res.text();
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}: ${txt.slice(0,500)}`);
|
||||
try { return JSON.parse(txt); } catch { return txt; }
|
||||
}
|
||||
|
||||
export async function voiceboxListProfiles() {
|
||||
try {
|
||||
const profiles = await fetchJSON(`${VOICEBOX_URL}/profiles`);
|
||||
const enriched = profiles.map(p => ({
|
||||
id: p.id,
|
||||
name: p.name,
|
||||
engine: p.default_engine,
|
||||
voice_type: p.voice_type,
|
||||
preset_voice_id: p.preset_voice_id,
|
||||
sample_count: p.sample_count,
|
||||
generation_count: p.generation_count,
|
||||
}));
|
||||
return { ok: true, url: VOICEBOX_URL, count: enriched.length, profiles: enriched, defaultBoyVoice: "Aiden", defaultGirlVoice: "Jessica", mapping: KNOWN_PROFILES };
|
||||
} catch (e) {
|
||||
return { ok: false, error: e.message, url: VOICEBOX_URL };
|
||||
}
|
||||
}
|
||||
|
||||
export async function voiceboxHealth() {
|
||||
try {
|
||||
const health = await fetchJSON(`${VOICEBOX_URL}/health`);
|
||||
return { ok: true, ...health, url: VOICEBOX_URL };
|
||||
} catch (e) {
|
||||
return { ok: false, error: e.message, url: VOICEBOX_URL };
|
||||
}
|
||||
}
|
||||
|
||||
async function pollGeneration(id, timeoutMs = 20000) {
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
try {
|
||||
const gen = await fetchJSON(`${VOICEBOX_URL}/history/${id}`);
|
||||
if (gen.status === "completed" && gen.audio_path) return gen;
|
||||
if (gen.status === "failed") throw new Error(gen.error || "generation failed");
|
||||
} catch (e) {
|
||||
if (e.message && e.message.toLowerCase().includes("failed")) throw e;
|
||||
}
|
||||
await new Promise(r => setTimeout(r, 500));
|
||||
}
|
||||
throw new Error("poll timeout");
|
||||
}
|
||||
|
||||
export async function voiceboxGenerate({ text, profile, engine, language, voice }) {
|
||||
if (!text) throw new Error("text required");
|
||||
let profileId = profile || voice || "Aiden";
|
||||
if (KNOWN_PROFILES[profileId]) profileId = KNOWN_PROFILES[profileId];
|
||||
if (!profileId.includes("-")) {
|
||||
try {
|
||||
const profiles = await fetchJSON(`${VOICEBOX_URL}/profiles`);
|
||||
const match = profiles.find(p => p.name.toLowerCase() === profileId.toLowerCase());
|
||||
if (match) profileId = match.id;
|
||||
} catch {}
|
||||
}
|
||||
let autoEngine = engine;
|
||||
if (!autoEngine && ["ff624ec6-5485-4173-a4f0-2ec2196efd39", "4d0ded93-3b12-465f-aeb4-aa4360f3dc5c", "d6c2e90f-ec01-4f5e-8efc-7822bd79ac56"].includes(profileId)) {
|
||||
autoEngine = "qwen_custom_voice";
|
||||
}
|
||||
const body = {
|
||||
profile_id: profileId,
|
||||
text: String(text).slice(0, 1000),
|
||||
language: language || "en",
|
||||
engine: autoEngine || undefined,
|
||||
};
|
||||
Object.keys(body).forEach(k => body[k] === undefined && delete body[k]);
|
||||
try {
|
||||
const gen = await fetchJSON(`${VOICEBOX_URL}/generate`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
let finalGen = gen;
|
||||
if (gen.status === "generating" || !gen.audio_path) {
|
||||
try {
|
||||
finalGen = await pollGeneration(gen.id, 20000);
|
||||
} catch (pollErr) {
|
||||
return { ok: false, id: gen.id, status: gen.status, error: pollErr.message, url: VOICEBOX_URL, polling: true, profile_id: profileId, text: body.text };
|
||||
}
|
||||
}
|
||||
return {
|
||||
ok: true,
|
||||
id: finalGen.id,
|
||||
profile_id: finalGen.profile_id,
|
||||
text: finalGen.text,
|
||||
audio_path: finalGen.audio_path,
|
||||
duration: finalGen.duration,
|
||||
engine: finalGen.engine,
|
||||
status: finalGen.status,
|
||||
url: VOICEBOX_URL,
|
||||
};
|
||||
} catch (e) {
|
||||
throw new Error(`voicebox generate failed: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function voiceboxGenerateBase64({ text, profile, voice, engine, language }) {
|
||||
const gen = await voiceboxGenerate({ text, profile, voice, engine, language });
|
||||
if (!gen.ok || !gen.audio_path) return { ...gen, ok: false, error: gen.error || `no audio_path, status ${gen.status}`, fileExists: false };
|
||||
|
||||
// audio_path may be relative "generations/xxx.wav" or absolute
|
||||
let filePath = gen.audio_path;
|
||||
let candidates = [filePath];
|
||||
if (!filePath.startsWith("/") && !filePath.startsWith("~")) {
|
||||
candidates.push(path.join(os.homedir(), "Library", "Application Support", "sh.voicebox.app", filePath));
|
||||
candidates.push(path.join(os.homedir(), "Library", "Application Support", "sh.voicebox.app", "generations", path.basename(filePath)));
|
||||
}
|
||||
let foundPath = null;
|
||||
for (const cand of candidates) {
|
||||
try { await stat(cand); foundPath = cand; break; } catch {}
|
||||
}
|
||||
if (!foundPath) {
|
||||
try { await stat(filePath); foundPath = filePath; } catch {}
|
||||
}
|
||||
if (!foundPath) {
|
||||
// try glob latest file matching id
|
||||
return { ...gen, ok: false, error: `audio_path not found on disk: ${filePath}, tried ${candidates.join(",")}`, fileExists: false };
|
||||
}
|
||||
filePath = foundPath;
|
||||
try {
|
||||
const buf = await readFile(filePath);
|
||||
const b64 = buf.toString("base64");
|
||||
return {
|
||||
ok: true,
|
||||
id: gen.id,
|
||||
profile_id: gen.profile_id,
|
||||
text: gen.text,
|
||||
filePath,
|
||||
duration: gen.duration,
|
||||
engine: gen.engine,
|
||||
wavBase64: b64,
|
||||
base64Length: b64.length,
|
||||
size: buf.length,
|
||||
format: filePath.endsWith(".wav") ? "wav" : "mp3",
|
||||
url: VOICEBOX_URL,
|
||||
profile: profile || voice || "Aiden",
|
||||
};
|
||||
} catch (e) {
|
||||
return { ...gen, ok: false, error: `read failed: ${e.message}` };
|
||||
}
|
||||
}
|
||||
|
||||
export async function voiceboxQuickReply({ text, voice, profile }) {
|
||||
const chosenProfile = profile || voice || "Aiden";
|
||||
const result = await voiceboxGenerateBase64({ text, profile: chosenProfile, language: "en", engine: undefined });
|
||||
return result;
|
||||
}
|
||||
@@ -0,0 +1,620 @@
|
||||
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
FOCUS_CALENDAR,
|
||||
createEvent,
|
||||
listCalendars,
|
||||
listEvents,
|
||||
} from "./integrations/calendar.js";
|
||||
import {
|
||||
createContact,
|
||||
readContact,
|
||||
searchContacts,
|
||||
} from "./integrations/contacts.js";
|
||||
import {
|
||||
generateCodexImage,
|
||||
getCodexImageConfigStatus,
|
||||
} from "./integrations/codex-image.js";
|
||||
import { getDecoConfigStatus, getDecoStats } from "./integrations/deco.js";
|
||||
import {
|
||||
buildGeminiChromePrompt,
|
||||
getGeminiChromePromptConfigStatus,
|
||||
} from "./integrations/gemini-chrome-prompt.js";
|
||||
import {
|
||||
generateGeminiImage,
|
||||
getGeminiImageConfigStatus,
|
||||
} from "./integrations/gemini-image.js";
|
||||
import { createNote, listNotes, readNote } from "./integrations/notes.js";
|
||||
import {
|
||||
listMailAccounts,
|
||||
listMailboxes,
|
||||
listMailMessages,
|
||||
readMailMessage,
|
||||
} from "./integrations/mail.js";
|
||||
import {
|
||||
createReminder,
|
||||
listReminderLists,
|
||||
listReminders,
|
||||
} from "./integrations/reminders.js";
|
||||
import { getSystemInfo, getSpeechApiStatus } from "./integrations/system.js";
|
||||
import {
|
||||
speechListVoices,
|
||||
speechSynthesize,
|
||||
speechSynthesizeBase64,
|
||||
} from "./integrations/speech-tts.js";
|
||||
import {
|
||||
speechKokoroStatus,
|
||||
speechKokoroSynthesize,
|
||||
speechKokoroSynthesizeBase64,
|
||||
} from "./integrations/kokoro-tts.js";
|
||||
import {
|
||||
voiceboxListProfiles,
|
||||
voiceboxHealth,
|
||||
voiceboxGenerate,
|
||||
voiceboxGenerateBase64,
|
||||
voiceboxQuickReply,
|
||||
} from "./integrations/voicebox.js";
|
||||
import {
|
||||
speechTranscribeFile,
|
||||
speechListLocales,
|
||||
speechQuickTest,
|
||||
} from "./integrations/speech-transcribe.js";
|
||||
import {
|
||||
speechLiveTranscribe,
|
||||
speechLiveClose,
|
||||
speechLiveStatus,
|
||||
} from "./integrations/speech-live.js";
|
||||
import {
|
||||
appleLLMCheck,
|
||||
appleLLMPolish,
|
||||
appleLLMQuickReply,
|
||||
appleLLMChat,
|
||||
appleLLMClose,
|
||||
appleLLMStatus,
|
||||
} from "./integrations/apple-llm.js";
|
||||
|
||||
const jsonResult = (value) => ({
|
||||
content: [{ type: "text", text: JSON.stringify(value, null, 2) }],
|
||||
});
|
||||
|
||||
const handled = (operation) => async (input) => {
|
||||
try {
|
||||
return jsonResult(await operation(input));
|
||||
} catch (error) {
|
||||
return {
|
||||
isError: true,
|
||||
content: [{ type: "text", text: error.message }],
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
function withFocusedCalendar(input) {
|
||||
if (input.calendarIndex !== undefined) {
|
||||
return { ...input, calendarIndex: input.calendarIndex };
|
||||
}
|
||||
if (input.calendar !== undefined) {
|
||||
return { ...input, calendarIndex: null };
|
||||
}
|
||||
return { ...input, ...FOCUS_CALENDAR };
|
||||
}
|
||||
|
||||
export function createMacMiniMcpServer() {
|
||||
const server = new McpServer({
|
||||
name: "macmini-mcp",
|
||||
version: "0.1.0",
|
||||
});
|
||||
|
||||
server.tool(
|
||||
"notes_list",
|
||||
"Find notes by optional text or folder filter. Titles and metadata are returned by default; request previews explicitly.",
|
||||
{
|
||||
query: z.string().optional().describe("Text to search in note title and plaintext body."),
|
||||
folder: z.string().optional().describe("Exact Notes folder name."),
|
||||
includePreview: z.boolean().default(false).describe("Include plaintext previews only when needed; note content can be sensitive."),
|
||||
limit: z.number().int().positive().max(100).default(20),
|
||||
},
|
||||
handled(listNotes),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"notes_read",
|
||||
"Read one Apple Note after locating its ID with notes_list.",
|
||||
{ id: z.string().min(1).describe("Notes unique ID.") },
|
||||
handled(({ id }) => readNote(id)),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"notes_create",
|
||||
"Create a new Apple Note from plaintext content.",
|
||||
{
|
||||
title: z.string().min(1).max(300),
|
||||
body: z.string().default(""),
|
||||
folder: z.string().optional().describe("Exact destination folder; defaults to the first available folder."),
|
||||
},
|
||||
handled(createNote),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"mail_accounts",
|
||||
"List configured Apple Mail accounts and their email addresses. Read-only.",
|
||||
{},
|
||||
handled(() => listMailAccounts()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"mail_list_mailboxes",
|
||||
"List mailboxes for one selected Apple Mail account. Call mail_accounts first. Read-only.",
|
||||
{
|
||||
accountId: z.string().min(1).describe("Account ID returned by mail_accounts."),
|
||||
},
|
||||
handled(listMailboxes),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"mail_list_messages",
|
||||
"List message metadata in one selected Apple Mail account and mailbox. Read-only; does not fetch message bodies.",
|
||||
{
|
||||
accountId: z.string().min(1).describe("Account ID returned by mail_accounts."),
|
||||
mailbox: z.string().min(1).describe("Exact mailbox name returned by mail_list_mailboxes."),
|
||||
unreadOnly: z.boolean().default(false).describe("Return only unread messages."),
|
||||
limit: z.number().int().positive().max(50).default(10),
|
||||
},
|
||||
handled(listMailMessages),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"mail_read_message",
|
||||
"Read one Apple Mail message after selecting it with mail_list_messages. Read-only.",
|
||||
{
|
||||
accountId: z.string().min(1).describe("Account ID returned by mail_accounts."),
|
||||
mailbox: z.string().min(1).describe("Exact mailbox name containing the selected message."),
|
||||
id: z.string().min(1).describe("Message ID returned by mail_list_messages."),
|
||||
},
|
||||
handled(readMailMessage),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"calendar_list_calendars",
|
||||
"List macOS Calendar calendars with indexes and indicate which accept new events. The server is focused on the event-rich Home calendar by default.",
|
||||
{},
|
||||
handled(listCalendars),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"calendar_list_events",
|
||||
"List events in the focused Home calendar over a required ISO-8601 time range. Optional selectors override the focus calendar.",
|
||||
{
|
||||
start: z.string().describe("Range start as an ISO-8601 datetime."),
|
||||
end: z.string().describe("Range end as an ISO-8601 datetime."),
|
||||
calendar: z.string().optional().describe("Advanced override: exact calendar name."),
|
||||
calendarIndex: z.number().int().nonnegative().optional().describe("Advanced override: index returned by calendar_list_calendars."),
|
||||
limit: z.number().int().positive().max(250).default(50),
|
||||
},
|
||||
handled((input) => listEvents(withFocusedCalendar(input))),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"calendar_create_event",
|
||||
"Create an event in the focused Home calendar. Optional selectors override the focus calendar.",
|
||||
{
|
||||
calendar: z.string().optional().describe("Advanced override: exact destination calendar name."),
|
||||
calendarIndex: z.number().int().nonnegative().optional().describe("Advanced override: destination index."),
|
||||
title: z.string().min(1).max(500),
|
||||
start: z.string().describe("Event start as an ISO-8601 datetime."),
|
||||
end: z.string().describe("Event end as an ISO-8601 datetime."),
|
||||
allDay: z.boolean().default(false),
|
||||
notes: z.string().optional(),
|
||||
location: z.string().optional(),
|
||||
},
|
||||
handled((input) => createEvent(withFocusedCalendar(input))),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"reminders_list_lists",
|
||||
"List Reminders lists, including account context and assignment metadata availability for shared lists.",
|
||||
{},
|
||||
handled(listReminderLists),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"reminders_list",
|
||||
"List reminders with optional list and completion filters. Returns assignment details for shared-list reminders when available, plus title/notes-derived assignment hints.",
|
||||
{
|
||||
list: z.string().optional().describe("Exact Reminders list name."),
|
||||
completed: z.boolean().nullable().default(false).describe("Use null to return both complete and incomplete items."),
|
||||
limit: z.number().int().positive().max(100).default(25),
|
||||
},
|
||||
handled(listReminders),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"reminders_create",
|
||||
"Create a reminder in an optional named Reminders list.",
|
||||
{
|
||||
title: z.string().min(1).max(500),
|
||||
list: z.string().optional().describe("Exact destination list; defaults to the first available list."),
|
||||
notes: z.string().optional(),
|
||||
due: z.string().optional().describe("Optional due date as an ISO-8601 datetime."),
|
||||
},
|
||||
handled(createReminder),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"contacts_search",
|
||||
"Search Contacts by name or organization. Returns metadata only; use contacts_read for contact methods.",
|
||||
{
|
||||
query: z.string().optional().describe("Text to search in display name or organization."),
|
||||
limit: z.number().int().positive().max(100).default(20),
|
||||
},
|
||||
handled(searchContacts),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"contacts_read",
|
||||
"Read contact details, including email addresses and phone numbers, after selecting an ID with contacts_search.",
|
||||
{
|
||||
id: z.string().min(1).describe("Persistent Contacts person ID."),
|
||||
},
|
||||
handled(({ id }) => readContact(id)),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"contacts_create",
|
||||
"Create a new contact with optional email address and phone number.",
|
||||
{
|
||||
firstName: z.string().max(200).optional(),
|
||||
lastName: z.string().max(200).optional(),
|
||||
organization: z.string().max(300).optional(),
|
||||
jobTitle: z.string().max(300).optional(),
|
||||
note: z.string().max(2000).optional(),
|
||||
email: z.object({
|
||||
label: z.string().max(100).default("work"),
|
||||
value: z.string().email(),
|
||||
}).optional(),
|
||||
phone: z.object({
|
||||
label: z.string().max(100).default("mobile"),
|
||||
value: z.string().min(1).max(100),
|
||||
}).optional(),
|
||||
},
|
||||
handled((input) => {
|
||||
if (!input.firstName && !input.lastName && !input.organization) {
|
||||
throw new Error("firstName, lastName, or organization is required.");
|
||||
}
|
||||
return createContact(input);
|
||||
}),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"deco_get_config_status",
|
||||
"Show TP-Link Deco connection configuration without revealing the password.",
|
||||
{},
|
||||
handled(getDecoConfigStatus),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"deco_get_overview",
|
||||
"Read TP-Link Deco overview stats: WAN/LAN addresses, CPU/memory usage, client counts, Wi-Fi enablement, and firmware.",
|
||||
{},
|
||||
handled(() => getDecoStats("overview")),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"deco_list_clients",
|
||||
"List online TP-Link Deco clients with hostname, IP, MAC, connection type, current up/down speeds, and linked mesh node when available.",
|
||||
{},
|
||||
handled(() => getDecoStats("clients")),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"deco_get_ipv4_status",
|
||||
"Read TP-Link Deco WAN/LAN IPv4 status including gateway, DNS, netmasks, and connection type.",
|
||||
{},
|
||||
handled(() => getDecoStats("ipv4")),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"deco_get_firmware",
|
||||
"Read TP-Link Deco model, hardware version, and firmware version.",
|
||||
{},
|
||||
handled(() => getDecoStats("firmware")),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"system_get_info",
|
||||
"Get Mac mini system info: macOS version (sw_vers), uname, hardware model, and whether macOS 26+ SpeechAnalyzer is available.",
|
||||
{},
|
||||
handled(() => getSystemInfo()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"system_speech_api_status",
|
||||
"Check if Apple's new SpeechAnalyzer / SpeechTranscriber (macOS 26+) is available — returns macOS version, build, and Swift availability probe.",
|
||||
{},
|
||||
handled(() => getSpeechApiStatus()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_list_locales",
|
||||
"List SpeechTranscriber locales: isAvailable, supported, installed, reserved, maxReserved. Use to check if your language model is ready before transcribing.",
|
||||
{},
|
||||
handled(() => speechListLocales()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_transcribe_file",
|
||||
"Transcribe an audio file using Apple's new SpeechAnalyzer + SpeechTranscriber (macOS 26+). Fastest and most accurate English engine on-device per Inscribe benchmark (2.12% WER). Supports m4a, wav, aiff, mp3, etc via AVFoundation. Returns transcript, segments, timing, realtime factor.",
|
||||
{
|
||||
filePath: z.string().min(1).describe("Absolute path to audio file on the Mac. Can be ~/path or /tmp/ etc."),
|
||||
locale: z.string().optional().describe("Locale like en-US, es-ES, etc. Defaults to en-US. Use speech_list_locales to see options."),
|
||||
},
|
||||
handled(({ filePath, locale }) => speechTranscribeFile({ filePath, locale })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_quick_test",
|
||||
"End-to-end loopback test: generates speech with macOS say command, then transcribes it with SpeechAnalyzer to verify the whole pipeline works. Returns both input text and transcript for comparison.",
|
||||
{
|
||||
text: z.string().optional().describe("Text to synthesize and transcribe. Default: hello world test."),
|
||||
voice: z.string().optional().describe("macOS say voice, e.g. Alex, Samantha. Default: Alex."),
|
||||
},
|
||||
handled(({ text, voice }) => speechQuickTest({ text, voice })),
|
||||
);
|
||||
|
||||
|
||||
server.tool(
|
||||
"speech_live_transcribe",
|
||||
"LIVE draft transcription via persistent Apple pipe (ApplePipeTranscriber macOS 26+). Reuses your whisper-translation pipe protocol: 4-byte BE len + WAV payload -> draft/final JSON streaming. Input: base64-encoded 16kHz mono WAV. Output: {text, drafts[], finals[], isFinal}. Keep session alive for <200ms incremental feedback while streaming ESP32 PCM. Auto-closes after 60s idle.",
|
||||
{
|
||||
audioBase64: z.string().min(100).describe("Base64-encoded WAV file (16kHz mono s16le). Use 0.5-3s chunks for draft streaming."),
|
||||
locale: z.string().optional().describe("Locale like en-US. Default en-US."),
|
||||
},
|
||||
handled(({ audioBase64, locale }) => speechLiveTranscribe({ audioBase64, locale })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_live_close",
|
||||
"Close a persistent live transcription pipe session (frees Swift process).",
|
||||
{
|
||||
locale: z.string().optional().describe("Locale to close. Default en-US."),
|
||||
},
|
||||
handled(({ locale }) => speechLiveClose({ locale })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_live_status",
|
||||
"Show active live pipe sessions: locale, pid, chunkIdx, lastActivity.",
|
||||
{},
|
||||
handled(() => speechLiveStatus()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"codex_image_get_config_status",
|
||||
"Show local Codex CLI image generation configuration.",
|
||||
{},
|
||||
handled(getCodexImageConfigStatus),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"codex_image_generate",
|
||||
"Generate an image using this Mac's Codex CLI image generation and save it to the MacMiniMCP generated-images directory.",
|
||||
{
|
||||
prompt: z.string().min(1).max(8000).describe("Text prompt describing the image to generate."),
|
||||
filename: z.string().min(1).max(200).optional().describe("Optional local filename. Directory components are ignored."),
|
||||
size: z.string().max(100).optional().describe("Optional size or aspect request, such as 1024x1024, 16:9, or square."),
|
||||
quality: z.string().max(100).optional().describe("Optional quality request, such as draft, standard, or high."),
|
||||
style: z.string().max(500).optional().describe("Optional visual style guidance."),
|
||||
referenceImage: z.string().optional().describe("Optional absolute path to a local reference image for Codex CLI --image."),
|
||||
},
|
||||
handled(generateCodexImage),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"gemini_image_get_config_status",
|
||||
"Show Gemini image generation configuration without revealing the API key.",
|
||||
{},
|
||||
handled(getGeminiImageConfigStatus),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"gemini_image_generate",
|
||||
"Generate one image with the Gemini API and save it to the local generated-images directory. This tool does not expose browser control.",
|
||||
{
|
||||
prompt: z.string().min(1).max(8000).describe("Text prompt describing the image to generate."),
|
||||
filename: z.string().min(1).max(200).optional().describe("Optional local PNG filename. Directory components are ignored."),
|
||||
model: z.string().min(1).max(100).default("gemini-3.1-flash-image"),
|
||||
aspectRatio: z.enum(["1:1", "3:4", "4:3", "9:16", "16:9"]).optional(),
|
||||
imageSize: z.enum(["1K", "2K", "4K"]).optional(),
|
||||
useGoogleSearch: z.boolean().default(false).describe("Allow Gemini to use Google Search for prompts that need current real-world context."),
|
||||
},
|
||||
handled(generateGeminiImage),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"gemini_chrome_prompt_get_config_status",
|
||||
"Show configuration for the Codex Chrome-skill Gemini image prompt builder.",
|
||||
{},
|
||||
handled(getGeminiChromePromptConfigStatus),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"gemini_chrome_prompt_build",
|
||||
"Build a ready-to-run Codex prompt for generating one Gemini web-app image through the verified ReynaFamilyBot Chrome profile. This MCP tool does not control Chrome.",
|
||||
{
|
||||
prompt: z.string().min(1).max(8000).describe("Text prompt describing the image to generate in Gemini."),
|
||||
filename: z.string().min(1).max(200).optional().describe("Optional local PNG filename. Directory components are ignored."),
|
||||
},
|
||||
handled(buildGeminiChromePrompt),
|
||||
);
|
||||
|
||||
|
||||
server.tool(
|
||||
"apple_llm_check",
|
||||
"Check if Apple on-device 3B LLM (FoundationModels SystemLanguageModel) is available. ANE-accelerated, offline, ~0.6s vs Ollama 2-3s. Required for polish/quick_reply/chat.",
|
||||
{},
|
||||
handled(() => appleLLMCheck()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"apple_llm_polish",
|
||||
"Polish a caption line/paragraph using Apple on-device 3B LLM ANE. From whisper-translation engine_apple_llm.py: same as AppleLLMPolish binary pipe. Fixes punctuation, casing, typos from live transcript. Mode: line (default) or paragraph. Reuses single warm session.",
|
||||
{
|
||||
text: z.string().min(1).describe("Text to polish (STT draft or final)."),
|
||||
prev1: z.string().optional().describe("Previous line for context (line mode)."),
|
||||
prev2: z.string().optional().describe("Second previous line."),
|
||||
mode: z.enum(["line", "paragraph"]).optional().describe("line (default) or paragraph."),
|
||||
},
|
||||
handled(({ text, prev1, prev2, mode }) => appleLLMPolish({ text, prev1, prev2, mode })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"apple_llm_quick_reply",
|
||||
"INSTANT smart reply from live draft — for voice gateway. Feed the volatile draft transcript from speech_live_transcribe while user is speaking. Apple 3B ANE returns <20-word ACK preview in ~0.6s, before final transcription. Use this to provide almost immediate response when voice stops: send draft to quick_reply, get instant text to speak/display, then full Hermes turn in background. Reuses same ANE session as polish.",
|
||||
{
|
||||
draft: z.string().min(1).describe("Live draft transcript (partial, may have typos) from speech_live_transcribe. Used to infer intent for instant preview reply."),
|
||||
context: z.string().optional().describe("Optional previous conversation context or last assistant reply."),
|
||||
instructions: z.string().optional().describe("Optional system instructions for quick reply persona."),
|
||||
},
|
||||
handled(({ draft, context, instructions }) => appleLLMQuickReply({ draft, context, instructions })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"apple_llm_chat",
|
||||
"Fast voice-assistant chat via Apple 3B ANE (FoundationModels). Under 40 words, warm, kid-safe, for ESP32. Use when you want Mac mini to directly answer draft/final without calling Hermes gateway. Accepts history array and current text. ~0.6-1.2s.",
|
||||
{
|
||||
text: z.string().min(1).describe("User message (draft or final transcript)."),
|
||||
history: z.array(z.object({ role: z.string(), text: z.string() })).optional().describe("Optional conversation history [{role, text}]."),
|
||||
instructions: z.string().optional().describe("Optional custom instructions."),
|
||||
},
|
||||
handled(({ text, history, instructions }) => appleLLMChat({ text, history, instructions })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"apple_llm_status",
|
||||
"Show Apple LLM session status: active, ready, pid, lastActivity.",
|
||||
{},
|
||||
handled(() => appleLLMStatus()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"apple_llm_close",
|
||||
"Close Apple LLM ANE session (frees Swift process + KV cache). Auto-closes after 2min idle anyway.",
|
||||
{},
|
||||
handled(() => appleLLMClose()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_list_voices",
|
||||
"List available macOS say voices with locale tags. Use for TTS voice selection.",
|
||||
{},
|
||||
handled(() => speechListVoices()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_synthesize",
|
||||
"Synthesize text to speech using macOS say command (native Apple Neural voices). Generates AIFF and 16k wav for iPhone playback. Returns file paths.",
|
||||
{
|
||||
text: z.string().min(1).max(5000).describe("Text to speak (max 5000 chars)"),
|
||||
voice: z.string().optional().describe("Voice name like Alex, Samantha, Ava, etc. Use speech_list_voices to see options"),
|
||||
rate: z.number().int().min(80).max(500).optional().describe("Speech rate in wpm, 80-500, default system"),
|
||||
},
|
||||
handled(({ text, voice, rate }) => speechSynthesize({ text, voice, rate })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_synthesize_base64",
|
||||
"Synthesize text to 16k mono WAV base64 using macOS say + afconvert. Ideal for sending to iPhone play_audio_base64. Returns full base64 payload.",
|
||||
{
|
||||
text: z.string().min(1).max(5000).describe("Text to speak"),
|
||||
voice: z.string().optional().describe("Voice name"),
|
||||
rate: z.number().int().min(80).max(500).optional(),
|
||||
},
|
||||
handled(({ text, voice, rate }) => speechSynthesizeBase64({ text, voice, rate })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_kokoro_status",
|
||||
"Check the warm Kokoro ksay TTS daemon status: loaded model, default voice, default language, and load timing.",
|
||||
{},
|
||||
handled(() => speechKokoroStatus()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_kokoro_synthesize",
|
||||
"Fast neural TTS using the warm mlx-audio Kokoro ksay daemon. Generates a 24kHz WAV and returns the local file path. Use this for low-latency local speech instead of macOS say.",
|
||||
{
|
||||
text: z.string().min(1).max(8000).describe("Text to speak."),
|
||||
voice: z.string().optional().describe("Kokoro voice preset, e.g. af_heart, af_bella, af_nova, am_adam, bf_alice. Default af_heart."),
|
||||
speed: z.number().min(0.5).max(2.0).optional().describe("Kokoro speed multiplier. Default 1.0."),
|
||||
langCode: z.string().max(8).optional().describe("Kokoro language code: a American English, b British English, j Japanese, z Mandarin, etc. Default a."),
|
||||
outputPath: z.string().optional().describe("Optional absolute output WAV path. Defaults to generated-audio/ksay-*.wav."),
|
||||
},
|
||||
handled(({ text, voice, speed, langCode, outputPath }) => speechKokoroSynthesize({ text, voice, speed, langCode, outputPath })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"speech_kokoro_synthesize_base64",
|
||||
"Fast neural TTS using the warm mlx-audio Kokoro ksay daemon. Returns full 24kHz WAV base64 for clients that need immediate audio bytes.",
|
||||
{
|
||||
text: z.string().min(1).max(3000).describe("Text to speak. Keep short for fast playback and smaller MCP payloads."),
|
||||
voice: z.string().optional().describe("Kokoro voice preset. Default af_heart."),
|
||||
speed: z.number().min(0.5).max(2.0).optional().describe("Kokoro speed multiplier. Default 1.0."),
|
||||
langCode: z.string().max(8).optional().describe("Kokoro language code. Default a."),
|
||||
},
|
||||
handled(({ text, voice, speed, langCode }) => speechKokoroSynthesizeBase64({ text, voice, speed, langCode })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"voicebox_list_profiles",
|
||||
"List Voicebox voice profiles on Mac mini (Qwen3-TTS 1.7B MPS). Includes Aiden boy voice, Jessica/Nicole girl, Adolfo cloned. Use for TTS.",
|
||||
{},
|
||||
handled(() => voiceboxListProfiles()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"voicebox_health",
|
||||
"Check Voicebox TTS health: model_loaded, gpu_available (MPS), backend mlx, version. Port 17493 Qwen3 1.7B.",
|
||||
{},
|
||||
handled(() => voiceboxHealth()),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"voicebox_generate",
|
||||
"Generate speech via Voicebox on Mac mini: text + profile (Aiden boy voice default, Jessica/Nicole girl, Adolfo cloned). Qwen3-TTS 1.7B MPS GPU. Returns audio_path, duration, engine. Use voicebox_generate_base64 for base64 audio to stream to iPhone/WiFi.",
|
||||
{
|
||||
text: z.string().min(1).max(1000).describe("Text to synthesize (max 1000 chars)."),
|
||||
profile: z.string().optional().describe("Profile name or id: Aiden (boy voice default), Adolfo (cloned), Jessica/Nicole girl, Dora, Alex. Default Aiden."),
|
||||
voice: z.string().optional().describe("Alias for profile — same as profile."),
|
||||
engine: z.string().optional().describe("Optional engine override: qwen, qwen_custom_voice, kokoro, etc."),
|
||||
language: z.string().optional().describe("Language code: en (default), es, ko, zh, etc."),
|
||||
},
|
||||
handled(({ text, profile, voice, engine, language }) => voiceboxGenerate({ text, profile, voice, engine, language })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"voicebox_generate_base64",
|
||||
"Generate speech via Voicebox and return base64 WAV audio for immediate playback on iPhone ESP32. Uses Mac mini Qwen3-TTS 1.7B boy voice Aiden by default. Ideal for instant fast reply audio: generate contextual reply with apple_llm_quick_reply, then speak it with this tool, then play_audio_base64 on ESP32 screen. Returns wavBase64 ready for play_audio_base64 MCP.",
|
||||
{
|
||||
text: z.string().min(1).max(500).describe("Text to speak (max 500 chars for fast instant reply, keep short)."),
|
||||
profile: z.string().optional().describe("Profile: Aiden (boy default for instant), Jessica/Nicole girl, Adolfo cloned. Default Aiden."),
|
||||
voice: z.string().optional().describe("Alias for profile"),
|
||||
engine: z.string().optional().describe("Engine override"),
|
||||
language: z.string().optional().describe("Language, default en"),
|
||||
},
|
||||
handled(({ text, profile, voice, engine, language }) => voiceboxGenerateBase64({ text, profile, voice, engine, language })),
|
||||
);
|
||||
|
||||
server.tool(
|
||||
"voicebox_quick_reply",
|
||||
"ONE-CALL fast instant reply audio: text -> Voicebox boy voice Aiden base64 wav. Combines quick text already generated. For voice gateway: after apple_llm_quick_reply gives text like Got it! Switching to boy voice, immediately call this with that text to get audio base64 to play on iPhone while full Hermes answer generates. Boy voice Aiden default as requested.",
|
||||
{
|
||||
text: z.string().min(1).max(300).describe("Instant reply text from apple_llm_quick_reply (max 300 chars)."),
|
||||
profile: z.string().optional().describe("Profile, default Aiden boy voice as requested for boys voice."),
|
||||
voice: z.string().optional().describe("Alias for profile"),
|
||||
},
|
||||
handled(({ text, profile, voice }) => voiceboxQuickReply({ text, voice, profile })),
|
||||
);
|
||||
|
||||
|
||||
|
||||
return server;
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
|
||||
import { createMacMiniMcpServer } from "./server.js";
|
||||
|
||||
const server = createMacMiniMcpServer();
|
||||
const transport = new StdioServerTransport();
|
||||
|
||||
await server.connect(transport);
|
||||
@@ -0,0 +1,15 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { dateFromInput, plainTextToNoteHtml } from "../src/apple-events.js";
|
||||
|
||||
test("plainTextToNoteHtml escapes input and preserves line breaks", () => {
|
||||
assert.equal(
|
||||
plainTextToNoteHtml("R&D <today>", 'First\n"second"'),
|
||||
"<h1>R&D <today></h1><div>First<br>"second"</div>",
|
||||
);
|
||||
});
|
||||
|
||||
test("dateFromInput accepts valid datetimes and rejects invalid input", () => {
|
||||
assert.equal(dateFromInput("2026-05-26T10:00:00-04:00", "start").toISOString(), "2026-05-26T14:00:00.000Z");
|
||||
assert.throws(() => dateFromInput("not-a-date", "start"), /valid ISO-8601/);
|
||||
});
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import {
|
||||
listMailAccounts,
|
||||
listMailboxes,
|
||||
listMailMessages,
|
||||
readMailMessage,
|
||||
} from "../src/integrations/mail.js";
|
||||
|
||||
const enabled = process.env.RUN_APPLE_INTEGRATION_TESTS === "1";
|
||||
|
||||
test("Apple Mail read-only integration: enumerate accounts and read a selected inbox message", { skip: !enabled }, async () => {
|
||||
const accounts = await listMailAccounts();
|
||||
assert.ok(accounts.length > 0, "Apple Mail must have at least one configured account");
|
||||
|
||||
const mailboxes = await listMailboxes({ accountId: accounts[0].id });
|
||||
const inbox = mailboxes.find((mailbox) => mailbox.role === "inbox");
|
||||
assert.ok(inbox, "Apple Mail must expose a global inbox");
|
||||
|
||||
let accountWithMessage = null;
|
||||
for (const account of accounts) {
|
||||
const messages = await listMailMessages({ accountId: account.id, mailbox: inbox.name, limit: 1 });
|
||||
if (messages.length) {
|
||||
accountWithMessage = account;
|
||||
break;
|
||||
}
|
||||
}
|
||||
assert.ok(accountWithMessage, "At least one configured account must contain an inbox message");
|
||||
|
||||
const messages = await listMailMessages({ accountId: accountWithMessage.id, mailbox: inbox.name, limit: 1 });
|
||||
assert.equal(messages.length, 1);
|
||||
const message = await readMailMessage({ accountId: accountWithMessage.id, mailbox: inbox.name, id: messages[0].id });
|
||||
assert.equal(message.id, messages[0].id);
|
||||
assert.equal(message.accountId, accountWithMessage.id);
|
||||
assert.equal(typeof message.body, "string");
|
||||
});
|
||||
@@ -0,0 +1,81 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { createMailClient } from "../src/integrations/mail.js";
|
||||
|
||||
function fakeRunJxa(result) {
|
||||
const calls = [];
|
||||
return {
|
||||
calls,
|
||||
client: createMailClient(async (script, input) => {
|
||||
calls.push({ script, input });
|
||||
return result;
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
test("mail_accounts returns normalized account identities without message data", async () => {
|
||||
const { client, calls } = fakeRunJxa([
|
||||
{ id: "acct-personal", name: "Personal", emailAddresses: ["me@example.com"] },
|
||||
{ id: "acct-emi", name: "EMI", emailAddresses: ["info@emmint.com"] },
|
||||
]);
|
||||
|
||||
const result = await client.accounts();
|
||||
|
||||
assert.deepEqual(result, [
|
||||
{ id: "acct-personal", name: "Personal", emailAddresses: ["me@example.com"] },
|
||||
{ id: "acct-emi", name: "EMI", emailAddresses: ["info@emmint.com"] },
|
||||
]);
|
||||
assert.equal(calls.length, 1);
|
||||
assert.deepEqual(calls[0].input, {});
|
||||
});
|
||||
|
||||
test("mail_list_messages scopes the request to an account mailbox and bounded limit", async () => {
|
||||
const { client, calls } = fakeRunJxa([
|
||||
{
|
||||
id: "message-1",
|
||||
accountId: "acct-emi",
|
||||
account: "EMI",
|
||||
mailbox: "INBOX",
|
||||
subject: "Board update",
|
||||
sender: "Board <board@example.org>",
|
||||
dateSent: "2026-08-03T12:00:00.000Z",
|
||||
read: false,
|
||||
},
|
||||
]);
|
||||
|
||||
const result = await client.listMessages({ accountId: "acct-emi", mailbox: "INBOX", limit: 5, unreadOnly: true });
|
||||
|
||||
assert.equal(result.length, 1);
|
||||
assert.equal(result[0].subject, "Board update");
|
||||
assert.deepEqual(calls[0].input, { accountId: "acct-emi", mailbox: "INBOX", limit: 5, unreadOnly: true });
|
||||
assert.match(calls[0].script, /input\.limit/);
|
||||
});
|
||||
|
||||
test("mail_read_message requires the selected message ID and never returns data from another message", async () => {
|
||||
const { client, calls } = fakeRunJxa({
|
||||
id: "message-1",
|
||||
accountId: "acct-personal",
|
||||
mailbox: "INBOX",
|
||||
subject: "Receipt",
|
||||
sender: "Store <sales@example.org>",
|
||||
dateSent: "2026-08-03T12:00:00.000Z",
|
||||
read: true,
|
||||
body: "Thanks for your order.",
|
||||
});
|
||||
|
||||
const result = await client.readMessage({ accountId: "acct-personal", mailbox: "INBOX", id: "message-1" });
|
||||
|
||||
assert.equal(result.id, "message-1");
|
||||
assert.equal(result.body, "Thanks for your order.");
|
||||
assert.deepEqual(calls[0].input, { accountId: "acct-personal", mailbox: "INBOX", id: "message-1" });
|
||||
});
|
||||
|
||||
test("mail_list_messages rejects out-of-range limits before asking Mail", async () => {
|
||||
const { client, calls } = fakeRunJxa([]);
|
||||
|
||||
assert.throws(
|
||||
() => client.listMessages({ accountId: "acct-emi", mailbox: "INBOX", limit: 51, unreadOnly: false }),
|
||||
/limit must be between 1 and 50/,
|
||||
);
|
||||
assert.equal(calls.length, 0);
|
||||
});
|
||||
Reference in New Issue
Block a user