feat: Apple on-device LLM (ANE) polish replaces Ollama as primary

- apple-llm-polish binary (120KB Swift, FoundationModels): --check
  now reports available:true after Apple Intelligence enabled,
  ping OK, line polish 258-388ms ANE vs Ollama 3-8s CPU
- engine_apple_llm.py: AppleLLM class with persistent pipe,
  thread reader, polish_line() + polish_paragraph(), JSONL protocol
  {id,mode,text,prev1,prev2,context} -> {id,text,ok,ms}
- engine_llm.py: try Apple ANE first (permissiveContentTransformations,
  temp 0.1 line / 0.2 para), log provider=apple apple_ms, fallback
  to Ollama on guardrail/timeout. Fixes missing LLM log — now
  logs provider=apple with ms.
- main_v3.py: --apple-llm / --no-apple-llm flag (on by default)
- Verified: direct polish 258ms, last log provider=apple ms=498,
  speech binary still 240KB with 31 word-by-word drafts

Co-authored-by: internal-model

.gitignore

@@ -1,2 +1,25 @@
-__pycache__/\n*.pyc\n.DS_Store
-build/\ndist/\n*.spec
+__pycache__/
+*.pyc
+.DS_Store
+build/
+dist/
+*.spec
+venv/
+.env
+*.log
+apple_speech/.build/
+apple-speech-transcribe
+apple_speech/apple-speech-transcribe
+apple-llm-polish
+apple_speech/apple-llm-polish
+apple_speech/.build/release/apple-llm-polish
+logs/apple_bench.json
+logs/context*.json
+logs/prompt*.json
+logs/final*.json
+logs/paragraph*.json
+logs/llm_requests.jsonl
+prayer*.json
+test.json
+/tmp/*.wav
+/tmp/*.aiff

README.md

@@ -0,0 +1,140 @@
+# Python Whisper Live Transcription & Translation
+
+A real-time, low-latency audio transcription and translation tool utilizing OpenAI's Whisper (via `mlx-whisper` for Apple Silicon optimization), Silero VAD for speech detection, and Helsinki-NLP's Opus-MT models for translation.
+
+## Features
+- **Live Transcription:** Real-time speech-to-text with automatic language detection.
+- **On-the-fly Translation:** Bridge translations to English and then to target languages (Spanish, Arabic, French, etc.).
+- **Voice Activity Detection (VAD):** Intelligent audio buffering using Silero VAD to process only actual speech.
+- **Apple Silicon Optimized:** Uses MLX for high performance on Mac (MPS).
+- **Background Ingestion:** Optional background thread to send JSON payloads to a remote server.
+- **Configurable:** Command-line parameters to select languages, devices, and ingestion.
+
+## Installation
+
+1. **Clone the repository:**
+   ```bash
+   git clone <repository-url>
+   cd pythonwhisper
+   ```
+
+2. **Install dependencies:**
+   Ensure you have Python 3.9+ and the required libraries:
+   ```bash
+   pip install mlx-whisper numpy sounddevice torch requests transformers silero-vad
+   ```
+   *Note: On Apple Silicon, ensure `mlx` and `torch` with MPS support are correctly installed.*
+
+## Usage
+
+Run the script using `python3 transcribe.py` with optional flags.
+
+### LLM Prompt Testing
+You can now send the line-correction or paragraph-refinement prompts directly without starting live transcription. Every LLM request is appended as JSON Lines to `logs/llm_requests.jsonl` by default.
+
+- **Test the line-correction prompt:**
+  ```bash
+  python3 main_v2.py --post-correct-model qwen3.5:0.8b --llm-test-line "so um we should probably ship it tomorrow" --llm-test-prev1 "We finished the staging deploy." --llm-test-prev2 "QA signed off this morning."
+  ```
+- **Test the paragraph-refinement prompt:**
+  ```bash
+  python3 main_v2.py --post-correct-model qwen3.5:0.8b --llm-test-context "We reviewed the launch checklist." --llm-test-segments "we confirmed monitoring we confirmed rollback and then talked about the release window"
+  ```
+- **Change where requests are logged:**
+  ```bash
+  python3 main_v2.py --llm-request-log-path tmp/my_llm_requests.jsonl --llm-test-line "example text"
+  ```
+- **Replay the most recent logged request:**
+  ```bash
+  python3 main_v2.py --llm-test-from-log -1
+  ```
+- **Replay a logged request using the original model from the log entry:**
+  ```bash
+  python3 main_v2.py --llm-test-from-log 12 --llm-test-use-logged-model
+  ```
+
+Each log entry includes the timestamp, provider, model, mode, temperature, and full `messages` payload that was sent to the LLM.
+
+### Freeflow-style Local Polish
+This implementation includes a Python port of the portable parts of Freeflow's polish pipeline:
+
+- spoken punctuation commands such as `comma`, `question mark`, `new paragraph`, `at sign`, and `hashtag`
+- filler/noise stripping before any LLM call
+- protected `<keep>...</keep>` symbols so the LLM does not reinterpret dictated symbols
+- a clean-transcript skip heuristic for low-latency local runs
+- Freeflow-inspired English/minimal/Qwen line-polish prompts
+
+Run deterministic cleanup only:
+```bash
+python3 main_v2.py --post-correct
+```
+
+Run deterministic cleanup plus a local Ollama line-polish model when needed:
+```bash
+python3 main_v2.py --post-correct-llm --post-correct-model qwen3.5:0.8b --post-correct-prompt-style qwen
+```
+
+Force every line through the LLM, even if the deterministic output already looks clean:
+```bash
+python3 main_v2.py --post-correct-llm --no-post-correct-skip-clean
+```
+
+### Common Commands
+- **List available audio devices:**
+  ```bash
+  python3 transcribe.py -l
+  ```
+- **Caption system audio (speakers) using a loopback device:**
+  ```bash
+  python3 transcribe.py --loopback -es
+  ```
+- **Transcribe and translate to Spanish (screen only):**
+  ```bash
+  python3 transcribe.py -es
+  ```
+- **Enable Spanish, Arabic, and French with English bridging, and send to server:**
+  ```bash
+  python3 transcribe.py -es -ar -fr -en -i
+  ```
+- **Use a specific input device (e.g., index 3) and translate to Spanish:**
+  ```bash
+  python3 transcribe.py -d 3 -es
+  ```
+
+### Arguments
+- `-es`: Enable Spanish translation.
+- `-en`: Enable English (shows original if detected, or bridged if not).
+- `-ar`: Enable Arabic translation.
+- `-fr`: Enable French translation.
+- `-i`, `--ingest`: Enable data transmission to the remote server.
+- `-l`, `--list-devices`: Show available audio devices and exit.
+- `-d`, `--device [ID]`: Input device index (bypasses selection prompt).
+- `--loopback`: Automatically select a loopback device (e.g., BlackHole, Stereo Mix) to caption system audio.
+- `-q`, `--quantize`: Use 4-bit quantized Whisper model for faster transcription (Strategy 3).
+- `-s`, `--stream`: Enable real-time streaming transcription/draft mode (Strategy 1).
+- `-c`, `--context`: Enable prompt caching/rolling context to help the model maintain sentence continuity across chunks.
+- `--lang [CODE]`: Hardcode the source language (e.g., `en`, `es`) to bypass automatic language detection for faster processing.
+- `--silence [MS]`: Set the minimum silence duration in milliseconds to end a chunk. Defaults to 1000ms. Increase to force longer sentences before translation.
+- `--max-buffer [SEC]`: Maximum buffer duration in seconds before forcing a flush (default: 20).
+- `--channels [N]`: Number of input channels (default: 1).
+- `--pick-channel [0|1]`: If stereo, select channel 0 (Left) or 1 (Right) to focus transcription.
+- `--filter-lang`: If used with `--lang`, discards segments that do not match the target language.
+
+---
+
+## Technical Note: Universal Translation Models
+
+While the current implementation uses specialized, per-language models from the **Helsinki-NLP Opus-MT** project (e.g., `opus-mt-en-es`, `opus-mt-en-ar`), there is an alternative approach: **Universal Models**.
+
+### Universal Model Alternative (e.g., Meta's NLLB-200)
+The current per-language model approach is highly accurate and memory-efficient if you only need 1 or 2 target languages. However, if you require support for many languages simultaneously, loading multiple specialized models can consume significant RAM/VRAM.
+
+We have the option to switch the translation engine to a single, universal model such as **NLLB-200 (No Language Left Behind)**:
+- **Model ID:** `facebook/nllb-200-distilled-600M`
+- **Benefits:**
+    - Supports over **200 languages** in a single model.
+    - Simplified code: no need to load/manage multiple model objects.
+    - More efficient for complex multilingual environments.
+- **Trade-off:** Slightly higher memory footprint for the single model compared to a single specialized model, but more efficient than 3+ specialized models.
+
+If you wish to switch to a universal model, the `transcribe.py` logic can be updated to use a single `M2M100` or `NLLB` pipeline instead of the current `MarianMT` loop.

README_APPLE_V3.md

@@ -0,0 +1,105 @@
+# Apple Speech API v3 — Real-time transcription using macOS 26's new engine
+
+Based on [Inscribe's benchmark](https://get-inscribe.com/blog/apple-speech-api-benchmark.html): Apple's `SpeechAnalyzer` + `SpeechTranscriber` (iOS/macOS 26) hits **2.12% WER** on LibriSpeech test-clean vs Whisper Small's 3.74%, at ~3x speed. No performance numbers from Apple until this blog.
+
+## What's in v3
+
+Two implementations:
+
+### 1. Swift CLI (`apple_speech/`)
+
+Tiny Swift 6 package compiled directly (no SPM version dance):
+
+```bash
+cd apple_speech && bash build.sh
+```
+
+Modes:
+- `apple-speech-transcribe <file> --locale en-US` → JSONL segments, timestamped
+- `apple-speech-transcribe --bench <file>` → single JSON with RTF, full transcript
+- `apple-speech-transcribe --live --locale en-US` → mic streaming JSONL (volatile + final)
+- `apple-speech-transcribe --check` → availability + installed/supported locales
+- `apple-speech-transcribe --list-locales`
+
+Correct API usage discovered by testing against crash logs:
+- File: `SpeechAnalyzer(inputAudioFile: file, modules: [transcriber], finishAfterFile: true)` then `for try await result in transcriber.results` — results loop terminates naturally, not via task cancellation. This is the path that emits final segments correctly; `analyzeSequence(from:)` returned early and missed finals.
+- Live: `AudioBufferQueue` actor + `LiveInputSequence: AsyncSequence<AnalyzerInput>` fed to `SpeechAnalyzer(inputSequence:modules:)` with AVAudioEngine tap + optional AVAudioConverter to bestAvailableAudioFormat.
+
+### 2. Python engine (`engine_apple_transcribe.py`)
+
+Drop-in for `engine_transcribe.py`:
+
+- Same Silero VAD chunking (`silence`, `max_buffer`)
+- Each VAD-finalized chunk → temp 16-bit WAV → `apple-speech-transcribe --bench` subprocess
+- Emits dict `{original, en_bridge, detected_lang, speaker, ts, _apple_meta{rtf, segments}}` same shape as whisper engine
+- Queue-compatible with full pipeline
+
+### 3. Main v3 entry (`main_v3.py`)
+
+`--engine {whisper,apple}` selector:
+
+```bash
+./venv/bin/python main_v3.py --engine apple --lang en -v
+./venv/bin/python main_v3.py --engine apple --lang es -v
+./venv/bin/python main_v3.py --engine whisper --lang en   # old path
+./venv/bin/python main_v3.py --apple-bench-file /tmp/audio.wav --lang en
+```
+
+## Benchmark results (this machine, M-series, macOS 26.5.2)
+
+Generated via `benchmark_v3.py --test-say` (TTS roundtrip):
+
+| Locale | Text | WER | RTF |
+|--------|------|-----|-----|
+| en-US | Hello world, this is a test... | 0.0% | 28.4x |
+| en-US | quick brown fox + $35.99 normalization | 38.9%* | 30.7x |
+| en-US | email to john@example.com | 14.3%* | 23.7x |
+| es-ES | Hola mundo... | 0.0% | 14.6x |
+| es-ES | Buenos días... | 0.0% | 38.4x |
+| fr-FR | Bonjour... (voice mismatch) | 92.9% | — |
+| de-DE | Hallo Welt... | 8.3% | 15.2x |
+
+*WER inflated due to smart normalization ($35.99, john@example.com) — actually *better* output.
+
+Real claim from Inscribe: **2.12% vs Whisper Small 3.74%**, ~3x faster, on LibriSpeech 5,559 utterances, M2 Pro 32GB macOS 26.5.1.
+
+## Locales on this machine
+
+After auto-download triggers:
+
+```
+installed: de_AT de_CH de_DE en_* es_CL es_ES es_MX es_US fr_BE fr_CA fr_CH fr_FR (20)
+supported: + it_CH it_IT ja_JP ko_KR pt_BR pt_PT yue_CN zh_CN zh_HK zh_TW (30 total)
+```
+
+Install is lazy: first transcribe in a locale triggers `AssetInventory.assetInstallationRequest`.
+
+## Architecture comparison
+
+- **v2 Whisper**: audio -> Silero VAD chunks -> mlx-whisper (460MB Small) -> freeflow_polish -> LLM paragraph -> MarianMT -> ingest
+- **v3 Apple**: audio -> Silero VAD chunks -> temp WAV -> apple-speech-transcribe SFSpeechAnalyzer (system model, ~few 100MB on-demand) -> same LLM/MT pipeline
+- **Future**: native streaming inputSequence path from Python without temp files (requires PyObjC or Swift extension)
+
+## Files
+
+- `apple_speech/Sources/AppleSpeechCLI/main.swift` — Swift binary
+- `apple_speech/build.sh` — swiftc build (bypass SPM .v26 manifest issue)
+- `engine_apple_transcribe.py` — Python VAD + subprocess wrapper
+- `main_v3.py` — engine selector main
+- `benchmark_v3.py` — quick TTS bench
+- `config_v3.json` — example config with engine=apple
+
+## Next steps to go production
+
+- [ ] Mic streaming without temp files (PyObjC bridge or keep long-lived Swift daemon)
+- [ ] Speaker diarization (pyannote still works same)
+- [ ] WER eval on LibriSpeech subset
+- [ ] CI for binary rebuild on macOS version bump
+
+## Gotchas discovered
+
+1. `Package.swift` with `.macOS(.v26)` needs PackageDescription 6.2+ but CLT's swift-pm is 6.0 — must compile via `swiftc` directly.
+2. `SpeechAnalyzer`'s `init(inputAudioFile:)` vs `analyzeSequence(from:)` have different lifetime: the former finishes when results AsyncSequence ends, the latter returns before final results (requires extra sleep + task cancellation). Fixed by using file initializer.
+3. Silence-only audio returns 0 segments, not error.
+4. Assets auto-download on first transcribe; status transitions `supported` -> `downloading` -> `installed`.
+5. Binary crashes with SIGTRAP if you call `start(inputAudioFile:)` AFTER `init(inputAudioFile:)` — double start.

apple_speech/Package.swift

@@ -0,0 +1,16 @@
+// swift-tools-version: 6.0
+import PackageDescription
+
+let package = Package(
+    name: "AppleSpeech",
+    platforms: [.macOS(.v26)],
+    products: [
+        .executable(name: "apple-speech-transcribe", targets: ["AppleSpeechCLI"])
+    ],
+    targets: [
+        .executableTarget(
+            name: "AppleSpeechCLI",
+            path: "Sources/AppleSpeechCLI"
+        )
+    ]
+)

apple_speech/Sources/AppleLLMPolish/main.swift

@@ -0,0 +1,228 @@
+import Foundation
+import FoundationModels
+
+// apple-llm-polish: pipe JSON {id?, mode: "line"|"paragraph", text, prev1?, prev2?, context?, language?} -> {id, text, ok, error?}
+// Uses Apple on-device 3B Foundation Model (SystemLanguageModel)
+// Much faster than Ollama, ANE-accelerated, offline
+
+struct InMsg: Decodable {
+    var id: String?
+    var mode: String // "line" | "paragraph" | "check" | "bench"
+    var text: String?
+    var prev1: String?
+    var prev2: String?
+    var context: String?
+    var prevSource: String?
+    var language: String?
+    var bench: String?
+}
+
+struct OutMsg: Encodable {
+    var id: String?
+    var ok: Bool
+    var text: String
+    var error: String?
+    var ms: Int?
+}
+
+func log(_ s: String) { fputs(s+"\n", stderr) }
+
+@main
+struct AppleLLMPolish {
+    static func main() async {
+        let args = CommandLine.arguments
+        if args.contains("--help") || args.contains("-h") {
+            fputs("Usage: apple-llm-polish [--check] [--bench \"text\"] [--pipe]\nPipe JSONL in stdin, JSONL out\n", stderr); exit(0)
+        }
+        if args.contains("--check") {
+            await runCheck(); return
+        }
+        if let idx = args.firstIndex(of: "--bench"), idx+1 < args.count {
+            await runBench(args[idx+1]); return
+        }
+        if args.contains("--bench-text") {
+            // bench via stdin text file? not needed
+        }
+        await runPipe()
+    }
+
+    static func runCheck() async {
+        let m = SystemLanguageModel.default
+        print("{\"available\": \(m.isAvailable), \"availability\": \"\(String(describing: m.availability))\"}")
+        if case .unavailable(let reason) = m.availability {
+            print("{\"reason\":\"\(String(describing: reason))\"}")
+        }
+        if m.isAvailable {
+            // quick ping
+            do {
+                let session = LanguageModelSession(model: m, instructions: "You are a concise assistant.")
+                let r = try await session.respond(to: "Say ok")
+                print("{\"ping\":\"\(r.content)\",\"ok\":true}")
+            } catch {
+                print("{\"ping_error\":\"\(error.localizedDescription)\",\"ok\":false}")
+            }
+        }
+    }
+
+    static func runBench(_ text: String) async {
+        let m = SystemLanguageModel.default
+        guard m.isAvailable else { print("{\"ok\":false,\"error\":\"model unavailable \(m.availability)\"}"); return }
+        let session = LanguageModelSession(model: m, instructions: instructionsFor(mode: "line", language: nil))
+        let t0 = Date()
+        do {
+            let r = try await session.respond(to: promptFor(mode: "line", text: text, prev1: "", prev2: "", context: nil, prevSource: nil, language: nil))
+            let ms = Int(Date().timeIntervalSince(t0)*1000)
+            print("{\"ok\":true,\"text\":\"\(r.content.replacingOccurrences(of: "\"", with: "\\\""))\",\"ms\":\(ms)}")
+        } catch {
+            print("{\"ok\":false,\"error\":\"\(error.localizedDescription)\"}")
+        }
+    }
+
+    static func runPipe() async {
+        let m = SystemLanguageModel.default
+        guard m.isAvailable else {
+            // emit errors for all input lines so caller doesn't hang
+            let reason = "\(m.availability)"
+            while let line = readLine() {
+                if line.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { continue }
+                var idv: String? = nil
+                if let data = line.data(using: .utf8), let dict = try? JSONSerialization.jsonObject(with: data) as? [String:Any] {
+                    idv = dict["id"] as? String
+                }
+                let out = OutMsg(id: idv, ok: false, text: "", error: "model unavailable: \(reason)", ms: nil)
+                if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
+            }
+            return
+        }
+
+        // Create single session reused across requests (keeps KV cache warm)
+        let lineSession = LanguageModelSession(model: SystemLanguageModel(useCase: .general, guardrails: .permissiveContentTransformations), instructions: freeflowSystemPrompt(language: nil, mode: "line"))
+        let paraSession = LanguageModelSession(model: SystemLanguageModel(useCase: .general, guardrails: .permissiveContentTransformations), instructions: freeflowSystemPrompt(language: nil, mode: "paragraph"))
+        // Prewarm
+        lineSession.prewarm()
+        paraSession.prewarm()
+
+        log("[apple-llm-polish] ready, ANE-backed, model available")
+
+        while let line = readLine() {
+            let trimmed = line.trimmingCharacters(in: .whitespacesAndNewlines)
+            if trimmed.isEmpty { continue }
+            guard let data = line.data(using: .utf8) else { continue }
+            guard let req = try? JSONDecoder().decode(InMsg.self, from: data) else {
+                let out = OutMsg(id: nil, ok: false, text: "", error: "bad json", ms: nil)
+                if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
+                continue
+            }
+            if req.mode == "check" {
+                let out = OutMsg(id: req.id, ok: m.isAvailable, text: "\(m.availability)", error: nil, ms: 0)
+                if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
+                continue
+            }
+            let t0 = Date()
+            do {
+                let (session, prompt): (LanguageModelSession, String)
+                let lang = req.language
+                switch req.mode {
+                case "paragraph":
+                    // For paragraph we want deterministic polish first? We do full LLM here, but freeflow deterministic already did pre-clean.
+                    // Use freeflow system prompt
+                    session = paraSession
+                    prompt = buildParagraphPrompt(context: req.context ?? "", prevSource: req.prevSource ?? "", newText: req.text ?? "")
+                default: // line
+                    session = lineSession
+                    prompt = buildLinePrompt(text: req.text ?? "", prev1: req.prev1 ?? "", prev2: req.prev2 ?? "", language: lang)
+                }
+                // Generation options - low temp for deterministic polish
+                var opts = GenerationOptions()
+                opts.temperature = req.mode == "paragraph" ? 0.2 : 0.1
+                let resp = try await session.respond(to: prompt, options: opts)
+                let ms = Int(Date().timeIntervalSince(t0)*1000)
+                let cleaned = resp.content.trimmingCharacters(in: .whitespacesAndNewlines)
+                let out = OutMsg(id: req.id, ok: true, text: cleaned.isEmpty ? (req.text ?? "") : cleaned, error: nil, ms: ms)
+                if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
+            } catch {
+                let ms = Int(Date().timeIntervalSince(t0)*1000)
+                let err = error.localizedDescription
+                // guardrail or unavailable -> fallback to original text so pipeline doesn't break
+                let fallback = req.text ?? ""
+                let out = OutMsg(id: req.id, ok: false, text: fallback, error: err, ms: ms)
+                if let d = try? JSONEncoder().encode(out), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
+            }
+        }
+    }
+
+    // MARK: - Prompts (ported from freeflow_polish.py + engine_llm.py freeflow style)
+
+    static func freeflowSystemPrompt(language: String?, mode: String) -> String {
+        if mode == "paragraph" {
+            return """
+            You are a careful live transcript editor working from noisy translated source text.
+
+            Goal:
+            Produce the most faithful readable English update.
+
+            Decision rules:
+            1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+            2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+            3. The final answer must be English only.
+            4. Never copy non-English words into the output unless they are proper names.
+            5. If a non-English fragment appears as an isolated clause, trailing fragment, or side comment without a clear English anchor, drop it.
+            6. Only translate a non-English fragment when it is clearly central to the same idea and its meaning is reasonably obvious from context.
+            7. If the new material starts a fresh thought, output only the new material.
+            8. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+            9. Do not add explanations, disclaimers, or meta commentary.
+            10. Prefer conservative wording over guessed meaning.
+            11. Return only the revised transcript text.
+            """
+        } else {
+            // line
+            return """
+            You are a real-time caption polisher.
+
+            Task: Polish the current caption line to be readable, preserving meaning exactly.
+
+            Rules:
+            - Fix punctuation, casing, obvious STT typos.
+            - Expand spoken punctuation like <keep>comma</keep> already expanded upstream.
+            - Remove filler words (uh, um, erm) if clearly disfluency.
+            - Keep all semantic content; do not rewrite with prior context.
+            - If uncertain, return original.
+            - Output only one polished line, no explanation.
+            """
+        }
+    }
+
+    static func buildLinePrompt(text: String, prev1: String, prev2: String, language: String?) -> String {
+        var p = ""
+        if !prev2.isEmpty { p += "Previous line 2: \(prev2)\n" }
+        if !prev1.isEmpty { p += "Previous line 1: \(prev1)\n" }
+        p += "Current line to polish:\n\(text)\n\nPolished:"
+        return p
+    }
+
+    static func buildParagraphPrompt(context: String, prevSource: String, newText: String) -> String {
+        return """
+        Edit this transcript update.
+        [PREVIOUS REFINED CONTEXT]
+        \(context)
+
+        [PREVIOUS SOURCE TEXT]
+        \(prevSource)
+
+        [NEW SOURCE TEXT]
+        \(newText)
+        """
+    }
+
+    static func instructionsFor(mode: String, language: String?) -> String {
+        return freeflowSystemPrompt(language: language, mode: mode)
+    }
+
+    static func promptFor(mode: String, text: String, prev1: String, prev2: String, context: String?, prevSource: String?, language: String?) -> String {
+        if mode == "paragraph" {
+            return buildParagraphPrompt(context: context ?? "", prevSource: prevSource ?? "", newText: text)
+        } else {
+            return buildLinePrompt(text: text, prev1: prev1, prev2: prev2, language: language)
+        }
+    }
+}

apple_speech/Sources/AppleSpeechCLI/main.swift

@@ -0,0 +1,436 @@
+import AVFoundation
+import Foundation
+import Speech
+
+enum Mode: String { case file, live, pipe, listLocales, bench, check }
+
+struct CLIArgs {
+    var mode: Mode = .file
+    var filePath: String? = nil
+    var locale: String = "en-US"
+    var verbose: Bool = false
+    var includeVolatile: Bool = false
+}
+
+func parseArgs() -> CLIArgs {
+    var args = CLIArgs()
+    var i = 1
+    let raw = CommandLine.arguments
+    while i < raw.count {
+        let a = raw[i]
+        switch a {
+        case "--live": args.mode = .live
+        case "--pipe": args.mode = .pipe
+        case "--list-locales": args.mode = .listLocales
+        case "--bench": args.mode = .bench
+        case "--check": args.mode = .check
+        case "--locale": if i + 1 < raw.count { args.locale = raw[i+1]; i += 1 }
+        case "--verbose", "-v": args.verbose = true
+        case "--include-volatile": args.includeVolatile = true
+        case "--help", "-h":
+            fputs("""
+            Usage:
+              apple-speech-transcribe <audio-file> [--locale en-US] [-v] [--include-volatile]
+              apple-speech-transcribe --bench <audio-file> [--locale en-US] [-v]
+              apple-speech-transcribe --live [--locale en-US] [-v]
+              apple-speech-transcribe --pipe [--locale en-US] [-v]   (stdin wav chunks: 4-byte BE len + wav payload)
+              apple-speech-transcribe --check
+              apple-speech-transcribe --list-locales
+
+            Output (file mode): JSON lines per segment on stdout
+            Output (bench):      single JSON object with full transcript + timing
+            Output (live/pipe):  JSON lines streaming (isFinal + text)
+            """, stderr)
+            exit(0)
+        default:
+            if !a.hasPrefix("-") && args.filePath == nil && (args.mode == .file || args.mode == .bench) {
+                args.filePath = a
+            }
+        }
+        i += 1
+    }
+    return args
+}
+
+func logv(_ msg: String, verbose: Bool) { if verbose { fputs("[apple-speech] \(msg)\n", stderr) } }
+
+func resolveLocale(_ id: String) async -> Locale {
+    if let sup = await SpeechTranscriber.supportedLocale(equivalentTo: Locale(identifier: id)) { return sup }
+    return Locale(identifier: id)
+}
+
+func makeTranscriber(locale: Locale, includeVolatile: Bool) -> SpeechTranscriber {
+    let rep: Set<SpeechTranscriber.ReportingOption> = includeVolatile ? [.volatileResults, .alternativeTranscriptions] : [.alternativeTranscriptions]
+    return SpeechTranscriber(locale: locale, transcriptionOptions: [], reportingOptions: rep, attributeOptions: [.audioTimeRange, .transcriptionConfidence])
+}
+
+func ensureAssets(_ transcriber: SpeechTranscriber, verbose: Bool) async -> Bool {
+    let status = await AssetInventory.status(forModules: [transcriber])
+    switch status {
+    case .installed: logv("Assets installed", verbose: verbose); return true
+    case .unsupported: fputs("Locale unsupported\n", stderr); return false
+    case .supported:
+        logv("Assets not installed, downloading...", verbose: true)
+        do {
+            if let req = try await AssetInventory.assetInstallationRequest(supporting: [transcriber]) { try await req.downloadAndInstall() }
+            return true
+        } catch { fputs("Asset download failed: \(error)\n", stderr); return false }
+    case .downloading:
+        logv("Assets downloading, waiting up to 30s...", verbose: true)
+        for _ in 0..<30 {
+            try? await Task.sleep(nanoseconds: 1_000_000_000)
+            let s2 = await AssetInventory.status(forModules: [transcriber])
+            if s2 == .installed { return true }
+        }
+        return false
+    @unknown default: return true
+    }
+}
+
+// MARK: - Check
+
+func runCheck() async {
+    print("{\"event\":\"check_start\"}")
+    let isAvailable = SpeechTranscriber.isAvailable
+    print("{\"event\":\"availability\",\"isAvailable\":\(isAvailable)}")
+    if !isAvailable { print("{\"event\":\"check_end\",\"ok\":false}"); return }
+    let supported = await SpeechTranscriber.supportedLocales
+    let installed = await SpeechTranscriber.installedLocales
+    let dict: [String: Any] = [
+        "event": "locales",
+        "supported": supported.map { $0.identifier }.sorted(),
+        "installed": installed.map { $0.identifier }.sorted(),
+        "supported_count": supported.count,
+        "installed_count": installed.count
+    ]
+    if let d = try? JSONSerialization.data(withJSONObject: dict), let s = String(data: d, encoding: .utf8) { print(s) }
+    let t = SpeechTranscriber(locale: Locale(identifier: "en-US"), preset: .transcription)
+    let st = await AssetInventory.status(forModules: [t])
+    let str: String
+    switch st { case .unsupported: str="unsupported"; case .supported: str="supported"; case .downloading: str="downloading"; case .installed: str="installed"; @unknown default: str="unknown" }
+    print("{\"event\":\"asset_status\",\"locale\":\"en-US\",\"status\":\"\(str)\"}")
+    print("{\"event\":\"check_end\",\"ok\":true}")
+}
+
+func runListLocales() async {
+    if !SpeechTranscriber.isAvailable { fputs("Not available\n", stderr); exit(1) }
+    for loc in (await SpeechTranscriber.supportedLocales).sorted(by: { $0.identifier < $1.identifier }) { print(loc.identifier) }
+}
+
+// MARK: - File transcription (fixed)
+
+struct SegmentJSON: Codable {
+    var text: String
+    var isFinal: Bool
+    var start: Double?
+    var duration: Double?
+    var alternatives: [String]
+}
+
+func runFile(filePath: String, localeId: String, includeVolatile: Bool, bench: Bool, verbose: Bool) async {
+    guard FileManager.default.fileExists(atPath: filePath) else { fputs("File not found: \(filePath)\n", stderr); exit(1) }
+    guard SpeechTranscriber.isAvailable else { fputs("Not available\n", stderr); exit(1) }
+
+    let locale = await resolveLocale(localeId)
+    logv("Locale resolved: \(locale.identifier) (req \(localeId))", verbose: verbose)
+    let finalTranscriber: SpeechTranscriber = {
+        if bench { return makeTranscriber(locale: locale, includeVolatile: false) }
+        return makeTranscriber(locale: locale, includeVolatile: includeVolatile)
+    }()
+
+    guard await ensureAssets(finalTranscriber, verbose: verbose) else { exit(2) }
+
+    let fileURL = URL(fileURLWithPath: filePath)
+    let audioFile: AVAudioFile
+    do { audioFile = try AVAudioFile(forReading: fileURL) } catch { fputs("Open failed: \(error)\n", stderr); exit(4) }
+    let audioDuration = Double(audioFile.length) / audioFile.fileFormat.sampleRate
+    logv("File: \(filePath) sr=\(audioFile.fileFormat.sampleRate) ch=\(audioFile.fileFormat.channelCount) dur=\(String(format: "%.2f", audioDuration))s", verbose: verbose)
+
+    // Use init(inputAudioFile:finishAfterFile:true) which is the path that emits volatile correctly
+    // and automatically handles format.
+    let analyzer: SpeechAnalyzer
+    do {
+        analyzer = try await SpeechAnalyzer(inputAudioFile: audioFile, modules: [finalTranscriber], finishAfterFile: true)
+    } catch { fputs("Analyzer init failed: \(error)\n", stderr); exit(5) }
+
+    var segments: [SegmentJSON] = []
+    var volatileCount = 0
+    var finalCount = 0
+    let t0 = Date()
+
+    // Consume results; loop ends naturally when analyzer finishes file
+    do {
+        for try await result in finalTranscriber.results {
+            let text = String(result.text.characters)
+            if text.isEmpty { continue }
+            let isF = result.isFinal
+            if isF { finalCount += 1 } else { volatileCount += 1 }
+
+            let seg = SegmentJSON(
+                text: text,
+                isFinal: isF,
+                start: result.range.start.seconds,
+                duration: result.range.duration.seconds,
+                alternatives: result.alternatives.map { String($0.characters) }
+            )
+
+            if bench {
+                if isF { segments.append(seg) }
+            } else {
+                if !includeVolatile && !isF { continue }
+                segments.append(seg)
+                if let data = try? JSONEncoder().encode(seg), let line = String(data: data, encoding: .utf8) {
+                    print(line); fflush(stdout)
+                }
+            }
+        }
+    } catch {
+        fputs("Results error: \(error)\n", stderr)
+    }
+
+    // Ensure analyzer is done (it should be since results ended)
+    _ = analyzer // keep alive
+
+    let elapsed = Date().timeIntervalSince(t0)
+    let rtf = audioDuration / max(elapsed, 0.001)
+
+    if bench {
+        let fullText = segments.filter { $0.isFinal }.map { $0.text }.joined(separator: " ")
+        let out: [String: Any] = [
+            "file": filePath,
+            "locale": locale.identifier,
+            "requested_locale": localeId,
+            "audio_duration_sec": audioDuration,
+            "processing_sec": elapsed,
+            "rtf": rtf,
+            "volatile_count": volatileCount,
+            "final_count": finalCount,
+            "text": fullText,
+            "segments": segments.map { ["text": $0.text, "start": $0.start ?? 0, "duration": $0.duration ?? 0] }
+        ]
+        if let data = try? JSONSerialization.data(withJSONObject: out, options: [.prettyPrinted, .sortedKeys]),
+           let str = String(data: data, encoding: .utf8) { print(str) }
+    } else {
+        if verbose {
+            fputs("Done: \(finalCount) final / \(volatileCount) volatile, RTF=\(String(format: "%.1f", rtf))x elapsed=\(String(format: "%.2f", elapsed))s\n", stderr)
+        }
+    }
+}
+
+// MARK: - Live (AnalyzerInput)
+
+actor AudioBufferQueue {
+    private var buffers: [AVAudioPCMBuffer] = []
+    private var conts: [CheckedContinuation<AVAudioPCMBuffer?, Never>] = []
+    private var finished = false
+    func enqueue(_ b: AVAudioPCMBuffer) {
+        if let c = conts.first { conts.removeFirst(); c.resume(returning: b) } else { buffers.append(b) }
+    }
+    func dequeue() async -> AVAudioPCMBuffer? {
+        if !buffers.isEmpty { return buffers.removeFirst() }
+        if finished { return nil }
+        return await withCheckedContinuation { conts.append($0) }
+    }
+    func finish() { finished = true; for c in conts { c.resume(returning: nil) }; conts.removeAll() }
+}
+
+struct LiveInputSequence: AsyncSequence, Sendable {
+    let queue: AudioBufferQueue
+    struct AsyncIterator: AsyncIteratorProtocol {
+        let queue: AudioBufferQueue
+        mutating func next() async -> AnalyzerInput? {
+            guard let buf = await queue.dequeue() else { return nil }
+            return AnalyzerInput(buffer: buf)
+        }
+    }
+    func makeAsyncIterator() -> AsyncIterator { AsyncIterator(queue: queue) }
+    typealias Element = AnalyzerInput
+}
+
+func runLive(localeId: String, verbose: Bool) async {
+    guard SpeechTranscriber.isAvailable else { fputs("Not available\n", stderr); exit(1) }
+    let locale = await resolveLocale(localeId)
+    logv("Live locale \(locale.identifier) (req \(localeId))", verbose: verbose)
+    let transcriber = SpeechTranscriber(locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults, .alternativeTranscriptions], attributeOptions: [.audioTimeRange, .transcriptionConfidence])
+    guard await ensureAssets(transcriber, verbose: verbose) else { exit(2) }
+
+    let engine = AVAudioEngine()
+    let inputNode = engine.inputNode
+    let inputFormat = inputNode.inputFormat(forBus: 0)
+    logv("Mic format: \(inputFormat)", verbose: verbose)
+
+    let queue = AudioBufferQueue()
+    let inputSequence = LiveInputSequence(queue: queue)
+    let analyzer = SpeechAnalyzer(inputSequence: inputSequence, modules: [transcriber])
+    let targetFormat = await SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith: [transcriber], considering: inputFormat)
+    logv("Target format: \(String(describing: targetFormat))", verbose: verbose)
+
+    let converter: AVAudioConverter? = {
+        guard let tf = targetFormat, tf != inputFormat else { return nil }
+        return AVAudioConverter(from: inputFormat, to: tf)
+    }()
+
+    var shouldStop = false
+    let sig = DispatchSource.makeSignalSource(signal: SIGINT, queue: .main)
+    signal(SIGINT, SIG_IGN)
+    sig.setEventHandler { fputs("\nStopping...\n", stderr); shouldStop = true; Task { await queue.finish() } }
+    sig.resume()
+
+    let analyzerTask = Task { do { try await analyzer.start(inputSequence: inputSequence) } catch { fputs("Analyzer err: \(error)\n", stderr) } }
+    let resultsTask = Task {
+        do {
+            for try await result in transcriber.results {
+                let dict: [String: Any] = [
+                    "text": String(result.text.characters),
+                    "isFinal": result.isFinal,
+                    "start": result.range.start.seconds,
+                    "duration": result.range.duration.seconds,
+                    "alternatives": result.alternatives.map { String($0.characters) }
+                ]
+                if let d = try? JSONSerialization.data(withJSONObject: dict), let s = String(data: d, encoding: .utf8) { print(s); fflush(stdout) }
+            }
+        } catch { fputs("Results err: \(error)\n", stderr) }
+    }
+
+    if let tf = targetFormat, let conv = converter {
+        inputNode.installTap(onBus: 0, bufferSize: 4096, format: inputFormat) { buffer, _ in
+            if shouldStop { return }
+            let cap = AVAudioFrameCount(Double(buffer.frameLength) * tf.sampleRate / inputFormat.sampleRate + 10)
+            guard let out = AVAudioPCMBuffer(pcmFormat: tf, frameCapacity: cap) else { return }
+            var err: NSError?
+            let block: AVAudioConverterInputBlock = { _, outStatus in outStatus.pointee = .haveData; return buffer }
+            conv.convert(to: out, error: &err, withInputFrom: block)
+            if err == nil && out.frameLength > 0 { Task { await queue.enqueue(out) } }
+        }
+    } else {
+        inputNode.installTap(onBus: 0, bufferSize: 4096, format: inputFormat) { buffer, _ in
+            if shouldStop { return }
+            guard let copy = AVAudioPCMBuffer(pcmFormat: buffer.format, frameCapacity: buffer.frameCapacity) else { return }
+            copy.frameLength = buffer.frameLength
+            if let src = buffer.floatChannelData, let dst = copy.floatChannelData {
+                for ch in 0..<Int(buffer.format.channelCount) { memcpy(dst[ch], src[ch], Int(buffer.frameLength) * MemoryLayout<Float>.size) }
+            }
+            Task { await queue.enqueue(copy) }
+        }
+    }
+
+    engine.prepare()
+    do { try engine.start() } catch { fputs("Audio engine start failed: \(error)\n", stderr); exit(6) }
+    logv("Listening... Ctrl-C to stop", verbose: true)
+    while !shouldStop { try? await Task.sleep(nanoseconds: 100_000_000) }
+    inputNode.removeTap(onBus: 0)
+    engine.stop()
+    await queue.finish()
+    analyzerTask.cancel()
+    resultsTask.cancel()
+}
+
+@main
+struct AppleSpeechCLI {
+    static func main() async {
+        let cli = parseArgs()
+        switch cli.mode {
+        case .check: await runCheck()
+        case .listLocales: await runListLocales()
+        case .file:
+            guard let fp = cli.filePath else {
+                fputs("Usage: apple-speech-transcribe <file> [--locale en-US] [-v] [--include-volatile]\n", stderr); exit(1)
+            }
+            await runFile(filePath: fp, localeId: cli.locale, includeVolatile: cli.includeVolatile, bench: false, verbose: cli.verbose)
+        case .bench:
+            guard let fp = cli.filePath else { fputs("Usage: --bench <file> [--locale]\n", stderr); exit(1) }
+            await runFile(filePath: fp, localeId: cli.locale, includeVolatile: false, bench: true, verbose: cli.verbose)
+        case .live: await runLive(localeId: cli.locale, verbose: cli.verbose)
+        case .pipe: await runPipe(localeId: cli.locale, verbose: cli.verbose)
+        }
+    }
+}
+
+// MARK: - Pipe (stdin wav chunks) for draft streaming from Python VAD
+
+actor WavChunkQueue {
+    private var chunks: [URL] = []
+    private var conts: [CheckedContinuation<URL?, Never>] = []
+    private var finished = false
+    func enqueue(_ url: URL) {
+        if let c = conts.first { conts.removeFirst(); c.resume(returning: url) } else { chunks.append(url) }
+    }
+    func dequeue() async -> URL? {
+        if !chunks.isEmpty { return chunks.removeFirst() }
+        if finished { return nil }
+        return await withCheckedContinuation { conts.append($0) }
+    }
+    func finish() { finished = true; for c in conts { c.resume(returning: nil) }; conts.removeAll() }
+}
+
+func runPipe(localeId: String, verbose: Bool) async {
+    guard SpeechTranscriber.isAvailable else { fputs("Not available\n", stderr); exit(1) }
+    let locale = await resolveLocale(localeId)
+    let warm = SpeechTranscriber(locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults], attributeOptions: [])
+    guard await ensureAssets(warm, verbose: verbose) else { exit(2) }
+    logv("Pipe ready locale=\(locale.identifier)", verbose: true)
+
+    let stdinH = FileHandle.standardInput
+    var leftover = Data()
+    var chunkIdx = 0
+
+    func readExact(_ n: Int) -> Data? {
+        var out = Data()
+        out.reserveCapacity(n)
+        // first consume leftover
+        if leftover.count >= n {
+            let d = leftover.prefix(n); leftover = leftover.dropFirst(n); return Data(d)
+        }
+        if leftover.count > 0 { out.append(leftover); leftover = Data() }
+        while out.count < n {
+            let d = stdinH.readData(ofLength: n - out.count)
+            if d.isEmpty {
+                // EOF? try one more small sleep then retry once if we have partial
+                if out.count == 0 { return nil }
+                // partial -> return nil -> broken stream
+                return nil
+            }
+            out.append(d)
+        }
+        return out
+    }
+
+    while true {
+        guard let lenData = readExact(4) else { break }
+        let length = lenData.withUnsafeBytes { $0.load(as: UInt32.self).bigEndian }
+        if length == 0 { logv("Pipe EOF", verbose: true); break }
+        if length > 20_000_000 { fputs("Pipe: chunk too large \(length)\n", stderr); break }
+        guard let chunkData = readExact(Int(length)) else {
+            fputs("Pipe: truncated, expected \(length)\n", stderr); break
+        }
+        chunkIdx += 1
+        let tmpURL = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("apple-pipe-\(ProcessInfo.processInfo.processIdentifier)-\(chunkIdx).wav")
+        do { try chunkData.write(to: tmpURL) } catch { fputs("Pipe write err: \(error)\n", stderr); continue }
+        guard let audioFile = try? AVAudioFile(forReading: tmpURL) else {
+            try? FileManager.default.removeItem(at: tmpURL)
+            fputs("Pipe open fail \(chunkIdx)\n", stderr); continue
+        }
+        let t = SpeechTranscriber(locale: locale, transcriptionOptions: [], reportingOptions: [.volatileResults], attributeOptions: [.audioTimeRange])
+        guard let analyzer = try? await SpeechAnalyzer(inputAudioFile: audioFile, modules: [t], finishAfterFile: true) else {
+            try? FileManager.default.removeItem(at: tmpURL); continue
+        }
+        do {
+            for try await res in t.results {
+                let txt = String(res.text.characters).trimmingCharacters(in: .whitespacesAndNewlines)
+                if txt.isEmpty { continue }
+                let d: [String: Any] = [
+                    "event": res.isFinal ? "final" : "draft",
+                    "text": txt, "isFinal": res.isFinal,
+                    "chunk": chunkIdx, "start": res.range.start.seconds,
+                    "duration": res.range.duration.seconds
+                ]
+                if let jd = try? JSONSerialization.data(withJSONObject: d), let s = String(data: jd, encoding: .utf8) {
+                    print(s); fflush(stdout)
+                }
+            }
+        } catch { fputs("Pipe results err \(chunkIdx): \(error)\n", stderr) }
+        _ = analyzer
+        try? FileManager.default.removeItem(at: tmpURL)
+    }
+    logv("Pipe done \(chunkIdx) chunks", verbose: true)
+}

apple_speech/build.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+set -euo pipefail
+cd "$(dirname "$0")"
+SRC="Sources/AppleSpeechCLI/main.swift"
+OUT=".build/release/apple-speech-transcribe"
+mkdir -p .build/release
+
+# Use swiftc directly with macOS 26.5 SDK
+# Swift 6.3.3 supports typed throws, Speech new API compiles fine with swiftc
+echo "Compiling apple-speech-transcribe with swiftc..."
+swiftc -O -parse-as-library \
+  -target arm64-apple-macosx26.0 \
+  -sdk /Library/Developer/CommandLineTools/SDKs/MacOSX26.5.sdk \
+  "$SRC" -o "$OUT" \
+  -framework Speech -framework AVFoundation -framework Foundation
+
+echo "Built: $OUT"
+ls -lh "$OUT"
+"$OUT" --check 2>&1 || true
+
+# Also copy to project root for python use
+cp "$OUT" ../apple-speech-transcribe 2>/dev/null || cp "$OUT" ./apple-speech-transcribe
+echo "Copied to ../apple-speech-transcribe and ./apple-speech-transcribe"

apple_speech/build_llm.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+set -e
+cd "$(dirname "$0")"
+SDK="/Library/Developer/CommandLineTools/SDKs/MacOSX26.5.sdk"
+if [ ! -d "$SDK" ]; then SDK="/Library/Developer/CommandLineTools/SDKs/MacOSX26.sdk"; fi
+mkdir -p .build/release
+echo "Compiling apple-llm-polish with swiftc..."
+swiftc -O -parse-as-library -target arm64-apple-macosx26.0 -sdk "$SDK" \
+  Sources/AppleLLMPolish/main.swift -o .build/release/apple-llm-polish \
+  -framework Foundation -framework FoundationModels
+
+echo "Built: .build/release/apple-llm-polish"
+ls -lh .build/release/apple-llm-polish
+
+# check
+echo "=== check ==="
+.build/release/apple-llm-polish --check 2>&1 | head -20
+
+echo "Copying to root copies..."
+cp .build/release/apple-llm-polish ../apple-llm-polish 2>/dev/null || true
+cp .build/release/apple-llm-polish ./apple-llm-polish 2>/dev/null || true
+echo "Done"

benchmark_v3.py

@@ -0,0 +1,145 @@
+#!/usr/bin/env python3
+"""
+benchmark_v3.py - Quick WER-free benchmark comparing Apple vs Whisper on file(s)
+
+- For each audio file: run Apple (--bench) and optionally Whisper
+- Measures wall time, RTF, segment counts
+- For known reference texts, computes naive WER (word-level)
+- Outputs table + JSON
+
+Usage:
+  python benchmark_v3.py /tmp/long.wav
+  python benchmark_v3.py prayer.json --extract-audio  # future
+  python benchmark_v3.py --test-say  # generates TTS samples + benches
+"""
+
+import subprocess
+import json
+import time
+import os
+import sys
+import tempfile
+from pathlib import Path
+
+BIN = Path(__file__).parent / "apple_speech" / ".build" / "release" / "apple-speech-transcribe"
+
+def run_apple_bench(wav_path: str, locale: str = "en-US"):
+    cmd = [str(BIN), "--bench", wav_path, "--locale", locale]
+    start = time.time()
+    result = subprocess.run(cmd, capture_output=True, text=True, timeout=20)
+    wall = time.time() - start
+    if result.returncode != 0:
+        return {"ok": False, "stderr": result.stderr, "wall": wall}
+    try:
+        data = json.loads(result.stdout)
+        data["ok"] = True
+        data["bench_wall"] = wall
+        return data
+    except Exception as e:
+        return {"ok": False, "error": str(e), "stdout": result.stdout[:500], "wall": wall}
+
+def normalize_words(s: str):
+    import re
+    return re.findall(r"[a-z0-9']+", s.lower())
+
+def wer(ref: str, hyp: str) -> float:
+    """Simple word error rate via edit distance"""
+    ref_words = normalize_words(ref)
+    hyp_words = normalize_words(hyp)
+    if not ref_words:
+        return 0.0 if not hyp_words else 1.0
+    # DP
+    n, m = len(ref_words), len(hyp_words)
+    dp = [[0]*(m+1) for _ in range(n+1)]
+    for i in range(n+1): dp[i][0] = i
+    for j in range(m+1): dp[0][j] = j
+    for i in range(1, n+1):
+        for j in range(1, m+1):
+            cost = 0 if ref_words[i-1] == hyp_words[j-1] else 1
+            dp[i][j] = min(dp[i-1][j]+1, dp[i][j-1]+1, dp[i-1][j-1]+cost)
+    return dp[n][m] / n
+
+def test_with_say():
+    tests = [
+        ("en-US", "Hello world, this is a test of Apple's new speech transcription API."),
+        ("en-US", "The quick brown fox jumps over the lazy dog while pricing thirty five dollars and ninety nine cents."),
+        ("en-US", "Please send an email to john at example dot com with the meeting notes."),
+        ("es-ES", "Hola mundo, esto es una prueba de la nueva API de transcripción de Apple."),
+        ("es-ES", "Buenos días, ¿cómo estás? Espero que tengas un excelente día."),
+        ("fr-FR", "Bonjour le monde, ceci est un test de la nouvelle API de transcription d'Apple."),
+        ("de-DE", "Hallo Welt, dies ist ein Test der neuen Apple Speech Transkriptions-API."),
+    ]
+
+    voice_map = {
+        "en-US": "Samantha",
+        "es-ES": "Paulina",
+        "fr-FR": "Amelie",
+        "de-DE": "Anna",
+    }
+
+    results = []
+    for locale, ref_text in tests:
+        voice = voice_map.get(locale, "Samantha")
+        # Generate aiff
+        tmp_aiff = tempfile.mktemp(suffix=".aiff")
+        tmp_wav = tempfile.mktemp(suffix=".wav")
+        try:
+            subprocess.run(["say", "-v", voice, "-o", tmp_aiff, ref_text], timeout=10)
+            subprocess.run(["afconvert", tmp_aiff, "-f", "WAVE", "-d", "LEI16@16000", tmp_wav], timeout=10)
+            if not os.path.exists(tmp_wav):
+                continue
+            apple = run_apple_bench(tmp_wav, locale=locale)
+            hyp = apple.get("text","") if apple.get("ok") else ""
+            score = wer(ref_text, hyp) if apple.get("ok") else 1.0
+            results.append({
+                "locale": locale,
+                "ref": ref_text,
+                "hyp": hyp,
+                "wer": score,
+                "rtf": apple.get("rtf", 0),
+                "processing_sec": apple.get("processing_sec", 0),
+                "ok": apple.get("ok", False)
+            })
+            print(f"[{locale}] WER={score:.1%} RTF={apple.get('rtf',0):.1f}x")
+            print(f"  REF: {ref_text}")
+            print(f"  HYP: {hyp}")
+            print()
+        except Exception as e:
+            print(f"[{locale}] error: {e}")
+        finally:
+            for p in [tmp_aiff, tmp_wav]:
+                if os.path.exists(p):
+                    os.unlink(p)
+
+    print("\n=== Summary ===")
+    for r in results:
+        print(f"{r['locale']:6} WER={r['wer']:.1%} RTF={r['rtf']:.1f}x ok={r['ok']}")
+
+    # Save
+    out_path = Path(__file__).parent / "logs" / "apple_bench.json"
+    out_path.parent.mkdir(exist_ok=True)
+    with open(out_path, "w") as f:
+        json.dump(results, f, indent=2, ensure_ascii=False)
+    print(f"\nSaved to {out_path}")
+
+if __name__ == "__main__":
+    import argparse
+    parser = argparse.ArgumentParser()
+    parser.add_argument("files", nargs="*", help="WAV files to bench")
+    parser.add_argument("--locale", default="en-US")
+    parser.add_argument("--test-say", action="store_true", help="Generate TTS samples and bench")
+    args = parser.parse_args()
+
+    if not BIN.exists():
+        print(f"Binary not found: {BIN}")
+        print("Run: cd apple_speech && bash build.sh")
+        sys.exit(1)
+
+    if args.test_say or not args.files:
+        print("=== Running TTS-based benchmark ===")
+        test_with_say()
+    else:
+        for fp in args.files:
+            print(f"\n=== {fp} ===")
+            res = run_apple_bench(fp, locale=args.locale)
+            print(json.dumps(res, indent=2, ensure_ascii=False))

config.json

@@ -0,0 +1,42 @@
+{
+    "model": "mlx-community/whisper-base-mlx",
+    "device": null,
+    "lang": null,
+    "silence": 1000,
+    "max_buffer": 20,
+    "channels": 1,
+    "es": false,
+    "fr": false,
+    "ar": false,
+    "en": false,
+    "ingest": false,
+    "filter_lang": false,
+    "stream": false,
+    "context": false,
+    "quantize": false,
+    "speaker_diarization": false,
+    "post_correct": false,
+    "post_correct_llm": false,
+    "post_correct_model": "qwen3.5:0.8b",
+    "post_correct_ollama_url": "http://127.0.0.1:11434/api/generate",
+    "post_correct_llm_timeout": 8.0,
+    "post_correct_keep_alive": "30m",
+    "post_correct_warmup_timeout": 20.0,
+    "post_correct_min_overlap": 0.45,
+    "post_correct_debug": false,
+    "freeflow_polish": true,
+    "post_correct_skip_clean": true,
+    "post_correct_prompt_style": "freeflow",
+    "llm_paragraph": false,
+    "llm_paragraph_temperature": 0.2,
+    "llm_paragraph_top_p": 0.8,
+    "llm_paragraph_repeat_penalty": 1.0,
+    "llm_request_log_path": "logs/llm_requests.jsonl",
+    "only_translate_llm": false,
+    "temperature_fallback": [0.0, 0.2, 0.4, 0.6, 0.8, 1.0],
+    "logprob_threshold": -0.8,
+    "compression_threshold": 2.2,
+    "mt_max_chars": 250,
+    "mt_max_new_tokens": 150,
+    "mt_num_beams": 4
+}

config_v3.json

@@ -0,0 +1,45 @@
+{
+    "model": "mlx-community/whisper-base-mlx",
+    "device": null,
+    "lang": null,
+    "silence": 1000,
+    "max_buffer": 20,
+    "channels": 1,
+    "es": false,
+    "fr": false,
+    "ar": false,
+    "en": false,
+    "ingest": false,
+    "filter_lang": false,
+    "stream": false,
+    "context": false,
+    "quantize": false,
+    "speaker_diarization": false,
+    "post_correct": true,
+    "post_correct_llm": false,
+    "post_correct_model": "qwen3.5:0.8b",
+    "post_correct_ollama_url": "http://127.0.0.1:11434/api/generate",
+    "post_correct_llm_timeout": 8.0,
+    "post_correct_keep_alive": "30m",
+    "post_correct_warmup_timeout": 20.0,
+    "post_correct_min_overlap": 0.45,
+    "post_correct_debug": false,
+    "freeflow_polish": true,
+    "post_correct_skip_clean": true,
+    "post_correct_prompt_style": "freeflow",
+    "llm_paragraph": false,
+    "llm_paragraph_temperature": 0.2,
+    "llm_paragraph_top_p": 0.8,
+    "llm_paragraph_repeat_penalty": 1.0,
+    "llm_request_log_path": "logs/llm_requests.jsonl",
+    "only_translate_llm": false,
+    "temperature_fallback": [0.0, 0.2, 0.4, 0.6, 0.8, 1.0],
+    "logprob_threshold": -0.8,
+    "compression_threshold": 2.2,
+    "mt_max_chars": 250,
+    "mt_max_new_tokens": 150,
+    "mt_num_beams": 4,
+    "engine": "apple",
+    "apple_locale": null,
+    "verbose": false
+}

engine_apple_llm.py

@@ -0,0 +1,190 @@
+"""
+engine_apple_llm.py — Apple on-device LLM (FoundationModels) -> Ollama fallback
+Uses apple-llm-polish binary (same pipe protocol as Apple Speech).
+ANE-accelerated, ~0.6s vs 2-3s Ollama.
+
+Wire: line polish + paragraph rolling refine.
+"""
+
+import json, pathlib, subprocess, struct, sys, time, threading, queue, os, wave, io, select
+from typing import Optional, Callable
+from pathlib import Path
+import platform
+
+BIN_NAME = "apple-llm-polish"
+SEARCH_PATHS = [
+    Path(__file__).parent / "apple_speech" / ".build" / "release" / BIN_NAME,
+    Path(__file__).parent / BIN_NAME,
+    Path.home() / "Projects" / "pythonwhisper" / "apple_speech" / ".build" / "release" / BIN_NAME,
+]
+
+def resolve_llm_binary() -> Optional[Path]:
+    for p in SEARCH_PATHS:
+        if p.exists():
+            return p
+    import shutil
+    which = shutil.which(BIN_NAME)
+    if which:
+        return Path(which)
+    which2 = shutil.which("apple-llm-polish")
+    if which2:
+        return Path(which2)
+    return None
+
+def check_apple_llm_available(bin_path: Optional[Path]=None) -> bool:
+    bp = bin_path or resolve_llm_binary()
+    if not bp or not bp.exists():
+        return False
+    try:
+        r = subprocess.run([str(bp), "--check"], capture_output=True, text=True, timeout=10)
+        if r.returncode != 0:
+            return False
+        # parse: {"available": true}
+        for line in r.stdout.splitlines():
+            line=line.strip()
+            if not line:
+                continue
+            try:
+                j=json.loads(line)
+                if j.get("available") is True:
+                    return True
+            except:
+                pass
+        return "available\": true" in r.stdout.lower() or '"available":true' in r.stdout.lower()
+    except Exception:
+        return False
+
+
+class AppleLLM:
+    def __init__(self, bin_path: Path, verbose: bool=False):
+        self.bin_path = Path(bin_path)
+        self.verbose = verbose
+        self.proc: Optional[subprocess.Popen] = None
+        self.lock = threading.Lock()
+        self._req_id = 0
+        self._pending = {}  # id -> queue
+        self._reader_thread: Optional[threading.Thread] = None
+        self._start()
+
+    def _start(self):
+        if not self.bin_path.exists():
+            raise FileNotFoundError(str(self.bin_path))
+        self.proc = subprocess.Popen(
+            [str(self.bin_path)],
+            stdin=subprocess.PIPE,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            bufsize=1,
+            text=True,
+        )
+        # small warmup wait for [ready] log on stderr
+        time.sleep(0.5)
+        # start reader
+        self._reader_thread = threading.Thread(target=self._reader_loop, daemon=True)
+        self._reader_thread.start()
+        if self.verbose:
+            print(f"[AppleLLM] started {self.bin_path}")
+
+    def _reader_loop(self):
+        assert self.proc and self.proc.stdout
+        for line in self.proc.stdout:
+            line=line.strip()
+            if not line:
+                continue
+            try:
+                obj=json.loads(line)
+            except:
+                continue
+            rid = obj.get("id")
+            if rid is not None and rid in self._pending:
+                self._pending[rid].put(obj)
+            else:
+                # no id? try broadcast
+                for q in list(self._pending.values()):
+                    q.put(obj)
+
+    def _call(self, payload: dict, timeout: float=15.0) -> dict:
+        if not self.proc or self.proc.poll() is not None:
+            # restart
+            try:
+                self._start()
+            except Exception as e:
+                return {"ok": False, "text": payload.get("text",""), "error": f"restart failed {e}"}
+        rid = str(self._req_id)
+        self._req_id += 1
+        payload["id"] = rid
+        q: queue.Queue = queue.Queue()
+        self._pending[rid] = q
+        try:
+            line = json.dumps(payload, ensure_ascii=False)
+            assert self.proc and self.proc.stdin
+            with self.lock:
+                self.proc.stdin.write(line+"\n")
+                self.proc.stdin.flush()
+            try:
+                resp = q.get(timeout=timeout)
+                return resp
+            except queue.Empty:
+                return {"ok": False, "text": payload.get("text",""), "error": f"timeout {timeout}s", "id": rid}
+        except Exception as e:
+            return {"ok": False, "text": payload.get("text",""), "error": str(e), "id": rid}
+        finally:
+            self._pending.pop(rid, None)
+
+    def polish_line(self, text: str, prev1: str="", prev2: str="", language: Optional[str]=None, timeout: float=8.0) -> tuple[str,bool,float]:
+        """returns (polished_text, ok, ms)"""
+        resp = self._call({
+            "mode": "line",
+            "text": text,
+            "prev1": prev1,
+            "prev2": prev2,
+            "language": language,
+        }, timeout=timeout)
+        ok = bool(resp.get("ok"))
+        return (resp.get("text","") or text, ok, float(resp.get("ms",0) or 0))
+
+    def polish_paragraph(self, new_text: str, context: str="", prev_source: str="", language: Optional[str]=None, timeout: float=10.0) -> tuple[str,bool,float]:
+        resp = self._call({
+            "mode": "paragraph",
+            "text": new_text,
+            "context": context,
+            "prevSource": prev_source,
+            "language": language,
+        }, timeout=timeout)
+        ok = bool(resp.get("ok"))
+        return (resp.get("text","") or new_text, ok, float(resp.get("ms",0) or 0))
+
+    def close(self):
+        try:
+            if self.proc and self.proc.stdin:
+                try:
+                    self.proc.stdin.close()
+                except: pass
+            if self.proc:
+                self.proc.wait(timeout=2)
+        except:
+            try:
+                if self.proc:
+                    self.proc.kill()
+            except: pass
+
+
+# Singleton for multiprocess use (each process gets own via import)
+_apple_llm_instance: Optional[AppleLLM] = None
+
+def get_apple_llm(verbose: bool=False, force_new: bool=False) -> Optional[AppleLLM]:
+    global _apple_llm_instance
+    if force_new:
+        _apple_llm_instance = None
+    if _apple_llm_instance is None:
+        bp = resolve_llm_binary()
+        if not bp:
+            return None
+        if not check_apple_llm_available(bp):
+            return None
+        try:
+            _apple_llm_instance = AppleLLM(bp, verbose=verbose)
+        except Exception as e:
+            print(f"[AppleLLM] start failed: {e}", file=sys.stderr)
+            return None
+    return _apple_llm_instance

engine_apple_transcribe.py

@@ -0,0 +1,668 @@
+#!/usr/bin/env python3
+"""
+engine_apple_transcribe.py - v3 Apple Speech backend
+
+Drop-in replacement for engine_transcribe.py using Apple's new
+SpeechAnalyzer + SpeechTranscriber (macOS 26+).
+
+Architecture:
+  - Uses same Silero VAD chunking as whisper engine (audio_buffer + silence threshold)
+  - Each finalized chunk -> temp WAV 16k mono -> apple-speech-transcribe subprocess
+  - Parses bench JSON output -> emits same dict shape as whisper engine
+
+This keeps the rest of the pipeline (LLM polish, translation, distribution) unchanged.
+
+Binary location: apple_speech/.build/release/apple-speech-transcribe
+Fallback: ../apple-speech-transcribe (project root copy)
+"""
+
+from __future__ import annotations
+import sys
+from unittest.mock import MagicMock
+try:
+    import lzma
+except ImportError:
+    mock_lzma = MagicMock()
+    mock_lzma.FORMAT_XZ, mock_lzma.FORMAT_ALONE, mock_lzma.FORMAT_RAW = 1, 2, 3
+    mock_lzma.CHECK_NONE, mock_lzma.CHECK_CRC32, mock_lzma.CHECK_CRC64, mock_lzma.CHECK_SHA256 = 0, 1, 4, 10
+    sys.modules["_lzma"] = MagicMock()
+    sys.modules["lzma"] = mock_lzma
+
+import time
+import os
+import argparse
+import json
+import subprocess
+import tempfile
+import shutil
+from pathlib import Path
+from typing import Optional
+
+APPLE_BIN_CANDIDATES = [
+    Path(__file__).parent / "apple_speech" / ".build" / "release" / "apple-speech-transcribe",
+    Path(__file__).parent / "apple_speech" / "apple-speech-transcribe",
+    Path(__file__).parent / "apple-speech-transcribe",
+    Path("/tmp/apple-speech-transcribe"),
+]
+
+LANG_TO_LOCALE = {
+    "en": "en-US",
+    "en-us": "en-US",
+    "en-gb": "en-GB",
+    "en-au": "en-AU",
+    "es": "es-ES",
+    "es-es": "es-ES",
+    "es-mx": "es-MX",
+    "es-us": "es-US",
+    "es-cl": "es-CL",
+    "fr": "fr-FR",
+    "fr-fr": "fr-FR",
+    "fr-ca": "fr-CA",
+    "de": "de-DE",
+    "de-de": "de-DE",
+    "it": "it-IT",
+    "ja": "ja-JP",
+    "ko": "ko-KR",
+    "pt": "pt-PT",
+    "pt-br": "pt-BR",
+    "zh": "zh-CN",
+    "zh-cn": "zh-CN",
+    "zh-tw": "zh-TW",
+    "yue": "yue-CN",
+}
+
+APPLE_SUPPORTED_LANGS = {"en", "es", "fr", "de", "it", "ja", "ko", "pt", "zh", "yue"}
+
+def resolve_binary() -> Optional[Path]:
+    for p in APPLE_BIN_CANDIDATES:
+        if p.exists() and os.access(p, os.X_OK):
+            return p
+    return None
+
+def resolve_locale(lang: Optional[str], apple_locale_arg: Optional[str] = None) -> str:
+    if apple_locale_arg:
+        return apple_locale_arg
+    if not lang:
+        return "en-US"
+    lang_lower = lang.lower().strip()
+    if lang_lower in LANG_TO_LOCALE:
+        return LANG_TO_LOCALE[lang_lower]
+    # try prefix
+    prefix = lang_lower.split("-")[0]
+    if prefix in LANG_TO_LOCALE:
+        return LANG_TO_LOCALE[prefix]
+    # if lang itself looks like a locale (contains -) try as-is
+    if "-" in lang and len(lang) >= 4:
+        return lang
+    return "en-US"
+
+def list_audio_devices():
+    try:
+        import sounddevice as sd
+        print("\nAvailable Audio Devices:")
+        print(sd.query_devices())
+    except ImportError:
+        print("[Error] sounddevice not installed. Cannot list devices.")
+
+def _ensure_soundfile():
+    try:
+        import soundfile as sf
+        return sf
+    except ImportError:
+        return None
+
+def transcribe_chunk_apple(audio_np, sample_rate, binary_path: Path, locale: str, verbose: bool = False) -> Optional[dict]:
+    """
+    Transcribe a numpy audio chunk via apple-speech-transcribe --bench
+    Returns parsed JSON dict or None.
+    """
+    return _transcribe_chunk_wav(audio_np, sample_rate, binary_path, locale, verbose, mode="bench")
+
+def _wav_bytes_from_np(audio_np, sample_rate: int):
+    import numpy as np, wave, io
+    if audio_np.dtype != np.float32:
+        if audio_np.dtype == np.int16:
+            audio_float = audio_np.astype(np.float32) / 32768.0
+        else:
+            audio_float = audio_np.astype(np.float32)
+    else:
+        audio_float = audio_np
+    if audio_float.ndim > 1:
+        audio_float = audio_float.mean(axis=1)
+    audio_float = np.clip(audio_float, -1.0, 1.0)
+    if sample_rate != 16000:
+        duration = len(audio_float) / sample_rate
+        new_len = int(duration * 16000)
+        if new_len > 0:
+            x_old = np.linspace(0, 1, len(audio_float))
+            x_new = np.linspace(0, 1, new_len)
+            audio_float = np.interp(x_new, x_old, audio_float).astype(np.float32)
+            sample_rate = 16000
+    int16_data = (audio_float * 32767).astype(np.int16)
+    bio = io.BytesIO()
+    with wave.open(bio, 'w') as wf:
+        wf.setnchannels(1); wf.setsampwidth(2); wf.setframerate(sample_rate)
+        wf.writeframes(int16_data.tobytes())
+    return bio.getvalue(), sample_rate
+
+def _transcribe_chunk_wav(audio_np, sample_rate, binary_path: Path, locale: str, verbose: bool, mode: str = "bench") -> Optional[dict]:
+    import numpy as np
+    tmp_path = None
+    try:
+        import wave, struct, io
+        wav_bytes, sr = _wav_bytes_from_np(audio_np, sample_rate)
+        fd, tmp_path = tempfile.mkstemp(suffix=".wav")
+        os.close(fd)
+        with open(tmp_path, "wb") as f:
+            f.write(wav_bytes)
+        if mode == "bench":
+            cmd = [str(binary_path), "--bench", tmp_path, "--locale", locale]
+        else:
+            # include volatile to get drafts
+            cmd = [str(binary_path), tmp_path, "--locale", locale, "--include-volatile"]
+        result = subprocess.run(cmd, capture_output=True, text=True, timeout=8.0)
+        if result.returncode != 0:
+            if verbose:
+                print(f"[Apple] Binary failed {result.returncode}: {result.stderr[:500]}")
+            return None
+        if mode == "bench":
+            try:
+                payload = json.loads(result.stdout)
+            except json.JSONDecodeError as e:
+                if verbose:
+                    print(f"[Apple] JSON parse failed: {e} stdout={result.stdout[:500]}")
+                return None
+            return payload
+        else:
+            # parse JSONL with volatile+final
+            finals = []
+            volatiles = []
+            for line in result.stdout.splitlines():
+                line=line.strip()
+                if not line: continue
+                try:
+                    seg=json.loads(line)
+                    if seg.get("isFinal"):
+                        finals.append(seg)
+                    else:
+                        volatiles.append(seg)
+                except: pass
+            # build combined text from finals
+            full_text = " ".join(s.get("text","") for s in finals).strip()
+            full_text = " ".join(full_text.split())
+            return {"text": full_text, "segments": finals, "volatiles": volatiles, "audio_duration_sec": len(audio_np)/16000}
+    except subprocess.TimeoutExpired:
+        if verbose: print("[Apple] Transcription timed out")
+        return None
+    except Exception as e:
+        if verbose: print(f"[Apple] Error: {e}")
+        return None
+    finally:
+        if tmp_path and os.path.exists(tmp_path):
+            try: os.unlink(tmp_path)
+            except: pass
+
+def transcribe_chunk_apple_with_draft(audio_np, sample_rate, binary_path: Path, locale: str, verbose: bool = False) -> Optional[dict]:
+    return _transcribe_chunk_wav(audio_np, sample_rate, binary_path, locale, verbose, mode="draft")
+
+
+class ApplePipeTranscriber:
+    """
+    Keeps one Swift binary process alive in --pipe mode.
+    Write wav chunk -> reads draft/final JSON lines with ultra-low overhead (no subprocess spawn per chunk).
+    Falls back to bench if pipe not available.
+    """
+    def __init__(self, binary_path: Path, locale: str, verbose=False):
+        self.binary_path = binary_path
+        self.locale = locale
+        self.verbose = verbose
+        self.proc = None
+        self.lock = None
+        self._start()
+
+    def _start(self):
+        import threading
+        self.lock = threading.Lock()
+        cmd = [str(self.binary_path), "--pipe", "--locale", self.locale]
+        if self.verbose:
+            cmd.append("-v")
+        try:
+            self.proc = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=False, bufsize=0)
+            # check alive after 1s
+            import time
+            time.sleep(0.2)
+            if self.proc.poll() is not None:
+                err = self.proc.stderr.read().decode(errors="ignore") if self.proc.stderr else ""
+                if self.verbose:
+                    print(f"[ApplePipe] Failed to start, exit={self.proc.returncode} {err[:500]}")
+                self.proc=None
+            else:
+                if self.verbose:
+                    print(f"[ApplePipe] Started pid={self.proc.pid} locale={self.locale}")
+        except Exception as e:
+            if self.verbose:
+                print(f"[ApplePipe] Start error: {e}")
+            self.proc=None
+
+    def transcribe(self, audio_np, sample_rate) -> Optional[dict]:
+        if not self.proc or self.proc.poll() is not None:
+            self._start()
+            if not self.proc:
+                return None
+        try:
+            wav_bytes, _ = _wav_bytes_from_np(audio_np, sample_rate)
+            import struct
+            header = struct.pack(">I", len(wav_bytes))
+            with self.lock:
+                self.proc.stdin.write(header + wav_bytes)
+                self.proc.stdin.flush()
+                import time, select
+                finals=[]; volatiles=[]
+                start=time.time()
+                while True:
+                    remaining = 6.0 - (time.time()-start)
+                    if remaining<=0: break
+                    rready,_,_ = select.select([self.proc.stdout], [], [], min(remaining, 0.2))
+                    if not rready:
+                        if finals: break
+                        continue
+                    line = self.proc.stdout.readline()
+                    if not line: break
+                    try:
+                        txt=line.decode('utf-8', errors='ignore').strip()
+                        if not txt: continue
+                        obj=json.loads(txt)
+                        t=obj.get("text","").strip()
+                        if not t: continue
+                        if obj.get("isFinal") or obj.get("event")=="final":
+                            finals.append(obj)
+                            stall=time.time()+0.35
+                            while time.time()<stall:
+                                rr,_,_ = select.select([self.proc.stdout], [], [], 0.05)
+                                if rr:
+                                    extra=self.proc.stdout.readline()
+                                    if not extra: break
+                                    try:
+                                        eo=json.loads(extra.decode('utf-8',errors='ignore'))
+                                        if eo.get("text","").strip():
+                                            if eo.get("isFinal") or eo.get("event")=="final":
+                                                finals.append(eo)
+                                            else: volatiles.append(eo)
+                                        stall=time.time()+0.35
+                                    except: pass
+                            break
+                        else:
+                            volatiles.append(obj)
+                    except: continue
+            if not finals and not volatiles: return None
+            full_text = " ".join(f.get("text","") for f in finals).strip() if finals else (volatiles[-1].get("text","") if volatiles else "")
+            full_text = " ".join(full_text.split())
+            return {"text": full_text, "segments": finals, "volatiles": volatiles, "audio_duration_sec": len(audio_np)/16000, "pipe": True}
+        except Exception as e:
+            if self.verbose: print(f"[ApplePipe] transcribe error: {e}")
+            try: self.proc.kill()
+            except: pass
+            self.proc=None
+            return None
+
+    def transcribe_with_draft_callback(self, audio_np, sample_rate, draft_callback=None) -> Optional[dict]:
+        """
+        Streaming version: calls draft_callback(text) for each volatile update (word-by-word),
+        then returns final dict. Use this for ultra-low latency draft captions.
+        """
+        if not self.proc or self.proc.poll() is not None:
+            self._start()
+            if not self.proc:
+                return None
+        try:
+            wav_bytes, _ = _wav_bytes_from_np(audio_np, sample_rate)
+            import struct, time, select
+            header = struct.pack(">I", len(wav_bytes))
+            with self.lock:
+                self.proc.stdin.write(header + wav_bytes)
+                self.proc.stdin.flush()
+                finals=[]; volatiles=[]
+                start=time.time()
+                last_draft=""
+                while True:
+                    remaining = 6.0 - (time.time()-start)
+                    if remaining<=0: break
+                    rr,_,_ = select.select([self.proc.stdout], [], [], min(remaining, 0.15))
+                    if not rr:
+                        if finals: break
+                        continue
+                    line=self.proc.stdout.readline()
+                    if not line: break
+                    try:
+                        obj=json.loads(line.decode('utf-8',errors='ignore').strip())
+                        t=obj.get("text","").strip()
+                        if not t: continue
+                        if obj.get("isFinal") or obj.get("event")=="final":
+                            finals.append(obj)
+                            stall=time.time()+0.35
+                            while time.time()<stall:
+                                r2,_,_ = select.select([self.proc.stdout], [], [], 0.05)
+                                if r2:
+                                    extra=self.proc.stdout.readline()
+                                    if not extra: break
+                                    try:
+                                        eo=json.loads(extra.decode('utf-8',errors='ignore'))
+                                        if eo.get("text","").strip():
+                                            if eo.get("isFinal") or eo.get("event")=="final": finals.append(eo)
+                                            else:
+                                                volatiles.append(eo)
+                                                if draft_callback and eo.get("text","").strip()!=last_draft:
+                                                    draft_callback(eo["text"])
+                                                    last_draft=eo["text"]
+                                        stall=time.time()+0.35
+                                    except: pass
+                            break
+                        else:
+                            volatiles.append(obj)
+                            if draft_callback and t!=last_draft:
+                                draft_callback(t)
+                                last_draft=t
+                    except: continue
+            if not finals and not volatiles: return None
+            full_text=" ".join(f.get("text","") for f in finals).strip() if finals else (volatiles[-1].get("text","") if volatiles else "")
+            full_text=" ".join(full_text.split())
+            return {"text": full_text, "segments": finals, "volatiles": volatiles, "audio_duration_sec": len(audio_np)/16000, "pipe": True}
+        except Exception as e:
+            if self.verbose: print(f"[ApplePipe] draft cb error: {e}")
+            try: self.proc.kill()
+            except: pass
+            self.proc=None
+            return None
+
+    def close(self):
+        try:
+            if self.proc and self.proc.poll() is None:
+                import struct
+                with self.lock:
+                    try:
+                        self.proc.stdin.write(struct.pack(">I", 0))
+                        self.proc.stdin.flush()
+                        self.proc.stdin.close()
+                    except: pass
+                self.proc.wait(timeout=3)
+        except:
+            try: self.proc.kill()
+            except: pass
+        self.proc=None
+
+
+
+def run_transcription(out_queue, args):
+    """
+    Main entry compatible with multiprocessing.Process(target=run_transcription)
+    Same signature as engine_transcribe.py
+    """
+    import numpy as np
+    import sounddevice as sd
+    import torch
+    from silero_vad import load_silero_vad, get_speech_timestamps
+    import queue
+
+    SAMPLERATE, BLOCK_SIZE, VAD_THRESHOLD = 16000, 512, 0.5
+
+    binary_path = resolve_binary()
+    if binary_path is None:
+        print(f"[AppleTranscribe] ERROR: binary not found. Searched: {APPLE_BIN_CANDIDATES}")
+        print("[AppleTranscribe] Run: cd apple_speech && bash build.sh")
+        return
+
+    apple_locale = resolve_locale(getattr(args, "lang", None), getattr(args, "apple_locale", None))
+    verbose = getattr(args, "verbose", False) or getattr(args, "post_correct_debug", False)
+
+    print(f"[AppleTranscribe] Using binary: {binary_path}")
+    print(f"[AppleTranscribe] Locale: {apple_locale} (from lang={getattr(args,'lang',None)})")
+    print(f"[AppleTranscribe] VAD model loading...")
+    vad_model = load_silero_vad()
+
+    # Quick health check
+    try:
+        chk = subprocess.run([str(binary_path), "--check"], capture_output=True, text=True, timeout=5)
+        if chk.returncode != 0:
+            print(f"[AppleTranscribe] --check failed: {chk.stderr}")
+        else:
+            # parse last line
+            print(f"[AppleTranscribe] Binary check OK")
+            if verbose:
+                for line in chk.stdout.splitlines():
+                    print(f"  {line}")
+    except Exception as e:
+        print(f"[AppleTranscribe] Binary check error: {e}")
+
+    # Optional: preload by transcribing 0.1s silence to warm model
+    try:
+        silence = np.zeros(int(SAMPLERATE * 0.1), dtype=np.float32)
+        _ = transcribe_chunk_apple(silence, SAMPLERATE, binary_path, apple_locale, verbose=False)
+        print(f"[AppleTranscribe] Warmup done")
+    except:
+        pass
+
+    audio_queue = queue.Queue()
+
+    def audio_callback(indata, frames, time_info, status):
+        if status:
+            print(status, file=sys.stderr)
+        audio_queue.put(indata.copy())
+
+    audio_buffer, speech_started = [], False
+    last_stream_time = time.time()
+    last_draft_text = ""
+    # Feature flags
+    use_stream = getattr(args, "stream", False) or getattr(args, "apple_stream", True)  # default ON for Apple (cheap)
+    stream_interval = getattr(args, "apple_stream_interval", 1.0)  # seconds between draft transcribes
+    use_pipe = getattr(args, "apple_pipe", True)  # keep one Swift process alive (faster than spawn per chunk)
+    pipe_transcriber = None
+
+    if use_pipe:
+        try:
+            pipe_transcriber = ApplePipeTranscriber(binary_path, apple_locale, verbose=verbose)
+            if pipe_transcriber.proc is None:
+                pipe_transcriber = None
+                print("[AppleTranscribe] Pipe mode unavailable, falling back to subprocess per chunk")
+        except Exception as e:
+            if verbose:
+                print(f"[AppleTranscribe] Pipe init failed: {e}")
+            pipe_transcriber = None
+
+    device = "mps" if torch.backends.mps.is_available() else "cpu"
+
+    def do_transcribe_final(audio_np):
+        if pipe_transcriber and pipe_transcriber.proc and pipe_transcriber.proc.poll() is None:
+            return pipe_transcriber.transcribe(audio_np, SAMPLERATE)
+        return transcribe_chunk_apple(audio_np, SAMPLERATE, binary_path, apple_locale, verbose=verbose)
+
+    def do_transcribe_with_streaming_drafts(audio_np, on_draft):
+        """Use pipe's volatile streaming for real-time drafts (60ms granularity)."""
+        if pipe_transcriber and pipe_transcriber.proc and pipe_transcriber.proc.poll() is None:
+            return pipe_transcriber.transcribe_with_draft_callback(audio_np, SAMPLERATE, draft_callback=on_draft)
+        # fallback: no streaming, just final
+        return do_transcribe_final(audio_np)
+
+    try:
+        with sd.InputStream(samplerate=SAMPLERATE, channels=getattr(args, "channels", 1),
+                            callback=audio_callback, blocksize=BLOCK_SIZE, device=getattr(args, "device", None)):
+            print(f"[AppleTranscribe] Listening (silence={getattr(args,'silence',1000)}ms max_buffer={getattr(args,'max_buffer',20)}s stream={use_stream} pipe={pipe_transcriber is not None})...")
+            while True:
+                while not audio_queue.empty():
+                    data = audio_queue.get()
+                    if data is not None and data.size > 0:
+                        if getattr(args, "channels", 1) > 1:
+                            data = np.mean(data, axis=1)
+                        audio_buffer.append(data.flatten())
+
+                if audio_buffer:
+                    current_audio = np.concatenate(audio_buffer)
+                    audio_tensor = torch.from_numpy(current_audio)
+
+                    speech_timestamps = get_speech_timestamps(
+                        audio_tensor, vad_model,
+                        sampling_rate=SAMPLERATE,
+                        threshold=VAD_THRESHOLD,
+                        min_silence_duration_ms=getattr(args, "silence", 1000)
+                    )
+
+                    if speech_timestamps:
+                        speech_started = True
+                        # --- draft streaming (non-blocking poll every stream_interval) ---
+                        if use_stream and (time.time() - last_stream_time) > stream_interval:
+                            if len(current_audio) > SAMPLERATE * 0.6:
+                                def _emit_draft(txt):
+                                    nonlocal last_draft_text
+                                    t = " ".join(txt.split()).strip()
+                                    if t and t != last_draft_text and len(t) > 1:
+                                        out_queue.put({"draft": t, "ts": time.time(), "isApple": True})
+                                        if verbose:
+                                            print(f"\r[DRAFT]: {t}    ", end="", flush=True)
+                                        last_draft_text = t
+
+                                # quick draft via pipe streaming - but we are inside input stream callback thread?
+                                # Use quick final call with draft callback if pipe else interim transcribe
+                                try:
+                                    # For low-latency drafts while speaking, transcribe current buffer
+                                    # with incremental volatile updates showing word-by-word
+                                    snap = current_audio.copy()  # avoid racing with audio_buffer appending
+                                    # If pipe not available, fall back to fast volatile parse
+                                    if pipe_transcriber and pipe_transcriber.proc:
+                                        # Use a separate thread to not block VAD loop too long? But okay
+                                        # We'll call blocking but it returns within 200ms
+                                        _ = pipe_transcriber.transcribe_with_draft_callback(
+                                            snap, SAMPLERATE, draft_callback=_emit_draft
+                                        )
+                                    else:
+                                        dr = transcribe_chunk_apple_with_draft(snap, SAMPLERATE, binary_path, apple_locale, verbose=False)
+                                        if dr:
+                                            txt = dr.get("text", "").strip()
+                                            if txt:
+                                                _emit_draft(txt)
+                                except Exception as e:
+                                    if verbose:
+                                        print(f"[Apple DRAFT err] {e}")
+                            last_stream_time = time.time()
+
+                        silence_frames = len(current_audio) - speech_timestamps[-1]['end']
+                        should_flush = (silence_frames > (SAMPLERATE * getattr(args, "silence", 1000) / 1000)) or \
+                                       len(current_audio) > (SAMPLERATE * getattr(args, "max_buffer", 20))
+
+                        if should_flush:
+                            # Cancel any pending draft display and transcribe final
+                            # Use streaming final for best latency + true partials
+                            t0 = time.time()
+
+                            # Use pipe final (fast) without draft callback for this flush
+                            result = do_transcribe_final(current_audio)
+                            dt = time.time() - t0
+
+                            if result is None:
+                                print(f"[AppleTranscribe] Chunk failed (len={len(current_audio)/SAMPLERATE:.2f}s)")
+                                audio_buffer, speech_started = [], False
+                                last_draft_text=""
+                                last_stream_time = time.time()
+                                continue
+
+                            text = result.get("text", "").strip()
+                            text = " ".join(text.split())
+
+                            if not text:
+                                audio_buffer, speech_started = [], False
+                                last_draft_text=""
+                                last_stream_time = time.time()
+                                continue
+
+                            detected_lang = getattr(args, "lang", None) or apple_locale.split("-")[0]
+                            audio_dur = result.get("audio_duration_sec", len(current_audio)/SAMPLERATE)
+                            rtf = result.get("rtf", result.get("processing_sec", 0) and (audio_dur / max(result.get("processing_sec",0.001),0.001)) or 0)
+                            if use_stream:
+                                print(flush=True)
+
+                            out_queue.put({
+                                "original": text,
+                                "en_bridge": text,
+                                "detected_lang": detected_lang,
+                                "speaker": None,
+                                "ts": time.time(),
+                                "_apple_meta": {
+                                    "locale": apple_locale,
+                                    "rtf": rtf,
+                                    "processing_sec": result.get("processing_sec"),
+                                    "segments": result.get("segments", []),
+                                    "pipe": result.get("pipe", False),
+                                }
+                            })
+
+                            print(f"[AppleTranscribe] {detected_lang.upper()} [{audio_dur:.1f}s-> {dt*1000:.0f}ms RTF={rtf:.1f}x]: {text}")
+                            audio_buffer, speech_started = [], False
+                            last_draft_text=""
+                            last_stream_time = time.time()
+
+                    elif not speech_started and len(current_audio) > SAMPLERATE * 2:
+                        audio_buffer = []
+                        last_draft_text=""
+
+    except Exception as e:
+        print(f"[AppleTranscribe] Error: {e}")
+        import traceback
+        traceback.print_exc()
+    finally:
+        if pipe_transcriber:
+            pipe_transcriber.close()
+
+
+# For --file mode testing standalone (non-live)
+def transcribe_file(file_path: str, locale: str = "en-US", include_volatile: bool = False) -> Optional[dict]:
+    binary_path = resolve_binary()
+    if not binary_path:
+        raise FileNotFoundError(f"Apple speech binary not found. Candidates: {APPLE_BIN_CANDIDATES}")
+    # For file mode we can call binary directly with JSONL or bench
+    cmd = [str(binary_path), file_path, "--locale", locale]
+    if include_volatile:
+        cmd.append("--include-volatile")
+    result = subprocess.run(cmd, capture_output=True, text=True, timeout=20)
+    if result.returncode != 0:
+        print(f"Failed: {result.stderr}")
+        return None
+    # JSONL lines
+    segments = []
+    for line in result.stdout.splitlines():
+        line=line.strip()
+        if not line:
+            continue
+        try:
+            seg = json.loads(line)
+            segments.append(seg)
+        except:
+            pass
+    full_text = " ".join(s.get("text","") for s in segments if s.get("isFinal")).strip()
+    full_text = " ".join(full_text.split())
+    return {"text": full_text, "segments": segments}
+
+if __name__ == "__main__":
+    import argparse
+    parser = argparse.ArgumentParser()
+    parser.add_argument("file", nargs="?", help="Audio file to transcribe")
+    parser.add_argument("--locale", default="en-US")
+    parser.add_argument("--bench", action="store_true")
+    parser.add_argument("--include-volatile", action="store_true")
+    parser.add_argument("--verbose", "-v", action="store_true")
+    args = parser.parse_args()
+
+    binary_path = resolve_binary()
+    if not binary_path:
+        print("Binary not found")
+        sys.exit(1)
+
+    if args.file:
+        if args.bench:
+            cmd = [str(binary_path), "--bench", args.file, "--locale", args.locale]
+            if args.verbose:
+                cmd.append("-v")
+            subprocess.run(cmd)
+        else:
+            r = transcribe_file(args.file, args.locale, args.include_volatile)
+            print(json.dumps(r, indent=2, ensure_ascii=False))
+    else:
+        # live test (just list devices)
+        list_audio_devices()

engine_distribute.py

@@ -0,0 +1,128 @@
+import requests
+import time
+import argparse
+import multiprocessing
+import sys
+import os
+
+INGEST_URL = "https://emiapi.reynafamily.com/live-captions/ingest"
+
+def post_with_retries(payload, timeout, max_attempts=3):
+    delay = 1
+    for attempt in range(1, max_attempts + 1):
+        try:
+            response = requests.post(INGEST_URL, json=payload, timeout=timeout)
+            if response.status_code == 200:
+                return True
+            print(f"[Distribute Error] {response.status_code} on attempt {attempt}/{max_attempts}.")
+        except Exception as exc:
+            print(f"[Distribute Error] {exc} on attempt {attempt}/{max_attempts}.")
+        if attempt < max_attempts:
+            time.sleep(delay)
+            delay = min(delay * 2, 5)
+    return False
+
+def log_debug(message):
+    try:
+        with open("distribute_debug.log", "a") as f:
+            f.write(f"{time.strftime('%H:%M:%S')} {message}\n")
+            f.flush()
+            os.fsync(f.fileno())
+    except:
+        pass
+
+def run_distribution(in_queue, args):
+    print("[Distribute] Starting distribution service...", flush=True)
+    log_debug("Starting distribution service...")
+    
+    while True:
+        try:
+            payload = in_queue.get()
+            if payload is None: break
+            
+            # Draft support
+            if "draft" in payload:
+                draft_text = payload["draft"]
+                sys.stdout.write(f"\r\033[K[DRAFT]: {draft_text}")
+                sys.stdout.flush()
+                if args.ingest:
+                    post_with_retries({"draft": draft_text}, timeout=2, max_attempts=2)
+                continue
+
+            # Finalized segment
+            speaker_label = payload.get('speaker')
+            speaker_prefix = f"[{speaker_label}]: " if speaker_label else ""
+            
+            msg = f"[Final]: {speaker_prefix}{payload.get('original', '')}"
+            print(msg, flush=True)
+            log_debug(msg)
+
+            bridge_text = payload.get("en_bridge")
+            original_text = payload.get("original", "")
+            corrected_text = payload.get("corrected")
+            paragraph_text = payload.get("paragraph")
+
+            if bridge_text and bridge_text.strip() != original_text.strip():
+                bridge_msg = f"[EN-BRIDGE]: {speaker_prefix}{bridge_text}"
+                print(bridge_msg, flush=True)
+                log_debug(bridge_msg)
+
+            if payload.get("used_llm_line") and corrected_text and corrected_text.strip() != bridge_text.strip():
+                llm_line_msg = f"[LLM-LINE]: {speaker_prefix}{corrected_text}"
+                print(llm_line_msg, flush=True)
+                log_debug(llm_line_msg)
+
+            if payload.get("used_llm_paragraph") and paragraph_text:
+                print("", flush=True)
+                log_debug("")
+                llm_paragraph_msg = f"[LLM-PARAGRAPH]: {speaker_prefix}{paragraph_text}"
+                print(llm_paragraph_msg, flush=True)
+                log_debug(llm_paragraph_msg)
+                print("", flush=True)
+                log_debug("")
+
+            if payload.get("paragraph_fallback") and paragraph_text:
+                paragraph_msg = f"[PARAGRAPH]: {speaker_prefix}{paragraph_text}"
+                print(paragraph_msg, flush=True)
+                log_debug(paragraph_msg)
+            
+            for lang in ["es", "fr", "ar", "en"]:
+                if payload.get(lang):
+                    lang_msg = f"[{lang.upper()}]: {speaker_prefix}{payload[lang]}"
+                    print(lang_msg, flush=True)
+                    log_debug(lang_msg)
+
+            if args.ingest:
+                # Flat JSON Schema: Keep only what's needed for the server
+                ingest_payload = {
+                    "original": payload.get("original"),
+                    "speaker": payload.get("speaker"),
+                    "ts": payload.get("ts")
+                }
+                english_output = payload.get("english_output") or payload.get("en")
+                if english_output:
+                    ingest_payload["en"] = english_output
+                has_any_translation = False
+                for lang in ["es", "fr", "ar"]:
+                    if payload.get(lang):
+                        ingest_payload[lang] = payload[lang]
+                        has_any_translation = True
+                if english_output:
+                    has_any_translation = True
+                
+                # If only-translate-llm is on, ONLY send when we have a translation (paragraph-level)
+                should_send = True
+                if getattr(args, "only_translate_llm", False) and not has_any_translation:
+                    should_send = False
+                
+                if should_send:
+                    success = post_with_retries(ingest_payload, timeout=5, max_attempts=3)
+                    if not success:
+                        log_debug(f"Dropped ingest payload after retries: {ingest_payload}")
+                
+        except Exception as e:
+            print(f"[Distribute] Error: {e}")
+
+if __name__ == "__main__":
+    import multiprocessing
+    run_distribution(multiprocessing.Queue(), argparse.Namespace())

engine_llm.py

@@ -0,0 +1,736 @@
+import sys
+from unittest.mock import MagicMock
+
+# MANDATORY: Mock lzma BEFORE any other imports
+try:
+    import lzma
+except ImportError:
+    mock_lzma = MagicMock()
+    mock_lzma.FORMAT_XZ, mock_lzma.FORMAT_ALONE, mock_lzma.FORMAT_RAW = 1, 2, 3
+    mock_lzma.CHECK_NONE, mock_lzma.CHECK_CRC32, mock_lzma.CHECK_CRC64, mock_lzma.CHECK_SHA256 = 0, 1, 4, 10
+    sys.modules["_lzma"] = MagicMock()
+    sys.modules["lzma"] = mock_lzma
+
+import time
+import argparse
+import multiprocessing
+import os
+import re
+import json
+import requests
+import queue
+from collections import deque
+from datetime import datetime, timezone
+from freeflow_polish import (
+    build_user_prompt,
+    deterministic_polish,
+    guard_against_truncation,
+    is_clean_enough_to_skip_llm,
+    match_input_casing,
+    normalize_formatting,
+    strip_keep_tags,
+    system_prompt_for_language,
+)
+
+
+def build_paragraph_messages(previous_refined_context, previous_source_text, new_source_text):
+    system_prompt = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+3. The final answer must be English only.
+4. Never copy non-English words into the output unless they are proper names.
+5. If a non-English fragment appears as an isolated clause, trailing fragment, or side comment without a clear English anchor, drop it.
+6. Only translate a non-English fragment when it is clearly central to the same idea and its meaning is reasonably obvious from context.
+7. If the new material starts a fresh thought, output only the new material.
+8. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+9. Do not add explanations, disclaimers, or meta commentary.
+10. Prefer conservative wording over guessed meaning.
+11. Return only the revised transcript text."""
+
+    prompt = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT]
+{previous_refined_context}
+
+[PREVIOUS SOURCE TEXT]
+{previous_source_text}
+
+[NEW SOURCE TEXT]
+{new_source_text}
+"""
+    return [
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": prompt},
+    ], prompt
+
+
+def build_line_messages(prev1, prev2, corrected):
+    system_prompt = """You are a real-time English caption corrector for live speech.
+
+Task:
+Clean only the current caption line.
+
+Hard rules:
+1. Preserve meaning exactly. Never replace current content with prior context.
+2. Remove disfluencies and false starts.
+3. Fix punctuation, casing, and obvious STT typos.
+4. If uncertain, return the original line unchanged.
+5. Output only one corrected English line."""
+
+    prompt = (
+        "Context (reference only):\n"
+        f"Previous line 1: {prev1}\n"
+        f"Previous line 2: {prev2}\n"
+        "Current line to correct:\n"
+        f"{corrected}\n"
+        "Corrected:"
+    )
+    return [
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": prompt},
+    ], prompt
+
+
+def build_freeflow_line_messages(text, language=None, model=""):
+    system_prompt = system_prompt_for_language(language, model=model)
+    prompt = build_user_prompt(text, language=language)
+    return [
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": prompt},
+    ], prompt
+
+
+def append_llm_request_log(log_path, entry):
+    if not log_path:
+        return
+    log_dir = os.path.dirname(log_path)
+    if log_dir:
+        os.makedirs(log_dir, exist_ok=True)
+    with open(log_path, "a", encoding="utf-8") as handle:
+        json.dump(entry, handle, ensure_ascii=False)
+        handle.write("\n")
+
+
+def load_llm_request_log_entry(log_path, entry_index):
+    if not log_path or not os.path.exists(log_path):
+        raise FileNotFoundError(f"LLM request log not found: {log_path}")
+
+    with open(log_path, "r", encoding="utf-8") as handle:
+        entries = [json.loads(line) for line in handle if line.strip()]
+
+    if not entries:
+        raise ValueError(f"LLM request log is empty: {log_path}")
+
+    if entry_index is None or entry_index == -1:
+        return entries[-1]
+
+    if entry_index < 0:
+        entry_index = len(entries) + entry_index
+
+    if entry_index < 0 or entry_index >= len(entries):
+        raise IndexError(f"LLM request log index {entry_index} is out of range for {len(entries)} entries")
+
+    return entries[entry_index]
+
+
+def run_llm_prompt_test(args):
+    from dotenv import load_dotenv
+    load_dotenv()
+
+    api_key = os.environ.get("OPENAI_API_KEY")
+    log_path = getattr(args, "llm_request_log_path", "logs/llm_requests.jsonl")
+
+    def call_openai(model, messages):
+        response = requests.post(
+            "https://api.openai.com/v1/chat/completions",
+            json={"model": model, "messages": messages},
+            headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
+            timeout=getattr(args, "post_correct_llm_timeout", 8.0),
+        )
+        if response.status_code >= 400:
+            try:
+                payload = response.json()
+                detail = json.dumps(payload, ensure_ascii=True)
+            except Exception:
+                detail = response.text.strip()
+            raise RuntimeError(f"OpenAI API error {response.status_code}: {detail}")
+        return response.json()["choices"][0]["message"]["content"].strip()
+
+    def call_ollama(model, prompt, temperature, system=None, extra_options=None):
+        options = {"temperature": temperature}
+        if extra_options:
+            options.update(extra_options)
+        response = requests.post(
+            getattr(args, "post_correct_ollama_url", "http://127.0.0.1:11434/api/generate"),
+            json={
+                "model": model,
+                "prompt": prompt,
+                "system": system or "",
+                "stream": False,
+                "options": options,
+                "keep_alive": getattr(args, "post_correct_keep_alive", "30m"),
+            },
+            timeout=getattr(args, "post_correct_llm_timeout", 8.0),
+        )
+        response.raise_for_status()
+        return response.json().get("response", "").strip()
+
+    def send_test_request(mode, messages, prompt, temperature, metadata, model=None, extra_options=None):
+        request_model = model or args.post_correct_model
+        is_openai = request_model.startswith("gpt-")
+        entry = {
+            "timestamp": datetime.now(timezone.utc).isoformat(),
+            "mode": mode,
+            "provider": "openai" if is_openai else "ollama",
+            "model": request_model,
+            "temperature": temperature,
+            "options": extra_options or {},
+            "messages": messages,
+            "metadata": metadata,
+        }
+        started_at = time.time()
+        if is_openai:
+            if not api_key:
+                raise RuntimeError("OPENAI_API_KEY is not set")
+            try:
+                response_text = call_openai(request_model, messages)
+                entry["response"] = {"status": "ok", "content": response_text}
+                return response_text
+            except Exception as exc:
+                entry["response"] = {"status": "error", "error": str(exc)}
+                raise
+            finally:
+                entry["duration_ms"] = round((time.time() - started_at) * 1000, 2)
+                append_llm_request_log(log_path, entry)
+        system = ""
+        for message in messages:
+            if message.get("role") == "system":
+                system = message.get("content", "")
+                break
+        try:
+            response_text = call_ollama(request_model, prompt, temperature, system=system, extra_options=extra_options)
+            entry["response"] = {"status": "ok", "content": response_text}
+            return response_text
+        except Exception as exc:
+            entry["response"] = {"status": "error", "error": str(exc)}
+            raise
+        finally:
+            entry["duration_ms"] = round((time.time() - started_at) * 1000, 2)
+            append_llm_request_log(log_path, entry)
+
+    if getattr(args, "llm_test_from_log", None) is not None:
+        logged_entry = load_llm_request_log_entry(log_path, args.llm_test_from_log)
+        logged_messages = logged_entry.get("messages")
+        if not logged_messages:
+            raise ValueError("Selected log entry does not contain messages")
+        logged_prompt = next((m.get("content", "") for m in logged_messages if m.get("role") == "user"), "")
+        logged_model = logged_entry.get("model", args.post_correct_model)
+        replay_model = logged_model if getattr(args, "llm_test_use_logged_model", False) else args.post_correct_model
+        response = send_test_request(
+            logged_entry.get("mode", "replay"),
+            logged_messages,
+            logged_prompt,
+            logged_entry.get("temperature", 0.1),
+            {
+                "replay_source_log_path": log_path,
+                "replay_source_index": args.llm_test_from_log,
+                "replay_source_timestamp": logged_entry.get("timestamp"),
+                "replay_source_model": logged_model,
+            },
+            model=replay_model,
+        )
+        print("[LLM TEST] Replayed request from log:")
+        print(f"source_index={args.llm_test_from_log} source_model={logged_model} replay_model={replay_model}")
+        print(response)
+
+    if getattr(args, "llm_test_line", None):
+        use_freeflow = getattr(args, "post_correct_prompt_style", "freeflow") != "legacy"
+        if use_freeflow:
+            preprocessed, substituted = deterministic_polish(args.llm_test_line)
+            messages, prompt = build_freeflow_line_messages(
+                substituted,
+                language=getattr(args, "lang", None),
+                model=getattr(args, "post_correct_model", ""),
+            )
+            metadata_input = preprocessed
+        else:
+            messages, prompt = build_line_messages(
+                getattr(args, "llm_test_prev1", ""),
+                getattr(args, "llm_test_prev2", ""),
+                args.llm_test_line,
+            )
+            metadata_input = args.llm_test_line
+        response = send_test_request(
+            "line",
+            messages,
+            prompt,
+            0.1,
+            {
+                "prev1": getattr(args, "llm_test_prev1", ""),
+                "prev2": getattr(args, "llm_test_prev2", ""),
+                "input": metadata_input,
+                "prompt_style": getattr(args, "post_correct_prompt_style", "freeflow"),
+            },
+        )
+        print("[LLM TEST] Line response:")
+        print(response)
+
+    if getattr(args, "llm_test_segments", None):
+        messages, prompt = build_paragraph_messages(
+            getattr(args, "llm_test_context", ""),
+            "",
+            args.llm_test_segments,
+        )
+        response = send_test_request(
+            "paragraph",
+            messages,
+            prompt,
+            0.1,
+            {
+                "context": getattr(args, "llm_test_context", ""),
+                "segments": args.llm_test_segments,
+            },
+        )
+        print("[LLM TEST] Paragraph response:")
+        print(response)
+
+def run_llm_processor(in_queue, out_queue, args):
+    from dotenv import load_dotenv
+    load_dotenv()
+    
+    print("[LLM] Starting Rolling Paragraph Engine...")
+    
+    api_key = os.environ.get("OPENAI_API_KEY")
+    is_openai = args.post_correct_model.startswith("gpt-")
+    llm_state = {"line_warned": False, "paragraph_warned": False, "apple_warned": False}
+    log_path = getattr(args, "llm_request_log_path", "logs/llm_requests.jsonl")
+
+    # --- Apple on-device LLM (ANE) – primary if available ---
+    apple_llm = None
+    apple_provider_enabled = getattr(args, "apple_llm", True)  # on by default
+    use_apple_llm = False
+    if apple_provider_enabled and not is_openai:
+        try:
+            from engine_apple_llm import resolve_llm_binary, check_apple_llm_available, AppleLLM
+            bp = resolve_llm_binary()
+            if bp and check_apple_llm_available(bp):
+                print(f"[LLM] Apple on-device LLM available at {bp} — using ANE path (primary), Ollama fallback")
+                try:
+                    apple_llm = AppleLLM(bp, verbose=getattr(args, "verbose", False))
+                    use_apple_llm = True
+                    print(f"[LLM] Apple LLM ready: {bp} (ANE-backed)")
+                except Exception as e:
+                    print(f"[LLM] Apple LLM start failed ({e}) — falling back to Ollama")
+            else:
+                if getattr(args, "verbose", False):
+                    print(f"[LLM] Apple LLM not available (bin={bp}) — using Ollama")
+        except Exception as e:
+            if getattr(args, "verbose", False):
+                print(f"[LLM] Apple LLM probe failed ({e})")
+
+    def check_ollama_health():
+        try:
+            response = requests.get("http://127.0.0.1:11434/api/tags", timeout=2.5)
+            response.raise_for_status()
+            data = response.json()
+            models = data.get("models", [])
+            available = {model.get("name") for model in models if model.get("name")}
+            if args.post_correct_model not in available and f"{args.post_correct_model}:latest" not in available:
+                print(f"[LLM] Configured Ollama model '{args.post_correct_model}' is not installed.")
+            return True
+        except Exception as exc:
+            print(f"[LLM] Ollama health check failed ({exc}). Local LLM features may fall back.")
+            return False
+
+    def warmup_ollama_model():
+        try:
+            response = requests.post(
+                getattr(args, "post_correct_ollama_url", "http://127.0.0.1:11434/api/generate"),
+                json={
+                    "model": args.post_correct_model,
+                    "prompt": "Return exactly: ok",
+                    "system": "You are a concise assistant.",
+                    "stream": False,
+                    "options": {"temperature": 0.0},
+                    "keep_alive": getattr(args, "post_correct_keep_alive", "30m"),
+                },
+                timeout=getattr(args, "post_correct_warmup_timeout", 20.0),
+            )
+            response.raise_for_status()
+            body = response.json()
+            if body.get("response", "").strip():
+                total_s = body.get("total_duration", 0) / 1_000_000_000
+                load_s = body.get("load_duration", 0) / 1_000_000_000
+                print(f"[LLM] Ollama warmup OK for {args.post_correct_model} (load={load_s:.2f}s total={total_s:.2f}s).")
+                return True
+        except Exception as exc:
+            print(f"[LLM] Ollama warmup failed for {args.post_correct_model} ({exc}).")
+        return False
+
+    if not is_openai and not use_apple_llm:
+        check_ollama_health()
+        if getattr(args, "post_correct_llm", False) or getattr(args, "llm_paragraph", False):
+            warmup_ollama_model()
+
+    def normalize_english_caption(text):
+        normalized = " ".join((text or "").split()).strip()
+        if normalized and normalized[0].isalpha():
+            normalized = normalized[0].upper() + normalized[1:]
+        return normalized
+
+    def apply_rule_based_post_correction(text):
+        if getattr(args, "freeflow_polish", True):
+            corrected, _ = deterministic_polish(text)
+            return corrected
+        corrected = normalize_english_caption(text)
+        corrected = re.sub(r"\s+", " ", corrected).strip()
+        corrected = re.sub(r"\s+([,.;!?])", r"\1", corrected)
+        corrected = re.sub(r"([,.;!?]){2,}", r"\1", corrected)
+        corrected = re.sub(r"\b(uh+|um+|erm+|ah+|hmm+)\b", "", corrected, flags=re.IGNORECASE)
+        corrected = re.sub(r"\s+", " ", corrected).strip()
+        corrected = re.sub(r"\b(\w+)\s+\1\b", r"\1", corrected, flags=re.IGNORECASE)
+        return normalize_english_caption(corrected)
+
+    def word_overlap_ratio(source_text, candidate_text):
+        src_tokens = set(re.findall(r"[a-z0-9']+", source_text.lower()))
+        cand_tokens = set(re.findall(r"[a-z0-9']+", candidate_text.lower()))
+        if not src_tokens:
+            return 1.0
+        return len(src_tokens & cand_tokens) / len(src_tokens)
+
+    def call_openai(messages):
+        response = requests.post(
+            "https://api.openai.com/v1/chat/completions",
+            json={"model": args.post_correct_model, "messages": messages},
+            headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
+            timeout=getattr(args, "post_correct_llm_timeout", 8.0),
+        )
+        if response.status_code >= 400:
+            try:
+                payload = response.json()
+                detail = json.dumps(payload, ensure_ascii=True)
+            except Exception:
+                detail = response.text.strip()
+            raise RuntimeError(f"OpenAI API error {response.status_code}: {detail}")
+        return response.json()["choices"][0]["message"]["content"].strip()
+
+    def call_ollama(prompt, temperature, system=None, extra_options=None):
+        options = {"temperature": temperature}
+        if extra_options:
+            options.update(extra_options)
+        response = requests.post(
+            getattr(args, "post_correct_ollama_url", "http://127.0.0.1:11434/api/generate"),
+            json={
+                "model": args.post_correct_model,
+                "prompt": prompt,
+                "system": system or "",
+                "stream": False,
+                "options": options,
+                "keep_alive": getattr(args, "post_correct_keep_alive", "30m"),
+            },
+            timeout=getattr(args, "post_correct_llm_timeout", 8.0),
+        )
+        response.raise_for_status()
+        return response.json().get("response", "").strip()
+
+    def call_llm(messages, prompt, temperature, warn_key, extra_options=None):
+        # Apple LLM path (ANE) - try first if enabled
+        if use_apple_llm and apple_llm is not None:
+            # Determine mode from warn_key
+            is_paragraph = warn_key == "paragraph_warned"
+            # Extract from prompt/messages best-effort
+            # For line: prompt is the substituted line, we need recent lines for prev1/prev2
+            # For paragraph: prompt built from paragraph_messages – we need previous_refined_context, previous_source, new_segments
+            # We will keep the signature but also intercept via wrapper functions below that pass raw components via closure.
+            # This generic call_llm fallback will still attempt Ollama if Apple fails inside specific wrapper.
+            pass  # actual Apple dispatch is in wrappers below, this is preserved for direct callers
+
+        entry = {
+            "timestamp": datetime.now(timezone.utc).isoformat(),
+            "mode": "paragraph" if warn_key == "paragraph_warned" else "line",
+            "provider": "openai" if is_openai else ("apple" if use_apple_llm else "ollama"),
+            "model": args.post_correct_model if not use_apple_llm else "apple-fm",
+            "temperature": temperature,
+            "options": extra_options or {},
+            "messages": messages,
+        }
+        started_at = time.time()
+        try:
+            if is_openai:
+                if not api_key:
+                    raise RuntimeError("OPENAI_API_KEY is not set")
+                response_text = call_openai(messages)
+                entry["response"] = {"status": "ok", "content": response_text}
+                return response_text
+            # If we are in Apple mode, this generic path should not be hit for line/para polish
+            # because post_correct_line and call_llm_rolling_refine now directly call Apple.
+            # But keep Ollama fallback for any direct call_llm usage.
+            system = ""
+            if messages:
+                for message in messages:
+                    if message.get("role") == "system":
+                        system = message.get("content", "")
+                        break
+            response_text = call_ollama(prompt, temperature, system=system, extra_options=extra_options)
+            entry["response"] = {"status": "ok", "content": response_text}
+            return response_text
+        except Exception as exc:
+            entry["response"] = {"status": "error", "error": str(exc)}
+            if not llm_state[warn_key]:
+                label = "Paragraph LLM" if warn_key == "paragraph_warned" else "Line LLM"
+                print(f"[LLM] {label} unavailable ({exc}). Falling back to deterministic text.")
+                llm_state[warn_key] = True
+            return ""
+        finally:
+            entry["duration_ms"] = round((time.time() - started_at) * 1000, 2)
+            append_llm_request_log(log_path, entry)
+
+    # State: Keep a short rolling window of refined paragraphs as context.
+    refined_context_history = deque(maxlen=2)
+    previous_source_window = ""
+    recent_lines = deque(maxlen=2)
+    
+    def call_llm_rolling_refine(new_segments_str, previous_refined_context, previous_source_text):
+        # Apple ANE path first
+        if use_apple_llm and apple_llm is not None:
+            t0 = time.time()
+            try:
+                polished, ok, apple_ms = apple_llm.polish_paragraph(
+                    new_text=new_segments_str,
+                    context=previous_refined_context,
+                    prev_source=previous_source_text,
+                    language=getattr(args, "lang", None),
+                    timeout=getattr(args, "post_correct_llm_timeout", 8.0),
+                )
+                entry = {
+                    "timestamp": datetime.now(timezone.utc).isoformat(),
+                    "mode": "paragraph",
+                    "provider": "apple",
+                    "model": "apple-fm",
+                    "duration_ms": int((time.time()-t0)*1000),
+                    "apple_ms": apple_ms,
+                    "ok": ok,
+                }
+                append_llm_request_log(log_path, entry)
+                if ok and polished:
+                    print(f"[LLM] Apple paragraph polish OK {apple_ms:.0f}ms -> {polished[:80]}")
+                    return polished
+                else:
+                    print(f"[LLM] Apple paragraph fallback ok={ok} ms={apple_ms} err, trying Ollama")
+            except Exception as e:
+                print(f"[LLM] Apple paragraph error {e}, falling back to Ollama")
+                if not llm_state["apple_warned"]:
+                    llm_state["apple_warned"] = True
+
+        messages, prompt = build_paragraph_messages(previous_refined_context, previous_source_text, new_segments_str)
+        paragraph_options = {
+            "top_p": getattr(args, "llm_paragraph_top_p", 0.8),
+            "repeat_penalty": getattr(args, "llm_paragraph_repeat_penalty", 1.0),
+        }
+        return call_llm(
+            messages,
+            prompt,
+            getattr(args, "llm_paragraph_temperature", 0.2),
+            "paragraph_warned",
+            extra_options=paragraph_options,
+        )
+
+    def post_correct_line(text):
+        if not getattr(args, "post_correct", False):
+            return normalize_english_caption(text)
+
+        corrected, substituted = deterministic_polish(text) if getattr(args, "freeflow_polish", True) else (apply_rule_based_post_correction(text), text)
+        if not getattr(args, "post_correct_llm", False):
+            return corrected
+        if getattr(args, "post_correct_skip_clean", True) and is_clean_enough_to_skip_llm(corrected):
+            if getattr(args, "post_correct_debug", False):
+                print("[LLM] Skipping line LLM polish; deterministic output looks clean.")
+            return corrected
+
+        # Try Apple ANE path first (fast, offline)
+        if use_apple_llm and apple_llm is not None:
+            try:
+                t0 = time.time()
+                prev1 = recent_lines[-1] if len(recent_lines) >= 1 else ""
+                prev2 = recent_lines[-2] if len(recent_lines) >= 2 else ""
+                polished, ok, apple_ms = apple_llm.polish_line(
+                    text=substituted if getattr(args, "freeflow_polish", True) else corrected,
+                    prev1=prev1, prev2=prev2,
+                    language=getattr(args, "lang", None),
+                    timeout=getattr(args, "post_correct_llm_timeout", 8.0),
+                )
+                dur = int((time.time()-t0)*1000)
+                # Log Apple call
+                entry = {
+                    "timestamp": datetime.now(timezone.utc).isoformat(),
+                    "mode": "line",
+                    "provider": "apple",
+                    "model": "apple-fm",
+                    "duration_ms": dur,
+                    "apple_ms": apple_ms,
+                    "ok": ok,
+                    "input": substituted[:200],
+                    "response": {"content": polished},
+                }
+                append_llm_request_log(log_path, entry)
+                if ok and polished:
+                    # Apply same post-processing as Ollama path
+                    prompt_style = getattr(args, "post_correct_prompt_style", "freeflow")
+                    cand = polished
+                    if prompt_style != "legacy":
+                        fb = guard_against_truncation(cand, corrected)
+                        if fb:
+                            if getattr(args, "post_correct_debug", False):
+                                print(f"[LLM] Apple guard fallback triggered, keeping deterministic: {fb[:60]}")
+                            return fb
+                        cand = strip_keep_tags(cand)
+                        cand = normalize_formatting(cand)
+                        cand = match_input_casing(cand, substituted)
+                    else:
+                        cand = normalize_english_caption(cand)
+                    if cand and word_overlap_ratio(corrected, cand) >= getattr(args, "post_correct_min_overlap", 0.45):
+                        print(f"[LLM] Apple line OK {apple_ms:.0f}ms ({dur}ms total) -> {cand[:80]}")
+                        return cand
+                    else:
+                        if getattr(args, "verbose", False):
+                            print(f"[LLM] Apple line low overlap {word_overlap_ratio(corrected, cand):.2f}, keep deterministic")
+                        return corrected
+                else:
+                    print(f"[LLM] Apple line not ok ok={ok} ms={apple_ms}, falling back to Ollama")
+            except Exception as e:
+                print(f"[LLM] Apple line error {e}, falling back to Ollama")
+                if not llm_state["apple_warned"]:
+                    llm_state["apple_warned"] = True
+
+        prompt_style = getattr(args, "post_correct_prompt_style", "freeflow")
+        if prompt_style == "legacy":
+            prev2 = recent_lines[-2] if len(recent_lines) >= 2 else ""
+            prev1 = recent_lines[-1] if len(recent_lines) >= 1 else ""
+            messages, prompt = build_line_messages(prev1, prev2, corrected)
+        else:
+            language = getattr(args, "lang", None)
+            messages, prompt = build_freeflow_line_messages(
+                substituted,
+                language=language,
+                model=args.post_correct_model if prompt_style == "qwen" else "",
+            )
+        candidate = call_llm(messages, prompt, 0.1, "line_warned").strip()
+        if prompt_style != "legacy":
+            fallback = guard_against_truncation(candidate, corrected)
+            if fallback:
+                return fallback
+            candidate = strip_keep_tags(candidate)
+            candidate = normalize_formatting(candidate)
+            candidate = match_input_casing(candidate, substituted)
+        else:
+            candidate = normalize_english_caption(candidate)
+        if candidate and word_overlap_ratio(corrected, candidate) >= getattr(args, "post_correct_min_overlap", 0.45):
+            return candidate
+        return corrected
+
+    pending_buffer = []
+
+    def flush_pending_outputs(processed_items):
+        nonlocal pending_buffer, previous_source_window
+        if not processed_items:
+            return
+
+        structured_paragraph = None
+        paragraph_from_llm = False
+        if args.llm_paragraph and pending_buffer:
+            new_batch = " ".join(pending_buffer)
+            previous_refined_context = "\n\n".join(refined_context_history)
+            result = call_llm_rolling_refine(new_batch, previous_refined_context, previous_source_window)
+
+            if result:
+                if "\n\n" in result:
+                    parts = result.split("\n\n")
+                    refined_context_history.clear()
+                    for part in parts:
+                        cleaned_part = part.strip()
+                        if cleaned_part:
+                            refined_context_history.append(cleaned_part)
+                    structured_paragraph = result
+                else:
+                    cleaned_result = result.strip()
+                    if cleaned_result:
+                        refined_context_history.append(cleaned_result)
+                    structured_paragraph = result
+                paragraph_from_llm = True
+            else:
+                structured_paragraph = new_batch
+                cleaned_batch = new_batch.strip()
+                if cleaned_batch:
+                    refined_context_history.append(cleaned_batch)
+
+            previous_source_window = new_batch
+
+        last_index = len(processed_items) - 1
+        for index, processed_item in enumerate(processed_items):
+            out_queue.put({
+                "raw": processed_item["raw"],
+                "corrected": processed_item["corrected"],
+                "paragraph": structured_paragraph if index == last_index else None,
+                "en_bridge": processed_item["en_bridge"],
+                "used_llm_line": bool(getattr(args, "post_correct_llm", False)),
+                "used_llm_paragraph": paragraph_from_llm if index == last_index else False,
+                "detected_lang": processed_item.get("detected_lang"),
+                "speaker": processed_item.get("speaker"),
+                "ts": processed_item.get("ts", time.time())
+            })
+
+        pending_buffer = []
+    
+    while True:
+        try:
+            item = in_queue.get()
+            if item is None: break
+            if "draft" in item:
+                out_queue.put(item)
+                continue
+            processed_items = []
+            pending_item = item
+
+            while pending_item is not None:
+                raw_text = pending_item.get("original", "").strip()
+                bridge_text = pending_item.get("en_bridge", raw_text).strip()
+                if raw_text:
+                    corrected_text = post_correct_line(bridge_text)
+
+                    # Hallucination check
+                    words = corrected_text.lower().split()
+                    if not (len(words) > 10 and len(set(words)) < 3):
+                        pending_buffer.append(corrected_text)
+                        recent_lines.append(corrected_text)
+                        processed_items.append({
+                            "raw": raw_text,
+                            "corrected": corrected_text,
+                            "en_bridge": bridge_text,
+                            "detected_lang": pending_item.get("detected_lang"),
+                            "speaker": pending_item.get("speaker"),
+                            "ts": pending_item.get("ts", time.time()),
+                        })
+
+                try:
+                    pending_item = in_queue.get_nowait()
+                    if pending_item is None:
+                        flush_pending_outputs(processed_items)
+                        return
+                    if "draft" in pending_item:
+                        out_queue.put(pending_item)
+                        pending_item = None
+                except queue.Empty:
+                    pending_item = None
+
+            flush_pending_outputs(processed_items)
+            
+        except Exception as e:
+            print(f"[LLM] Error: {e}")
+
+if __name__ == "__main__":
+    run_llm_processor(multiprocessing.Queue(), multiprocessing.Queue(), argparse.Namespace())

engine_transcribe.py

@@ -0,0 +1,206 @@
+import sys
+from unittest.mock import MagicMock
+
+# MANDATORY: Mock lzma BEFORE any other imports (especially pyannote/torchvision)
+try:
+    import lzma
+except ImportError:
+    mock_lzma = MagicMock()
+    mock_lzma.FORMAT_XZ, mock_lzma.FORMAT_ALONE, mock_lzma.FORMAT_RAW = 1, 2, 3
+    mock_lzma.CHECK_NONE, mock_lzma.CHECK_CRC32, mock_lzma.CHECK_CRC64, mock_lzma.CHECK_SHA256 = 0, 1, 4, 10
+    sys.modules["_lzma"] = MagicMock()
+    sys.modules["lzma"] = mock_lzma
+
+import time
+import os
+import argparse
+
+def list_audio_devices():
+    try:
+        import sounddevice as sd
+        print("\nAvailable Audio Devices:")
+        print(sd.query_devices())
+    except ImportError:
+        print("[Error] sounddevice not installed. Cannot list devices.")
+
+def assign_speakers_to_segments(segments, diarization_result):
+    if not diarization_result or not segments:
+        return segments
+    
+    # pyannote 4.0.4 returns a DiarizeOutput object with a speaker_diarization attribute
+    annotation = None
+    if hasattr(diarization_result, 'speaker_diarization'):
+        annotation = diarization_result.speaker_diarization
+    elif hasattr(diarization_result, 'itertracks'):
+        annotation = diarization_result
+    
+    if annotation is None:
+        return segments
+
+    for segment in segments:
+        segment_start, segment_end = segment.get('start', 0), segment.get('end', 0)
+        speaker_intersections = {}
+        
+        try:
+            for turn, _, speaker in annotation.itertracks(yield_label=True):
+                intersection_start = max(segment_start, turn.start)
+                intersection_end = min(segment_end, turn.end)
+                if intersection_end > intersection_start:
+                    duration = intersection_end - intersection_start
+                    speaker_intersections[speaker] = speaker_intersections.get(speaker, 0) + duration
+        except Exception:
+            continue
+        
+        if speaker_intersections:
+            segment['speaker'] = max(speaker_intersections, key=speaker_intersections.get)
+        else:
+            segment['speaker'] = 'UNKNOWN'
+    return segments
+
+def run_transcription(out_queue, args):
+    import numpy as np
+    import sounddevice as sd
+    import torch
+    import mlx_whisper
+    from silero_vad import load_silero_vad, get_speech_timestamps
+    import queue
+
+    SAMPLERATE, BLOCK_SIZE, VAD_THRESHOLD = 16000, 512, 0.5
+
+    def transcribe_with_controls(audio, transcribe_kwargs):
+        try:
+            return mlx_whisper.transcribe(audio, **transcribe_kwargs)
+        except Exception as exc:
+            if "beam_size" in transcribe_kwargs:
+                transcribe_kwargs.pop("beam_size")
+                return mlx_whisper.transcribe(audio, **transcribe_kwargs)
+            raise exc
+
+    device = "mps" if torch.backends.mps.is_available() else "cpu"
+    model_name = args.model
+    if args.quantize and "4bit" not in model_name:
+        model_name = "mlx-community/whisper-small-mlx-4bit"
+    
+    print(f"[Transcribe] Loading Whisper model '{model_name}' on {device}...")
+    vad_model = load_silero_vad()
+    
+    diarization_pipeline = None
+    if args.speaker_diarization:
+        from pyannote.audio import Pipeline
+        print("[Transcribe] Loading speaker diarization pipeline...")
+        hf_token = os.environ.get("HF_TOKEN")
+        try:
+            diarization_pipeline = Pipeline.from_pretrained("pyannote/speaker-diarization-3.1", token=hf_token)
+            diarization_pipeline.to(torch.device(device))
+        except Exception as e:
+            print(f"[Transcribe] Diarization load failed: {e}")
+
+    audio_queue = queue.Queue()
+    def audio_callback(indata, frames, time, status):
+        if status: print(status, file=sys.stderr)
+        audio_queue.put(indata.copy())
+
+    audio_buffer, speech_started, rolling_context, rolling_context_en = [], False, "", ""
+    last_stream_time = time.time()
+    
+    try:
+        with sd.InputStream(samplerate=SAMPLERATE, channels=args.channels, callback=audio_callback, blocksize=BLOCK_SIZE, device=args.device):
+            while True:
+                while not audio_queue.empty():
+                    data = audio_queue.get()
+                    if data is not None and data.size > 0:
+                        if args.channels > 1:
+                            data = np.mean(data, axis=1)
+                        audio_buffer.append(data.flatten())
+
+                if audio_buffer:
+                    current_audio = np.concatenate(audio_buffer)
+                    audio_tensor = torch.from_numpy(current_audio)
+                    
+                    speech_timestamps = get_speech_timestamps(audio_tensor, vad_model, sampling_rate=SAMPLERATE, threshold=VAD_THRESHOLD, min_silence_duration_ms=args.silence)
+
+                    if speech_timestamps:
+                        speech_started = True
+                        if (len(current_audio) - speech_timestamps[-1]['end']) > (SAMPLERATE * args.silence / 1000) or len(current_audio) > (SAMPLERATE * args.max_buffer):
+                            
+                            # STEP 1: ALWAYS TRANSCRIBE FIRST
+                            transcribe_kwargs = {
+                                "path_or_hf_repo": model_name,
+                                "temperature": args.temperature_fallback,
+                                "word_timestamps": args.speaker_diarization,
+                                "logprob_threshold": args.logprob_threshold,
+                                "compression_ratio_threshold": args.compression_threshold,
+                            }
+                            if args.lang: transcribe_kwargs["language"] = args.lang
+                            if args.context and rolling_context: transcribe_kwargs["initial_prompt"] = rolling_context
+
+                            res_asr = transcribe_with_controls(current_audio, transcribe_kwargs)
+                            text = res_asr['text'].strip()
+                            detected_lang = res_asr.get('language', args.lang or 'en')
+
+                            if text:
+                                if args.filter_lang and args.lang and detected_lang != args.lang:
+                                    print(f"[Transcribe] Discarding segment (Detected: {detected_lang}, Expected: {args.lang})")
+                                    audio_buffer, speech_started = [], False
+                                    continue
+
+                                english_text = text
+                                if detected_lang != "en":
+                                    bridge_kwargs = {
+                                        "path_or_hf_repo": model_name,
+                                        "task": "translate",
+                                        "temperature": args.temperature_fallback,
+                                        "logprob_threshold": args.logprob_threshold,
+                                        "compression_ratio_threshold": args.compression_threshold,
+                                    }
+                                    if args.lang:
+                                        bridge_kwargs["language"] = args.lang
+                                    if args.context and rolling_context_en:
+                                        bridge_kwargs["initial_prompt"] = rolling_context_en
+                                    res_bridge = transcribe_with_controls(current_audio, bridge_kwargs)
+                                    english_text = res_bridge.get("text", "").strip() or text
+
+                                speaker_label = None
+                                if diarization_pipeline:
+                                    try:
+                                        diar_out = diarization_pipeline({"waveform": torch.from_numpy(current_audio).float().unsqueeze(0), "sample_rate": SAMPLERATE})
+                                        asr_segments = res_asr.get('segments', [])
+                                        diarized_segments = assign_speakers_to_segments(asr_segments, diar_out)
+                                        
+                                        speakers = [s.get('speaker') for s in diarized_segments if s.get('speaker') and s.get('speaker') != 'UNKNOWN']
+                                        if speakers:
+                                            from collections import Counter
+                                            speaker_label = Counter(speakers).most_common(1)[0][0]
+                                    except Exception as diar_err:
+                                        print(f"[Transcribe] Diarization error: {diar_err}")
+
+                                out_queue.put({
+                                    "original": text, 
+                                    "en_bridge": english_text,
+                                    "detected_lang": detected_lang, 
+                                    "speaker": speaker_label,
+                                    "ts": time.time()
+                                })
+                                
+                                speaker_tag = f" ({speaker_label})" if speaker_label else ""
+                                print(f"[Transcribe] {detected_lang.upper()}{speaker_tag}: {text}")
+                                if args.context:
+                                    rolling_context = (rolling_context + " " + text)[-200:].strip()
+                                    rolling_context_en = (rolling_context_en + " " + english_text)[-200:].strip()
+
+                            audio_buffer, speech_started = [], False
+                        
+                        elif args.stream and (time.time() - last_stream_time) > 1.5:
+                            draft_result = transcribe_with_controls(current_audio, {"path_or_hf_repo": model_name, "temperature": 0.0})
+                            if draft_result['text'].strip():
+                                out_queue.put({"draft": draft_result['text'].strip(), "ts": time.time()})
+                            last_stream_time = time.time()
+                    
+                    elif not speech_started and len(current_audio) > SAMPLERATE * 2:
+                        audio_buffer = []
+
+    except Exception as e: print(f"[Transcribe] Error: {e}")
+
+if __name__ == "__main__":
+    import multiprocessing
+    run_transcription(multiprocessing.Queue(), argparse.Namespace())

engine_translate.py

@@ -0,0 +1,112 @@
+import argparse
+import multiprocessing
+import time
+import re
+
+def run_translation(in_queue, out_queue, args):
+    import sys
+    from unittest.mock import MagicMock
+    # Comprehensive workaround for missing _lzma in some Python builds
+    try:
+        import lzma
+    except ImportError:
+        mock_lzma = MagicMock()
+        mock_lzma.FORMAT_XZ, mock_lzma.FORMAT_ALONE, mock_lzma.FORMAT_RAW = 1, 2, 3
+        mock_lzma.CHECK_NONE, mock_lzma.CHECK_CRC32, mock_lzma.CHECK_CRC64, mock_lzma.CHECK_SHA256 = 0, 1, 4, 10
+        sys.modules["_lzma"] = MagicMock()
+        sys.modules["lzma"] = mock_lzma
+
+    import torch
+    from transformers import MarianMTModel, MarianTokenizer
+
+    TARGET_LANGS = {
+        "es": "Helsinki-NLP/opus-mt-en-es",
+        "fr": "Helsinki-NLP/opus-mt-en-fr",
+        "ar": "Helsinki-NLP/opus-mt-en-ar"
+    }
+
+    def split_text_for_translation(text, max_chars=250):
+        normalized = " ".join(text.split()).strip()
+        if not normalized or len(normalized) <= max_chars: return [normalized] if normalized else []
+        chunks, current = [], ""
+        sentences = [s for s in re.split(r"(?<=[.!?])\s+", normalized) if s]
+        for sentence in sentences:
+            if len(sentence) > max_chars:
+                words, word_chunk = sentence.split(), ""
+                for word in words:
+                    candidate = f"{word_chunk} {word}".strip()
+                    if len(candidate) <= max_chars: word_chunk = candidate
+                    else:
+                        if word_chunk: chunks.append(word_chunk)
+                        word_chunk = word
+                if word_chunk: chunks.append(word_chunk)
+                continue
+            candidate = f"{current} {sentence}".strip()
+            if candidate and len(candidate) <= max_chars: current = candidate
+            else:
+                if current: chunks.append(current)
+                current = sentence
+        if current: chunks.append(current)
+        return chunks
+
+    device = "mps" if torch.backends.mps.is_available() else "cpu"
+    print(f"[Translate] Loading translation models on {device}...")
+    
+    translation_engines = {}
+    for lang_key, model_id in TARGET_LANGS.items():
+        if getattr(args, lang_key, False):
+            print(f"[Translate] Loading {lang_key} model...")
+            translation_engines[lang_key] = (
+                MarianMTModel.from_pretrained(model_id).to(device),
+                MarianTokenizer.from_pretrained(model_id)
+            )
+
+    while True:
+        try:
+            item = in_queue.get()
+            if item is None: break
+            if "draft" in item:
+                out_queue.put(item)
+                continue
+
+            raw_text, corrected_text, paragraph_text = item.get("raw"), item.get("corrected"), item.get("paragraph")
+            payload = {
+                "original": raw_text, 
+                "corrected": corrected_text, 
+                "paragraph": paragraph_text, 
+                "en_bridge": item.get("en_bridge"),
+                "used_llm_line": item.get("used_llm_line", False),
+                "used_llm_paragraph": item.get("used_llm_paragraph", False),
+                "paragraph_fallback": bool(paragraph_text) and not item.get("used_llm_paragraph", False),
+                "speaker": item.get("speaker"),
+                "ts": item.get("ts", time.time())
+            }
+            
+            if args.only_translate_llm:
+                text_to_translate = paragraph_text
+            else:
+                text_to_translate = paragraph_text or corrected_text or raw_text
+
+            english_output = text_to_translate or ""
+            if english_output:
+                payload["english_output"] = english_output
+            if args.en and english_output:
+                payload["en"] = english_output
+
+            if text_to_translate and translation_engines:
+                for lang_key, (model, tokenizer) in translation_engines.items():
+                    chunks = split_text_for_translation(text_to_translate, args.mt_max_chars)
+                    translated_parts = []
+                    for chunk in chunks:
+                        inputs = tokenizer(chunk, return_tensors="pt", padding=True).to(device)
+                        with torch.no_grad():
+                            translated_tokens = model.generate(**inputs, max_new_tokens=args.mt_max_new_tokens, num_beams=args.mt_num_beams)
+                        translated_parts.append(tokenizer.decode(translated_tokens[0], skip_special_tokens=True).strip())
+                    payload[lang_key] = " ".join(translated_parts)
+            
+            # ALWAYS put in queue, even if no translations were done
+            out_queue.put(payload)
+        except Exception as e: print(f"[Translate] Error: {e}")
+
+if __name__ == "__main__":
+    run_translation(multiprocessing.Queue(), multiprocessing.Queue(), argparse.Namespace())

freeflow_polish.py

@@ -0,0 +1,317 @@
+import re
+
+
+# Python port of the Apache-2.0 FreeFlow polish pipeline ideas:
+# https://github.com/mrinalwadhwa/freeflow
+
+SYSTEM_PROMPT_QWEN = """Clean up this dictated text.
+Fix punctuation and capitalization.
+Return only the cleaned text."""
+
+SYSTEM_PROMPT_MINIMAL = """You are a speech-to-text cleanup assistant. The user dictated text in a non-English language and a speech-to-text engine transcribed it. Your job is to clean up the transcription into polished written text.
+
+Fix filler words, false starts, consecutive repetitions, mid-sentence corrections, punctuation, capitalization, and obvious number formatting. Keep the user's original words. Do not fabricate text. Do not translate. Preserve every <keep>...</keep> block exactly as it appears.
+
+If the transcription is already clean, return it unchanged. Return only the cleaned text."""
+
+SYSTEM_PROMPT_ENGLISH = """You are a speech-to-text cleanup assistant. The user dictated text and a speech-to-text engine transcribed it. Your job is to clean up the transcription into polished written text. If the transcription is already clean, return it unchanged. Do not wrap your output in quotes or add any preamble. Return only the cleaned text. If the input starts with a lowercase letter, keep it lowercase. This happens when the user is continuing a sentence.
+
+Fix punctuation, capitalization, and spelling. Prefer periods, commas, or colons over em-dashes unless an em-dash appears inside a <keep> tag.
+
+When the input already contains explicit punctuation like "!", "?", ":", or ";", preserve it.
+
+When "period" appears at the end of a clause and does not make sense as a noun, treat it as a punctuation command and replace it with ".".
+
+Remove filler words and throat-clearing preambles that add no content. Remove unintentional stuttered repetitions. When the speaker corrects themselves with phrases like "no wait", "actually", "sorry", "I mean", "let me rephrase", "never mind", "or rather", or "make that", drop the abandoned version and keep the final version.
+
+Convert spelled-out numbers to digits where appropriate. Format times, phone numbers, ratios, and currency when the speaker clearly dictated them.
+
+When the speaker mentions 3 or more items, format them as a vertical list. Use numbered items if the order is explicit; otherwise use bullets. Keep 2-item phrases inline.
+
+Keep the speaker's word choices. Do not substitute synonyms, rephrase sentences, expand contractions, or add words the speaker did not say. Preserve informal words such as kinda, gonna, wanna, dunno, lemme, sorta, and gotta.
+
+When "at" appears between a name and a domain, and "dot" appears before the TLD, format it as an email address.
+
+The input may contain <keep>...</keep> tags around symbols inserted by preprocessing. Preserve every <keep>...</keep> block in your output exactly as-is, including the tags themselves."""
+
+
+PUNCTUATION_RULES = [
+    (r"\bnew paragraph\b", "[PAR]", True),
+    (r"\bnew line\b", "[NL]", True),
+    (r"\bnewline\b", "[NL]", True),
+    (r"\bquestion mark\b", "?", False),
+    (r"\bexclamation point\b", "!", False),
+    (r"\bexclamation mark\b", "!", False),
+    (r"\bcomma\b", ",", False),
+    (r"\bcolon\b", ":", False),
+    (r"\bsemicolon\b", ";", False),
+    (r"\bem dash\b", "\u2014", True),
+    (r"\ben dash\b", "\u2013", True),
+    (r"\bhyphen\b", "-", True),
+    (r"\bminus\s+(?:sign|symbol)\b", "-", True),
+    (r"\bopen paren(?:t|thesis)?\b", "(", False),
+    (r"\bclose paren(?:t|thesis)?\b", ")", False),
+    (r"\bopen quote\b", "\u201c", False),
+    (r"\b(?:close|end) quote\b", "\u201d", False),
+    (r"\bunquote\b", "\u201d", False),
+    (r"\b(?:apostrophe|single quote)\b", "'", False),
+    (r"\bopen bracket\b", "[", False),
+    (r"\bclose bracket\b", "]", False),
+    (r"\b(?:open )?angle bracket\b", "<", True),
+    (r"\bless[- ]than sign\b", "<", True),
+    (r"\bclose angle bracket\b", ">", True),
+    (r"\bgreater[- ]than sign\b", ">", True),
+    (r"\bdot dot dot\b", "\u2026", True),
+    (r"\bellipsis\b", "\u2026", True),
+    (r"\b(?:ampersand|and sign|and symbol)\b", "&", True),
+    (r"\b(?:at sign|at symbol)\b", "@", True),
+    (r"\bhashtag\b", "#", True),
+    (r"\b(?:back ?slash|slash en)\b", "\\", True),
+    (r"\bforward slash\b", "/", True),
+    (r"\b(?:asterisk|asterisk sign)\b", "*", True),
+    (r"\bunderscore\b", "_", True),
+    (r"\b(?:percent sign|per cent|percentage symbol)\b", "%", True),
+    (r"\bdollar sign\b", "$", True),
+    (r"\b(?:equals sign|equals symbol)\b", "=", True),
+    (r"\b(?:plus sign|plus symbol)\b", "+", True),
+    (r"\btrademark sign\b", "\u2122", True),
+    (r"\btm\b", "\u2122", True),
+    (r"\bcopyright sign\b", "\u00a9", True),
+    (r"\bcopyright symbol\b", "\u00a9", True),
+    (r"\bdegrees?\s+fahrenheit\b", "\u00b0F", True),
+    (r"\bdegrees?\s+f\b", "\u00b0F", True),
+    (r"\bdegrees?\s+celsius\b", "\u00b0C", True),
+    (r"\bdegrees?\s+centigrade\b", "\u00b0C", True),
+    (r"\b(?:degree sign|degree symbol)\b", "\u00b0", True),
+]
+
+KNOWN_TERMS = {
+    "redis": "Redis",
+    "postgresql": "PostgreSQL",
+    "postgres": "Postgres",
+    "mongodb": "MongoDB",
+    "mysql": "MySQL",
+    "sqlite": "SQLite",
+    "kubernetes": "Kubernetes",
+    "docker": "Docker",
+    "terraform": "Terraform",
+    "nginx": "Nginx",
+    "grafana": "Grafana",
+    "prometheus": "Prometheus",
+    "cloudflare": "Cloudflare",
+    "vercel": "Vercel",
+    "rabbitmq": "RabbitMQ",
+    "kafka": "Kafka",
+    "javascript": "JavaScript",
+    "typescript": "TypeScript",
+    "python": "Python",
+    "swift": "Swift",
+    "rust": "Rust",
+    "react": "React",
+    "django": "Django",
+    "fastapi": "FastAPI",
+    "nextjs": "Next.js",
+    "nodejs": "Node.js",
+    "numpy": "NumPy",
+    "pandas": "Pandas",
+    "pytorch": "PyTorch",
+    "github": "GitHub",
+    "slack": "Slack",
+    "figma": "Figma",
+    "macos": "macOS",
+    "ios": "iOS",
+    "iphone": "iPhone",
+    "xcode": "Xcode",
+    "aws": "AWS",
+    "gcp": "GCP",
+    "api": "API",
+    "sql": "SQL",
+    "css": "CSS",
+    "html": "HTML",
+    "json": "JSON",
+    "xml": "XML",
+    "yaml": "YAML",
+    "url": "URL",
+    "cli": "CLI",
+    "sdk": "SDK",
+}
+
+
+def ends_at_sentence_boundary(text):
+    stripped = (text or "").rstrip()
+    return bool(stripped) and stripped[-1] in ".?!"
+
+
+def _capitalize_after_pattern(text, pattern):
+    def repl(match):
+        return match.group(1) + match.group(2).upper()
+
+    return re.sub(pattern, repl, text)
+
+
+def collapse_adjacent_punctuation(text):
+    strength = {",": 1, ":": 2, ";": 3, ".": 4, "?": 5, "!": 6}
+
+    def repl(match):
+        marks = [ch for ch in match.group(0) if ch in strength]
+        return max(marks, key=lambda ch: strength[ch]) if marks else match.group(0)
+
+    return re.sub(r"([.,;:?!])(?:\s*[.,;:?!])+", repl, text)
+
+
+def strip_noise_phrases(text):
+    text = re.sub(r"(?i)\b(?:uh huh|uh-huh|mm hmm|mm-hmm)\b[,.]?\s*", "", text)
+    return re.sub(r" {2,}", " ", text).strip()
+
+
+def strip_filler_sounds(text):
+    fillers = "um|eh|mmm|uhh|hm|umm|mm|uh|uhhh|uhm|ah|hmm|mh|ehh"
+    text = re.sub(rf"(?i)\b(?:{fillers})\b[,.]?\s*", "", text)
+    return re.sub(r" {2,}", " ", text).strip()
+
+
+def clean_spurious_commas(text):
+    text = re.sub(r",{2,}", ",", text)
+    text = re.sub(r",\s*(AM|PM|a\.m\.|p\.m\.)", r" \1", text)
+    text = re.sub(r",(\s*[.!?])", r"\1", text)
+    text = re.sub(r",\s*$", "", text)
+    text = re.sub(r",\s*,", ",", text)
+    return re.sub(r" {2,}", " ", text).strip()
+
+
+def substitute_dictated_punctuation(text, preceding_text=None, casual=False):
+    result = text or ""
+    for pattern, replacement, protect in PUNCTUATION_RULES:
+        value = f"<keep>{replacement}</keep>" if protect else replacement
+        result = re.sub(pattern, lambda _m, v=value: v, result, flags=re.IGNORECASE)
+
+    result = result.replace("...", "<keep>\u2026</keep>")
+    result = re.sub(r" +([.,;:?!)\]\u201d])", r"\1", result)
+    result = re.sub(r"([(\[\u201c]) +", r"\1", result)
+    result = re.sub(r" {2,}", " ", result)
+    result = collapse_adjacent_punctuation(result)
+    result = re.sub(r"[,;]\s*(<keep>\[(?:PAR|NL)\]</keep>)", r".\1", result)
+    result = re.sub(r"([^.!?\s])\s*(<keep>\[(?:PAR|NL)\]</keep>)", r"\1.\2", result)
+    result = re.sub(r" *\n *", "\n", result)
+
+    if casual:
+        if result[:1].isupper():
+            first_word = re.match(r"[A-Za-z]+", result)
+            token = first_word.group(0) if first_word else ""
+            if token != "I" and not (len(token) > 1 and token.isupper()):
+                result = result[0].lower() + result[1:]
+    else:
+        result = _capitalize_after_pattern(result, r"([.!?]\s+)(\w)")
+        if not (preceding_text and not ends_at_sentence_boundary(preceding_text)):
+            result = result[:1].upper() + result[1:] if result[:1].isalpha() else result
+
+    result = clean_spurious_commas(result)
+    result = strip_noise_phrases(result)
+    result = strip_filler_sounds(result)
+    return result.strip()
+
+
+def strip_keep_tags(text, casual=False):
+    result = re.sub(r"<keep>(.*?)</keep>", r"\1", text or "", flags=re.DOTALL)
+    result = re.sub(r" *\[PAR\] *", "\n\n", result)
+    result = re.sub(r" *\[NL\] *", "\n", result)
+    result = re.sub(r" *\u00b6 *", "\n\n", result)
+    result = re.sub(r" *\u21b5 *", "\n", result)
+    result = re.sub(r" +([.,;:?!)\]>\u201d\u2026%\u2122\u00a9\u00b0])", r"\1", result)
+    result = re.sub(r"([(\[\u201c#$<]) +", r"\1", result)
+    result = re.sub(r" *([-@/\\_'\u2013\u2014]) +", r"\1", result)
+    result = re.sub(r"\*[\w* ]*\*", lambda m: m.group(0).replace(" ", ""), result)
+    result = re.sub(r" {2,}", " ", result)
+    if not casual:
+        result = _capitalize_after_pattern(result, r"(\n)(\w)")
+    return result.strip()
+
+
+def capitalize_known_terms(text):
+    result = text
+    for source, replacement in KNOWN_TERMS.items():
+        result = re.sub(rf"(?<!/)\b{re.escape(source)}\b", replacement, result, flags=re.IGNORECASE)
+    return result
+
+
+def normalize_formatting(text, casual=False):
+    result = text or ""
+    result = re.sub(r" *\[PAR\] *", "\n\n", result)
+    result = re.sub(r" *\[NL\] *", "\n", result)
+    result = re.sub(r" *\u00b6 *", "\n\n", result)
+    result = re.sub(r" *\u21b5 *", "\n", result)
+    result = re.sub(r"(?<!:)//", "/", result)
+    result = re.sub(r"(?<=\w)\\\\(?=\w)", r"\\", result)
+    result = re.sub(r"\ba\.m\.(?= )", "AM", result)
+    result = re.sub(r"\ba\.m\.(?=$|\n)", "AM.", result)
+    result = re.sub(r"\bp\.m\.(?= )", "PM", result)
+    result = re.sub(r"\bp\.m\.(?=$|\n)", "PM.", result)
+    lines = []
+    for line in result.split("\n"):
+        line = line.rstrip()
+        line = re.sub(r"^(\s*)-(\S)", r"\1- \2", line)
+        if line.endswith(".") and re.search(r"^\s*(?:-|\d+\.)\s+", line):
+            line = line[:-1]
+        lines.append(line)
+    result = "\n".join(lines)
+    return result if casual else capitalize_known_terms(result)
+
+
+def match_input_casing(text, preprocessed_input, casual=False):
+    if casual or not text or not preprocessed_input:
+        return text
+    if not text[0].isalpha() or not preprocessed_input[0].isalpha():
+        return text
+    if preprocessed_input[0].islower() and text[0].isupper():
+        first_word = re.match(r"[A-Za-z]+", text)
+        token = first_word.group(0) if first_word else ""
+        if len(token) > 1 and token.isupper():
+            return text
+        return text[0].lower() + text[1:]
+    if preprocessed_input[0].isupper() and text[0].islower():
+        return text[0].upper() + text[1:]
+    return text
+
+
+def guard_against_truncation(polished, preprocessed):
+    if len(preprocessed or "") < 40:
+        return None
+    if not polished:
+        return preprocessed
+    return preprocessed if len(polished) / len(preprocessed) < 0.25 else None
+
+
+def deterministic_polish(text, preceding_text=None, casual=False):
+    substituted = substitute_dictated_punctuation(text, preceding_text=preceding_text, casual=casual)
+    stripped = strip_keep_tags(substituted, casual=casual)
+    return normalize_formatting(stripped, casual=casual), substituted
+
+
+def build_user_prompt(text, language=None):
+    parts = [f"Transcription:\n{text}"]
+    if language:
+        parts.append(f"Language: {language}")
+    return "\n\n".join(parts)
+
+
+def system_prompt_for_language(language=None, model=""):
+    if "qwen" in (model or "").lower():
+        return SYSTEM_PROMPT_QWEN
+    if not language or language == "en":
+        return SYSTEM_PROMPT_ENGLISH
+    return SYSTEM_PROMPT_MINIMAL
+
+
+def is_clean_enough_to_skip_llm(text):
+    normalized = " ".join((text or "").split())
+    if not normalized:
+        return True
+    if normalized[0].isalpha() and not normalized[0].isupper():
+        return False
+    if not ends_at_sentence_boundary(normalized):
+        return False
+    if re.search(r"(?i)\b(um|uh|uhh|umm|ah|hmm|you know|i mean|no wait|actually|sorry|let me rephrase)\b", normalized):
+        return False
+    if re.search(r"(?i)\b(\w+)(?:\s+\1\b){1,}", normalized):
+        return False
+    return True

history.md

@@ -80,3 +80,238 @@ The project now features a high-performance, Apple Silicon-optimized pipeline th
     - Implemented a background `ingest_worker` thread to handle HTTP POST requests without stalling the audio processing.
     - **Flat JSON Schema:** Used a key-value format as requested (e.g., `original`, `es`, `en`, `fr`).
     - **Reliability:** Integrated exponential backoff retries (1s to 15s) to handle network or server failures.
+
+## Phase 10: CLI Configuration & Documentation
+- **Goal:** Make the tool fully configurable without editing the code.
+- **Approach:** 
+    - Integrated `argparse` to allow dynamic selection of translation languages (`-es`, `-en`, `-ar`, `-fr`), data ingestion (`-i`), and audio device (`-d`).
+    - Added device listing capability (`-l`).
+    - Excluded the `venv` directory from git and generated `requirements.txt`.
+    - Created a comprehensive `README.md` containing setup instructions, usage examples, and a technical note on universal models (like NLLB-200).
+- **Fix:** Added missing `sentencepiece` dependency required by MarianMT models.
+
+## Phase 11: Speed Optimization (Quantization & Streaming)
+- **Goal:** Reduce latency and improve real-time feedback.
+- **Approach:**
+    - **Streaming Mode (`-s`):** Implemented a 1-second rolling draft transcription that continuously updates the console while the user is still speaking.
+    - **Quantization (`-q`):** Added support for dynamically swapping to the 4-bit quantized Whisper model (`mlx-community/whisper-small-mlx-4bit`) for faster inference on Apple Silicon with lower memory bandwidth.
+
+## Phase 12: Accuracy & Context Optimization
+- **Goal:** Improve translation quality of short or broken audio chunks.
+- **Approach:**
+    - **Prompt Caching (`-c`):** Implemented a rolling context buffer that feeds the last 200 characters of previously transcribed text back into Whisper as an `initial_prompt`, maintaining sentence continuity.
+    - **Language Bypassing (`--lang`):** Added the ability to hardcode the source language to skip the Whisper language identification phase on every chunk.
+    - **Heuristic Punctuation Buffering (Reverted):** Briefly implemented a system to hold English translations until a definitive punctuation mark was reached to prevent grammatical errors. This was reverted because Whisper's punctuation generation is not 100% reliable, leading to translations getting "stuck" in the buffer indefinitely if no period was generated.
+
+## Phase 13: Stability & Stuck Detection (Watchdog)
+- **Goal:** Prevent captions from getting "stuck" during long periods of music, singing, or model hallucinations.
+- **Approach:**
+    - **Transcription Watchdog:** Implemented a logic that monitors the audio buffer duration and the time since the last *unique* draft change. If the buffer exceeds 12s without a change for 7s, or if it hits an absolute limit of 25s, it forces a clean reset.
+    - **Hallucination Filtering:** Added regex-based detection for common Whisper hallucinations (e.g., "Thanks for watching", "Please subscribe") and a repetition counter to identify and discard high-frequency loops (e.g., "Hallelujah" repeated 20 times).
+    - **Buffer Optimization:** Reduced the default maximum buffer from 30s to 20s (now configurable via `--max-buffer`) to ensure more frequent flushes during continuous sound.
+- **Outcome:** Significantly improved reliability during live church services and musical performances where Whisper previously tended to "hang" or hallucinate.
+
+## Phase 14: Multi-Channel & Language Filtering
+- **Goal:** Isolate specific speakers or languages when the audio feed contains multiple mixed sources.
+- **Approach:**
+    - **Stereo Channel Selection:** Added `--channels` and `--pick-channel [0|1]` flags. This allows the tool to pull a specific audio channel (e.g., English on Left, Translation on Right) from a stereo feed, ignoring the other.
+    - **Language Filtering (`--filter-lang`):** Integrated a mechanism to discard any segment where the detected language does not match the hardcoded `--lang` parameter. 
+- **Outcome:** Enabled the ability to "focus" the transcription engine on a single speaker even when the audio input is a complex mix.
+
+## Phase 15: Potential Future Optimizations (Backlog)
+- **Decoding Parameters:**
+    - **Temperature Fallback:** Use a tuple `(0.0, 0.2, 0.4, 0.6, 0.8, 1.0)` to allow Whisper to re-try failed transcriptions with higher randomness (crucial for music/singing).
+    - **Beam Size Tuning:** Set `beam_size=1` for Draft Mode (3x faster) and `beam_size=5` for final segments (higher accuracy).
+    - **Token Suppression:** Native suppression of music symbols or common hallucination tokens at the decoder level.
+    - **Heuristic Thresholds:** Utilize `logprob_threshold` (confidence) and `compression_ratio_threshold` (repetition) to programmatically identify and discard bad transcriptions before they reach the user.
+- **Structural Features:**
+    - **Word-Level Timestamps:** Enable `word_timestamps=True` to provide granular timing data for front-end caption highlighting.
+
+## Phase 16: Decoder Controls + Bridge Context Refinements
+- **Goal:** Improve English caption stability and translation quality without sacrificing runtime compatibility.
+- **Approach:**
+    - **Whisper Decoder Controls (implemented):** Added configurable decoding flags for final segments and drafts, including temperature fallback and quality thresholds (`logprob_threshold`, `compression_ratio_threshold`), with compatibility fallback when unsupported.
+    - **Beam Compatibility Hardening:** Defaulted beam sizes to greedy-safe values and auto-retry without `beam_size` when the runtime reports "Beam search decoder is not yet implemented."
+    - **Dedicated English Bridge Context:** Split context handling into source-language context and a separate English context used specifically for `task="translate"` bridge generation.
+    - **Sentence-Aware MT Chunking:** Replaced hard truncation with sentence-aware splitting for long English bridge text before translation.
+    - **Marian Generation Tuning:** Added configurable generation controls (`num_beams`, `no_repeat_ngram_size`, `length_penalty`, `repetition_penalty`, `early_stopping`) to reduce repetitive or unstable outputs.
+- **Outcome:** Better continuity for non-English to English bridging, fewer clipped translations on long segments, and cleaner target-language output with safer defaults for current `mlx_whisper` builds.
+
+## Phase 17: Intelligent English Caption Correction (V1)
+- **Goal:** Improve English caption readability and resilience without introducing heavy latency.
+- **Approach:**
+    - **Confidence-Triggered Re-Decode:** Added optional retry logic that re-transcribes low-confidence segments when Whisper quality signals indicate likely errors (`avg_logprob` / `compression_ratio` thresholds).
+    - **Glossary Replacements:** Added configurable `SOURCE=TARGET` term normalization (`--glossary-pair`) to consistently correct names/terms in English captions.
+    - **Rolling Caption Revision Buffer:** Added a short correction window (`--caption-correction-window`, default 3s) with a 2-line rolling buffer that can merge incomplete English lines into a cleaner sentence as new context arrives.
+- **Outcome:** Captions now support lightweight post-correction behavior that improves continuity and term consistency while remaining compatible with real-time streaming.
+
+## Phase 18: File-Based Glossary Loading
+- **Goal:** Make glossary corrections persistent and easier to maintain without long CLI commands.
+- **Approach:**
+    - Added `--glossary-file` (default `glossary.txt`) to load `SOURCE=TARGET` term mappings when the file exists.
+    - Implemented comment/blank-line support and invalid-line skipping with line-level warnings.
+    - Merged file-based glossary entries with repeatable `--glossary-pair` CLI entries for runtime overrides.
+- **Outcome:** English caption correction can now use a maintained glossary file automatically, while preserving ad-hoc CLI term fixes.
+
+## Phase 19: Arabic Terminal Rendering Hardening
+- **Goal:** Improve readability of Arabic captions in terminal environments with mixed LTR/RTL output.
+- **Approach:**
+    - Switched Arabic output to a strict two-line format (`[AR]:` label line + dedicated RTL content line).
+    - Wrapped Arabic caption content in explicit RTL embedding marks for better bidirectional layout stability.
+    - Added punctuation normalization for Arabic display (`،`, `؛`, `؟`) to reduce LTR punctuation artifacts.
+- **Outcome:** Arabic captions render more consistently in terminal output, especially when adjacent to English/French/Spanish caption lines.
+
+## Phase 20: Post-Correct Mode (V2 Foundation)
+- **Goal:** Add a practical second-stage correction pass for finalized English captions.
+- **Approach:**
+    - Added `--post-correct` mode to run deterministic rule-based cleanup after ASR (spacing, punctuation cleanup, disfluency reduction, immediate duplicate-word collapse).
+    - Added optional local LLM correction via Ollama (`--post-correct-llm`, model/url/timeout flags) with strict fallback to rules-only when unavailable.
+    - Integrated post-correction after bridge-to-English and before downstream translation so improved English text propagates to target-language MT.
+- **Outcome:** The pipeline now supports a two-stage transcription flow (ASR -> correction) with low-latency defaults and optional local generative enhancement.
+
+## Phase 21: LLM Merge-Decider for Line Revisions
+- **Goal:** Improve English line-merge quality when smart-correct revision is ambiguous.
+- **Approach:**
+    - Added optional `--llm-merge-decider` to validate/refine heuristic merge candidates using the local post-correct LLM.
+    - Enforced strict low-latency behavior with `--llm-merge-timeout` (default 0.7s) and automatic fallback to heuristic merges on errors/timeouts.
+    - Added safety guards to reject overly large LLM rewrites and preserve buffered caption state when merge is rejected.
+- **Outcome:** Merge decisions remain fast and deterministic by default, with optional LLM arbitration for cleaner sentence continuity.
+
+## Phase 22: Optional Speaker-Change Line Cutting
+- **Goal:** Split long live captions earlier when a different person starts speaking.
+- **Approach:**
+    - Added optional `--speaker-change-detect` mode with cosine-similarity comparison of lightweight voice signatures from buffered audio.
+    - Introduced tuning flags for sensitivity and runtime cost (`--speaker-sim-threshold`, `--speaker-min-buffer`, `--speaker-check-interval`).
+    - When a probable speaker switch is detected, the current buffered caption line is force-flushed early instead of waiting only for silence.
+    - Added speaker-state resets on watchdog/hallucination/silence reset paths to avoid stale identity drift.
+- **Outcome:** Users can opt into faster caption segmentation at speaker boundaries while keeping default behavior unchanged.
+
+## Phase 23: Prompt Hardening + Anti-Drift Guardrails
+- **Goal:** Reduce semantic drift in LLM post-correction and merge decisions.
+- **Approach:**
+    - Rewrote post-correction prompt with strict "current line only" and "if uncertain, keep original" constraints.
+    - Expanded post-correction context to up to two previous lines but marked as reference-only.
+    - Added token-overlap acceptance guard (`--post-correct-min-overlap`) to reject LLM rewrites that diverge too far from source text.
+    - Updated merge-decider prompt to stricter JSON behavior and added overlap-based fallback to heuristic merge.
+- **Outcome:** Improved protection against context bleed (e.g., replacing current line with prior sentence) while keeping optional LLM improvements.
+
+## Phase 24: Ollama Warmup + Keep-Alive
+- **Goal:** Reduce intermittent post-correction timeouts caused by cold model loads.
+- **Approach:**
+    - Added startup warmup request for the local post-correct model when LLM correction is enabled.
+    - Added configurable Ollama `keep_alive` setting so the model stays resident between caption calls.
+    - Routed post-correct and merge-decider requests through a shared Ollama caller with keep-alive support.
+    - Added flags for warmup timeout and optional warmup skip for troubleshooting.
+- **Outcome:** Lower first-request latency and fewer fallback-to-rules events due to local model cold starts.
+
+## Phase 25: Session Logging + Tuned Live Preset
+- **Goal:** Preserve reliable debugging context while locking in the best-performing live settings discovered through manual testing.
+- **Approach:**
+    - Added session file logging hooks so finalized captions, revisions, startup state, and runtime errors can be written to a dedicated log via `--session-log-file`.
+    - Improved interactive terminal behavior by clearing stale draft lines cleanly and right-aligning Arabic output more consistently against terminal width.
+    - Recorded the current preferred live preset for English-first filtered transcription with multilingual output and local LLM post-correction:
+    ```bash
+    python3 transcribe.py --silence 100 -q -s -c -es -en -fr -ar --lang en --filter-lang --mt-max-chars 450 --mt-max-new-tokens 220 --smart-correct --post-correct --post-correct-llm --post-correct-model qwen2.5:3b-instruct --session-log-file debug_session_1.log
+    ```
+- **Outcome:** The project now has a repeatable "known good" runtime profile plus persistent session diagnostics for reviewing caption issues after a run.
+
+## Phase 26: Multi-Process Decoupled Architecture
+- **Goal:** Improve system stability, reduce latency, and fully decouple audio capture from heavy LLM/Translation tasks.
+- **Approach:**
+    - **4-Process Pipeline:** Refactored the monolithic script into four independent services coordinated via `multiprocessing.Queue`:
+        1. **`engine_transcribe.py` (Whisper):** Dedicated to high-priority audio capture and ASR.
+        2. **`engine_llm.py` (Ollama):** Handles asynchronous post-correction and paragraph structuring without blocking the transcription loop.
+        3. **`engine_translate.py` (MarianMT):** Manages multi-language translation for both raw and refined text.
+        4. **`engine_distribute.py` (API/CLI):** Handles data delivery and terminal display.
+    - **Centralized Configuration:** Introduced `config.json` for managing all defaults and parameters in one place, with `main_v2.py` as the entry point.
+    - **Dual Payload Strategy:** Implemented a robust data flow where the translation and distribution engines receive both raw and LLM-corrected versions of the text, allowing for fallback and comparison.
+    - **Hardware Isolation:** Transcription and Translation processes independently leverage Apple Silicon (MPS), while the LLM process utilizes Ollama's external server, preventing resource contention.
+- **Outcome:** Significantly increased resilience. If the LLM or Translation engine stalls, the Transcription engine continues to capture and buffer audio safely, preventing data loss.
+
+## Phase 27: Intelligent LLM Refinement & Pipeline Hardening
+- **Goal:** Transform raw ASR segments into professional paragraphs and stabilize advanced features.
+- **Approach:**
+    - **Accumulative LLM Engine:** Developed a stateful refinement logic in `engine_llm.py` that maintains a "working paragraph," allowing the LLM to continuously integrate and polish new segments in real-time.
+    - **OpenAI Integration:** Added direct support for OpenAI's GPT API (via `requests` to avoid subprocess dependency issues), enabling higher-quality refinement than lightweight local models.
+    - **Rolling Context Window:** Implemented a smart context management system that "finalizes" paragraphs once they reach a natural break (detected via `\n\n`), clearing the prompt history to save tokens and maintain focus.
+    - **Speaker Diarization Hardening:** Refactored `engine_transcribe.py` to support `pyannote.audio` 4.0.4, specifically handling the new `DiarizeOutput` object structure and ensuring speaker labels (`SPEAKER_XX`) propagate through the entire multi-process pipeline.
+    - **English Bridge Optimization:** Forced a two-pass transcription strategy (ASR first, then optional translation) to ensure word-level timestamps are captured for speaker mapping even when translating to English.
+    - **Stability Fixes:** Added hallucination filtering for Whisper's repetitive loops, enforced line-buffering for terminal logging, and implemented a global `lzma` mock to ensure compatibility across restricted Python environments.
+- **Outcome:** A robust, production-ready pipeline that produces high-quality, speaker-attributed, and professionally formatted live transcripts.
+
+## Phase 28: `main_v2` Bridge Contract Fixes + Live Reliability Review
+- **Goal:** Repair stage-contract regressions in the new multi-process pipeline and preserve live throughput under failure.
+- **Review Findings:**
+    - The Whisper English bridge in `engine_transcribe.py` reused source-language prompt context for `task="translate"` instead of a dedicated English context, which could bias or degrade translated bridge text.
+    - `--filter-lang` was exposed in `main_v2.py` but not enforced in the new transcription engine, so wrong-language speech could still be bridged and propagated downstream.
+    - `engine_llm.py` told the paragraph refiner to output the source language even though downstream Marian models require English input, breaking `llm_paragraph` for non-English sources.
+    - `engine_translate.py` could translate one English variant while publishing a different `[EN]` line, making the visible source text diverge from what target-language MT actually used.
+    - `engine_distribute.py` retried ingest forever in the queue consumer, allowing a network outage to stall the whole live pipeline.
+    - `main_v2.py` exposed post-correction settings that the new pipeline did not fully honor, making CLI behavior drift from the documented workflow.
+- **Approach:**
+    - Restored a dedicated rolling English prompt context for the Whisper bridge pass and rebuilt bridge kwargs explicitly instead of copying source-ASR kwargs.
+    - Reinstated `--filter-lang` enforcement before bridge generation and reduced unnecessary ASR cost by only asking Whisper for word timestamps when diarization is enabled.
+    - Updated the LLM stage so paragraph refinement always preserves English bridge text, added deterministic post-correction plus optional LLM line cleanup, and provided a safe fallback paragraph when the LLM backend is unavailable.
+    - Aligned published English output with the exact text chosen for translation and added the missing Ollama-related config/plumbing to `main_v2`.
+    - Replaced infinite ingest retry loops with bounded retry logic so failed delivery degrades gracefully instead of freezing caption flow.
+- **Outcome:** The `main_v2` pipeline now keeps an English-first contract across transcription, refinement, and Marian translation while behaving more predictably under API outages and mixed-language input.
+
+## Phase 29: Output Semantics + Model Defaults Cleanup
+- **Goal:** Make the live console reflect the real pipeline stages and choose safer small-model defaults for local LLM use.
+- **Approach:**
+    - Split raw ASR, Whisper English bridge, line-level LLM corrections, and paragraph output into distinct console labels instead of grouping multiple stages under `[LLM]`.
+    - Updated the default local post-correction model configuration to prefer a smaller Qwen 3.5 variant for experimentation, then kept the code flexible so model choice can be overridden per run.
+    - Added metadata flow through the translation/distribution stages so downstream output can distinguish true LLM paragraphs from deterministic paragraph fallback.
+- **Outcome:** Terminal output is easier to interpret during multilingual runs, especially when comparing source ASR against the English bridge and later refinement stages.
+
+## Phase 30: Local LLM Warmup + Fallback Transparency
+- **Goal:** Reduce cold-start timeout confusion for local Ollama-backed correction and paragraph modes.
+- **Approach:**
+    - Added Ollama startup health checks and model presence checks before live processing begins.
+    - Added a warmup request for local models so working Ollama models can pay the cold-load cost before live captions arrive.
+    - Split paragraph output labels into true LLM paragraphs versus deterministic fallback paragraphs to avoid overstating when a model actually answered.
+- **Outcome:** Local-model failures are surfaced earlier, cold-start behavior is easier to understand, and fallback output is clearly labeled.
+
+## Phase 31: OpenAI/Ollama Prompt Parity + Better API Diagnostics
+- **Goal:** Make remote and local LLM behavior easier to compare and make API failures actionable.
+- **Approach:**
+    - Improved OpenAI error reporting to print the full API error body instead of only a generic HTTP 400 message.
+    - Removed the explicit Chat Completions temperature override after discovering current GPT-5-family models reject non-default temperature values on this endpoint.
+    - Aligned the Ollama path with the OpenAI path by forwarding the same system instructions to local generation requests, including warmup calls.
+- **Outcome:** GPT-backed runs now fail with useful diagnostics, OpenAI requests are compatible with current GPT-5-mini behavior, and local-vs-remote prompting is more consistent.
+
+## Phase 32: LLM Request Logging + Prompt Replay Harness
+- **Goal:** Make prompt behavior observable and reproducible so local LLM tuning can happen without rerunning live transcription.
+- **Approach:**
+    - Added JSONL request logging for every LLM call, including full message payloads, timing, and the final response or error outcome.
+    - Added prompt-only test entry points in `main_v2.py` so line-correction and paragraph-refinement prompts can be sent directly without starting the transcription pipeline.
+    - Added replay-from-log support so a captured request can be resent with the current model or the originally logged model.
+    - Documented the new logging and replay workflow in `README.md` and added a default log path in `config.json`.
+- **Outcome:** We can now inspect exactly what the app sends to the LLM, replay real requests on demand, and compare prompt changes against stored payloads and outputs.
+
+## Phase 33: Grounded Paragraph Prompt Experiments
+- **Goal:** Reduce paragraph drift and self-reinforcing hallucinations in the rolling LLM refiner.
+- **Approach:**
+    - Built `prompt_experiments.py` to reconstruct representative paragraph-refinement samples from `distribute_debug.log` and send multiple prompt variants directly to Ollama.
+    - Compared several context arrangements, including the previous refined paragraph alone, raw-only windows, and mixed refined-plus-raw grounding strategies.
+    - Identified the best-performing variant as one that treats the new source window as primary evidence while using prior refined and prior raw text only when they clearly continue the thought.
+    - Updated `engine_llm.py` so live paragraph refinement now sends `PREVIOUS REFINED CONTEXT`, `PREVIOUS SOURCE TEXT`, and `NEW SOURCE TEXT` with a more conservative, grounding-oriented system prompt.
+- **Outcome:** Paragraph refinement is now substantially better anchored to current source text, with less stale-context carryover and a much lower rate of unsupported content in direct Ollama tests.
+
+## Phase 34: Multilingual Noise Filtering + Paragraph Generation Sweep
+- **Goal:** Make paragraph refinement more robust when stray Spanish or other non-English fragments leak into the English bridge text.
+- **Approach:**
+    - Extended `prompt_experiments.py` with multilingual-noise test cases that inject Spanish, Portuguese, and French fragments into reconstructed live samples.
+    - Ran direct-Ollama comparisons across multiple paragraph prompt variants and generation settings, including temperature, `top_p`, and `repeat_penalty`.
+    - Found that the best overall paragraph generation settings for `qwen2.5:3b-instruct` remained conservative: `temperature=0.2`, `top_p=0.8`, `repeat_penalty=1.0`.
+    - Tightened the live paragraph system prompt in `engine_llm.py` so output must stay English-only, non-English fragments should not be copied through, isolated trailing foreign fragments should be dropped, and only clearly central foreign content should be translated into English.
+    - Added paragraph-specific generation controls to `main_v2.py` and `config.json` so paragraph tuning can evolve independently of line-correction behavior.
+- **Outcome:** Paragraph refinement is now stricter about keeping the final output in English while preserving the more grounded context layout and the best-tested Ollama generation settings.
+
+## Phase 35: Previous-Context Scope + Previous-Source Final Check
+- **Goal:** Decide whether the paragraph refiner should keep receiving previous source text when experimenting with multiple prior refined paragraphs as context.
+- **Approach:**
+    - Extended `prompt_experiments.py` to compare `2` and `4` previous refined paragraphs with and without `PREVIOUS SOURCE TEXT`.
+    - Ran the final focused comparison on later reconstructed samples using deterministic generation (`temperature=0.0`) to minimize sampling noise.
+    - Compared the resulting tradeoffs in unsupported content, repetition, and output length instead of looking only at one-off wins.
+- **Outcome:** Keeping `PREVIOUS SOURCE TEXT` remains the safer choice. The best result from this final sweep was `2` previous refined paragraphs plus previous source text; removing previous source text consistently hurt grounding, while larger refined-context windows increased drift risk.

main_v2.py

@@ -0,0 +1,152 @@
+import multiprocessing
+import argparse
+import json
+import os
+import sys
+from dotenv import load_dotenv
+
+# Load environment variables from .env if present
+load_dotenv()
+
+# Light imports (heavy ones moved inside run functions)
+from engine_transcribe import run_transcription, list_audio_devices
+from engine_llm import run_llm_processor, run_llm_prompt_test
+from engine_translate import run_translation
+from engine_distribute import run_distribution
+
+def parse_temperature_fallback(value):
+    try:
+        return tuple(float(x.strip()) for x in value.split(",") if x.strip())
+    except:
+        return (0.0, 0.2, 0.4, 0.6, 0.8, 1.0)
+
+def main():
+    config_path = "config.json"
+    defaults = {}
+    if os.path.exists(config_path):
+        with open(config_path, "r") as f:
+            defaults = json.load(f)
+
+    parser = argparse.ArgumentParser(description="Multi-process Transcription and Translation.")
+    
+    # Transcribe Args
+    parser.add_argument("-l", "--list-devices", action="store_true", help="Show available audio devices and exit")
+    parser.add_argument("--model", type=str, default=defaults.get("model", "mlx-community/whisper-base-mlx"))
+    parser.add_argument("--device", type=int, default=defaults.get("device"))
+    parser.add_argument("--lang", type=str, default=defaults.get("lang"))
+    parser.add_argument("--silence", type=int, default=defaults.get("silence", 1000))
+    parser.add_argument("--max-buffer", type=int, default=defaults.get("max_buffer", 20))
+    parser.add_argument("--channels", type=int, default=defaults.get("channels", 1))
+    parser.add_argument("--filter-lang", action="store_true", default=defaults.get("filter_lang", False))
+    parser.add_argument("-q", "--quantize", action="store_true", default=defaults.get("quantize", False))
+    parser.add_argument("-s", "--stream", action="store_true", default=defaults.get("stream", False))
+    parser.add_argument("-c", "--context", action="store_true", default=defaults.get("context", False))
+    parser.add_argument("--speaker-diarization", action="store_true", default=defaults.get("speaker_diarization", False))
+    parser.add_argument("--temperature-fallback", type=parse_temperature_fallback, default=tuple(defaults.get("temperature_fallback", [0.0, 0.2, 0.4, 0.6, 0.8, 1.0])))
+    parser.add_argument("--logprob-threshold", type=float, default=defaults.get("logprob_threshold", -0.8))
+    parser.add_argument("--compression-threshold", type=float, default=defaults.get("compression_threshold", 2.2))
+    
+    # LLM Args
+    parser.add_argument("--post-correct", action="store_true", default=defaults.get("post_correct", False))
+    parser.add_argument("--post-correct-llm", action="store_true", default=defaults.get("post_correct_llm", False))
+    parser.add_argument("--post-correct-model", type=str, default=defaults.get("post_correct_model", "qwen3.5:0.8b"))
+    parser.add_argument("--post-correct-ollama-url", type=str, default=defaults.get("post_correct_ollama_url", "http://127.0.0.1:11434/api/generate"))
+    parser.add_argument("--post-correct-llm-timeout", type=float, default=defaults.get("post_correct_llm_timeout", 8.0))
+    parser.add_argument("--post-correct-keep-alive", type=str, default=defaults.get("post_correct_keep_alive", "30m"))
+    parser.add_argument("--post-correct-warmup-timeout", type=float, default=defaults.get("post_correct_warmup_timeout", 20.0))
+    parser.add_argument("--post-correct-min-overlap", type=float, default=defaults.get("post_correct_min_overlap", 0.45))
+    parser.add_argument("--post-correct-debug", action="store_true", default=defaults.get("post_correct_debug", False))
+    parser.add_argument("--freeflow-polish", action="store_true", default=defaults.get("freeflow_polish", True), help="Use Freeflow-style deterministic punctuation/filler polishing before optional LLM cleanup.")
+    parser.add_argument("--no-freeflow-polish", action="store_false", dest="freeflow_polish", help="Use the older local regex cleanup instead of Freeflow-style deterministic polishing.")
+    parser.add_argument("--post-correct-skip-clean", action="store_true", default=defaults.get("post_correct_skip_clean", True), help="Skip the line LLM when deterministic polish already looks clean.")
+    parser.add_argument("--no-post-correct-skip-clean", action="store_false", dest="post_correct_skip_clean", help="Always call the line LLM when --post-correct-llm is enabled.")
+    parser.add_argument("--post-correct-prompt-style", choices=["freeflow", "qwen", "legacy"], default=defaults.get("post_correct_prompt_style", "freeflow"), help="Prompt style for line LLM cleanup.")
+    parser.add_argument("--llm-paragraph", action="store_true", default=defaults.get("llm_paragraph", False))
+    parser.add_argument("--llm-paragraph-temperature", type=float, default=defaults.get("llm_paragraph_temperature", 0.2))
+    parser.add_argument("--llm-paragraph-top-p", type=float, default=defaults.get("llm_paragraph_top_p", 0.8))
+    parser.add_argument("--llm-paragraph-repeat-penalty", type=float, default=defaults.get("llm_paragraph_repeat_penalty", 1.0))
+    parser.add_argument("--llm-request-log-path", type=str, default=defaults.get("llm_request_log_path", "logs/llm_requests.jsonl"))
+    parser.add_argument("--llm-test-line", type=str, default=None, help="Send a single line-correction prompt without starting transcription.")
+    parser.add_argument("--llm-test-prev1", type=str, default="", help="Optional previous line 1 context for --llm-test-line.")
+    parser.add_argument("--llm-test-prev2", type=str, default="", help="Optional previous line 2 context for --llm-test-line.")
+    parser.add_argument("--llm-test-segments", type=str, default=None, help="Send a single paragraph-refinement prompt without starting transcription.")
+    parser.add_argument("--llm-test-context", type=str, default="", help="Optional existing paragraph context for --llm-test-segments.")
+    parser.add_argument("--llm-test-from-log", type=int, default=None, help="Replay a logged LLM request by JSONL entry index. Use -1 for the most recent entry.")
+    parser.add_argument("--llm-test-use-logged-model", action="store_true", default=False, help="When replaying from log, use the model stored in the log entry instead of --post-correct-model.")
+
+    # Translate Args
+    parser.add_argument("-es", action="store_true", default=defaults.get("es", False))
+    parser.add_argument("-fr", action="store_true", default=defaults.get("fr", False))
+    parser.add_argument("-ar", action="store_true", default=defaults.get("ar", False))
+    parser.add_argument("-en", action="store_true", default=defaults.get("en", False))
+    parser.add_argument("--only-translate-llm", action="store_true", default=defaults.get("only_translate_llm", False), help="Only translate when the LLM produces a refined paragraph.")
+    parser.add_argument("--mt-max-chars", type=int, default=defaults.get("mt_max_chars", 250))
+    parser.add_argument("--mt-max-new-tokens", type=int, default=defaults.get("mt_max_new_tokens", 150))
+    parser.add_argument("--mt-num-beams", type=int, default=defaults.get("mt_num_beams", 4))
+    
+    # Distribute Args
+    parser.add_argument("-i", "--ingest", action="store_true", default=defaults.get("ingest", False))
+    
+    args = parser.parse_args()
+
+    # Auto-enable required flags
+    if args.post_correct_llm and not args.post_correct:
+        args.post_correct = True
+    if args.only_translate_llm and not args.llm_paragraph:
+        print("[Main] Enabling --llm-paragraph because --only-translate-llm was requested.")
+        args.llm_paragraph = True
+
+    if args.llm_test_line or args.llm_test_segments or args.llm_test_from_log is not None:
+        run_llm_prompt_test(args)
+        return
+
+    # Handle device listing
+    if args.list_devices:
+        list_audio_devices()
+        return
+
+    # Handle interactive device selection
+    if args.device is None:
+        list_audio_devices()
+        try:
+            val = input("\nSelect input device index (or press Enter for default): ").strip()
+            if val:
+                args.device = int(val)
+        except EOFError:
+            pass # Non-interactive environment
+        except ValueError:
+            print("[Main] Invalid index, using default.")
+
+    # Queues
+    q_trans_to_llm = multiprocessing.Queue()
+    q_llm_to_tl = multiprocessing.Queue()
+    q_tl_to_dist = multiprocessing.Queue()
+
+    # Processes
+    p_transcribe = multiprocessing.Process(target=run_transcription, args=(q_trans_to_llm, args))
+    p_llm = multiprocessing.Process(target=run_llm_processor, args=(q_trans_to_llm, q_llm_to_tl, args))
+    p_translate = multiprocessing.Process(target=run_translation, args=(q_llm_to_tl, q_tl_to_dist, args))
+    p_distribute = multiprocessing.Process(target=run_distribution, args=(q_tl_to_dist, args))
+
+    print("[Main] Starting processes...")
+    p_transcribe.start()
+    p_llm.start()
+    p_translate.start()
+    p_distribute.start()
+
+    try:
+        p_transcribe.join()
+        p_llm.join()
+        p_translate.join()
+        p_distribute.join()
+    except KeyboardInterrupt:
+        print("\n[Main] Stopping processes...")
+        p_transcribe.terminate()
+        p_llm.terminate()
+        p_translate.terminate()
+        p_distribute.terminate()
+        sys.exit(0)
+
+if __name__ == "__main__":
+    multiprocessing.freeze_support()
+    main()

main_v3.py

@@ -0,0 +1,245 @@
+import multiprocessing
+import argparse
+import json as _json
+import os
+import sys
+from dotenv import load_dotenv
+
+load_dotenv()
+
+from engine_transcribe import run_transcription as run_whisper_transcription, list_audio_devices
+
+def _lazy_apple():
+    try:
+        from engine_apple_transcribe import run_transcription as run_apple
+        return run_apple
+    except Exception as e:
+        print(f"[Main] Apple engine import failed: {e}")
+        return None
+
+from engine_llm import run_llm_processor, run_llm_prompt_test
+# engine_translate is imported lazily inside main() to avoid PIL crash when not needed
+from engine_distribute import run_distribution
+
+def _run_translation_passthrough(in_q, out_q, a):
+    """No MarianMT import — just map LLM output to distribution shape. Top-level for spawn pickling."""
+    import time as _time
+    while True:
+        try:
+            item = in_q.get()
+            if item is None:
+                break
+            if "draft" in item:
+                out_q.put(item)
+                continue
+            raw_text = item.get("raw")
+            corrected = item.get("corrected")
+            paragraph = item.get("paragraph")
+            to_show = paragraph if getattr(a, "only_translate_llm", False) else (paragraph or corrected or raw_text)
+            payload = {
+                "original": raw_text,
+                "corrected": corrected,
+                "paragraph": paragraph,
+                "en_bridge": item.get("en_bridge"),
+                "english_output": to_show or "",
+                "used_llm_line": item.get("used_llm_line", False),
+                "used_llm_paragraph": item.get("used_llm_paragraph", False),
+                "paragraph_fallback": bool(paragraph) and not item.get("used_llm_paragraph", False),
+                "speaker": item.get("speaker"),
+                "ts": item.get("ts", _time.time()),
+            }
+            if getattr(a, "en", False) and to_show:
+                payload["en"] = to_show
+            out_q.put(payload)
+        except Exception as e:
+            print(f"[Translate-passthrough] Error: {e}")
+
+def parse_temperature_fallback(value):
+    try:
+        return tuple(float(x.strip()) for x in value.split(",") if x.strip())
+    except:
+        return (0.0, 0.2, 0.4, 0.6, 0.8, 1.0)
+
+def main():
+    config_path = "config.json"
+    defaults = {}
+    if os.path.exists(config_path):
+        with open(config_path, "r") as f:
+            defaults = _json.load(f)
+
+    parser = argparse.ArgumentParser(description="Multi-process Transcription and Translation (v3: whisper + apple).")
+
+    # Engine selector
+    parser.add_argument("--engine", choices=["whisper", "apple"], default=defaults.get("engine", "whisper"),
+                        help="Transcription engine: whisper (mlx) or apple (SpeechAnalyzer macOS 26+) (default: whisper)")
+    parser.add_argument("--apple-locale", type=str, default=defaults.get("apple_locale", None),
+                        help="Apple locale override e.g. en-US, es-ES, fr-FR (default: auto from --lang)")
+    parser.add_argument("--apple-bench-file", type=str, default=None,
+                        help="Quick bench: transcribe a file with apple engine and exit")
+    parser.add_argument("--apple-stream", action="store_true", default=defaults.get("apple_stream", True),
+                        help="Apple: enable draft streaming every ~1s while speaking (default: on)")
+    parser.add_argument("--no-apple-stream", action="store_false", dest="apple_stream")
+    parser.add_argument("--apple-stream-interval", type=float, default=defaults.get("apple_stream_interval", 1.0),
+                        help="Apple: seconds between draft transcribes (default: 1.0)")
+    parser.add_argument("--apple-pipe", action="store_true", default=defaults.get("apple_pipe", True),
+                        help="Apple: keep one Swift process alive via --pipe (default: on, faster)")
+    parser.add_argument("--no-apple-pipe", action="store_false", dest="apple_pipe")
+
+    # Transcribe Args (shared)
+    parser.add_argument("-l", "--list-devices", action="store_true", help="Show available audio devices and exit")
+    parser.add_argument("--model", type=str, default=defaults.get("model", "mlx-community/whisper-base-mlx"))
+    parser.add_argument("--device", type=int, default=defaults.get("device"))
+    parser.add_argument("--lang", type=str, default=defaults.get("lang"))
+    parser.add_argument("--silence", type=int, default=defaults.get("silence", 1000))
+    parser.add_argument("--max-buffer", type=int, default=defaults.get("max_buffer", 20))
+    parser.add_argument("--channels", type=int, default=defaults.get("channels", 1))
+    parser.add_argument("--filter-lang", action="store_true", default=defaults.get("filter_lang", False))
+    parser.add_argument("-q", "--quantize", action="store_true", default=defaults.get("quantize", False))
+    parser.add_argument("-s", "--stream", action="store_true", default=defaults.get("stream", False))
+    parser.add_argument("-c", "--context", action="store_true", default=defaults.get("context", False))
+    parser.add_argument("--speaker-diarization", action="store_true", default=defaults.get("speaker_diarization", False))
+    parser.add_argument("--temperature-fallback", type=parse_temperature_fallback, default=tuple(defaults.get("temperature_fallback", [0.0, 0.2, 0.4, 0.6, 0.8, 1.0])))
+    parser.add_argument("--logprob-threshold", type=float, default=defaults.get("logprob_threshold", -0.8))
+    parser.add_argument("--compression-threshold", type=float, default=defaults.get("compression_threshold", 2.2))
+    parser.add_argument("--verbose", "-v", action="store_true", default=defaults.get("verbose", False))
+
+    # LLM Args
+    parser.add_argument("--post-correct", action="store_true", default=defaults.get("post_correct", False))
+    parser.add_argument("--post-correct-llm", action="store_true", default=defaults.get("post_correct_llm", False))
+    parser.add_argument("--post-correct-model", type=str, default=defaults.get("post_correct_model", "qwen3.5:0.8b"))
+    parser.add_argument("--post-correct-ollama-url", type=str, default=defaults.get("post_correct_ollama_url", "http://127.0.0.1:11434/api/generate"))
+    parser.add_argument("--post-correct-llm-timeout", type=float, default=defaults.get("post_correct_llm_timeout", 8.0))
+    parser.add_argument("--post-correct-keep-alive", type=str, default=defaults.get("post_correct_keep_alive", "30m"))
+    parser.add_argument("--post-correct-warmup-timeout", type=float, default=defaults.get("post_correct_warmup_timeout", 20.0))
+    parser.add_argument("--post-correct-min-overlap", type=float, default=defaults.get("post_correct_min_overlap", 0.45))
+    parser.add_argument("--post-correct-debug", action="store_true", default=defaults.get("post_correct_debug", False))
+    parser.add_argument("--freeflow-polish", action="store_true", default=defaults.get("freeflow_polish", True))
+    parser.add_argument("--no-freeflow-polish", action="store_false", dest="freeflow_polish")
+    parser.add_argument("--post-correct-skip-clean", action="store_true", default=defaults.get("post_correct_skip_clean", True))
+    parser.add_argument("--no-post-correct-skip-clean", action="store_false", dest="post_correct_skip_clean")
+    parser.add_argument("--post-correct-prompt-style", choices=["freeflow", "qwen", "legacy"], default=defaults.get("post_correct_prompt_style", "freeflow"))
+    parser.add_argument("--apple-llm", action="store_true", default=defaults.get("apple_llm", True),
+                        help="Use Apple on-device LLM (FoundationModels, ANE) for polish when available, Ollama fallback")
+    parser.add_argument("--no-apple-llm", action="store_false", dest="apple_llm")
+    parser.add_argument("--llm-paragraph", action="store_true", default=defaults.get("llm_paragraph", False))
+    parser.add_argument("--llm-paragraph-temperature", type=float, default=defaults.get("llm_paragraph_temperature", 0.2))
+    parser.add_argument("--llm-paragraph-top-p", type=float, default=defaults.get("llm_paragraph_top_p", 0.8))
+    parser.add_argument("--llm-paragraph-repeat-penalty", type=float, default=defaults.get("llm_paragraph_repeat_penalty", 1.0))
+    parser.add_argument("--llm-request-log-path", type=str, default=defaults.get("llm_request_log_path", "logs/llm_requests.jsonl"))
+    parser.add_argument("--llm-test-line", type=str, default=None)
+    parser.add_argument("--llm-test-prev1", type=str, default="")
+    parser.add_argument("--llm-test-prev2", type=str, default="")
+    parser.add_argument("--llm-test-segments", type=str, default=None)
+    parser.add_argument("--llm-test-context", type=str, default="")
+    parser.add_argument("--llm-test-from-log", type=int, default=None)
+    parser.add_argument("--llm-test-use-logged-model", action="store_true", default=False)
+
+    # Translate Args
+    parser.add_argument("-es", action="store_true", default=defaults.get("es", False))
+    parser.add_argument("-fr", action="store_true", default=defaults.get("fr", False))
+    parser.add_argument("-ar", action="store_true", default=defaults.get("ar", False))
+    parser.add_argument("-en", action="store_true", default=defaults.get("en", False))
+    parser.add_argument("--only-translate-llm", action="store_true", default=defaults.get("only_translate_llm", False))
+    parser.add_argument("--mt-max-chars", type=int, default=defaults.get("mt_max_chars", 250))
+    parser.add_argument("--mt-max-new-tokens", type=int, default=defaults.get("mt_max_new_tokens", 150))
+    parser.add_argument("--mt-num-beams", type=int, default=defaults.get("mt_num_beams", 4))
+
+    # Distribute
+    parser.add_argument("-i", "--ingest", action="store_true", default=defaults.get("ingest", False))
+
+    args = parser.parse_args()
+
+    # Auto-enable
+    if args.post_correct_llm and not args.post_correct:
+        args.post_correct = True
+    if args.only_translate_llm and not args.llm_paragraph:
+        print("[Main] Enabling --llm-paragraph because --only-translate-llm was requested.")
+        args.llm_paragraph = True
+
+    # Apple bench file shortcut
+    if args.apple_bench_file:
+        from engine_apple_transcribe import resolve_binary, resolve_locale
+        import subprocess, json
+        binary = resolve_binary()
+        if not binary:
+            print("Apple binary not found. Run cd apple_speech && bash build.sh")
+            sys.exit(1)
+        locale = resolve_locale(args.lang, args.apple_locale)
+        cmd = [str(binary), "--bench", args.apple_bench_file, "--locale", locale, "-v"]
+        print(f"[Main] Running: {' '.join(cmd)}")
+        subprocess.run(cmd)
+        return
+
+    if args.llm_test_line or args.llm_test_segments or args.llm_test_from_log is not None:
+        run_llm_prompt_test(args)
+        return
+
+    if args.list_devices:
+        list_audio_devices()
+        return
+
+    if args.device is None:
+        list_audio_devices()
+        try:
+            val = input("\nSelect input device index (or press Enter for default): ").strip()
+            if val:
+                args.device = int(val)
+        except EOFError:
+            pass
+        except ValueError:
+            print("[Main] Invalid index, using default.")
+
+    # Choose engine
+    if args.engine == "apple":
+        run_transcription_fn = _lazy_apple()
+        if run_transcription_fn is None:
+            print("[Main] Falling back to whisper because apple engine unavailable")
+            run_transcription_fn = run_whisper_transcription
+    else:
+        run_transcription_fn = run_whisper_transcription
+
+    print(f"[Main] Engine: {args.engine}")
+
+    needs_translate = bool(args.es or args.fr or args.ar or args.en)
+    q_trans_to_llm = multiprocessing.Queue()
+    q_llm_to_tl = multiprocessing.Queue()
+    q_tl_to_dist = multiprocessing.Queue()
+
+    p_transcribe = multiprocessing.Process(target=run_transcription_fn, args=(q_trans_to_llm, args))
+    p_llm = multiprocessing.Process(target=run_llm_processor, args=(q_trans_to_llm, q_llm_to_tl, args))
+
+    if not needs_translate:
+        print("[Main] Translate disabled (no -es/-fr/-ar/-en) — using passthrough (no MarianMT/PIL)")
+        p_translate = multiprocessing.Process(target=_run_translation_passthrough, args=(q_llm_to_tl, q_tl_to_dist, args))
+    else:
+        try:
+            from engine_translate import run_translation as _real_translate
+            p_translate = multiprocessing.Process(target=_real_translate, args=(q_llm_to_tl, q_tl_to_dist, args))
+        except Exception as e:
+            print(f"[Main] Translate import failed ({e}) — falling back to passthrough")
+            p_translate = multiprocessing.Process(target=_run_translation_passthrough, args=(q_llm_to_tl, q_tl_to_dist, args))
+
+    p_distribute = multiprocessing.Process(target=run_distribution, args=(q_tl_to_dist, args))
+
+    print("[Main] Starting processes...")
+    p_transcribe.start()
+    p_llm.start()
+    p_translate.start()
+    p_distribute.start()
+
+    try:
+        p_transcribe.join()
+        p_llm.join()
+        p_translate.join()
+        p_distribute.join()
+    except KeyboardInterrupt:
+        print("\n[Main] Stopping processes...")
+        p_transcribe.terminate()
+        p_llm.terminate()
+        p_translate.terminate()
+        p_distribute.terminate()
+        sys.exit(0)
+
+if __name__ == "__main__":
+    multiprocessing.freeze_support()
+    main()

prompt_experiments.py

@@ -0,0 +1,727 @@
+import argparse
+import json
+import os
+import re
+import statistics
+from collections import Counter
+from urllib import request, error
+
+
+FOREIGN_INSERTIONS = [
+    "pero en espanol dice que no retrocedas",
+    "y tambien dijo que Dios responde",
+    "mas ele continuou em fe",
+    "et il a dit de rester ferme",
+]
+
+
+def parse_distribute_debug_log(path):
+    entries = []
+    current = {"final": None, "en_bridge": None, "llm_paragraph": None}
+
+    with open(path, "r", encoding="utf-8") as handle:
+        for raw_line in handle:
+            line = raw_line.rstrip("\n")
+            if "[Final]:" in line:
+                current["final"] = line.split("[Final]:", 1)[1].strip()
+            elif "[EN-BRIDGE]:" in line:
+                current["en_bridge"] = line.split("[EN-BRIDGE]:", 1)[1].strip()
+            elif "[LLM-PARAGRAPH]:" in line:
+                current["llm_paragraph"] = line.split("[LLM-PARAGRAPH]:", 1)[1].strip()
+                entries.append(current)
+                current = {"final": None, "en_bridge": None, "llm_paragraph": None}
+            elif not line.strip():
+                continue
+
+    return entries
+
+
+def normalize_paragraph(text):
+    return (text or "").strip().strip('"').strip()
+
+
+def reconstruct_samples(path):
+    samples = []
+    rows = parse_distribute_debug_log(path)
+    previous_llm = ""
+    previous_llm_history = []
+    pending_bridges = []
+    recent_raw_windows = []
+
+    for row in rows:
+        if row.get("en_bridge"):
+            pending_bridges.append(row["en_bridge"])
+        if not row.get("llm_paragraph"):
+            continue
+
+        llm_output = normalize_paragraph(row["llm_paragraph"])
+        new_raw = " ".join(pending_bridges).strip()
+        prev_raw = recent_raw_windows[-1] if recent_raw_windows else ""
+
+        samples.append(
+            {
+                "previous_refined": previous_llm,
+                "previous_refined_history": list(previous_llm_history),
+                "previous_raw": prev_raw,
+                "new_raw": new_raw,
+                "observed_output": llm_output,
+            }
+        )
+
+        previous_llm = llm_output
+        previous_llm_history.append(llm_output)
+        recent_raw_windows.append(new_raw)
+        pending_bridges = []
+
+    return samples
+
+
+def inject_multilingual_noise(text, sample_index, slot_name):
+    if not text:
+        return text
+    inserts = [
+        FOREIGN_INSERTIONS[(sample_index + 0) % len(FOREIGN_INSERTIONS)],
+        FOREIGN_INSERTIONS[(sample_index + 1) % len(FOREIGN_INSERTIONS)],
+    ]
+    sentences = [part.strip() for part in re.split(r"(?<=[.!?])\s+", text.strip()) if part.strip()]
+    if not sentences:
+        return text
+    if len(sentences) == 1:
+        return f'{sentences[0]} {inserts[0]}.'
+    head = sentences[0]
+    tail = " ".join(sentences[1:])
+    if slot_name == "previous_raw":
+        return f"{head} {inserts[0]}. {tail}"
+    return f"{head} {tail} {inserts[1]}."
+
+
+def maybe_augment_samples(samples, mode):
+    if mode != "multilingual_noise":
+        return samples
+
+    augmented = []
+    for idx, sample in enumerate(samples):
+        updated = dict(sample)
+        updated["previous_raw"] = inject_multilingual_noise(sample["previous_raw"], idx, "previous_raw")
+        updated["new_raw"] = inject_multilingual_noise(sample["new_raw"], idx, "new_raw")
+        augmented.append(updated)
+    return augmented
+
+
+def build_variants(sample, variant_filter=None):
+    prev_refined = sample["previous_refined"]
+    prev_refined_history = sample.get("previous_refined_history", [])
+    prev_raw = sample["previous_raw"]
+    new_raw = sample["new_raw"]
+
+    variants = []
+
+    system_a = """You are a live transcript editor.
+
+Task:
+Refine live transcription stream into clean, professional paragraphs.
+
+Rules:
+1. Integrate new segments into the flow of the previous working context.
+2. Remove redundant repetitions and translator echoes.
+3. Use a double newline (\\n\\n) to start a new paragraph when a topic changes or the current one is complete.
+4. Output only in English.
+5. Return only the refined, consolidated text with no explanation."""
+    user_a = f"""Refine the following live transcription stream.
+
+[PREVIOUS WORKING CONTEXT]
+"{prev_refined}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+    variants.append(("current_refined_plus_new", system_a, user_a))
+
+    system_b = """You are a live transcript editor for noisy speech recognition output.
+
+Goal:
+Produce a clean English paragraph update grounded in the source text.
+
+Grounding rules:
+1. Treat PREVIOUS REFINED CONTEXT as style and continuity help, not as unquestionable truth.
+2. Use PREVIOUS SOURCE TEXT and NEW SOURCE TEXT as the factual anchor.
+3. Do not add details that are not supported by the source text.
+4. Preserve uncertainty when the source is unclear instead of inventing a cleaner claim.
+5. Remove obvious repetitions and translator echoes.
+6. Start a new paragraph with a double newline only when there is a clear topic shift.
+7. Return only the refined English text with no explanation."""
+    user_b = f"""Update the transcript using the source text below.
+
+[PREVIOUS REFINED CONTEXT]
+"{prev_refined}"
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+    variants.append(("refined_plus_prev_raw_plus_new_raw", system_b, user_b))
+
+    system_c = """You are a careful transcript editor.
+
+Goal:
+Produce clean English paragraphs directly from noisy source text.
+
+Rules:
+1. Use only the supplied source text as evidence.
+2. Do not infer missing facts.
+3. Keep wording conservative when the source is noisy.
+4. Remove repeated fragments and obvious translation artifacts.
+5. Use a double newline only for a clear paragraph break.
+6. Return only the refined English text."""
+    user_c = f"""Refine this transcript source.
+
+[RECENT SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+    variants.append(("prev_raw_plus_new_raw_only", system_c, user_c))
+
+    system_d = """You are a live transcript editor for sermon audio translated into English.
+
+Goal:
+Create the most faithful readable English paragraph you can from imperfect source text.
+
+Priority order:
+1. Faithfulness to source text.
+2. Remove duplicated phrases, stutters, and translation echoes.
+3. Maintain continuity with the previous refined context only when it matches the source text.
+4. Prefer slight awkwardness over hallucination.
+5. If a phrase is unclear, keep it modestly literal instead of guessing.
+6. Use a double newline only for a real topic change.
+7. Return only the revised English transcript.
+
+Hard constraints:
+- Do not add people, events, or meanings absent from the source text.
+- Do not turn rhetorical questions into factual claims unless the source clearly does that.
+- Do not overwrite NEW SOURCE TEXT with PREVIOUS REFINED CONTEXT."""
+    user_d = f"""Revise the transcript update.
+
+[PREVIOUS REFINED CONTEXT]
+"{prev_refined}"
+
+[RECENT SOURCE WINDOW]
+"{prev_raw}"
+
+[CURRENT SOURCE WINDOW]
+"{new_raw}"
+"""
+    variants.append(("grounded_sermon_editor", system_d, user_d))
+
+    system_e = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+3. If the new material starts a fresh thought, output only the new material.
+4. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+5. Do not add explanations, disclaimers, or meta commentary.
+6. Prefer conservative wording over guessed meaning.
+7. Return only the revised transcript text."""
+    user_e = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT]
+"{prev_refined}"
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+    variants.append(("grounded_new_dominant", system_e, user_e))
+
+    system_g = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+3. The final answer must be English only.
+4. If source text contains Spanish, Portuguese, French, or other non-English fragments, translate their meaning into English when reasonably clear.
+5. Never copy non-English words into the output unless they are proper names.
+6. If the new material starts a fresh thought, output only the new material.
+7. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+8. Do not add explanations, disclaimers, or meta commentary.
+9. Prefer conservative wording over guessed meaning.
+10. Return only the revised transcript text."""
+    user_g = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT]
+"{prev_refined}"
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+    variants.append(("grounded_english_only_translate", system_g, user_g))
+
+    system_h = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+3. The final answer must be English only.
+4. Treat any Spanish, Portuguese, French, or other non-English fragments as contamination from surrounding audio or translation noise unless they clearly repeat the same idea as nearby English.
+5. When a non-English fragment clearly repeats nearby English content, keep only the English meaning once.
+6. Never copy non-English words into the output unless they are proper names.
+7. If the new material starts a fresh thought, output only the new material.
+8. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+9. Do not add explanations, disclaimers, or meta commentary.
+10. Prefer conservative wording over guessed meaning.
+11. Return only the revised transcript text."""
+    user_h = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT]
+"{prev_refined}"
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+    variants.append(("grounded_english_only_ignore_noise", system_h, user_h))
+
+    system_i = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+3. The final answer must be English only.
+4. Never copy non-English words into the output unless they are proper names.
+5. If a non-English fragment appears as an isolated clause, trailing fragment, or side comment without a clear English anchor, drop it.
+6. Only translate a non-English fragment when it is clearly central to the same idea and its meaning is reasonably obvious from context.
+7. If the new material starts a fresh thought, output only the new material.
+8. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+9. Do not add explanations, disclaimers, or meta commentary.
+10. Prefer conservative wording over guessed meaning.
+11. Return only the revised transcript text."""
+    user_i = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT]
+"{prev_refined}"
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+    variants.append(("grounded_english_only_drop_isolated", system_i, user_i))
+
+    system_f = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+3. If the new material starts a fresh thought, output only the new material.
+4. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+5. If the source repeats the same short clause three or more times in a row, keep at most two repetitions unless losing more would change the speaker's emphasis.
+6. Do not add explanations, disclaimers, or meta commentary.
+7. Prefer conservative wording over guessed meaning.
+8. Return only the revised transcript text."""
+    user_f = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT]
+"{prev_refined}"
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+    variants.append(("grounded_new_dominant_dedupe", system_f, user_f))
+
+    for window_size in [2, 4, 6, 8, 10]:
+        context_window = "\n\n".join(prev_refined_history[-window_size:]).strip()
+        system_ctx = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS REFINED CONTEXT and PREVIOUS SOURCE TEXT only when they clearly help resolve or continue the new material.
+3. The final answer must be English only.
+4. Never copy non-English words into the output unless they are proper names.
+5. If a non-English fragment appears as an isolated clause, trailing fragment, or side comment without a clear English anchor, drop it.
+6. Only translate a non-English fragment when it is clearly central to the same idea and its meaning is reasonably obvious from context.
+7. If the new material starts a fresh thought, output only the new material.
+8. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+9. Do not add explanations, disclaimers, or meta commentary.
+10. Prefer conservative wording over guessed meaning.
+11. Return only the revised transcript text."""
+        user_ctx = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT WINDOW]
+{context_window}
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+        """
+        variants.append((f"grounded_ctx{window_size}", system_ctx, user_ctx))
+
+        if window_size in (2, 4):
+            user_ctx_no_prev_source = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT WINDOW]
+{context_window}
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+            variants.append((f"grounded_ctx{window_size}_no_prev_source", system_ctx, user_ctx_no_prev_source))
+
+        if window_size == 2:
+            system_ctx_paragraph = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce faithful, readable English paragraphs.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+3. The final answer must be English only.
+4. Never copy non-English words into the output unless they are proper names.
+5. If a non-English fragment appears as an isolated clause, trailing fragment, or side comment without a clear English anchor, drop it.
+6. Only translate a non-English fragment when it is clearly central to the same idea and its meaning is reasonably obvious from context.
+7. If NEW SOURCE TEXT continues the same thought as PREVIOUS REFINED CONTEXT, merge them into one coherent paragraph.
+8. If NEW SOURCE TEXT starts a fresh thought, start a new paragraph instead of forcing a merge.
+9. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+10. Make obvious caption-word mistakes make sense only when the intended English wording is reasonably clear from context.
+11. Prefer conservative wording over guessed meaning.
+12. Use a double newline between distinct paragraphs.
+13. Do not add explanations, disclaimers, labels, or meta commentary.
+14. Return only the revised paragraph text."""
+            user_ctx_paragraph = f"""Synthesize the transcript into coherent paragraphs.
+
+[PREVIOUS REFINED CONTEXT]
+{context_window}
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+            variants.append(("grounded_ctx2_paragraph_live", system_ctx_paragraph, user_ctx_paragraph))
+
+            system_ctx_paragraph_soft = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS SOURCE TEXT and PREVIOUS REFINED CONTEXT only when they clearly help resolve or continue the new material.
+3. The final answer must be English only.
+4. Never copy non-English words into the output unless they are proper names.
+5. If a non-English fragment appears as an isolated clause, trailing fragment, or side comment without a clear English anchor, drop it.
+6. Only translate a non-English fragment when it is clearly central to the same idea and its meaning is reasonably obvious from context.
+7. If NEW SOURCE TEXT clearly continues PREVIOUS REFINED CONTEXT, you may merge them naturally.
+8. If NEW SOURCE TEXT starts a fresh thought, output only the new material and do not drag older context forward.
+9. If both pieces are relevant but distinct adjacent thoughts, separate them with a double newline.
+10. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+11. Make obvious caption-word mistakes make sense only when the intended English wording is reasonably clear from context.
+12. Prefer conservative wording over guessed meaning.
+13. Do not add explanations, disclaimers, labels, or meta commentary.
+14. Return only the revised transcript text."""
+            user_ctx_paragraph_soft = f"""Edit this transcript update as readable paragraph text.
+
+[PREVIOUS REFINED CONTEXT]
+{context_window}
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+            variants.append(("grounded_ctx2_paragraph_soft", system_ctx_paragraph_soft, user_ctx_paragraph_soft))
+
+        system_ctx_repair = """You are a careful live transcript editor working from noisy translated source text.
+
+Goal:
+Produce the most faithful readable English update.
+
+Decision rules:
+1. NEW SOURCE TEXT is the primary evidence and should dominate the output.
+2. Use PREVIOUS REFINED CONTEXT and PREVIOUS SOURCE TEXT only when they clearly help resolve or continue the new material.
+3. The final answer must be English only.
+4. Never copy non-English words into the output unless they are proper names.
+5. If a non-English fragment appears as an isolated clause, trailing fragment, or side comment without a clear English anchor, drop it.
+6. Correct likely caption-word mistakes only when the surrounding context makes the intended English wording reasonably clear.
+7. If a word or phrase is unclear, keep it conservative instead of guessing.
+8. If the new material starts a fresh thought, output only the new material.
+9. Remove exact or near-exact repetition unless the repetition is clearly intentional rhetoric.
+10. Do not add explanations, disclaimers, or meta commentary.
+11. Return only the revised transcript text."""
+        user_ctx_repair = f"""Edit this transcript update.
+
+[PREVIOUS REFINED CONTEXT WINDOW]
+{context_window}
+
+[PREVIOUS SOURCE TEXT]
+"{prev_raw}"
+
+[NEW SOURCE TEXT]
+"{new_raw}"
+"""
+        variants.append((f"grounded_repair_ctx{window_size}", system_ctx_repair, user_ctx_repair))
+
+    if variant_filter:
+        allowed = set(x.strip() for x in variant_filter.split(",") if x.strip())
+        variants = [variant for variant in variants if variant[0] in allowed]
+    return variants
+
+
+def ollama_generate(model, system_prompt, user_prompt, url, timeout, keep_alive, temperature, top_p, repeat_penalty):
+    payload = json.dumps(
+        {
+            "model": model,
+            "prompt": user_prompt,
+            "system": system_prompt,
+            "stream": False,
+            "options": {
+                "temperature": temperature,
+                "top_p": top_p,
+                "repeat_penalty": repeat_penalty,
+            },
+            "keep_alive": keep_alive,
+        }
+    ).encode("utf-8")
+    req = request.Request(
+        url,
+        data=payload,
+        headers={"Content-Type": "application/json"},
+        method="POST",
+    )
+    try:
+        with request.urlopen(req, timeout=timeout) as response:
+            body = json.loads(response.read().decode("utf-8"))
+    except error.HTTPError as exc:
+        detail = exc.read().decode("utf-8", errors="replace")
+        raise RuntimeError(f"Ollama HTTP error {exc.code}: {detail}") from exc
+    return body.get("response", "").strip()
+
+
+def tokenize(text):
+    return re.findall(r"[a-z0-9']+", (text or "").lower())
+
+
+def repeated_ngram_ratio(text, n=3):
+    tokens = tokenize(text)
+    if len(tokens) < n:
+        return 0.0
+    grams = [tuple(tokens[i:i + n]) for i in range(len(tokens) - n + 1)]
+    counts = Counter(grams)
+    repeated = sum(count - 1 for count in counts.values() if count > 1)
+    return repeated / max(1, len(grams))
+
+
+def unsupported_token_ratio(output, evidence):
+    evidence_tokens = set(tokenize(evidence))
+    output_tokens = tokenize(output)
+    if not output_tokens:
+        return 0.0
+    unsupported = [tok for tok in output_tokens if tok not in evidence_tokens]
+    return len(unsupported) / len(output_tokens)
+
+
+def evaluate_output(output, sample):
+    evidence = " ".join(
+        part for part in [sample["previous_raw"], sample["new_raw"]] if part
+    )
+    return {
+        "output_len": len(tokenize(output)),
+        "repeated_ngram_ratio": round(repeated_ngram_ratio(output), 4),
+        "unsupported_token_ratio": round(unsupported_token_ratio(output, evidence), 4),
+        "foreign_leak_ratio": round(foreign_leak_ratio(output), 4),
+    }
+
+
+def foreign_leak_ratio(output):
+    foreign_markers = {
+        "pero", "espanol", "tambien", "dijo", "dios", "responde",
+        "mas", "ele", "continuou", "fe",
+        "et", "rester", "ferme",
+    }
+    tokens = tokenize(output)
+    if not tokens:
+        return 0.0
+    leaked = [tok for tok in tokens if tok in foreign_markers]
+    return len(leaked) / len(tokens)
+
+
+def parse_float_grid(raw_value, default_values):
+    if not raw_value:
+        return default_values
+    return [float(x.strip()) for x in raw_value.split(",") if x.strip()]
+
+
+def build_param_sets(args):
+    temperatures = parse_float_grid(args.temperature_grid, [0.1])
+    top_ps = parse_float_grid(args.top_p_grid, [0.9])
+    repeat_penalties = parse_float_grid(args.repeat_penalty_grid, [1.1])
+
+    param_sets = []
+    for temperature in temperatures:
+        for top_p in top_ps:
+            for repeat_penalty in repeat_penalties:
+                label = f"temp={temperature}_top_p={top_p}_repeat_penalty={repeat_penalty}"
+                param_sets.append(
+                    {
+                        "label": label,
+                        "temperature": temperature,
+                        "top_p": top_p,
+                        "repeat_penalty": repeat_penalty,
+                    }
+                )
+    return param_sets
+
+
+def run_experiments(args):
+    samples = reconstruct_samples(args.log_file)
+    samples = maybe_augment_samples(samples, args.sample_mode)
+    if args.start_index:
+        samples = samples[args.start_index:]
+    if args.limit:
+        samples = samples[: args.limit]
+
+    param_sets = build_param_sets(args)
+    results = []
+    for index, sample in enumerate(samples):
+        print(f"Running sample {index + 1}/{len(samples)}")
+        sample_results = []
+        for variant_name, system_prompt, user_prompt in build_variants(sample, args.variant_filter):
+            print(f"  variant={variant_name}")
+            for params in param_sets:
+                print(f"    params={params['label']}")
+                try:
+                    output = ollama_generate(
+                        args.model,
+                        system_prompt,
+                        user_prompt,
+                        args.ollama_url,
+                        args.timeout,
+                        args.keep_alive,
+                        params["temperature"],
+                        params["top_p"],
+                        params["repeat_penalty"],
+                    )
+                    metrics = evaluate_output(output, sample)
+                    error_message = None
+                except Exception as exc:
+                    output = ""
+                    metrics = None
+                    error_message = str(exc)
+                sample_results.append(
+                    {
+                        "variant": variant_name,
+                        "params": params,
+                        "output": output,
+                        "metrics": metrics,
+                        "error": error_message,
+                    }
+                )
+        results.append(
+            {
+                "sample_index": index,
+                "sample": sample,
+                "results": sample_results,
+            }
+        )
+    return results
+
+
+def summarize_results(results):
+    summary = {}
+    for sample_result in results:
+        for variant_result in sample_result["results"]:
+            summary_key = f"{variant_result['variant']}|{variant_result['params']['label']}"
+            bucket = summary.setdefault(
+                summary_key,
+                {
+                    "repeated_ngram_ratio": [],
+                    "unsupported_token_ratio": [],
+                    "foreign_leak_ratio": [],
+                    "output_len": [],
+                    "errors": 0,
+                },
+            )
+            if variant_result.get("error"):
+                bucket["errors"] += 1
+                continue
+            for key, value in variant_result["metrics"].items():
+                bucket[key].append(value)
+
+    for variant_name, metrics in summary.items():
+        summary[variant_name] = {
+            key: round(statistics.mean(values), 4) if isinstance(values, list) and values else values
+            for key, values in metrics.items()
+        }
+    return summary
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Run prompt experiments against Ollama using distribute_debug.log samples.")
+    parser.add_argument("--log-file", type=str, default="distribute_debug.log")
+    parser.add_argument("--model", type=str, default="qwen2.5:3b-instruct")
+    parser.add_argument("--ollama-url", type=str, default="http://127.0.0.1:11434/api/generate")
+    parser.add_argument("--timeout", type=float, default=40.0)
+    parser.add_argument("--keep-alive", type=str, default="30m")
+    parser.add_argument("--limit", type=int, default=0)
+    parser.add_argument("--start-index", type=int, default=0)
+    parser.add_argument("--output-file", type=str, default="logs/prompt_experiment_results.json")
+    parser.add_argument("--variant-filter", type=str, default="", help="Comma-separated subset of variants to run.")
+    parser.add_argument("--temperature-grid", type=str, default="", help="Comma-separated temperatures to test.")
+    parser.add_argument("--top-p-grid", type=str, default="", help="Comma-separated top_p values to test.")
+    parser.add_argument("--repeat-penalty-grid", type=str, default="", help="Comma-separated repeat_penalty values to test.")
+    parser.add_argument("--sample-mode", type=str, default="base", help="Sample mode: base or multilingual_noise.")
+    args = parser.parse_args()
+
+    results = run_experiments(args)
+    summary = summarize_results(results)
+    payload = {"model": args.model, "summary": summary, "results": results}
+
+    output_dir = os.path.dirname(args.output_file)
+    if output_dir:
+        os.makedirs(output_dir, exist_ok=True)
+    with open(args.output_file, "w", encoding="utf-8") as handle:
+        json.dump(payload, handle, ensure_ascii=False, indent=2)
+
+    print(json.dumps(summary, ensure_ascii=False, indent=2))
+    print(f"Wrote detailed results to {args.output_file}")
+
+
+if __name__ == "__main__":
+    main()

requirements.txt

@@ -0,0 +1,16 @@
+mlx-whisper
+numpy
+sounddevice
+torch
+requests
+transformers
+silero-vad
+pyinstaller
+sacremoses
+joblib
+sentencepiece
+pyannote.audio
+soundfile
+langdetect
+python-dotenv
+openai