feat: Apple on-device LLM (ANE) polish replaces Ollama as primary

- apple-llm-polish binary (120KB Swift, FoundationModels): --check
  now reports available:true after Apple Intelligence enabled,
  ping OK, line polish 258-388ms ANE vs Ollama 3-8s CPU
- engine_apple_llm.py: AppleLLM class with persistent pipe,
  thread reader, polish_line() + polish_paragraph(), JSONL protocol
  {id,mode,text,prev1,prev2,context} -> {id,text,ok,ms}
- engine_llm.py: try Apple ANE first (permissiveContentTransformations,
  temp 0.1 line / 0.2 para), log provider=apple apple_ms, fallback
  to Ollama on guardrail/timeout. Fixes missing LLM log — now
  logs provider=apple with ms.
- main_v3.py: --apple-llm / --no-apple-llm flag (on by default)
- Verified: direct polish 258ms, last log provider=apple ms=498,
  speech binary still 240KB with 31 word-by-word drafts

Co-authored-by: internal-model
This commit is contained in:
Adolfo Reyna
2026-07-13 21:31:24 -04:00
parent 80f0bf309f
commit 2e6d6d24ce
6 changed files with 573 additions and 4 deletions
+3
View File
@@ -10,6 +10,9 @@ venv/
apple_speech/.build/
apple-speech-transcribe
apple_speech/apple-speech-transcribe
apple-llm-polish
apple_speech/apple-llm-polish
apple_speech/.build/release/apple-llm-polish
logs/apple_bench.json
logs/context*.json
logs/prompt*.json