feat: Apple on-device LLM (ANE) polish replaces Ollama as primary

- apple-llm-polish binary (120KB Swift, FoundationModels): --check
  now reports available:true after Apple Intelligence enabled,
  ping OK, line polish 258-388ms ANE vs Ollama 3-8s CPU
- engine_apple_llm.py: AppleLLM class with persistent pipe,
  thread reader, polish_line() + polish_paragraph(), JSONL protocol
  {id,mode,text,prev1,prev2,context} -> {id,text,ok,ms}
- engine_llm.py: try Apple ANE first (permissiveContentTransformations,
  temp 0.1 line / 0.2 para), log provider=apple apple_ms, fallback
  to Ollama on guardrail/timeout. Fixes missing LLM log — now
  logs provider=apple with ms.
- main_v3.py: --apple-llm / --no-apple-llm flag (on by default)
- Verified: direct polish 258ms, last log provider=apple ms=498,
  speech binary still 240KB with 31 word-by-word drafts

Co-authored-by: internal-model
This commit is contained in:
Adolfo Reyna
2026-07-13 21:31:24 -04:00
parent 80f0bf309f
commit 2e6d6d24ce
6 changed files with 573 additions and 4 deletions
+3
View File
@@ -118,6 +118,9 @@ def main():
parser.add_argument("--post-correct-skip-clean", action="store_true", default=defaults.get("post_correct_skip_clean", True))
parser.add_argument("--no-post-correct-skip-clean", action="store_false", dest="post_correct_skip_clean")
parser.add_argument("--post-correct-prompt-style", choices=["freeflow", "qwen", "legacy"], default=defaults.get("post_correct_prompt_style", "freeflow"))
parser.add_argument("--apple-llm", action="store_true", default=defaults.get("apple_llm", True),
help="Use Apple on-device LLM (FoundationModels, ANE) for polish when available, Ollama fallback")
parser.add_argument("--no-apple-llm", action="store_false", dest="apple_llm")
parser.add_argument("--llm-paragraph", action="store_true", default=defaults.get("llm_paragraph", False))
parser.add_argument("--llm-paragraph-temperature", type=float, default=defaults.get("llm_paragraph_temperature", 0.2))
parser.add_argument("--llm-paragraph-top-p", type=float, default=defaults.get("llm_paragraph_top_p", 0.8))