feat: Apple on-device LLM (ANE) polish replaces Ollama as primary
- apple-llm-polish binary (120KB Swift, FoundationModels): --check
now reports available:true after Apple Intelligence enabled,
ping OK, line polish 258-388ms ANE vs Ollama 3-8s CPU
- engine_apple_llm.py: AppleLLM class with persistent pipe,
thread reader, polish_line() + polish_paragraph(), JSONL protocol
{id,mode,text,prev1,prev2,context} -> {id,text,ok,ms}
- engine_llm.py: try Apple ANE first (permissiveContentTransformations,
temp 0.1 line / 0.2 para), log provider=apple apple_ms, fallback
to Ollama on guardrail/timeout. Fixes missing LLM log — now
logs provider=apple with ms.
- main_v3.py: --apple-llm / --no-apple-llm flag (on by default)
- Verified: direct polish 258ms, last log provider=apple ms=498,
speech binary still 240KB with 31 word-by-word drafts
Co-authored-by: internal-model
This commit is contained in:
@@ -118,6 +118,9 @@ def main():
|
||||
parser.add_argument("--post-correct-skip-clean", action="store_true", default=defaults.get("post_correct_skip_clean", True))
|
||||
parser.add_argument("--no-post-correct-skip-clean", action="store_false", dest="post_correct_skip_clean")
|
||||
parser.add_argument("--post-correct-prompt-style", choices=["freeflow", "qwen", "legacy"], default=defaults.get("post_correct_prompt_style", "freeflow"))
|
||||
parser.add_argument("--apple-llm", action="store_true", default=defaults.get("apple_llm", True),
|
||||
help="Use Apple on-device LLM (FoundationModels, ANE) for polish when available, Ollama fallback")
|
||||
parser.add_argument("--no-apple-llm", action="store_false", dest="apple_llm")
|
||||
parser.add_argument("--llm-paragraph", action="store_true", default=defaults.get("llm_paragraph", False))
|
||||
parser.add_argument("--llm-paragraph-temperature", type=float, default=defaults.get("llm_paragraph_temperature", 0.2))
|
||||
parser.add_argument("--llm-paragraph-top-p", type=float, default=defaults.get("llm_paragraph_top_p", 0.8))
|
||||
|
||||
Reference in New Issue
Block a user