feat: warm up ollama and keep post-correct model loaded
This commit is contained in:
@@ -194,3 +194,12 @@ The project now features a high-performance, Apple Silicon-optimized pipeline th
|
||||
- Added token-overlap acceptance guard (`--post-correct-min-overlap`) to reject LLM rewrites that diverge too far from source text.
|
||||
- Updated merge-decider prompt to stricter JSON behavior and added overlap-based fallback to heuristic merge.
|
||||
- **Outcome:** Improved protection against context bleed (e.g., replacing current line with prior sentence) while keeping optional LLM improvements.
|
||||
|
||||
## Phase 24: Ollama Warmup + Keep-Alive
|
||||
- **Goal:** Reduce intermittent post-correction timeouts caused by cold model loads.
|
||||
- **Approach:**
|
||||
- Added startup warmup request for the local post-correct model when LLM correction is enabled.
|
||||
- Added configurable Ollama `keep_alive` setting so the model stays resident between caption calls.
|
||||
- Routed post-correct and merge-decider requests through a shared Ollama caller with keep-alive support.
|
||||
- Added flags for warmup timeout and optional warmup skip for troubleshooting.
|
||||
- **Outcome:** Lower first-request latency and fewer fallback-to-rules events due to local model cold starts.
|
||||
|
||||
Reference in New Issue
Block a user