adolforeyna/whisper-translation
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Tune paragraph prompts for multilingual noise

Browse Source
This commit is contained in:
Adolfo Reyna
2026-03-17 11:51:53 -04:00
parent 7da02a9697
commit 7451d164b7
5 changed files with 304 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
history.md
+10
View File
@@ -297,3 +297,13 @@ The project now features a high-performance, Apple Silicon-optimized pipeline th
- Identified the best-performing variant as one that treats the new source window as primary evidence while using prior refined and prior raw text only when they clearly continue the thought.
- Updated `engine_llm.py` so live paragraph refinement now sends `PREVIOUS REFINED CONTEXT`, `PREVIOUS SOURCE TEXT`, and `NEW SOURCE TEXT` with a more conservative, grounding-oriented system prompt.
- **Outcome:** Paragraph refinement is now substantially better anchored to current source text, with less stale-context carryover and a much lower rate of unsupported content in direct Ollama tests.
## Phase 34: Multilingual Noise Filtering + Paragraph Generation Sweep
- **Goal:** Make paragraph refinement more robust when stray Spanish or other non-English fragments leak into the English bridge text.
- **Approach:**
- Extended `prompt_experiments.py` with multilingual-noise test cases that inject Spanish, Portuguese, and French fragments into reconstructed live samples.
- Ran direct-Ollama comparisons across multiple paragraph prompt variants and generation settings, including temperature, `top_p`, and `repeat_penalty`.
- Found that the best overall paragraph generation settings for `qwen2.5:3b-instruct` remained conservative: `temperature=0.2`, `top_p=0.8`, `repeat_penalty=1.0`.
- Tightened the live paragraph system prompt in `engine_llm.py` so output must stay English-only, non-English fragments should not be copied through, isolated trailing foreign fragments should be dropped, and only clearly central foreign content should be translated into English.
- Added paragraph-specific generation controls to `main_v2.py` and `config.json` so paragraph tuning can evolve independently of line-correction behavior.
- **Outcome:** Paragraph refinement is now stricter about keeping the final output in English while preserving the more grounded context layout and the best-tested Ollama generation settings.