whisper-translation/debug_session_1_issues.md

Debug Session 1: Observed Issues and Low-Effort Fixes

This note summarizes concrete problems observed in debug_session_1.log, with examples and pragmatic fixes that do not require a major redesign.

1. Duplicate Final English Captions

The pipeline frequently emits the same English line twice in quick succession, usually as a raw/final pair or a minimally corrected pair.

Examples

• It has a lot of sweet things. appears twice at debug_session_1.log:449 and debug_session_1.log:450.
• Because the crisis we're going through is the crisis Cuba is going through. appears twice at debug_session_1.log:827 and debug_session_1.log:828.
• And one thing I can say about my brother, Daniel. appears twice at debug_session_1.log:923 and debug_session_1.log:924.

Low-Hanging Fix

• Track the last emitted finalized English caption and suppress exact or near-exact duplicates before printing or sending downstream.
• Prefer a single user-facing final English line instead of exposing both pre-corrected and corrected variants.

2. Over-Eager Merge Logic

The short-window merge logic sometimes combines adjacent fragments that should remain separate, producing unnatural or noisy caption revisions.

Examples

• restaurants, bakeries. is merged into the previous line as Right now I'm a partner of that business group, which has Restaurants, bakeries. at debug_session_1.log:461 and debug_session_1.log:462.
• A-A-M is preserved and merged into A-A-M Prepare ourselves for the future in that sense. at debug_session_1.log:534 and debug_session_1.log:541.
• Short fragments become a clumsy combined sentence around Right now. / What's coming in the future. at debug_session_1.log:1019, debug_session_1.log:1025, and debug_session_1.log:1032.

Low-Hanging Fix

• Tighten merge conditions so a merge requires stronger evidence that the current line continues the previous one.
• Reject merges when the current line is too short, looks like noise, or resembles a standalone noun phrase.
• Reject merges when either line contains token patterns like A-A-M or other obvious non-lexical fragments.
• Require token overlap or a continuation cue before merging.

3. Runaway Repetition and Hallucination

The current hallucination checks do not catch repeated multi-word loops reliably, so some bad segments still reach output and translation.

Examples

• Repeated sentence loop at debug_session_1.log:28.
• He got a job. loop at debug_session_1.log:34.
• So what about ... repetition burst at debug_session_1.log:83.
• Extreme remember repetition at debug_session_1.log:152.

Low-Hanging Fix

• Expand hallucination detection beyond single-word repetition.
• Add repeated n-gram detection for 2- to 5-word phrases.
• Reject long lines with very low lexical diversity.
• Skip downstream translation when the English bridge is flagged as repetitive garbage.

4. Bad ASR Segments Survive Correction

Some obviously wrong segments remain wrong even after rule-based and LLM post-correction.

Examples

• red at this moment. at debug_session_1.log:516.
• Gong! at debug_session_1.log:599.
• an ungovernmental organization. at debug_session_1.log:647.
• So I posted what clarified that last part. at debug_session_1.log:875.
• It's such a fun to help. at debug_session_1.log:1109.

Low-Hanging Fix

• Add deterministic rejection for filler-only, malformed, or obviously low-information English fragments before translation.
• Expand rule-based cleanup to drop pure filler lines such as Uh., Um., or Hmm.
• Add heuristics to reject malformed token patterns before they become finalized captions.

5. Wrong English Pollutes All Translations

Because translation is English-bridge driven, wrong or noisy English becomes confident multilingual output.

Examples

• Gong! propagates into ES/FR/AR around debug_session_1.log:599, debug_session_1.log:601, debug_session_1.log:602, and debug_session_1.log:604.
• ungovernmental organization is normalized into target-language output around debug_session_1.log:647, debug_session_1.log:649, debug_session_1.log:650, and debug_session_1.log:652.

Low-Hanging Fix

• Add a quality gate before translation so clearly bad English is not propagated.
• For rejected English segments, prefer skipping translation over producing confident multilingual noise.

6. Language Filtering Was Necessary

Earlier runs without --filter-lang showed stray source-language outputs that do not fit the intended English-first workflow.

Examples

• [PT]: Que sempre pulamos. at debug_session_1.log:26.
• [RU]: и просуло им. at debug_session_1.log:32.
• [IT]: Three, one. at debug_session_1.log:133.

Low-Hanging Fix

• Keep --filter-lang --lang en as part of the preferred operating profile for this use case.
• Preserve this as the default recommendation for live runs unless a multilingual-source workflow is explicitly desired.

Recommended First Fixes

If the goal is best return on effort, the first three fixes to implement are:

1. Suppress duplicate finalized English captions.
2. Tighten merge heuristics so only clear continuations are merged.
3. Add repeated-phrase hallucination detection and skip translation for clearly bad English.

These changes target the most visible issues in the log without requiring a new model stack or a broader pipeline redesign.