Adolfo Reyna
119 lines
4.7 KiB
Markdown
119 lines
4.7 KiB
Markdown
# Video Captioning & Translation Tool (OpenAI API)
|
|
|
|
This is a lightweight Command Line Interface (CLI) tool that transcribes, translates, and captions video files. It uses **OpenAI's cloud APIs** for transcription (Whisper) and translation (GPT), combined with a **local FFmpeg pipeline** to extract audio and burn the translated subtitles into the output video.
|
|
|
|
## Architecture & Features
|
|
|
|
- **No Heavy Model Downloads**: Swapping local models for OpenAI APIs means zero local machine inference overhead and a tiny installation footprint (no PyTorch/CUDA/CoreML weights needed).
|
|
- **Whisper API Transcription**: Calls OpenAI's `whisper-1` model with `verbose_json` to fetch accurate segment timestamps.
|
|
- **Context-Aware Translation**: Translates transcription segments with a rolling context window (remembers prior segments) via the OpenAI Chat Completions API (`gpt-4o-mini` by default). This maintains high translation cohesion.
|
|
- **Local Subtitle Rendering**: Uses local FFmpeg filters to hardcode (burn) subtitles into the output video.
|
|
|
|
---
|
|
|
|
## Dependencies & Setup
|
|
|
|
### 1. System Requirements
|
|
You must have **FFmpeg** installed locally to handle audio extraction and video rendering.
|
|
On macOS, install it using Homebrew:
|
|
```bash
|
|
brew install ffmpeg
|
|
```
|
|
|
|
### 2. Python Environment Setup
|
|
Create a virtual environment and install the dependencies:
|
|
```bash
|
|
# Navigate to the project folder
|
|
cd /Users/adolforeyna/.gemini/antigravity/scratch/local-translator
|
|
|
|
# Create a virtual environment
|
|
python3 -m venv venv
|
|
source venv/bin/activate
|
|
|
|
# Install requirements
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
### 3. API Configuration
|
|
The tool reads `.env` configuration. A `.env` file has been automatically copied from your `pythonwhisper` project containing your `OPENAI_API_KEY`.
|
|
|
|
If you need to update it, modify the `.env` file in the project folder:
|
|
```env
|
|
OPENAI_API_KEY=your_openai_api_key_here
|
|
```
|
|
|
|
---
|
|
|
|
## Usage Examples
|
|
|
|
Always activate your virtual environment before running the tool:
|
|
```bash
|
|
source venv/bin/activate
|
|
```
|
|
|
|
### 1. Translate English Video to Spanish (Default)
|
|
```bash
|
|
python main.py -v /path/to/my_video.mp4
|
|
```
|
|
*Outputs: `/path/to/my_video_es_captioned.mp4`*
|
|
|
|
### 2. Translate Spanish Video to English
|
|
```bash
|
|
python main.py -v /path/to/video.mp4 -s es -t en
|
|
```
|
|
|
|
### 3. Keep Temporary Subtitles and Audio Files
|
|
If you want to keep the intermediate `.srt` subtitle file and `.wav` extracted audio for manual editing or verification, add `--keep-temp`:
|
|
```bash
|
|
python main.py -v /path/to/video.mp4 --keep-temp
|
|
```
|
|
|
|
---
|
|
|
|
## Visuals & Style Customization
|
|
|
|
The tool supports subtitle visual customization via **FFmpeg ASS Style Presets** (available when burning subtitles):
|
|
- **`box`** (Default): White text on a semi-transparent dark background box. Offers the best readability.
|
|
- **`default`**: Standard white text with outline and drop shadow.
|
|
- **`yellow`**: Yellow text with black outline and drop shadow.
|
|
- **`clean`**: Larger white text on a minimal thin outline with a Trebuchet font style.
|
|
|
|
To select a preset:
|
|
```bash
|
|
python main.py -v my_video.mp4 --style yellow
|
|
```
|
|
|
|
For advanced styling, pass a raw ASS style string using `--custom-style`:
|
|
```bash
|
|
python main.py -v my_video.mp4 --custom-style "FontName=Arial,FontSize=24,PrimaryColour=&H0000FFFF,BorderStyle=3,BackColour=&H80000000"
|
|
```
|
|
|
|
---
|
|
|
|
## Cadence Customization (Segment Splitting)
|
|
|
|
By default, the translation tool preserves the transcription segment lengths returned by the Whisper API. If you have long sentences or fast spoken parts, subtitles can get cluttered.
|
|
|
|
You can enforce a **character limit per card** using the `--max-chars` flag. The tool will automatically split long sentences into smaller, logically divided segments and distribute the timing proportionally:
|
|
```bash
|
|
# Split subtitles so each card is at most 40 characters
|
|
python main.py -v my_video.mp4 --max-chars 40
|
|
```
|
|
|
|
---
|
|
|
|
## CLI Argument Reference
|
|
|
|
| Argument | Short | Description |
|
|
| :--- | :--- | :--- |
|
|
| `--video` | `-v` | **Required**. Path to the input video file. |
|
|
| `--source-lang` | `-s` | Source language of the video (default: `en`). |
|
|
| `--target-lang` | `-t` | Target language for the captions (default: `es`). |
|
|
| `--output` | `-o` | Custom path for the output captioned video. |
|
|
| `--whisper-model`| | OpenAI Whisper model to use (default: `whisper-1`). |
|
|
| `--model` | | OpenAI translation model to use (default: `gpt-4o-mini`). |
|
|
| `--keep-temp` | | Keep intermediate `.wav` and `.srt` files. |
|
|
| `--style` | | Subtitle style preset: `default`, `yellow`, `box`, `clean` (default: `box`). |
|
|
| `--custom-style` | | Raw ASS style string override (e.g. `FontName=Arial,FontSize=24`). |
|
|
| `--max-chars` | | Force subtitle line length limit by splitting segments (default: `0` for no splitting). |
|