feat: add command-line arguments for lang selection, ingest toggle, and device management; add README and requirements

2026-02-28 18:55:37 -05:00
parent 0ea605cc0c
commit 3c874c113c
4 changed files with 141 additions and 21 deletions
@@ -0,0 +1,76 @@
+# Python Whisper Live Transcription & Translation
+
+A real-time, low-latency audio transcription and translation tool utilizing OpenAI's Whisper (via `mlx-whisper` for Apple Silicon optimization), Silero VAD for speech detection, and Helsinki-NLP's Opus-MT models for translation.
+
+## Features
+- **Live Transcription:** Real-time speech-to-text with automatic language detection.
+- **On-the-fly Translation:** Bridge translations to English and then to target languages (Spanish, Arabic, French, etc.).
+- **Voice Activity Detection (VAD):** Intelligent audio buffering using Silero VAD to process only actual speech.
+- **Apple Silicon Optimized:** Uses MLX for high performance on Mac (MPS).
+- **Background Ingestion:** Optional background thread to send JSON payloads to a remote server.
+- **Configurable:** Command-line parameters to select languages, devices, and ingestion.
+
+## Installation
+
+1. **Clone the repository:**
+   ```bash
+   git clone <repository-url>
+   cd pythonwhisper
+   ```
+
+2. **Install dependencies:**
+   Ensure you have Python 3.9+ and the required libraries:
+   ```bash
+   pip install mlx-whisper numpy sounddevice torch requests transformers silero-vad
+   ```
+   *Note: On Apple Silicon, ensure `mlx` and `torch` with MPS support are correctly installed.*
+
+## Usage
+
+Run the script using `python3 transcribe.py` with optional flags.
+
+### Common Commands
+- **List available audio devices:**
+  ```bash
+  python3 transcribe.py -l
+  ```
+- **Transcribe and translate to Spanish (screen only):**
+  ```bash
+  python3 transcribe.py -es
+  ```
+- **Enable Spanish, Arabic, and French with English bridging, and send to server:**
+  ```bash
+  python3 transcribe.py -es -ar -fr -en -i
+  ```
+- **Use a specific input device (e.g., index 3) and translate to Spanish:**
+  ```bash
+  python3 transcribe.py -d 3 -es
+  ```
+
+### Arguments
+- `-es`: Enable Spanish translation.
+- `-en`: Enable English (shows original if detected, or bridged if not).
+- `-ar`: Enable Arabic translation.
+- `-fr`: Enable French translation.
+- `-i`, `--ingest`: Enable data transmission to the remote server.
+- `-l`, `--list-devices`: Show available audio devices and exit.
+- `-d`, `--device [ID]`: Input device index (bypasses selection prompt).
+
+---
+
+## Technical Note: Universal Translation Models
+
+While the current implementation uses specialized, per-language models from the **Helsinki-NLP Opus-MT** project (e.g., `opus-mt-en-es`, `opus-mt-en-ar`), there is an alternative approach: **Universal Models**.
+
+### Universal Model Alternative (e.g., Meta's NLLB-200)
+The current per-language model approach is highly accurate and memory-efficient if you only need 1 or 2 target languages. However, if you require support for many languages simultaneously, loading multiple specialized models can consume significant RAM/VRAM.
+
+We have the option to switch the translation engine to a single, universal model such as **NLLB-200 (No Language Left Behind)**:
+- **Model ID:** `facebook/nllb-200-distilled-600M`
+- **Benefits:**
+    - Supports over **200 languages** in a single model.
+    - Simplified code: no need to load/manage multiple model objects.
+    - More efficient for complex multilingual environments.
+- **Trade-off:** Slightly higher memory footprint for the single model compared to a single specialized model, but more efficient than 3+ specialized models.
+
+If you wish to switch to a universal model, the `transcribe.py` logic can be updated to use a single `M2M100` or `NLLB` pipeline instead of the current `MarianMT` loop.