Add LLM logging and prompt experiment harness
This commit is contained in:
@@ -29,6 +29,32 @@ A real-time, low-latency audio transcription and translation tool utilizing Open
|
||||
|
||||
Run the script using `python3 transcribe.py` with optional flags.
|
||||
|
||||
### LLM Prompt Testing
|
||||
You can now send the line-correction or paragraph-refinement prompts directly without starting live transcription. Every LLM request is appended as JSON Lines to `logs/llm_requests.jsonl` by default.
|
||||
|
||||
- **Test the line-correction prompt:**
|
||||
```bash
|
||||
python3 main_v2.py --post-correct-model qwen3.5:0.8b --llm-test-line "so um we should probably ship it tomorrow" --llm-test-prev1 "We finished the staging deploy." --llm-test-prev2 "QA signed off this morning."
|
||||
```
|
||||
- **Test the paragraph-refinement prompt:**
|
||||
```bash
|
||||
python3 main_v2.py --post-correct-model qwen3.5:0.8b --llm-test-context "We reviewed the launch checklist." --llm-test-segments "we confirmed monitoring we confirmed rollback and then talked about the release window"
|
||||
```
|
||||
- **Change where requests are logged:**
|
||||
```bash
|
||||
python3 main_v2.py --llm-request-log-path tmp/my_llm_requests.jsonl --llm-test-line "example text"
|
||||
```
|
||||
- **Replay the most recent logged request:**
|
||||
```bash
|
||||
python3 main_v2.py --llm-test-from-log -1
|
||||
```
|
||||
- **Replay a logged request using the original model from the log entry:**
|
||||
```bash
|
||||
python3 main_v2.py --llm-test-from-log 12 --llm-test-use-logged-model
|
||||
```
|
||||
|
||||
Each log entry includes the timestamp, provider, model, mode, temperature, and full `messages` payload that was sent to the LLM.
|
||||
|
||||
### Common Commands
|
||||
- **List available audio devices:**
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user