144 lines
5.0 KiB
Markdown
144 lines
5.0 KiB
Markdown
---
|
||
name: remotion-prompt-video
|
||
description: Generate MP4 videos from text prompts using Remotion. Use when the user asks to create a video, explainer clip, social media reel, or animated presentation from a text prompt. The agent writes a VideoScript JSON, then Remotion renders it to MP4. Requires Node.js.
|
||
metadata: {"openclaw":{"requires":{"bins":["node","npm"]}}}
|
||
---
|
||
|
||
# Remotion Prompt Video
|
||
|
||
Render text-based videos from a structured JSON script using Remotion. The agent authors the script directly (no external API calls needed), then `render.mjs` converts it to MP4.
|
||
|
||
## Workflow
|
||
|
||
1. **Setup** — Scaffold the Remotion project from templates (once per workspace)
|
||
2. **Author script** — Generate a `VideoScript` JSON file from the user’s prompt
|
||
3. **Render** — Run `node render.mjs script.json output.mp4`
|
||
4. **Iterate** — Edit the JSON and re-render as needed
|
||
|
||
## Step 1: Setup (First Use Only)
|
||
|
||
```bash
|
||
bash {baseDir}/scripts/setup_project.sh <project_dir>
|
||
```
|
||
|
||
This copies templates, runs `npm install`, and downloads Chromium headless for rendering (~1-2 min). Skip if the project directory already exists.
|
||
|
||
## Step 2: Author the VideoScript JSON
|
||
|
||
Write a JSON file matching this structure. See `{baseDir}/references/video-script-schema.md` for the full schema and all valid values.
|
||
|
||
```json
|
||
{
|
||
"title": "Video Title",
|
||
"description": "Brief description",
|
||
"fps": 30,
|
||
"width": 1920,
|
||
"height": 1080,
|
||
"totalDurationInSeconds": 12,
|
||
"scenes": [
|
||
{
|
||
"id": 1,
|
||
"title": "Intro",
|
||
"text": "Main text on screen",
|
||
"subtitle": "Optional smaller text",
|
||
"durationInSeconds": 3,
|
||
"style": {
|
||
"backgroundColor": "#1a1a2e",
|
||
"textColor": "#ffffff",
|
||
"fontSize": 72,
|
||
"fontFamily": "Arial",
|
||
"textAlign": "center"
|
||
},
|
||
"animation": { "entrance": "zoomIn", "exit": "fadeOut" },
|
||
"backgroundGradient": { "from": "#0f0c29", "to": "#302b63", "direction": "to bottom right" },
|
||
"icon": "🎬"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
### Script authoring guidelines
|
||
|
||
- Create 3–8 scenes; each 2–5 seconds long
|
||
- First scene = title/intro, last scene = closing/outro
|
||
- Keep `text` concise (≤15 words) — this is video, not an article
|
||
- Use gradient backgrounds with good text contrast
|
||
- Vary animations across scenes for visual interest
|
||
- Available entrance animations: `fadeIn`, `slideUp`, `slideLeft`, `slideRight`, `zoomIn`, `typewriter`, `none`
|
||
- Available exit animations: `fadeOut`, `slideDown`, `slideLeft`, `slideRight`, `zoomOut`, `none`
|
||
- Fonts: `Arial`, `Georgia`, `Courier New`, `Verdana`, `Impact`, `Trebuchet MS`
|
||
- `totalDurationInSeconds` must equal the sum of all scene `durationInSeconds`
|
||
- Scene `id` values must be sequential starting from 1
|
||
|
||
### Resolution presets
|
||
|
||
| Aspect | Width | Height |
|
||
|--------|-------|--------|
|
||
| 16:9 | 1920 | 1080 |
|
||
| 9:16 | 1080 | 1920 |
|
||
| 1:1 | 1080 | 1080 |
|
||
|
||
## Step 3: Render
|
||
|
||
```bash
|
||
cd <project_dir>
|
||
node render.mjs <script.json> [output.mp4]
|
||
```
|
||
|
||
If no output path is given, the video is saved to `out/video.mp4`. Rendering a 20-second 1080p video takes ~3-5 minutes.
|
||
|
||
## Step 3: Render
|
||
|
||
```bash
|
||
cd <project_dir>
|
||
node render.mjs <script.json> [output.mp4]
|
||
```
|
||
|
||
If no output path is given, the video is saved to `out/video.mp4`. Rendering a 20-second 1080p video takes ~3-5 minutes.
|
||
|
||
|
||
### Sending renders to messaging tools
|
||
|
||
Messaging bridges like Telegram only accept media files from a handful of allowed directories (for macOS that means stuff under `/var/folders/*/T/`). After the render finishes, create a temp copy there and send from that path:```bash
|
||
tmp=$(mktemp -t remotion-video.mp4)
|
||
cp out/video.mp4 "$tmp"
|
||
message ... media="$tmp"
|
||
```
|
||
If you automate this inside the skill, always copy into `mktemp` before invoking the messaging tool, so future renders can ship without manual work.
|
||
|
||
## Step 4: Iterate
|
||
|
||
Edit the JSON script and re-run `node render.mjs` to update the video. No rebuild needed — the bundler re-bundles automatically each run.
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
project/
|
||
├── render.mjs # script.json → MP4
|
||
├── package.json
|
||
├── tsconfig.json
|
||
├── src/
|
||
│ ├── index.ts # Remotion entry point
|
||
│ ├── Root.tsx # Composition registration
|
||
│ ├── components/
|
||
│ │ ├── Video.tsx # Sequences all scenes
|
||
│ │ └── Scene.tsx # Scene renderer with animations
|
||
│ └── lib/
|
||
│ └── types.ts # VideoScript, Scene types
|
||
└── out/ # Output directory
|
||
```
|
||
|
||
## References
|
||
|
||
- **Full JSON schema and all valid field values**: `{baseDir}/references/video-script-schema.md`
|
||
- **Extending animations, adding images/audio, custom fonts**: `{baseDir}/references/customization.md`
|
||
|
||
## Troubleshooting
|
||
|
||
| Issue | Fix |
|
||
|-------|-----|
|
||
| `Can't resolve './Root.js'` | Imports in `.ts`/`.tsx` must omit `.js` extension |
|
||
| `zod version mismatch` | `npm install zod@3.22.3` |
|
||
| Render very slow | Use fewer scenes or smaller resolution (`1:1`) |
|
||
| Missing browser | `npx remotion browser ensure` |
|