What you get, and what you do not
- You get 9:16 clips at the height of your source (720 by 1280 from a 720p file), captions 3 words at a time, each clip cut on whole sentences, no watermark and no credits.
- You don't get face tracking. The crop is the center of the frame, so if your speaker leans or two people sit side by side, you set the crop yourself or lose part of a face.
- You need Python 3, ffmpeg built with libass (most builds are), about 1 GB of disk for the Whisper model, and any chat model you already use.
The 4 steps, with our times
| Step | Tool | Time on a 22-minute 42-second video |
|---|---|---|
| 1. Transcript with a time for every word | faster-whisper, small English model | 9 minutes 18 seconds |
| 2. Pick the best moments | One chat model prompt | 64 seconds |
| 3. Move each cut to a sentence edge | A 38-line script | Under 1 second |
| 4. Cut, crop and caption | ffmpeg | 3 minutes 47 seconds for 10 clips |
Our runs used 2 CPU threads and no graphics card. A recent laptop with more cores will be faster on steps 1 and 4.
Set up once
python3 -m venv clips && . clips/bin/activate
pip install faster-whisper soundfile
ffmpeg -hide_banner -filters | grep -w ass # must print a line, or captions will not burn inStep 1: a transcript with word times
First pull the sound out of the video as 16 kHz mono, which is what Whisper reads:
ffmpeg -i talk.mp4 -vn -ac 1 -ar 16000 talk.wavThen run this script. It works through the audio one minute at a time, so memory stays low on a small machine, and it keeps a start and end time for every word. Step 3 and the captions both need those word times.
"""Transcribes a 16 kHz mono WAV with faster-whisper small.en on CPU in 1-minute chunks, so peak memory stays under the render cap, and times the run."""
import json, sys, time
import soundfile as sf
from faster_whisper import WhisperModel
t0 = time.time()
model = WhisperModel("small.en", device="cpu", compute_type="int8", cpu_threads=2)
audio, rate = sf.read(sys.argv[1], dtype="float32")
chunk = 60 * rate
out = []
for i in range(0, len(audio), chunk):
off = i / rate
segs, _ = model.transcribe(audio[i:i + chunk], word_timestamps=True, vad_filter=True)
out += [{"start": s.start + off, "end": s.end + off, "text": s.text.strip(), "words": [[w.start + off, w.end + off, w.word] for w in s.words]} for s in segs]
json.dump({"seconds": round(time.time() - t0, 1), "duration": len(audio) / rate, "segments": out}, open(sys.argv[2], "w"))
print("transcribed in", round(time.time() - t0, 1), "s for", round(len(audio) / rate), "s of audio")python3 transcribe.py talk.wav transcript.jsonThe small English model was accurate enough for captions on our test. It spelled terms such as "cupola", "Artemis", "Z9" and "neutral density" correctly. Read the captions once anyway, since a wrong word burned into a clip can't be fixed later.
Step 2: let a chat model pick the moments
This script prints the whole transcript as timed lines, followed by the ask. The rule in the ask is the one we judge clips by: a clip stands alone, starts at the start of a sentence, ends at the end of a thought, answers one question, and is not a greeting, a radio check or a thank-you. Edit the first line to describe your own video.
"""Prints the clip-picking prompt for a chat model: the transcript as timestamped lines, then the ask.
Usage: python3 pick_prompt.py transcript.json > prompt.txt
The same prompt can be pasted into any free chat model; we sent it to one model call and timed it.
"""
import json
import sys
segs = json.load(open(sys.argv[1]))["segments"]
lines = [f"[{s['start']:.1f}-{s['end']:.1f}] {s['text']}" for s in segs]
print("""Below is the transcript of a 23-minute live call between the ISS and a space photographer, as [start-end seconds] lines.
Pick the 10 best moments for 20 to 90 second vertical clips. Each clip must stand alone: it starts at the start of a sentence, ends at the end of a thought, answers one question, and is not a greeting, a radio check or a thank-you. No two clips may cover the same answer.
Answer with JSON only: a list of {"start": seconds, "end": seconds, "title": "short title"}, using the start of the first line and the end of the last line you keep.
""" + "\n".join(lines))python3 pick_prompt.py transcript.json > prompt.txtPaste prompt.txt into any chat model you use, and save the JSON it answers with as picks.json. We sent it to Claude Haiku in one call. It took 64 seconds and picked 10 moments, and 9 of them cover a moment that Opus Clip, a paid clip tool, also picked from the same video.
The model was good at picking moments and careless about edges: only 2 of its 10 clips started and ended on a whole sentence. That's what step 3 is for.
Step 3: snap every cut to a sentence edge
This script moves each start to the nearest sentence start and each end to the nearest sentence end, using the word times from step 1. It also stops a clip from ending on a question, because a question opens the next answer, and stops one clip from running into the next.
"""Moves each pick to whole sentences, using the faster-whisper word timestamps.
The start moves to the nearest sentence start and the end to the nearest sentence end, back or forward.
A sentence ends on . ? or !, but not on "...", which whisper writes for a trailing-off half sentence.
A clip never ends on a question, since a question opens the next answer, and never runs into the next pick.
The end pads 0.3 s but stops before the next word starts.
Usage: snap.py transcript.json picks.json > snapped.json
"""
import json, sys
words = [w for s in json.load(open(sys.argv[1]))["segments"] for w in s["words"]]
def ends(k):
t = words[k][2].strip()
return t[-1:] in ".?!" and not t.endswith("...")
starts = [0] + [k + 1 for k in range(len(words) - 1) if ends(k)]
stops = [k for k in range(len(words)) if ends(k)]
def nearest(cands, t, key):
return min(cands, key=lambda k: abs(key(k) - t))
picks = json.load(open(sys.argv[2]))
out = []
for n, p in enumerate(picks):
i = nearest(starts, p["start"], lambda k: words[k][0])
limit = picks[n + 1]["start"] if n + 1 < len(picks) else 1e9
ok = [k for k in stops if k > i and words[k][1] <= limit]
j = nearest(ok, p["end"], lambda k: words[k][1])
while words[j][2].strip().endswith("?") and ok.index(j) > 0 and words[ok[ok.index(j) - 1]][1] - words[i][0] >= 20:
j = ok[ok.index(j) - 1]
out.append({**p, "start": round(words[i][0] - 0.1, 2), "end": round(min(words[j][1] + 0.3, words[j + 1][0] - 0.02) if j + 1 < len(words) else words[j][1] + 0.3, 2)})
print(json.dumps(out, indent=1))python3 snap.py transcript.json picks.json > snapped.jsonAfter this step, 8 of our 10 clips were usable, against 2 before it. The 2 left over show what a script can't see. One ends on a whole sentence but before the picture we wanted, a filter coming off a window. The other runs 113 seconds and opens on the host's leftover remark. Watch every clip once before you post it.
Step 4: cut, crop to 9:16 and burn in captions
For each pick, this script writes a caption file with 3 words per line, in capitals, from the word times. Then it has ffmpeg cut the clip, crop the center of the frame to 9:16 and burn the captions in.
"""Cuts picked moments from the source into 9:16 clips with burned-in word captions.
Usage: cut.py source.mp4 transcript.json picks.json outdir
picks.json is a list of {"start": s, "end": s, "title": str}. The crop is the center of the frame (no speaker
tracking), and captions show 3 words at a time from the faster-whisper word timestamps.
"""
import json, subprocess, sys, time
src, tr, picks, out = sys.argv[1:5]
words = [w for s in json.load(open(tr))["segments"] for w in s["words"]]
def ts(t):
h, r = divmod(max(t, 0), 3600); m, s = divmod(r, 60)
return f"{int(h)}:{int(m):02d}:{s:05.2f}"
HEAD = """[Script Info]
PlayResX: 720
PlayResY: 1280
[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, OutlineColour, Bold, BorderStyle, Outline, Shadow, Alignment, MarginV
Style: Cap,DejaVu Sans,64,&H00FFFFFF,&H00000000,1,1,5,0,2,260
[Events]
Format: Layer, Start, End, Style, Text
"""
t0 = time.time()
for i, p in enumerate(json.load(open(picks)), 1):
p0 = p["start"]
ws = [w for w in words if w[0] >= p["start"] and w[1] <= p["end"]]
lines = [HEAD]
for j in range(0, len(ws), 3):
g = ws[j:j + 3]
text = " ".join(w[2].strip() for w in g).upper()
lines.append(f"Dialogue: 0,{ts(g[0][0] - p0)},{ts(g[-1][1] - p0)},Cap,{text}\n")
ass = f"{out}/{i:02d}.ass"
open(ass, "w").write("".join(lines))
subprocess.run(["ffmpeg", "-nostdin", "-v", "error", "-y", "-ss", str(p["start"]), "-to", str(p["end"]), "-i", src,
"-vf", f"crop=ih*9/16:ih,scale=720:1280,ass={ass}", "-c:v", "libx264", "-preset", "veryfast", "-crf", "23",
"-threads", "2", "-c:a", "aac", "-b:a", "128k", f"{out}/{i:02d}.mp4"], check=True)
print("cut", i, "clips in", round(time.time() - t0, 1), "s")mkdir -p out
python3 cut.py talk.mp4 transcript.json snapped.json outChange two things for your own video. The font is DejaVu Sans, which most Linux machines have. On a Mac, use a font you have, such as Helvetica. And if your speaker sits off center, change crop=ih*9/16:ih to crop=ih*9/16:ih:X:0, where X is the pixel column where the crop starts.
When a paid tool is worth it
The free route is weakest where a paid clip tool is strongest: framing that follows the speaker, a finished look, and no setup. On the same 22-minute video, Opus Clip gave 11 clips we would post in about 10 minutes, and its free plan puts a watermark on every clip. The numbers for both routes are in our side-by-side test.
If you clip often and the speaker moves, Opus Clip Starter is 15 US dollars per month on its plans page, read 2026-10-18, and removes the watermark. If you clip now and then and can set a crop, the 4 scripts above do the job for nothing.
Frequently asked questions
Do I need a graphics card?
No. Our whole run used 2 CPU threads: 14 minutes of machine time for a 22-minute video. A graphics card makes step 1 faster.
Can I do this without paying for a chat model?
Yes. Step 2 is one prompt, so a free chat window works. Paste prompt.txt in, and save the JSON answer as picks.json. Through a pay-per-call API, our one call would have cost about 0.09 US dollars.
Why are my clips cut in the middle of a sentence?
Chat models give rough times. Run step 3: on our test it took clean clips from 2 of 10 to 8 of 10.
Sources
- faster-whisper: the transcript step, with word timestamps.
- ffmpeg ass filter: burns the caption file into the clip.
- Opus Clip plans: the Starter price and the free plan watermark, read 2026-10-18.
- Our own run on 2026-10-18 on a NASA video from the NASA Image and Video Library, used only as a test file. NASA does not endorse this post or any tool in it.