Image generation became a commodity; video generation has not. It costs far more per second, it is much less forgiving of a single bad frame, and the things that go wrong are different in kind — not a wrong texture, but a person who changes face between cuts.
Models, pricing and licensing in this category change monthly. Verify current terms with the vendor before any commercial use.
Three categories
Hosted frontier models
Video generation from the large labs, usually released as limited-access products — OpenAI's Sora line, Google's Veo family, and similar. These tend to produce the most coherent motion and the best physical plausibility at their best.
- Strength: strongest sense of physical realism and camera movement
- Strength: audio and dialogue generation in some models, which changes what a "clip" even is
- Limitation: access, quotas and pricing are unstable by design during preview periods
- Limitation: limited fine control; you describe and hope, rather than direct
- Limitation: content restrictions are strictest here
Editing-suite and creative tools
Video-focused products built for people who actually make things — Runway being the long-standing example, alongside Kling, Hailuo/MiniMax and others with strong followings.
- Strength: control — motion brushes, camera keys, inpainting, extending a shot, matching a reference
- Strength: built into a real workflow with timeline, layers and export options
- Strength: image-to-video is the default mode, which is the more controllable way to work
- Limitation: quality varies noticeably shot to shot, and you will generate many attempts
- Limitation: credit systems make cost hard to predict
Open-weight and local
Models you can run or fine-tune yourself, in the lineage of open video diffusion work.
- Strength: no per-generation cost, no content filter, full privacy, and fine-tuning on your own footage
- Strength: reproducible — the same seed and weights give the same result, which matters for real production
- Limitation: heavy hardware requirements, significant setup, and slower iteration
- Limitation: quality at a given size generally trails the hosted frontier, and it is your job to maintain
Image-to-video beats text-to-video
If there is one practical piece of advice worth more than the rest of this article: generate the still first, then animate it.
- You can iterate on composition, lighting and subject cheaply as an image — where generation is fast and inexpensive
- The video model then only has to solve motion, not composition and motion at once
- You get consistency across shots by reusing a consistent character or product image
- Text-to-video is best reserved for abstract or atmospheric shots where specific composition doesn't matter
What still breaks
- Identity drift — a face that changes subtly across a few seconds, or a character who becomes someone else after a cut. The single biggest blocker for narrative work.
- Object permanence — items that appear, vanish, duplicate or change shape mid-shot
- Physics — liquids that don't pour, cloth that doesn't fold, weight that doesn't transfer, objects passing through each other
- Hands and interaction — much improved, still the most reliable tell
- Legible text — signage, screens, labels. Same weakness as image models, compounded by motion
- Causality — actions that produce no consequence, or consequences with no cause
- Long duration — coherence degrades with length; almost everything works better in short shots stitched together
- Precise timing — "she turns on the third beat" is still closer to a wish than an instruction
Cost is a different shape
This is the part people under-budget. A usable four-second shot may take ten to thirty attempts. Multiply by length and by re-editing, and per-second pricing becomes misleading.
- Budget by finished second, not by generation
- Short shots cut together are cheaper and more reliable than one long one
- Use video only where motion carries meaning — a slow push on a still is often enough, and free
- Iterate at low resolution where the tool allows it, then upscale the one that works
The part that matters beyond the tool
Video generation is the first consumer-accessible technology that can produce convincing footage of events that never happened, featuring real people. That has consequences worth stating clearly:
- Provenance matters. Prefer tools that embed content credentials or watermarks, and preserve them. Stripping provenance from generated footage is what makes it dangerous.
- Consent is not optional. Generating a recognisable real person — including a colleague, a public figure, or a private individual — without consent is a legal and ethical problem regardless of what the tool permits.
- Disclosure is increasingly required. Many platforms require labelling synthetic media, and some jurisdictions regulate it. Check before publishing.
- Never generate footage of real events — news, disasters, conflicts, medical claims — that could be mistaken for documentation.
Choosing
- Best-looking motion, willing to accept limited control: a hosted frontier model
- Actual production work, need control and a workflow: an editing-suite tool
- Privacy, volume, or fine-tuning on your own footage: open weights locally
- Character consistency across shots: test this specifically before committing — it is where projects fail
- Anything published: check licensing and labelling requirements first
Getting a usable shot
- Start from a still you love, then animate it
- Describe motion, not content — the image already establishes content
- Keep shots short — three to five seconds, then cut
- Change one variable per attempt and keep notes on what worked
- Reject early. If the first second is wrong, the shot is wrong; don't wait to see the end
- Plan to hide weaknesses with editing — cuts, motion blur, close-ups and sound cover a great deal
- Keep the seed and prompt for anything you may need to reproduce or extend
Bottom line
Hosted frontier models produce the most convincing motion with the least control; editing-suite tools give you the control real work requires; local open weights give privacy, volume and reproducibility at the cost of quality and setup. Generate stills first, then animate them, keep shots short, and budget by finished second rather than by attempt. And treat provenance and consent as part of the work, not an afterthought — this is the one category where a careless output can do damage well beyond a bad render.