Video & audio
Video and audio tools are tested the way you would test a camera and a microphone: the same request, two results, and a written score for motion, faces, flicker, voices, and transcripts. Public boards already rank many models by votes or by word error rate, and you can still run a smaller Swedish check on files you have the right to use.
Beginner
Same prompt, two clips, one voice check
A video model turns a written description into a short moving clip. An audio model turns text into a spoken voice, or turns a recording back into text. You can compare both with the same habit: freeze the request, change one service, and write down what you see or hear.
Pick one sentence. Example: a person in a red coat walks toward the camera on a wet street, looks at the camera, and stops. Paste that sentence into two video services. Ask each for about five seconds. Play both clips one after the other, and pause often.
Check three things. Motion: do the legs keep a walk, or does the body glide? Face: do the eyes, teeth, and ears stay in place when the head turns? Flicker: does the coat, the street, or the light pulse between frames? Hands, if they appear, often gain or lose fingers. Write a score from 1 to 5 for each point, plus the model name, the date, the length, and the resolution the service showed.
Then test sound. Read one Swedish sentence aloud, or use a short clip you recorded yourself, and ask a transcription tool for the text. Read the result against your sentence. A missed word and an extra word both count. Separately, ask a voice tool to speak the same sentence, and listen for names, numbers, and a flat tone.
The market and the job board open later. You can run this comparison on your own today.
Five-second clip and a Swedish line
Use this exact request in two video tools, and ask each for about 5 seconds. Then use the Swedish line in a voice tool and in a transcription tool. Video: A person in a red coat walks toward the camera on a wet street, looks at the camera, and stops. Same person, same coat, same street for the whole clip. No words on screen. Voice: speak this sentence in a calm adult Swedish voice: Vi ses på stationen klockan åtta. Transcript check: I will paste a transcript below. Compare it with that sentence. List every wrong word, missing word, and extra word. Do not smooth the spelling.
Intermediate
Three video jobs, then a word-error count
Text-to-video starts from words. Image-to-video starts from a still you supply, and the model invents the motion. Video edit starts from a clip you already have and changes the part you name, such as a coat color or a car behind the person. Run those as three tests. A strong result on one job can sit beside a weak result on another.
Use one prompt in two text-to-video tools, then one still in two image-to-video tools. Match length, shape, and resolution as closely as the tools allow, and write down any setting you could not match. Arena publishes boards for text-to-video, image-to-video, and video edit, from blind votes on pairs of clips. Read the table on the day you open it, and keep your own scores beside it.
For speech, use files you may share: your own voice, a person who agreed in writing, or Swedish audio from Mozilla Common Voice, released under the CC0 public-domain dedication. Do not try to identify a Common Voice speaker. Transcribe the same files in two tools. Count a wrong word, a missing word, and an extra word. Word error rate adds those three and divides by the words in the correct transcript. The Open ASR Leaderboard uses that ratio. A lower number is a closer transcript. A name such as Bolagsverket, or a regional vowel, moves the score more than a studio sentence.
Divide the price shown by the seconds of finished video. That is the cost per second. Note whether the price included sound.
Three jobs, one log
I am comparing video and audio tools. Help me fill a log. For each result I paste, label the job as text-to-video, image-to-video, or video edit. Score motion, face stability, flicker, and hands from 1 to 5, and quote the moment the clip fails. Then compute cost per second from the price and the clip length I give you. If I paste a Swedish reference transcript and a model transcript, compute word error rate as (wrong words + missing words + extra words) / words in the reference, after ignoring punctuation and capitals. Ask me for any number you do not have. Do not invent scores.
Expert
Frames, crowd votes, Swedish errors, and call delay
VBench scores video generators on separate points: a subject that stays the same person, smooth motion, and flicker over time. Export one frame every half second into a contact sheet. Score identity, flicker, hands, and simple physics, such as a cup that stays on a table. The paper and the code list prompts and automatic checks.
On the Arena boards, read the rank spread and the vote count. Many votes with a narrow spread are steadier than a wide spread on a few hundred votes. Use the open-weights filter for a model you could run yourself.
The Open ASR Leaderboard reports word error rate and inverse real-time factor, RTFx. RTFx is seconds of audio divided by seconds of compute. An RTFx of 1 keeps pace with the speaker. An RTFx of 2 finishes in half the speaking time. The multilingual tab lists German, French, Italian, Spanish, Portuguese, Dutch, Armenian, and Hindi. For Swedish dialect and trade language, label your own set with region, speaking style, and terms that must match. Strip punctuation and capitals before scoring, as that leaderboard does.
ITU-T Recommendation G.114 tells planners to stay within 400 milliseconds of one-way delay, and says voice calls and video meetings can suffer at lower delays. Time the gap from the end of your Swedish sentence to the first sound of the reply. Record the network, the model, and the milliseconds.
In the same row, record currency, price, output seconds, resolution, audio on or off, and seconds from submit until the file arrived. Divide that wait by the clip length.
Frame sheet, error rate, and the reply gap
Act as a test scribe. I will give you frame notes from a clip (one frame every 0.5 seconds), an Arena rank with its vote count and rank spread, a Swedish reference, a model transcript, a price, a wait time, and a measured gap from the end of my sentence to the reply audio. Produce one table row with: identity, flicker, hands, physics, whether the crowd score looks steady from the vote count and spread, word error rate, cost per second, wait per second of video, and the reply gap in milliseconds. State the network and model only if I supplied them. Use ITU-T G.114 only as context: a 400 ms one-way planning limit, with interactive calls sensitive below that. Do not turn my gap into a pass or fail badge.
What to write on each row
Keep one row per clip or per audio file. Leave a cell blank when the service hides the figure.
| Field | What to write |
|---|---|
| Date and tool | Day, product name, and model name as shown |
| Job | Text-to-video, image-to-video, video edit, voice, or transcript |
| Request | The full prompt, or the file name of the still, clip, or audio |
| Output | Seconds, resolution, and whether the clip has sound |
| Motion, face, flicker, hands | Each scored 1 to 5, with one short note |
| Price | Currency charged, then price divided by output seconds |
| Wait | Seconds from submit to file, then that number divided by output seconds |
| Transcript | Reference text, word error rate, and any term that failed |
| Call gap | Milliseconds from your last word to the reply's first sound, plus the network |
Use footage you filmed, or a clip whose license you have read. On Pexels, photos and videos are free to use and modify, and attribution is optional. That license forbids selling an unaltered copy, implying that a person or brand endorses you, showing an identifiable person in an offensive way, and reposting the file on another stock site. For a public test set, prefer material the project filmed and cleared.
A shared community DGX Station could run the same open video and speech models on one recorded machine, so a heavy text-to-video pass or a Swedish word-error-rate run becomes a dated row members can compare. About the shared station
Sources
- Text-to-Video ArenaBlind-vote ranking of text-to-video models, with score, vote count, and an open-weights filter. Ranks move, so read the date on the page.
- Image-to-Video ArenaThe same style of ranking for models that start from a still image.
- Video Edit ArenaBlind votes on models that change an existing clip. Fewer votes than the other two boards, so check the vote count.
- Open ASR LeaderboardWord error rate (lower is better) and inverse real-time factor (higher is faster). The About tab shows the formula and the languages actually scored. Swedish is absent from the multilingual list.
- VBench paperSixteen separate scores for generated video, including subject consistency, motion smoothness, and temporal flickering. Code and prompts: https://github.com/Vchitect/VBench
- Mozilla Common Voice, SwedishRecord or check Swedish speech. Common Voice releases donated clips under the CC0 public-domain dedication. Do not try to identify speakers.
- ITU-T G.114, One-way transmission timeOfficial guidance: do not plan a general network above 400 ms one-way delay, and expect voice calls and video meetings to feel delayed well below that. Free summary and PDF on the page.
- Pexels licenseFree to use and modify. Attribution optional. No unaltered resale, no implied endorsement, no offensive use of identifiable people, no upload to other stock sites.