Evaluation video where the right answer is known
A test set is only useful if you know what should happen in every clip. Because we write the script, we do: each scene has a stated action, order of events and outcome, filmed under conditions you control and held out from anything public.
Examples of what we stage
- The same action repeated with one factor changed at a time
- Scenes with a stated order of events, for testing temporal understanding
- Fixed camera moves such as pans, push-ins and orbits, for testing camera control
- Matched pairs that differ in a single detail
- Lighting and background changes on an otherwise identical scene
- Deliberately hard cases: occlusion, clutter, low light, look-alike objects
How the variation works
A test set is small and precise. A single shoot day can cover a dozen scripted scenes with several controlled variations each, and the set can be refreshed with new scenes each cycle so it never leaks into training data.
Why it has to be staged
Public benchmarks get absorbed into training sets, and crowd footage has no ground truth. A commissioned set stays private, and its answers come from the script, not from after-the-fact labelling.
What you receive
Video files for each camera, one metadata record per clip with the script, the variation and the expected outcome, a manifest, a QA report and the performer releases. Folder layout, file formats and field names follow your spec. See the default delivery structure.
Rights and handling
Every performer signs a release that names AI training and likeness. Exclusivity, retention and reuse are written into the contract before the shoot, and delivery goes to the storage you approve.
Getting started
Most projects begin with a pilot: one shoot day of about 10 to 15 scripted scenarios. Book a free planning call and we reply with a shoot plan and a quote.
Other scenario types: Safety events · Human-robot interaction · Performance and conversation