VANTAGE-Bench

Evaluate your model using the official evaluation harness to generate predictions, then upload them here. Predictions are scored against held-out ground truth, and results will be emailed to you.

01Identity

Candidate name *

The name that will appear on the leaderboard.

Organization *

Your company, university, or team name.

Model card or paper URL

Link to your model (Hugging Face, arXiv, or project page). Required for leaderboard linking, optional otherwise.

Contact email *

Results and updates will be sent here.

02Submission type

Pipeline type *

Single modelOne model used for all tasks. One forward pass per sample.

System / pipelineA combination of multiple models, components, or post-processing steps.

System access type *

What is the most restrictive access level required to reproduce this submission?

Fully open-weightAll components are publicly available.

Mixed (open + proprietary)Combination of open and closed components.

ProprietaryRequires closed models or APIs.

03Model configuration

Provide details for the primary or most representative model in your submission. If multiple models are used equally, you may write multiple and describe them in the pipeline description in Section 02.

Primary model / checkpoint *

The main or representative model in your system.

Parameter count

Write undisclosed if not public.

Inference precision *

Numerical precision used during inference.

Training type *

Zero-shotBase model evaluated without task-specific training.

Fine-tunedModel was further trained before evaluation.

04Inference setup

Primary inference infrastructure *

Hardware used for the main reasoning component. For API-based systems, write API.

Official evaluation harness

Did you use the official VANTAGE-Bench evaluation harness to generate these predictions?

YesPredictions were generated using the official harness.

No — custom inference codeI used my own inference pipeline.

Additional hyperparameters

Non-default inference settings. Leave blank if using harness defaults.

05Pillars submitted

Select all pillars you are submitting *

At least one required. Submissions must cover all tasks within a pillar — partial pillar submissions are not accepted. Overall score is computed only over submitted pillars.

Pillar ISemantic

Event VerificationMacro F1

Video QATop-1 Accuracy

Pillar IISpatial

Referring ExpressionsmIoU

Spatial PointingTop-1 Accuracy

Object LocalizationF1@0.5

Pillar IIITemporal

Temporal LocalizationmIoU

Dense Video CaptioningSODAc

Pillar IVSpatio-Temporal

Single Object TrackingSuccess AUC

06Predictions file

Predictions archive *

Upload a .tar.gz archive containing one .jsonl file per task. Maximum size: 500MB.

📦

Click to upload or drag and drop

.tar.gz · max 500 MB

07Acknowledgements

Please confirm all statements:

I am submitting predictions generated using the model and configuration described above.The results reflect actual model performance without manual modification.These results are reproducible given the same setup.I understand submission limits (2/day, 30 lifetime per email). I confirm this submission represents a distinct model configuration from any prior submissions.I agree to the VANTAGE-Bench evaluation terms.

Submit your predictions.