Arena Elo — Vision

A separate Elo leaderboard for vision-language models, built only from Arena battles whose prompt included an image.

Also known as: Chatbot Arena Vision leaderboard, Multimodal Arena, LMArena Vision

unassessed

What it measures

arena_elo_vision is a separate Arena leaderboard computed only from anonymous battles whose prompt included an image: two vision-language models are compared head-to-head on the same image-plus-text input, and a voter picks the preferred response before identities are revealed. It launched in June 2024 as the Multimodal ("Vision") Arena, drawing over 17,000 votes across more than 60 languages in its first two weeks, on tasks the team observed spanning captioning, visual math questions, document understanding and meme explanation. It is rated with the same Bradley-Terry pipeline as the Text arena, but over its own, separate vote pool.

Task format

Anonymous, randomized side-by-side chat where the user's prompt includes at least one image; a user votes for the preferred response.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub