Artificial Analysis has released version 5.0 of its Intelligence Index, a benchmarking framework designed to provide a holistic synthesis of language model capabilities. Unlike single-metric evaluations, the Intelligence Index aggregates performance across four distinct categories—Knowledge Work, Coding, Science, and General—each weighted at 25%. This structure is intended to offer a more balanced comparison of model intelligence than any single benchmark, though the authors note that the index has limitations and may not apply to every specific use case.
This image represents the Artificial Analysis Intelligence Index framework described in the article.
The v5.0 index incorporates nine specific evaluations to populate these categories. For Knowledge Work, it utilizes AA-Briefcase v1.1 and AutomationBench-AA, which test agentic task completion and SaaS workflow automation. The Coding category is measured via FrontierCode 2.0 and Terminal-Bench 4.0, focusing on open-source repository changes and terminal-based task execution. Science capabilities are assessed through Terminal-Bench-Science 0.1 and CritPt v2 (Hard Subset), while General intelligence is measured using AA-Omniscience v1.1 and HLE-Diamond. These evaluations prioritize real-world applicability, often requiring models to interact with tools or produce specific file outputs in a sandbox environment.
To ensure fairness and transparency, the methodology follows four core principles: standardization of conditions (such as temperature and prompting), unbiased evaluation that accommodates valid variations in output, zero-shot instruction prompting to test raw following of directions, and full disclosure of methodology. For example, the AA-Briefcase v1.1 evaluation uses a multi-week simulated workplace scenario where agents must complete tasks using a code-execution tool in an E2B sandbox. The index also provides a framework for calculating the cost and time per task, using a weighted average of input, output, and cached tokens to help users understand the economic and temporal trade-offs of different models across the entire suite of tasks. The Intelligence Index is primarily text-based and English-language; separate indices are maintained for multilingual, visual, and mathematical performance.