GAIA: A Benchmark for General AI Assistants

Grégoire Mialon (Meta AI), Clémentine Fourrier (Hugging Face)

Abstract

We introduce GAIA, a benchmark for general AI assistants that tests real-world capabilities requiring multi-step reasoning, tool use, and information synthesis. GAIA questions require an average of 5.4 steps to solve.

Key Contributions

→Real-world multi-step benchmark
→Tool use evaluation
→Difficulty stratification

Topics

general AI benchmarkmulti-step reasoningtool usereal-world tasks

Relevance Scores

Long-Horizon Score90

Enterprise Score84

Completeness84

Paper Info

Year2023

VenueICLR 2024

Typebenchmark

ChapterCh. 1

Authors2

Frameworks

PASF AEGIS