HomeResearch LibraryAgent Benchmarks Fail Public Sector Requirements
Chapter 6 · 2026

Agent Benchmarks Fail Public Sector Requirements

Jonathan Rystrøm, Chris Schmitz, Karolina Korgul

Abstract

This paper argues that existing benchmarks for LLM agents fail to meet the stringent legal, procedural, and structural requirements of the public sector. It defines criteria for public sector-relevant benchmarks, including process-based, realistic, public-sector-specific, and metrics-driven. An analysis of over 1,300 benchmark papers reveals that no single benchmark meets all these criteria, calling for new research and application of these criteria by public-sector officials.

Eigenvector Insight — Zone III / PASF-PADE AnalysisNot part of the original paper
Eigenvector Research — Marco van Hurne
How this paper contributes to solving the Zone III problem (PASF-PADE)

This paper directly addresses one of the core structural challenges in Zone III deployments. The research on LLM agents, benchmarking, public sector provides evidence-based foundations that enterprise architects cannot ignore when designing long-horizon autonomous workflows. The findings challenge the assumption that a base language model — however capable — can handle the complexity of durable, governed, multi-step execution without explicit architectural intervention. For Zone III practitioners, this paper belongs in the required reading list.

Why AI is not sufficient for Zone III without this

Zone III refers to high-complexity, high-risk, long-running agentic workflows — the class of enterprise AI deployments where a single failure can cascade across hundreds of steps. Standard AI models, trained to predict the next token, are not inherently designed for durable, governed, multi-step execution. This paper addresses one or more of the structural gaps that make Zone III deployments unsafe without explicit architectural intervention.

Topics

LLM agentsbenchmarkingpublic sectorevaluation criteriagovernance