HomeResearch LibraryAuditing Large Language Models: A Three-Layer Approach
governanceChapter 3AI & Ethics · 2023

Auditing Large Language Models: A Three-Layer Approach

Jakob Mökander (Oxford), Jonas Schuett (Cambridge)

Abstract

We propose a three-layer approach to auditing large language models: governance audits, model audits, and application audits. The framework provides a systematic methodology for enterprise AI accountability.

Eigenvector Insight — Zone III / PASF-PADE AnalysisNot part of the original paper
Eigenvector Research — Marco van Hurne
How this paper contributes to solving the Zone III problem (PASF-PADE)

The three-layer audit framework provides the systematic methodology that enterprise compliance teams need for Zone III deployments. Governance, model, and application audits must all be in place for regulated industry deployments.

Why AI is not sufficient for Zone III without this

Zone III refers to high-complexity, high-risk, long-running agentic workflows — the class of enterprise AI deployments where a single failure can cascade across hundreds of steps. Standard AI models, trained to predict the next token, are not inherently designed for durable, governed, multi-step execution. This paper addresses one or more of the structural gaps that make Zone III deployments unsafe without explicit architectural intervention.

Key Contributions

  • →Three-layer audit framework
  • →Governance-model-application audit hierarchy
  • →Enterprise accountability methodology

Topics

AI auditingaccountabilitygovernanceLLM evaluation
Relevance Scores
Long-Horizon Score76
Enterprise Score93
Completeness78
Paper Info
Year2023
VenueAI & Ethics
Typegovernance
ChapterCh. 3
Authors2
Zone III Analysis
Frameworks