HomeResearch LibrarySemantic Uncertainty: Linguistic Invariances for Uncert…
theoretical frameworkChapter 3ICLR 2023 · 2023

Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation

Lorenz Kuhn (Oxford), Yarin Gal (Oxford), Sebastian Farquhar (Oxford)

Abstract

We introduce semantic uncertainty, an entropy-based uncertainty measure for free-form natural language generation. Semantic uncertainty accounts for the fact that many different sentences can express the same meaning.

Key Contributions

  • →Semantic uncertainty measure
  • →Entropy-based confidence estimation
  • →Linguistic invariance for NLG evaluation
Eigenvector Commentary

Semantic uncertainty is the right tool for detecting when an agent is operating outside its reliable knowledge boundary. For enterprise governance, knowing when to escalate to human review requires exactly this kind of calibrated confidence signal.

Topics

uncertainty estimationsemantic uncertaintyNLGreliability