Emergent Abilities of Large Language Models

Jason Wei (Google Brain), Yi Tay (Google Brain)

Abstract

We discuss emergent abilities of large language models — abilities that are not present in smaller models but appear in larger models. We survey 137 emergent abilities across 8 models.

Key Contributions

→Emergent ability taxonomy
→Scaling-capability relationship
→137 emergent abilities documented

Eigenvector Commentary

Emergent abilities are both the promise and the peril of Zone III. The promise: capabilities that enable autonomous enterprise workflows emerge at scale. The peril: emergent behaviors are unpredictable and may include undesired capabilities that complicate governance.

Topics

emergent abilitiesscalingLLM capabilitiesphase transitions

Relevance Scores

Long-Horizon Score75

Enterprise Score80

Completeness76

Paper Info

Year2022

VenueTMLR

Typeempirical study

ChapterCh. 7

Authors2

Frameworks

PASF