HomeResearch LibraryAttention Is All You Need
system architectureChapter 7NeurIPS 2017 · 2017

Attention Is All You Need

Ashish Vaswani (Google Brain), Noam Shazeer (Google Brain)

Abstract

We propose the Transformer, a model architecture based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. The Transformer achieves state-of-the-art results on machine translation tasks.

Eigenvector Insight — Zone III / PASF-PADE AnalysisNot part of the original paper
Eigenvector Research — Marco van Hurne
How this paper contributes to solving the Zone III problem (PASF-PADE)

The Transformer is the foundation upon which all modern LLM agents are built. Including it in this corpus is a reminder that Zone III is built on a decade of foundational research — and that the architectural choices made in 2017 still constrain what agents can do today.

Why AI is not sufficient for Zone III without this

Zone III refers to high-complexity, high-risk, long-running agentic workflows — the class of enterprise AI deployments where a single failure can cascade across hundreds of steps. Standard AI models, trained to predict the next token, are not inherently designed for durable, governed, multi-step execution. This paper addresses one or more of the structural gaps that make Zone III deployments unsafe without explicit architectural intervention.

Key Contributions

  • →Transformer architecture
  • →Self-attention mechanism
  • →Multi-head attention

Topics

transformerattention mechanismneural architecturefoundational