The architecture behind modern AI
The Transformer is a neural network architecture published by a Google research team in 2017, in the paper “Attention Is All You Need”. It displaced the previously dominant approaches and now underlies essentially every large language model. The T in GPT is this.
Why it was a breakthrough
The core is attention. When reading, a word’s meaning depends on its relationship to the words around it — bank in a financial sentence and bank beside a river are the same string with different meanings. Attention computes which words in a passage relate to which, allowing understanding that accounts for the whole context.
Earlier approaches processed text strictly in order. A Transformer processes a passage in parallel, which made efficient training on large clusters possible and opened the path to scaling models up — that is, to LLMs.
Beyond language
Transformers are not only for text. They are applied in image recognition, speech recognition, video generation, and even protein structure prediction. Treating data as a sequence and capturing relationships through attention turned out to be that general.
The short version
You do not need the mechanics. Two points suffice: the 2017 Transformer is the technical starting point of the current AI era, and attention — capturing context — is the core idea. It appears in every article about the history of the field.
The paper’s title, “Attention Is All You Need”, is among the most recognized in AI research. It is barely an exaggeration to say ChatGPT, modern machine translation, and the present boom all begin there.