The attention mechanism is the specific innovation that made Transformers possible — understanding it concretely, beyond just “tokens look at each other,” clarifies a lot about how LLMs actually process context.
Queries, Keys, and Values
For each token, attention computes three vectors: Query - "what am I looking for?" Key - "what do I represent, for others to find?" Value - "what information do I actually carry?" Attention score = how well a token's QUERY matches another token's KEY The output is a weighted sum of VALUES, weighted by those attention scores
A Concrete Intuition
In the sentence “The animal didn’t cross the street because it was too tired,” resolving what “it” refers to requires attention — the model computes a high attention score between “it” and “animal” (not “street”), effectively letting “it” pull in relevant context from “animal” when building its representation. This is exactly the kind of long-range dependency older sequential models struggled to capture reliably.
Why This Scales So Well
Because every token can attend to every other token in a single parallel computation (rather than sequentially, one token influencing the next only through a chain), attention is genuinely well-suited to modern GPU hardware, which excels at large parallel matrix operations — a major reason Transformers scale to enormous model sizes and dataset sizes more gracefully than the sequential architectures that came before them.