Attention Mechanism

A neural network component that assigns different weights to different parts of the input, allowing the model to focus on the most relevant information and providing a form of interpretability.

The part of an AI that decides what to focus on. When translating a sentence, the attention mechanism might focus on the verb first, then the subject; similar to how your eyes move when reading.