Which Transformer Mechanism Lets Distant Tokens Inform Each Other

Company: Deloitte USI_5sep

Difficulty: medium

Problem Statement

Question 1

A Transformer processes a sentence and needs each token representation to incorporate information from other relevant tokens, even when those tokens are far apart. Which mechanism primarily provides this capability?