Lost-in-the-middle

Attention phenomenon (Liu et al. 2023, arXiv:2307.03172) where models show degraded recall for information in the middle of long contexts. U-shaped performance curve: primacy and recency positions show 27-98% accuracy vs 27% for middle positions.