From PageRank to Transformers: Tracing the Evolution of Attention and Ranking Mechanisms in Modern AI
Keywords:
PageRank; attention; Transformers; Markov chains; eigenvectors; graph neural networks; personalized PageRank; rankingAbstract
Two of the most consequential ideas in modern computing—PageRank, which ordered the early web, andthe attention mechanism, which powers today's large language models—are usually told as separate stories. This article argues they are chapters of one story about computing importance over a set of related elements.PageRank assigns each web page
References
S. Brin and L. Page, “The anatomy of a large-scale hypertextual web search engine,” Computer Networks and ISDN Systems, vol. 30, no. 1–7, pp. 107–117, 1998.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017.
Downloads
Published
2023-04-20
How to Cite
Ravi Teja Gurram. (2023). From PageRank to Transformers: Tracing the Evolution of Attention and Ranking Mechanisms in Modern AI . Journal of Computational Analysis and Applications (JoCAAA), 31(4), 3045–3054. Retrieved from https://eudoxuspress.com/index.php/pub/article/view/5618
Issue
Section
Articles
