A Comprehensive Review on Designing a New Model and Architecture for  Effective Web Crawling:

Somika Kumari,Dr. Radhe Shyam Soni

A Comprehensive Review on Designing a New Model and Architecture for Effective Web Crawling:

Authors

Somika Kumari,Dr. Radhe Shyam Soni

Keywords:

web crawling, intelligent crawlers, machine learning, URL prioritisation, distributed architecture, reinforcement learning, focused crawling, dynamic content, scalable systems, PageRank, information retrieval.

Abstract

The World Wide Web has grown into one of the largest and most complex informationsystems in human history, containing billions of pages across millions of domains and expandingat an unprecedented rate. Web crawling — the automated systematic traversal and retrieval of web content — forms the foundational infrastructure

References

Abiteboul, S., Preda, M., & Cobena, G. (2003). Adaptive on-line page importance computation. Proceedings of the 12th International Conference on World Wide Web (WWW '03), 280 290. https://doi.org/10.1145/775152.775192

Downloads

Published

2024-10-12

How to Cite

Somika Kumari,Dr. Radhe Shyam Soni. (2024). A Comprehensive Review on Designing a New Model and Architecture for Effective Web Crawling: . Journal of Computational Analysis and Applications (JoCAAA), 33(08), 8007–8026. Retrieved from https://eudoxuspress.com/index.php/pub/article/view/5124