Aug 24, 2026Aug 24, 2026 · 2 min read · 80 viewsDesign a web crawlerAt the start we seed a few webpage which recursively add more pages from the links found on the seed pages. Uses Cases Used for search engine indexing, web archiving, web minining (scraping), web moni2 min read · 80 views
