The Best open source Web Crawling Frameworks On my hunt for the right back-end crawler for my startup I took a look at several open source systems. A fter some initial research I narrowed the choice down to the three systems that seemed to be the most mature and widely used: Scrapy (Python), Heritrix (Java) and Apache Nutch(Java). What is …