Heritrix

Your main task is scrape specific pages from the web site.

Nutch: Open-source web-search software, built on Lucene Java

Heritrix: is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project

So I think Heritrix is much better than Nutch for your project.

Learning a framework/library is a valuable exercise. But it takes some time. Since you task is not very complex one, sometimes it would be less painful to write a simple crawler from the scratch in Java

Nutch

StackOverfow

crawler
Nutch is the best you can do when it comes to a free crawler. It is built off of the concept of Lucene (in an enterprise scaled manner) and is supported by the Hadoop back end using MapReduce (similar to Google) for large scale data querying. Great products! I am currently reading all about Hadoop in the new (not yet released) Hadoop in Action from manning. If you go this route I suggest getting onto their technical review team to get an early copy of this title!

These are all Java based. If you are a .net guy (like me!!) then you might be more interested in Lucene.NET, Nutch.NET, and Hadoop.NET which are all class by class and api by api ports to C#.

Pablo Hoffman, Lead developer of Scrapy

http://www.quora.com/Web-Crawling/What-is-the-best-open-source-web-crawler-and-why There is also Scrapy (Python based) which is faster than Mechanize but not as scalable as Nutch or Heritrix, which means that it's not meant to be used for crawling the entire web, but it's OK for crawling a lot (5000+) of sites, even huge ones like Amazon.

Heritrix vs Nutch

http://webcache.googleusercontent.com/search?q=cache:qN3PZw2Hd7UJ:omricohen.me/post/9034412576/the-crawler-heritrix-vs-nutch+nutch+vs+heritrix&cd=10&hl=en&ct=clnk&gl=us

The Crawler: Heritrix vs. Nutch

So we started to work on our Crawler… most definitely one of the more challenging tasks…

At the beginning we considered very seriously to work with Nutch to give us the full solution of indexing, crawling and searching, since we are going to use Solr in any case (with Lucene). Nutch, for the ones of you who don’t very familiar with the family of Apache Lucene products, is a whole solution that use Lucene as an index engine and Solr as a Search server, and it also comes with a Crawler.

Nutch crawler is good for certain purposes.. and it is great for out of the box use. But we have to remmember that the “Crawling” will be one of our most major tasks. Therefore, we decided to look for a more generic, open-source crawler. The search for such crawler led us to Heritrix.. the quest after a great crawler cam to an end.

Heritrix comes in three main versions (plain heritrix, heritrix 2 and heritrix 3), we sticked with the plain Heritrix (v1.14.4), it was (and actually still is) great. For those of you that prefer using it out of the box, it comes with a great Web UI, that makes life really easy for first setup and use. For the developers among you, I can only say that this is, in my humble opinion, the best choice for a crawler.

Heritrix designed in a generic way and the documentation both for plain users and for developers is much over the expected. The code is very well documented, and all at all it is very convenient to add modules of your own and to implement your own logic and needs in it. The Heritrix community is also very good and active. We found many modules with special capabilities we needed already written.

So to sum up, if you first came here to find a comparison between Nutch and Heritrix and found me praising to Heritrix, well there is a very good reason.. It really is one of the more well designed piece of software I have ever encountered..