爬虫工具默认一个被抓取页面的外链接限制在100，修改db.max.outlinks.per.page的值到一个更高的数值(比方说1000)或者设置为没有限制(-1)
file:conf/nutch-default.xml
db.max.outlinks.per.page
-1
The maximum number of outlinks that we'll process for a page.
If this value is nonnegative (>=0), at most db.max.outlinks.per.page outlinks
will be processed for a page; otherwise, all outlinks will be processed.
阅读(536) | 评论(0) | 转发(0) |
