Package org.terrier.services.websitesearch.crawler4j

Provides classes for crawling websites on the fly by linking into Crawler4J.

See: Description

Class Summary

Class	Description
CrawlStrategy	Overrides Crawler4J methods in WebCrawler to enable restriction to a named host and to connect to the Terrier index.
CustomIndexData	This is a data structure that holds all of the information that the crawler needs to determine what to crawl and what to do with the pages when done crawling

Package org.terrier.services.websitesearch.crawler4j Description

Provides classes for crawling websites on the fly by linking into Crawler4J.