org.terrier.applications

Class ThreadedBatchIndexing

java.lang.Object

org.terrier.applications.BatchIndexing

org.terrier.applications.ThreadedBatchIndexing

public class ThreadedBatchIndexing
extends BatchIndexing

An implementation of BatchIndexing that uses Java 8 parallel streams to increase indexing speed on multi-core machines.

Since:

4.2

Author:

Craig Macdonald

Nested Class Summary

Nested classes/interfaces inherited from class org.terrier.applications.BatchIndexing

BatchIndexing.Command

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	logger The logger used

Fields inherited from class org.terrier.applications.BatchIndexing

blocks, path, prefix

Constructor Summary

Constructors

Constructor and Description
ThreadedBatchIndexing(String _path, String _prefix, boolean _singlePass)
ThreadedBatchIndexing(String _path, String _prefix, boolean _singlePass, int threads)

Method Summary

Modifier and Type	Method and Description
void	index()
protected static Collection	loadCollection(List<String> files)
void	setMaxThreads(int threads) Define maximum number of threads in use.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static org.slf4j.Logger logger

The logger used

Constructor Detail

ThreadedBatchIndexing

public ThreadedBatchIndexing(String _path,
                             String _prefix,
                             boolean _singlePass)

ThreadedBatchIndexing

public ThreadedBatchIndexing(String _path,
                             String _prefix,
                             boolean _singlePass,
                             int threads)

Method Detail

loadCollection

protected static Collection loadCollection(List<String> files)

setMaxThreads

public void setMaxThreads(int threads)

Define maximum number of threads in use. -1 for no limit.

index

public void index()

Specified by:

index in class BatchIndexing