Details
-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 3.0
-
Fix Version/s: 3.0
-
Component/s: None
-
Labels:None
Description
The use of the MetaIndex during the reduce phase to merge the meta indexes. If there are lots of map tasks, this can result in too many meta indices being loaded into memory at once.
The solution is to access the MetaIndex as a stream.
The solution is to access the MetaIndex as a stream.
Committed to SVN. Also uses MapReduce job to do metaindex inversion, i.e. meta->docid