[TR-72] FSOrderedMapFile.EntryIterator.skip() breaks FSOrderedMapFile.EntryIterator.hasNext() Created: 20/Oct/09  Updated: 05/Mar/10  Resolved: 20/Oct/09

Status: Resolved
Project: Terrier Core
Component/s: .structures
Affects Version/s: 3.0
Fix Version/s: 3.0

Type: Bug Priority: Trivial
Reporter: Craig Macdonald Assignee: Craig Macdonald
Resolution: Fixed  
Labels: None


 Description   
FSOrderedMapFile.EntryIterator.skip() breaks FSOrderedMapFile.EntryIterator.hasNext().

This only seems to affect a corner case in Inverted2DirectMR.



 Comments   
Comment by Craig Macdonald [ 20/Oct/09 ]

Here's the patch, including test case:

Index: src/uk/ac/gla/terrier/structures/collections/FSOrderedMapFile.java
===================================================================
--- src/uk/ac/gla/terrier/structures/collections/FSOrderedMapFile.java	(revision 2750)
+++ src/uk/ac/gla/terrier/structures/collections/FSOrderedMapFile.java	(working copy)
@@ -191,6 +191,7 @@
 				di.skipBytes(toSkip);
 				actualSkipped += (long)toSkip;
 			}
+			counter += numEntries;
 		}
     }
     
Index: src_test/uk/ac/gla/terrier/structures/collections/TestFSOrderedMapFile.java
===================================================================
--- src_test/uk/ac/gla/terrier/structures/collections/TestFSOrderedMapFile.java	(revision 2694)
+++ src_test/uk/ac/gla/terrier/structures/collections/TestFSOrderedMapFile.java	(working copy)
@@ -12,6 +12,7 @@
 import org.apache.hadoop.io.Text;
 import org.junit.Test;
 
+import uk.ac.gla.terrier.structures.Skipable;
 import uk.ac.gla.terrier.structures.collections.FSOrderedMapFile.MapFileWriter;
 import uk.ac.gla.terrier.structures.seralization.FixedSizeIntWritableFactory;
 import uk.ac.gla.terrier.structures.seralization.FixedSizeTextFactory;
@@ -94,6 +95,23 @@
 		assertTrue(copyKey2Id.size() == 0);
 	}
 	
+	protected void readStreamSkip(Iterator<Map.Entry<Text, IntWritable>> iterator, int totalNumEntries) throws Exception
+	{
+		int skip = 3;
+		int entryIndex = 0;
+		((Skipable)iterator).skip(skip);
+		entryIndex += skip;
+		while(iterator.hasNext())
+		{
+			Map.Entry<Text, IntWritable> e = iterator.next();
+			assertNotNull(e);
+			assertNotNull(e.getKey());
+			assertEquals(testKeys[entryIndex], e.getKey().toString());
+			entryIndex++;
+		}
+		assertEquals(testKeys.length, entryIndex);
+	}
+	
 	@Test public void testStream() throws Exception
 	{
 		FixedSizeTextFactory keyFactory = new FixedSizeTextFactory(20);
@@ -103,6 +121,15 @@
 		inputStream.close();
 	}
 	
+	@Test public void testStreamSkip() throws Exception
+	{
+		FixedSizeTextFactory keyFactory = new FixedSizeTextFactory(20);
+		FSOrderedMapFile.EntryIterator<Text, IntWritable> inputStream = new FSOrderedMapFile.EntryIterator<Text, IntWritable>(
+				file, keyFactory, new FixedSizeIntWritableFactory());
+		readStreamSkip(inputStream, testKeys.length);
+		inputStream.close();
+	}
+	
 	@Test public void testOnDisk() throws Exception
 	{
 		FixedSizeTextFactory keyFactory = new FixedSizeTextFactory(20);
Comment by Craig Macdonald [ 20/Oct/09 ]

Committed to trunk.

Generated at Tue Dec 01 12:21:38 GMT 2020 using JIRA 7.1.1#71004-sha1:d6b2c0d9b7051e9fb5e4eb8ce177ca56d91d7bd8.