Terrier Users :  Terrier Forum terrier.org
General discussion about using/developing applications using Terrier 
java.lang.OutOfMemoryError: GC overhead limit exceeded
Posted by: serina ()
Date: February 09, 2018 07:12PM

Hi terrier users,


I encountered this error during my collection index:
------------------------------------------------------------------
20:52:19.478 [main] INFO o.t.indexing.SimpleXMLCollection - Found 1 documents i
n C:\Users\Serina\mahak\mahak1\WebIR-000.xml
java.lang.OutOfMemoryError: GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOfRange(Unknown Source)
at java.lang.String.<init>(Unknown Source)
at java.lang.String.toLowerCase(Unknown Source)
at java.lang.String.toLowerCase(Unknown Source)
at org.terrier.indexing.SimpleXMLCollection$XMLDocument.doNode(SimpleXML
Collection.java:162)
at org.terrier.indexing.SimpleXMLCollection$XMLDocument.doRecursive(Simp
leXMLCollection.java:126)

--------------------------------------------------------------------

I know this error is due to lack of memory. Please tell me if I can do something to solve this problem on my system?
Or should I change my system and use a higher ram system?

Options: ReplyQuote
Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Posted by: craigm ()
Date: February 10, 2018 03:21PM

You can increase the amount of memory used by Terrier by setting the TERRIER_HEAP_MEM environment variable.

Craig

Options: ReplyQuote
Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Posted by: serina ()
Date: February 10, 2018 07:34PM

Hi sir,

Thank you so much

I just did not understand well. Should I set the following attribute in the terrier.properties file?

------------------------------
memory.reserved=50000000
memory.heap.usage=0.85
------------------------------

If yes, what should I put on?
And if so, where exactly should I change the amount of memory?

Sorry I ask a lot of questions. My work is very dependent on the index of this collection

Options: ReplyQuote
Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Posted by: craigm ()
Date: February 11, 2018 11:07AM

An environment variable is different from a Terrier properties file. See
[www.google.co.uk]

Does this error occur in indexing the first file of the collection?

Craig

Options: ReplyQuote
Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Posted by: serina ()
Date: February 11, 2018 11:13PM

Hi Mr.Craigm

Yes, this is exactly the first Xml file that wants to index.
I know how to set the Environment variable. I just do not know how much I want. What do you mean by adding TERRIER_HEAP_MEM as this variable? How much do I spend on that variable?

Do you think I can index my collections on a strong server?
Can I tell if this error is due to the terrier setting or because my system is weak?
Should I change this environment variable on a strong server?

Forgive me for taking your time.
please guide me.

With respect,serina.



Edited 1 time(s). Last edit at 02/11/2018 11:37PM by serina.

Options: ReplyQuote
Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Posted by: craigm ()
Date: February 15, 2018 10:13AM

**PLEASE DO NOT START NEW THREADS FOR THE SAME TOPIC***
I have deleted your duplicate post.

Default TERRIER_HEAP_MEM values are 1024MB.

I guess your machine has more memory. So try a larger value, e.g.

TERRIER_HEAP_MEM=8G

Options: ReplyQuote
Re: java.lang.OutOfMemoryError: GC overhead limit exceeded
Posted by: serina ()
Date: February 15, 2018 01:46PM

Hi,

Thank you for your feedback
I'm sorry if I break your forum rules.Unfortunately, I face a severe constraint
I had to post in a new thread to answer.

thanks alot

Options: ReplyQuote


Sorry, only registered users may post in this forum.
This forum powered by Phorum.