This note will provide a quick overview of different out of memory errors you might enounter, how to identify the type of the error, how you can analyze them further and how you can try to avoid them in future. We will also see some JVM flags which can help us in this analysis.

We will see the four main cases of out of memory error:

1. Not enough native memory for JVM

2. Not enough permgen/metaspace memory in heap

3. Not enough heap memory

4. GC overhead limit exceeded

Not enough native memory for JVM

Identification: Can be identified using 'native' in error message.

Analysis: In this case, we should look further into the actual native memory issue rather than any heap tuning.

Not enough permgen/metaspace memory in heap

Identification: Look for 'metaspace' or 'permgen space' in the error. 

Possible Causes: Can be caused due to too many classes to be loaded or a classloader memory leak.

Possible Solution: 

If you are getting Out Of Memory Error for permgen/metaspace, then try increasing the max size.

Classloader memory leak is a bug, increasing size of permgen/metaspace to solve such an error will only defer error. Hence it should be analyzed and fixed. Classloading memory leak can be identified by analyzing heap dumps or you can use jmap -permstat (Java 7) or jmap -clstat in (Java 8).

However if it is a third party product, you are left with only options to raise a bug with them or increase the permgen/metaspace max size. You can also do a simple search to see if this is a known issue with the product.

Minimum and maximum permgen is set via -XX:PermSize and -XX:MaxPermSize

Minimum and maximum metaspace is set via -XX:MetaSpaceSize and -XX:MaxMetaSpaceSize

Not enough heap memory

Identification: Look for 'heap space' in error 

Possible Causes: Can be caused due to too many live objects or a memory leak(unused objects do not go out of scope).

Possible Solution: If too many objects are there and if that is expected, increase size of heap. A memory leak is a bug, and increasing size of heap in such a case, will only defer error. Collection classes are most common cause of memory leak as application inserts into collection and never frees them. Items should be proactively discard objects when not needed or Use weak or soft references.'

JVM flags to take heap dumps automatically

Heap dumps can help you analyze out of memory errors in heap and you can use some JVM flags to take heap dumps automatically on an out of memory error to analyze in detail.

-XX:+HeapDumpOnOutOfMemoryError (default value is false) - Will cause the JVM to create a heap dump whenever an out of memory error is thrown.

-XX:HeapDumpPath=<path> - This flag specifies the location where the heap dump will be written; the default is java_pid<pid>.hprof in the application’s current working directory. Can specify file name or directory.

-XX:+HeapDumpAfterFullGC - This flag generates a heap dump after running a full GC.

-XX:+HeapDumpBeforeFullGC - This flag generates a heap dump before running a full GC.

If the application has a memory leak, then take successive heap dumps a few minutes apart and compare them using a tool. Eclipse Memory Analyzer (MAT) is one useful tool: if two heap dumps are open, then mat has an option to calculate the difference in the histograms between the two heaps.

GC overhead limit exceeded

Identification: Look for ‘GC overhead limit exceeded’ message in error.

Cause: There are four conditions that should hold true:

1. Time spent in full GCs exceed 98% (can be changed using flag: GCTimeLimit)

2. Memory reclaimed by full GC is less than 2% (can be changed using flag: GCHeapFreeLimit)

3. Above two conditions hold true for 5 consecutive Full GC cycles.

4. UseGCOverheadLimit flag is not disabled (it is enabled by default).

If the first two conditions hold true for 4 consecuting full GCs, all soft references are freed before 5th full GC so that there is more chance that more than 2% of heap is freed (condition 2). However, this might throw away the cashed results unexpectedly. If you notice such a situation, you can try to reduce value of the flag SoftRefLRUPolicyMSPerMB, so that soft references will be reclaimed more frequently and avoid filling up. See the note on Soft references for more details on this flag.

Possible Solution: Can tune GC to avoid this.