EMA :5: Exception Logging Causing Memory Pressure

Too many exceptions can also cause your application to perform poorly, as exception object creation is an expensive operation and an exception object takes more memory than a normal object as it stores the complete stack trace. And if you have incidents enabled on your environment or server, then too many exception can cause an adverse affect, as shown in my next example. In this case, one of the server was throwing a lot of exceptions and the log writing mechanism was storing the stack trace in a LogRecord[] array before writing it to the incident dump file. The problem was that the frequency of the occurance of exceptions outperformed the server mechanism of incident creation. The result was that the server went OOM and crashed. LogRecord is reported as retaining around 14% of heap, and the relevant classloader was holding onto 18% of heap as shown in the Leak Suspects Report below:

On digging further, figured out that each LogRecord is having a HashMap, which is having multiple keys stored in HashMap.Entry. And within that, the key SRC_LINES stores the complete stack of 1 or multiple exception stack traces, as shown below:

When I was analyzing the issue, I was not aware what was going on. When I saw the stack trace which showed reference to incident, I became certain that its related to incident dumping. The above figure shows it’s related to incident dumping. The above stack usually ended as shown below:

When I checked the incident directories, figured out that the exceptions with the same stack trace are getting listed in the incident dump files. The next step was to work with the Development team to fix these errors.