Java Stacktrace filtering utility.

Introduction: Problem definition and sugested solution idea

This article is a a technical article for Java developers that suggest a solution for a major pain point of analyzing very long stacktraces searching for meaningful information in a pile of frameworks related stacktrace lines. The core idea of the solution is to provide a capability to intelligently filter out irrelevant parts of the stacktrace without losing important and meaningful information. The benefits are two-fold:

1. Making stacktrace much easier to read and analyze, making it more clear and concise

2. Making stacktrace much shorter and saving space

Stacktrace is a lifesaver when debugging or trying to figure out what went wrong in your application. However, when working with logs on the server side you can come across huge stacktrace that contains the longest useless tail of various frameworks and Application Server related packages. And somewhere in this pile, there are several lines of a relevant trace and they may be in different segments separated by useless information. It becomes a nightmare to search for a relevant stuff. Here is a link that describes the same problem with real-life examples (not for the fainthearted :)) https://dzone.com/articles/filtering-stack-trace-hell

Despite the obvious value of this capability, the Java ecosystem offers very few, if any, libraries with built-in support for stacktrace filtering out of the box. Developers often resort to writing custom code or regex filters to parse and shorten stacktraces—an ad-hoc and fragile solution that’s hard to maintain and reuse. Some logging frameworks such as Log4J and Logback might provide basic filtering options based on log levels or format, but they don't typically allow for the granular control over stack trace

How the solution works and how to use it The Utility is provided as part of Open Source Java library called MgntUtils. It is available on Maven Central as well as on Github (including source code and Javadoc). Here is a direct link to Javadoc. The solution implementation is provided in class TextUtils in method getStacktrace() with several overridden signatures. Here is the direct Javadoc link to getStacktrace() method with detailed explanation of the functionality.

So the solution is that user can set a relevant package prefix (or multiple prefixes srting with MgntUtils library version 1.7.0.0) of the packages that are relevant. The stacktrace filtering will work based on the provided prefixes in the following way:

1. The error message is always printed.

2. The first lines of the stacktrace are always printed as well until the first line matching one of the prefixes is found.

3. Once the first line matching one of the prefixes is found this and all the following lines that ARE matching one of the prefixes will be printed

4. Once the first line that is NOT matching any of the prefixes is found - this first non-matching line is printed but all the following non-matching lines are replaced with single line ". . ."

5. If at some point another line matching one of the prefixes is found this and all the following matching lines will be printed. and now the logic just keep looping between points 4 and 5

Stacktrace could consist of several parts such as the main section, "Caused by" section and "Supressed" Section. Each part is filtered as a separate section according to the logic described above.

Also, the same utility (starting from version 1.5.0.3) has method getStacktrace() that takes CharSequence interface instead of Throwable and thus allows to filter and shorten stacktrace stored as a string the same way as a stacktrace extracted from Throwable. So, essentially stacktraces could be filtered "on the fly" at run time or later on from any text source such as a log. (Just to clarify - the utility does not support parsing and modifying the entire log file. It supports filtering just a stacktrace that as passed as a String. So if anyone wants to filter exceptions in a log file they would have to parse the log file and extract stacktrace(s) as separate strings and then can use this utility to filter each individual stacktrace).

Here is a usage example. Note that the first parameter of getStacktrace() method in this example is Throwable. Let's say your company's code always resides in packages that start with "com.plain.*" So you set such a prefix and do this

logger.info(TextUtils.getStacktrace(e,true,"com.plain."));

this will filter out all the useless parts of the trace according to the logic described above leaving you with very concise stacktrace. Also, user can pre-set the prefix (or multiple prefixes) and then just use the convenience method

TextUtils.getStacktrace(e);

It will do the same. To preset the prefix just use the method

TextUtils.setRelevantPackage("com.plain.");

Method setRelevantPackage() supports setting multiple prefixes, so you can use it like this:

TextUtils.setRelevantPackage("com.plain.", "com.encrypted.");

If you would like to pre-set this value by configuration then starting with the library version 1.1.0.1 you can set Environment Variable "MGNT_RELEVANT_PACKAGE" or System Property "mgnt.relevant.package" to value "com.plain." and the property will be set to that value without you invoking method TextUtils.setRelevantPackage("com.plain."); explicitly in your code. Note that System property value would take precedence over the environment variable if both were set. Just a reminder that with System property you can add it in your command line using -D flag:

"-Dmgnt.relevant.package=com.plain."

Note that System property value would take precedence over environment variable if both are set. IMOPRTANT: Note that for both environment variable and system property if multiple prefixes needed to be set than list them one after another separated by semicolon (;) For Example: "com.plain.;com.encrypted."

There is also a flexibility here: If you do have pre-set prefixes but for some particular case you would wish to filter according to different set of prefixes you can use the method signature that takes prefixes as parameter and it will override the globally pre-set prefixes just for this time:

logger.info(TextUtils.getStacktrace(e,true,"org.alternative."));

Here is an example of filtered vs unfiltered stacktrace: you will get a following filtered stacktrace:

com.plain.BookListNotFoundException: Internal access error
at com.plain.BookService.listBooks()
at com.plain.BookService$$FastClassByCGLIB$$e7645040.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke()
...
at com.plain.LoggingAspect.logging()
at sun.reflect.NativeMethodAccessorImpl.invoke0()
...
at com.plain.BookService$$EnhancerByCGLIB$$7cb147e4.listBooks()
at com.plain.web.BookController.listBooks()

instead of unfiltered version

com.plain.BookListNotFoundException: Internal access error
at com.plain.BookService.listBooks()
at com.plain.BookService$$FastClassByCGLIB$$e7645040.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke()
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint()
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed()
at com.plain.LoggingAspect.logging()
at sun.reflect.NativeMethodAccessorImpl.invoke0()
at sun.reflect.NativeMethodAccessorImpl.invoke()
at sun.reflect.DelegatingMethodAccessorImpl.invoke()
at java.lang.reflect.Method.invoke()
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs()
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod()
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke()
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()
at org.springframework.aop.interceptor.AbstractTraceInterceptor.invoke()
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke()
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke()
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed()
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept()
at com.plain.BookService$$EnhancerByCGLIB$$7cb147e4.listBooks()
at com.plain.web.BookController.listBooks()

In Conclusion The MgntUtils library is written and maintained by me. If you require any support or have any question or would like a short demo you can contact me through LinkedIn - send me a message or request connection, or send me email directly at michael_gantman@yahoo.com. I will do my best to respond