Audit Log Field Guide — Time Travel and Humblebrags

In a previous post, I wrote about audit logs and the compliance frameworks which rely on them. What I didn’t cover is what you need to include in your audit logs to make them useful. After many years of painfully trying to develop and update audit logs for various products and even more time spent scouring the internet looking for a solution, I can share what I’ve found to be the optimal answer.

First, we’ll start with what you absolutely need to know. At the core of every audit log is the basic question of “who did what when?” It’s not too hard to take that question and turn it into three simple fields:

• Timestamp — When was the auditable activity performed?

• Actor — Who performed the auditable activity?

• Message — What auditable activity was performed?

That seems pretty straightforward — easy peasy. If only life was that simple. The problems with this simple approach start to creep in when you introduce the nuance of real-world use cases. Consider the following scenario.

Say we’ve built an exercise app for tracking a person’s runs. A customer calls up and reports many of her recorded runs from this year have been deleted. She’s very distraught because she can no longer humblebrag to her friends by “accidentally” letting them see her total mileage on her phone (this is for sure not based on a real person).

A search of the system’s audit records should be able to help us get to the bottom of this urgent mystery. However, when we search the audit logs, we realize we have no easy way to search for “delete” actions with the fields we’ve implemented. Further, we have no easy way to limit our search to just this customer’s activity. Fortunately for us, we have a kitted out DeLorean and drive 88 MPH into the past to update our audit design by adding a few more fields:

• Action — We can go with traditional CRUD actions here or have actions specific to our app. This allows us to limit our search to just “delete” actions in this use case.

• Target — Which records were targeted by this action. This can include record IDs and the ID of the owner of the record. In this case, we’d store the id of the run and the ID of the customer.

• Status — Here, we can record whether the result of the action was successful or not. Unsuccessful “delete” actions are of no interest to us in this example.

Armed with these new fields, we can easily search our audit logs and get to the bottom of this deep state conspiracy. At least, we thought we could. You see, the logs are telling us that it was her account that deleted all her runs, but she swears that she didn’t do it (“I would never”). She sounds serious, so we take her word for it. We hop back in our time machine and add one more field to our logs:

• Source — With source, we can record the IP address, user-agent, and location from which the changes were made.

Uh oh, when we review the source of the activity, we’re legitimately shocked to see it’s coming from North Korea. Well, maybe that’s where she lives? Nope. After reviewing her other activity, it’s clear that sunny southern California is her home base. Apparently, North Korea really hates a humblebragger and has made it their mission to take our customer down a peg.

On the bright side, we search the source field for any other activity from North Korea across all our other customers, and we find none. It must’ve been a targeted nation-state attack designed to ostracize her from her social circle.

We advise her to update her password and enable 2FA, so it doesn’t happen again — but she wants her data back… NOW. She needs this information to reestablish herself as the alpha in her group.

What to do? What to do? An idea strikes like lightning on a clock tower, and we do our best Dukes of Hazzard across the hood of our car, lift open those gull-wing doors, and head back one last time to add:

• Old value/New Value — These two fields are used to record the value of a field before and after a change was made. One might ask, why do you need “new value,” isn’t that just the current value? It could be… if the record was only updated once. New and old values allow audit records to record the complete history of changes over many updates.

So, returning one last time to the present, we update all her runs to the correct mileage, using the “old value” field, and our customer is back to subtly crushing the spirit of her unathletic friends.

While this example is a little tongue-in-cheek, this represents an amalgamation of real-world experience that has led to the following complete set of field recommendations:

• Timestamp

• Actor

• Source

• Action

• Target

• Status

• Message

• Old Value

• New Value

With these fields, you’ll be able to confidently answer the tough questions you or your customers may ask. Further, when it comes time for an auditor to talk compliance, they’ll have an easier time reviewing your logs and validating that they meet the requirements. If you’d like to try a solution that works out of the box, with SDKs in Javascript, Python, and Go, sign up for Pangea’s Secure Audit Log API. Now go forth, and audit.