mod_sec audit logs are atrocious. There have been a few attempts to make parsing audit data more palatable- BitsOfInfo recently wrote up a proof of concept of working through audit logs with logstash, and the AuditConsole project from JWall provides a more comprehensive approach to collecting log data, but neither solutions addresses the inherent problem of how messy native audit log data is.

As part of my Master’s thesis I need a way to efficiently audit, sort and store audit logs; building my own parser, while doable, would be a waste of time, so I’ve forked the ModSecurity project on Github and built a patch that implements JSON logging directly in the application, replacing the old data log structures entirely. There’s still a long way to go, but the initial patch seems stable and produces valid JSON. Below is an example log entry using the core CRS in DetectionOnly mode:

Right now every matched rule is being logged, so I snipped the majority of matches for brevity’s sake (again, keeping in mind this is the stock CRS). Getting this patch made was a two-part process; editing the log writer to build JSON instead of writing out to the audit file was easy. The hard part was spelunking through the disruption functions to find out where ruleset data is pushed onto an APR array (hint- the same functionality is called three different times, and the one that applies to audit logs isn’t the one called “perform_disruptive_action”… go figure). This meant changing a few helper functions to build and return JSON objects instead of strings; a future pull request might include some function duplication and allow for a compile-time option of traditional or JSON logging, but that’s a bit out of scope of what I wanted to get done initially.

Building this requires the JSON-C library; I’ve updated the autoconf files to include this library in the make process, and it’s building on my test system, but my make-fu is amateur, and this could use another pair of eyes. I’ve also only tested this on my proxy setup so far (using Nginx as a reverse proxy), so a lot more testing needs to be done before building it into a production Apache deployment, but the core idea of JSON logging is there. The branch is available here if anyone’s interested in taking a look.

Update: I’ve revisited and refactored this approach.