Last year I took a look at generating Mod_Security audit logs as JSON data, rather than the module’s native format (which is… err… difficult to parse). My initial approach was incomplete, needlessly introduced additional dependencies, and leaked like a sieve; I ended up abandoning this to work on FreeWAF. Some new use uses came up that would benefit from more structured Mod_Security audit logs, so I’ve revamped a patch to emit JSON data using a more sane approach.

My new approach uses the included YAJL library, which contains a fast, streaming JSON generator. The API is relatively simple, and with a few macros added for simplicity, it’s easy to stream in new key-value pairs and manipulate arrays and maps.

The generated JSON contains five primary top-level keys: transaction (corresponding to SecLogAuditPart ‘A’), request (containing maps for request headers and body content, as well as the sanitized request line), response (response status, headers, and body), audit_data (SecLogAuditPart ‘H’ data), and matched_rules (represented as a JSON array). Below is an beautified example of an audit log entry generated from a simple rule designed to mitigate an ongoing spoofed Googlebot WordPress login bruteforce botnet:

And the corresponding Mod_Security rules:

Of note, new functions were added for generating stopwatch and producer data in JSON format. Request/response sanitation is also properly handled, using the same memory overwrites that native logging employs.

YAJL’s generator allocator provides the ability for users to specify a struct containing function pointers to memory allocation routines to replace the default (malloc, realloc, and free). This patch does not leverage that or the available APR pools, largely to simplify the changeset, and because performance testing indicated no significant difference between the two. Indeed, benchmark tests for both patched JSON and natively audit log generation shows no significant difference; JSON logging maybe be slightly more performant because it contains only a single atomic write (native audit logs may make several dozen or hundred write syscalls). The choice to buffer the entire audit log entry and call a single write is based on the desire to avoid having broken JSON structures written to disk.

Building with this patch is straightforward; just configure with –enable-json-logging=yes and build Configuring this option is straightforward, with a new configuration option ‘SecAuditLogFormat’, which takes the string ‘Native’ or ‘JSON’:

Audit log data is not written in a beautified fashion (what a pointless endeavor would that have been!), and is newline-separated. I’ve submitted a pull request for this patch, though I’m not holding my breath, as the json_logging branch on upstream looks like the developers want to enable a runtime option for enabling JSON logging, an approach this patch now follows. I’m hoping people find this useful, and will happily entertain comments/requests regarding the structure and data points present within the generated entries.

Update: Since posting this, I’ve made a few updates to the json_logging_audit branch, including removing the compile-time flag, and updating the matched_rules portion of the structure (see this commit for more details). The example audit log entry shown above is the correct version of the data emitted.