Making Sense of ModSecurity: JSON Audit Logs
DreamHost uses ModSecurity extensively throughout our infrastructure to provide multiple layers of web application security to customer servers and applications. As a powerful, flexible WAF (web application firewall), ModSecurity allows our security team to greatly improve the security of our web services, protect against zero-day threats, mitigate DDoS attacks, and provide insight into trends across our server landscape. It’s a powerful tool, with one glaring weakness: the logs it produces can be difficult to read, impossible to parse, and quite frankly, a nightmare to work with.
ModSecurity provides audit logs when it intercepts attacks to give server operators forensic information about a malicious request. These logs spell out all kinds of detailed information about the request – header information, request payload, port information, and more. Below is an example of a ModSecurity audit log entry:
These logs are great for eyeballing, but terrible for working with programmatically. This problem is only exacerbated when working with scale – we have tens of thousands of machines running ModSecurity, producing millions of lines of output daily. A handful of projects, tools and scripts have popped up in the past few years to try to wrangle ModSecurity logs into some form of meaningful data, and these are certainly excellent contributions to the community, but trying to shoehorn in a solution to transpose logging data after the fact at our scale is a thankless chore and a waste of resources. We needed a way to natively generate logging data in a usable format.
Determining that patching ModSecurity to generate data in a useful format, such as JSON, was a worthwhile endeavor, we devoted some time last summer to building a method to generate JSON audit logs. Thankfully, ModSecurity is a stable, mature project, and we were able to test and integrate our patch into a wide variety of environments. Now our WAF environments generate logging data in a sane way, allowing us to scale our data aggregation and analysis without the need to wrangle a pile of non-standard data 37 different ways before shoving it into storage.
As a security team, we felt that our change would benefit the ModSecurity community as a whole. Luckily, DreamHost has always had a love-love relationship with open source projects. The use of free and open software is a core philosophy as an organization. The beauty of embracing open source projects is that, as a company, we’re given the opportunity to interact with the open source community beyond simple implementation. We submitted a pull request to the upstream ModSecurity project run by SpiderLabs, and eventually the patch was committed and will be deployed in the latest stable release of ModSecurity (2.9.1).
We think this is a pretty big win for both DreamHost and the open source community. We saw a need in an open source project we use extensively, filled that need, and contributed our work back to the community. Doing so has also allowed us to continue working closely with the upstream ModSecurity dev team, providing feedback on future design, hunting down bugs, and providing a voice in the open source community. We’re excited about the future opportunities that this and other open security projects hold for our teams and our customers.