Too much security can kill your server

Sometimes you have to learn the hard way and that is what I did a few days ago.
But let me start at the beginning. At the company I am working for we have one golden rule
that says that every public website that we launch needs to be tested. I am talking about a pentest here.
A pentest bascially checks your website for any kind of vulnerability like XSS (cross site scripting),
SQL injection and all these nasty kind of attacks. These test are performed by an external company
pentest.co.uk. They are doing a great job.
So a few weeks ago we launched a new website that I am responsible for and of course a pentest
was executed last week. The results are pretty impressive (thanks to jZonic) and no vulnerabilities were found.
Anyway of course they usually also complain if your servers are too noisy. For example if they show up
their name and version in the HTTP header or on a 404 page. Also they firgured out that the HTTP TRACE
method was still available. Nothing serious but it usually reveals too much internal information.
Our livesystem usually combines an apache2.2 running upfront and then resin3.2 for the webapplication
itself and both working together using mod_proxy.
Now I tried to fix the issues but changing the server header in apache is not that simple. But there was
mod_security to the rescue. I installed it and simply used the default configuration. In case you do not
know mod_proxy you can check it out here. There is a huge ruleset available that you can use.
The installation is pretty simple and you can for example change the server header. After the installation
I was pretty happy since I solved the open issues and went back to my regular work.
During the day I saw that all of the sudden the performance of the website went nuts. A "htop" revealed that all of a sudden the machine was really busy. But traffic did not went up as well. Even a restart of resin did not solved the problem. After some digging I figured out that it probably was a problem with mod_security. It was the only thing that I changed that day so it had to be it. First step was to disable it and the server went
back to stable conditions immediately.
Now it was the time to probably take a deeper look at the configuration of the mod_security.
That was actually my problem. You can download a HUGE set of rules for mod_security and that is what I did and I enabled ALL of them. So it was time to take a closer look at these rules. I was able to get rid of 70% of the rules. After enabling the mod again the server is still in stable conditions.
The conclusion is that mod_security is great. It should be part of any apache installation. BUT take a closer look before you use it.
That is the story how too much security killed my poor server.