Get the words best practice out of your vocabulary.” I have been at many customer sites that needed my expertise, and someone says to me in a meeting, “well what is the best practice to secure our information.” I’m going to tell you right now, the bad guys are reading the same best practice white papers and poking holes through them left and right. In addition, that audit report you received saying you are in compliance with <state your regulation> may be factually correct at that moment in time, but your information is still not secure.
For each of your systems, bring five of your senior system administrators into a room and ask them a simple question. “How would you compromise your system?” Then sit back and listen. If they are good inside of thirty minutes you are going to start hearing things that will scare you. Let me give you some examples from my life.
We are generating audit reports but no one is actually reviewing them. At one customer, I generated audit reports that showed invalid logins’, ip address and username logins when the the connections were simultaneous, and a host of other audit reports. I then went to our security people and asked them how often they wanted this report, daily, or weekly. I was told to review them and only bring things up to them when I find something interesting. Well for one, under the concept of separation of duties your SA should not have sole responsibility for reviewing audit logs. Yes, they should but a copy needs to be sent out for review. As the Oracle DBA I can logon as oracle or logon as sys, I am god. I can do almost anything I want and cover my tracks. Can you spell Edward Snowden? Can you say with complete confidence that you do not have an Edward Snowden in your shop?
The patch schedule is to drawn out. One shop we were twelve months behind in our Oracle CPU patches because of the perception that patching the database would impact the development schedule. This is one of those times where I got maybe a little bit forceful in a meeting and pushed for getting everything patched. But the customer was adamant, “you will not do anything that will slowdown development and testing.” We finally got everything patched when it was announced there was going to be an audit. The audit happened, and there was no findings. Our immediate management was happy, me, I was not so happy. When a patch comes out, the bad guys are reading what is being fixed and they are quite adapt at exploiting the weaknesses that are being patched.
A webserver that is miss-configured, I walked into one customer and as I was setting up my audit scripts, I noticed there were over a thousand invalid login attempts from a handful of webservers every thirty minutes. The DBAs were not talking to the webserver SA’s and the webserver SA’s would allow the invalid login attempts to continue if the application that webserver supported was no longer in use and again, no one was actually reviewing the audit logs. In fact this was a known issue that was explained as “normal” to the security group. This is the perfect way to hide password cracking attempts. One of the webservers that was in the DMZ had not been used for over a six months, but was still running. When I dug into the audit trail to see what was actually going on, I found several attempts to connect to the database as sys, system, admin, sa, root and a host of other attempts. That server had been compromised along with a few others. When I brought the evidence up to management they were shocked. Finally passwords were changed on the webservers and those webservers that were no longer in use were pulled from the network and a complete scan of the network was completed.
Excessive privileges to developers and developers using a common development account. Yup, this still happens. I walk into a shop, and given the application username password to do all my work. Folks your audit trail is now toast. At that same shop all developers have the passwords for sys and system. I never got a good reason for this. NO ONE should ever logon at sys. Application accounts should never be used for development.
This is just a small fraction of the issues I have seen in different shops.
All of these shops either followed “best practices” or modified their practice when the learned there was going to be an audit. Everyone of those shops are staffed with professional SA’s who can tell you where the weakness is. And every shop I have ever worked in has weaknesses. Your job in the C suite is to ask these professional SA’s to come into a meeting and have a safe and secure conversation on how they would crack the system. If they can not come up with anything you are either working at NSA, they are trying to hide something or you need smarter SA’s.
And don’t just do this once and say, we’re done, schedule these meetings either quarterly or semiannually. This is a conversation between the C suite and your professional SA’s. You need to understand where your risk are, these people are real smart and if you listen to them they can tell you what’s wrong and what needs to be done to fix it.
Once you have this information, give your professional SA’s and their management the tools and resources they need to close these security holes.