Talk with any incident responder and you'll learn that there are a few less glamorous parts of the job. Writing the final report and preparation in advance to an incident are probably top contenders. In this article I want to focus on preparation and explain to you why if you do this part right, responding to a security incident will no longer be a nightmare of weeks of chaotic 15+ hour work days with an unclear outcome, but instead an engagement you can actually come out of winning. Defense is doable and good preparation is a major element of a successful defense.
A few days ago I participated in a two-day workshop to plan an incident response mission at a site that has not yet done any preparation for a cyber security incident. The good news is that I'm going into this case with an incident response team that knows about the importance of preparation done right, how to do it and how to stay on top of a situation that will be more than a little chaotic at first. This brings me right to the value of preparation in a nutshell:
Preparation forces you to think about at minimum, the most likely scenarios you might encounter some day. Thus you will not be taken by surprise and you will be able to act much faster and in a more efficient way.
Now that might sound obvious and you might be thinking "everybody knows that." And still, a number of recent articles in German newspapers mentioned that overall, less than forty percent of German companies are prepared to deal with a serious cyber security incident. And the situation probably doesn't look all that much different in other countries. And while I am writing this, my Twitter timeline is full with comments and article links on the Equifax breach, quite probably the largest theft of personal data so far and unfortunately a sterling example of how not to handle the communications and remediation part of a cyber security incident. And finally, having an incident response plan is an important aspect for all organizations that have to comply with the coming European GDPR legislation.
So here is some advice on how to prepare for incident response. I will keep these at a high level in the hope they might also be of value to those folks not having permanent internal incident response teams, which is quite probably the majority.
So lets dive right into it. On a high level, I like to divide preparation into three different areas:
- Communications
- Environment
- Toolkit
Communications is from my perspective, the most important aspect of any kind of crisis management and incident response is well inside that domain. Everyone can easily think of a couple of examples where at least the public part of post breach communications went spectacularly wrong. Again keeping it in general terms, there are usually three different areas of communication to cover:
- IT Management
- Operations Teams either inside IT or outside, e.g. ICS Operations Teams
- Stakeholders outside the IT/operations realm. These can be for example departments immediately suffering from stolen or destroyed data or loss of IT services or the PR department that has to manage public communications and handle customers affected by a breach.
From my experience, at least two dedicated incident response team staff are needed to handle communications, so step one in preparing for communications is to assign these tasks to team members regardless of whether you have a permanent incident response team or not. Step two is to make a list of all points of contact for each area and keeping that list up to date. Once all necessary PoC's are identified, they of course also need to know what is expected of them during a breach so some kind of role description is needed. If you want to do this right, I highly recommend running table top exercises so everyone involved can get some practice in their assigned roles. These don't have to be fancy organization spanning crisis management exercises. Playing small "what if" games for a few different scenarios is a really good way to get started.
Want to do some more preparation? Think of alternative lines of communication. It might very well happen that in a data breach your mail server gets compromised, so communicating your countermeasures via internal email might not be a viable option.
Environment
You can only defend what you know about. So after you have determined who to talk to inside and outside your IT organization, it's now time to move to figuring out what is inside your IT environment. Collect all necessary documentation (e.g. network maps, asset lists, firewall rulesets, operating systems and applications used, etc.) and - this is the important part - verify the documentation with what is actually deployed in your environment. If your organization is anything like the ones I've worked in, documentation will be incomplete, outdated and might sometimes only reflect what the IT architects intention of an application was, not the actual implementation. So consider documentation to be a starting point to get your own picture of your IT environment. Want to know more on how to do that? Come to my SANS class, ICS515: ICS Active Defense and Incident Response, where we spend a whole day on asset identification and network security monitoring.
Toolkit
Trying to figure out how your tools work while an incident is ongoing is not going to work. Once the stress level is high you will neither have the time nor mental capacity to learn new things. So get familiar with the tools you'll depend on during an incident in advance. Just upgraded to a new version of some piece of forensics or monitoring software? Have you made sure it's working correctly and not breaking your workflow? And those automation scripts you wished you had during your last incident and never gotten around to finishing? No better time to do that than downtime. And while you're at it, don't stop at tools. Procedures and playbooks are part of your toolkit as well, so make sure these are up to date and refine them with all the experience you identified in your last after-action/lessons learned report. Oh, you didn't get around to finishing that either? I will leave that for another article ;-)
I hope this helps some of you to get started on preparing to deal with cyber security incidents. If you'd like to dive deeper into this, think about taking ICS515 where we delve deeper into preparing for and executing incident response in ICS environments..
Kai has been working in various IT Security roles for more than 15 years. Currently he is the DFIR lead at the premium automaker AUDI AG. Kai also designs and runs Red Team exercises at Audi that integrate IT, business, and physical aspects.
Before Audi he worked for more than 12 years at the engineering company SMS Group where he designed and implemented defensible LANs as well as running DFIR in traditional IT and ICS environments.
Kai holds an MA in Computer Science and English and American Literature.
Resources
- See Upcoming SANS ICS515 Training Opportunities
- Earn the GRID certification to actively defend and respond to ICS cyber attacks
- Join the SANS Community ICS Forum and be part of the conversation where 3800+ ICS professionals share, ask, and discuss ICS-related issue
Free Stuff Reminder
- Download the "What Will Your Attack Look Like" poster here
- Get the latest ICS resources here
- Join the conversation in the ICS Community Forum where ICS professionals share lessons learned, ask questions and connect with others passionate about securing our critical infrastructure.
