Four Keys to Effective ICS Incident Response

While incident response in Information Technology (IT) and Operational Technology (OT) or Industrial Control Systems (ICS) may appear to be very similar, incident response in an ICS environment has different considerations and priorities. Many organizations leverage their existing IT incident response capabilities in an OT environment which may not be ideal for successful incident response and safe, reliable operations. Understanding these gaps and closing them ahead of the incident is key to a successful ICS incident response.

The skills needed in an OT/ICS environment are different too but can be obtained either through training like SANS ICS515: ICS Active Defense and Incident Response or by leveraging experts in the field.

There are many excellent references available to assist in planning an incident response that are focused on responding to IT incidents. One such reference is the Incident Handler's Handbook by Craig Wright available in the SANS Reading Room. While this information should be leveraged to the greatest extent possible, there are several key differences that need to be considered for an ICS incident response scenario.

Preparation and Planning

Roles, responsibilities and decision making processes need to be incredibly clear when responding to OT cybersecurity incidents. Unlike IT incidents, ICS incidents need to take into consideration potential life, safety and property consequences that can result from both the incident and the actions taken by the response team. Successful ICS incident response plans define a clear decision making process that include leadership from the business, safety, operations, IT and the incident response team (IRT). This will avoid confusion when key decisions are needed, especially ones with potential safety consequences or significant financial impact. To the greatest extend possible develop pre-determined actions that have been planned and tested to equip operations and engineering to deal with responses.

When planning for an incident response, it's also important to make sure your IR plan has been reviewed by your compliance and safety officers. Integration with safety plans are key as safety actions such as spill response or evacuation plans may conflict with incident response objectives and it's important to map out a path forward prior to an actual incident. Regulatory reliability standards, notification requirements and other impositions must be carefully evaluated for your particular industry to ensure you remain compliant from the initial response through recovery.

Before an incident occurs, it is critical that organizations undertake an effort to identify all of the assets that can interact with the ICS environment. This is one of the key gaps when conducting incident response activities that I have observed and without this fundamental understanding of what assets are in place organizations will struggle to respond quickly and restore operations. Asset identification need not be active scanning on a network but could be as simple as a physical site survey, reconciliation of as-designed and as-built drawings with current state, or looking at network traffic flow information that traverses both internal to the control network and crosses control-corporate firewall.

Visibility and Detection

ICS environments are typically highly static in nature, very prescriptive in their communications and do not have a great number of human users with unpredictable behaviors. These environments are ripe for behavioral analysis and anomaly detection making detection of anomalies much easier for defenders. Contrast that with IT networks where it is often near impossible to develop a well-established network baseline.

A great example of this in action would be if a new domain is accessed by an ICS network asset. After a baseline has been established, new communications should be investigated by the network monitoring team to evaluate if they are in fact part of an approved changed and a new-normal operations. As ICS systems are unlikely to communicate with new external servers after install, this should happen with a very low frequency but is a great technique for detecting intrusions before significant penetrations are made.

Coordination and Communication

While coordination with other parts of your company is important in IT incident response, it is absolutely essential when conducting incident response in an ICS environment. While excellent planning and preparation may occur, ICS incident response actions that would otherwise be routine in IT environments may have impacts to the safe and reliable operation of the process. For example, bringing down a user's laptop to create a drive image may be an inconvenience for the user but taking down an HMI to create a drive image may remove key process visibility or have an impact to emergency response procedures.

This is why it is critical for information to flow between the leadership, incident response, operations and safety teams to ensure everyone has situational awareness on the IRT's actions and can provide input into the decision making process. Incident responders need to ensure they have a full understand of the consequences of their actions, with buy-in from the operation team, before taking those actions.

Incidents in ICS environments also tend to get significantly more attention from external stakeholders than do typical IT incidents. Even a malware infection as common as Conficker can turn into a headline if it's at a critical facility. Therefore, it's also important to involve the public relations team early in the process to develop a communications plan for both internal and external stakeholders.

Sharing

The ICS community is a small and tight knit group made up of operators, engineers and technicians that enable our modern way of life. Despite this familiar and critical role there is still a mentality of shame and concealment when it comes to reports of incidents in our ICS environments. Culturally this is a challenge we must overcome as intrusions into one facility could hold key information about what tactics, techniques and procedures will be leveraged by adversaries in another facility. Therefore, it is critical that organizations have a plan to share information with peer organizations both inside and outside of the sector during and after incident response activities. Only when working together can we minimize the risk to these critical systems.

While methods for IT incident response are well known, when applying them to ICS incident response a level of tailoring is needed to ensure success. Following these key points will help position your incident response program up for success. We cover this and other key incident response topics in more detail in SANS ICS515: ICS Active Defense and Incident Response.

Free Stuff Reminder!

Download the newest digital ICS poster Here

If you would like printed copies please send a request to: info@sans.org

Join the conversation in the ICS Community Forum where more than 4000 ICS professionals share lessons learned, ask questions and connect with others passionate about securing our critical infrastructure.