SEC568: Product Security Penetration Testing - Safeguarding Supply Chains and Managing Third-Party Risk

Overview

The first section of this course describes the principles associated with both supply chain attacks and product security testing. We start to navigate the "why" and the "what" to product security testing, followed by which skills are important for success. We introduce our main methodology, our toolbox included in the supplied virtual machines, and the name of our fictitious company you will be working for during the class, "Think Red, Act Blue". The main technical emphasis for section one will be to explore the basic, yet critical concept, of online product research followed by our first sections of basic enumeration and threat modeling. Through lectures and 3 hands-on exercises we will explore how to use simple tools such as binwalk, Corellium, APKLab, APKLeaks and associated product security methodologies to begin to understand how your target accomplishes its main goals. Students will be provided with access to a cloud-based mobile emulation platform (Corellium) to complete the last lab of this section.

Exercises

• Initial recon of product, existing CVEs, and vendor website
• Unpacking firmware and examining components
• Basic enumeration of an Android Package Kit (APK) with Corellium

Topics

Course overview and methodology

• This module is focused on an introduction to product security assessments and its application to combating supply chain attacks. We will start by covering the who, what, when, where, and how methodologies for product security assessments, with a particular emphasis on online product research and basic enumeration.

General and software supply chain application

• In this module we will explore several practical examples of when you want to deploy the concepts and processes of this course in real-world scenarios not related to supply chain attacks, followed by diving into supply chain attacks and ultimately demonstrating how product security testing is a critical component of your defenses.

Online product research

• With an overview of the process under our belt, in this module we will start to follow the Product Security Testing flow chart. We will look at how to effectively acquire system requirements and efficiently perform online product research, a concept that is recurring and critical to product security testing.

Basic enumeration on hardware devices, Linux & Android

• In this module we will embark on understanding the core technical concepts needed to start evaluating a target using basic enumeration. We will start with the overall methodology of basic enumeration and then dive into the specific core concepts and tools needed for hardware devices, Linux, and Android.

Overview

The second section of the course will close out our study of basic enumeration by looking at the Windows platform and networking concepts that fit within the basic enumeration methodology. This will include using tools such as Microsoft Attack Surface Analyzer (ASA), Microsoft Sysinternal Suite, ProcDOT and more. This will allow for an in-depth conversation around the important role threat modeling plays in product security assessments and is a crucial first step to reducing the impact of supply chain attacks. Doing threat modeling and answering questions related to product security testing requires sifting through a large amount of data. In this section, we will introduce Exploratory Data Analysis (EDA), a common workflow to conduct analysis used to try to make sense of the data and present the results. Through lectures, demos, and hands-on exercises we will learn how to explore this data using data science tools like Python, Jupyter Notebooks, Pandas DataFrame, and graphical libraries like Matplotlib, among others. Lastly, this section will start to segue into the course's deeper technical concepts by introducing deep enumeration.

Exercises

• Basic enumeration on Windows
• Initial network traffic profiling with Pandas DataFrame & Jupyter Notebooks
• Investigating network traffic on Android

Topics

Basic enumeration on Windows

• Continuing to build on concepts from the previous section on basic enumeration, this section will look at how to apply basic enumeration to the Windows platform, arguable the most used platform in enterprise make it a critical component to understand. We will cover important concepts related to product security testing, walk through relevant Windows fundamentals, and provide useful tools for performing basic enumeration on Windows.

Basic networking analysis

• This module will introduce network analysis at a basic level as it pertains to basic enumeration. Besides exploring important fundamental networking concepts, this module will leverage some basic data analysis and traffic profiling with Jupyter Notebooks and Pandas DataFrame to help demonstrate how to turn complex analysis into a more simplistic and visual task.

Threat modeling

• Threat modeling is a crucial concept in successful product security testing but even more important is when to perform threat modeling. This module will discuss different types of threat modeling, demonstrate one suggested approach within product security testing, and cover the information required to build a useful model. This module will conclude with a review of how our process up to this point has impacted the way we defend against supply chain attacks.

Deep enumeration

• Section 2 will just begin to explore the concept of deep enumeration. We will introduce deep enumeration, explore how a threat model influences deep enumeration, and discuss when to deploy these techniques. This module will begin increasing the technical depth of the course and push students' knowledge of finer topics and conclude with a section on networking concepts related to deep enumeration.

Overview

This section is all about deep enumeration, the process of deeper technical analysis to answer critical questions from a threat model. We will focus on two deep enumeration skills: binary code analysis and network analysis of unknown protocols. In the first half of section 3, we will learn how to use decompiler tools to access the underlying code of an application and unpack archives when needed. This will culminate in a hands-on lab using JetBrain's dotPeek to answer critical questions related to supply chain attacks regarding how an application is being updated. Network traffic is often a key input to a system and a common entry point for attackers. To dive deeper into network analysis, this section will also take a pause to provide a primer into Scapy, a Python framework designed to manipulate networking packets. Armed with the knowledge and skills covered in this section, you will be better prepared to dissect proprietary or unknown protocols later in the course.

Exercises

• Using APKLab and HTTP Toolkit to defeat cert pinning
• Learning from decompiled source code
• Using Scapy to create a 3-way handshake
• Searching for networking artifacts within payloads
• Bonus: Learning from decompiled source code on Android

Topics

Intro to Deep Network Analysis

• This module will build on the principles discussed during basic network analysis but dive deeper starting with encryption. While encryption is a great tool for security, we will discuss how to verify if it's implemented in a way that makes it more difficult for an attacker to access your information. This module will include a lab providing the student with hands-on experience bypass encryption used on a popular Android application.

Binary code analysis

• The ground truth to how a target or system works is its source code. Unless the application or device you are auditing is open source, you will not have access to this code. Binary code analysis is the concept of performing a code review without having access to the source code. This module walks through some basic skills to apply binary code analysis during product security testing as one of the two core concepts covered in this course within deep enumeration.

Scapy primer

• In order to dive deeper into network analysis, it is beneficial to be able to programmatically analysis and create networking packets. Scapy is a python module that helps take our analysis and testing to the next level. This module of section 3 will teach the basics of Scapy needed for product security testing.

Understanding proprietary protocols

• We will leverage the knowledge of Scapy acquired in the previous module and apply it to the deep enumeration concept of understanding unknown or proprietary protocols. This last module of section 3 will be an introduction to proprietary protocols that will be continued in section 4.

Overview

This section will conclude our exploration of deep enumeration and work toward finalizing a product security test project. We begin with a continuation and finalization of dissecting proprietary protocols using Scapy to create custom layers and explain the basics of networking fuzzing. This will bring us to our final risk analysis section where we will focus on creating attack trees and applying risk scoring methods to assess the risk of supply chain attacks, among other risks to the Think Red, Act Blue organization. The section will conclude with important final topics that will allow us to wrap up our product security assessment, such as reporting and vulnerability disclosure.

Exercises

• Learning through enumerating patterns
• Finding your network traffic's heartbeat
• Using basic Linux tools to analyze unknown payloads
• Basic fuzzing with Scapy
• Results analysis

Topics

Dissecting proprietary protocols

• This module is one of the most technically challenging sections of the course, as we discuss how to use techniques such as enumerating patterns and finding heartbeat packets to help gain a glimpse of the purpose of every byte in a packet payload. We will examine several real protocols and lean on previous modules such as data analysis with Pandas DataFrame and Scapy to gain as much insight as possible.

Fuzzing

• Every threat model should contain a risk of unknown vulnerabilities. While fuzzing is often considered a vast and sometimes daunting topic in security, it can help reduce the risk of unknown vulnerabilities plaguing your network. This module will explore the basic concepts of networking fuzzing as part of deep enumeration and the value it can bring to a product security test.

Risk analysis

• Determining the risk associated with our technical findings is the most important step of a product security test event. This module will focus on questions like: What does this mean for my organization? What steps can I take to mitigate my findings? Here we will study how to turn your previous threat model into realistic attack trees and quantify the risk for each scenario using a modified and improved version of the DREAD risk scoring method.

Reporting

• In many cases it will be required to report your findings and processes throughout your organization. This module will talk about industry best practices in reporting your findings from product security testing and highlight important items directly related to supply chain attacks.

Vulnerability disclosure

• In some cases, your analysis will result in the discovery of new vulnerabilities. This module will highlight best practices in reporting your team's finding to the affected vendors.

Overview

The course culminates in an all-day hands-on lab designed to give each student the experience of completing a product security test from start to finish. Students will be given a real application to test during this course section, which will apply the most crucial concepts learned throughout each previous section using the toolbox included in the supplied virtual machines and cloud-based labs.

Exercises

• Basic enumeration: Installation analysis
• Basic enumeration: Execution analysis
• Threat modeling
• Deep enumeration: Network analysis
• Results analysis

Topics

Introduction to target

• The section will start with an introduction to the target application and the basics of how it functions. This will help frame the following exercises and reduce the time needed to learn the basics of the program.

Hands-on product security testing event

• The product security test for this section has been broken down into 5 hands-on exercises. Each exercise will guide the student through one of the core concepts of the product security testing process, including basic enumeration, threat modeling, deep enumeration, and results analysis.

Instructor-led discussion

• Each lab exercise will be followed by an instructor-led discussion on what was uncovered during the lab. This will help ensure all students stay on the best path to provide the highest amount of value from the event.