At Symantec, I worked on implementing machine learning techniques to strengthen defenses capabilities and enhance the ATP product with advanced detections and protections. It certainly has increased the cost and effort of being an attacker, in addition to allowing us to play around with the telemetry and uncover attacks that went undetected for quite a while. That was a few years back. Although it is still essentially applied to defensive security, machine learning and deep learning aren’t only for blue teams anymore, and I wanted to take a look at how it is being used by pen-testers and red-teamers, and especially on what specific offensive tasks are aided by ML.
MITRE tactics describe the process that an offensive team would follow to either disrupt or breach a target system. At a high level, it basically represents the stages of the kill chain. Each stage would include techniques and sub-techniques that are known as being used by attackers. A high-level flow of an offensive process goes like this: reconnaissance, get a foothold, persist, elevate privileges, pivot, and take action; e.g., persist, exfiltrate data, or do whatever else you came for. In each of these steps, an important aspect for the offensive team is to protect its IP, i.e., whatever TTPs (Tactics, Techniques, and Procedures) they are using (to still be able to use them the next day!) So the aim here is to go through these steps while avoiding detection, especially in the active phases, and the role of ML would be to help with that in any way possible. Now, a cyber attack can also be only one of those techniques or sub-techniques, e.g., a DDoS or a brute force attempt, and any of these attacks and techniques might involve the use of ML.
In data analysis, you need data to work with! That’s a basic fact. The more you know about target people and environments, defensive technologies, how reversing is done, how incident response and DFIR is done, etc. the more you are able to take advantage of machine learning automation to carry out offensive operations and throw off defenses and defenders.
Note that there are two other aspects to the use of ML in offensive operations. The first one is deception; e.g., creating fake audios or videos (also known as deep fakes), which has already been used to scam businesses and could be used to spread disinformation or disrupt a target as part of an offensive campaign. There are a few surveys for deep fakes generation and detection out there. The second is adversarial attacks on ML and DL; which show that models are vulnerable to perturbations in their inputs. A basic taxonomy would include two main aspects; security, e.g., the change of output, and privacy, e.g., data leakage. Here is an example library used to test the adversarial robustness of ML-based Windows detectors.
Now back to the use of ML in cyber attacks. A quick note here on the use case of vulnerability detection, although not necessarily an offensive security use case, as it is also largely relevant to defense. This is one of the areas that has received the most attention. ML/DL models have been applied to fuzzing, static code analysis, as well as dynamic analysis of executables. The most interesting use case for me though is the automation of the process of exploitation, as described earlier.
Looking through arXiv, I found two interesting papers that really show how relevant this could be for offensive teams; the first one is about social engineering; a model that learns how to socially engineer specific users into clicking on deceptive URLs based on their social media posts. The second is about sandbox detection, using process list data. Voila, you’re more likely to be successful in your initial access attempt if your spearphishing campaign is highly targeted and your payload is likely to evade analysis.
Nevertheless, because the data within execution environments is quite hard to come by, there are practically no similar studies that focus on specific offensive tasks and strategies, such as identifying the target’s defenses for instance, or generating custom-made exploits that are specifically tailored to the target’s environment. There are, however, attempts in that direction that will certainly mature with time. DeepExploit was one of those attempts to automate pen-testing by examining target system configurations and selects the optimal payload (it works with Metasploit) according to the input information, e.g., OS type, or product name and version, etc. Another similar project specifically targeting Web servers is called GyoiThon. With the availability of data from execution environments, these systems will certainly become more relevant, effective, and less noisy.
The use of ML in offensive security is still in its infancy. It will not replace pentest teams anytime soon. The fact that ML models are only good at very specific tasks and offensive security (and cyber security in general) itself is a rapidly evolving field, will make progress even slower. Machines will still have problems for years to come to autonomously explore all possible solutions and build chains of exploitation in a way that doesn’t defeat the purpose of offensive security itself. Having said that, I’m sure it’ll become a great hacking partner to offensive security professionals.
