Persistence Testing vs Penetration Testing
Hey all! For the last 6 months I've been involved in some really cool and innovative takes on traditional penetration testing, something I call persistence testing. Let me go into the philosophy behind this testing style first, before we dive into the details.
My recent experiences have really taught me one thing, which is that anyone can get owned. This is the simple nature of computer security, exploits are always ahead of the curve, an 0-day can surface at any time, and you can quickly become compromised where you once had a strong defense. And even penetration testers know that the weakest element of security is often the human operating the computer, making spear-phishing a massively successful vector, something advanced attackers regularly take advantage of in the real world. Mandiant's M-Trends report from 2015 shows that most organizations don't even know they are compromised for 200 days, and on top of that only 30% of companies detect the breach on their own! As a penetration tester, these statistics and the reality of seeing so many companies routinely getting owned tells me we are doing something wrong, a change of strategy is clearly needed. Penetration tests find holes, but rarely are unsuccessful because it's so easy to find holes in the current information technology landscape. Therefore, I've started developing new pro-active strategies, based around Mandiant's general philosophy of needing to respond to the breach over preventing the breach, in today's current landscape at least. I call these new strategies "Persistence Testing", and the real goal here is to emulate APT campaign tactics or common host and network persistence techniques, in a scalable way such that the Red Team can audit and ensure an organization is able to detect these malicious actions. To emphasize the importance of detecting and containing a breach I've included a small interview w/ Kevin Mandia below.
There are a lot of differences between traditional penetration testing and persistence testing. Where as with traditional penetration testing the goal is to find a weakness in the organizations IT infrastructure, with persistence testing the goal is to find shortcomings in an organizations ability to detect and respond to security events. How is this different from traditional red teaming? With persistence testing we often assume the initial vector of compromise, such as a spear phish or an workstation gets infected while at an employee's home, and focus on emulating malicious post-exploitation techniques. To properly pull this off, the attacker will likely need an in-depth knowledge of the organization they are working with, including network topology, various security controls, key targets in the network, which detection capabilities currently exist, and where the client organization is looking to increase their response capabilities. Further, one of the core tenants of persistence testing is taking detailed metrics on which techniques the attacker is executing, which security controls the attacker is bypassing, and what roadblocks the attacker has encountered. When they compare these metrics with the incident response teams incident notes they can see which techniques went undetected and which techniques were alerted on. Then the next exercise can avoid or repeat the past actions, with an emphasis on increasing response time or detecting actions that were previously missed. Further, a skilled tester with in-depth knowledge of the client organization can create specialized malware that will go generally undetected, but then trigger specifically targeted detection mechanisms, such as setting off host based alerts for a specific malware family or setting off network based alerts for a typical insider threat (such as accessing high security assets they typically don't access, or exfiltrating large, encrypted files). In this manner, firms can take a pro-active and scientific methodology based approach to increasing security incident detection capabilities. This is something I've done a lot latly, both in the real world and with my CCDC experiences, so I think I will be talking on these specific techniques more in the future.
This isn't a totally unique idea either, some industry thought leaders have thrown around similar ideas, such as the following videos I've included from Dave Kennedy and Raphael Mudge. That said, I think it's really important to differentiate this type of security testing from traditional penetration testing, and thus execute these methodologies in a more advanced and specialized way.
My recent experiences have really taught me one thing, which is that anyone can get owned. This is the simple nature of computer security, exploits are always ahead of the curve, an 0-day can surface at any time, and you can quickly become compromised where you once had a strong defense. And even penetration testers know that the weakest element of security is often the human operating the computer, making spear-phishing a massively successful vector, something advanced attackers regularly take advantage of in the real world. Mandiant's M-Trends report from 2015 shows that most organizations don't even know they are compromised for 200 days, and on top of that only 30% of companies detect the breach on their own! As a penetration tester, these statistics and the reality of seeing so many companies routinely getting owned tells me we are doing something wrong, a change of strategy is clearly needed. Penetration tests find holes, but rarely are unsuccessful because it's so easy to find holes in the current information technology landscape. Therefore, I've started developing new pro-active strategies, based around Mandiant's general philosophy of needing to respond to the breach over preventing the breach, in today's current landscape at least. I call these new strategies "Persistence Testing", and the real goal here is to emulate APT campaign tactics or common host and network persistence techniques, in a scalable way such that the Red Team can audit and ensure an organization is able to detect these malicious actions. To emphasize the importance of detecting and containing a breach I've included a small interview w/ Kevin Mandia below.
There are a lot of differences between traditional penetration testing and persistence testing. Where as with traditional penetration testing the goal is to find a weakness in the organizations IT infrastructure, with persistence testing the goal is to find shortcomings in an organizations ability to detect and respond to security events. How is this different from traditional red teaming? With persistence testing we often assume the initial vector of compromise, such as a spear phish or an workstation gets infected while at an employee's home, and focus on emulating malicious post-exploitation techniques. To properly pull this off, the attacker will likely need an in-depth knowledge of the organization they are working with, including network topology, various security controls, key targets in the network, which detection capabilities currently exist, and where the client organization is looking to increase their response capabilities. Further, one of the core tenants of persistence testing is taking detailed metrics on which techniques the attacker is executing, which security controls the attacker is bypassing, and what roadblocks the attacker has encountered. When they compare these metrics with the incident response teams incident notes they can see which techniques went undetected and which techniques were alerted on. Then the next exercise can avoid or repeat the past actions, with an emphasis on increasing response time or detecting actions that were previously missed. Further, a skilled tester with in-depth knowledge of the client organization can create specialized malware that will go generally undetected, but then trigger specifically targeted detection mechanisms, such as setting off host based alerts for a specific malware family or setting off network based alerts for a typical insider threat (such as accessing high security assets they typically don't access, or exfiltrating large, encrypted files). In this manner, firms can take a pro-active and scientific methodology based approach to increasing security incident detection capabilities. This is something I've done a lot latly, both in the real world and with my CCDC experiences, so I think I will be talking on these specific techniques more in the future.
This isn't a totally unique idea either, some industry thought leaders have thrown around similar ideas, such as the following videos I've included from Dave Kennedy and Raphael Mudge. That said, I think it's really important to differentiate this type of security testing from traditional penetration testing, and thus execute these methodologies in a more advanced and specialized way.
