Thoughts on APT
There's been a great deal of discussion lately about the advanced persistent threat, or APT, and I've seen list posts from folks adding their thoughts, or asking others to weigh in and provide any insight they may have. I see this as healthy, not only for customers, but also for the forensics community as a whole.
There are some things that are being said, quite clearly and repeatedly about this threat. For example, take a look at Wendi's post on the Mandiant blog; she presents some statistics from the M-Trends report that can give you an idea of what to look for if you suspect you've been compromised. I also think that if you view it the right way, and perhaps have a bit of context from other sources, you'll see that this upholds the Least Frequency of Occurrence (LFO) principle that Pete Silberman has described. So what this means is that responders and analysts need to look for the anomalies; not the massive spikes in activity, but the small, infrequent things that we may not notice in all the noise on a system, or infrastructure. The Mandiant folks mention this, and so do the HBGary folks...so, whether you're using LFO or MRI (thanks again, Pete!), or you're looking at digital DNA, you're looking for what is or should be standing out as anomalous and infrequent.
Okay, so...what about APT?
As I see it, there are three major groups of actors here...the good, the bad, and theugly victim. The victims are pretty clear. The bad guys are the developers, purveyors and operators of exploits and other mechanisms (i.e., code, malware, etc...call it what you will) for malicious purposes. The good guys are LE, responders, corporate consultants, etc...those folks trying to assist the victims, most often after a data breach.
Now, a number of the good guys have been (or started) posting reports (see the Reports section at the end of this post) illustrating statistics based on the incidents they've responded to and the work they've done. Reading through these, we see a lot of information much like what Wendi included in her post. Perhaps the most important thing, in my mind, is that the numbers and information from these reports indicate that there was a cultural shift in the bad guy's realm. What I mean by that is that back "in the day", most of what we saw was malware that ran amok on networks, and folks blowing out SubSeven or NetBus to systems so that they could open and close the CD-Rom tray. No more. Systems are being targeted for either the access they provide or the data that they store and process. Malware is being modified enough so that current AV products don't detect new variants, and footprints of that malware are minimized, using mutexes so that the system is only infected once. I attended a conference in Redmond back in November, 2009, and in several of the presentations, LE stated that the bad guys are dedicated, patient, smart, well-funded, and they have an economic goal behind what they're doing.
From my perspective as a responder and analyst, as well as from reading the reports and compiled statistics, what I'm not seeing is a corresponding paradigm shift on the part of the organizations that fall victim to these intrusions and compromises. Intrusions are still going undetected; victims are being notified by external third parties weeks or months after the fact. Systems are still being compromised via SQL injection and the use of poor passwords by administrators.
One thing that really stands out in my mind is that looking at my own experience, as well as the experience of others (via reports and postings on the web), the victims are not experiencing a cultural shift that corresponds to what the bad guys have gone through. Even in the face of information that indicates that the cost of data breaches has increased, organizations continue to be breached. In all fairness, breach attempts are going to happen; however, at least one report indicates that as many as 70% of data breach victims responded to find out well after the breach from an external third party.
The point is that the bad guys have identified targets and have an economic stimulus of some kind for attaining their goals. They're dedicated and compartmentalized...someone is dedicated to discovering vulnerabilities, and often it appears to be a different party all together that employs the exploit and some new piece of malware. For the victims, we're still seeing incident prevention, detection, and response all being secondary or tertiary duties for overworked IT staff...so while the bad guys can dedicate time and resources toward getting into an organization, IF there are dedicated responders within the organization, and IF they have any recent training or experience, and IF anyone actually knows where the data resides...well, you can see my point. From the perspective of a historical military analogy, this appears to be akin to special operations forces attacking villages defended by farmers and shopkeepers.
Maybe I'm way off base here, but this whole discussion of APT seems to be showing us something that's a bit more of an expansive issue. My thinking here is that if those organizations that are storing and processing "sensitive data" (choose your definition du jour for "sensitive data") were to have a corresponding cultural paradigm shift, we might begin to see intrusions detected and responded to in a manner that would provide data and intel to law enforcement, such that there could ultimately be arrests. I know, this is easier said than done...look at the issues that have sprung up around compliance; all compliance is...really...is an attempt to mandate or legislate minimum levels of security that organizations should have already had in place. I don't want to cloud the issue (no pun intended), but my overall point here is that maybe law enforcement would be able to make arrests if they had data and intel. As a responder, too often have I arrived on-site for an incident where the customer was informed of an issue by an outside third party; no one knows definitively where critical data resides, there are no logs available, and administrators have already done "nothing", which in reality amounts to an extensive list of removing systems from the network, scanning them with AV, deleting files, and even wiping entire systems.
So we know that the bad guys are having fairly high rates of success compromising systems and infrastructures using, in some cases, well-known vulnerabilities that simply hadn't been patched. We know that in many cases, they don't need to use special privilege escalation exploits, because they get in with Administrator/root/superuser privileges. We know that in most cases, they don't upload massive sets of tools, but instead use native utilities or only one or two malware files. We know that rootkits simply don't have to be used to hide the bad guy's presence...why hide from someone who's not looking for you?
So the take away, for me, from these reports is simply that there needs to be a cultural shift on the part of those who store and process sensitive data, and it has to come from the top down. It's 2010, folks...do we still need to sell infosec to senior management? What should be the CEO's concern...that his email and IM are up and running, or that the sensitive data that his company stores and processes is secure, and his infrastructure monitored?
Reports
7Safe (UK)
Verizon
Mandiant
Addendum: There's a bit of a different perspective on APT and what it really means over at TaoSecurity (here, and commentary on the M-Trends report here). For another view or perspective on the M-Trends report, see what IntelFusion says.
One thing to keep in mind about the reports...remember that they're based on numbers compiled by the perspective groups. Each group may have a different customer base and primary line of business when it comes to what they do. What this means is that each report is going to represent a slightly different culture when it comes not only to the numbers but also what they represent.
There are some things that are being said, quite clearly and repeatedly about this threat. For example, take a look at Wendi's post on the Mandiant blog; she presents some statistics from the M-Trends report that can give you an idea of what to look for if you suspect you've been compromised. I also think that if you view it the right way, and perhaps have a bit of context from other sources, you'll see that this upholds the Least Frequency of Occurrence (LFO) principle that Pete Silberman has described. So what this means is that responders and analysts need to look for the anomalies; not the massive spikes in activity, but the small, infrequent things that we may not notice in all the noise on a system, or infrastructure. The Mandiant folks mention this, and so do the HBGary folks...so, whether you're using LFO or MRI (thanks again, Pete!), or you're looking at digital DNA, you're looking for what is or should be standing out as anomalous and infrequent.
Okay, so...what about APT?
As I see it, there are three major groups of actors here...the good, the bad, and the
Now, a number of the good guys have been (or started) posting reports (see the Reports section at the end of this post) illustrating statistics based on the incidents they've responded to and the work they've done. Reading through these, we see a lot of information much like what Wendi included in her post. Perhaps the most important thing, in my mind, is that the numbers and information from these reports indicate that there was a cultural shift in the bad guy's realm. What I mean by that is that back "in the day", most of what we saw was malware that ran amok on networks, and folks blowing out SubSeven or NetBus to systems so that they could open and close the CD-Rom tray. No more. Systems are being targeted for either the access they provide or the data that they store and process. Malware is being modified enough so that current AV products don't detect new variants, and footprints of that malware are minimized, using mutexes so that the system is only infected once. I attended a conference in Redmond back in November, 2009, and in several of the presentations, LE stated that the bad guys are dedicated, patient, smart, well-funded, and they have an economic goal behind what they're doing.
From my perspective as a responder and analyst, as well as from reading the reports and compiled statistics, what I'm not seeing is a corresponding paradigm shift on the part of the organizations that fall victim to these intrusions and compromises. Intrusions are still going undetected; victims are being notified by external third parties weeks or months after the fact. Systems are still being compromised via SQL injection and the use of poor passwords by administrators.
One thing that really stands out in my mind is that looking at my own experience, as well as the experience of others (via reports and postings on the web), the victims are not experiencing a cultural shift that corresponds to what the bad guys have gone through. Even in the face of information that indicates that the cost of data breaches has increased, organizations continue to be breached. In all fairness, breach attempts are going to happen; however, at least one report indicates that as many as 70% of data breach victims responded to find out well after the breach from an external third party.
The point is that the bad guys have identified targets and have an economic stimulus of some kind for attaining their goals. They're dedicated and compartmentalized...someone is dedicated to discovering vulnerabilities, and often it appears to be a different party all together that employs the exploit and some new piece of malware. For the victims, we're still seeing incident prevention, detection, and response all being secondary or tertiary duties for overworked IT staff...so while the bad guys can dedicate time and resources toward getting into an organization, IF there are dedicated responders within the organization, and IF they have any recent training or experience, and IF anyone actually knows where the data resides...well, you can see my point. From the perspective of a historical military analogy, this appears to be akin to special operations forces attacking villages defended by farmers and shopkeepers.
Maybe I'm way off base here, but this whole discussion of APT seems to be showing us something that's a bit more of an expansive issue. My thinking here is that if those organizations that are storing and processing "sensitive data" (choose your definition du jour for "sensitive data") were to have a corresponding cultural paradigm shift, we might begin to see intrusions detected and responded to in a manner that would provide data and intel to law enforcement, such that there could ultimately be arrests. I know, this is easier said than done...look at the issues that have sprung up around compliance; all compliance is...really...is an attempt to mandate or legislate minimum levels of security that organizations should have already had in place. I don't want to cloud the issue (no pun intended), but my overall point here is that maybe law enforcement would be able to make arrests if they had data and intel. As a responder, too often have I arrived on-site for an incident where the customer was informed of an issue by an outside third party; no one knows definitively where critical data resides, there are no logs available, and administrators have already done "nothing", which in reality amounts to an extensive list of removing systems from the network, scanning them with AV, deleting files, and even wiping entire systems.
So we know that the bad guys are having fairly high rates of success compromising systems and infrastructures using, in some cases, well-known vulnerabilities that simply hadn't been patched. We know that in many cases, they don't need to use special privilege escalation exploits, because they get in with Administrator/root/superuser privileges. We know that in most cases, they don't upload massive sets of tools, but instead use native utilities or only one or two malware files. We know that rootkits simply don't have to be used to hide the bad guy's presence...why hide from someone who's not looking for you?
So the take away, for me, from these reports is simply that there needs to be a cultural shift on the part of those who store and process sensitive data, and it has to come from the top down. It's 2010, folks...do we still need to sell infosec to senior management? What should be the CEO's concern...that his email and IM are up and running, or that the sensitive data that his company stores and processes is secure, and his infrastructure monitored?
Reports
7Safe (UK)
Verizon
Mandiant
Addendum: There's a bit of a different perspective on APT and what it really means over at TaoSecurity (here, and commentary on the M-Trends report here). For another view or perspective on the M-Trends report, see what IntelFusion says.
One thing to keep in mind about the reports...remember that they're based on numbers compiled by the perspective groups. Each group may have a different customer base and primary line of business when it comes to what they do. What this means is that each report is going to represent a slightly different culture when it comes not only to the numbers but also what they represent.