Sorry...no pun intended.  No...wait.  I totally meant it.

I was reading through some news items recently, and came across an article on Yahoo! news that refers to the recent Epsilon breach, (also read about it on KrebsonSecurity.com) in which, it appears, a number of email addresses were exposed.  However, some things about the article I read caught my attention...

From the article:
Epsilon said that while hackers had stolen customer email addresses, a rigorous assessment determined that no other personal information was compromised. By itself, without passwords and other sensitive data, email addresses are of little use to criminals. But they can be used to craft dangerous online attacks.

Okay, is it just me, or are the last two sentences contradictory?  I mean, without other information, email addresses are useless to criminals...except when they are used to "craft dangerous online attacks."  What are "dangerous online attacks"?  Well, Uri Rivner, the Head of the Security Division at EMC, recently posted "Anatomy of an Attack" on the RSA blog, and states in that post that the breach to their systems started with a phishing attack...specifically crafted emails were sent to specific individuals, knowing that at least some of them would be likely to click on the attachment.

Now, dear reader, please do not assume that these two incidents are tied together in any way, as that's not what I'm saying, nor am I suggesting it.  But what I am saying is that there is a considerable disconnect between what some think online criminals want or need, and what they actually end up going after.  For example, what does the second sentence in the above quoted paragraph sound like to you?  To me, it sounds like a justification to NOT have to notify, based on state notification laws (the first of which was California's SB 1386).  Think about it.  By specifically stating that all of this other personally identifiable information (PII) was not exposed along with the email addresses, there's now justification that the breach laws don't come into play, and by extension/implication, neither do any compliance regulations.

Okay, but someone still accessed your systems and took this information.  And by "took", I'm referring to the fact that while you still "have" the information, so do they.  So this isn't like real-world theft where someone steals your car and you no longer have possession...this is an instance where the confidentiality of the information on which your business runs has been compromised.

In addition, the article goes on to indicate that more than just the email addresses themselves were exposed...the businesses (banks, hotels, etc.) that the owners of those email addresses frequent were also exposed.

Do you know what this reminds me of?  The designer drug trade.  Apparently, designer drugs are outlawed based on chemical structure, so once one drug is outlawed, a chemist comes up with a new, potentially more powerful drug, with a different chemical structure, which is therefore legal until it's discovered by law enforcement and broken down enough to be uniquely identified.  The reason this breach reminds me of this sort of thing is that an organization was breached, and the critical information on which that organization's business relies was compromised...but not enough information to require notification by state breach laws, based on the specific definition of PII.  However, the information that was compromised...this email address is for someone who uses CitiBank, etc...can still be employed to devastating effect (refer back to the RSA breach).

So, Epsilon is notifying its customers...and folks like @briankrebs on Twitter are tracking notifications, and even receiving responses from individuals who are receiving five or more notifications...so that's a good thing.  We see in the media all the time where those who get out in front of an incident and are very open about it fare much better in the long run than those who cover it up in legalese or just flat out deny that it happened.  But I wonder how things would have worked out had the organization taken a proactive approach and done a better job of preparing for an incident.  For example, I wonder what the effect of having Carbon Black installed on systems would have had on the overall incident detection, and ultimately, response.

Folks, the fact is that the instant you think that you don't have anything anyone would want or could use, you've lost..it's a total Sun Tzu thing.  Even if you can not possibly imagine how someone would use or profit from the data that you process or store, someone else likely already has.  At the very minimum, you've likely got CPU cycles, RAM, and storage space that can be used as a staging area or jumping-off point.

So, what do you do about this?  One way to address the situation of the inevitable security incident or breach...after all, the last couple of months should have clearly demonstrated to all that NO ONE is immune...is to be ready for it to happen.  So why not seek out a trusted adviser, someone who has dealt with breaches and incidents across a wide range of clients, and cultivate a relationship?  Incident preparation is more about a change to your corporate culture than it is about purchasing devices and software; a great deal of preparation can be done without purchasing a single device.  However, the lack of visibility that most organizations have will likely be addressed by some sort of purchase, but we're not talking about dropping a truck-load of gear off at your doorstep.  There's a great deal that can be done, and you're going to likely be sold whatever solution is provided by the vendor that you call.

Breaches
As many of you know, I am (quite thankfully) no longer involved in Visa PCI breach investigations.  However, something I saw in the news recently did attract my attention; specifically, Briar Group LLC is the first restaurant chain fined under the Massachusetts data breach law (here).

So why is this interesting?  Well, most of the time I was doing this PCI work, I was rarely aware that any of the organizations I encountered got fined.  Yes, there were one or two that I heard about later, but like I said, it was rare.  Mostly, I'd go on-site, scope the incident, collect data, return to the lab to analyze the data and submit my report...and that was it.  Now, I didn't really need to know what happened, to be honest, but after the first year, organizations that had been breached would ask me, "What happens after you submit your report?", and I really didn't have much of an answer.  As far as I could tell, QIRA firms had only two functions with respect to PCI breaches...pay to keep their analysts certified, and submit reports within the timeframe specified.

This article provides a good deal of information, and for me, a bit of deja vu.  For example, while the actual issue isn't described in any detail, the incident itself is described as having first occurred in April 2009, and went undetected until December 2009.  If you have reviewed any of the cumulative annual reports that have come out over the years, and in particular the Verizon Business Security report, you'll notice that this sort of thing isn't unusual at all; organizations rarely detect an intrusion while it's in progress, but rather get notified by an external third party, well after the incident occurred.  I think that this is in part due to a lack of visibility into their networks, but I also think that there is a lack of intelligence provided by the oversight organization (collects up all reports), and then acted upon by those organizations who fall victim to these breaches.

Restaurants aren't the only organizations affected by these breaches.  Other businesses, such as credit unions, are being breached, as well.  While it may appear that credit unions are subject to a different set of compliance measures, these measures really aren't all that different from what's been mandated for other organizations.

With respect to the RSA breach, Federal News Radio 1500 AM has an interview with Gene Spafford, Executive Director, CERIAS, that is well worth a listen.

Getting Help
I'm not going to pick on LE for this one, because this same sort of thing applies to others, as well...but even with "simple" customer service stuff, I tend to get names, titles, and such when talking to folks.  In this particular case, it appears that the information on the cell phone was thought to be important, so one would think that all due care in handling the device would be the order of the day.

I recently had an issue I was attempting to address and wanted to get some credible information before proceeding.  As such, I reached out to Eoghan and Terrence at cmdLabs, and I have to say that their response and support was very greatly appreciated.

The point of all this is something that I think has its roots in the community fragmentation discussion that took place recently.  For my money, if the situation were more than just idle curiosity, I'd seek credible assistance.  However, having observed the industry for a number of years, I still believe that many analysts are simply afraid (for whatever reason) to ask for or seek out assistance, and others are simply asking the wrong questions to begin with.

What are your thoughts?

APT Tabletop Exercise
This recent post (link above) over on the SANS ISC blog caught my attention.  One statement in particular that caught my eye was:

If you're not on the obvious-targets list already...

Kevin goes on in the post to point out that every organization has its secrets, and he's absolutely right.  More importantly, I would suggest that it's the height of folly to think that you don't have something someone wants, whether that something is intellectual property, or simply storage space and CPU cycles.  I mean, here in the US, it's getting close to tax time, so if you were to survey a wide range of systems, you would probably find tax-related software (TurboTax, etc.) and information on many of these systems.  Also, loading a key logger or 'data jacker' on a system will often result in online banking or shopping credentials being revealed.  Ultimately, a compromise may come down to simply the connectivity and processor power, as the compromised system is part of a botnet.

Speaking of APT and preparing, it would probably be a good idea to take a look at the RSA blog, and specifically the Anatomy of an Attack post, written by Uri Rivner.  The post does a very good job of covering the general methodology of how these attacks occur, and even provides information about the specific details regarding the spear phishing attack that led to the initial entry.

For folks who are more detailed oriented, such as myself, the rest of the post is light on actual details, with the exception of some domains used by the attackers.  There are some details, such as the use of FTP to transfer files, but in other cases, the methodology is discussed without details.  That being said, there is enough information there to infer the nature of what occurs on systems compromised by these actors.

Looking at the details that were provided, I have to wonder what that post would look like had Carbon Black been installed on critical systems within the infrastructure.  Speaking of which, the early release of Carbon Black starts today...drop on by the Kyrus web page and check it out!


Blog
Brad Garnett moved his blog recently...it's now the Digital Forensic Source blog.  Brad hasn't been terribly prolific, but if you follow the Case Leads posts on the SANS Forensics Blog, you'll find Brad's posts to be similar, providing links to items you might otherwise have missed.  Sometimes you may run across something that will lead to another blog or blog post or media article that's of interest.



Reading
Speaking of blogs, Richard took the time to answer some questions from a previous post (on Reading) over on the TaoSecurity blog.  I thought that the previous post was interesting, but found the questions that Richard chose to respond to even more so.


Over the years, I've found Richard's reviews of various books to be very insightful, and it has been clear that he's put some considerable effort into the reviews.  I have always suspected that he does this because he puts his name on the review (ie, doesn't post it under Anonymous), and realizes that the entire review reflects upon him and his position.  I've seen quite a number of reviews of digital forensics that have amounted to "It was good" or simply "It sucked."  I've seen other reviews that were really nothing more than a listing of chapters and a regurgitation of the content in each.  As such, I tend to put a lot of value in reviews such as those that Richard writes, as they are very insightful.

I especially enjoyed the first question (and response) in the blog post.  I won't copy-paste it here, as you really should head on over to Richard's blog and check it out.  Regardless, I have to say that the question is pretty typical; like others, Richard is doing something that he enjoys doing, and he's doing it entirely for free.  Yet, there's always going to be someone who will ask for something more, many times without offering anything of their own. 

With respect to the final question on the blog, I pretty much followed what Richard mentioned when I did a recent review of the ebook edition of Cybercrime and Espionage; while I took notes directly in the material using the Kindle functionality, I also took notes in a notebook.  I know that some will probably look at that as extra work, but I usually find when I read books such as the one from Gragido and Pirc that I not only get ideas and insights about the material presented, but I will also sometimes find tie-ins to other books or online materials, so having handwritten notes is a great way of solidifying those thoughts in my mind, and having something right in front of me to review later.

Windows 8
Finally, I caught an article on CNNMoney this morning that indicates that Microsoft is getting ready to release Windows 8.  I love job security.

As a follow-on (or one of them, anyway) to my previous post on investigating breaches, I wanted to perhaps scratch the surface a bit more (as opposed to "digging deeper") regarding the subject.

Take for instance a Windows XP laptop, which is a pretty typical corporate configuration. Let's say that some suspicious activity was seen originating from this system by network admins, when the system was connected to the corporate network. Management feels that the employee is loyal and not the issue, and they suspect that the system may have been compromised with it was connected to another network, such as the employee's home or a hotel network. As such, management would like you to examine the system and determine if it had been compromised, and if so, how and when.

Pretty easy, right? Maybe, maybe not. Like I said, often it's relatively easy to find secondary indicators of a compromise...tools that the intruder may have loaded onto the system following the initial compromise. However, it may not be so easy to find the primary indicators of the initial intrusion.

So, when starting such an investigation, what would you want to look for? Well, I'd start by getting an idea of what services and applications are installed on the system. For example, was there a web server (hey, I've seen it!) installed? FTP server? Anything used for remote access? PCAnywhere? VNC? What about applications...any P2P? Also, be sure to check the Windows firewall configuration...I've seen indicators of malware there.

Another aspect to look at is, what is the user's default browser (yes, there is RegRipper plugin for that!)? IE? Well, what files are in their cache (browser drive-bys or "browse-bys" are a big issue)? Any documents? PDF files? JavaScript? Anything that can download or carry executable code? Email? What about Outlook, and any files downloaded as Outlook attachments?

While we're on the subject of IE, index.dat files, and browsing history, don't forget about checking the Default User profile for indications of web browsing history...another quick check but hey, I've found some pretty amazing things here.

An alternative means of analysis is to mount the image and scan it for malware, being sure to see what AV was installed on the system (if any...be sure to check for MRT) and then use something else (PCTools, AVG, Avira...be sure to check licenses), looking for malware, keyloggers, etc. Other quick checks for a better view include checking the hosts file for modifications, the firewall configuration (mentioned above), running tools like wfpcheck, etc. All of this can be done rather quickly, and provide a much more comprehensive analysis than just running AV. From this, you might hope to find some secondary indicators that might at least provide a point-in-time reference, the reasoning being that a secondary indicator would occur after a primary one.

Other areas of analysis include Event Logs, Registry autostart analysis (more of a follow-on to looking into running services), and even analyzing any available crash dump logs for indications of unusual processes.

What I'd really like to do is find a forensic challenge available online that consists of analyzing a compromised Windows system, and provide a walk-through of the methods and procedures used in the analysis. While there are a number of images online for download as part of challenges, the ones I have found involve malicious activity on the system, conducted by the user...I have yet to find one that involves a system being compromised or hacked remotely.

I recently received an email from someone who said that he wanted to learn more about "network intrusion investigations". Seeing this, I asked him to clarify, and he said that he was interested in learning what to look for when someone breaks into a system from the network.

This got me to thinking...how would one go about teaching this subject? What's a better way to see if you really understand something than to sit down and try to communicate a how-to to another person? So I started thinking about this, and it brought up another conversation I had with some other folks...actually, a series of conversations I had over about 18 months. Specifically, I had conversations about intrusion investigations with some guys who went about discovering vulnerabilities and writing exploits. My thinking was that in order for these guys to be able to report a successful exploit using a vulnerability they found, they would have to have a test system or application, and a condition to define success. I won't got into the details of the exchange...what matters here is that at one point, one of them said, "you aren't looking for artifacts of the initial intrusion...you're looking for artifacts of what the bad guy does after the exploit succeeds."

Well, I have to say, I disagreed with that at that time, but for the purposes of investigating data breaches, one of the primary things you need to determine is, was the system in question, in fact, breached? One way to answer that question is to look for indications of "suspicious" activity on the system. Was malware or some means of access or persistence installed? AV scans are a good place to start, but I'd suggest that analyzing the acquired image for indications of AV tools already installed should be the first step. Why is that? How many analysts mount an acquired image and scan it with an updated commercial AV scanning tool? Okay, you can put your hands down. Now, how many check the system or the image first to see that the tool they use wasn't the one installed on the system? See my point? If the AV scanner missed something once, what's to say it won't miss it again?

Anyway, I've already talked about malware detection, so I won't belabor the point here.

Peter Silberman of Mandiant once mentioned that malware is often "the least frequency of occurrence" on a system, and in many instance, the same may apply to an intrusion. As such, the analyst will not be looking for sudden increases in activity on a system but instead looking for very specific data points within a discrete time window. I've created several timelines in which the predominance of file system activity was the result of system or application updates, as well as AV scans run by the administrators. Often, the necessary data points or time window may be established via other means of analysis.

Overall, I don't believe that you can teach one specific means for investigating breaches, and consider that sufficient. Rather, what must be done instead is to develop an overall understanding of data sources, and what a breach "looks like", and then conduct specific analysis from there.

For example, by what means could a system be breached? Well, let's narrow the field a bit and say that we're looking at a Windows XP laptop...what possibilities are there? There are those services that may be running and listening for connections (check the firewall configuration as it may not be the default), or there may be indications of a breach as a result of user activity, such as downloads via the browser (intentional or otherwise), email, P2P applications, etc. What we may end up doing is examining the system for secondary indications of a breach (i.e., creation of persistence mechanisms, such as user accounts, etc.), and working from there to, at the very least, establish a timeline or at least an initial reference point.

Another point to remember about investigations in general is that the more data points that you have to support your findings, the better. This not only helps you build a more solid picture (and eliminate speculation) of what happened on the system (and when) but it also allows you to build that picture when some data points do not exist.

Let's look at an example...let's say that you suspect that the system you're examining may have been accessed via Terminal Services, using compromised credentials. You examine the Windows Services found in the Registry and determine that Terminal Services was set to start when the system started, and other data points to the fact that remote connections were allowed. So your next step might be to look at Security Event Log entries that would show signs of logins...but you find through Registry and Event Log analysis that auditing of login events wasn't enabled. What then? Well, you might think to look for the time that specific files had last been accessed...but if the system you're examining is a Vista system, well, by default, the updating of last access times is disabled. So what do you do?

Again...the more data points you have to support a finding, the better off you'll be in your analysis. Where one data point is good, six may be better. A knowledgeable analyst will know that while some modicum of work will be needed to establish those other five data points, much of it can be automated and made repeatable, increasing efficiency and reducing analysis time.

Perhaps a way to illustrate the overall theme of this post is to look at examples, and that's what we'll be doing in the future. In the meantime, questions and comments are always welcome.

Beware...here's your warning: This post is NOT specifically about Windows incident response or forensic analysis. Okay. There. You've been warned.

Brian Krebs of the SecurityFix blog posted recently that data breaches are more costly than ever. I can't say as I'm surprised...security folks have known for a long time that the cost of cleaning up a mess is much, much more than the cost of finding and fixing the issues before an incident happens (ie, how much does it really cost to put a password on that 'sa' account?). In some ways, this is common sense, but apparently, not "common" enough.

Look at the first sentence of Brian's post: Organizations that experienced a data breach paid an average of $6.6 million last year to rebuild their brand image and retain customers following public disclosures of the incidents...

I think that this makes a couple of key points...

I found a similar article on DarkReading, as well. Both articles reference the Ponemon Institute study, the US Cost of Data Breach Study. A couple excerpts from the articles that I found interesting:

Data breaches experienced by "first timers" are more expensive than those experienced by organizations that have had previous data breaches....

No kidding. You get caught with your pants down (I wish I had a graphic, but hey, this is a PG-13 blog), and that's what happens. You don't know enough about your architecture or infrastructure, about where sensitive data is stored or processed...and then someone intrudes into your network and figures it out...but not for you. For themselves. And takes stuff.

Also, Brian pointed out hard costs associated with breaches, such as hiring forensic analyst and responder firms, setting up call centers, and discounts on future products. These all go beyond the other hard costs, such as notification (estimated now at ~ $100 per record), law suits (don't believe me? Check out this article...), etc.

The fact of the matter is that breaches themselves are far less expensive than the cost of preparation. There's no question...incidents WILL happen. We've seen the surveys, and we've seen this in the media. So why wait for a breach to happen in order to justify spending to prevent or detect security incidents? I mean, seriously...do companies hire employees and wait until someone brings a law suit to start paying them? Do companies offer a product without sales, marketing, maintenance, and some way of billing customers (invoice, collections, etc.)?

So where most organizations are right now is that they're likely sitting at a table with someone who wants to sell them services of some kind that have to do with security. The decision point is the certainty of the purchase order that the sales guy is pushing across the table versus the possibility of a major data breach occurring at some point in the future. I understand this. However, what I do not understand is why organizations that store and process MY sensitive data do not recognize that that possibility or probability is rapidly approaching certainty. If you cannot clearly demonstrate your controls and you have no visibility into your infrastructure, the I would suggest that it's already there.

Despite their experiences, however, most companies still don't plan financially for data breaches.
Now, see this is the part I don't get. It's one thing to look at the Hannafords and the Heartlands, and to think, hey, it won't happen to ME. However, it's another thing entirely for it to happen to you, for you to be informed as to just how vulnerable you really are, and then for you to think, hey, it won't happen to me AGAIN.

The study also does not measure the cost of intellectual property that is lost or stolen as a result of a data breach.

Ugh. How many organizations out there are smuggly grinning ear-to-ear, because they don't store or process "sensitive data", as defined by PCI, NCUA, or any of the abundant state legislation, all while their intellectual property is being siphoned off? "No, we don't have to meet compliance because we're not required to." Well, don't you want to...you know...protect your intellectual property (R&D, manufacturing process specs, new drug/medication design specs, etc.) just because its the RIGHT THING to do? Well, no...where's the ROI in that?

Oh, and since I mentioned Heartland, I simply can't let this pass without mentioning Cory's comments on this recent StoreFrontBackTalk article. To me, as an experienced responder, this is a great example of "denial isn't just a river in Egypt". Yeah, it's also a great example of how these things get wildly miscommunicated through Public Relations or Corporate Communications, and then on to the media, but it also reminds me that folks are more likely to tell their friends, "hey, I got mugged on the way home from work and beat up by the Pittsburgh Steeler's defensive line", when in fact the truth is that a 12 yr old girl took your wallet. Another way to say it is, make the story sound a lot better than it really is...don't let the facts get in the way of a good story.