How many times do we launch a tool to parse some data, and then sit there looking at the output, wondering how someone would see something "suspicious" or "malicious" in the output?  How many times do we look at lines of data, wondering how someone else could easily look at the same data and say, "there it is...there's the malware"?  I've done IR engagements where I could look at the output of a couple of tools and identify the "bad" stuff, after someone else had spent several days trying to find out what was going wrong with their systems.  How do we go about doing this?

The best and most effective way I've found to get to this point is to take what I learned on one engagement and roll it into the next.  If I find something unusual...a file path of interest, something particular within the binary contents of a file, etc...I'll attempt to incorporate that information into my overall analysis process and use it during future engagements.  Anything that's interesting, as a result of either direct or ancillary analysis will be incorporated into my analysis process.  Over time, I've found that some things keep coming back, while other artifacts are only seen every now and then.  Those artifacts that are less frequent are no less important, not simply because of the specific artifacts themselves, but also for the trends that they illustrate over time.

Before too long, the analysis process includes, "access this data, run this tool, and look for these things..."; we can then make this process easier on ourselves by taking the "look for these things" section of the process and automating it.  After all, we're human, get tired from looking at a lot of data, and we can make mistakes, particularly when there is a LOT of data.  By automating what we look for (or, what we've have found before), we can speed up those searches and reduce the potential for mistakes.

Okay, I know what you're going to say..."I already do keyword searches, so I'm good".  Great, that's fine...but what I'm talking about goes beyond keyword searches.  Sure, I'll open up a lot of lines of output (RegRipper output, web server logs) in UltraEdit or Notepad++, and search for specific items, based on information I have about the particular analysis that I'm working on (what are my goals, etc.).  However, more often than not, I tend to take that keyword search one step further...the keyword itself will indicate items of interest, but will be loose enough that I'm going have a number of false positives.  Once I locate a hit, I'll look for other items in the same line that are of interest.

For example, let's take a look at Corey Harrell's recent post regarding locating an injected iframe.  This is an excellent, very detailed post where Corey walks through his analysis process, and at one point, locates two 'suspicious' process names in the output of a volatile data collection script.  The names of the processes themselves are likely random, and therefore difficult to include in a keyword list when conducting a search.  However, what we can take away from just that section of the blog post is that executable files located in the root of the ProgramData folder would be suspicious, and potentially malicious.  Therefore, a script that that parses the file path and looks for that condition would be extremely useful, and written in Perl, might look something like this:

my @path = split(/\\/,$filepath);
my $len = scalar(@path);
if (lc($path[$len - 2]) eq "programdata" && lc($path[$len - 1]) =~ m/\.exe$/) {
  print "Suspicious path found: ".$filepath."\n";
}

Similar paths of interest might include "AppData\Local\Temp"; we see this one and the previous one in one of the images that Corey posted of his timeline later in the blog post, specifically associated with the AppCompatCache data output.


Adding intel is generally difficult to do with third-party, closed source tools that we download from someone else's web site, particularly GUI tools.  In such cases, we have to access the data in question, export that data out to a different format, and then run our analysis process against that data.  This is why I recommend that DFIR analysts develop some modicum of programming skill...you can either modify someone else's open source code, or write your own parsing tool to meet your own specific needs.  I tend to do this...many of the tools I've written and use, including those for creating timelines, will incorporate some modicum of alerting functionality.  For example, RegRipper version 2.8 incorporates alerting functionality directly into the plugins. This alerting functionality can greatly enhance our analysis processes when it comes to detecting persistence mechanisms, as well as illustrating suspicious artifacts as a result of program execution.


We don't always have to get our intelligence just from our own analysis. For example, this morning on Twitter, I saw a tweet from +Chris Obscuresec indicating that he'd found another DLL search order issue, this one on Windows 8 (application looked for cryptbase.dll in the ehome folder before looking in system32); as soon as I saw that, I thought, "note to self: add checking for this specific issue to my Win8 analysis process, and incorporate it into my overall DLL search order analysis process".

The key here is that no one of us knows everything, but together, we're smarter than any one of us.

I know that what we've discussed so far in this post sounds a lot like the purpose behind the OpenIOC framework.  I agree that there needs to be a common framework or "language" for representing and sharing this sort of information, but it would appear that some of the available frameworks may be too stringent, not offer enough flexibility, or are simply internal to some organizations.  Or, the issue may be as Chris Pogue mentioned during the 2012 SANS DFIR Summit..."no one is going to share their secret sauce."  I still believe that this is the case, but I also believe that there are some fantastic opportunities being missed because so much is being incorporated under the umbrella of "secret sauce"; sometimes, simply sharing that you're seeing something similar to what others are seeing can be a very powerful data point.

Regardless of the reason, we need to overcome our own (possibly self-imposed) roadblocks for sharing those things that we learn, as sharing information between analysts has considerable value.  Consider this post...who had heard of the issue with imm32.dll prior to reading that post?  We all become smarter through sharing information and intelligence.  This way, we're able to incorporate not just our own intelligence into our analysis processes, but we're also able to extend our capabilities by adding intelligence derived and shared by others.

I recently ran across this very interesting paper by Dan Guido; the paper is titled, "A Case Study of Intelligence-Driven Defense".  Dan's research point out the fallacy of the current implementation of the Defender's Paradigm, and how attackers, however unknowingly, are exploiting this paradigm.  Dan's approach throughout the paper is to make his very valid points based on analysis of information, rather than vague statements and speculation.

In the paper, Dan takes the stance, in part, that:

- Over the past year, MITRE's CVE identified and tracked more than 8,000 vulnerabilities
- Organizations expend considerable resources (man-power, money, etc.)
- In 2010, only 13 vulnerabilities "were exploited to install SpyEye, Zeus, Gozi, Clampi and other info-stealing Trojans in massive exploitation campaigns".

As such, his point is that rather than focusing on compliance and having to address all 8K+ vulnerabilities, a more effective use of resources would be to put more focus on those vulnerabilities that are actually being included in the crimeware packs for mass malware distribution.  Based on the information that Dan's included in the paper, this approach would also work for targeted attacks by many of the "advanced" adversaries, as well.

Dan goes on to say:

"Analysis of attacker data, and a focus on vulnerabilities exploited rather than vulnerabilities discovered, might yield more effective defenses."

This sounds very Deming-esque, and I have to agree.  Admittedly, Dan's paper focuses on mass malware, but in many ways, I think that the approach Dan advocates can be used across a number of other data breach and compromise issues.  For example, many of the folks who are working the PCI forensic audits are very likely still seeing a lot of the same issues across the board...the same or similar malware placed on systems that are compromised using some of the same techniques.  So, Dan's approach to intel-driven defense can be applied to other aspects of DFIR besides mass malware infections in an equally effective manner.

Through the analysis he described in his paper, Dan was able to demonstrate how focusing on a few, low-cost (or even free), centrally-managed updates could have significantly reduced the attack surface of an organization and limited, inhibited, or even stopped mass malware infections.  The same approach could clearly be applied to many of the more targeted attacks, as well.

Where the current approach to infosec defense falls short is the lack of adequate detection, response, and analysis.

Detection - look at the recent reports available from TrustWave, Verizon, and Mandiant, and consider the percentage of their respective customers for whom "detection" consists of third-party notification.  So determining that an organization is infected or compromised at all can be difficult.  Detection itself often requires that monitoring be put in place, and this can be scary, and thought of as expensive to those who don't have it already.  However, tools like Carbon Black (Cb) can provide a great deal of ROI, as it's not just a security monitoring tool.  When used as part of a security monitoring infrastructure, Cb will retain copies of binaries (many of the binaries that are downloaded to infected systems will be run and deleted) as well as information about the processes themselves...information that persists after the process has exited and the binary has been deleted as part of the attack process.  The next version of Cb will include Registry modifications and network initiations, which means that the entire monitored infrastructure can then be searched for other infected systems, all from a central location and without adding anything additional to the systems themselves.

Response - What happens most often when systems are found to be infected?  The predominant response methodology appears to be that systems suspected to be compromised or infected are taken offline, wiped, and the operating system and data are reinstalled.  As such, critical information (and intelligence) is lost.

Analysis - Analysis of mass malware is often impossible because a sample isn't collected.  If a sample is available, critical information about "in the wild" artifacts are not documented, and AV vendor write-ups based on the provided samples will only give a partial view of the malware capabilities, and will very often include self-inflicted artifacts.  I've seen and found malware on compromised systems for which if no intelligence had been provided, the malware analyst would have had a very difficult time performing any useful analysis.

In addition, most often, the systems themselves are not subject to analysis...memory is not collected, nor is an image acquired from the system.  In most cases, this simply isn't part of the response process, and even when it is, these actions aren't taken because the necessary training and/or tools aren't available, or they are but they simply aren't on-hand at the time.  Military members are familiar with the term "immediate actions"...these are actions that are drilled into members so that they become part of "muscle memory" and can be performed effectively while under stress.  A similar "no-excuses" approach needs to be applied as part of a top-down security posture that is driven by senior management.

Another important aspect of analysis is sharing.  The cycle of developing usable, actionable intelligence depends on analysis being performed.  That cycle can move much faster if the analysis methodologies are shared (and open to review and improvement), and if the results of the analysis are also shared.  Consider the current state of affairs...as Dan points out, the cycle of developing the crimeware packs for mass malware infections is highly specialized and compartmentalized, and clearly has an economic/monetary stimulus.  That is, someone who's really good at one specific step in the chain (i.e., locating vulnerabilities and developing exploits, or pulling the exploits together into a nice, easy to use package) performs that function, and then passes it along to the next step, usually for a fee.  As such, there's the economic motivation to provide a quality product and remain relevant.  Ultimately, the final product in the chain is being deployed against infrastructures with little monitoring in place (evidenced by the external third party notification...).  When the IT staff (who are, by definition, generalists) are notified of an issue, the default reaction is to take the offending systems offline, wipe them and get them back into service.  As such, analysis is not being done and a great deal of valuable intelligence is being lost.

Why is that?  Is it because keeping systems up and running, and getting systems back online as quickly as possible are the primary goals of infected organizations?  Perhaps this is the case.  But what if there were some way to perform the necessary analysis in a timely manner, either because your staff has the training to do so, or because you've collected the necessary information and have a partner or trusted adviser who can perform that analysis?  How valuable would it be to you if that partner could then provide not only the results of the analysis in a timely manner, but also provide additional insight or intelligence to help you defend your organization due to partnership with law enforcement and other intel-sharing organizations?

Consider this...massive botnets have been taken down (Khelios, Zeus, etc.) when small, dedicated groups have worked together, with a focused approach, to achieve a common goal.  This approach has been proven to be highly effective...but it doesn't have to be restricted to just those folks involved in those groups.  This has simply been a matter of a couple of folks saying, "we can do this", and then doing it.

The state of DFIR affairs is not going to get better unless the Defender's Paradigm is changed.  Information needs to be collected and analyzed, and from that, better defenses can be put in place in a cost effective manner.  From the defender's perspective, there may seem to be chaos on the security landscape, but that's because we're not operating from an intelligence-driven perspective...there are too many unknowns, but these unknowns are there because the right resources haven't been focused on the problem.

Here are Shawn Henry's (former exec. assistant director at the FBI) thoughts on sharing intel:
“We have to share threat data. My threat data will help enable you to be more secure. It will help you be predictive and will allow you to be proactive. Despite our best efforts we need to assume that we will get breached, hence we need to ensure our organisations have consequence management in its systems that allow us to minimise any damage.” 

Resources
Dan's Exploit Intel Project video

Not long ago, I read Greg's blog post on using HBGary's DDNA to detect APT attackers in memory, and ran across an interesting statement near the end of the post, specifically, the first paragraph in the Using IOC’s effectively subsection (it's kind of long so I won't copy/paste it here...though I highly recommend that you take a look).  I have to say, I agree with Greg's statements...IOCs can be effective tools, but like any tool, they are only as effective as how they are used.  I think that IOCs have great potential in data reduction and providing an automated facility for removing the "low-hanging fruit" from analysis, thereby leaving the analyst to...you know...analyze.  But like Greg says, if your IOCs are just lists of file names and MD5 hashes, then they really aren't all that effective.

Back when I was performing PCI forensic exams, one of the most frustrating things was the lists of "IOCs" we got from one of the major card brands; we'd get these lists of mostly file names and MD5 hashes, two of the most mutable aspects of IOCs, and we were told that we had to include these ever-expanding lists in searches across all acquired images.  Okay, so let's say we found bp0.exe during an exam, but the hash was different from the one provided in the issued list...we noted that and sent it back in, and then the next list we saw would have the new hash added to it.  However, the lack of sharing any real intel was frustrating, as having a more comprehensive intel model would have allowed all analyst firms to move beyond the relatively simple, "yes, you were compromised" and more toward, "this is how you were compromised, this is your window of compromise, and this is what you can do to prevent it in the future."  For example, we'd get file names, but in most cases, not paths.  We'd get MD5 hashes, but not fuzzy hashes, and Chris Pogue would push for fuzzy hashes.  In most cases, we wouldn't even get things like Registry artifacts that would let us see when something had been installed and run, but then deleted.

So, per Greg's statements, one way to make IOCs more effective is to create them so that they're not just a simple blacklist.  Rather than looking for a specific Registry value used for persistence (because file names and Registry value names can be random and change...), have the IOC either dump all of the values in the key for manual review, or correlate multiple keys and values in order to narrow the focus a bit.  Again referring back to past PCI engagements, when looking for credit card numbers (CCNs) within an image, we had three checks...length, BIN, and Luhn check...that we would use, but we got a lot of false positives (such as MS GUIDs).  However, adding additional checks for track 1 and 2 data not only narrowed down our criteria, but increased our confidence that what we'd found was actual useful data and NOT false positives.  So, as Greg suggests, looking for more general things, or adding additional items to your IOCs, can be very effective.

The effectiveness of IOCs is increased though open source intel sharing, by having someone who discovers something develop an effective (and well-documented) IOC and sharing with their team.  These IOCs can be even more effective if they're shared openly (even if it's within a relatively closed community), because then they're not only subject to a more open review, but they can also be expanded further.

The effectiveness of IOCs is obviated and completely lost when the folks using IOCs do nothing but consume them; that is, download the latest IOCs and run them without any review or understanding.  Tools that use a plugin-based approach (Nessus comes to mind...) are only as effective as the effort put into the plugins, and what's actually reported.  A recent InformationWeek article describes eight lessons learned from the recent Nortel revelation...number 6 is "conduct a thorough forensic analysis".  If you suspect that you have something untoward going on in your infrastructure, and you run a list of IOCs and don't get anything definitive...maybe it's time to break out the toolkit and take things a step further.  You may find something entirely new...or you may simply get peace of mind.

Let's take a look at a couple of tandem blog posts that addressed some malware analysis recently.  I caught wind of this on Twitter...we all know how difficult it is to share anything on Twitter, because you have squeeze things down into 140 characters, or make multiple, abbreviated posts...and a lot gets lost that way.  First, we have Keith's post on the Digital4rensics blog, and then there's the Digitalsec4u post on the New Age Security blog.  Both address different components of the analysis of something that hit an infrastructure and was shared, in order to complete some analysis.  You have to look at both of these blog posts together to pull out the IOCs; read the Digitalsec4u post first to see what hit the infrastructure, and what it "looked like" when it came in, and get the Initial Infection Vector.  Then read Keith's post to begin getting a view of what happened once the original bit of malware was executed.

Now, I know from some perspectives, neither the malware that was analyzed nor the actual results are particularly "earth shattering", particularly from the perspective of these two gents who very graciously shared their analysis.  However, there is considerable value in the data that has been posted and shared.  For example, this malware came in as an email attachment, disguised to look like a plane ticket.  We can see hashes on the original payload, and we know that these artifacts are easily mutable.  We can also see that the malware is packed...that's something we can latch on to (IOC: search email attachment directories for packed executables).  Moving over to Keith's post, we begin to see what happens when the original malware is executed.  We can use this information to see if our IR (and DF) processes are sufficient to deal with this type of issue; assume that it wasn't detected and blocked by your in-place security mechanisms, how would you go about responding to this?  Or, better yet, if part of it was blocked, would your current response and analysis process be suitable to determine that the follow-on stages hadn't actually executed?  This is an extremely important question, and one that not a lot of responders and analysts have been trained to consider, or answer.

Some lessons from the two blog posts:
Engage with those with adjacent skill sets; the above two blog posts are great examples of malware analysts doing their thing, but there are a number of artifacts and other bits of information that would be extremely useful for responders and analysts.  Keith's blog has some request URLs that network analysts can search for in proxy and firewall logs, but there's a question as to how these requests are made - does the malware use the WinInet APIs?  If so, we would expect to see other artifacts on the system, wouldn't we?  So, it's not unusual for an analyst to highlight or share what makes sense to them, but they have the information right there in front of them that would be useful to someone else, perhaps someone with an adjacent skill set (responder, DF analyst, network analyst, etc.), so sharing that other information with them would reduce the amount of time looking for these things.

When sharing that information, specificity of language is important.  Does the malware create a Registry key...or does it add a value to an existing key?  If it adds a value to the Run key...which one?  The same is true for file and directory paths, as well as identifying the platform you're infecting during your analysis.

In short, being clear and concise in presenting your findings makes it much easier for someone to not only replicate your findings (taking variations in platform...WinXP vs Win7...into account), but to also use that information in an immediate response mode, and begin seeing if their systems are infected.

Okay, so why are these lessons important?  There's a bit more to this than was mentioned in the blog posts; Keith mentioned that one of the bits of malware was identified by Microsoft (via VirusTotal) as PWS:Win32/Sypak.A...which is not just a password stealer, but it's a keystroke logger.  This means that it just doesn't capture passwords...it can potentially capture everything, and this can be extremely detrimental to an organization.

A little over a month ago, I purchased a Dell Inspiron 1545 from the Dell Refurbished shelf. Most of the systems I've purchased from Dell have been procured through this route, and I've been pretty happy with the systems.

Until now. Tonight, I was p0wned by MS.

See, I'd purchased the laptop to do Windows 7 forensic (and in particular Registry) research. You know, use it like a user would and then see what the system "looks like" from a forensic perspective. Do what a user would do, then do like a forensic nerd would do.

Well, it seems that the laptop I purchased is running an Intel T4200 processor. It has 410 Million transistors, but does not support hardware virtualization.

Okay, my thought was that I was going to get an almost-brand-spanking-new system...no way it wouldn't support hardware virtualization. Well, it doesn't. What this means is that this laptop doesn't support XP Mode. Wow, so much for that rather critical portion of research.

So, the lesson learned here is, don't assume that the latest and greatest box, even one birthed in the past year, is going to have the necessary functionality to support what you want to do. In fact, as far as XP Mode is concerned, if that system you've got your eye on has an Intel processor, assume that it doesn't until proven otherwise. Wonder where I got that? My favorite forensics tool, Cory Altheide, found this at the Parallels site...notice what it says at the bottom of the page about AMD microprocessors.

Addendum: It seems that while the laptop I just purchased does not have a processor that supports hardware virtualization, the Dell Latitude D820 that I purchased in 2006 DOES! All I need to do is enable it in the BIOS...