Are You Being Served, pt II
This article isn't going to be directed toward digital analysts; rather, it will be directed more to folks who hire or contract with analysts or firms, and are the recipients (or customers) of the technical work performed by those digital forensics analysts. My goal here is to simply express some thoughts on how customers might go about determining if the results of the work that they contracted for are meeting their needs.
Previously in this blog, I asked the question, Are you being served? If you've asked yourself that question, you may be wondering...how would I know? Selecting a DFIR analyst (either an individual or a firm) is really no different for evaluating and hiring any other provider of services, such as a plumber or auto mechanic. The difference is that plumbers and mechanics fix something for you, and you can evaluate their services based on if the problem is fixed, and for how long. For customers of digital analysis services, determining if you're getting what you paid for is a bit more difficult.
In exploring the subject of finding a digital forensics expert, I ran across this article at the Law.com web site. The article contains a number of aspects of the overall digital analysis services that lawyers should consider when looking for a digital forensics expert. For example, the article suggests that when asked to identify methods of data exfiltration, analysts should include USB devices. This is good to know, but more importantly, does the analyst identify all such devices, or only the thumb drives? How do you know? Does the analyst make an attempt to determine the use of counter-forensics techniques, where a user might delete certain artifacts in an attempt to hide the fact that they connected a specific device to the system? What details can the analyst provide with respect to the device being connected to the system, and how a user may have interacted with that device? Regardless of the data exfiltration method used (USB device, web mail, BlueTooth, etc.), how does the analyst address data movement, in particular?
Beyond those items addressed in the article, some other things to consider include (but are not limited to):
Does the analyst explore historical data, such as Volume Shadow Copies (VSCs), when and where it is appropriate to do so? If not, why? If the methodology used by the analyst fails to find any VSCs, what does the analyst state as the reason for this finding?
What about other artifacts? When the analyst provides a finding, do they have additional artifacts to support their findings, or are their findings based on that one artifact? If artifacts (such as Prefetch files) are not examined or missing, what reason does the analyst provide?
If you're interested in the existence of malware on a system, what does the analyst do to address this issue? Do they run AV against the mounted image? What else do they do? If malware is found, do they determine the initial infection vector? Do they determine if the malware ever actually executed?
When you look at the report provided, does the information in it answer your questions and address your concerns, or are there gaps? Does the analyst connect the dots in the report, or do they skip over many of the dots, and fill in the gaps using speculation?
One question that you might consider asking is, what tools does the analyst use, but I would suggest that it's more important to know how the tools are used. For example, having access to one of the commercial analysis suites can be a good thing, particularly if the analyst states that they will use it on your case to perform a keyword search. But does it make sense to do so? Did they work with you to develop a list of keywords to use in the search? I've heard of examinations that were delayed for some time while the data was being preprocessed and indexed in preparation for a keyword search, yet none of the analysts could state why the keyword search was necessary or of value to the case itself.
There is often much more to digital analysis than simply finding one or two artifacts in order to "solve the case". Systems today are sufficiently complex that multiple artifacts are needed to identify the context of a single artifact, such as a tool not finding VSCs within an image of a Windows 7 system. Digital analysis is very often used as the basis for making critical business decisions or addressing legal questions, so the question remains...are you being served? Are you getting the data that you need, in a timely manner, and in a manner that you can understand and use?
Resources
Law.com - How to Find a Digital Forensics Expert
Interested in Windows DFIR training? Windows Forensic Analysis, 11-12 Mar; Timeline Analysis, 9-10 Apr. Pricing and Calendar. Send email here to register.
Previously in this blog, I asked the question, Are you being served? If you've asked yourself that question, you may be wondering...how would I know? Selecting a DFIR analyst (either an individual or a firm) is really no different for evaluating and hiring any other provider of services, such as a plumber or auto mechanic. The difference is that plumbers and mechanics fix something for you, and you can evaluate their services based on if the problem is fixed, and for how long. For customers of digital analysis services, determining if you're getting what you paid for is a bit more difficult.
In exploring the subject of finding a digital forensics expert, I ran across this article at the Law.com web site. The article contains a number of aspects of the overall digital analysis services that lawyers should consider when looking for a digital forensics expert. For example, the article suggests that when asked to identify methods of data exfiltration, analysts should include USB devices. This is good to know, but more importantly, does the analyst identify all such devices, or only the thumb drives? How do you know? Does the analyst make an attempt to determine the use of counter-forensics techniques, where a user might delete certain artifacts in an attempt to hide the fact that they connected a specific device to the system? What details can the analyst provide with respect to the device being connected to the system, and how a user may have interacted with that device? Regardless of the data exfiltration method used (USB device, web mail, BlueTooth, etc.), how does the analyst address data movement, in particular?
Beyond those items addressed in the article, some other things to consider include (but are not limited to):
Does the analyst explore historical data, such as Volume Shadow Copies (VSCs), when and where it is appropriate to do so? If not, why? If the methodology used by the analyst fails to find any VSCs, what does the analyst state as the reason for this finding?
What about other artifacts? When the analyst provides a finding, do they have additional artifacts to support their findings, or are their findings based on that one artifact? If artifacts (such as Prefetch files) are not examined or missing, what reason does the analyst provide?
If you're interested in the existence of malware on a system, what does the analyst do to address this issue? Do they run AV against the mounted image? What else do they do? If malware is found, do they determine the initial infection vector? Do they determine if the malware ever actually executed?
When you look at the report provided, does the information in it answer your questions and address your concerns, or are there gaps? Does the analyst connect the dots in the report, or do they skip over many of the dots, and fill in the gaps using speculation?
One question that you might consider asking is, what tools does the analyst use, but I would suggest that it's more important to know how the tools are used. For example, having access to one of the commercial analysis suites can be a good thing, particularly if the analyst states that they will use it on your case to perform a keyword search. But does it make sense to do so? Did they work with you to develop a list of keywords to use in the search? I've heard of examinations that were delayed for some time while the data was being preprocessed and indexed in preparation for a keyword search, yet none of the analysts could state why the keyword search was necessary or of value to the case itself.
There is often much more to digital analysis than simply finding one or two artifacts in order to "solve the case". Systems today are sufficiently complex that multiple artifacts are needed to identify the context of a single artifact, such as a tool not finding VSCs within an image of a Windows 7 system. Digital analysis is very often used as the basis for making critical business decisions or addressing legal questions, so the question remains...are you being served? Are you getting the data that you need, in a timely manner, and in a manner that you can understand and use?
Resources
Law.com - How to Find a Digital Forensics Expert
Interested in Windows DFIR training? Windows Forensic Analysis, 11-12 Mar; Timeline Analysis, 9-10 Apr. Pricing and Calendar. Send email here to register.