Are You Being Served?
Larry Daniel recently posted to his Ex Forensis blog regarding a very interesting topic, regarding "The Perils of Using the Local Computer Shop for Computer Forensics". I've thought about this before...when I was on the ISS ERS (and later the IBM ISS ERS) team, on more than one occasion we'd arrive on-site to work with another team, or to take over after someone else had already done some work. In a couple of instances, I worked with other teams that, while technically skilled, were not full-time DFIR folks. Larry's post got me to thinking about who is being asked to perform DFIR work, and the overall effect that it has on the industry.
There's a question that I ask myself sometimes, particularly when working on exams...am I doing all I can to provide the best possible product to my customers? As best I can, I work closely with the customer to establish the goals of the exam, to determine parameters of what they are most interested in. I do this, because like most analysts, I can spend weeks finding all manner of "interesting" stuff, but my primary interest lies in locating artifacts that pertain to what the customer's interested in, so that I can provide them with what they need in order to make the decisions that they need to make. As much as I can, I try to find multiple artifacts to clearly support my findings, and I avoid blanket statements and speculation, as much as I can.
Also, something that I do after every exam is take a look at what I did and what I needed to do, and ask myself if there's a way I could do it better (faster, more comprehensive and complete, etc.) the next time.
Let's take a step away from DFIR work for a moment. Like many, I make use of other's services. I own a vehicle, which requires regular upkeep and preventative maintenance. Sometimes, if all I need is an oil change, I'll go to one of the commercial in-and-out places, because I've looked into the service that they provide, what it entails, and that's all I need at the moment. However, when it comes to other, perhaps more specialized maintenance...brake work, inspections recommended by the manufacturer, as well as inspections of a trailer I own...I'm going to go with someone I know and trust to do the work correctly. Another thing I like about working with folks like this is that we tend to develop a relationship where, if during the course of their work, they find something else that requires my attention, they'll let me know, inform me about the issue, and let me make the decision. After all, they're the experts.
Years ago...1992, in fact...I owned an Isuzu Rodeo. I'd take it to one of the drive-in places to get the oil changed on a Saturday morning. The first time I took it to one place, I got an extra charge on my bill for a 4-wheel drive vehicle. Hold on, I said! Why are you adding a charge for a 4-wheel drive vehicle, when the vehicle is clearly 2-wheel drive? The manager apologized, and gave me a discount on my next oil change. However, a couple of months later, I came back to the same shop with the same vehicle and went through the same thing all over again. Needless to say, had I relied on the "expertise" of the mechanics, I'd have paid more than I needed to, several times over. I never went back to that shop again, and from that point on, I made sure to check everything on the list of services performed before paying the bill.
Like many, I own a home, and there are a number of reasons for me to seek services...HVAC, as well as other specialists (particularly as a result of Super Storm Sandy). I tend to follow the same sort of path with my home that I do with my vehicles...small stuff that I can do myself, I do. Larger stuff that requires more specialized work, I want to bring in someone I know and trust. I'm a computer nerd...I'm not an expert in automobile design, nor am I an expert in home design and maintenance. I can write code to parse Registry data and shell items, but I am not an expert in building codes.
So, the question I have for you, reader, is this...how do you know that you're getting quality work? To Larry's point, who are you hiring to perform the work?
At the first SANS Forensic Summit, I was on a panel with a number of the big names in DFIR, several of whom are SANS instructors. One of the questions that was asked was, "what qualities do you look for in someone you're looking to hire to do DFIR work?" At the time, my response was simply, "what did they do last week?" My point was, are you going to hire someone to do DFIR work, if last week they'd done a PCI assessment and the week prior to that, they'd performed a pen test? Or would you be more likely to hire someone who does DFIR work all the time? I stand by that response, but would add other qualifications to it. For example, how "tied in" are the examiners? Do they simply rely on the training they received at the beginning of their careers, or do they continually progress in their knowledge and education? Do they seek professional improvement and continuing education? More importantly, do they use it? Maybe the big question is not so much that the examiners do these things, but do their managers require that the examiners do these things, and make them part of performance evaluations?
Are you being served?
Addendum: Why does any of this matter? So what? Well, something to consider is, what will a CEO be reporting to the board, as well as to the SEC? Will the report state, "nothing found", or worse, will the report be speculation of a "browser drive-by"? In my experience, most regulatory organizations want to know the root cause of an issue (such as a compromise or data leakage)...they don't want a laundry list of what the issue could have been.
In addition, consider the costs associated with PCI (or any other sensitive information) data theft; if an organization is compromised, and they hire the local computer repair shop to perform the "investigation", what happens when PCI data is discovered to be involved, or potentially involved? Well, you have to go pay for the investigation all over again, only this time it's after someone else has come in an "investigated", and this is going to have a potentially negative effect on the final report. I think plumbers have a special fee for helping folks who have already tried to "fix" something themselves. ;-)
Look at the services that you currently have in your business. Benefits management. Management of a retirement plan. Payroll. Do you go out every month and select the lowest bidder to provide these services? Why treat the information security posture of the your organization this way?
There's a question that I ask myself sometimes, particularly when working on exams...am I doing all I can to provide the best possible product to my customers? As best I can, I work closely with the customer to establish the goals of the exam, to determine parameters of what they are most interested in. I do this, because like most analysts, I can spend weeks finding all manner of "interesting" stuff, but my primary interest lies in locating artifacts that pertain to what the customer's interested in, so that I can provide them with what they need in order to make the decisions that they need to make. As much as I can, I try to find multiple artifacts to clearly support my findings, and I avoid blanket statements and speculation, as much as I can.
Also, something that I do after every exam is take a look at what I did and what I needed to do, and ask myself if there's a way I could do it better (faster, more comprehensive and complete, etc.) the next time.
Let's take a step away from DFIR work for a moment. Like many, I make use of other's services. I own a vehicle, which requires regular upkeep and preventative maintenance. Sometimes, if all I need is an oil change, I'll go to one of the commercial in-and-out places, because I've looked into the service that they provide, what it entails, and that's all I need at the moment. However, when it comes to other, perhaps more specialized maintenance...brake work, inspections recommended by the manufacturer, as well as inspections of a trailer I own...I'm going to go with someone I know and trust to do the work correctly. Another thing I like about working with folks like this is that we tend to develop a relationship where, if during the course of their work, they find something else that requires my attention, they'll let me know, inform me about the issue, and let me make the decision. After all, they're the experts.
Years ago...1992, in fact...I owned an Isuzu Rodeo. I'd take it to one of the drive-in places to get the oil changed on a Saturday morning. The first time I took it to one place, I got an extra charge on my bill for a 4-wheel drive vehicle. Hold on, I said! Why are you adding a charge for a 4-wheel drive vehicle, when the vehicle is clearly 2-wheel drive? The manager apologized, and gave me a discount on my next oil change. However, a couple of months later, I came back to the same shop with the same vehicle and went through the same thing all over again. Needless to say, had I relied on the "expertise" of the mechanics, I'd have paid more than I needed to, several times over. I never went back to that shop again, and from that point on, I made sure to check everything on the list of services performed before paying the bill.
Like many, I own a home, and there are a number of reasons for me to seek services...HVAC, as well as other specialists (particularly as a result of Super Storm Sandy). I tend to follow the same sort of path with my home that I do with my vehicles...small stuff that I can do myself, I do. Larger stuff that requires more specialized work, I want to bring in someone I know and trust. I'm a computer nerd...I'm not an expert in automobile design, nor am I an expert in home design and maintenance. I can write code to parse Registry data and shell items, but I am not an expert in building codes.
So, the question I have for you, reader, is this...how do you know that you're getting quality work? To Larry's point, who are you hiring to perform the work?
At the first SANS Forensic Summit, I was on a panel with a number of the big names in DFIR, several of whom are SANS instructors. One of the questions that was asked was, "what qualities do you look for in someone you're looking to hire to do DFIR work?" At the time, my response was simply, "what did they do last week?" My point was, are you going to hire someone to do DFIR work, if last week they'd done a PCI assessment and the week prior to that, they'd performed a pen test? Or would you be more likely to hire someone who does DFIR work all the time? I stand by that response, but would add other qualifications to it. For example, how "tied in" are the examiners? Do they simply rely on the training they received at the beginning of their careers, or do they continually progress in their knowledge and education? Do they seek professional improvement and continuing education? More importantly, do they use it? Maybe the big question is not so much that the examiners do these things, but do their managers require that the examiners do these things, and make them part of performance evaluations?
Are you being served?
Addendum: Why does any of this matter? So what? Well, something to consider is, what will a CEO be reporting to the board, as well as to the SEC? Will the report state, "nothing found", or worse, will the report be speculation of a "browser drive-by"? In my experience, most regulatory organizations want to know the root cause of an issue (such as a compromise or data leakage)...they don't want a laundry list of what the issue could have been.
In addition, consider the costs associated with PCI (or any other sensitive information) data theft; if an organization is compromised, and they hire the local computer repair shop to perform the "investigation", what happens when PCI data is discovered to be involved, or potentially involved? Well, you have to go pay for the investigation all over again, only this time it's after someone else has come in an "investigated", and this is going to have a potentially negative effect on the final report. I think plumbers have a special fee for helping folks who have already tried to "fix" something themselves. ;-)
Look at the services that you currently have in your business. Benefits management. Management of a retirement plan. Payroll. Do you go out every month and select the lowest bidder to provide these services? Why treat the information security posture of the your organization this way?