Posted by Cory Stoker
Tue, 22 Aug 2006 23:42:00 GMT
In part 2 of my VA auditing experience I told you all about our "training" for the VA assessment. I am going to finish this out with my thoughts on the first site experience. If you missed it here is part 1. With all the things that had gone on with this project I was very interested in how the actual audit was going to go for each site. Before I could think long on it I was off to the wonderful state of Maine in February.
Now I live in Colorado and most people's preconception of Colorado in the winter is exactly what Maine was... Cold, snowy, and dark. For those of you that don't know, Denver Colorado has a very mild winter and snow barely stays a week on the ground. In the mountains is a different story but Denver is on the plains not the mountains.
So back in Virginia we were told that we needed to car pool with the other auditors and that each auditor was responsible for ensuring the whole team got to the site. This was interesting to say the least as the audit teams were thrown together maybe 2 days before we actually flew out. Each trip I went on had a team with different people. This fact was great for meeting new people but horrible for car pooling as the one person who had the car was expected to ferry us around! Now the issue that greeted me first was that I got to Portland, Maine at about 11:00 PM EST and had to get to Augusta which is about 1 1/2 hours away. Trying to get ahold of the guy with the car did not happen as it went to VM suprisingly enough. Suffice to say I had to take a taxi to Augusta which costs about 170 dollars, footed by the tax payers of course. For people that don't know Maine, Portland is in the south and Augusta, the capital, is in the lower center of the state so a taxi ride was costly.
The second issue was that none of the audit staff could get ahold of each other. In fact I didn't even get to the facility till later on Monday cause we all were staying at different hotels. Hotels, flights, and rental cars were chosen by the coordinators not the auditors so this was not negotiable. Anyhow we were scheduled to be at the facility for 4 days and leaving the 5th day so I was already thinking of how much fun I was going to have.
Onto Monday we go! After I get to the facility with my chauffeur. I finally find out how many computers we are testing. Lets see the audit team had 3 "windows testers" including me so that means we can get pretty good coverage in 4 days right? Well we had to test a grand total of 26 computers and all the mobile nursing stations for a grand total of 30. Remember the checklist, the one that takes about 20 minutes per computer max? 30 / 3 = 10 computers over 4 days. So doing some more math we can estimate about a 4 hour work day including lunch. Now this facility was pretty big. So big that I would have easily gotten lost without my VA companion. Off I went to verify the VA is secure with my clipboard! Suffice to say that my VA companion was pleased to only waste 4 hours running MBSA and Dumpsec.
At this point I am sure a few of you are thinking that it was easier for me to test this minuscule amount of computers and then just chill till it is time to leave but it wasn't. We were not allowed to have cell phones on in the building because of possible interference with medical equipment, we were not allowed to go onto the VA network with our laptops, which makes sense, and we were in the middle of nowhere. Luckily we got to go home on the 3rd day meaning that we had only spent 4 days total in snowy Maine.
A few thoughts on my whole VA auditing experience. First, I did actually like meeting the other auditors and the technical VA personnel. They were great and made the whole project actually move forward. I also got to go to places I would never have gone to if not on business. What a waste of money the whole endeavor was. As Bruce Schneier likes to always say, this definitely had the perception of being a proactive security measure but that is all it was, a perception. I think that there were some serious loopholes somewhere that allows this sort of thing to go on. Like I said earlier, if this kind of project happened elsewhere everyone would be fired, unless of course they are interested in the perception. We ended up doing 10 facilities before we just could not take it anymore. We were not alone in that feeling as I think every team I was on had people that were new who had replaced someone that went to the "training".
Tags Accreditation, Audit, C and A, Certification, ClearNet Security, Cory Stoker, security, VA, Veterans Affairs | 1 comment | no trackbacks
Posted by Cory Stoker
Sat, 29 Jul 2006 20:58:00 GMT
OK so I guess I have touched a nerve with this subject as our traffic to our blog has spiked since Richard Bejtlich's blog linked to my "VA and Bureaucracy" post. As one to not let a good story go to waste I will finish the story in 2 more parts before I leave for BlackHat 2006.
As you remember from my previous post on the subject Tate and I were part of a large team of people contracted to go audit the VA computer networks and systems at every VA facility in the country. We had thought that we would be working with other individuals of our technical caliber on a comprehensive audit process that follows along with the NIST SP 800 series of documents. As we flew from Colorado to Virginia we had some expectations of this project that were brutally shattered in the coming days.
Before the trip our expertise had been doing security in the corporate space, i.e. a company would hire us to conduct a penetration test or a vulnerability assessment, etc. The government space with its money capital and processes at its disposal must be better, at least in my mind. However, it quickly became apparent that what we were really tasked to do at the VA would get us fired at any of our other private corporate space clients for negligence.
One note about the ethics of what we could do in this particular situation. Two things:
- First is that in the initial meetings with VA respresentatives there was spirited push back on the VA and
contract companies that this whole thing was just not right. In fact I think that many people just gave up after
the introductory meetings because no one was listening.
- Second is that we stayed on the project at this point mostly because we just couldn't forsee that it could
be as bad as it turned out to be. We were always looking for the gotcha that would dispel the myth and make
the project make sense.
And on with the story...
So Tate and I were flabbergasted at the first meeting with the VA but we were at least optimistic on what the next day would hold as we were being trained on the specific audit procedures for each technical area we would be qualified to test in. The technical areas we were going to test were Windows, network, and policy. So the next day bright and early we had to report to the main office of the chief company controlling all the audit teams.
First up for us was Windows testing. We had a lot of ideas of what we would want to hear, like which scanners were going to be run, what tools to follow up results with and what kind of forensic analysis would happen if a computer was exploited, infected, or warez'ed. Well basically a checklist was handed out and a so-called “trainer” read through the procedure. It went something like this:
- Write down info about computer like name, room location, date, OS installed.
- Run MBSA.
- Dump Registry.
- Dump users and groups
- Dump logs if any are even there
- Take a screenshot of the screensaver properties
Gee that sure is comprehensive huh? At least it is super expensive so it must be good. Basically our job as high paid and trained security professionals was to dictate step-by-step procedures (click here, click there, click save-as, etc.) to a VA employee while shoulder surfing. Then after they completed a step we would check it off along with the time it took to run it. The hardest part would be to get the room number and address of the computer we were on as a lot of the VA facilities did not label every room.
Right after our "training" a person asked how many computers we would have to do this to at each facility. The answer was a sampling and possibly all the Windows Servers. Later on at my first facility I went to I tested 10 computers at a VA facility that was about a ~1000 computer facility. I will tell more detail on this in my next entry.
Then another person in the room brought up scripting, "Hey you could write a script that could be run on logon or log off to grab these results from every computer in the facility"
The trainer replied "Scripting is not allowed because it is too dangerous; it could bring down a critical computer"
"OK then why not just leave the critical computers out and do those by hand and leave the non-critical computers in the script"
"No. No scripting can be done as was agreed earlier."
That was the end of that. No scripting because it is too dangerous. The network training was basically the same thing but added in that architecture was not to be looked at. For example if a facility left their network on the Internet with no firewall, it was not to be noted. Just stay to the checklist, don’t look left or right.
At this point we were seriously considering dropping off the project but we decided to give it a shot and remain open-minded. But I can tell you it was hard. I mean if I saw by happenstance (and I am not saying I did) that a computer was running a warez site, if it wasn't caught by the checklist then according to the VA’s audit procedures, it was OK. Again concerns were raised to the company we were contracted under and I believe they had sent it up the ladder but I never heard anything. The checklists were even revised multiple times because many people still had a hard time following it step by step, but the revisions they made never really meant much with respect to security.
In the next part I will talk about my first experience at a VA facility - as a screenshot properties collector, err I mean security auditor.
Tags Accreditation, Audit, C and A, Certification, ClearNet Security, Cory Stoker, security, VA, Veterans Affairs | no comments
Posted by Cory Stoker
Fri, 09 Jun 2006 06:52:00 GMT
So it has been awhile since I blogged. Sorry! So anyways I have been following the security breach that happened to the Veterans Affairs (VA) with interest. For those of that do not know, basically a VA worker had been taking veterans data like SSN and name etc, home so that he could work on his project from home. What happened was his laptop with USB drive got stolen from his house and the VA data went with it. No one knows what if anything happened to the data but it does leave millions of veterans open to identity theft. More info here.
This is near and dear to my heart as it was one of the first projects that Tate and I worked on. The project was at the time the largest Certification and Accreditation (C&A) process for the federal government was happening at the VA. Tate and I jumped onto a contract with a company that had head count and we were off to Virginia for training. Now for those of you that do not know, the C&A process is very large and detailed. It is created and kept by the National Institute of Science and Technology (NIST) and is the process all federal agencies need to follow to be compliant. The documents themselves are actually really well written and freely available. Basically the C&A process is summed up as this: develop a policy, test against it, determine risk points, and then remediation plans. The certification part is where the auditors, audit against the policy and the standards set by the C&A documents. The accreditation part is where the big hunchos of the agency either accept the risk and keep their IT going or stop it until the risk is remediated. This process was what we were "thinking" we were getting into. At this point we did not have C&A experience so this was worth it for us.
Ok.... so we get to Virginia and start what we thought was going to be some hard security work. In fact the company we were working under thought that our skill sets might not be up to par enough.... We had to go to a meeting with all the auditors and the VA staff where they were going to let us in on the work involved and this is where we had our first exasperating moment on this project. The main person involved on the VA side stands up and tells us this is the biggest C&A process ever and blah, blah......Oh yeah, no one other than VA personnel is able to touch ANY computer either physically or virtually! Wait a sec! I still remember the whole crowd of 200 or so auditors all collectively looking around and I think some people in the back row made a run for it at this point. Everyone was thinking exactly what you are thinking at this moment, how can you test "technical controls" without actually testing... Well they came up with the answer, which was to pair us up with our very own set of VA hands, still attached to a VA employee at each site we visit. Yup, now instead of us actually typing and testing a computer we were supposed to relay commands to a VA staff person and they would type it in! Sweet, I can give my carpal tunnel a rest and set my jaw wagging. I can just see it now, "Oh look, it looks like this computer has some malware. Click here, load this tool, select this hex field and check the registry....NO NOT THAT KEY! Run!"
Right about this time the second bomb shell went off.... The guy up front promptly says that all test results we collect are to be given to the VA. This makes sense as it is their computers and they are entitled to our analyzed results right? Wrong! The guy corrects himself and says that the results are not to be analyzed by the auditors but by VA personnel. Hmm...so at this point I am not touching a computer nor am I analyzing the results for risk or what is wrong. Something seems very broken about this process at this point.
In the next part I will explain the next day and our first site experience. In reading this I am sure you are now not surprised to hear about data breaches and lameness on the part of the VA. After all they pretty much subverted the C&A process to insure they pass.
Tags Accreditation, Audit, C and A, Certification, ClearNet Security, Cory Stoker, security, VA, Veterans Affairs | no comments
