Posted by Tate Hansen
Mon, 05 May 2008 04:45:00 GMT
This from the new
PCI information supplement: (regarding the required annual penetration testing for compliance)
The penetration tests should attempt to exploit vulnerabilities […] attempting to penetrate both at the network level and key applications
Really? I laughed when I read this, seriously. It made me think for a second about how many consultants really have the skills to chef-boy-ar-dee exploits under pressure. It’s clear too; this is not about a vulnerability sweep, they want you to bust in.
Penetration testing [..] should occur from both outside the network trying to come in (external testing) and from inside the network.
Wow. True penetration testing from inside the network? How many internal networks have you seen that would survive a blitzkrieg attack from a good penetration test team?
PCI states:
“resources must be experienced penetration testers”
What does that mean?
I’m sure the PCI council is of compos mentis, and I’m not trying to rain on the PCI council or ASVs or QSAs, though it’s funny the council points out that “The PCI DSS does not require that a QSA or ASV perform the penetration test”. That statement wouldn’t be because most of them couldn’t penetration test there way out of a paper bag even if they were handed a loaded metasploit gun, right?
With the huge number of companies bemoaning PCI compliance, I just don’t see most getting a true penetration test. I guess I could be reading too much into this. Maybe the skills bar level I consider for experienced penetration testers is way higher than what the PCI council considers experienced or what others consider experienced or good?
Do you have penetration testing skills? What does that mean to you? Do you think most of the companies that buy a penetration test actually get one?
Tags ASV, ClearNet, ClearNet Security, exploits, PCI, Penetration Testing, QSA, security, Tate Hansen, vulnerability | 3 comments
Posted by Tate Hansen
Wed, 07 Nov 2007 01:31:00 GMT
Today I was talking with a colleague from a partner company about the PCI certification - I think he's up for recertification.
The interesting thing is he was talking to a Qualys representative recently whom, affably speaking, offered tips on how to tune the Qualys scans based on new modifications made at Mastercard's test lab. The representative also said he could review the report Qualys automatically builds. My colleague exclaimed to me "It sounded like they already have the answers".
Of course they do. Qualys pays PCI to verify their ability to discover what PCI wants them to discover. People pay and use Qualys so they can become PCI certified. Anybody willing to click "start scan" has the ability to be an Approved Scanning Vendor.
What's my problem with all this? For one, the certification process is rotten:
http://blog.clearnetsec.com/articles/2007/05/16/pci-not-our-problem.
http://blog.clearnetsec.com/articles/2007/05/04/pci-misleading-racket
On top of that, it's costly and does little to vet engineers claiming competency. Its design is to weed out small security firms, which is probably why it fires me up in the first place and turns me into a cynical punk all day.
Tags ASV, Certification, ClearNet, ClearNet Security, PCI, Tate Hansen, visa | no comments
Posted by Tate Hansen
Thu, 17 May 2007 02:51:00 GMT
What happens when the test environment operated by MasterCard (they “own” the testing lab) is misbehaving? I know. They yank the wheel, swerve away from responsibility, and point to the PCI council. And PCI? They point back. Beautiful, no?
You see because they refuse to disclose missed results to you they duck responsibility for anything that may have been their fault. They also clearly imply if anything is missed in your attempts to identify vulnerabilities then it is surely your fault or a problem with the tools you used.
I love it: No clear pass criteria, no way to challenge a decision, and no transparency of what or how they are doing. For all this great service you get to spend thousands every year!
So what happens when you call bullshit and raise hell? They pass you. :) Let me not forget to mention we had a few extra bullets in our clip they may have unexpected us to have – bullets provided to us by friends with information.
Be forewarned; this process has serious issues.
Tags ASV, cisp, ClearNet, ClearNet Security, mastercard, PCI, scanning, security, Tate Hansen, testing, visa, vulnerability | 1 comment
Posted by Tate Hansen
Fri, 04 May 2007 17:52:00 GMT
The world of the PCI Security Standards, ASVs (Approved Scanning Vendors), and commercial scan vendors is, from my limited interactions, not exactly on the straight.
Having recently spent considerable time preparing, scanning, and writing reports with the explicit goal of becoming an ASV, I’m disturbed by my communications with all involved.
It doesn’t help that the pass criteria to become an ASV is not clear. Is it based on discovering all vulnerabilities on their test network? A subset? Which parts are subjectively reviewed?
I like to use Qualys to baseline vulnerabilities, which a test representative caught as one of the tools we’d be using based on the source IP blocks for scanning. He said something like “If you use Qualys, you’ll get 95% of what you need”. From that I guessed the example web application would have vulnerabilities which would be missed by Qualys and other network-based tools. As expected, that was true.
Fast forward. We received feedback that our reports have not been reviewed because Qualys changed something recently causing expected results to be absent which PCI requires for passing. The representative said “If you’d have scanned a month ago with Qualys, you’d have passed with flying colors”. He added that they are in communication with Qualys to resolve the issue.
Nevermind that Qualys was only one of the tools we used and we had added vulnerabilities not discovered by the popular free or commercial scanners. It was clear the representative didn’t review the report, which he said as well (and may have done to protect us from an automatic failure -- even though our current pass status is pending). But they would not reveal the gaps, which obviously makes it hard to understand what the problem is. I appealed by mentioning I had added vulnerabilities not discovered by Qualys. He then modified his previous statements by saying he in fact spot checked the reports and the items he was looking for were absent. My gut reaction to all this: bullshit.
The connections apparent make for a nice racket. You pay lots of money to PCI. PCI “communicates” with selective vendors to ensure all the vulnerabilities they expect to be discovered are discovered. You pay money to the scan vendors. And what about if you find additional vulnerabilities or a superset? Sounds like they don’t check and don’t care. I would think the idea behind all this is to make sure you are adding sufficient value to entities subject to PCI regulations (even if that means you didn't catch 100% of everything bad). If passing simply means doing a blind Qualys scan when it works right (i.e. does what PCI wants), well then, you now know what to do to as an attacker -- just go after something Qualys doesn't check.
So much for trusting this process and what it does to vet competent assessment companies.
Tags ASV, ClearNet, ClearNet Security, PCI, Qualys, security, Tate Hansen | 1 comment
