Posted by Tate Hansen
Mon, 05 May 2008 04:45:00 GMT
This from the new
PCI information supplement: (regarding the required annual penetration testing for compliance)
The penetration tests should attempt to exploit vulnerabilities […] attempting to penetrate both at the network level and key applications
Really? I laughed when I read this, seriously. It made me think for a second about how many consultants really have the skills to chef-boy-ar-dee exploits under pressure. It’s clear too; this is not about a vulnerability sweep, they want you to bust in.
Penetration testing [..] should occur from both outside the network trying to come in (external testing) and from inside the network.
Wow. True penetration testing from inside the network? How many internal networks have you seen that would survive a blitzkrieg attack from a good penetration test team?
PCI states:
“resources must be experienced penetration testers”
What does that mean?
I’m sure the PCI council is of compos mentis, and I’m not trying to rain on the PCI council or ASVs or QSAs, though it’s funny the council points out that “The PCI DSS does not require that a QSA or ASV perform the penetration test”. That statement wouldn’t be because most of them couldn’t penetration test there way out of a paper bag even if they were handed a loaded metasploit gun, right?
With the huge number of companies bemoaning PCI compliance, I just don’t see most getting a true penetration test. I guess I could be reading too much into this. Maybe the skills bar level I consider for experienced penetration testers is way higher than what the PCI council considers experienced or what others consider experienced or good?
Do you have penetration testing skills? What does that mean to you? Do you think most of the companies that buy a penetration test actually get one?
Tags ASV, ClearNet, ClearNet Security, exploits, PCI, Penetration Testing, QSA, security, Tate Hansen, vulnerability | 3 comments
Posted by Tate Hansen
Mon, 24 Sep 2007 20:24:00 GMT
I can’t resist rehashing this topic.
Why not, after performing a penetration test, ask the customer what they observed? And when doing so, using commensurate effort to clearly surface the importance and point from asking that question.
So many are ordering up vulnerability and penetration tests like it’s a quick immunization shot against being hacked. They may get a wee bit nauseous after viewing their vulnerability report, but once they mitigate and recover they feel invulnerable, however temporary.
Penetration testing proves nothing of invulnerability, and it naturally follows, ad nauseam, to ask, again, what does it really prove then? Marcus Ranum summarizes Gary McGraw’s answer to that:
Gary McGraw likes to refer to pen testing as the "badness-o-meter" - it's a test that registers, at one end of the dial "your network sucks" and at the other end, "we don't know."
If the “We don’t know” club doesn’t already list as members all penetration testers, it surely should. No one has the knowledge or possession of everything needed to break all systems, so everyone is playing their own part in the “don’t know”.
Even if you are named Mr. Exploiter and pay cash to subscribe to every 0day exploit factory, the value of showing your customer they get owned when you call an 0day blitz is not only from proving you can sprint past their defense. Anyone with an exploit can do that.
The value rises from the conversation you have with the customer after the party. Why should the customer care if the latest 0day worked? What was the customer to do about it anyway?
Companies whom take their security seriously need to report back to you. Instead of hearing the customer ask you, “What did you find?”, you should ask the customer, “What did you observe?”.
Imagine how shocked you’d be to listen to a customer detail all of your efforts during a penetration test: They report to you which systems you attacked, when and how, and what information you had obtained.
That would be good security. Penetration testing should move away from “I got you with this 0day” to “You identified 90% of my efforts to compromise your systems”.
Flipping the reporting paints a clearer picture of the overall security of your customer: Those that do well you could posit have good policies in place and have built respectable awareness and response capabilities.
Tags assessment, ClearNet, ClearNet Security, penetration, security, Tate Hansen, testing, vulnerability | no comments
Posted by Tate Hansen
Thu, 17 May 2007 02:51:00 GMT
What happens when the test environment operated by MasterCard (they “own” the testing lab) is misbehaving? I know. They yank the wheel, swerve away from responsibility, and point to the PCI council. And PCI? They point back. Beautiful, no?
You see because they refuse to disclose missed results to you they duck responsibility for anything that may have been their fault. They also clearly imply if anything is missed in your attempts to identify vulnerabilities then it is surely your fault or a problem with the tools you used.
I love it: No clear pass criteria, no way to challenge a decision, and no transparency of what or how they are doing. For all this great service you get to spend thousands every year!
So what happens when you call bullshit and raise hell? They pass you. :) Let me not forget to mention we had a few extra bullets in our clip they may have unexpected us to have – bullets provided to us by friends with information.
Be forewarned; this process has serious issues.
Tags ASV, cisp, ClearNet, ClearNet Security, mastercard, PCI, scanning, security, Tate Hansen, testing, visa, vulnerability | 1 comment
Posted by Tate Hansen
Sun, 07 Jan 2007 20:44:00 GMT
Along with others I’m sure, we’re seeing a nascent phase for security services firms in the business of performing external security assessments. Rather than blast through the battery of scanners the game is changing to include tasks which traditionally fell outside the scope of a basic external assessment.
I like the changes because it follows more of a risk assessment approach. For example, as part of an “external assessment pack” we might obtain the configurations for routers, firewalls, switches, wireless access points, concentrators, review relevant policies, architecture diagrams, and to the best of the client’s ability incorporate their identification of segments or servers which touch sensitive data.
Contrary to acquiring infrastructure visibility, running blind external assessments are losing effectiveness - especially with the growth of the commercial exploit factories (e.g. argeniss, gleg, core security, immunity sec ). Making a statement like “We scanned with Nmap, Nessus, Qualys, Webinspect, checked Secunia, SF, etc., and found no major findings” is losing value - its assurance offered more yesterday than today.
My participation in a few recent incident response/forensic assignments is testament to the weakening value of blind assessments; the cause of each compromise would not have been revealed as a vulnerability by vulnerability scanners.
Tags ClearNet Security, external assessment, security, security assessments, Tate Hansen, vulnerability | no comments
Posted by Tate Hansen
Tue, 19 Sep 2006 21:21:00 GMT
When competing for security assessment projects it is often painful for the customer to distinguish the level of service or effort between proposals. We used to respond to RFPs with the intention of satisfying all the services the customer is soliciting – of course in the end in nearly every case that isn’t what wins the bid.
We came up with a quick flow diagram to illustrate the differences in the level of effort between network-based security assessments. This has helped us tremendously with clients and with keeping the playing field level. It’s not complete or exact by any means, but it works.
We add some verbiage to help customers relate it to real world:
Sample attacker profile:
Basic: Attacker spending minimal effort; downloading free 'hacking' tools and running them with minimal attention
Intermediate: A motivated attacker spending more time and resources with greater attention to detail and actively searching for a weakness
Advanced: A serious attacker with intent to harm or steal information assests
Security assurance profile:
Basic: Minimal; relies on a limited set of tools to discover weaknesses
Intermediate: Good; relies on running many tools with overlapping functions, specialty tools, tuned for bandwidth and latency conditions, and includes manual investigation, validation, and research into findings
Advanced: Excellent; goes beyond Intermediate to prove the existence of vulnerabilities, includes checking non-public domains for the existence of 0-day exploits
Tags Audit, ClearNet Security, penetration test, scan, security, security assessments, Tate Hansen, vulnerability | no comments
Posted by Tate Hansen
Sun, 22 Jan 2006 18:11:00 GMT
I wish that was true, but it is furthest from the truth. There is an unfortunate conception for many tasked with reviewing or choosing a scanner (a conception espoused by the marketing of several Vulnerability Assessment vendors) which is the quality of a scanner is directly based on how fast it can scan. In this race for speed, vendors’ almost always default to performing less than complete checks. In fact, some vendors shy away from adding checks to their products in an attempt to retain speed (i.e. adding more checks will slow a scanner down). That is probably not what you want. I also have yet to see a single commercial scanner default to performing a full TCP port check. Totally understandable; if a scanner’s default policy opted for full coverage then you would surely dismiss the quality after running a test scan and learning you have to wait 40 hours for the results. Too bad. In the world of network-based vulnerability scanners there is a trade-off that spans long: speed and accuracy. Speed kills accuracy.
“Watch this man, I can scan a class B in 10 seconds.”
“Really? How did you do that?”
“Oh, well, it is only checking if one port is open.”
“Ah, nice.” |
 |
So if you want accuracy, then slow it down. Don’t run a thousand threads, allow more time for remote devices to respond, choose complete port coverage, and don’t parallelize so much you saturate switch ports or test your operating system's TCP/IP stack limitations.
Tags ClearNet Security, scanning, Tate Hansen, vulnerability, vulnerability scanning | no comments
