True penetration testing?

Posted by Tate Hansen Mon, 05 May 2008 04:45:00 GMT

This from the new PCI information supplement: (regarding the required annual penetration testing for compliance)

The penetration tests should attempt to exploit vulnerabilities […] attempting to penetrate both at the network level and key applications

Really? I laughed when I read this, seriously. It made me think for a second about how many consultants really have the skills to chef-boy-ar-dee exploits under pressure. It’s clear too; this is not about a vulnerability sweep, they want you to bust in.

Penetration testing [..] should occur from both outside the network trying to come in (external testing) and from inside the network.

Wow. True penetration testing from inside the network? How many internal networks have you seen that would survive a blitzkrieg attack from a good penetration test team?

PCI states:
“resources must be experienced penetration testers”

What does that mean?

I’m sure the PCI council is of compos mentis, and I’m not trying to rain on the PCI council or ASVs or QSAs, though it’s funny the council points out that “The PCI DSS does not require that a QSA or ASV perform the penetration test”. That statement wouldn’t be because most of them couldn’t penetration test there way out of a paper bag even if they were handed a loaded metasploit gun, right?

With the huge number of companies bemoaning PCI compliance, I just don’t see most getting a true penetration test. I guess I could be reading too much into this. Maybe the skills bar level I consider for experienced penetration testers is way higher than what the PCI council considers experienced or what others consider experienced or good?

Do you have penetration testing skills? What does that mean to you? Do you think most of the companies that buy a penetration test actually get one?

Tags , , , , , , , , ,  | 3 comments

Predictive markets & betting on when apps or companies get owned

Posted by Tate Hansen Thu, 01 May 2008 05:30:00 GMT

A recent WSJ article titled “Trading on the Wisdom of Crowds” sparked my interest as it may relate to security. Are there ways to build a business around helping organizations understand the risk to their data assets by using predictive market models? Or maybe building it around betting on commercial applications?

“Betting odds are generally taken as the best indicator of probable results in presidential campaigns," this newspaper explained in 1924.

I’m placing a bet that retail store XYZ gets owned and reveals grandma’s credit card details in ‘08. I’m placing another bet that application ABC will have a remote admin level vulnerability by October ’08.

Alas, we must have more transparency and trust in the publicly disclosed information to play. Participation is key as well:

Predicting markets seem to work so long as there are enough traders whose aggregate information is fully reflected in bets.

Would enough people find it worthwhile to become active traders? Maybe. There was an active predictive market created around the following question:

What will the government's 2007 computer security grade be?

It’s probably a big stretch to build a successful predictive market business around the types of security bets which would benefit organizations. By that I mean if I was responsible for a commercial application in which 75% of the traders were betting on my application being owned within the year, I’d probably work hard to change the odds (i.e. allocate resources to improving the security of my app).

Tags , , , ,  | no comments

When virtual servers play havoc

Posted by Tate Hansen Mon, 14 Apr 2008 21:19:00 GMT

I recorded a tidbit which came from a comment spoken at one of this year's RSA panel tracks. I hadn't thought of this issue on a big scale. It was a comment on how disruptive an environment which frequently "resets" virtual servers as part of normal business is to security.

It's obvious such an environment can have a significant impact on security tools, especially those which strive to learn patterns or look at history or both.

I was just imagining if I was a security admin responsible for a large block of EC2 virtual servers. As part of that, maybe the use of these blocks of servers is similar to a class lab whereby students get to install and do anything they want on the servers. When they're done, the instructor runs around and resets all the servers. Extrapolate this and it can lead to a hard problem, security speaking, for general cases.

I haven't meditated on this issue, but I'm guessing it'll become more visible in short time.

Tags , , , , ,  | no comments

Test commercial web app scanners for free and without restrictions?

Posted by Tate Hansen Mon, 24 Mar 2008 17:54:00 GMT

If your software licensing ethics tend to contort a tad here and there, then you may find the below tricks helpful when you want to evaluate commercial web app scanners. Partaking in these tricks is slippery, and you may fall into ethical perdition, so prepare yourself!

Super simple trick #1: Request search and replace proxy

Fire up a proxy that supports request search and replace.

suru

Let’s say an app restricts you to scan only their target site (e.g. demo.testsite.net), but you want to point the scanner to a different target.

No problema.

As shown above, I typed in demo.testsite.net for Search, and blog.clearnetsec.com for Replace. Every HTTP request passing through this proxy with a Request-URI matching the string demo.testsite.net will get replaced with blog.clearnetsec.com. The result? The app scans my blog.

Note: Super simple trick #1 only works for apps which restrict via the hostname. If the app is smarter and adds IP validation, then move along to Sort of simple trick #2.

Sort of simple trick #2: IPTables magic

This trick can add license evading umpf to your smokin’ renegade style. :)

Let’s say the app tries to validate you are only scanning the sites you're licensed for by checking the target IP addresses (regardless of how the hostnames are resolved). For example, maybe the demo version of the app allows you to only scan IP address 55.55.55.55.

No problema.

One line explanation: Setup a linux box to do routing, configure a VIP, and add two IPTable NAT rules.

The long answer.

Setup a linux box. A linux VMware image works too. Configure the network as normal – give it a standard IP, gateway, list of name servers, etc., as you would when configuring any other box on your local subnet.

Once you got that, then check out the script below.

cli

Follow? If not, I’ll explain in more detail.

  • Again, the first thing you need is a normal networked working linux box.
  • There are 4 steps to facilitate a destination IP switch-a-roo:
    • Configure a Virtual IP. Pick an IP located on a different subnet, don’t pick an IP on the same subnet as the primary IP.
    • Enable IP forwarding.
    • Setup a DNAT (Destination NAT) rule to replace the destination IP on the fly. The first IP (e.g. 55.55.55.55) is the licensed locked IP. The second IP is (e.g. 216.241.191.205) is the IP you want to scan.
    • Setup a SNAT (Source NAT) rule to replace the source IP on the fly. The first IP (e.g. 192.168.1.101) is your workstation IP. The second IP is the primary IP address of the linux box.

The last step there, #4, is overloaded. Once you go through the steps above on the linux box, then you need to change the IP address of your workstation with the app scanner installed. You want to pick a new IP for your workstation which is on the same subnet as the VIP you configured on the linux box. You also want to change your workstation to use the linux box as your gateway to the Internet, so change the default route address to match the IP of the VIP on the linux box.

That should do it. Replace the IPs in the IPTable rules above with the IPs that work for you and scan away.

Note: You can’t always do the IPTables trick by itself; one reason is due to virtual hosting. If only one website is being hosted on the IP, then you probably can do this. If the target IP is hosting lots of domains, then you need to chain the request and replace proxy with the IPTables magic to ensure each Request-URI is for the correct host and domain.

For example, GET http://216.241.191.205/... HTTP/1.1 may not be the same as GET http://blog.clearnetsec.com/... HTTP/1.1

Super Simple trick #3: VMware snapshots

Most everyone is likely familiar with this trick. If you have a web app installed in a VMware image and you have a working license (e.g. trial license), but it expires at a certain time and date, the trick is to create a snapshot of the VMware image with the app in a working state.

Anytime you want to scan, change your host OS clock back to a time that is within the licensing window (or ensure your VMware guest image doesn’t sync the clock with the host OS when you restore the snapshot), and restore the snapshot.

Note: Some apps may call home when first launched, so it helps to create the VM snapshot when you have the app open and ready to scan.

Trick addendum:

For those web app scanners which restrict you from scanning SSL enabled sites, have no hesistation, you can work around that too.

One way is via stunnel. From www.stunnel.org.

Encrypted version with STUNNEL

+---------+      | |     +--------+    +---------+
| non-SSL | -ST- | | --- | Apache | -- | non-SSL |
| enabled |      | |     | WITH   |    | enabled |
| client  |      | |     | SSL    |    | server  |
+---------+      | |     +--------+    +---------+
   CLIENT        NET     WEB SERVER      SERVICE
Note the position of STUNNEL : the "-ST-" in the diagram above. Below is an example stunnel configuration:

client=yes
verify=0
[psuedo-https]
accept = 8080
connect = blog.clearnetsec.com:443
TIMEOUTclose=0

Configure your app or browser to use the stunnel proxy listening on port 8080 and you'll be able to hit the site using HTTPS (via the proxy), but your local app or browser will be only speaking HTTP.

Quick alternate to #1 for Apache fan boys and girls:

Via Apache 2.2.x, with mod proxy and mod rewrite enabled, setup a proxy like so:

ProxyRequests On
<Proxy *>
RewriteEngine on
RewriteRule ^(.+) http://blog.clearnetsec.com/$1 [P] (or something close to this)
Order deny,allow
Deny from all
Allow from all
</Proxy>

Configure your browser or app scanner to use this Apache proxy server and all Request-URIs passed through will be re- written to target http://blog.clearnetsec.com/.

YMMV. Have fun.

Tags , , , , , , , , , , , , , ,  | 2 comments

Rogue modems are still plentiful?

Posted by Tate Hansen Sat, 16 Feb 2008 05:18:00 GMT

I was doing a scope call for a large grocery chain recently when they mentioned they discover around 20 rogue modems per quarter per division (and they have more than 12 divisions). That number is way higher than I would've guessed, though maybe lots of the modems are legitimate but not on their official roster.

Whatever the case may be, wardialing is not a moribund activity, or as close to it as I thought.

Tags , , , , , , ,  | 2 comments

The booming exploit market and bye bye to swaths of products

Posted by Tate Hansen Fri, 01 Feb 2008 06:50:00 GMT

There are lots of articles mentioning the Digital Armaments bounty for exploits. I wrote a snippet on the commercial exploit market about a month ago, whereby I was simply listing the prices for subscribing to the different exploit houses.

I guess I forgot to consider another complexity of all this and that is from the influence the organizations who compete to purchase exploits are having (e.g. iDefense, 3COM/TippingPoint, Governments, people and groups w/lots of money).

I wonder how extensive this really goes – I mean, it seems this market is in a boom of sorts which implies there are lots of private exploits trading hands. Exactly how many would be interesting to know. Hell, any numbers would be nice.

One thing is apparent though, if this market continues to grow then how can any security products based on “knowing attacks” succeed? They won't. An IDS vendor is not going to be able to afford to purchase all; no company will have a monopoly.

Tags , , , , , , ,  | no comments

“Big money! Big prizes! I love it!”

Posted by Tate Hansen Fri, 28 Dec 2007 15:57:00 GMT

Smash TV quotes. Love ‘em.

Speaking of big money, the commercial exploit market’s growth isn’t making it any easier to bid on penetration test gigs. If you want to provide the highest assurance you’re capable of to clients, then of course you would like to have your hands on all the exploits out there, both public and private.

product to start quarterly total
d2 $1,950 $850 $5,350
gleg $1,400 $700 $4,200
argeniss $1,000 $500 $3,000
canvas $1,450 $730 $4,370

And the crème of the crop:
Immunity Sec’s Vulnerability Sharing Club $50,000 - $100,000 per year

Attacking with anything less in hand tends toward negligence, especially if you do so without disclosing what you’re missing. Pay to have all and you’ve likely priced yourself out of competitive bids.

The winners here, again, are the attackers.

“Good Luck… you’ll need it!”


Tags , , , , , , , , ,  | 2 comments

Follow-up on using unicornscan for a big scan (400,000+ public IPs)

Posted by Tate Hansen Thu, 27 Dec 2007 19:36:00 GMT

I’m happy to report our growing experience using unicornscan for large discovery sweeps is a positive one. Our confidence in using this tool has increased and it is now our preferred weapon of choice for scanning large IP swaths.

To recap: We performed a sweep of 400,000+ public IPs across multiple continents by configuring the scans to do a full TCP port scan of each IP, sustained ~55 Mbits/s using between 3 and 5 systems, and completed it in a matter of days.

This is pretty good considering by sending two SYN probes per port it meant sending ~52.5 billion packets and producing some 3 Terabytes of data.

Nmap is often our preferred tool, and we used it to spot check our results with unicornscan, but from now on it will come down to the details of the gig to make the choice.

Tech note: We avoided problems with table overflows and other like issues by placing the systems directly on the internet and with iptables turned off.

Tags , , , , , , ,  | no comments

Another tidbit on PCI

Posted by Tate Hansen Wed, 07 Nov 2007 01:31:00 GMT

Today I was talking with a colleague from a partner company about the PCI certification - I think he's up for recertification.

The interesting thing is he was talking to a Qualys representative recently whom, affably speaking, offered tips on how to tune the Qualys scans based on new modifications made at Mastercard's test lab. The representative also said he could review the report Qualys automatically builds. My colleague exclaimed to me "It sounded like they already have the answers".

Of course they do. Qualys pays PCI to verify their ability to discover what PCI wants them to discover. People pay and use Qualys so they can become PCI certified. Anybody willing to click "start scan" has the ability to be an Approved Scanning Vendor.

What's my problem with all this? For one, the certification process is rotten:

http://blog.clearnetsec.com/articles/2007/05/16/pci-not-our-problem.
http://blog.clearnetsec.com/articles/2007/05/04/pci-misleading-racket

On top of that, it's costly and does little to vet engineers claiming competency. Its design is to weed out small security firms, which is probably why it fires me up in the first place and turns me into a cynical punk all day.

Tags , , , , , ,  | no comments

Trying out unicornscan

Posted by Tate Hansen Mon, 15 Oct 2007 04:10:00 GMT

We’ve hit a new high. We’ve soaked ourselves in a bandwidth bath on behalf of a client whom would like us to discover active services across a range of six public /16 blocks plus some scattered /17s, /24s, etc. The range is close to a total of 400,000 IPs.

We started out with five dual Xeon systems running 20 to 40 instances of nmap, each tuned, and each instance targeting 64 IPs. This client wants the job completed in weeks, so we decided it was a good time to get more experience with unicornscan.

By luck, we tapped into a Danish provider that is allowing us to push 55Mbits/s. I have no idea how much that amount of bandwidth would normally cost, especially if sustaining it 24x7 for a few weeks, but I’m guessing it is way over $10,000. Our client would allow us to go up to 100Mbits/s, alas, our luck doesn’t go that far.

Anyway, we now have faster dual-core systems each pushing ~25 Mbits/s via unicornscan like so:

sudo nohup /usr/local/bin/unicornscan -mT -p –r25000 -vv xxx.zz.0.0/16:a -w unicorn.output.for.xxx.zz..0.0.fullTCP > unicorn.output.fullTCP &

We have lots of results from nmap; so far unicornscan is matching the nmap results. Having the ability to specify packets per second with unicornscan is super nice.

We’ll create a follow up post on how all our scanning worked out on this gig when we’re finished (sometime in late November).

Tags , , , , ,  | no comments

Flip the reporting and get the customer to speak

Posted by Tate Hansen Mon, 24 Sep 2007 20:24:00 GMT

I can’t resist rehashing this topic.

Why not, after performing a penetration test, ask the customer what they observed? And when doing so, using commensurate effort to clearly surface the importance and point from asking that question.

So many are ordering up vulnerability and penetration tests like it’s a quick immunization shot against being hacked. They may get a wee bit nauseous after viewing their vulnerability report, but once they mitigate and recover they feel invulnerable, however temporary.

Penetration testing proves nothing of invulnerability, and it naturally follows, ad nauseam, to ask, again, what does it really prove then? Marcus Ranum summarizes Gary McGraw’s answer to that:

Gary McGraw likes to refer to pen testing as the "badness-o-meter" - it's a test that registers, at one end of the dial "your network sucks" and at the other end, "we don't know."

If the “We don’t know” club doesn’t already list as members all penetration testers, it surely should. No one has the knowledge or possession of everything needed to break all systems, so everyone is playing their own part in the “don’t know”.

Even if you are named Mr. Exploiter and pay cash to subscribe to every 0day exploit factory, the value of showing your customer they get owned when you call an 0day blitz is not only from proving you can sprint past their defense. Anyone with an exploit can do that.

The value rises from the conversation you have with the customer after the party. Why should the customer care if the latest 0day worked? What was the customer to do about it anyway?

Companies whom take their security seriously need to report back to you. Instead of hearing the customer ask you, “What did you find?”, you should ask the customer, “What did you observe?”.

Imagine how shocked you’d be to listen to a customer detail all of your efforts during a penetration test: They report to you which systems you attacked, when and how, and what information you had obtained.

That would be good security. Penetration testing should move away from “I got you with this 0day” to “You identified 90% of my efforts to compromise your systems”.

Flipping the reporting paints a clearer picture of the overall security of your customer: Those that do well you could posit have good policies in place and have built respectable awareness and response capabilities.

Tags , , , , , , ,  | no comments

Singularity Summit

Posted by Tate Hansen Tue, 11 Sep 2007 05:04:00 GMT

For those not in the know, of which I was a member, there is a far out annual summit whereby distinguished researches speak of impending craziness. The summit is about the Singularity.

One of the presenters, Eliezer Yudkowsky, described how the definition of the Singularity has taken on a few different meanings. If you’re interested, you can check this video interview of Eliezer here: http://video.google.com/videoplay?docid=6315588532367156746

Peter Thiel (co-founder of PayPal + lots of other things) offered several interesting snippets, which are captured in this wired blog entry discussing his presentation at the summit: http://blog.wired.co/business/2007/09/peter-thiel-exp.html

After attending the event, I sort of feel teased by the potential of seeing an Artificial General Intelligence within my lifetime. I think I hope to see it, although it sounds like the number of paths to a happy ending is small relative to the number of paths which lead to the end of the world.

Tags , , , , , ,  | no comments

Anything alert you?

Posted by Tate Hansen Thu, 06 Sep 2007 01:37:00 GMT

There is nothing I’ve seen recently to promote a valuable exercise to do after receiving a security assessment. That is, as the client, what did you see?

Did you have anything alert you? If so, what did it suggest? Did you have enough information to piece together what was happening? (Bonus: do you know which tools were fired towards your IPs?)

The majority of my clients have no clue if anything occurred. That’s bad. Businesses which have little to lose may decide to ignore investing in monitoring and detection, but for others it’s turning a blind eye.

I’m going to dig a little deeper on future exit calls to get more information. I often ask clients if they detected any strange behavior, but there is definitely more room to expand the discussion.

Tags , , , , , , , ,  | no comments

Tech note on Syslog, TCP, and Cisco ASA/PIX

Posted by Tate Hansen Fri, 24 Aug 2007 01:05:00 GMT

Absent of Cisco wizard skills caused me a little pain yesterday. I remotely configured my Cisco ASA to forward syslog via TCP to a central log host. When I subsequently rebooted the central log host, I lost the ability to establish new connections to anything behind the ASA.

Luckily, I had an established session to a system with a serial connection which enabled me to recover.

I hadn’t run into this before, but I confirmed my experience:

1. If it is unable to log via a defined TCP syslog session, a PIX will not create any new connections (although connections opened before the failure of the session will continue to work). The PIX will log a message to the console stating that it is disallowing new connections.
2. In order to re-establish connection activity, the privileged set logging command, with the correct parameters, will have to be entered or the PIX reloaded.

Tags , , , , , , ,  | 1 comment

Attackers will win so what can you do?

Posted by Tate Hansen Mon, 06 Aug 2007 14:36:00 GMT

The cat and mouse game you’re playing to protect your network against the enmity of motivated attackers is perilous.  You’re going to lose. (more and more 0day is coming: see Bejtlich's summary at the bottom of his post for Black Hat Day 1 '07 or Immuntiy Sec's recent presentation which makes a nice point, "Our time to exploit is shorter than your ability to patch")


The problem is you have to play if you want to be connected. Playing then is about choosing the best strategy, or the dominant strategy if one exist. I think there is a dominant strategy for securing your network - traffic analysis (augmented with content & context when available).


From the game theory book Thinking Strategically:

In general, a player has a dominant strategy when he has one course of action that outperforms all others no matter what the other players do.  If a player has such a strategy, his decision becomes very simple: he can choose the dominant strategy without worrying about the rival’s moves.  Therefore it is the first thing one should seek.

Traffic analysis (augmented with content & context) is the best solution when you want pervasive security (i.e. proactive in identifying all types of normal and anomalous activities and strong incident response support due to having the history of communications)


Let me make the point that I do not think traffic analysis in isolation is the winning strategy, but it is a winner when combined with other data freely available on your network. And because I'm tackling this from a defensive perspective (i.e. you have ownership of the network you're protecting), then I'm assuming you get extra defensive observation muscle - snippets of content and context parceled and sent to you by services like syslog.


To take a step back, wikipedia defines traffic analysis as the process of intercepting and examining messages in order to deduce information from patterns in communication. That means learning what's going on from only analyzing the metadata surrounding communication (e.g. the sender, the receiver, the time and length of messages, etc.).

It's amazing what traffic analysis can uncover. Taken from Blink:

The Germans were [in WW II], of course, broadcasting in code, so - at least in the early part of the war - the British couldn't understand what was being said. But that didn't necessarily matter, because before long, just by listening to the cadence of the transmission, the interceptors began to pick up on the individual fists of the German operators, and by doing so, they knew something nearly as important, which was who was doing the sending. [..] After they identified the person who was sending the message, the interceptors would the locate their signal. So now they knew something more. They new who was where.

This goes on and is only a glimpse of what can be learned of course. In IT security, then you can imagine it's possible to:

  • uniquely identify all users on a network only by observing patterns (e.g. quirks about how a user types on a keyboard, command sequences a user typically executes, patterns on how they peruse Internet and Intranet sites, plus 100s or 1000s of additional ways)
  • to always identify an attacker by observing that nothing the attacker is doing matches any known trusted users' patterns

At Blackhat '07, a presentation on traffic analysis had a slide titled "Why do this?", which speaks to the advantages of traffic analysis (I added stuff between []):

  • crypto [i.e. you can't see the content anyways]
  • too much data, already [i.e. the need to aggregate and summarize]
  • it's easier than analyzing everything
  • it's hard to evade [i.e. you'll either catch the attacker or possess the data to reconstruct communication paths]

Now extending traffic analysis with relevant content and context data (syslog, authentication logs, alerts from point products, etc.) allows for very powerful detection for all types of attacks, likely with much greater precision and breadth of coverage versus doing anything else (or relying on a mix of prevention focused systems). Hence, the reason why I think it is a dominant strategy for pervasive security.

There is a major challenge in analyzing all this related information though, which is called the curse of dimensionality. I'll save that one for later.

 

Tags , , , , ,  | 2 comments

Network Visualization

Posted by Tate Hansen Sun, 22 Jul 2007 20:38:00 GMT

There is debate over the value of visualization techniques. I’m a proponent. Of course the point is to be able to deduce the information you’re looking for faster versus alternatives. Regardless, I’m fatigued by the run of the mill table views, and as such, I like to read about successful stories.

From: http://secviz.org/?q=node/74

“This directed graph reminds me of the social network you might see in a suburban high school, and revealed to us some interesting things, including the existence of a new network monitoring tool quietly installed by a rogue internal unix admin team... us and them, we're having a "come to Jesus" meeting tomorrow ;)”

That rocks.


A relation browser like http://www.der-mo.net/relationBrowser/index.html may make for a great forensics tool. Actually, imagine for a minute all the cool uses.

A timeline carousel: http://simile.mit.edu/timeline/

A big list of visual techniques with thumbnails for fast viewing: http://www.visualcomplexity.com/vc/
http://datamining.typepad.com/data_mining/graphs/index.html

Gapminder is popular: http://www.gapminder.org/downloads/applications/

Tags , , , , , , ,  | no comments

building a better security events system

Posted by Tate Hansen Sat, 30 Jun 2007 21:56:00 GMT

It’s hard to build a decision support system based on partial views of the world.

My goal is to identify interesting events on a network and to prioritize those events based on sets of attributes. Yes, there are lots of products that do this. But most focus on a slice of the world (e.g. an IDS fires an alert based on a regex match on a single packet). And that is boring.

Doing it for the whole world is where the action is at.

Capture an alert fired from an IDS, check netflow for a session, note a “first-time” event recorded in a syslog message, mix in statistical data mining and learning techniques – and do it all in near real time. This is how things get interesting.

Unfortunately it’s hard to get complete visibility (i.e. get all syslog, all netflow, all application logs, etc.). There must be a point though where I can get enough information to successfully prioritize interesting events. I’m not sure exactly where that’s at, but it’s a fun problem to work on.

The picture is of the inside of IT-Universitetet in Copenhagen where I’m working for a few weeks. The meeting rooms all jet out into the open space in the middle – a pretty cool design.

Tags , , , , , , ,  | 5 comments

PCI: Not our problem...

Posted by Tate Hansen Thu, 17 May 2007 02:51:00 GMT

What happens when the test environment operated by MasterCard (they “own” the testing lab) is misbehaving? I know. They yank the wheel, swerve away from responsibility, and point to the PCI council. And PCI? They point back. Beautiful, no?

You see because they refuse to disclose missed results to you they duck responsibility for anything that may have been their fault. They also clearly imply if anything is missed in your attempts to identify vulnerabilities then it is surely your fault or a problem with the tools you used.

I love it: No clear pass criteria, no way to challenge a decision, and no transparency of what or how they are doing. For all this great service you get to spend thousands every year!

So what happens when you call bullshit and raise hell? They pass you. :) Let me not forget to mention we had a few extra bullets in our clip they may have unexpected us to have – bullets provided to us by friends with information.

Be forewarned; this process has serious issues.

Tags , , , , , , , , , , ,  | 1 comment

PCI ASV wonderland

Posted by Tate Hansen Fri, 04 May 2007 17:52:00 GMT

The world of the PCI Security Standards, ASVs (Approved Scanning Vendors), and commercial scan vendors is, from my limited interactions, not exactly on the straight.

Having recently spent considerable time preparing, scanning, and writing reports with the explicit goal of becoming an ASV, I’m disturbed by my communications with all involved.

It doesn’t help that the pass criteria to become an ASV is not clear. Is it based on discovering all vulnerabilities on their test network? A subset? Which parts are subjectively reviewed?

I like to use Qualys to baseline vulnerabilities, which a test representative caught as one of the tools we’d be using based on the source IP blocks for scanning. He said something like “If you use Qualys, you’ll get 95% of what you need”. From that I guessed the example web application would have vulnerabilities which would be missed by Qualys and other network-based tools. As expected, that was true.

Fast forward. We received feedback that our reports have not been reviewed because Qualys changed something recently causing expected results to be absent which PCI requires for passing. The representative said “If you’d have scanned a month ago with Qualys, you’d have passed with flying colors”. He added that they are in communication with Qualys to resolve the issue.

Nevermind that Qualys was only one of the tools we used and we had added vulnerabilities not discovered by the popular free or commercial scanners. It was clear the representative didn’t review the report, which he said as well (and may have done to protect us from an automatic failure -- even though our current pass status is pending). But they would not reveal the gaps, which obviously makes it hard to understand what the problem is. I appealed by mentioning I had added vulnerabilities not discovered by Qualys. He then modified his previous statements by saying he in fact spot checked the reports and the items he was looking for were absent. My gut reaction to all this: bullshit.

The connections apparent make for a nice racket. You pay lots of money to PCI. PCI “communicates” with selective vendors to ensure all the vulnerabilities they expect to be discovered are discovered. You pay money to the scan vendors. And what about if you find additional vulnerabilities or a superset? Sounds like they don’t check and don’t care. I would think the idea behind all this is to make sure you are adding sufficient value to entities subject to PCI regulations (even if that means you didn't catch 100% of everything bad). If passing simply means doing a blind Qualys scan when it works right (i.e. does what PCI wants), well then, you now know what to do to as an attacker -- just go after something Qualys doesn't check.

So much for trusting this process and what it does to vet competent assessment companies.

Tags , , , , , ,  | 1 comment

An example of why human effort is helpful when assessing web applications

Posted by Tate Hansen Fri, 13 Apr 2007 22:37:00 GMT

It can take some digging to discover if you’ve successfully injected any code into a web application. I was using the ALL-FUZZ-STRINGS that comes with Suru (added additional strings from sources like ha.ckers.org XSS Cheat Sheet) to run through a list of popular input validation attacks.

Suru is a Man In The Middle (MITM) proxy that sits between the user's browser and the web application.

I was keying on the fuzzy logic results window in Suru which helps the assessor determine if any attacks were successful.

The Fuzzy logic trigger box is used to fine-tune the results received during a fuzzing run. The user has multiple options that range from content extraction to brute force fine-tuning. The results from the fuzzing run can be mined or filtered for false positives, positives or false negative findings.

Here is a screenshot of the some of strings that are included in the ALL-FUZZ-STRINGS

During this phase of testing I was interested to see if any of the XSS type attacks succeeded. The fuzzy logic results window shows calculated scores that are derived from running an equation on each response and on a base response.

In order to conduct Fuzz logic or content extraction runs, Suru needs to compare the results to a base response. For example, if we need to brute force an application we need to compare a failed attempt with a successful attempt to gauge successes or failures.

I was surprised when all of the scores were basically the same – implying that none of the attacks caused a response which was measurably different from the base response. I already knew XSS/code injection vulnerabilities existed in this part of the application.

Upon a little investigation, it was clear the reason why Suru’s fuzzing scores were remaining more or less the same. For this particular web application, the response page created from submitting values was simply an acknowledgement of the POST request.

This response page was always the same -- there was no deviation from the base response. But by manually crawling the site and clicking on deeper links, I ran across a link where the injected code was executed and displayed in the response. One of the attacks was successful.

The take-away here is that it is important to manually walk the web application after performing XSS, code injection, etc. attacks to really see if any were successful. The attacks may cause the web application to respond with the exact expected attack string (i.e. if you put in as part of the attack the string “URL=javascript:alert('XSS-IS-HERE');”, then you can scan the HTML response for the string “XSS-IS-HERE”), but the web application may modify the string enough to evade automated checks yet still be vulnerable.

And in the case above only discovered by manually crawling the app to see where the attack successfully surfaces.

Tags , , , , , ,  | no comments

Older posts: 1 2