ClearNet Security

Jeremiah Grossman wrote an article titled: "Technology alone cannot defeat Web application attacks: Understanding technical vs. logical vulnerabilities" (requires registration). I pulled a couple sections out which explain why automated scanners can not find 'logical' vulnerabilities (hence, capture < 50% of the potential vulnerabilities present):

[...]

"This has made the automated process of finding simple vulnerabilities hard -- and finding difficult ones impossible.

Consider the following example. If we visit a Web site and are presented with the following URL:
http://example/order.asp?item=50&price=300.00

Can we guess what the application order.asp combined with the parameters item and price do? Using intelligence unique to humans, we can quickly deduce their purpose with relative certainty. This is a product ordering application. The item parameter is the particular product we are interested in. In our case, let's say an iPod. The price parameter is the amount we are going to pay for our portable music player. What happens if we changed the price of 300.00 to 100.00? Or 1.00? Does the Web site still sell us the iPod? If so, we can easily understand that the Web site should not have allowed the price alteration. As humans, we possess a natural ability to assess context, and we aptly refer to these types of issues as "logical vulnerabilities," issues that only humans can identify.

Now, if an automated scanner attempted the very same attack in a generic fashion, how would it decide if a custom Web site's response was good or bad? How would it know if the attack worked or was adequately defended? Or what the item and price parameters were supposed to do in the first place? The answer is clear: Scanners cannot reliably make these assumptions."

[...]

"Scanners are great at tackling technical vulnerabilities such as cross-site scripting and SQL injection, and not effective at identifying price list modification, credential/session prediction, insufficient authorization, and other logical vulnerabilities.

It would be nice to fully categorize which vulnerabilities (or classes of vulnerabilities) can be checked for via automated tests and which only a human can check.

A recent thread found on web app security reveals some insights regarding web app scanners.  It is really more of the same (i.e. they suck with out lots of manual help), but this thread includes vendor opinions.  It kicked off when Dr. Holger Peine published a comparison report on web app security scanners which triggered some defensive posts.  I care less about the defensive posturing, but what is interesting to read is the post by Ory Segal on how to properly configure AppScan.  It’s another notch on why these tools are not just ‘click and go’ tools.  You need some level of prior knowledge of the app you’re going to scan in order to configure whatever tool for maximum accuracy. 

Even if you scan with “optimal” settings (i.e. fully customized) you’re likely missing lots of true vulnerabilities.  From Jeremiah Grossman (develops web app scanning tools):

“In my experience, scanners can only reliably TEST for about half of the possible vulnerabilities in a website.”

So, to state the obvious, lots of manual work is required.  I agree with Jeremiah’s other statements:

“These observations have brought us to the following conclusions:

1. Scanners find vulnerabilities and save time.
2. Experienced security engineers must find the “logical flaws”

Only by combining the two is it possible to achieve comprehensiveness and due diligence.”

However, comprehensiveness for me would include code analysis.  I’m taking his statement out of context a little, but it would be the only way to capture certain risks.  Say for example, a developer was testing a piece of code or had setup a backdoor for whatever reason.  A ror example:

if @params[:post][:login] == “letMeIn”
  flash[:notice] = “Superuser login successful”
  redirect_to :controller => “admin”, :action => “setup”
end

Superuser privileges.  Automated web app scanners will always fail to capture things like above.  It’s possible some brute force tool or fuzzer tool may get lucky and discover a backdoor like above, but this is a simple contrived example. Another post asserts:

"I remember I encountered one case in which the session id is sequential; the scanner did not even pick up such an obvious flaw. In another case, after user login, the user id is embedded as a hidden value and used to authenticate the user. And the scanner failed to pick this up again."

Martin O'Neal summarized it well:

Automated tools are essential for making the assessment process achievable in a reasonable period of time, and are an important part of the process, however (and it is a big however) they should never be seen as being "the process" in themselves.

I had a conversation with a law enforcement friend the other day and I learned of two trends I wasn't aware of: one is the increasing number of people sending threats via email and two, the number of people installing key loggers illegally. He mentioned many of the key logger cases are couples in the midst of divorce or separation. I was unaware the police will always respond if you report receiving an email threat. The statute:

Under 18 U.S.C. 875(c), it is a federal crime, punishable by up to five years in prison and a fine of up to $250,000, to transmit any communication in interstate or foreign commerce containing a threat to injure the person of another. Section 875(c) applies to any communication actually transmitted in interstate or foreign commerce - thus it includes threats transmitted in interstate or foreign commerce via the telephone, e-mail, beepers, or the Internet.

Regarding key loggers, it is unclear to me whether their use always violates federal wiretap laws or not. The most recent article I found discussing this in general is here: http://www.theregister.com/2005/10/04/secfocus_keyloggers/print.html

I’m seeing RFPs (Request for Proposal) with language like:

"Exploitation should be performed using tools and techniques for which there is a reasonable expectation would mirror common tools and techniques used by potentially malicious users."

Ok, that’s loaded. What are 'common tools and techniques'?  For tools, is it Fyodor's Top 75 Security Tools? Metasploit is not listed there, CANVAS and Core Impact are mentioned but are not listed as one of the 75, so are those reasonable?  How about app specific tools like WebInspect or Appscan?  What about converting a PoC exploit from FrSIRT's database?  You have no developer friends? Alright, but if you're a steadfast malicious user and somewhat savvy, I'd bet you would pay for a nice VulnDisco Pack, especially if what you're after is more valuable to you or someone else.

This gets tricky to accurately nail down – the what is reasonable part – and think about the costs of using, say, 5 of these commercial tools.  You’d be at $20,000 after negotiations.  That’ll sure win you the bid.

My point is you can easily make arguments for any of the tools listed above, not to mention the 100s of free tools available.  Technique likely has less common ground.    Some like to say penetration testing is worthless, but I think that argument fits the idealist versus realist debate.  If there is a demand for this service, I see no reason why not to try win work if you play in this field.  It is just super hard to succinctly provide enough information to a client to enable them to differentiate and make a quality decision. 

Probably a good question to submit when responding to similar RFPs goes something like:  ‘Ok.  You want reasonable assurance you can’t be compromised.  Considering your definition of a ‘malicious user’, what do you consider the greatest amount of resources (i.e. time and money) a malicious user may dedicate?’  I think that get’s you closer to what the client really wants.

There have been some recent newsgroup postings about consistency problems with nmap results (especially when scanning over the internet). In my experience, if you want the best chance of obtaining accurate nmap scan results, you need to:

• check the latency to your targets (ping some device on the target net block for awhile, what is the average rtt value?)
  • use the average to configure the --max_rtt_timeout nmap option (e.g. if your ping time is 300ms, you obviously want to set this option to more than 300ms to prevent nmap for missing an open port if/when a probe takes longer than 300ms. Maybe set it to 500ms or even higher. Speed suffers, but we're chasing accuracy)
• if you can, check the available bandwidth limit for you and the target net (e.g. T3:45mbps, T1:1.544mbps, DSL, etc.)
  • use this to configure the --min-hostgroup and --max-hostgroup nmap options and the --min-parallelism and --max-parallelism options (i.e. the more bandwidth each side has available, the more hosts and ports you can scan in parallel). This can significantly reduce overall scan time while keeping accuracy as the highest priority. I typically use IPTraf to watch pps (packets per second) to guage my bandwidth usage. If my options are too aggressive (i.e. consuming too much bandwidth), I'll kill the current scan and start over with different tuning values. The objective here is to set --min-hostgroup and min-parallelism to something that both sides can handle without sacrificing accuracy. If bandwidth permits, I like to set min_parallelism to 100 (number of parallel SYN requests or whatever scan type you're doing). I've gone up to 300 for --min-hostgroup (had lots of bandwidth and plenty of server horsepower). You don't want to cause your scan server, the target, or any device the probes traverse to drop packets, so the more information you have about all the components involved the better you'll be able to tune these values.
• if you can, verify there are no 'smart' devices impeding scans (e.g. IPS, rate limiting device, etc.)
• run multiple nmap scans of the same target, preferably at different times of the day and different days of the week (e.g. nightly backups can saturate certain hosts/links which causes nmap to fail to discover open ports). I like to run multiple services scans (~1600 ports) and do one or two 65k scans depending on circumstances. If time permits, the confidence boost you get knowing you've maximized accuracy by running repeated scans is satisfying. It also shows your assiduity; you can tell your customer you fired off 5 scans at different times to obtain your final list of exposed services. If you missed something after repeated scans (assuming you used safe tuning values), something is likely wrong.
• set --host_timeout to something you are comfortable with.
  • if you determine a scan may take 20 hours to perform a 65k port scan of the target device, then set this option accordingly. Note: you can use 's', 'm', or 'h' to specify seconds, minutes, or hours.
• set --max-scan-delay to something low if nothing is impeding scans
  • I typically set this option to 0 because I'm confident in my other tuning values (no delay between probes, just fire them off)

While tuning nmap options can do lots of good things it is still important to verify your scan server can keep up. We've knocked down 8-way Opteron systems w/16GB of RAM, so optimal nmap options may not be realistic for your hardware.