ClearNet Security

A recent thread found on web app security reveals some insights regarding web app scanners.  It is really more of the same (i.e. they suck with out lots of manual help), but this thread includes vendor opinions.  It kicked off when Dr. Holger Peine published a comparison report on web app security scanners which triggered some defensive posts.  I care less about the defensive posturing, but what is interesting to read is the post by Ory Segal on how to properly configure AppScan.  It’s another notch on why these tools are not just ‘click and go’ tools.  You need some level of prior knowledge of the app you’re going to scan in order to configure whatever tool for maximum accuracy. 

Even if you scan with “optimal” settings (i.e. fully customized) you’re likely missing lots of true vulnerabilities.  From Jeremiah Grossman (develops web app scanning tools):

“In my experience, scanners can only reliably TEST for about half of the possible vulnerabilities in a website.”

So, to state the obvious, lots of manual work is required.  I agree with Jeremiah’s other statements:

“These observations have brought us to the following conclusions:

1. Scanners find vulnerabilities and save time.
2. Experienced security engineers must find the “logical flaws”

Only by combining the two is it possible to achieve comprehensiveness and due diligence.”

However, comprehensiveness for me would include code analysis.  I’m taking his statement out of context a little, but it would be the only way to capture certain risks.  Say for example, a developer was testing a piece of code or had setup a backdoor for whatever reason.  A ror example:

if @params[:post][:login] == “letMeIn”
  flash[:notice] = “Superuser login successful”
  redirect_to :controller => “admin”, :action => “setup”
end

Superuser privileges.  Automated web app scanners will always fail to capture things like above.  It’s possible some brute force tool or fuzzer tool may get lucky and discover a backdoor like above, but this is a simple contrived example. Another post asserts:

"I remember I encountered one case in which the session id is sequential; the scanner did not even pick up such an obvious flaw. In another case, after user login, the user id is embedded as a hidden value and used to authenticate the user. And the scanner failed to pick this up again."

Martin O'Neal summarized it well:

Automated tools are essential for making the assessment process achievable in a reasonable period of time, and are an important part of the process, however (and it is a big however) they should never be seen as being "the process" in themselves.