ClearNet Security

Which vulnerability tests did Nessus run? How long did it take? Which tests take the most time?

Here is a quick perl piece I wrote awhile back to answer these questions. My main goal at the time was to find which vulnerability tests were consuming the most time. The script parsers the nessusd.messages file and generates an output which looks like the following (I added color to this sample to emphasize the different components):

[suse] > ./parseNessusdMessages.pl 10

===========================================================
xx.xxx.78.65: completed checks = 2262: Time to complete host scan = 369.65 (0:6:9)
68.785 (%18.608) invision_gallery_st_sql_injection.nasl
55.707 (%15.070) lighttpd_cgi.nasl
49.862 (%13.489) DDI_IIS_Compromised.nasl
47.847 (%12.944) horde_detect.nasl
31.121 (%8.419) rpc_portmap.nasl
30.254 (%8.184) amanda_detect.nasl
18.491 (%5.002) h323_detection.nasl
17.578 (%4.755) dont_print_on_printers.nasl
17.525 (%4.741) basilix_inc_files.nasl
16.554 (%4.478) ventrilo_detect.nasl
16.474 (%4.457) krb_pingpong.nasl
not showing remaining list... (only showing 10)

===========================================================
===========================================================
xx.xxx.217.250: completed checks = 2262: Time to complete host scan = 677.22 (0:11:17)
224.901 (%33.209) office_files.nasl
62.777 (%9.270) sql_injection.nasl
61.661 (%9.105) mozilla_default_perms.nasl
60.220 (%8.892) NetSphere.nasl
55.007 (%8.122) shells.nasl
47.335 (%6.990) mozilla_176.nasl
46.044 (%6.799) bnc_auth_bypass.nasl
45.214 (%6.676) qpopper2.nasl
45.110 (%6.661) squid_dos.nasl
44.425 (%6.560) dont_print_on_printers.nasl
44.394 (%6.555) CA_License_Service_Stack_Overflow.nasl
not showing remaining list... (only showing 10)

===========================================================
===========================================================
xx.xxx.78.6: completed checks = 2350: Time to complete host scan = 647.36 (0:10:47)
163.406 (%25.242) sendmail_ident.nasl
163.162 (%25.204) traceroute.nasl
120.236 (%18.573) nessus_detect.nasl
106.080 (%16.387) ftp_backdoor.nasl
101.724 (%15.714) mozilla_nntp_heap_overflow.nasl
96.309 (%14.877) quote.nasl
74.986 (%11.583) eserv_dir_traversal.nasl
60.184 (%9.297) cfengine_authdiag.nasl
53.321 (%8.237) crlinux_file_reading.nasl
45.546 (%7.036) bnc_auth_bypass.nasl
45.315 (%7.000) ircd_ignition_ircop_vuln2.nasl
not showing remaining list... (only showing 10)

The script parses all the tests found in the nessusd.messages file which were executed (not skipped) and displays the results organized by target IP address. It also sorts and displays in descending order the vulnerability tests which consumed the most time. The parseNessusdMessages.pl takes a single argument which specifies how many vulnerability checks to display per IP address (i.e. in the above example, 10). Few notes:

• This works best if you start with a empty nessusd.messages file. Run a nessus scan and after it is complete then you can run the script to generate the output like above.
• The nessusd.messages file is not always located in the same spot. You may have to modify the path in the script (likely locations include /opt/nessus/var/nessus/logs/nessusd.messages or /usr/local/var/nessus/logs/nessusd.messages). By default, you need root permission to view the nessusd.messages file. I often just copy the nessusd.messages (using sudo) file to my homedir and change the path in the script to check "~/nessusd.messages".
• The 1st column is the number of seconds the vulnerability test took followed by the percentage of overall time (i.e. how long it took to scan the target IP).

So, if you are in the business of tuning your Nessus scanner, then this script may provide you some value.

 

I've worked with firewalls and various network filtering devices for a long time.  It's still a fairly popular area of development but as it has matured it has stayed fairly primitive.  There are a lot of reasons for that and I think that discussion is less constructive than talking about how I think it should be done and the various pitfalls.  

For starters,  I don't know if you've ever "inherited" a firewall or had to get one changed in a midsized corporation.  They tend to grow and never really shrink and after a while they take on this state where you're almost afraid to shrink them because you don't know the history.  Further, as the corporation grows in size and starts to take security more seriously you'll have to go through a process to actually change a firewall and it will take approval from people that probably don't care about what you need to accomplish that requires a change in the first place and they might not even know (or care as can be in some IT groups) what it is that is really needing protection in the first place.

Firewalls tend to get treated as static entities which is pretty good but the internet and networks in general are fairly dynamic,  good rules now might not be good rules tomorrow.  If your organization is in to the more static style of thinking,  change control your firewall.  Put it in to clearcase, subversion or whatever you use and put detailed comments in when you change a configuration. 

Also incorrectly built firewalls can be slow and inefficient. I know of a place that had a traffic shaper and Pix and their firewall became so complex that when you removed the traffic shaper the Pix became overloaded and in effect the traffic shaping device became and integral part of their security system which makes the whole network that much more complex. They were using tools incorrectly which also makes it more likely that the tool won't do some task that they may need in the future and ultimately it restricts the things you can do with the traffic shaper and firewall so that if there is ever a need to quickly change something you have to be more careful because that change may end up breaking the interaction between the firewall and traffic shaper that allows them both to work. 

I found a nice reference to this NSA how-to from a post on the Planet Security blog. Here is a local copy.

I wish that was true, but it is furthest from the truth. There is an unfortunate conception for many tasked with reviewing or choosing a scanner (a conception espoused by the marketing of several Vulnerability Assessment vendors) which is the quality of a scanner is directly based on how fast it can scan. In this race for speed, vendors’ almost always default to performing less than complete checks. In fact, some vendors shy away from adding checks to their products in an attempt to retain speed (i.e. adding more checks will slow a scanner down). That is probably not what you want. I also have yet to see a single commercial scanner default to performing a full TCP port check. Totally understandable; if a scanner’s default policy opted for full coverage then you would surely dismiss the quality after running a test scan and learning you have to wait 40 hours for the results. Too bad. In the world of network-based vulnerability scanners there is a trade-off that spans long: speed and accuracy. Speed kills accuracy.

“Watch this man, I can scan a class B in 10 seconds.” “Really? How did you do that?” “Oh, well, it is only checking if one port is open.” “Ah, nice.”

So if you want accuracy, then slow it down. Don’t run a thousand threads, allow more time for remote devices to respond, choose complete port coverage, and don’t parallelize so much you saturate switch ports or test your operating system's TCP/IP stack limitations.

I'm a long time emacs user.   It does just about everything, it runs on everything, and once you're used to the keys everything else seems clunky by comparison.   VIers and XEDITers say the same things.  While I love emacs and it is one of my favorite programs, I have some issues with it:  1) The migration to guile has stalled 2) The lisp compiler is aged and not terribly good, I run it on dual Opteron and dual G5 machines and it's the slowest part still, lisp heavy emacs tasks aren't that much better than on an old machine and lastly 3) it doesn't matter a ton but it's not as nice to look at aesthetically and just doesn't get a ton of attention in that department.

So not long ago I fired up eclipse and I've been quite impressed with it.  There are vague familiarities with LPEX and the old Visual Age products for Java, Smalltalk and C++.  It's definitely java centric out of the box and while I do program in Java that is a turn off because I also program in a lot of other languages and I like a versatile tool that does a lot of things well rather than only one thing.  I forced myself to use it though,  the emacs bindings are okay,  not great but with a little tuning they are tolerable.   There aren't any emacs like macros but you can do most things you need to with out.   Then I started exploring the world of plugins and that is what won me over.Most notable is CDT.  It is a full blown C/C++ "editor mode" for eclipse.   It has all the syntax highlighting and normal features you'd expect but it also has automatic code completion, code browsing and the assisting features you usually only see in Visual Studio or other language specific IDEs.  

There is also PyDev and RDT for Python and Ruby.    As with all plugin based applications like this in the opensource world the quality of different plugins can vary but there are some very high quality ones.  There are also a lot of plugins you can buy that do everything from UML modeling, visual GUI construction to a full blown Ada IDE.   I highly recommend having a look at eclipse,  I find myself using it more and more.   It's self-updating also,  eclipse can be programmed to check when new versions of CDT, RDT and whatever plugins you use have been updated.

Here is a set of decent plugins I've been using and the update URLs for them,  just paste these into eclipse (Help -> Software Updates) and it will automatically install them for you.

• Plugins for dealing with Maven based builds: http://mevenide.codehaus.org/release/eclipse/update/site.xml
• PyDev is a python plugin with code completion and browsing: http://pydev.sf.net/updates/
• CDT 3.1 is the newest stable version of the official C and C++ plugin: http://download.eclipse.org/tools/cdt/releases/eclipse3.1
• RDT: http://rubyeclipse.sf.net/updatesite
• subclipse allows eclipse to integrate with subversion: http://subclipse.tigris.org/update
• javasvn: http://tmate.org/svn/
• coverlipse: http://coverlipse.sf.net/update/
• jboss-ide, also has XML plugins: http://jboss.sourceforge.net/jbosside/updates
• Scheme: http://schemeway.sourceforge.net/update-site
• Perl EPIC: http://e-p-i-c.sf.net/updates
• Radrails: http://radrails.sourceforge.net/update

I was just digging into my notes about how to best tune Nessus for a particular network condition.  Back in Oct ’04 I posted a reply on the full-disclosure list which details some of my observations.  Those observations applied to the Nessus 2.2.x train.  It looks like most of the same tuning parameters exist in the new 3.x version.  I’m going to run tests to see if the purported 2 to 4x speed improvement in the new 3.x train is true.

One of my favorite diagrams I always reference when thinking of the best way to assess risk comes from, perhaps surprisingly to you, a presentation by a Microsoft security manager for threat and risk assessment (Jared Pfost). Here it is:

If I remember right, I believe they assign a value between 1 and 5 for the 'Impact' and 'Probability' statements. The result is a positive number whose value determines the speed of response. A high value may trigger an immediate and sustained effort to mitigate the risk. A low value may represent less risk and be addressed in a slower and less costly approach. I also believe it reduces alarmist tendencies because it forces you to stop and think through exactly what is at stake. Whatever it takes to stop colleagues from hitting the big emergency button every time a new disclosure is released is a good thing in my mind.

A newer and more detailed methodology which appears to build on the diagram above can be found here:
http://www.microsoft.com/technet/security/topics/policiesandprocedures/secrisk/ack_page.mspx

I run a Powerbook 15", as my main system. It handles most things pretty well except for running Windows in VirtualPC. VirtualPC is so painful on the Powerbook that I hardly ever use it. Now with the Apple line going Intel their could be a possibility to dual boot Mac OS X and Windows. It all depends on the the new BIOS called EFI. If Windows can boot with using EFI, then I think I have found my new scanning laptop!

For efficiency, thoroughness, or comparison you can chain several popular web application assessment tools together.  Three tools I sometimes chain in a series are the BURP Spider, Paros Proxy, and WebInspect.  To do this on a single system, you simply configure a listening port for each tool.  Check the diagram below:

You can configure each tool to do this by specifying a listening port for incoming requests and an IP address:listening port for outgoing requests.  In the diagram above, BURP Spider is listening on localhost:9002 (port #), Paros Proxy is listening on localhost:9001, and WebInspect on localhost:9000.  Each tool forwards incoming requests to the next in line (WebInspect, in the diagram above, sends the original request to the target site).     

Paros distinguishes the proxy setting configurations as follows: 

• “Local proxy”:  This is for incoming requests
• “Use an outgoing proxy server”:  This is for outgoing requests

BURP Spider:

• “Proxy running on port”:  This is for incoming requests
• “Use proxy server”:  This is for outgoing requests

WebInspect:

• “Step Mode Listening IP Address and Port”:  This is for incoming requests
• “Proxy server”:  This is for outgoing requests

Below are screenshots of the tools in action with the above configuration.

If you want to get super crazy, you can do exploratory investigating of target websites with the above tools and do it all anonymously with Tor and Privoxy (albeit potentially sacrificing thoroughness due to Privoxy filtering)

 

Wow. I'm amazed how fast you can build web apps with ruby on rails. I've been breezing through the book Agile Web Development with Rails; a couple nights of reading and coding and it's all working. I have lots of little projects I would like to quicky code with a web front-end to help me organize and sift through scan data. I'm going to spin a few up with the rails framework. So far, the book has been great and my sense of speed towards fluency feels fast. Ian is rightfully skeptical of rails in his post; I'm going to run with it for awhile anyway and maybe I'll throw out an interface to help sift through large data sets of nmap results.

Ruby on rails has been getting a lot of attention lately, and rightfully so.  It is cool.  I think it is great to see these kinds of technologies being picked and chosen to do major tasks.   I've always tried to avoid getting religion about programming technologies, just pick the best tool for the job.

So how do you decide to use Rails or Zope over J2EE?  Why do you pick them and what are the wrinkles?   Why not write everything in Python in the first place?   J2EE can be a really nice way to make a web application or any client/server type application,  there are a lot of compelling tools in that space but there can be a learning curve,  I'm not sure if it's becoming easier or if I've been exposed to so many of the different technologies that it's starting to seem easy.  There are enterprise grade tools though.

I have experimented with Zope and it's pretty cool,  there is a good sized community with plenty of code to reuse, it's a well documented system,  it's very easy to quickly produce some fairly cool applications,  the problem I've experienced with it is deployment.   Major versions of Zope aren't always easy to migrate between.  While I love the python programming language,  Zope isn't the only thing in python that can be difficult or troublesome in that regard;  python has no formal application deployment method and by the time I get my workstation setup and running and start deploying python apps I quickly find myself in DLL hell.   While I'm bitching about it,  Perl has this problem too,  there generally is no separation between the language and the libraries that are installed as part of a system. Python app tend to include modules in the standard python directories that are part of python,  there is a "classpath" but most apps don't really do anything with it.  If you change your version of python, you potentially need to change all of the modules, regardless of whether they are pure python or linked to object code.  Not long ago I upgraded GNOME and ended up upgrading dozens of python modules and even python itself because of the python bindings for GNOME and GTK+ and not surprisingly some of the python apps didn't deal with it all without some hiccups.   I understand the problem and it's not entirely Python's problem but it's still annoying.  Python does support loading of classes from zip files but it's still very primitive compared to what Java does with jar files.  It's a nice first step and hopefully python 3000 will start to really leverage it.   I have Java apps that have worked seamlessly when I've changed JDKs under it.

Now that Rails is picking up steam, I've been looking at it to see how app deployment is dealt with.  Is everything thing tightly coupled?   Can I deploy an application with version 1.1 of ruby library foo and then another application on the same system with version 2.0 of library foo?   Java solves those problems and while they may not matter for you blog or your little mom and pop shopping cart,  they can become substantial when an application is in production and a business that is making money is using it.  At a glance it still has the same set of problems that Python and Perl have as far as application deployment go.   Rails apps seem to have a nicer way to deploy apps it's still not there.  If Ruby wants to really separate itself from the free language competition,  they'd develop a robust app deployment model;  I'd switch all the way from python if they did. There is something to be said for just dropping a war or ear file in to a directory and having it unpack itself with all of its libraries and just run.

Skill2Die4 posted a new version of the web application cheatsheet on SecGuru. I downloaded the PDF version and it is available here.

To put it in simple terms I feel sorry for many businesses, especially start-up and small, which must conform to the computer and network security demands listed in some regulations. We're doing policy work for a young startup; their IT staff consists of a few developers and one outside contractor. They have been damned, err, deemed a Service Provider Level 1 entity by Visa’s CISP program. That means they'll have to spend lots of money on high maintenance activities which are usually reserved for much larger organizations.

I’m all for best practices and doing things right, but the cost of playing and doing business in this context is skyrocketing.  If I had a good business idea, I likely would feel very depressed when realizing I must combine the risks of trying something new with the enormous costs of meeting regulatory requirements.  For me, the motivation to operate securely is already there – if you fail, your company will suffer damage or be shut down.  Like with all things that go this way, we get less market choices and more expensive services.

PIX 7.x…. This long awaited release of the Cisco Firewall OS has finally been released and has had time to bake. I have had a chance to use some of its new features and definitely learned a few things. Of course the features of 7.0 were pretty large in itself with the advent of “Bridging Mode” and "Virtual Firewalls" PIX 7.0 is trying to be the all in one firewall. So when you upgrade the firewall a few things become apparent right away. First off the 7.0 command line is much more like a router than it is a traditional PIX. I think this is Cisco further assimilating the PIX technology into the Cisco technology collective. I am personally divided on this approach as it is now important to remember what mode you are in for certain commands where in the old days of PIX slinging you could execute almost all commands in the config mode. Having to traverse config levels like a router makes half your commands in a config file deal with moving around the OS and not real configuration commands.

Here is an example of old vs. new:

Changing interface details on PIX 6.x:



Changing interface details on PIX 7.x:



We had an interesting situation that would warrant the use of PIX 7.0. We had two external internet address spaces on the same physical link. Now the logistics of this project is that one space needs to be administered by one individual and the other by someone else. Now if this was the old days, it would be difficult to allow each person to administer the access list for its space only as the access lists are ultimately the same in PIX 6.x. Enter in Virtual Firewalls… Virtual firewalls was the answer here. Basically you create multiple images of the PIX OS and have each instance administered by the prospective admin. The virtual firewalls bind virtual interfaces to physical interfaces but for security they cannot affect the physical properties of the interfaces. This means that if an admin were to shutdown his virtual interface it would not affect any other virtual interface nor would it affect the physical interface. Also since each virtual firewall is separate each access list is wholly controlled by the virtual firewall admin.

Later I will show how to configure and use this virtual firewall and discuss the drawbacks of it as well.

We use 1and1 for domain registration. It is cheap. The consequence is spending 4 days attempting to contact their support (3x waiting around an hour on the phone only to get dropped), finally getting through, and then receiving the response 'We can't help you because our Administrators in the US are not available'. Lovely. 24x7 support is really 8x5 for help of any value. We'll be switching ASAP.

When I'm ready to spend money on upgrading a computer or purchasing a new one it is always tough for me to decide whether to spend the money on a high-end portable system or to keep my primary workstation in top shape. 90% of my work happens on my workstation, but when I need power on the road, it sucks to work on my antique Dell Inspiron 8100. We have run into several gigs whereby we need to have some portable powerhouses to run the tools we use at any reasonable speed. So now I'm going on the hunt for the best portable 'tools' box to run things like WebInspect, Paros, Nessus, Burp Spider, and other power hungry things. Those vendors selling the more sexy and expensive are:

• Alienware Aurora m7700 laptops
• Voodoo Laptops
• Falcon Northwest
• Hypersonic Laptops

The coolness factor is hip, for sure, but the price is steep for the models sporting workstation like performance - upwards of $5000 or more. Now, Hypersonic offers an option to airbrush your corporate logo on the top - that would be a fun prop.

By the way, these CPU charts are great to help make buying decisions: Intel CPU Chart / AMD CPU Chart.

I just spent a few minutes checking out the guest speakers for the upcoming RSA 2006 Conference in San Jose. This will be my 5th RSA Conference and I've learned it is best to create an agenda around speakers you know to be good versus picking sessions based on a title only. Regardless of topic, I like to see:

Dan Geer, Paul Kocher, Bruce Schneier, Whitfield Diffie (speaker details are here: https://cm.rsaconference.com/US06/catalog/speakers.do)

Around the beginning of 2005 Cory and I offered to help Richard Bejtlich with authoring material for his 'Extrusion Detection' book. Unfortunately our schedule got super heavy and we were unable to commit full energy to it, but I see Richard is speaking this year at RSA so hopefully we'll have a chance to meet in person.

Session topics that look interesting for 2006 are:

• Detecting Security Vulnerabilities through Automated Binary Analysis
• The Inevitability of Security at the Point of Use (Dan Geer)
• How to Break Software Security
• Managing Business Risk via Information Classification
• Beyond "Black Box" Security Penetration Testing
• Tools for Security Risk Assessments (NIST methodologies)

(session abstracts are here: https://cm.rsaconference.com/US06/catalog/eventguide/publicSchedule.jsp)