ClearNet Security

True penetration testing?

tate@ClearNetSec.com (Tate Hansen) — Sun, 04 May 2008 22:45:00 -0600

This from the new PCI information supplement: (regarding the required annual penetration testing for compliance)

The penetration tests should attempt to exploit vulnerabilities […] attempting to penetrate both at the network level and key applications

Really? I laughed when I read this, seriously. It made me think for a second about how many consultants really have the skills to chef-boy-ar-dee exploits under pressure. It’s clear too; this is not about a vulnerability sweep, they want you to bust in.

Penetration testing [..] should occur from both outside the network trying to come in (external testing) and from inside the network.

Wow. True penetration testing from inside the network? How many internal networks have you seen that would survive a blitzkrieg attack from a good penetration test team?

PCI states:

“resources must be experienced penetration testers”

What does that mean?

I’m sure the PCI council is of compos mentis, and I’m not trying to rain on the PCI council or ASVs or QSAs, though it’s funny the council points out that “The PCI DSS does not require that a QSA or ASV perform the penetration test”. That statement wouldn’t be because most of them couldn’t penetration test there way out of a paper bag even if they were handed a loaded metasploit gun, right?

With the huge number of companies bemoaning PCI compliance, I just don’t see most getting a true penetration test. I guess I could be reading too much into this. Maybe the skills bar level I consider for experienced penetration testers is way higher than what the PCI council considers experienced or what others consider experienced or good?

Do you have penetration testing skills? What does that mean to you? Do you think most of the companies that buy a penetration test actually get one?

Predictive markets & betting on when apps or companies get owned

tate@ClearNetSec.com (Tate Hansen) — Wed, 30 Apr 2008 23:30:00 -0600

A recent WSJ article titled “Trading on the Wisdom of Crowds” sparked my interest as it may relate to security. Are there ways to build a business around helping organizations understand the risk to their data assets by using predictive market models? Or maybe building it around betting on commercial applications?

“Betting odds are generally taken as the best indicator of probable results in presidential campaigns," this newspaper explained in 1924.

I’m placing a bet that retail store XYZ gets owned and reveals grandma’s credit card details in ‘08. I’m placing another bet that application ABC will have a remote admin level vulnerability by October ’08.

Alas, we must have more transparency and trust in the publicly disclosed information to play. Participation is key as well:

Predicting markets seem to work so long as there are enough traders whose aggregate information is fully reflected in bets.

Would enough people find it worthwhile to become active traders? Maybe. There was an active predictive market created around the following question:

What will the government's 2007 computer security grade be?

It’s probably a big stretch to build a successful predictive market business around the types of security bets which would benefit organizations. By that I mean if I was responsible for a commercial application in which 75% of the traders were betting on my application being owned within the year, I’d probably work hard to change the odds (i.e. allocate resources to improving the security of my app).

When virtual servers play havoc

tate@ClearNetSec.com (Tate Hansen) — Mon, 14 Apr 2008 15:19:00 -0600

I recorded a tidbit which came from a comment spoken at one of this year's RSA panel tracks. I hadn't thought of this issue on a big scale. It was a comment on how disruptive an environment which frequently "resets" virtual servers as part of normal business is to security.

It's obvious such an environment can have a significant impact on security tools, especially those which strive to learn patterns or look at history or both.

I was just imagining if I was a security admin responsible for a large block of EC2 virtual servers. As part of that, maybe the use of these blocks of servers is similar to a class lab whereby students get to install and do anything they want on the servers. When they're done, the instructor runs around and resets all the servers. Extrapolate this and it can lead to a hard problem, security speaking, for general cases.

I haven't meditated on this issue, but I'm guessing it'll become more visible in short time.

Test commercial web app scanners for free and without restrictions?

tate@ClearNetSec.com (Tate Hansen) — Mon, 24 Mar 2008 11:54:00 -0600

If your software licensing ethics tend to contort a tad here and there, then you may find the below tricks helpful when you want to evaluate commercial web app scanners. Partaking in these tricks is slippery, and you may fall into ethical perdition, so prepare yourself!

Super simple trick #1: Request search and replace proxy

Fire up a proxy that supports request search and replace.

Let’s say an app restricts you to scan only their target site (e.g. demo.testsite.net), but you want to point the scanner to a different target.

No problema.

As shown above, I typed in demo.testsite.net for Search, and blog.clearnetsec.com for Replace. Every HTTP request passing through this proxy with a Request-URI matching the string demo.testsite.net will get replaced with blog.clearnetsec.com. The result? The app scans my blog.

Note: Super simple trick #1 only works for apps which restrict via the hostname. If the app is smarter and adds IP validation, then move along to Sort of simple trick #2.

Sort of simple trick #2: IPTables magic

This trick can add license evading umpf to your smokin’ renegade style. :)

Let’s say the app tries to validate you are only scanning the sites you're licensed for by checking the target IP addresses (regardless of how the hostnames are resolved). For example, maybe the demo version of the app allows you to only scan IP address 55.55.55.55.

No problema.

One line explanation: Setup a linux box to do routing, configure a VIP, and add two IPTable NAT rules.

The long answer.

Setup a linux box. A linux VMware image works too. Configure the network as normal – give it a standard IP, gateway, list of name servers, etc., as you would when configuring any other box on your local subnet.

Once you got that, then check out the script below.

Follow? If not, I’ll explain in more detail.

Again, the first thing you need is a normal networked working linux box.
There are 4 steps to facilitate a destination IP switch-a-roo:

Configure a Virtual IP. Pick an IP located on a different subnet, don’t pick an IP on the same subnet as the primary IP.
Enable IP forwarding.
Setup a DNAT (Destination NAT) rule to replace the destination IP on the fly. The first IP (e.g. 55.55.55.55) is the licensed locked IP. The second IP is (e.g. 216.241.191.205) is the IP you want to scan.
Setup a SNAT (Source NAT) rule to replace the source IP on the fly. The first IP (e.g. 192.168.1.101) is your workstation IP. The second IP is the primary IP address of the linux box.

The last step there, #4, is overloaded. Once you go through the steps above on the linux box, then you need to change the IP address of your workstation with the app scanner installed. You want to pick a new IP for your workstation which is on the same subnet as the VIP you configured on the linux box. You also want to change your workstation to use the linux box as your gateway to the Internet, so change the default route address to match the IP of the VIP on the linux box.

That should do it. Replace the IPs in the IPTable rules above with the IPs that work for you and scan away.

Note: You can’t always do the IPTables trick by itself; one reason is due to virtual hosting. If only one website is being hosted on the IP, then you probably can do this. If the target IP is hosting lots of domains, then you need to chain the request and replace proxy with the IPTables magic to ensure each Request-URI is for the correct host and domain.

For example, GET http://216.241.191.205/... HTTP/1.1 may not be the same as GET http://blog.clearnetsec.com/... HTTP/1.1

Super Simple trick #3: VMware snapshots

Most everyone is likely familiar with this trick. If you have a web app installed in a VMware image and you have a working license (e.g. trial license), but it expires at a certain time and date, the trick is to create a snapshot of the VMware image with the app in a working state.

Anytime you want to scan, change your host OS clock back to a time that is within the licensing window (or ensure your VMware guest image doesn’t sync the clock with the host OS when you restore the snapshot), and restore the snapshot.

Note: Some apps may call home when first launched, so it helps to create the VM snapshot when you have the app open and ready to scan.

Trick addendum:

For those web app scanners which restrict you from scanning SSL enabled sites, have no hesistation, you can work around that too.

One way is via stunnel. From www.stunnel.org.

Encrypted version with STUNNEL

+---------+      | |     +--------+    +---------+
| non-SSL | -ST- | | --- | Apache | -- | non-SSL |
| enabled |      | |     | WITH   |    | enabled |
| client  |      | |     | SSL    |    | server  |
+---------+      | |     +--------+    +---------+
   CLIENT        NET     WEB SERVER      SERVICE

Note the position of STUNNEL : the "-ST-" in the diagram above. Below is an example stunnel configuration:

client=yes
verify=0
[psuedo-https]
accept = 8080
connect = blog.clearnetsec.com:443
TIMEOUTclose=0

Configure your app or browser to use the stunnel proxy listening on port 8080 and you'll be able to hit the site using HTTPS (via the proxy), but your local app or browser will be only speaking HTTP.

Quick alternate to #1 for Apache fan boys and girls:

Via Apache 2.2.x, with mod proxy and mod rewrite enabled, setup a proxy like so:

ProxyRequests On
<Proxy *>
RewriteEngine on
RewriteRule ^(.+) http://blog.clearnetsec.com/$1 [P] (or something close to this)
Order deny,allow
Deny from all
Allow from all
</Proxy>

Configure your browser or app scanner to use this Apache proxy server and all Request-URIs passed through will be re- written to target http://blog.clearnetsec.com/.

YMMV. Have fun.

Us and them development

ian@ClearNetSec.com (Ian S. Nelson) — Fri, 14 Mar 2008 10:57:00 -0600

This actually relates to security. It's round about but it'll tie together.

I was speaking with a college buddy the other day, a guy I respect the hell out of who happens to be an extremely senior engineer and now a fairly wealthy man due to applying that skill in some successful companies. I can't remember the exact thing I said but I was chatting about work, sort of complaining or venting and he called me out for speaking negatively about my current company's QA. I was basically saying that they don't seem to try very hard and he pretty much asked me why they ever would if you treated them like inferiors?

I was a little taken back, I don't think I treat them that way, it's not a conscious thing but clearly there is a caste. I haven't had that many jobs but the 2 places where test was treated as equals and with the same respect as development they also had the same level of responsibility and we ultimately had much much better testing. It works better when everyone takes ownership of the whole product and it generally doesn't work that well when people try to just slice off little pieces and ignore the rest. Everywhere else, most other places, QA were second class citizens, it seems like a normal sort of way of operating. People can rise to the level of expectation, if that level is too low then that's what you'll get.

So how does this affect security? Network ops and marketing usually have very different missions. In the last 10 years, businesses have added security groups and security officers to “add security” to the business, and to basically fill in a hole that nobody else owned. It doesn't work very well. A CSO should be more like an ombudsman. If it's not everybody's responsibility and everybody's job then it simply won't work, there are no tools you can simply buy that will make your business “secure.”

Rogue modems are still plentiful?

tate@ClearNetSec.com (Tate Hansen) — Fri, 15 Feb 2008 22:18:00 -0700

I was doing a scope call for a large grocery chain recently when they mentioned they discover around 20 rogue modems per quarter per division (and they have more than 12 divisions). That number is way higher than I would've guessed, though maybe lots of the modems are legitimate but not on their official roster.

Whatever the case may be, wardialing is not a moribund activity, or as close to it as I thought.

The booming exploit market and bye bye to swaths of products

tate@ClearNetSec.com (Tate Hansen) — Thu, 31 Jan 2008 23:50:00 -0700

There are lots of articles mentioning the Digital Armaments bounty for exploits. I wrote a snippet on the commercial exploit market about a month ago, whereby I was simply listing the prices for subscribing to the different exploit houses.

I guess I forgot to consider another complexity of all this and that is from the influence the organizations who compete to purchase exploits are having (e.g. iDefense, 3COM/TippingPoint, Governments, people and groups w/lots of money).

I wonder how extensive this really goes – I mean, it seems this market is in a boom of sorts which implies there are lots of private exploits trading hands. Exactly how many would be interesting to know. Hell, any numbers would be nice.

One thing is apparent though, if this market continues to grow then how can any security products based on “knowing attacks” succeed? They won't. An IDS vendor is not going to be able to afford to purchase all; no company will have a monopoly.

How not to build a server

ian@ClearNetSec.com (Ian S. Nelson) — Mon, 28 Jan 2008 10:18:00 -0700

A few years ago I was in a job I wasn't particularly fond of. One of the things that I disliked was that it felt like we have a bad heuristic for deciding when to roll our own verison of something and when to use something off the shelf. This is critical for 21st century business, it's laughable the things companies did as experiments last century, the IBM PC was arguably an experiment that IBM was never fully committed, it was never their core competency...

This company did things completely backwards. They'd homebrew things like Linux platforms (or if you were marketing the effort you could call it "hardening" which it really wasn't) and then for the core technology of the products we'd take off the shelf products and try to brand it our own and put our own little spin on it.

One particular episode really stands out to me, we were migrating our offices and we had to start running our own mail server. The old office set us up with an active directory, you'd think we'd just drop in exchange, flip on pop, imap and webmail and call it done. Instead a multi-week process that involved "hardening" a Redhat linux box was undertaken by our Security Architect. Part of this process was default replacing bash with c-shell, for that old skool BSD-like flavor, it puzzled me because in the 2.5 years there I never once saw a c-shell script, most of that stuff was done in perl which sort of makes the shell irrelevant. The thing with c-shell is that nobody but sickos will want to use the system should it ever be compromised so there might be some security value but... Some of the other hardening procedures included system scripts that deleted history and log files on logouts, some odd changes to the default Redhat "lokkit" firewall and then a fairly bizarre process of mirroring authentication from the active directory. I'll be honest, I don't know how it really worked but every account was duplicated and you could actually log in with your AD password. When the project was announced as "finished" we had a real email server that you could get email at and it used your windows password.

It also had a number of interesting side effects, like the fact that I could simply ssh in to it. There was a host based IDS that did record my ssh attempts but every user could ssh in to it, it was inside a fairly friendly environment so that's not terrible but most mail servers you'd get off the shelf wouldn't allow just anyone to log in. Another side effect was that our Windows authentication hashes were all NTLM which made them a bit easier to crack with l0pht crack, this is probably still lost on them, it wasn't clear to me when I left that they'd ever recognize the significance of this or recognize it at all. Then maybe the best side-effect of all was very nature of the mail server and how the custom chopper had been built, it was pure spool files and sendmail (or maybe it was postfix, I don't remember.) One day we stumbled on to this gem: ls -lh /var/spool/mail

...
-rw------- 1 cory users     5.8M 2005-01-24 15:33 cory
-rw------- 1 ian  users     3.4M 2005-01-24 15:33 ian
-rw------- 1 root root      2.8M 2005-01-24 03:31 root
-rw-r--r-- 1 someguy users 10.2M 2005-01-24 15:33 someguy
-rw------- 1 tate users     3.5M 2005-01-24 15:33 tate
...

That's just a recreation, names have been changed. At a glance you might not notice anything. Turns out that "someguy" must have manually edited his spool file with the wrong umask, also turns out that "someguy" was the Security Architect and the initiator of this experiment. His spool file was world readable for months and months. When I first saw it, I immediately read the file with "cat." I deleted my history file and logged out and back in to check if I had left any obvious tracks. Turns out that the system automatically did the same thing for reasons I to this day don't understand, someguy added that logic to the various .exit system files. I waited a couple days for some fallout to see if anything happened, thinking I might have tripped an alarm or left a gory log message but nothing happened and nothing changed so I started downloading "someguy's" email on a daily basis. To my chagrin I learned of another defect.

I could run this command:

ssh ian@mailserver "cat /var/spool/mail/someguy" > someguys.email.dump

and it wasn't even recording a "login" on the HIDS that was running, further I know this because I was reading the emails from the HIDS in "someguy's" email. I shouldn't confess this, it was unethical, but this went on for months. I got to read about who he wanted fired, salaries, messages from his wife about his dog getting sick from eating too many treats, and private messages to his friends about how he hated his job there, messages about how he thought he should be the next CTO and some other guy got the job; there was some good stuff and I could pile on but I won't. I had the whole thing streamlined, I'd dump the file, zip it up and send it to myself at home where I could read it in safety and comfort while drinking a cocktail.

Lessons?

What's funniest to me about this was the amount of effort that went in to being "non-Microsoft," a fresh exchange server would have cost much less man time (and therefore money,) been maintainable, easily outsource-ready and more secure than this "secure" hand rolled linux system. To a degree, the same could be said about simply slamming Suse or Redhat on a box and installing Cyrus or dbmail.
What's also funny is that just about any out of the box Linux without this "hardening" would have been more secure too, I'd have left tracks and marks and various events would have been logged.
None of this had anything to do with what the company was in place to actually do. We could have used hotmail accounts if we had to and that would have been more secure.
There was no thought about when you should be on "main street" and when you should "roll your own" or build a "custom chopper." If there was some clear reason to hand build it then it should be done but there was no competitive advantage or any really compelling reason to do it.
We all tend to go to our strengths when we are given a challenge we may not know the solution for. There is something to be said for actually knowing your strengths. "Someguy" liked to act a BSD guy and liked to be a "security expert" and this project was one of the more clear demonstrations of how little of each he is/was. You have to know what your strengths are and you have to know if they are actually strengths. I can't help but think that most "BSD guys" would have realized how janky the whole thing was long before it was deployed.
Lastly, watch what you say in email, especially at work where you use computers and networks that don't belong to you. Even security experts botch it from time to time.

“Big money! Big prizes! I love it!”

tate@ClearNetSec.com (Tate Hansen) — Fri, 28 Dec 2007 08:57:00 -0700

Smash TV quotes. Love ‘em.

Speaking of big money, the commercial exploit market’s growth isn’t making it any easier to bid on penetration test gigs. If you want to provide the highest assurance you’re capable of to clients, then of course you would like to have your hands on all the exploits out there, both public and private.

product	to start	quarterly	total
d2	$1,950	$850	$5,350
gleg	$1,400	$700	$4,200
argeniss	$1,000	$500	$3,000
canvas	$1,450	$730	$4,370

And the crème of the crop:
Immunity Sec’s Vulnerability Sharing Club $50,000 - $100,000 per year

Attacking with anything less in hand tends toward negligence, especially if you do so without disclosing what you’re missing. Pay to have all and you’ve likely priced yourself out of competitive bids.

The winners here, again, are the attackers.

“Good Luck… you’ll need it!”

Follow-up on using unicornscan for a big scan (400,000+ public IPs)

tate@ClearNetSec.com (Tate Hansen) — Thu, 27 Dec 2007 12:36:00 -0700

I’m happy to report our growing experience using unicornscan for large discovery sweeps is a positive one. Our confidence in using this tool has increased and it is now our preferred weapon of choice for scanning large IP swaths.

To recap: We performed a sweep of 400,000+ public IPs across multiple continents by configuring the scans to do a full TCP port scan of each IP, sustained ~55 Mbits/s using between 3 and 5 systems, and completed it in a matter of days.

This is pretty good considering by sending two SYN probes per port it meant sending ~52.5 billion packets and producing some 3 Terabytes of data.

Nmap is often our preferred tool, and we used it to spot check our results with unicornscan, but from now on it will come down to the details of the gig to make the choice.

Tech note: We avoided problems with table overflows and other like issues by placing the systems directly on the internet and with iptables turned off.