Posted by Tate Hansen
Mon, 24 Mar 2008 17:54:00 GMT
If your software licensing ethics tend to contort a tad here and there, then you may find the below tricks helpful when
you want to evaluate commercial web app scanners. Partaking in these tricks is slippery, and you may fall into ethical
perdition, so prepare yourself!
Super simple trick #1: Request search and replace proxy
Fire up a proxy that supports request search and replace.
Let’s say an app restricts you to scan only their target site (e.g. demo.testsite.net), but you want to point the scanner to a different target.
No problema.
As shown above, I typed in demo.testsite.net for Search, and blog.clearnetsec.com for Replace. Every HTTP request passing through this proxy with a Request-URI matching the string demo.testsite.net will get replaced with blog.clearnetsec.com. The result? The app scans my blog.
Note: Super simple trick #1 only works for apps which restrict via the hostname. If the app is smarter
and adds IP validation, then move along to Sort of simple trick #2.
Sort of simple trick #2: IPTables magic
This trick can add license evading umpf to your smokin’ renegade style. :)
Let’s say the app tries to validate you are only scanning the sites you're licensed for by checking the target IP addresses
(regardless of how the hostnames are resolved). For example, maybe the demo version of the app allows you to only scan IP
address 55.55.55.55.
No problema.
One line explanation: Setup a linux box to do routing, configure a VIP, and add two IPTable NAT rules.
The long answer.
Setup a linux box. A linux VMware image works too. Configure the network as normal – give it a standard IP, gateway,
list of name servers, etc., as you would when configuring any other box on your local subnet.
Once you got that, then check out the script below.
Follow? If not, I’ll explain in more detail.
- Again, the first thing you need is a normal networked working linux box.
- There are 4 steps to facilitate a destination IP switch-a-roo:
- Configure a Virtual IP. Pick an IP located on a different subnet, don’t pick an IP on the same subnet as the
primary IP.
- Enable IP forwarding.
- Setup a DNAT (Destination NAT) rule to replace the destination IP on the fly. The first IP (e.g. 55.55.55.55) is the
licensed locked IP. The second IP is (e.g. 216.241.191.205) is the IP you want to scan.
- Setup a SNAT (Source NAT) rule to replace the source IP on the fly. The first IP (e.g. 192.168.1.101) is your
workstation IP. The second IP is the primary IP address of the linux box.
The last step there, #4, is overloaded. Once you go through the steps above on the linux box, then you need to change the
IP address of your workstation with the app scanner installed. You want to pick a new IP for your workstation which is on
the same subnet as the VIP you configured on the linux box. You also want to change your workstation to use the linux box as
your gateway to the Internet, so change the default route address to match the IP of the VIP on the linux box.
That should do it. Replace the IPs in the IPTable rules above with the IPs that work for you and scan away.
Note: You can’t always do the IPTables trick by itself; one reason is due to virtual hosting. If only
one website is being hosted on the IP, then you probably can do this. If the target IP is hosting lots of domains, then you
need to chain the request and replace proxy with the IPTables magic to ensure each Request-URI is for the correct host and
domain.
For example, GET http://216.241.191.205/... HTTP/1.1 may not be the same as GET
http://blog.clearnetsec.com/... HTTP/1.1
Super Simple trick #3: VMware snapshots
Most everyone is likely familiar with this trick. If you have a web app installed in a VMware image and you have a working
license (e.g. trial license), but it expires at a certain time and date, the trick is to create a snapshot of the VMware
image with the app in a working state.
Anytime you want to scan, change your host OS clock back to a time that is within the licensing window (or ensure your
VMware guest image doesn’t sync the clock with the host OS when you restore the snapshot), and restore the snapshot.
Note: Some apps may call home when first launched, so it helps to create the VM snapshot when you have
the app open and ready to scan.
Trick addendum:
For those web app scanners which restrict you from scanning SSL enabled sites, have no hesistation, you can work around
that too.
One way is via stunnel. From www.stunnel.org.
Encrypted version with STUNNEL
+---------+ | | +--------+ +---------+
| non-SSL | -ST- | | --- | Apache | -- | non-SSL |
| enabled | | | | WITH | | enabled |
| client | | | | SSL | | server |
+---------+ | | +--------+ +---------+
CLIENT NET WEB SERVER SERVICE
Note the position of STUNNEL : the "-ST-" in the diagram above. Below is an example stunnel configuration:
client=yes
verify=0
[psuedo-https]
accept = 8080
connect = blog.clearnetsec.com:443
TIMEOUTclose=0
Configure your app or browser to use the stunnel proxy listening on port 8080 and you'll be able to hit the site using HTTPS (via the proxy),
but your local app or browser will be only speaking HTTP.
Quick alternate to #1 for Apache fan boys and girls:
Via Apache 2.2.x, with mod proxy and mod rewrite enabled, setup a proxy like so:
ProxyRequests On
<Proxy *>
RewriteEngine on
RewriteRule ^(.+) http://blog.clearnetsec.com/$1 [P] (or something close to this)
Order deny,allow
Deny from all
Allow from all
</Proxy>
Configure your browser or app scanner to use this Apache proxy server and all Request-URIs passed through will be re-
written to target http://blog.clearnetsec.com/.
YMMV. Have fun.
Tags apache, bypass, ClearNet, ClearNet Security, license, licensing, proxy, scanners, security, software, stunnel, Suru, Tate Hansen, vmware, web application | 2 comments
Posted by Ian S. Nelson
Mon, 28 Jan 2008 17:18:00 GMT
A few years ago I was in a job I wasn't particularly fond of. One of the things that I disliked was that it felt like we have a bad heuristic for deciding when to roll our own verison of something and when to use something off the shelf. This is critical for 21st century business, it's laughable the things companies did as experiments last century, the IBM PC was arguably an experiment that IBM was never fully committed, it was never their core competency...
This company did things completely backwards. They'd homebrew things like Linux platforms (or if you were marketing the effort you could call it "hardening" which it really wasn't) and then for the core technology of the products we'd take off the shelf products and try to brand it our own and put our own little spin on it.
One particular episode really stands out to me, we were migrating our offices and we had to start running our own mail server. The old office set us up with an active directory, you'd think we'd just drop in exchange, flip on pop, imap and webmail and call it done. Instead a multi-week process that involved "hardening" a Redhat linux box was undertaken by our Security Architect. Part of this process was default replacing bash with c-shell, for that old skool BSD-like flavor, it puzzled me because in the 2.5 years there I never once saw a c-shell script, most of that stuff was done in perl which sort of makes the shell irrelevant. The thing with c-shell is that nobody but sickos will want to use the system should it ever be compromised so there might be some security value but... Some of the other hardening procedures included system scripts that deleted history and log files on logouts, some odd changes to the default Redhat "lokkit" firewall and then a fairly bizarre process of mirroring authentication from the active directory. I'll be honest, I don't know how it really worked but every account was duplicated and you could actually log in with your AD password. When the project was announced as "finished" we had a real email server that you could get email at and it used your windows password.
It also had a number of interesting side effects, like the fact that I could simply ssh in to it. There was a host based IDS that did record my ssh attempts but every user could ssh in to it, it was inside a fairly friendly environment so that's not terrible but most mail servers you'd get off the shelf wouldn't allow just anyone to log in. Another side effect was that our Windows authentication hashes were all NTLM which made them a bit easier to crack with l0pht crack, this is probably still lost on them, it wasn't clear to me when I left that they'd ever recognize the significance of this or recognize it at all. Then maybe the best side-effect of all was very nature of the mail server and how the custom chopper had been built, it was pure spool files and sendmail (or maybe it was postfix, I don't remember.) One day we stumbled on to this gem:
ls -lh /var/spool/mail
...
-rw------- 1 cory users 5.8M 2005-01-24 15:33 cory
-rw------- 1 ian users 3.4M 2005-01-24 15:33 ian
-rw------- 1 root root 2.8M 2005-01-24 03:31 root
-rw-r--r-- 1 someguy users 10.2M 2005-01-24 15:33 someguy
-rw------- 1 tate users 3.5M 2005-01-24 15:33 tate
...
That's just a recreation, names have been changed. At a glance you might not notice anything. Turns out that "someguy" must have manually edited his spool file with the wrong umask, also turns out that "someguy" was the Security Architect and the initiator of this experiment. His spool file was world readable for months and months. When I first saw it, I immediately read the file with "cat." I deleted my history file and logged out and back in to check if I had left any obvious tracks. Turns out that the system automatically did the same thing for reasons I to this day don't understand, someguy added that logic to the various .exit system files. I waited a couple days for some fallout to see if anything happened, thinking I might have tripped an alarm or left a gory log message but nothing happened and nothing changed so I started downloading "someguy's" email on a daily basis. To my chagrin I learned of another defect.
I could run this command:
ssh ian@mailserver "cat /var/spool/mail/someguy" > someguys.email.dump
and it wasn't even recording a "login" on the HIDS that was running, further I know this because I was reading the emails from the HIDS in "someguy's" email. I shouldn't confess this, it was unethical, but this went on for months. I got to read about who he wanted fired, salaries, messages from his wife about his dog getting sick from eating too many treats, and private messages to his friends about how he hated his job there, messages about how he thought he should be the next CTO and some other guy got the job; there was some good stuff and I could pile on but I won't. I had the whole thing streamlined, I'd dump the file, zip it up and send it to myself at home where I could read it in safety and comfort while drinking a cocktail.
Lessons?
- What's funniest to me about this was the amount of effort that went in to being "non-Microsoft," a fresh exchange server would have cost much less man time (and therefore money,) been maintainable, easily outsource-ready and more secure than this "secure" hand rolled linux system. To a degree, the same could be said about simply slamming Suse or Redhat on a box and installing Cyrus or dbmail.
- What's also funny is that just about any out of the box Linux without this "hardening" would have been more secure too, I'd have left tracks and marks and various events would have been logged.
- None of this had anything to do with what the company was in place to actually do. We could have used hotmail accounts if we had to and that would have been more secure.
- There was no thought about when you should be on "main street" and when you should "roll your own" or build a "custom chopper." If there was some clear reason to hand build it then it should be done but there was no competitive advantage or any really compelling reason to do it.
- We all tend to go to our strengths when we are given a challenge we may not know the solution for. There is something to be said for actually knowing your strengths. "Someguy" liked to act a BSD guy and liked to be a "security expert" and this project was one of the more clear demonstrations of how little of each he is/was. You have to know what your strengths are and you have to know if they are actually strengths. I can't help but think that most "BSD guys" would have realized how janky the whole thing was long before it was deployed.
- Lastly, watch what you say in email, especially at work where you use computers and networks that don't belong to you. Even security experts botch it from time to time.
Tags building, do, it, own, security, servers, software, your, yourself | 3 comments
Posted by Ian S. Nelson
Tue, 16 Oct 2007 23:13:00 GMT
I've had this idea for a book for most of this new century. I left IBM because I didn't feel as if my contributions mattered very much, I proceeded to go to less and less capable software shops where my contributions mattered more and more but not exactly how I wanted and regardless of what I controlled it seemed like more and more stupid things were done. There have been a few shining moments where the places turned out to be collectively more capable than I had thought and there have also been some disasters. After thinking about it, I came up with a plan for a book. My plan has been to record the foolish things I've witnessed over the years, and believe me, I've seen a lot of really stupid stuff, and then produce a tome on the subject and somehow contribute to the cannon of software engineering or computer science by suggesting ways to avoid the mistakes I've seen and even made.
I guess I believe, perhaps foolishly, that if we know our mistakes and why they were made then somehow we can do a better job in the future. The more I look at my log, the more it's looking like my book will just be a bunch of Dilbert-like case studies which isn't what I want. There are just so many really stupid ways to screw up software... While I reformulate my plan, I'm going to blog up some of the lessons I've learned, distilled down and without naming names. Email me if you have some stories, or post them here.
Here are some thoughts on staffing. One thing I've also noticed is that some of the most stupid mistakes are made with honest and sincere good intentions. This might just be a general rule for life, I've never heard someone talking about how they were intentionally doing something stupid, there is always a logic no matter how twisted and screwed up it might be. For example, it's never a good time to fire somebody, there is always a holiday or a release or something coming up, it's usually not a personal problem and you want their kids to have a good Christmas and all that cliche stuff. It just sucks. So I've witnessed the fear of firing turn in to a real joke, I saw a completely incompetent employee kept on the job until "the time is right" and work was fabricated for them. This isn't as easy as you might think, you can't just pull assignments out of a textbook when you do that, it has to apply to the job in some way. At the same time, you've decided to terminate this person and you probably don't want them further contaminating the code base. Once I saw a place hire a contractor to do a job and then put the person who was sitting on the plank on the exact same assignment in parallel with the expectation that they are throwing one copy of the work out. Then some half-brained decision was made to some how incorporate both of the projects because "dead man walking" was upset that we weren't going to use his code. His complaint was unexpected, it was as if he saw through the crafty ruse. So instead of just doing the dirty work, there was a debate on who had better code between an hourly contractor and a soon to be ex-employee who's team didn't want his code. As if Salomon himself decreed it, they cut the baby in half and used both code bases. Even better, the team that knew of the "weak link" had their morale drop because it started looking like they weren't going to replace the guy after all. And the project didn't work quite right either.
The lesson? I've come around and become much more cold blooded on this one. The best day to fire someone is today. If you have firing to do, then get it on. No good ever comes from putting it off. I'm not advocating just firing people because you're having a bad day or something but if you decide to part ways with an employee and you have made a good decision that is well reasoned and thought through, then once you've made that decision you should act. I don't think the people that work at that particular company are stupid, I know many of them are anything but that. Emotion got in the way though, it took a simple but somewhat dirty chore and turned it into an engineering problem.
Tags engineering, software, Termination | no comments
Posted by Ian S. Nelson
Sat, 09 Jun 2007 16:52:00 GMT
I recently went on a vacation to Scotland. While I was there, I read a wonderful
book about
St. Kilda and the near utopian society that lived there until 1930, it is no uninhabited. St. Kilda is one of the most remote Hebrides and it's a particularly rocky set of islands that isn't very easy to approach by boat. It had been inhabited for at least 2000 years and isolated from the rest of the world for most of that time. As a result the islanders depended upon each other a great deal. Everyone had a say on every issue, they shared resources, crime didn't really exist. And then the outside world crashed in on them, people started visiting the island as tourists and eventually the St. Kildans' culture fell apart and they asked to be evacuated from the island.
What was fascinating to me is that Kilda wasn't utpoian by design, it was by need. Everyone depended upon everyone else on the island of only a couple hundred people. There weren't people that just did nothing while other people caught food. (Also fascinating, they didn't fish, they were fowlers and climbed up cliffs and caught sea birds) Every single member of the society pulled their weight and did something that the whole group needed. They had no leader, they had a daily "parliament" in which every man got to talk and an equal vote. While it wasn't easy living, outsides that looked in described the society as a utopia. Even more remarkably, every person on the island was equal when in the rest of the British Isles clans and royalty controlled society.
I compare this to the software industry. At only two companies have I ever seen "the process" work. One was IBM and it worked because everyone was dedicated to making it work and there were a lot of full time process people that pushed the process through as well as a couple super heroes that did above and beyond the call of duty to keep things floating. It was very expensive to make software and, honestly, it wasn't the most fun I've ever had. The other was a small startup where we had almost no management and a very tight team of developers and testers that all wanted to make the company successful; the executive staff was completely hands off. People didn't simply do their job and throw some output over the fence, everyone did more than their job description and there was almost no outside pressure in and we were fairly Agile and had enough process to provide some safety nets but we moved quickly. Also, everyone on the team didn't know anything but success, there was a lot of pride and "good enough" wasn't good enough for us and it was great while it lasted.
Everywhere else, they pretended. Process wasn't a need, so much as an excuse. (The lack of any process is the same thing, it's a process in its own right and there are usually bullies somewhere in the organization and it involves some sort of punishment feedback loop) I had this great experience once where I was encouraged to push back, but not until I exhausted every other option (that meant working 10+ hours 7 days a week, or at least that is how it was measured when you did "push back..") which is silly because it requires you to actually fail to prove that the plan didn't work in the first place. I'm watching it on a great scale now, there is a great effort being made to appear to follow a process but there really isn't one. It's easy for everyone to "do" the process but simply "doing" it doesn't make the product better or reduce risk or really even mean much of anything. It requires collective discipline, collective sacrifice and compromise, collective give and take and a lot of trust. Everyone in the organization has to take part, every single stakeholder has to be part of it. No process can create time when there isn't any or make an average team in to a great team. It's easy to pretend though. I'll talk more about that next time.
Tags agile, engineering, PACE, process, processes, software | no comments
Posted by Ian S. Nelson
Wed, 25 Oct 2006 18:11:00 GMT
I read this
article and was reminded of my own
post a couple months back. I generally try to avoid the religion disputes within the community but this is one I'm feeling more inclined to chime in on. I'm going to make an assertion to clarify the debate: basically, the "successful"
agile projects seem to pretty much build interfaces. I think a lot of developers may be sensitive to that description but that's pretty much what web programming is.
There are some great concepts in agility, there is an overwhelmingly strong tendency for companies to talk about stuff a lot more than actually doing things and at some point the talking always outweighs the effort of just "getting it done." Most teams need a shot of "agile" somewhere. You know, just a kick in the ass to start doing more and talking about doing less. At the same time, I know of no teams that are building things like compilers or operating systems agilely. I know of no widely used protocols that were constructed with agile. I doubt you'll ever see any agile embedded projects in the near future. Basically anything that is actually hard or where mistakes are costly due to limited debugging that might be available or where the customers are so far detached from the actual technologies that they cannot contribute much to the development beyond the interface ( a lot more people use compilers than have anything to add to conversation when it turns to intermediate representations.) Interface security is much different what is meant by security in a protocol. To be fair, I think the discussion should be scoped around this. Agile seems to be at its best in the interface. Maybe it will eventually move beyond that but I don't see any reason to think that will happen soon.
So agile and security... There are a couple things here. First off, security isn't a requirement from customers until it's too late. This is changing but unless they are coached or somehow lead, I doubt most customers would list it high on the list as a fundamental requirement of a new product. It's both implicit and not and when schedules are tight and there are other features that have been sold already it becomes a lot more like a "nice to have."
There aren't a lot of easy ways to automatically test security, I wish there were. More and more software is getting back to the point where there is an automated test suite that will verify functionality, the best projects are still under 100% coverage. I know of no projects that have test suites that verify security after various refactorings. In fact once you trust that something is secure, it's a huge anchor, it's hard to change it and maintain that level of trust. If something some how is perfectly secure, all you can really do is decrease or maintain that by refactoring it.
I also think that the notion that "working software" is the measured deliverable is faulty when you factor security in. Most software works fairly well and accomplishes the goals of the user to some degree, a lot of it also has known insecurities and vulnerabilities. Security is a benchmark above and beyond "working code." In fact most vulnerabilities go unnoticed by the user. Just today we found a hole in our blog software, Typo, that we haven't known about for quite a while and it was being abused. Everything seemed to work just fine though. When security matters, it's a requirement above and beyond "just working."
So are they contradictory like the article on The Server Side asks? I think so if you take agile in a literal sense. If you look at agile as more of a cultural thing rather than a software engineering methodology (and there are a lot of good reasons to do that) then they start to work together more closely, it's a very agile thing to respond to a vulnerability quickly with code, you have to be "agile" to do that. The other change is that security is becoming less and less of a boutique feature, pure security companies are dying and security is becoming just another expectation when you purchase software.
Tags agile, ClearNet Security, Ian S. Nelson, security, software | 1 comment | no trackbacks
Posted by Ian S. Nelson
Mon, 21 Aug 2006 23:51:00 GMT
Does security need to be baked in to a product from the very start? Or can you add it after the fact? Does the development model affect this? I think this is a really interesting question. My instinctual answer is that to build a properly secured application or system of applications you have to plan for it from the start. That's the parochial answer and I'm not sure that's it is 100% correct. The last few years various iterative development models have become more popular and while I cannot point to any examples of great success coming from that I also can't point out any failures that could have been avoided with any other development model. The trend is to rapidly develop with little or no up-front design, adapt to changing requirments dynamically and rapidly fix problems as they arrise. Do these development models lead to less secure products by their very nature?
The logical follow-up is how do you iterate security in to a product after the fact, if that's a valid way to do it? Any thoughts or experiences?
Tags engineering, security, software | 2 comments | no trackbacks
