ClearNet Security

I've had this idea for a book for most of this new century. I left IBM because I didn't feel as if my contributions mattered very much, I proceeded to go to less and less capable software shops where my contributions mattered more and more but not exactly how I wanted and regardless of what I controlled it seemed like more and more stupid things were done. There have been a few shining moments where the places turned out to be collectively more capable than I had thought and there have also been some disasters. After thinking about it, I came up with a plan for a book. My plan has been to record the foolish things I've witnessed over the years, and believe me, I've seen a lot of really stupid stuff, and then produce a tome on the subject and somehow contribute to the cannon of software engineering or computer science by suggesting ways to avoid the mistakes I've seen and even made.

I guess I believe, perhaps foolishly, that if we know our mistakes and why they were made then somehow we can do a better job in the future. The more I look at my log, the more it's looking like my book will just be a bunch of Dilbert-like case studies which isn't what I want. There are just so many really stupid ways to screw up software... While I reformulate my plan, I'm going to blog up some of the lessons I've learned, distilled down and without naming names. Email me if you have some stories, or post them here.

Here are some thoughts on staffing. One thing I've also noticed is that some of the most stupid mistakes are made with honest and sincere good intentions. This might just be a general rule for life, I've never heard someone talking about how they were intentionally doing something stupid, there is always a logic no matter how twisted and screwed up it might be. For example, it's never a good time to fire somebody, there is always a holiday or a release or something coming up, it's usually not a personal problem and you want their kids to have a good Christmas and all that cliche stuff. It just sucks. So I've witnessed the fear of firing turn in to a real joke, I saw a completely incompetent employee kept on the job until "the time is right" and work was fabricated for them. This isn't as easy as you might think, you can't just pull assignments out of a textbook when you do that, it has to apply to the job in some way. At the same time, you've decided to terminate this person and you probably don't want them further contaminating the code base. Once I saw a place hire a contractor to do a job and then put the person who was sitting on the plank on the exact same assignment in parallel with the expectation that they are throwing one copy of the work out. Then some half-brained decision was made to some how incorporate both of the projects because "dead man walking" was upset that we weren't going to use his code. His complaint was unexpected, it was as if he saw through the crafty ruse. So instead of just doing the dirty work, there was a debate on who had better code between an hourly contractor and a soon to be ex-employee who's team didn't want his code. As if Salomon himself decreed it, they cut the baby in half and used both code bases. Even better, the team that knew of the "weak link" had their morale drop because it started looking like they weren't going to replace the guy after all. And the project didn't work quite right either.

The lesson? I've come around and become much more cold blooded on this one. The best day to fire someone is today. If you have firing to do, then get it on. No good ever comes from putting it off. I'm not advocating just firing people because you're having a bad day or something but if you decide to part ways with an employee and you have made a good decision that is well reasoned and thought through, then once you've made that decision you should act. I don't think the people that work at that particular company are stupid, I know many of them are anything but that. Emotion got in the way though, it took a simple but somewhat dirty chore and turned it into an engineering problem.

We’ve hit a new high. We’ve soaked ourselves in a bandwidth bath on behalf of a client whom would like us to discover active services across a range of six public /16 blocks plus some scattered /17s, /24s, etc. The range is close to a total of 400,000 IPs.

We started out with five dual Xeon systems running 20 to 40 instances of nmap, each tuned, and each instance targeting 64 IPs. This client wants the job completed in weeks, so we decided it was a good time to get more experience with unicornscan.

By luck, we tapped into a Danish provider that is allowing us to push 55Mbits/s. I have no idea how much that amount of bandwidth would normally cost, especially if sustaining it 24x7 for a few weeks, but I’m guessing it is way over $10,000. Our client would allow us to go up to 100Mbits/s, alas, our luck doesn’t go that far.

Anyway, we now have faster dual-core systems each pushing ~25 Mbits/s via unicornscan like so:

sudo nohup /usr/local/bin/unicornscan -mT -p –r25000 -vv xxx.zz.0.0/16:a -w unicorn.output.for.xxx.zz..0.0.fullTCP > unicorn.output.fullTCP &

We have lots of results from nmap; so far unicornscan is matching the nmap results. Having the ability to specify packets per second with unicornscan is super nice.

We’ll create a follow up post on how all our scanning worked out on this gig when we’re finished (sometime in late November).