Why do we make processes?

ian@ClearNetSec.com (Ian S. Nelson) — Sat, 09 Jun 2007 10:52:00 -0600

I recently went on a vacation to Scotland. While I was there, I read a wonderful book about St. Kilda and the near utopian society that lived there until 1930, it is no uninhabited. St. Kilda is one of the most remote Hebrides and it's a particularly rocky set of islands that isn't very easy to approach by boat. It had been inhabited for at least 2000 years and isolated from the rest of the world for most of that time. As a result the islanders depended upon each other a great deal. Everyone had a say on every issue, they shared resources, crime didn't really exist. And then the outside world crashed in on them, people started visiting the island as tourists and eventually the St. Kildans' culture fell apart and they asked to be evacuated from the island.

What was fascinating to me is that Kilda wasn't utpoian by design, it was by need. Everyone depended upon everyone else on the island of only a couple hundred people. There weren't people that just did nothing while other people caught food. (Also fascinating, they didn't fish, they were fowlers and climbed up cliffs and caught sea birds) Every single member of the society pulled their weight and did something that the whole group needed. They had no leader, they had a daily "parliament" in which every man got to talk and an equal vote. While it wasn't easy living, outsides that looked in described the society as a utopia. Even more remarkably, every person on the island was equal when in the rest of the British Isles clans and royalty controlled society.

I compare this to the software industry. At only two companies have I ever seen "the process" work. One was IBM and it worked because everyone was dedicated to making it work and there were a lot of full time process people that pushed the process through as well as a couple super heroes that did above and beyond the call of duty to keep things floating. It was very expensive to make software and, honestly, it wasn't the most fun I've ever had. The other was a small startup where we had almost no management and a very tight team of developers and testers that all wanted to make the company successful; the executive staff was completely hands off. People didn't simply do their job and throw some output over the fence, everyone did more than their job description and there was almost no outside pressure in and we were fairly Agile and had enough process to provide some safety nets but we moved quickly. Also, everyone on the team didn't know anything but success, there was a lot of pride and "good enough" wasn't good enough for us and it was great while it lasted.

Everywhere else, they pretended. Process wasn't a need, so much as an excuse. (The lack of any process is the same thing, it's a process in its own right and there are usually bullies somewhere in the organization and it involves some sort of punishment feedback loop) I had this great experience once where I was encouraged to push back, but not until I exhausted every other option (that meant working 10+ hours 7 days a week, or at least that is how it was measured when you did "push back..") which is silly because it requires you to actually fail to prove that the plan didn't work in the first place. I'm watching it on a great scale now, there is a great effort being made to appear to follow a process but there really isn't one. It's easy for everyone to "do" the process but simply "doing" it doesn't make the product better or reduce risk or really even mean much of anything. It requires collective discipline, collective sacrifice and compromise, collective give and take and a lot of trust. Everyone in the organization has to take part, every single stakeholder has to be part of it. No process can create time when there isn't any or make an average team in to a great team. It's easy to pretend though. I'll talk more about that next time.

Agile vs. Security

ian@ClearNetSec.com (Ian S. Nelson) — Wed, 25 Oct 2006 12:11:00 -0600

I read this article and was reminded of my own post a couple months back. I generally try to avoid the religion disputes within the community but this is one I'm feeling more inclined to chime in on. I'm going to make an assertion to clarify the debate: basically, the "successful" agile projects seem to pretty much build interfaces. I think a lot of developers may be sensitive to that description but that's pretty much what web programming is.

There are some great concepts in agility, there is an overwhelmingly strong tendency for companies to talk about stuff a lot more than actually doing things and at some point the talking always outweighs the effort of just "getting it done." Most teams need a shot of "agile" somewhere. You know, just a kick in the ass to start doing more and talking about doing less. At the same time, I know of no teams that are building things like compilers or operating systems agilely. I know of no widely used protocols that were constructed with agile. I doubt you'll ever see any agile embedded projects in the near future. Basically anything that is actually hard or where mistakes are costly due to limited debugging that might be available or where the customers are so far detached from the actual technologies that they cannot contribute much to the development beyond the interface ( a lot more people use compilers than have anything to add to conversation when it turns to intermediate representations.) Interface security is much different what is meant by security in a protocol. To be fair, I think the discussion should be scoped around this. Agile seems to be at its best in the interface. Maybe it will eventually move beyond that but I don't see any reason to think that will happen soon.

So agile and security... There are a couple things here. First off, security isn't a requirement from customers until it's too late. This is changing but unless they are coached or somehow lead, I doubt most customers would list it high on the list as a fundamental requirement of a new product. It's both implicit and not and when schedules are tight and there are other features that have been sold already it becomes a lot more like a "nice to have."

There aren't a lot of easy ways to automatically test security, I wish there were. More and more software is getting back to the point where there is an automated test suite that will verify functionality, the best projects are still under 100% coverage. I know of no projects that have test suites that verify security after various refactorings. In fact once you trust that something is secure, it's a huge anchor, it's hard to change it and maintain that level of trust. If something some how is perfectly secure, all you can really do is decrease or maintain that by refactoring it.

I also think that the notion that "working software" is the measured deliverable is faulty when you factor security in. Most software works fairly well and accomplishes the goals of the user to some degree, a lot of it also has known insecurities and vulnerabilities. Security is a benchmark above and beyond "working code." In fact most vulnerabilities go unnoticed by the user. Just today we found a hole in our blog software, Typo, that we haven't known about for quite a while and it was being abused. Everything seemed to work just fine though. When security matters, it's a requirement above and beyond "just working."

So are they contradictory like the article on The Server Side asks? I think so if you take agile in a literal sense. If you look at agile as more of a cultural thing rather than a software engineering methodology (and there are a lot of good reasons to do that) then they start to work together more closely, it's a very agile thing to respond to a vulnerability quickly with code, you have to be "agile" to do that. The other change is that security is becoming less and less of a boutique feature, pure security companies are dying and security is becoming just another expectation when you purchase software.

ClearNet Security: Tag agile

Why do we make processes?

Agile vs. Security