Us and them development

ian@ClearNetSec.com (Ian S. Nelson) — Fri, 14 Mar 2008 10:57:00 -0600

This actually relates to security. It's round about but it'll tie together.

I was speaking with a college buddy the other day, a guy I respect the hell out of who happens to be an extremely senior engineer and now a fairly wealthy man due to applying that skill in some successful companies. I can't remember the exact thing I said but I was chatting about work, sort of complaining or venting and he called me out for speaking negatively about my current company's QA. I was basically saying that they don't seem to try very hard and he pretty much asked me why they ever would if you treated them like inferiors?

I was a little taken back, I don't think I treat them that way, it's not a conscious thing but clearly there is a caste. I haven't had that many jobs but the 2 places where test was treated as equals and with the same respect as development they also had the same level of responsibility and we ultimately had much much better testing. It works better when everyone takes ownership of the whole product and it generally doesn't work that well when people try to just slice off little pieces and ignore the rest. Everywhere else, most other places, QA were second class citizens, it seems like a normal sort of way of operating. People can rise to the level of expectation, if that level is too low then that's what you'll get.

So how does this affect security? Network ops and marketing usually have very different missions. In the last 10 years, businesses have added security groups and security officers to “add security” to the business, and to basically fill in a hole that nobody else owned. It doesn't work very well. A CSO should be more like an ombudsman. If it's not everybody's responsibility and everybody's job then it simply won't work, there are no tools you can simply buy that will make your business “secure.”

"Us and them development" by Andre Gironda

Fri, 14 Mar 2008 13:47:59 -0600

A traditional model, which is often the result of a primarily waterfall-based SDLC (or other classic SDLC) usually splits software development and SQA.

In this modern era of programming, where developers utilize practices such as Continuous Integration, Refactoring, the Dependency Injection pattern, and TDD/BDD - this “traditional” approach often does not work.

First, SQA is thought to be the validation or “check” for software development. If software developers utilize Continuous Integration, this means that developers are likely to also be doing Fagan inspection (or peer review of some kind) on all code check-ins, or component check-ins. At this point, SQA becomes redundant.

Continuous Integration and TDD also assume that developers are doing unit testing in their daily/nightly builds. This turns some or all developers into “developer-testers” which replace the need for unit testing done by SQA. This comes even more into play if the developers are doing continuous-prevention development and/or refactoring.

Worst of all for an already marginalized SQA team, the developers could also be doing all of the functional testing using Continuous Integration. Not only are unit tests written by developers, but also all functional tests using an automated testing tool such as Canoo WebTest driven by something like an Ant task at build time.

What does this leave SQA to do? Well, there is regression testing (replaced by continuous-prevention development) and finally - acceptance testing. Would an organization want to keep a SQA or SQC team around just to perform acceptance testing? Maybe. It’s my opinion that developer-testers can also do acceptance testing and thus - full elimination of all SQE’s is possible (however, it could naturally be that many SQE’s fill the developer-tester roles). In some cases, where extensive or very user-driven testing is necessary (or where it just doesn’t fit the culture), SQA/SQC should be kept around to perform acceptance testing.

The best place to put all of your current SQE’s is into test case/charter roles. Test cases are created in the earliest phases of the life cycle: planning and requirements gathering. Using the V-Model, test cases that will apply to any software project should be started before the software engineers sit down to decide on the design decisions.

Test case development isn’t the only way to provide testing throughout the life cycle. The best SQE’s today use techniques such as exploratory testing - however I also feel that this is best done during the programming phase (or the integration phase!) and not during a separate, post-build phase. Exploratory testing creates and works from a test charter, which is often everything that the test cases and unit tests are missing and more.

Exploratory testing takes into factors that involve the application as it is built and how it works internally besides all of the boundary value analysis, input validation, and code metrics. Domain testing and combinatorial explosions (especially using all-pairs testing) make good candidates for exploratory testing.

In summary, quality testers need to redefine their roles in this new era of TDD. There are many places for current quality people in early development lifecycle work, and there are many places to put “newbie” people (i.e. people with no computer science degree, experience, or certification). It’s probably best to list the positions still as SQE (software quality engineer), but make the role as a developer-tester. This is what Google and others do. For those that have quality tester certifications, utilize these people where they can provide the most benefit - such as requirements gathering and exploratory testing.

Yes, this means that SQA/SQC will have to integrate and work well with developers (and vice-versa). A documented and clean Fagan inspection process is necessary in order to make this successful.

ClearNet Security: Us and them development

Us and them development

"Us and them development" by Andre Gironda