This is a reprint of a blog post I wrote for Product Creation Studio

The Agile Manifesto, published in 2001, describes what the authors felt was most important when developing software:

Manifesto for Agile Software Development We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactionsover processes and tools
Working softwareover comprehensive documentation
Customer collaborationover contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

When I read the manifesto I’m struck by how reasonable the statements are, especially that responding to change is more important than following a plan. What’s the point of following a plan that’s based on out-of-date knowledge or bad assumptions?

The agile methodology is a project management approach that has grown from this manifesto. In agile, the features that the customer wants are listed, often on sticky notes attached to a board in the common workspace. The features are broken into groups that can be implemented in a short amount of time (usually a few weeks) called sprints. At the end of each sprint, a working and tested product is shown to the product owner (i.e. someone who speaks for the interests of the customer) that then gives feedback. This feedback is used to decide what features will be incorporated into the next sprint.

Agile is unlike waterfall in almost every way imaginable. In waterfall one needs requirements, a design, and a plan before you start executing. In agile, all you need to start writing code is a product owner, some developers, a vague definition of the problem you’re solving, and an idea of how you might solve that problem. Requirements might never be gathered together in a document. A product design isn’t created, but evolves. The plan is just a list of the features that will be implemented with a priority ranking, and this list will change at the end of each sprint. Agile focuses on executing short development cycles, creating value, and generating lots of customer feedback. In agile development, the creation of detailed requirements, documentation, and plans are seen as a waste of time because these artifacts constantly change and don’t create value for the customer.

Agile works well when developing a web site or software-as-a-service, where the time to release an iteration is short and the cost of a bad decision is low. These are the types of problems that agile was created to solve. When designing hardware products, agile’s focus on customer input, early and frequent testing, and adherence to creating value are all improvements over the waterfall method of project management. Unlike waterfall, agile does not force a project to stick with a plan when the information that was originally gathered has grown outdated.

One of agile’s strengths is the focus on testing. As code is developed, tests are written that demonstrate the code does what’s expected. Any time a change is made in the code, automated test tools are run, showing that the entire program still works as intended. These tests are often the best form of documentation because if they are out-of-sync with the code, the test will fail. This focus on testing catches errors early, when they are still easy to fix.

What agile does poorly is deal with technical risks, tasks that must be done in a particular order, and long lead-times. Imagine building a skyscraper with an agile approach: while you’re waiting for the crane to be installed, you order the steel and find out there’s a six-month wait for it to be shipped from China; the electrician shows up two weeks after the walls were closed up; the client looks at the half-built building and requests an additional floor for a gym he just decided he wants.

Rapid prototyping tools like 3-D printing have made it easier to bring some of the agile philosophy to hardware product development, but there’s still a fundamental difference in the way hardware and software are developed. With the automated testing and the light documentation requirements preferred by agile, it takes about an hour to release a new iteration of software. An iteration of a hardware product can easily take two months (plus the time needed to change the design) between altering or building manufacturing tools, obtaining all of the parts needed, building and populating new printed circuit boards, scheduling time on a production line, and building new test fixtures.

The agile approach allows one to start developing very quickly, but at the risk of heading down a dead-end and having to start over, which increases project costs and the time to complete. As described in the classic novel Zen and the Art of Motorcycle Maintenance by Robert M. Pirsig, having to rework sucks one’s gumption (the energy needed to do high-quality work). It is better to move forward thoughtfully than need to repeat work because you left a gasket out in an early step of rebuilding a carburetor. Zen’s discussion of gumption and quality should be required reading for all engineers.

One complex hardware project I worked on took the agile approach. They settled on a hardware architecture early on and worked on getting a prototype working as quickly as possible. By the time I joined the project, we had a working prototype where new software features could be tested. The agile approach worked well for the software, but the architecture we chose could not meet our required performance needs without starting over. Since the operating system was linked to the hardware, even the software needed to be redone. A product that should have taken about two years to develop reached the market seven years after project launch.

In developing a hardware product, it is best to have a destination (requirements) and a map (a design and a plan) before one begins a journey. For agile, the destination is vague and the map is non-existent. This works well for software development, where the cost of making a mistake is small. In hardware development, costs can be expensive in both time and money; one needs a process that reduces both the chance of making mistakes and the cost of the ones you will inevitably make.