Documentation has always been a weak spot in software systems - at least in all the systems I've been involved with. Neil McAllister brought up the topic again an Infoworld article and I realized that the problem is that most engineers don't understand why documentation is important. Simon Sinek's TEDx talk is a great explanation of why the why is important: without the why, people just aren't fully motivated to follow a process. So, why write documentation?

When I talk about documentation, I'm not talking about code comments. I agree with the school of thought that good engineers can just read code and thus don't need a lot of comments in the code. When I talk about documentation, I'm talking about explaining the design decisions that cannot be expressed in the code because the code itself is a point solution to that design space.

Software design is the art of taking the universe of all possible solutions and reducing it down to one precise solution: the bits that execute on the machine. The code, being that single precise solution, cannot explain why the designer chose, e.g., MySQL. Was that a deliberate decision because MySQL had a feature that was important to the system architecture, or an accidental decision because it happened to be what the first engineer knew how to use? Was the decision SQL over NoSQL, or MySQL specifically over other SQL databases? These decisions, and our desire that other current and future engineers understand them, are the reason for documentation.

Once we know the reason for documentation (to describe the design choices), it's easy to be motivated to write because if we don't write that documentation, our code, our legacy of excellence, will melt away and become yet another big ball of mud.

It's very common for engineers to work hard on a big release, finally get it out the door, and then take off for the beach. And who's to say "no" to a group that has worked hard to deliver for the company?

Well, I am going to say "no" because post-release is a really bad time to take a vacation because post-release is not the end of the project, it's actually just the midpoint! The conventional wisdom is that engineering effort ramps up to a maximum, then back down to the release:

In reality, the low points aren't at the releases because putting the system in customer's hands leads to a lot of questions from customers. Customers find defects, but even if your software is bug-free, customers find ways to use your system that you hadn't anticipated. Perhaps they are trying it on a different variant of Linux than you tried and it's logging interesting messages; perhaps they are using it to solve a problem that you hadn't considered and it's not as perfect for that; perhaps they are in Ecula, Australia and your timezone software hadn't anticipated a 45-minute offset; etc.

In practice, releases are somewhere in the mid-point of engineering effort. Post-release is more of an outward-facing effort where the pre-release is more inward-facing, but it's still effort:

The net net of this reality is that the team shouldn't plan a big vacation starting Saturday after the Friday release and upper management shouldn't plan an immediate post-release re-tasking of the team onto the next big thing. After all, your customers are the reason you're doing these releases, so why would you reduce your effort for your customers right when those customers are starting to use your new system?

It's the New Year and it's time to start a bunch of New Projects! One of my techniques for successful software projects is to start at version zero, also known as starting at the end. Starting at version zero means that the very first thing you do is build the delivery mechanism: the makefiles, the continuous integration, the packages, the deployment scripts, the monitoring tools and even tiny single-server staging and production systems. Version zero is the complete end-to-end, but deliberately content-free, "hello world" for your application.

Starting at version zero means that you're always ready to ship because every build is shippable - admittedly your version zero is not very useful, but it is shippable. And since you've built the deployment mechanism first, you won't be trying to get it all working under a crunch deadline at the end. In fact, starting with version zero helps reduce the need for crunch mode at all because the decision becomes "do we work overtime to ship with ten features instead of nine?" rather than "do we work overtime to ship instead of not-ship?"

Starting with version zero also means that you're always ready for a demo or a beta test, and it means that you're not waiting until the end to discover integration issues. So, start your new projects with version zero and deliver more with less stress.

Agile software development is mainstream. The productivity and quality advantages of short iterations, test driven development, and continuous integration are too large to ignore. But if these practices are so good, why don't companies apply them to more than just software? Some do: the Lean Startup movement is about applying those same principles to your business model. The continuous feedback in short iterations has proven just as effective for business models as it has for software.

But what about the third leg of the stool: your people? Why aren't businesses applying the same agile methods to developing their "greatest resource"? When I read about Microsoft's annual circus of employee disempowerment or when I talk to colleagues, even those at other small start-ups, about their annual self-reviews which are then promptly ignored, I cringe and avert my eyes.

At New Relic, I've established a policy of Agile Personnel Development. We use constant, honest, feedback on short cycles - basically it is "continuous review deployment". Keeping the intervals short has all the same benefits that it does in agile software development: deviations are minimized and feedback stays relevant. Our engineers advance in their skills and in their careers faster than those at companies which practice "Waterfall Annual Reviews" on their employees.

Our reviews are not just "list five objectives for the year" because our managers are actually spending time with the engineers, listening to where they want to learn and grow, and tailoring the feedback to help them do so. Some engineers want to move into engineering management, while others want to develop Principal Engineer level technical skills; some want to work on their public speaking; others want to learn more about business so that they can start their own start-up.

We use two nested cycles of reviews: weekly or bi-weekly focused 1-1s and quarterly written reviews. We've tried even more frequent reviews (e.g., monthly) but found that the kind of improvements our engineers were interested in took a little longer and thus quarterly was the most useful. There's nothing magical about our reviews other than the frequency and the care that we put into them. It's a serious time commitment by the managers, but the benefit is well worth the effort.

just as our engineers help the business by adhering to a rapid iteration agile software development process, we managers help our engineers by taking the time to do practice Agile Personnel Development. And, heck, it's a beneficial cycle because better, happier engineers means more productive efficient technical solutions which, of course, benefits the business.

New Year's resolutions and all that get me thinking about "conventional wisdom" in software engineering and why I have a despair.com poster on my office wall as inspiration. True, despair.com Demotivators™ are supposed to be spoofs of motivational posters but in one or two cases, they fail at that and become actually motivational. In this case:

As an engineer, I always want to improve on what I've done before so when I ask about a design or algorithm or process and the answer is "that's the way we do it" I used to have to bite my tongue. Now I just point to my poster and say "yeah, so what?" and then "tell me why". There could be a good reason that we do it that way; or it might be that we don't have a good reason.

For example, at one point we were doing our deploys late at night. Conventional wisdom was that we needed to do this to keep the site up and our customers happy. With a little thinking about headroom and migrations, and a switch to rolling restarts, we changed our deploys to the daylight hours, freeing our engineers and ops folks to spend more time with their families.

My job - our job - is to constantly improve what we do and that means questioning everything. Don't accept "we've always done it that way". Grab the New Year and start making things better!