The Danger of Local Optimisation

In the book 'The Goal' the protagonist discovers that within his factory, heavily optimised local production is causing large stockpiles of half-finished goods. This has far reached implications for the business, including skyrocketing complexity dealing with buffers of items and additional costs associated with half manufactured items.

In the production of software at its very core we have similar problems. Not only between departments that are of a similar technical ilk, but between separate parts of the business such as sales and marketing.

These buffers (or droughts) of value between teams are stopping the customer from receiving value. Let's look at a couple of scenarios and see why this can be a problem for a company that has many steps in a products preparation before the customer can take value from it.

Possible setup

Though not the same for every organisation. This might seem familiar.

• The customer has a problem to solve and engages us to solve it, possibly via a sales team.
• Analysts and technical leads talk to them about it and break down their needs
• We all create stories to cover the solution
• Engineering incrementally works towards a solution
• While QA makes sure it is up to standards and hits the requirements
• The customer delights in their value, neatly delivered and functional.

Agile working style means the main bulk are actually a loop of incremental deliverables, but the concept is sound. I'm trying to illustrate that there are steps, they happen in order and that different people do those steps. Sometimes they are parallel.

The customer

• What if they do not understand their problem yet and delay?
• What happens when they talk to us, then abandon a project and then return? Are we on standby?
• What about if they are held up by a third party?
• What if their points of contact are difficult to deal with?

Just because this is the start of the process it does not mean that there cannot be delays, in fact this first step can be silently the most complex, because these are some things, we cannot help a customer with: their own internal politics and process.

We can make it easier for them to engage though, but this is not the focus here. Let's focus on some antipatterns.

Analysis of the customer problems

Local Optimisations

• Saturated sales pipeline waiting for analysis
• Excessive bugs or issues coming back to haunt us
• Unfiltered client requests for features in the platform
• Quick handoffs without correct analysis
• The analysis and relationship are optimal, accurate and lots of work is queued up
• Aggressive client led deadlines
• Aggressive internally chosen deadlines

Bear in mind that there are two reasons we locally optimise:

• To genuinely push good work, that is well thought out into the production pipeline
• To make a department look good by churning numbers and skipping steps, seems like optimisation from the outside.

These failures in analysis will result in work being stockpiled in front of the next step and a backlog occurring.

Engineering

For engineering there are a few things that can cause work to pass through quicker. Some good, some not so good. Regardless, these are all things that reduce the amount of time value spends here.

Local Optimisations

• Skipping tests
• Skipping documentation
• Having a strong team that finishes work quickly.
• Using frameworks or tooling that only benefits engineering.
• Having a high performing engineer do all the hard work
• Having specialists do specific tasks instead of everybody becoming more of a generalist

This is not a full list, but you get the idea. These are things that enable work to go through quicker, for better or worse. This makes work pile up elsewhere. It also has the effect of reducing quality in some of the negative cases.