Measurement Paradox Continued – Avoid Short-term Goals and Measurements

This post is a continuation of two previous posts: Solving the Measuring Paradox and If You Measure This Then Your Children Will Be Next.

When setting quantity-based goals or metrics, it is usually better to measure over a period of time that is long as possible.
Consider the following example: suppose you want to motivate developers on your team to resolve as many bugs as possible. So you start measuring the number of bugs closed by developer and set the following goal: each developer must resolve at least two bugs a day.
Here’s what will happen: Alice, a developer on your team resolves her second bug of the day at 2PM. By 4PM she already has a fully tested solution to her third bug. Alice, being the perfectly logical creature she is and aware of the goal you set, will not check in her fix but rather wait until the next day. This way she will make it easier for herself to reach her goal tomorrow as well. In fact, your developers will start accumulating these “spare” bugs “just in case”.
I’ m against the measurement tactics altogether, but if you have to id it, it is much better to define the metric for a longer term.
Suppose that instead you define the goal to be “40 resolved bugs per month” (I’m assuming 20 work days per month here). This way, your developers still have more of an incentive to work on bugs throughout the period, because “good days” (with lots of resolved bugs) will balance with the “slow days”.

“What’s the difference?” you say. “This way or the other, 40 bugs have to be resolved over the month”.
Not exactly: first – the fact that bug fixes are being “held up” for later, stalls other things (such as other people waiting for the fix). Second – under the first version of the metric, a developer who already has a few bugs “reserved” has less incentive to work hard on new bugs, whereas under the second version of the metric, assuming the metric is not much less than the developer “usual” throughput, the “slow days” balance out the “good” ones and the developer has motivation to work on bugs to meet the long-term goal.

Here are two interesting analogies from the world of sports:
The football championships in Argentina have a unique (and genius IMO) system for relegation from and promotion to the top league: instead of relegation based on the performance in the last season (number of points) like everywhere else, teams are relegated based on their performance over the last three seasons. This avoids the common scenario where teams placed in the middle of the table have “nothing to play for” at the end of the season and games involving these teams tend to be rather boring.

Another example: players’ contracts often consist of a base salary and some performance based bonuses. These bonuses are always set based on a long period (e.g. average points scored over the whole season) and not a single game (bonus for scoring over X points in a single game).