If You Measure This Then Your Children Will Be Next

Well, that’s just the exception to the rule, that only proves the rule. (My Father, when wrong)

A while ago, I complained about how measuring the wrong thing will just make things worse, in particular, how measuring things in software almost never works.
Recently, I read the earth-shattering news that “Transformers” beat the all-time record for first-week revenue by a non-sequel.
“Transformers” beat the all-time record for first-week revenue by a non-sequel.
“Transformers” beat the all-time record for first-week revenue by a non-sequel.
Hmm. Is it, like, good?
Does it mean Transformers is the best thing since (user-generated) sliced bread?
Is it just a sucky flick?
I don’t know.
Now, the interesting thing is: how did this ever become the news? Why don’t they say “Transformers is the best movie ever”, or “John Torturo as a ‘section 7’ agent is as brilliant as cold fusion”?
And another puzzle, for the advanced reader - why don’t you hear about top grossing movies over a year or ten years’ period? It’s always “first weekend” or “first week”.
Here’s why:
Movie makers are in it for the money, just like the rest of us. And over the last decade or so, there’s this one major obstacle in their little money making business. It’s called piracy. It’s easier to get an illegal copy of a movie than to get some water from the fridge.
In fact, I’m kind of thirsty now. But I’m not going to get that water. I’m too busy downloading Spiderman 4.
Oh, it’s not out yet? No problem. They got it on bittorrent already.
So basically, most money you’re going to make on a movie is in the first week or so. After that, everyone has already downloaded it from the internet for free.
Of course, the money people run the movie, not the…hmm…movie people. That’s because the money people give the movie people money to buy food, and the movie people, well, they don’t give the money people any money.
Now we have money people who run the movie and they want to make some money and they have to do it all in the first week. So they’re going to measure, you guessed it, first week revenue.

Why do we care?
Here’s the problem - to increase that measurement, which is first week revenue, it is not necessary to make a better movie. You just do better marketing, and more hype, and whatever. Because people who see the movie in the theater during the week don’t really get a chance to hear from their friends if it’s good or bad. They go because the trailer is really cool, and the ads are everywhere. And so, the quality may or may not suffer. We don’t know. Because that’s not what we measure.
Here you have an external force (illegal movie copying) creating a random measurement (first week revenue) and potentially driving final product quality down.

And please, don’t measure bugs-per-developer either.

I’ve seen Transformers, by the way. It rocks. Hence my father’s quote.