Keeping Score, continued

Let me expand on the idea that we should track the accuracy of commentators. Too often, it seems like these people are using debate-team rules, where the goal is to out-argue the other guy using rhetorical tricks, and real-world truth is lost in the balance.

For example, take the current debate over whether global warming is real. This lends itself well to making specific, measurable predictions. For example, the average world-wide temperature in year X will be Y degrees, or by year Z the ocean level will have risen W inches. Make the predictions and results public. Do that, and I predict – no, I guarantee – that you will see a lot more waffling among the naysayers, as well as the enviro-extremist-doomsters. The overall quality of the discussion would go up, and maybe some of the people who are consistently, egregiously wrong would have to find more appropriate careers.

I’ve been thinking about this in terms of my job as well. My job is all about trying to make good decisions, which requires anticipating the effects of the software my team writes. And we frequently get into debates over the right course to take. But my motives aren’t always pure – besides my appraisal of the future, there is also what the decision means to me personally, i.e. how much extra work do I have to take on. And the same applies everyone. So I’ve started asking myself whether my decisions would change if I had money on the line. And interestingly, sometimes they would.

Which leads to an interesting thought experiment. What if a company gave everyone a budget, say a couple hundred bucks, to use in an internal sort of futures market? So for example, I’m designing some feature that doesn’t support a certain obscure use case. A colleague calls me on it, and I say, well, no customer would ever try that use case. At that point, if they really wanted to challenge me, we could put money on it; if I was too chicken to do that, then it meant I had to acknowledge that my design was faulty and fix it. Obviously, there would be a lot of pitfalls and details to be worked out, but it’s an interesting idea.