And why we have it all wrong.
The private sector has a universal scoreboard. There is only one goal: have more tomorrow than you have today. You won’t hear investors or entrepreneurs or CEOs debating how to define success. It’s understood by everyone.
The public and non-profit sectors don’t have a scoreboard. This isn’t our fault. Our work is distributed, nuanced, and complex. What is our fault is that we keep wasting time trying to build one.
When we try, we end up with two things:
1.) Quantitative measures (web traffic, bodies in a room, test scores) that can easily be mistaken for impact; and
2.) Qualitative measures (feedback, expert observations, personal stories) that prove impact in specific instances, but don’t provide a definitive ‘Yes, we were successful.’
Businesses use metrics to measure efficiencies and conversion rates. To track how employees are performing. To see if advertising dollars are being well spent. Metrics inform the path to success.
We, on the other hand, attempt to use metrics to define success. To have the metrics be our scoreboard. And then screw ourselves by working towards a standard that almost, but doesn’t really, indicate success.
To end the interminable ‘metrics’ discussions and put a lot of great people back to work on the stuff that matters, I suggest the following consensus:
Give up. Accept the scoreboard will never exist.
Instead, do one of two things:
1.) Define your mission as something that can be measured, success as achieving the desired score, and stop there. Make the metric the mission, not an indicator of the mission. Increase graduation rates by 5%. Reduce hospital wait times by 20 minutes. Get 100km above the earth.
2.) Or, if you’re particularly enlightened, agree that you’ll know it when you see it. That everyone working on the project is smart, well-intentioned, and will try to make the right decisions. Use metrics to help with those decisions.
We’ll be happier, our resources will be put to better use, and we’ll all stop chasing our tails.
UPDATE: Working on a constructive follow-up post mashing Most Significant Change with iterative development and open funding models.
Geoffrey MacDougall 
Interesting take on things – I’d like to know what inspired it. I mean, what niggling examples do you have of metrics indicating success and the not-for-profit organization being screwed in the process? Not sure if I agree with your underlying assumption. More please. :)
Remixed it a little. It’s more that a reliance on tangential / parallel metrics to define success is inherently flawed. That an effective problem statement either (i) has to be defined by metrics from the beginning or (ii) agreed to be evaluated in hindsight.
There is also the issue of the effective underlying problem statement itself. Nail the clearly defined problem statement upstream and some of the metrics issues downstream seem to vanish.
I attended a large business’ fast talking super rich owner’s office in manhatten to seek support for Shelternet for Abused Women, a site that connected women to the 550 shelters in Canada and Shelters to each other.
His opening question was how do you know you actually help anyone? I spouted our google tracked #’s and hits and uniques vrs regular users and he was completely underwhelmed with my pitch and threw me out saying until he knew we actually helped women and children he’d do a big fat zero dispite his Canadian Sr VP had been on Board for past 6 years.
I though he was a jerk….many wouldn’t so my question is: was he write to push us to a higher level of accountability or was he seeking an excuse, any excuse, not to give?