Getting in Shape, Just in Time for Summer

Introducing Bell Curve Grading

Posted by Charlie Moran on 04.29.09 @ 10:29 AM

Most of you probably woke up this morning and, just before checking the Times and your email, raced to see how the Power 150 was looking today. Hopefully none of you spit out your coffee when you saw that things had been shaken up a bit.

Early this morning we implemented a few changes on the back-end that will make computing your scores easier and more reliable for our server. Most of them will be invisible to the outside world, but, in the process, we've shifted to a more consistent, and, we think, fair scaling mechanism, which you may have noticed.

We don't disclose exactly how we scale our metrics, but we've always done it on a curve rather than a line. In the past, each metric had its own curve, but now we've applied a uniform curve to all scores that spreads them out more evenly across the 150 points and, in the process, makes relative differences between blog scores more pronounced. It's also a sliding scale as opposed to a fixed one, so it'll adjust on the fly to broad changes, which, for example, would leave you less likely to lose points if most of your peers are as well. Conversely, if you're far exceeding your peers on a relative basis, it's more likely to take that into account.

What's this mean for you, exactly? Not terribly much. There will be some rumbling underfoot today while things are getting re-settled, and some of the metrics that had far different curves will be brought into line and therefore cause more disruption. But, moving ahead, our new scaling system will be a little smarter and make our metrics more consistent with the relative weights we've given them. While it may disadvantage some for the time being, it should be a net win for everyone as a whole.