The Final Outrage: DNN Retires Power Rankings

After 4 1/2 years of issuing more-or-less regular monthly rankings of the top 25 teams in modern roller derby, along with accompanying analysis, this month we’ve decided to retire the DNN Power Rankings. We’ve considered this move carefully over several months. Here, then, are five reasons why we’re pulling the plug:

1) WFTDA Rankings have returned to a single ordered list.

We launched the DNN Power Rankings in February 2009 for a number of reasons. At the top of the list was the Women’s Flat Track Derby Association’s move from a single (then) national ranking to regional rankings only. We felt that the lack of a single measure for comparison hindered discussion and debate of potential matchups, which we consider to be an important part of fostering a long-term love for any sport.

Now that the WFTDA rankings are once again a single, ordered, global list, our separate process becomes less necessary. Importantly, in the interim WFTDA also did away with the practice of de-listing teams that haven’t fulfilled membership activity obligations, meaning that notable teams no longer disappear from this ranking (rendering it less meaningful for purposes of discussion and comparison).

2) Algorithm-based rankings have improved.

Another key motivation for the original launch of Power Rankings was the apparent deficiency, in 2009, of algorithm-based ranking systems. Flat Track Stats, always an incredibly useful tool for data analysis, was at the time struggling to successfully pick even 70% of Top 25 games; Derbytron would debut later in the year, with similar performance.

In the intervening years, though, both sites’ algorithms have been tweaked and tuned to perform better. Much better. In the past year, both are calling the victor of top 25 games correctly between 85% and 90% of the time, which is as well as or better than our human panel has ever achieved.

We recommend you start preparing for the robot holocaust, as we are.

3) Unaffiliated teams are no longer likely contenders for ranking.

In the early years of the Power Rankings, not all of the most capable teams were full WFTDA members. Notably, the flat track travel team of the San Diego Derby Dolls were Power Ranked from May 2009 through December 2010, peaking at #14.

Since then, though, only (then-apprentice) Columbia has cracked the top 25, appearing briefly in early 2012. While we’re glad we kept the DNN Power Rankings open to any team that demonstrated the ability to play in the bigs, the ground reality is that we don’t imagine this happening again in the foreseeable future.

4) Increasing parity at the top has made outcome prediction less reliable.

Go look at the results from the Golden Bowl and ECDX, and try to make a consistent ordering of teams based on those results. Go on. Just try it. We’ll wait.

Over the past year, and particularly in the past two months, we’ve seen a big uptick in “any given Friday/Saturday/Sunday” performance among the top couple dozen teams (with one obvious exception, of course). It’s unclear where this apparent increase in parity derives from; maybe more teams are learning to adapt quickly to the shape of a game, or perhaps it’s just a temporary situation while everyone gets used to the 2013 rules changes.

Note: we see this as a GOOD thing. Whatever it is, we’ll take it. Nobody likes a foregone conclusion.

5) We’d rather spend the time on other projects.

Creating and publishing the DNN Power Rankings is a time-consuming endeavor which burns 20+ person-hours of labor each month. This was clearly worthwhile to us for some time, but as all the above factors have become less important, the process has become a chore. The other rankings systems, as they exist today, now provide more than enough quality fodder for debate and discussion.

By retiring the Power Rankings, we regain the time and energy to experiment with other features, such as the new prediction-focused Weekend Preview we’re taking for a test drive. We’re interested in your input on what you’d like to see. Is it this? Is it something else?

It’s been a long, fun run for the Power Rankings, but we think it’s time to move on. We hope you’ll agree!