If there is one thing that matters to the team at Advanced Sports Logic, it is that our products help our customers win. The team is steeped in quality improvement practices and one of the well-known rules for achieving high quality is “You can’t improve what you don’t measure”. That’s why, early in 2011, we built a win probability metric system to capture how well our customers are doing.
I admit at being a bit nervous being measured on hard and fast results, but commitment to real results and continuous improvement overrode that nervousness. So at the end of 2011, when the results were in, I, as well as the rest of the team, were extremely pleased to find that customers that used The Machine for their draft and at least the first half of their season were almost twice as likely to win compared to average. Since 2011, many on the ASL staff have used The Machine as their primary weapon in public and highly competitive leagues and results have been both consistent and amazing:
- 73.2% winning percentage and playoffs three years in a row at FSTA
- Regular season lead in 32-team All-Pro League
- USA Today: The Oracle vs The Machine
However, ultimately it is our customers’ success that matters. ASL’s goal is to achieve a 50% win rate by our average customer. We have a long way to go, and a lot more to do to achieve this lofty goal, but in 2013 the peg moved in the right direction — we almost tripled our customers’ probability to win!
Here are the numbers:
This graph shows our customers’ win probability trends from week -1 through week 17. It includes all customers that played in 12-team leagues and purchased either The Machine Dynasty* or The Machine FullSeason* and loaded The Machine at least once in seven different weeks from two weeks before the start of the NFL season through the end of week 6. In a 12-team league, one out of twelve teams win, so the average win probability is 8.3%. During a draft, since The Machine uses the same math to make recommendations and to measure a team’s win probability, it is highly unlikely to follow The Machine’s guidance and finish the draft with less than a 15% probability to win. Therefore, we consider someone “trusted The Machine” when they start the season (measured in week 1) with a 15% or greater probability to win. Of course, the real measure of success is not the starting probability, but if that probability holds true for the season. We are careful not to let attrition skew our measurement. If we only measured the people that are still playing in weeks 16 and 17, then obviously the numbers would be much higher because these would only be our customers that are in the playoffs. So whatever probability to win a customer has at the last time they logged in gets propagates through all following weeks for measuring the performance of that week.
The average win probability of customers that started with greater than a 15% probability to win ended with an average probability to win of 21.6%. This is 2.6 times greater than the average of 8.3%.
Amazingly, even those that had the worst start to their season starting in the 0% to 6% win probability range, as a whole trended above average with a final probability win of 12%.
Toward the end of the season, we had made a lot of improvements in projection quality, lineup recommendations, and waiver recommendations. In the off-season, we have made many improvements for trade related features, and are working to make The Machine faster and easier to use, including support for Android and Apple mobile devices!
We are now eager to see how much closer we can get in 2014 toward our ultimate goal of a customer 50% win rate!
Thank you again to our customers for your patronage!
President and Founder
*”The Machine Dynasty” is now called “The Machine for Dynasty Teams”
*”The Machine Full-season” is now called “The Machine for Draft, Lineups, Waivers and Trades”.