Leonard LaPadula - CEO
As the CEO of Advanced Sports Logic, many people have asked me, “How did The Machine do in 2011?”
Based on how our customer’s did, my own league results, our solid software endeavors and industry impact, I feel that The Machine did very well and am proud of what we achieved. Over the next few weeks, I’ll be sharing with you in more detail, my thoughts and experiences on how we did over the 2011 fantasy football season. I'll start with how The Machine customers did overall.
The first thing anyone wants to know is how did people that used The Machine do in 2011?
First, let’s take a look at how players’ season compared with their week 1 win projections. For analysis purposes, I divided the grouped the players into three categories:
- Customers that only used the free introductory license (available at MyFantasyLeague and RealTime Fantasy Sports) just to run simulated drafts, get lineup advice and check out the win probability.
- Customers that purchased a full subscription license, but used The Machine in only 1 to 5 of the 10 weeks between the first NFL preseason game and week 6.
- Customers that purchased a full subscription license and used The Machine in at least 6 of the 10 weeks between the first NFL preseason game and week 6.
The chart below shows how these groups of players did over the entire season. Customers who purchased the full subscription and used The Machine in at least 6 of the 10 weeks had a 13.0% probability to win by the end of the season, compared to 10.5% that only benefited from the introductory features.
Chart 1: Probability to win based on subscription type and usage duration
In-Depth Observations of The Machine Customers
Looking a little more in depth at our customer results, we were able to draw some interesting conclusions regarding that last group of customers who used The Machine in at least 6 of the first 10 weeks including the NFL preseason.
For example, the data in the following chart shows how their teams did over the course of the entire season compared to their probability to win before the first week of the NFL games.
Chart 2: Probability to win over the course of the entire season
Clearly, the average customer in this group that had a higher probability to win after their draft carried a higher probability throughout the entire season. In our review, all instances of customers trusting The Machine to give draft guidance resulted in a probability to win that was higher than 15%; so we can assume that the vast majority of customers with a below 15% probability to win before week 1 did not follow the draft guidance as provided by The Machine.
Another observation in this group was that the top teams saw a bit of a slump in the beginning of the season, but trended upwards towards the end of the season. The drop could be contributed to early season issues we ran into regarding projection blending. However, the uptrend starting at week 14 could be a result of The Machine drafting teams with better resilience to NFL player injury and because of improvements we made to our projection blending. (Stay tuned to read more about the software endeavors in 2011)
Why Customer Results are Important
We use our customer results to measure how much The Machine helps fantasy football players and to find opportunities for further fine-tuning. The reason we can trust this customer analysis is because we capture the probability to win every time a customer runs The Machine and because we take extra steps to ensure more accurately filtered data. For example, if we only measured active customers at the end of the season, then it would artificially skew our results analysis favorably. To prevent this, we do the following:
- We determine our active customers by measuring their activity for only the preseason and the first six weeks of the season. So even if a customer becomes inactive after the first six weeks of the season, they are still considered one of overall active customers.
- We use the last time a customer runs The Machine to fill in any weeks that the customer does not run The Machine. For example, if a customer last runs The Machine in week 12 and has a 0% probability to win, we propagate the 0% probability value all the way to the last week of the season. So this gets factored equally with the customers that are still active.
Using the data from last season’s customer results allows us to draw some significant conclusions regarding performance of The Machine in 2011.
- Active customers that used The Machine for draft guidance - based on the assumption that they achieved 15% or greater probability to win in week 1 - more than doubled their probability to win by achieving a 21% probability to win compared with a roughly 9% probability to win for the average customer (as seen in Chart 2)
- Customers that purchased the full subscription license of The Machine show a higher probability to win compared with customers that only used the introductory license.
- While the in-season probabilities did not increase significantly throughout the season, the end of season upward trend was likely due to better resilience from NFL player injury and improvements made to project blending.
- The Machine’s preseason estimates are very good indicators of a team’s actual probability to win indicating the efficacy of the technology for our Trade Arbitration product.