We analyzed 570,786 picks to see if our picks game is a game of skill. We weren't surprised by the results, but you might be. We frequently see the same friends at the top of the Global Leaderboard. We see the same people when we have the Verdict Awards at the beginning of each year. Seeing a friend on the Global Leaderboard or top the Verdict Awards is anecdotal. We want to show that our picks game is undisputedly a game of skill through statistics.
The definition of a game of skill varies wildly for each jurisdiction. For our analysis, we define it as a game where you can improve your outcome over time. If we can prove that, we can prove that some players are better than others. As a counter-example to a game of skill, think of roulette. You can't improve over time; no player is better than another player.
The paper that we're summarizing below is available here. It analyzes our free game, which maps a set of picks to a leaderboard position. It's the foundation of Verdict Tournaments which is our free game but with money.
We took our previous data, which contains 570,786 picks and final placement on a leaderboard over 158 events. We added some extra information to the picks to show how many events the player previously played in before the pick. The additional information allows us to easily ask questions like, "Does a player place higher on average if they've participated in more events?". From there, we can feed the data into a program that can graph and interpret the data.
We created a graph showing the Average Placement Percent on the leaderboard versus the Previous Number of Events Played.
From there, we can interpret some results visually; it looks like as the Previous Number of Events Played increases, the Average Placement Percent also increases. Now, I wish we could conclude things there, but we dig deeper using a few known techniques in statistics.
We calculate a measure of the relationship between two datasets. That technique shows an extremely high probability that the two are related and that the Previous Number of Events Played increases as the Average Placement Percent increases. Additionally, the result indicates that the outcome is not due to chance.