We’ve received a fair amount of feedback on our current rating system. Based on this, we’ll be doing a number of changes, as well as resetting everybody’s ratings for a fresh start. We’re aiming to reset on Thursday next week. For a more detailed breakdown of why we’re doing this, read on past the break.
Let’s begin by breaking down the current system; how it works, why it was designed the way it was, and what flaws it has. In this post, I’ll only be discussing the ranked match ratings. Quick matches work using a very similar system, but those ratings are never shown to players.
Our current matchmaking system measures a player’s skill in two ways – a hidden value (HV) that starts at 1500, and a public value (PV) that starts at 0. The HV follows the Elo rating system. It’s a tried and true system that we figured would make a good starting point.
The PV, on the other hand, is a new concept. We figured it would be intimidating for newer players to start off at 1500 rating, and likely lose rating after their first match. The public value is what’s used for matching players against each other. When a player loses a match and their PV is less than their HV, the rating stays the same. When they win a match, the PV moves closer to the HV (by a percentage of the difference).
It’s a bit complex, but it’s designed to support an idea we like – to let newer players have a gentle start in the matchmaking system. Unfortunately, the system has a number of drawbacks; the most important one being that players can appear to lose rating on a win (when the PV is significantly higher than the HV).
With the upcoming rating reset, we’ll be changing this system a bit. Instead of having a hidden and public value, we’ll have one rating value (RV) and a coefficient (C). The RV will still use the Elo system, just like the old hidden value did. The coefficient will start at 0, and increase by a fixed value (possibly .1) with each win, until it reaches 1. In-game, we’ll be displaying RV * C as the player’s rating. This value will also be used for matchmaking. This means that newer users will still be eased into the system. After a fixed number of wins, you’ll be in the matchmaking full force.
When the original rating system was designed, we were very careful about retaining the zero-sum balance implicit in Elo; the average rating is always exactly the starting value, 1500. Over time, we’ve come to realise that the addition of new players and the loss of old means the zero-sum is mostly theoretical. Additionally, Scrolls is an ever-evolving game; changing rules and meta means very old ratings aren’t truly valid.
For these reasons, we’ll be introducing rating decay. This means that inactive players will lose rating (down towards 1500) over time, giving other players a chance to reach the top of the ladder. As this system requires somewhat larger changes, it’ll be introduced a while after the reset and new rating system.
See you on the battlefield!