Aaron Keller on Recent Changes to Overwatch 2 Matchmaking

[Lumy]

Aaron Keller on Recent Changes to Overwatch 2 Matchmaking

Director’s Take – Making a Great Match

Aaron Keller is back with an update about our changes to Matchmaking from our latest patch

---

Two weeks ago, we talked about balance and matchmaking in our blog update. Matchmaking has continued to be a big topic in the community, and it's something we've been focused on internally. If it's important to you then it’s important for us to talk about, so I'd like to update you all on what we're seeing and doing.

One note before we begin - I think we're at risk of sounding like a broken record with our communication on matchmaking. "We're working on it and it's getting better" isn't the best message, especially when players are still experiencing bad matches. So, I wanted to share some data to help you better understand the state of matchmaking and the impact of some of our recent changes. Let’s get into it!

Over the past week, we’ve made changes to the system that have brought down the skill gap between players in a match significantly for high and low MMR matches. If you look at the Ranked Skill Difference chart below, you can see two spikes that negatively impacted match quality in Season 3 and the recent changes that have started bringing this back down. We’re almost back to where we were when Season 3 started.

Looking at the high MMR group (GM players and above), roughly half of our matches have an MMR range of about 4-5 divisions, while 25% are at a difference of about 5-6 divisions and the worst 1% of matches are at roughly 10 divisions. Grouping behavior drives a large part of this gap – remember, in Masters, players can group with other players who are up to 5 divisions apart.

Competitive Role Queue Intra-match Player Delta Percentiles

High Ranked




Medium Ranked




Low Ranked



Yellow = 50th Percentile | Green = 75th Percentile | Cyan = 95th Percentile | Purple = 99th Percentile

Okay, so that’s where we were as of March 6. Let's talk a bit about how our matchmaker was working at this point and the changes we’re making to improve it. I'll try not to get too into the weeds!

When our matchmaker tries to make a match, it looks for players as close to each other's MMR as possible. As it continues to search, it will expand to players with larger differences in skill. Prior to Season 3, we had the ability to control the rate of expansion independently for each queue. In Season 3, we changed how our matchmaker expands over time, but removed some of our control over per queue behavior. This meant that if we wanted to reduce the size of skill differences in Competitive, we had to do the same thing in Unranked and Arcade. If we wanted to get shorter queue times for Unranked by expanding skill difference requirements, we could do that, but then Competitive would have wider skill differences too.

In the patch we released on Tuesday, we implemented the first of a series of systemic changes (more are coming in Season 4) that give us back the flexibility to tune these modes separately and gives us new ways to tune the system. Competitive can once again have its own set of parameters for how wide the skill gap can be that are separate from our other modes. This week we will be tuning these values to tighten the skill gap in Competitive as much as possible while keeping an eye on queue times for that mode.

Additionally, in the mid-season patch we added the capability to sort parties together that have a similar player delta. We’re now able to get similar parties together in a way that minimizes role delta, which has resulted in significantly lower party queue times. It also concentrates wide parties together, which has made a dent in our player MMR delta.

Hopefully, that provides some much-needed context for all of you and helps you to see the amount of work going into the system. However, your experiences are what really matter. We'll be collecting a lot of data this week and listening to all of you as well. I'll be providing some short updates soon, most likely through Twitter.

That's it! Hopefully we'll be moving on to other topics for future updates. See you all in game!

- Aaron

[Primohastat]

Wow, thank you Aaron Keller for such detailed information. Such insight, omg!

One could think that over those 7 years of Overwatch, all these well paid game developers could come up with a solution rather than another stupid "update". Each paragraph dumber than the previous one. Each sentence clouding the real issue and trying to redirect attention to some stupid thoughts written probably by "collateral damage handlers".

This "update" is so bland that it could be a meme - "wanna witness the state of play in Overwatch team, look up this update"

[Edge-]

Wow, thank you Aaron Keller for such detailed information. Such insight, omg!

One could think that over those 7 years of Overwatch, all these well paid game developers could come up with a solution rather than another stupid "update". Each paragraph dumber than the previous one. Each sentence clouding the real issue and trying to redirect attention to some stupid thoughts written probably by "collateral damage handlers".

This "update" is so bland that it could be a meme - "wanna witness the state of play in Overwatch team, look up this update"

So what's the obvious and easy fix that they're just too dumb to figure out, apparently?

[Primohastat]

Season 1 of OW2: let's start a system in which you never lose rating. You inw you gain. You lose you don't gain. This way players can pretty quickly find where is their spot on the ladder. Oh, and ranking points are also currency, which means you can earn ranking and currency at the same time. Say, 25 point = 25 OW2 dollars.

Make that work in season 1, gather feedback, think about something new for season 2.

It took me like 2 minutes.

[Edge-]

Season 1 of OW2: let's start a system in which you never lose rating. You inw you gain. You lose you don't gain. This way players can pretty quickly find where is their spot on the ladder. Oh, and ranking points are also currency, which means you can earn ranking and currency at the same time. Say, 25 point = 25 OW2 dollars.

Make that work in season 1, gather feedback, think about something new for season 2.

It took me like 2 minutes.

How much do you gain? How much do you lose? What if you performed really well but your team dragged you down? What if you sucked and got carried hard? It's a team-based game and all, but matchmaking is supposed to be based off "your" skill. So basing that purely around the performance of your whole team seems like it wouldn't actually provide a very accurate result.

That took about 30 seconds to poke a fairly obvious hole in. This is why it's not all quite so /5head easy.

Also, ranking points as currency would result in people trying to boost their way higher up for free currency and give "unfair" additional rewards to better players. You can bet people would complain about it, and that's potentially a lot of cash shop currency going out for free that Blizzard isn't going to want to miss out on (knowing that that's not all lost revenue, but some of it will be).

[Primohastat]

How much do you gain? How much do you lose? What if you performed really well but your team dragged you down? What if you sucked and got carried hard? It's a team-based game and all, but matchmaking is supposed to be based off "your" skill. So basing that purely around the performance of your whole team seems like it wouldn't actually provide a very accurate result.

I think that the devs fall into this mindflow. Add more indicators, more parameters, more... If someone in the dev team formulates goals this way, the team responds with addressing all those points one by one. Like this for example:

1. Instead of fixed 25 points per win, let's make that a variable which changes based on your rating, your role, your behavior in game, your latency, your endorsements, your team rating, individual ratings of your team mates, the opposite team ratings and meta information, etc.

2. You lose points - same thying as above, but reverse. Lots of data to gather, lots of data to process and then turn it into something that is suppose to increase... what exactly? What's the goal? With so many factors added, the dev team is prolly losing the main goal which should be FUN. This is a game and not programming contest. Heh, I'm already digressing, let's stop here.

3. Personal skill vs bad team. To me, that is impossible task to do with a game that is suppose to be fun and entertaining. In a life or death competition or when squeezing your employees to maximise profits, you can pursue such performance analytics to increase efficiency or profits, but in computer games or real life sports? You do not use ratings, you use role assignment and measures. If you suck and 100m sprint, and you suck at 200m sprint, but you feel that you are like running and you can be a winner, go for 800m, 3000m, 110 hurdles. Change the role rather than tying yourself to an impossible task other people might be cheating to gain or maintain. Dam, I'm digressing again. It is so easy to fall into that mind trap of "let's stick to the previous system rather than flipping the table"

4. Simple end game stats, which are very detailed in Overwatch, can provide the player and the team with insight about what to do next. Got carried? Still, you won, you gained points, you are happy. Team is more or less happy that they didn't lose. But even in a situation, you are hardcarried, you can see that you didn't add as much (kills, assists, death, objective) as the rest of the team. Showing such data to a player and giving the player medals for >>performing well in the role that player chose<< is way more educating than giving medals for kills to a healer, or medals for objective hold to a flanker dps. Right now devs made it that the game incentivize incorrect behaviour. This cannot be fixed by some ratings. But devs instead of fixing end game summary, they punish players (and teams) with losing many useless ratings.

5. Yeah, this is a wide known issue. If you good, pick solo discipline. Because you will be forever frustrated when overperforming in a team game. But Overwatch was a major game changer in that field. After decades of quake/counter strike approach of solo skill > over team work, we got a game which offers roles instead of headshots. And if there are roles, measuring all the roles with the same indicators is terrible dev mistake. It is so freaking stupid, develop a system that doesn't require ratings and then force ratings into it because everyone is adding ratings and ratings are with us since stone age and... and... Just drop this stupid idea of ratings. Just for one season. Just to see how this will impact the players and the gameplay.

6. Stats like kills, assists, heals, bubbles, soaking dmg, distance traveled, time on objective are not ratings. So modifying the end game summary to show the player/team how all the members performed in their role can be actually helpfull. Maybe highlight important factors to show what is more important in that role? Maybe add best performers in that role for comparison? For example Zarya player can compare his/her bubbles uptime vs best Zarya bubble count on this map this patch? Even if this will not be met with positive response from players, it is better to add something new like this rather than sticking to the same old shit for years.

But all this doesn't matter. It is all distraction from the problem.

The problem seems to be long wait times. Which might be addressed by:
1. Make the game player friendly so more players play the game, therefore more players in the draft pool (you can encourage players to play OW2 by NOT PUNISHING THEM BY LOSING RATING IN AN ENDLESS GRIND)
2. Make games and rounds shorter. Take out like 30 secs from each round (just for one season? is that possible? just for one season?) to see how it affects the queue.

No need to produce and post some bullshit rating vs queue "updates". But I guess that this is what OW2 devs are now. Burned out, bullshit grinding zombies sticking to their seats in a company they deeply hate. They couldn't change anything for so long, that they lost the capability to change anything.

[vizzle]

1. Make the game player friendly so more players play the game, therefore more players in the draft pool (you can encourage players to play OW2 by NOT PUNISHING THEM BY LOSING RATING IN AN ENDLESS GRIND

You haven't thought this through. You want a system where you gain rating by winning but you never lose rating by losing. Do you not realize that eventually every single player would reach their skill cap in rating and would just lose every single match? How would that be fun for anyone?

Rating isn't a currency, it's not something you continuously earn. It's something that reflects the average of your matches to help the matchmaking system give you matches that suit your skill level. If you never lose rating, then any player could eventually grind their way to the top out of luck, playing 20 matches where they're incredibly outskilled to win 1 match and gain a tiny bit of rating. That's not fun for anyone; not fun for the people who actually play at that rating and skill level playing against people who shouldn't be there, and not fun for the people who are gaming the system and just grinding out lucky wins in a level that outskills them.

You wrote this incredibly long rant while misunderstanding a fundamental of PVP games.

[Airlick]

Season 1 of OW2: let's start a system in which you never lose rating. You inw you gain. You lose you don't gain. This way players can pretty quickly find where is their spot on the ladder. Oh, and ranking points are also currency, which means you can earn ranking and currency at the same time. Say, 25 point = 25 OW2 dollars.

Make that work in season 1, gather feedback, think about something new for season 2.

It took me like 2 minutes.

That's just genius, you'd definitely find a comfortable spot on Blizzard dev team. Your solution to players complaining about having imbalanced matches is making the time played (because that's what not losing rating for losing means) the only metric for matchmaking? Yeah, that would definitely fix all issues and players "would find where their spot is". We're opening new horizons here, people.

[Primohastat]

You haven't thought this through. You want a system where you gain rating by winning but you never lose rating by losing. Do you not realize that eventually every single player would reach their skill cap in rating and would just lose every single match? How would that be fun for anyone?

I love bolded part. You don't need rankings/ratings to learn where your skill cap is. And knowing that I cannot be better with Soldier76 does not stop me from playing the game or experimenting with other hero or role.

Furthermore, reaching your skill cap doesn't equal losing interest in playing the game. Like, I never reached gold rating in OW, but it doesn't stopped me from playing each season. Long games and long queue times did. (as, wait times and breaks in between feeling longer than the match itself).

If you never lose rating, then any player could eventually grind their way to the top out of luck, playing 20 matches where they're incredibly outskilled to win 1 match and gain a tiny bit of rating.

What is at the end? What is the goal of having high or top rating? Deliver fun to player? Restart OW as an esport? Divide playerbase to elite and noobs? Continue programming kids and teenagers that grind is the game? Expose players that want to have fun in overwatch universe to additional stress?

I would love some developer actually answer this question: what is the goal of keeping artificial levels of beginner, medium, elite, top and top 1 ranking players in Overwatch. And how achieving that goal affects queue times and fun level.

You wrote this incredibly long rant while misunderstanding a fundamental of PVP games.

Right now this becomes just an offtopic created by my to response to Edge. The real issue is the content of this dev "update" on top of the page. And that it is not addressing the issue and shows that the approach of the OW2 developers is wrong which means no changes for the better anytime soon.

- - - Updated - - -

That's just genius, you'd definitely find a comfortable spot on Blizzard dev team. Your solution to players complaining about having imbalanced matches is making the time played (because that's what not losing rating for losing means) the only metric for matchmaking? Yeah, that would definitely fix all issues and players "would find where their spot is". We're opening new horizons here, people.

This is kinda offtopic since the original issue was long queue times for players across all levels of rating.

But if you think about it, rating IS INDEED A PROBLEM generator in Overwatch. Imagine that you have 3 players:

1. newbie on a low-end computer
2. player that spent 300h in game, mid-range computer
3. hardcore player, got skill, knows the game/maps/heroes inside out, high-end computer

Imagine how they play.

Now imagine that all the rating/ranking based solutions for queue times, for "fair game", for ladder climbing failed. BECAUSE THEY DID.

Now, you have an opportunity to create 2.0 version of the same game. You cannot change the in-game mechanics as they are iconic and they are an essence of Overwatch.

Changing what is invisible to players seems to be obvious choice.

Now, will you stick to old match making mechanics developed mostly to support FOMO and gambling implemented in game? Mechanics that are toxic, predatory, stress inducing? Mechanics that for years are being called bad and annoying?

Or maybe you will go with "Hey, it's 2.0! Let's do something completely new here shall we? We are not longer bound by balance patches. We got seasons now! Let's experiment rather than polish the old shit"

[vizzle]

I love bolded part. You don't need rankings/ratings to learn where your skill cap is. And knowing that I cannot be better with Soldier76 does not stop me from playing the game or experimenting with other hero or role.

Furthermore, reaching your skill cap doesn't equal losing interest in playing the game. Like, I never reached gold rating in OW, but it doesn't stopped me from playing each season. Long games and long queue times did. (as, wait times and breaks in between feeling longer than the match itself).



What is at the end? What is the goal of having high or top rating? Deliver fun to player? Restart OW as an esport? Divide playerbase to elite and noobs? Continue programming kids and teenagers that grind is the game? Expose players that want to have fun in overwatch universe to additional stress?

I would love some developer actually answer this question: what is the goal of keeping artificial levels of beginner, medium, elite, top and top 1 ranking players in Overwatch. And how achieving that goal affects queue times and fun level.

Right now this becomes just an offtopic created by my to response to Edge. The real issue is the content of this dev "update" on top of the page. And that it is not addressing the issue and shows that the approach of the OW2 developers is wrong which means no changes for the better anytime soon.

You are still misunderstanding how this works. Maybe take a break from gaming for a minute and really think this through, because I don't understand how you still aren't seeing why your solution is so broken.

Like, I never reached gold rating in OW, but it doesn't stopped me from playing each season. Long games and long queue times did. (as, wait times and breaks in between feeling longer than the match itself).

If you never lost rating during losses and only gained rating during wins, then you would end up with one end game -- your rating would end up peaking at a level where you would never win games anymore. You would never be able to escape that because your rating would be stuck there, no matter how many times you lose. Do you not understand? If the rating never goes down during losses, then you would peak at a place that is too hard for you. At that point, the only way you would get a win is by being carried or through luck, which would make your future matches even more difficult for you. Eventually, even luck wouldn't be enough to help you.

what is the goal of keeping artificial levels of beginner, medium, elite, top and top 1 ranking players in Overwatch. And how achieving that goal affects queue times and fun level.

? How do you not know the answer to this? How can you play PVP games and not know why players should be matched according to their skill level? Are you even thinking?

What is at the end? What is the goal of having high or top rating?

For better matchmaking. Duh. Jesus Christ. No one wants games where they steamroll everyone else or where they feel like everyone else is steamrolling them. No one wants to play with players who are several times better than them or worse than them. The best matches are those where everyone is roughly at a similar level. You don't want first timers and pros playing against each other. That's really fucking basic.

And let me add this:

Changing what is invisible to players seems to be obvious choice.

When every match turns into a game where the player skills are incredibly unbalanced, then I promise you, this wouldn't be "invisible" at all.

[Airlick]

This is kinda offtopic since the original issue was long queue times for players across all levels of rating.

But if you think about it, rating IS INDEED A PROBLEM generator in Overwatch. Imagine that you have 3 players:

1. newbie on a low-end computer
2. player that spent 300h in game, mid-range computer
3. hardcore player, got skill, knows the game/maps/heroes inside out, high-end computer

Imagine how they play.

Now imagine that all the rating/ranking based solutions for queue times, for "fair game", for ladder climbing failed. BECAUSE THEY DID.

Now, you have an opportunity to create 2.0 version of the same game. You cannot change the in-game mechanics as they are iconic and they are an essence of Overwatch.

Changing what is invisible to players seems to be obvious choice.

Now, will you stick to old match making mechanics developed mostly to support FOMO and gambling implemented in game? Mechanics that are toxic, predatory, stress inducing? Mechanics that for years are being called bad and annoying?

Or maybe you will go with "Hey, it's 2.0! Let's do something completely new here shall we? We are not longer bound by balance patches. We got seasons now! Let's experiment rather than polish the old shit"

Yes. You would indeed put the same old RANKED mechanic in RANKED, competitive mode. If you don't care or are too stressed out about rating, play UNRANKED. It exists for a reason.

Long queue times are not a result of bad matchmaking. Matchmaking being forced to gather players with wildly varied skill levels is the result of long queue times. You don't fix queue times by basically removing matchmaking altogether, as you suggest, you fix them by fixing the rest of the game, which in turn leads to players flocking to the game.