Overwatch 2 Beta Statistical Analysis

[Lumy]

Overwatch 2 Beta Statistical Analysis

Heroes of Overwatch, welcome back!

The second Overwatch 2 beta has ended, and that means it’s time to talk data–again! In our previous beta statistics blog, we took you behind the scenes to learn more about Overwatch data and statistics like weighted usage rate, unmirrored win rate, and more. This data, along with healthy amounts of player feedback, user research, internal discussion, and lots and lots of gameplay are the cornerstones of the hero balance process. In this blog, we’ll be walking back down that same road to show how we used these metrics to inform decisions that were made during the second beta.

The highlight of the second beta was, of course, Junker Queen. The design team wanted to ensure that she was just as powerful as the Queen we saw in her cinematic, but not so powerful that she dominated the entire beta. However, Queen wasn’t our only focus; this beta began with a smorgasbord of hero changes, including a full reimagining of the damage role passive. Finally, this beta was opened to console players, which meant ensuring that no hero was over or under tuned on all platforms was a chief concern.

Console joins the fray

Before we get too deep into the analysis, let’s talk about the PC versus Console balance split. Overwatch finds itself in a similar situation to other cross-platform titles, where we have a playerbase that enjoys our game on many platforms and with many different input control schemes. This can–and does–create different strategies that work better with either a controller or a mouse and keyboard. While there were differences in win rates for particular heroes on console versus PC, we were quite happy with the overall balance of heroes throughout the beta on both platforms. If anything, the win rates of heroes on console platforms were closer together than on PC! With this in mind, for the rest of this blog the metrics displayed will be a combination of PC and console data, but for all those data enthusiasts out there, here is a table of unmirrored win rate differences throughout the entire beta, split by PC versus console:

Hero	PC	Console	Difference
D.Va	43.80%	46.80%	3.00%
Zarya	49.80%	52.80%	3.00%
Orisa	40.80%	43.40%	2.60%
Lucio	52.50%	54.70%	2.20%
Roadhog	44.30%	46.20%	2.00%
Baptiste	43.50%	45.20%	1.70%
Bastion	46.30%	48.00%	1.60%
Sigma	50.30%	51.90%	1.60%
Symmetra	55.40%	57.00%	1.60%
Soldier:76	46.00%	47.50%	1.50%
Torbjorn	55.20%	56.50%	1.20%
Moira	49.20%	50.40%	1.20%
Sojourn	48.60%	49.70%	1.10%
Cassidy	47.40%	48.30%	0.90%
Sombra	45.20%	46.10%	0.90%
Brigitte	54.90%	55.50%	0.60%
Ashe	49.20%	49.60%	0.40%
Wrecking Ball	48.10%	48.40%	0.30%
Tracer	49.00%	49.30%	0.20%
Reaper	48.00%	48.00%	0.00%
Mei	54.80%	54.80%	0.00%
Winston	49.50%	49.50%	0.00%
Widowmaker	51.60%	51.30%	-0.30%
Junker Queen	56.10%	55.70%	-0.40%
Junkrat	53.30%	53.00%	-0.30%
Zenyatta	54.50%	54.20%	-0.30%
Mercy	48.80%	48.30%	-0.40%
Doomfist	46.80%	46.30%	-0.50%
Reinhardt	52.90%	52.20%	-0.70%
Hanzo	49.20%	48.10%	-1.10%
Echo	48.90%	47.70%	-1.20%
Pharah	56.00%	54.20%	-1.80%
Genji	54.50%	52.40%	-2.10%
Ana	47.50%	44.50%	-3.00%

Unmirrored win rate on PC and console– First week of Overwatch 2 beta

Gauging popularity and power

Now, we turn our attention to the blue-mohawked elephant in the room: Junker Queen. We love the Queen. Outside of maybe Junkrat, we might be Junker Queen’s biggest fans. When the beta launched, we were as excited to play Junker Queen as the community. Queen’s performance both in usage rate and win rate was fantastic–but she was definitely the all-star of the beta.

Weighted hero usage rates– First week of Overwatch 2 beta

This chart illustrates the relative usage rate of heroes compared to the equilibrium usage rate for their role. You can learn more about weighted usage rate in the prior stats blog, but you can think of it as how often a hero is played, relative to the number of heroes in its role. Junker Queen blasted into the Overwatch roster with a whopping 7.5 weighted usage rate, meaning that she was played more than seven times as often as the average for the tank role.

This metric is an excellent barometer for hero popularity, but this alone isn’t enough to indicate that a hero is too powerful. Sojourn, for example, was nearly as popular as Queen in this regard, at more than five weighted usage rate for most of the beta. We expected this kind of interest in Overwatch’s newest heroes, especially with a group of beta participants in console players who hadn’t had the chance to try them out yet. If we wanted to consider balance adjustments for either hero, we had to look at more than just their relative popularity. For that task, we examined their unmirrored win rates to get a better picture of their power level:

Hero unmirrored win rates– First week of Overwatch 2 beta

This chart shows the unmirrored win rates, or the win rates of heroes when only one team in the match fielded that hero. Unmirrored win rates paired with the high usage rate from the last chart show just how powerful Queen was at the start of the beta. Both Junker Queen and Sojourn had incredibly high usage in the beta, but Queen was not only far above a 50% win rate, she was also a head above all other heroes. At this point, we had some confirmation that we had likely overdone it a little bit with Queen.

This led to a fun investigation: if we wanted to tone back Queen’s power level, which ability should we target first? This sort of question is a microcosm of the whole balance process–we dive into player feedback, playtest potential changes, and look at data to confirm or deny assumptions we might have about a hero.

In Queen’s case, we heard a few common lines of player feedback. First, players found it difficult to differentiate Queen from other heroes, and more difficult to land hits on her given her (relative to other tanks) “skinnier” frame. Obviously, Queen is a tall, intimidating wall of muscle, but she certainly paints a smaller silhouette than Roadhog or Reinhardt. Second, pro players in particular had homed in on Commanding Shout as a major source of pain to play against in her kit.

The Queen reigns supreme

Commanding Shout is a focal point of Queen’s power because it applies an AOE health and speed boost to her and her entire team. In the past, hero changes that adjusted a hero’s health pool or speed had extremely strong effects on their performance. So, it certainly had the potential to affect the game in a similar way. Player feedback in hand, the next process began: could we verify these concerns in our data?

One of the ways we often look at heroes is to compare their output in several basic statistics to see if anything unusual stands out. When we pulled these stats for tanks in the first week of the beta, a certain column jumped off the page for Queen–can you see it too?

Hero	Final Blows per 10 minutes	Elims	Deaths	Hero Damage	Damage Taken
D.Va	8.48	21.43	6.19	8,152	13,205
Doomfist	6.67	15.28	8.85	6,427	11,191
Junker Queen	8.49	19.61	6.77	8,300	11,366
Orisa	6.65	17.05	8.11	8,261	12,491
Reinhardt	9.55	16.08	7.61	7,837	26,278
Roadhog	11.25	19.43	6.55	8,185	16,746
Sigma	8.18	19.47	5.33	10,228	21,985
Winston	7.97	18.64	7.46	7,481	25,459
Wrecking Ball	7.76	17.1	6.04	7,699	16,147
Zarya	9.52	22.96	5.92	9,973	15,655
Average	8.452	18.705	6883	8,254	17,052

Hero	Final Blows % Difference	Elims	Deaths	Hero Damage	Damage Taken
D.Va	0.30%	14.60%	-10.10%	0.30%	-22.60%
Doomfist	-21.10%	-18.30%	28.60%	21.10%	-34.40%
Junker Queen	0.40%	4.80%	-1.60%	0.40%	-33.30%
Orisa	-21.30%	-8.80%	17.80%	-21.30%	-26.70%
Reinhardt	13.00%	-14.00%	10.60%	13.00%	54.10%
Roadhog	33.10%	3.90%	-4.80%	33.10%	-1.80%
Sigma	-3.20%	4.10%	-22.60%	-3.20%	28.90%
Winston	-5.70%	-0.30%	8.40%	-5.70%	49.30%
Wrecking Ball	-8.20%	-8.60%	-12.20%	-8.20%	-5.30%
Zarya	12.60%	22.70%	-14.00%	12.60%	-8.20%

Select stat rates and their % from average for tanks– First week of Overwatch 2 beta

Does it seem odd that Junker Queen doesn’t take a lot of damage? She’s a tank who doesn’t have a shield, nor does she have ability that can block or absorb damage, yet she has the second-least damage taken in the first week of the beta! Even Doomfist, who does have an ability that blocks damage these days, seems to “pay” for his lesser damage taken in other ways, with lower performance in other stat rates. It’s worth noting that being better or worse in any of these stats isn’t necessarily a cause for concern. They indicate hero play styles and characteristics more than anything related to winning. However, for Junker Queen to take this little damage with no mitigation, the only explanation was to confirm the suspicions of the playerbase: she’s just harder to hit than other tanks.

Having confirmed Junker Queen is indeed a smaller target than Reinhardt or Roadhog, we turned our attention to the other player feedback on the influence of Commanding Shout on Queen’s games. Building from the above investigation, the theory was that since Queen has no other ways to save her team from incoming damage, her Shout must be contributing in some powerful fashion for her to continue winning games at such a high clip. To confirm whether this was the case, we analyzed death rates, but framed them in a unique context. We wanted to understand how often other heroes were dying when played alongside a particular Tank on the field. Below are the results of that study for the first week of the beta:

Death rates of heroes when played with specific tanks– First week of Overwatch 2 beta

We were expecting some signal here, but not anything this drastic. Junker Queen, a hero whose only methods to protect her own team are her Commanding Shout and blocking hits with her body (which isn’t easy, as shown above). This was somehow preventing teammate deaths at a rate almost on the level of Sigma, who has a shield and damage absorption ability.

Taken together, these two analyses painted a clear picture of Junker Queen at the beginning of the beta. She was incredibly popular, won far more matches than she lost, and was taking less damage than most of the other tanks. Confirming these phenomena with data gave us confidence in the direction we took her balance, resulting in a reduction in power of her Commanding Shout and a slight reduction in area of her Rampage ultimate.

It’s Norbin’ time

These and other changes, including the new Moira ability, Necrotic Orb, went live later on in the beta, which meant it was time once again for the most fun part of hero design data analysis: observing the effects of the changes on the power levels of heroes. Those changes went live on 7/11 and 7/14: let’s extend that win rate graph out and see how the heroes were impacted:

Hero unmirrored win rates— Overwatch 2 beta

For heroes who received changes directly related to balance, the changes had their intended effects. The nerfs to Zenyatta, Junker Queen, and Sojourn resulted in win rate reductions of 2-3% in each case, while the buffs to Doomfist, Symmetra, and Orisa had similar effects in the opposite direction. We were happy to see Junker Queen and Zenyatta fall back to Earth while still maintaining respectable win rates. While Sojourn now sits below 48% win rate, the changes to her Railgun alt fire feel more fun to play against. Orisa has been steadily climbing in win rate, and these changes were another step in the process of bringing her power level up to align with the fun level of her new kit.

Moira and Mercy were the subject of ability changes–in Moira’s case an entire ability–and their effects were less satisfactory. Mercy’s win rate was impacted a small amount by the refactoring of her Super Jump’s mechanics, and Moira lost a few percentage points off her win rate. While we were happy to have experimented with these ability changes, we have clear ideas for where we want to go with Mercy and Moira in the future, as stated in our blog on Necrotic Orb and Guardian Angel.

We rarely have a chance to study a hero before and after a major ability change like Moira’s, so even though Necrotic Orb is being shelved, let’s look at what effect of swapping Moira’s damaging Biotic Orb for Necrotic Orb had on her damage and final blow output:

Moira damage and final blow rates— Overwatch 2 beta

When Moira gained Necrotic Orb and all its Ultimate-defying benefits, she lost the damaging half of Biotic Orb, which impacted her hero damage and final blow rates in a similar fashion. For Final Blows, she dropped from roughly 5.8 per 10 minutes to 4.3, about a 25% drop. For Hero Damage, she dropped from about 5,950 per 10 minutes to 4,050, a drop of over 30%. The similarity in the reductions show us what we already knew: Moira’s damaging Biotic Orb isn’t just a source of damage, but also one of Moira’s main sources of lethality. These findings, alongside all the feedback mentioned in our Necrotic Orb blog, were factors in reverting this ability to Moira’s previous state.

Data, design, and community collaboration

The metrics and investigations we’ve shown in this blog are important for both evaluating heroes as they currently stand and for homing in on which portions of heroes’ kits are potential targets for adjustment. Sometimes we make changes that work the way we expect. Bringing Queen back down to Earth while still maintaining the thrill of playing the undisputed champion of the outback was a success, as was toning down Zenyatta without having to take away his new Snap Kick passive. Other times, changes don’t perform the way we hoped, like our Necrotic Orb experiment. Regardless of the outcome, having these data and investigations are an integral part of the hero design process.

Releasing new heroes, changing existing heroes, and gearing up to launch–it’s an exciting time for Overwatch! Thank you for being a part of this testing phase, and we hope you’ve enjoyed this peek behind the curtain for the second Overwatch 2 beta!

- Ben Trautman, Data Analyst, Overwatch

[Bakis]

OW2 will rock but it is too late. The sequal should haven been out at 2020 at latest and with more new maps and/or heroes meanwhile.
People moved on. OW is kinda dead at the moment and I dont sense there will be a big come back at launch in terms of the PvP.
Interest in OW(2) League is practically null.

I loved the first beta so I hope I'll be proven wrong but I doubt it.

Edit: also PUSH-mode is so flawed. Since as long as your team is in the lead the only focus is holding the best defensive spot and not pushing further and I cant think of a way to fix that without either redoing maps or make situational changes to the bot behaviour which is a bad thing, god knows what, attacking team getting 10% damage boost/less damage taken withing X meters or smt. Situational parameters bloats any game and should be avoided but I got no idea how this can be fixed.

[Darkeon]

Impressive, Now Let's See - meme.

Shame the reception to it was rather pathetic from what I've seen.

[Bakis]

Just why did they have to go down the super dodgy path of adding more abilities.
As a WoW player its the power creep.

Lets look at Hog. Shot (right/left)
Hook, 5 shots
Heal (while unable to do shit)
And an ulti that at best is 50% as good as the zoning ultis of soldier/mcree.

Junker Queen, she got it all.
Yes she will get nerfed but giving more tools and in this case moreusable tools on average is a slippery slope.
The good thing is BLZ wont add many heroes but the damage is done.
Why play X when Y is by default more versitile in the grand scheme of things?

DONT GIVE NEW HEROES A LARGER TOOL KIT. By default those got a bigger toolkit, no matter how optional.