The Trouble with Tier lists...

[Zodiark]

Back in the day when Tier lists were just becomming a thing the people putting them out used actual anecdotal evidence and real game testing and comparisons to make them for each class. Later several top performing players of each class and spec would contribute to form a sort of a meta aggregate Tier list for more accurate data because lets face it not EVERYONE is going to be a top performer for each spec.

Nowadays however what I have come to realize is that Tier lists are mostly inaccurate and just thrown together with sim data and theory crafting.

I've personally played some of the WORST class specs and done some of the MOST damage and actually saved some runs. Which led me to question what is even the point of these tier lists in the first place?

IF you roll a new character and just want a quick idea which spec is doing well right now you might think to yourself let me do a quick google search of a tier list so I'm not wasting my time and energy later with a class that is currently poorly tuned. Then you later find out the classes that scored worst are the ones topping damage meters later on.

The old axiom of "Play what you want" comes to mind. Because if a spec and class just "clicks" in your mind and you enjoy playing it then you will outperform most other players or classes regardless of where your spec falls on the tier list.

This is the trouble with Tier lists.

[Jimson]

I've personally played some of the WORST class specs and done some of the MOST damage and actually saved some runs.

I mean..... anecdotal evidence is what it is. ON AVERAGE, given equal skill/gear/conditions, the top specs/classes will outperform the bottom specs/classes.

[klaps_05]

Tier lists are just content. The more conflicting a tier list, the more viewers and clicks you get.

[arkanon]

Back in the day when Tier lists were just becomming a thing the people putting them out used actual anecdotal evidence and real game testing and comparisons to make them for each class. Later several top performing players of each class and spec would contribute to form a sort of a meta aggregate Tier list for more accurate data because lets face it not EVERYONE is going to be a top performer for each spec.

Nowadays however what I have come to realize is that Tier lists are mostly inaccurate and just thrown together with sim data and theory crafting.

I've personally played some of the WORST class specs and done some of the MOST damage and actually saved some runs. Which led me to question what is even the point of these tier lists in the first place?

IF you roll a new character and just want a quick idea which spec is doing well right now you might think to yourself let me do a quick google search of a tier list so I'm not wasting my time and energy later with a class that is currently poorly tuned. Then you later find out the classes that scored worst are the ones topping damage meters later on.

The old axiom of "Play what you want" comes to mind. Because if a spec and class just "clicks" in your mind and you enjoy playing it then you will outperform most other players or classes regardless of where your spec falls on the tier list.

This is the trouble with Tier lists.

I don't disagree, but there is a flip side to this as well - modest skill level players playing the most difficult to master and gear dependant specs in the game.

I remember this happening multiple times in my 'family and friends' team over the years. The conversation went exactly the same every time:

Why are you playing fire mage? (Example)

I love frost, but fires the best spec by far

You are playing the highly crit dependant build as well?

Yes, Its the highest performing spec

You know we are doing normal, right?

Yes

You know you are in blues and welfare epics, right?

Yes

You know you are doing 40% less damage than when you played frost, right?

Yes

So......why are you playing fire?

Because it's the best spec.


I also love doing what you are saying, and playing "bad" specs but playing it well, but yeah, two sides to every coin. Either way, outside of cutting edge mythic raiding, I say play whatever you want, so long as you enjoy it, and it works well for your raid team/pvp team/m+ team.

[Accendor]

Back in the day when Tier lists were just becomming a thing the people putting them out used actual anecdotal evidence and real game testing and comparisons to make them for each class. Later several top performing players of each class and spec would contribute to form a sort of a meta aggregate Tier list for more accurate data because lets face it not EVERYONE is going to be a top performer for each spec.

Nowadays however what I have come to realize is that Tier lists are mostly inaccurate and just thrown together with sim data and theory crafting.

I've personally played some of the WORST class specs and done some of the MOST damage and actually saved some runs. Which led me to question what is even the point of these tier lists in the first place?

IF you roll a new character and just want a quick idea which spec is doing well right now you might think to yourself let me do a quick google search of a tier list so I'm not wasting my time and energy later with a class that is currently poorly tuned. Then you later find out the classes that scored worst are the ones topping damage meters later on.

The old axiom of "Play what you want" comes to mind. Because if a spec and class just "clicks" in your mind and you enjoy playing it then you will outperform most other players or classes regardless of where your spec falls on the tier list.

This is the trouble with Tier lists.

We literally have an application that analyzes tens of thousands dungeon runs and raids across all difficulties and aggregates the collected data to visually present it to us. It's 3 clicks and I can check which spec performs how well in literally any environment of my chosing. There is no room for opinions, there are only hard facts. Some classes clearly perform better in literally every single scenario on avarage. The fact that a good player lof a bad spec can still hardcarry a group of bad players with good spec has nothing to with the objective evaluation of the meridian.

[huth]

We literally have an application that analyzes tens of thousands dungeon runs and raids across all difficulties and aggregates the collected data to visually present it to us. It's 3 clicks and I can check which spec performs how well in literally any environment of my chosing. There is no room for opinions, there are only hard facts. Some classes clearly perform better in literally every single scenario on avarage. The fact that a good player lof a bad spec can still hardcarry a group of bad players with good spec has nothing to with the objective evaluation of the meridian.

Those aren't facts. Those are statistics. Most people don't have a clue how to actually read statistics. You are one of them.

[Zodiark]

Those aren't facts. Those are statistics. Most people don't have a clue how to actually read statistics. You are one of them.

This may or may not be true but this whole response in which you are quoting does sound suspiciously like someone who staunchly believes in THIER statistical conclusions.

I have played the game long enough to readily spot a group of bad players and I have been in groups where I have been in several groups with people who are playing "good" specs and were doing just fine on the meters, while I playing a "bad" spec was still outperforming them to some degree.

This isn't ALWAYS the case as some specs which I generally enjoyed who are placed much higher on tier lists even when played fairly optimally still didn't perform as well as I had wanted.

This has led me to the conclusion that some specs are more reliant on certain tier set pieces than others and perform better regardless of the gear they have so long as certain sub stats are optimized such as crit, or haste, etc.

Not to mention some specs are heavily reliant on strict adherence to rotation while others are more benefitted by high APM's.

Conclusion once again, is that just like when you work out having a good mind muscle connection and proper form with even a smaller amount of exercise variety will produce superior results. Likewise a player with a spec that clicks with them that they feel passionate about will perform better than what ANY arbitrary tier list will belie .

- - - Updated - - -

I don't disagree, but there is a flip side to this as well - modest skill level players playing the most difficult to master and gear dependant specs in the game.

I remember this happening multiple times in my 'family and friends' team over the years. The conversation went exactly the same every time:

Why are you playing fire mage? (Example)

I love frost, but fires the best spec by far

You are playing the highly crit dependant build as well?

Yes, Its the highest performing spec

You know we are doing normal, right?

Yes

You know you are in blues and welfare epics, right?

Yes

You know you are doing 40% less damage than when you played frost, right?

Yes

So......why are you playing fire?

Because it's the best spec.


I also love doing what you are saying, and playing "bad" specs but playing it well, but yeah, two sides to every coin. Either way, outside of cutting edge mythic raiding, I say play whatever you want, so long as you enjoy it, and it works well for your raid team/pvp team/m+ team.

Yeah I agree. Right now I'm playing Survival hunter which is what got me on this topic. For one I'm doing very well numbers wise. For two when I actually looked at the supposed statistical analysis of a couple of these tier-lists I noticed that the actual number of survival hunters in which they based their data on was fairly low. So then I started questioning the real validity of these lists in the first place. Just look at how fast they come out as one example. People and sites know that these lists will garner clicks so they scramble to get them out. Just look at WoWhead and icy veins and read some of the posts on these lists. many people are scratching their heads on where this data even comes from.

[Mysterymask]

Tier lists are usually a statistic and are never meant to be fully concrete. Half the time it's who's mathematically the best or scwed by popularity. I'll use Marvel vs Capcom 3 for a few examples

Crimson Viper is top tier on the lists but barely anyone actually PLAYS her. Reason being is she can do some really insane combos and cancels but she has to work harder than Zero and Vergil to get the results.

Chun Li was considered like bottom of the barrel tier for a long time. She has risen considerably recently when people figured out she's pretty insane in Dark Phoenix teams being one of the best characters to build meter for her. Note that Capcom has ceased maintenance on this game over a decade ago.

MODOK and Phoenix Wright are also pretty terrible tier wise but a few players got top places in tournaments maining these 2.

Wesker has dropped to mid from high on the transition from vanilla to ultimate despite getting massive buffs. Only reason is he was so overplayed that the community as a whole learned how to counter him

On the flip side Vergil sits on top of the list and for good reason the dude is pretty busted showing that there is absolutely some truth in this

[arkanon]

Yeah I agree. Right now I'm playing Survival hunter which is what got me on this topic. For one I'm doing very well numbers wise. For two when I actually looked at the supposed statistical analysis of a couple of these tier-lists I noticed that the actual number of survival hunters in which they based their data on was fairly low. So then I started questioning the real validity of these lists in the first place. Just look at how fast they come out as one example. People and sites know that these lists will garner clicks so they scramble to get them out. Just look at WoWhead and icy veins and read some of the posts on these lists. many people are scratching their heads on where this data even comes from.

I dont remember which raid or patch it was, but fury warrior was performing EXTREMELY well, and that was compounded by method or whoever it was had 5 or 6 in their world first kill. I helped a friend get through LFR with some other friends who queued as tank/heals, so i went Arms, which was what I enjoyed. It was my main, so obviously i was horribly overgeared for the content, but the funny thing is i met "the other" arms warrior. We talked for ages and others in the raid noticed not one but TWO arms warriors and had a good chuckle. He joined after 2 bosses and a few people in the raid said "oh shit, its the other guy playing arms!"

I also dont remember the SPEC well, but i remember the scenario - our holy pallidan who had been consistent for YEARS decided to jump on the bandwagon and play the Glimmer(?) spec that was the FOTM. His performance was TERRIBLE even in easy content (again, family and friends team so normal/heroic). But he stuck with it, and after months and months of struggling and failing to perform he FINALLY..............went back to normal spec and everything was fine.

[huth]

This may or may not be true but this whole response in which you are quoting does sound suspiciously like someone who staunchly believes in THIER statistical conclusions.

I have played the game long enough to readily spot a group of bad players and I have been in groups where I have been in several groups with people who are playing "good" specs and were doing just fine on the meters, while I playing a "bad" spec was still outperforming them to some degree.

This isn't ALWAYS the case as some specs which I generally enjoyed who are placed much higher on tier lists even when played fairly optimally still didn't perform as well as I had wanted.

This has led me to the conclusion that some specs are more reliant on certain tier set pieces than others and perform better regardless of the gear they have so long as certain sub stats are optimized such as crit, or haste, etc.

Not to mention some specs are heavily reliant on strict adherence to rotation while others are more benefitted by high APM's.

Conclusion once again, is that just like when you work out having a good mind muscle connection and proper form with even a smaller amount of exercise variety will produce superior results. Likewise a player with a spec that clicks with them that they feel passionate about will perform better than what ANY arbitrary tier list will belie .

There's some more things. Like difficulty and complexity. A numerically "best" spec may actually be very inadvisable for most players due to requiring expert level play to actually manifest that superiority and being average or even subpar when you cannot do so, or only work under very specific circumstances that a non-top group/specificially geared group cannot provide consistently. Which leads people to play a spec that actually makes them perform worse.

[Zodiark]

There's some more things. Like difficulty and complexity. A numerically "best" spec may actually be very inadvisable for most players due to requiring expert level play to actually manifest that superiority and being average or even subpar when you cannot do so, or only work under very specific circumstances that a non-top group/specificially geared group cannot provide consistently. Which leads people to play a spec that actually makes them perform worse.

This may well be the case with some specs. One Example I can think of would be Windwalker monks vs Sub Rogue. WW in my experience ever since the class was introduced performs fairly well regardless of gear, tier sets, etc. Generally speaking having a high enough ilvl armor with high enough secondaries has for me produced some very good results. However with a rogue I've noticed much different results given the same parameters. My rogue DPS can be amazing only IF he has the right set pieces as well as everything else I mentioned. On TOP of that the group has to be on the money with mob control as well as their personal utility buffs which effect him as well as CD management like knowing when the optimal time to pop blood lust, etc.

Meanwhile my monk just goes in basically hit every keybind on my keyboard and can pull good numbers.

[Accendor]

Those aren't facts. Those are statistics. Most people don't have a clue how to actually read statistics. You are one of them.

Well, in that case it would be nice if you could enlighten me. Honestly, no trolling, I am eager to learn.

[Ezyah]

Those aren't facts. Those are statistics. Most people don't have a clue how to actually read statistics. You are one of them.

Statistics are facts. Or do you do your own research on Facebook ?

[arkanon]

Those aren't facts. Those are statistics.

Collins dictionary defines statistics as: facts which are obtained from analysing information expressed in numbers, for example information about the number of times that something happens.

You are correct that they can be misinterpreted, but that doesnt stop them being a fact.

[Malkiah]

Well, in that case it would be nice if you could enlighten me. Honestly, no trolling, I am eager to learn.

ok how to explain a visual metaphor in text...

imagine you have a bunch of finely calibrated and expertly tuned high performance sports cars - with nitrous boosters and world class rated tires and perfectly engineered aerodynamic body sculpts.
you want to pit these cars against each other to find out which is the fastest car.

so, you do a handful of races on a pro level course in temperate weather with expert drivers.
you run a series of tests in the salt flats where you can really see what they can do when opened up fully with trained professionals who know how to utilize the machine.
so, there you go... you obviously are going to be able to easily determine which is the fastest car, right?

well once you do all those tests, the reality is that the cars are exclusively used by 14 year olds with cerebral palsy, driving them on back roads in alaska in the autumn.
so which is the fastest car? or, arguably, the "best" car to drive in those specific conditions?

there isn't a problem with the data or the statistics per se, the problem is that the data being used is sheared off the upper end of the logs, which means you're looking at statistics for well geared high skilled players who are selectively biased towards pure numerical output in an extremely limited case scenarios.

BUT, the community looks at that graph and thinks that is universal data across all classes and specs in all situations.
let's say that we can simplify "performance" of a class to a scale of 1-10.
you're looking at data in the 7-10 range.
95% of people who play wow can only and will only ever perform in the 3-5 range, no matter what class they play or what spec.
playing a FOMO? 3-5 range. playing the bottom tier meta pile of garbage? 3-5 range. cookie cutter build following an exact rotation given to them on a hand written note by dratnos himself? 3-5 range.

this is a community perception problem.
people doing normal or 4/8 heroic raiding, or a 12 key, think that the spec tier lists matter to them. hell, for that matter people progressing in mythic or pushing 20s think that spec tier lists matter to them.
they don't.

the reality is that when you're in a soup made out of the faceless teeming hordes of mediocre players, the performance of the top end doesn't tell you *anything* about that you can expect from the bottom end or the middle, or what to do when the prot war is being played by a 67 year old who is sliding into dementia.

so really, "the problem with tier lists" is that by and large the wow community is incredibly stupid and doesn't understand what they're looking at when they see a tier list graph.
the tragic irony is that their stupidity in not understanding why the graph is irrelevant to them is what makes the existence of the graph in the first place problematic.

[huth]

Well, in that case it would be nice if you could enlighten me. Honestly, no trolling, I am eager to learn.

That would go way beyond the scope of this thread. It's not a simple matter.

Collins dictionary defines statistics as: facts which are obtained from analysing information expressed in numbers, for example information about the number of times that something happens.

You are correct that they can be misinterpreted, but that doesnt stop them being a fact.

Then Collins really needs to rework that definition. A statistic is not by itself a fact. Without interpretation and context, it is merely a collection of data that has no practical value.

[StrawberryZebra]

Nowadays however what I have come to realize is that Tier lists are mostly inaccurate and just thrown together with sim data and theory crafting.

To sumarise, a Tier list is an opinion of what the best classes are. Nothing more, nothing less. That opinion may be based on data, like logs, but it's very hard to measure contributions outside of DPS/Healing etc.

Abilities like Stampeding Roar, Mass Dispel, Death Grip etc are all amazing situational tools that don't directly contibute to performance. Any conversation on them is going to be opinion based because you can't really measure how good, or bad, they are in any numerical way.

Tier lists have value if you treat them as an opinion, unfortunately the common perception is that they're a hard, immutable fact. They're not that at all, they're a tool you can use measure how the wider WoW community might view your class. They're not the be-all and end-all of performance.

@Mysterymask
Every fighter always seems to have that one guy who picks a "low" tier character and mops the floor with the pro players. The FGC know better than most that tier lists only say so much and there's much more that goes into picking a character than if they've got fast jabs or meterless reversals.


@Malkiah
What you're describing here is selection bias. You're absolutely right in what you're saying - the top performers don't represent the average player. Analysing just the top 5% of players doesn't reflect what you're going to see in game outside most of the time. Obviously the higher level of content you're doing the more likely you are to run into players in that top 5%, but if you're grouping with them regularly chances are that you're also in that top 5%.

[Mysterymask]

Really the big BIG issue with tier lists is less the list itself and more human behavior. Humans in general don't like answers of "it depends" or "if X is the case the warlocks are amazing however if you don't have X then they will fall behind so you need to have a shadow priest". Even in the real world we are slowly coming to grips of "things are a spectrum" and a good chunk just aren't buying it or actively fighting against it.

Tier lists in wow are "in a vacuum with everything perfectly aligned this is best" but we don't live in a vacuum and there are humans playing these characters

Hell we technically knew this before

[arkanon]

Then Collins really needs to rework that definition. A statistic is not by itself a fact. Without interpretation and context, it is merely a collection of data that has no practical value.

Did you REALLY just claim to have a better understanding of the definition of a word than.....the Collins fucking dictionary?

At some point in your life, you need to accept "maybe I don't know everything, maybe I'm wrong about this". I think that point might be when you start arguing with the dictionary about the definition of a word.

Statistics are facts. If it's not a fact, it's not a statistic, by definition.

[Relapses]

There is no "trouble" with tier lists. The trouble is how this community interprets tier lists. Tier lists on their own are just that: Easy ways to summarize mountains of information into a nice, clean visual which compartmentalizes everything in a way that, at a glance, you know broadly where something lands on the spectrum of good to bad.

When you apply tier lists to the framework of WoW and specifically class balance (which seems to be the topic here), the issue is that the spectrum usually isn't good to bad. It's usually good to very good. And in some cases, so stupidly good that you'd be an idiot not to take this. (SV Hunter in 9.2 springs to mind.) It's because of this that it's kind of bad advice to tell people to play what they will enjoy because if they enjoy BM Hunter in a tier where SV is so ridiculously powerful you're actively trolling by playing BM, you are going to catch flak from people no matter how good you are as a player. Which leads right into one of the biggest blind spots with tier lists: Skill. An MDI-level player playing the statistically worst performing spec in the entire game would still run circles around a FotM reroll at the KSM level. When making a tier list, the creator often assumes that the class is going to be played optimally. And with some classes in WoW, playing optimally and playing the way that a player enjoys are diametrically opposed to one another. (Perhaps optimal play requires a talent set up that the individual player dislikes or something.) Because WoW players always assume they're playing 100% optimally even when they're very obviously not, you end up with weird incongruities where skilled players playing non-meta specs are deprioritized when compared against unskilled players on meta specs. That's not the fault of the tier lists, though, that's just the WoW community (once again) being incredibly bad at understanding how to properly parse information provided to it.

So how do you counteract this?

Easy. 3 steps:

Step 1.) Get good at the game; then
Step 2.) Play a meta spec even if you hate it; and, importantly...
Step 3.) Get good at the game.