The Trouble with Tier lists...

[Biomega]

@Relapses @Biomega @Ryzeth

yeah that was a rhetorical question - the answer(s) are as obvious as they are myriad, and serve to highlight the vast difference in raid logging and the resulting tier lists compared to trying to make tier lists out of ridiculously inconsequential M+ data.

It's still a good idea to explain it to people for whom it may not be as obvious. Logs are an incredible tool - and also probably the tool used the least correctly by the most people.

And in a way that dovetails nicely with tier lists. Sure they're not actually THAT useful, but they're also something that is wildly misused or misinterpreted SO MUCH. There's at least some merit in relative rankings (because that's what tier lists are, really) because liberating yourself from absolute numbers can help you articulate nuance better. But they're incredibly context-sensitive, and most people can't or simply just don't want to get into that nuance. They want a simple answer to a simple question, and don't want people telling them there ARE no simple questions OR simple answers, at least not in the way they want.

At the end of the day, it's about a very primitive relation: people want to pick the thing that makes them perform the best. So either they want information before they choose, or they want information that validates their choice. And they're rarely interested in the three pages it would take to explain everything - just like with a log, where they don't actually go into detailed graphs and timelines and check what's what, but just go "lol not even orange parse, scrub" after a 5-second look at the first column of data.

[chiddie]

Because most people running high keys don't care about parsing in them, so there's no real point to log. If nobody's uploading logs for them, then you're... Not gonna see logs for them... It's not as if the game automatically uploads fights to warcraftlogs, it's entirely opt-in and the website already has support for m+ logs as is.



The problem with tier lists have always been that most of them don't really take into account group utility. An enhance Shaman being 8th on a tier list isn't gonna "matter" when their windfury totem is giving the rogues/warriors/dks etc so much more dps as well, that logs don't show. Same can be said for something like a Shadow Priest... Logs don't reflect the increased dps of whoever they're giving PI to.

Also tier lists in multiplayer games do not take into account that often it’s not you vs the environment but you plus other X people vs the environment. No matter how good you are in playing a spec, you won’t win nor lose alone.

You can tier synergies also but this adds a lot of complexity layers.

[ArenaDk]

Really like how this one pvp tier list recently went.

Here is S A B tier etc. for the average player, BUT this C tier class is S tier in high rated and so on. So it would probably always be best to compare the aversge content and the high end content when going through statistics and make clear which classes are only S tier in high end content or in casual content so there is less suprise when class X isnt performing well, even though the tier list showed its the best class for mythic +25 or something

[Raisei]

Nowadays however what I have come to realize is that Tier lists are mostly inaccurate and just thrown together with sim data and theory crafting.

Mhmm, you are missunderstanding something. "Anecdotal evidence" is a hell of a lot more inaccurate then the computer-generated sims we have these days. Simply because the amount of testing is incomparable. Even if I do 100 dungeons a month (very unlikely) to collect numbers, the sim tests a few thousand times in a fraction of the time.

Now, that being said, you are not wrong. Both methods are in essence inaccurate because they do not take many things into consideration. A sim is basically a target dummy simulation. It uses all abilities at the perfect time, without any interruptions.

Human, or most of them, are unable to be this perfect, plus we have mechanics to do in almost all content that serves to interrupt our rotations.

So the tierlists basically show the absolute best a class can perform in a vaccuum, it is not a statement about what you should be doing or will be doing.

The networth then comes from comparing these "sims in vaccuum" with each other. For example in finding the best gear. "If both sims are done in a vaccuum, how does this piece of gear create a difference" is the question the sim answers.

For Tierlists, it is the comparison of the absolute best each class can perform against other class's best. This does not mean that Player A with an S-Tier class cannot be beaten by Player B with a C-Tier class, because Player skill will shift the numbers around a lot. It just tells you how the numbers look when fictional completely equal players would use those classes.

Personally I really do not give a lot about them. I am convinced and have seen evidence that even lower Tier classes can perform exceptional if the player behind it is good, while I have seen S-Tier specs perform poorly because the player barely knows how to play it.

"Play what you like" is a good rule tbh, because it ensures that a player will actually put the effort in to master a class whereas a reroller that just picks up the FOTM class is just carried by that classes performance and will fail if there are nerfs.

The real trouble with Tier lists is that many many players are not understanding the limit of their information and think that every other class instantly is unplayable. Hence in Pugs it is usually the FOTM classes that are sought after.

[huth]

Wait till you find out an average is a fact. I really think you should open up a dictionary some time, it's gonna blow your mind. Statistics are facts, and that is.........a fact.

No, it's not. They lack some important qualities for that. Otherwise, any random number would be a fact even when it doesn't refer to anything and is not actually real.

[arkanon]

No, it's not. They lack some important qualities for that. Otherwise, any random number would be a fact even when it doesn't refer to anything and is not actually real.

What you said is completely nonsensical. Averages are facts. Statistics are facts. You are going against the smartest minds the planet has ever seen when you refuse to accept that.

[Zodiark]

Mhmm, you are missunderstanding something. "Anecdotal evidence" is a hell of a lot more inaccurate then the computer-generated sims we have these days. Simply because the amount of testing is incomparable. Even if I do 100 dungeons a month (very unlikely) to collect numbers, the sim tests a few thousand times in a fraction of the time.

Now, that being said, you are not wrong. Both methods are in essence inaccurate because they do not take many things into consideration. A sim is basically a target dummy simulation. It uses all abilities at the perfect time, without any interruptions.

Human, or most of them, are unable to be this perfect, plus we have mechanics to do in almost all content that serves to interrupt our rotations.

So the tierlists basically show the absolute best a class can perform in a vaccuum, it is not a statement about what you should be doing or will be doing.

The networth then comes from comparing these "sims in vaccuum" with each other. For example in finding the best gear. "If both sims are done in a vaccuum, how does this piece of gear create a difference" is the question the sim answers.

For Tierlists, it is the comparison of the absolute best each class can perform against other class's best. This does not mean that Player A with an S-Tier class cannot be beaten by Player B with a C-Tier class, because Player skill will shift the numbers around a lot. It just tells you how the numbers look when fictional completely equal players would use those classes.

Personally I really do not give a lot about them. I am convinced and have seen evidence that even lower Tier classes can perform exceptional if the player behind it is good, while I have seen S-Tier specs perform poorly because the player barely knows how to play it.

"Play what you like" is a good rule tbh, because it ensures that a player will actually put the effort in to master a class whereas a reroller that just picks up the FOTM class is just carried by that classes performance and will fail if there are nerfs.

The real trouble with Tier lists is that many many players are not understanding the limit of their information and think that every other class instantly is unplayable. Hence in Pugs it is usually the FOTM classes that are sought after.

Real world evidence maters quite a bit because when you run content with actual people and see their parses consistently in-spite of what is supposed to be gospel on a Tier list then you TOO will become a believer.

I will argue that Tier Lists partly became popular because of people in MMO's becoming more and more independent and less social overall as games evolved. The quality of life features that we have grown to enjoy in games like WoW for instance have made it so players only group up when necessary. Look I'm guilty of this myself. However if you mostly only play solo then when the time comes that you DO need to group up with other players how do you pick other members of your group?

In comes Tier Lists. I get it. Also who has the time to group with consistently each spec with multiple different players to see how they perform in actual encounters?

So what's the TLDR; ? Maybe at best Tier Lists might be a good place to start if you need to get an example of how a spec is performing currently everything else being equal. But in the end you should always try to bring the player to the group not the spec. Because some people can make the lowest Tier specs do more damage than the highest and vice versa.

[BeepBoo]

Statistics are facts.

They're only facts insofar as using all of the appropriate qualifiers, and after stringing together the alphabet soup of qualifiers, they usually lose their meaning OR get abused incorrectly by the people quoting them.

"This spec, all else equal, should perform worse than this spec"

The issue I take is with people misusing the stats in inappropriate ways to make snap decisions because they're too lazy to account for the statistical variances and do their own analysis on each individual person, but that's always the case with stats. Idiots gonna be lazy idiots that wantonly apply stats. More news at 11.