Why you shouldn't use top 100 WoL parses to discuss balance

[Deleted]

Often, I see people complain about their class, stating that, because their class or spec doesn't appear in the top 100 of a certain fight, their spec is bad. this is a fake fallacy. people are forgetting that WoW is an RPG, and RPGs are notorious for one thing: RNG.

i'm going to use a general example using bland terms. suppose you are someone of class B, and your friend is of class A. there are 100,000 people who play each class. the numbers might be slightly off here and there, but not by so much that correct numbers would invalidate it.

Class A is a critproc sorceror: he relies on crits that have a chance to proc a spell that does much damage. the spell he fires does 100 damage every 3 seconds, has a 30% chance to crit for double damage, and those crits have a 50% chance to automatically cast an instant stronger spell that does double the damage of the original spell, and has 30% chance to crit as well, which has another 50% chance to proc the spell again, but those spells cannot proc it again. so for every 1000 of the first spell he casts, he does 700 normal hits for 100 each and 300 crits for 200 each. of those 300 crits, 100 proc a spell hit and another 50 proc a spell crit, and of those 17 proc the spell hit again, and another 8 procs of procs crit. so he does 70K from hit, 60K from crit, 20K from crit.proc.hit, 20K from crit.proc.crit, 3.4K from crit.proc.crit.proc.hit and 3.2K from crit.proc.crit.proc.crit, for a total of 176600 damage.

class B is a channeldot cleric: he casts 1 damage over time spell that does 77 damage every 3 seconds for 18 seconds and then does a channel that does 69 damage every 3 seconds over 15 seconds and has a 5% chance to proc an extra tick. he also has a 30% crit chance. he casts the dot and then does a 15 second channel and repeats this every 18 seconds. so over 1002 ticks of his DoT ( about same time period as the sorceror above), he does 100.3K damage from the DoT and can cast 167 channels, doing 835 ticks for 74900, with 42 extra ticks for a total of 3767 damage from the procs and a total damage of 178967 damage. he does steady damage.

so in the average scenario, B does slightly more damage than A, about 1%. however, you can clearly see that A is much more reliant on RNG. If B had a 10% increased proc rate, he would have 108K from the dot, 80661 from the channel and 4057 from the extra ticks, for a total of 192718. however, if A experienced a 10% higher crit rate, his spell distribution would shift from 600 normal hits for 100 each, 400 crit for 200, 120 proc hits for 200, 80 proc crits for 400, 24 proc of proc hits for 200 and 16 proc of proc crits for 400. the resulting numbers are 60K, 80k, 24K, 32k, 4.8K and 6.4K, for a total of 207,200. so by giving both a 10% extra crit rate, the gap between them has switched from 1% in favor of B, to 7% in the favor of A.

you might think: "But Nzall, we're comparing equal gear and fight length, how could the amounts of crits be different?" that's where RNG comes in. because you never cast spells for 3000 seconds in a fight. you often cast only 1/10th of that time, which means only 100 spells and therefore much more susceptible to RNG. another factor is that your crits aren't evenly distributed. sometimes you get lucky, and 40 of your 100 spells crit. other times, you get unlucky and only 25 of your 100 spells crit. that's the devil of RNG.

if we assume that every distribution of crits from 20% to 40% has 5% chance of occuring across all players, that means that 5,000 players of class B will do 1% higher damage than their comrades from A, but also 5,000 players of class A will have 7% higher damage than their friends from B. but on WoL, you'll only see the 100 best players overall, so you're only going to see 100 players of class A, while their average DPS with a gear-related critrate is lower than that of B. remember, both players had the same gear, the same stats and the same practical crit rate.

the same happens in WoW. for example, back in WotLK fire mages appeared on those logs as dominant, because their range of DPS encompassed a much larger range simply due to RNG. those players weren't just skilled or geared, they also had the luck to get that nice hot streak chain of pyroblast crits. meanwhile, elemental shamans had a strict rotation that wasn't as affected by crit or RNG, and had a much smaller DPS range. so if you had a mage and a shaman of equal gear and skill (and assume that their average DPS was balanced exactly the same), and a lot of luck for both, the mage would always be on top because he simply got a larger benefit from being lucky.

you see that in current logs as well. take the theoretical crit rates and proc rates the top 10 players have raid buffed, and compare it to the actual observed proc and crit rates in their logs, and you will notice that most of them were rather lucky as well. they had more crits, or a spell with a 10% proc rate proccing 15% of the time, and their trinkets always appear to line up with damage boost phases and cooldowns active. in addition, you will nearly always see the same classes leading, because they get a much bigger bonus from that extra luck.

luck is a factor in this game, and you shouldn't discount it. blizzard balances around average DPS, because if they were to balance around maximum DPS with maximum luck, you would either have every spec play exactly the same with exactly the same stats, or the average DPS would not be the same and thus instead of disparities at the top 5%, you would have disparities on the middle 50%, where the majority of the players are.

so next time that you complain that your class is not competitive because it isn't represented in the top 100, take a step back and consider that not everyone is equally affected by luck. Blizzard can certainly balance everyone's average DPS around a certain point, but they can't balance both average and maximum DPS at the same time without massive homogenization.

[l4808]

Often, I see people complain about their class, stating that, because their class or spec doesn't appear in the top 100 of a certain fight, their spec is bad. this is a fake fallacy. people are forgetting that WoW is an RPG, and RPGs are notorious for one thing: RNG.

i'm going to use a general example using bland terms. suppose you are someone of class B, and your friend is of class A. there are 100,000 people who play each class. the numbers might be slightly off here and there, but not by so much that correct numbers would invalidate it.

Class A is a critproc sorceror: he relies on crits that have a chance to proc a spell that does much damage. the spell he fires does 100 damage every 3 seconds, has a 30% chance to crit for double damage, and those crits have a 50% chance to automatically cast an instant stronger spell that does double the damage of the original spell, and has 30% chance to crit as well, which has another 50% chance to proc the spell again, but those spells cannot proc it again. so for every 1000 of the first spell he casts, he does 700 normal hits for 100 each and 300 crits for 200 each. of those 300 crits, 100 proc a spell hit and another 50 proc a spell crit, and of those 17 proc the spell hit again, and another 8 procs of procs crit. so he does 70K from hit, 60K from crit, 20K from crit.proc.hit, 20K from crit.proc.crit, 3.4K from crit.proc.crit.proc.hit and 3.2K from crit.proc.crit.proc.crit, for a total of 176600 damage.

class B is a channeldot cleric: he casts 1 damage over time spell that does 77 damage every 3 seconds for 18 seconds and then does a channel that does 69 damage every 3 seconds over 15 seconds and has a 5% chance to proc an extra tick. he also has a 30% crit chance. he casts the dot and then does a 15 second channel and repeats this every 18 seconds. so over 1002 ticks of his DoT ( about same time period as the sorceror above), he does 100.3K damage from the DoT and can cast 167 channels, doing 835 ticks for 74900, with 42 extra ticks for a total of 3767 damage from the procs and a total damage of 178967 damage. he does steady damage.

so in the average scenario, B does slightly more damage than A, about 1%. however, you can clearly see that A is much more reliant on RNG. If B had a 10% increased proc rate, he would have 108K from the dot, 80661 from the channel and 4057 from the extra ticks, for a total of 192718. however, if A experienced a 10% higher crit rate, his spell distribution would shift from 600 normal hits for 100 each, 400 crit for 200, 120 proc hits for 200, 80 proc crits for 400, 24 proc of proc hits for 200 and 16 proc of proc crits for 400. the resulting numbers are 60K, 80k, 24K, 32k, 4.8K and 6.4K, for a total of 207,200. so by giving both a 10% extra crit rate, the gap between them has switched from 1% in favor of B, to 7% in the favor of A.

you might think: "But Nzall, we're comparing equal gear and fight length, how could the amounts of crits be different?" that's where RNG comes in. because you never cast spells for 3000 seconds in a fight. you often cast only 1/10th of that time, which means only 100 spells and therefore much more susceptible to RNG. another factor is that your crits aren't evenly distributed. sometimes you get lucky, and 40 of your 100 spells crit. other times, you get unlucky and only 25 of your 100 spells crit. that's the devil of RNG.

if we assume that every distribution of crits from 20% to 40% has 5% chance of occuring across all players, that means that 5,000 players of class B will do 1% higher damage than their comrades from A, but also 5,000 players of class A will have 7% higher damage than their friends from B. but on WoL, you'll only see the 100 best players overall, so you're only going to see 100 players of class A, while their average DPS with a gear-related critrate is lower than that of B. remember, both players had the same gear, the same stats and the same practical crit rate.

the same happens in WoW. for example, back in WotLK fire mages appeared on those logs as dominant, because their range of DPS encompassed a much larger range simply due to RNG. those players weren't just skilled or geared, they also had the luck to get that nice hot streak chain of pyroblast crits. meanwhile, elemental shamans had a strict rotation that wasn't as affected by crit or RNG, and had a much smaller DPS range. so if you had a mage and a shaman of equal gear and skill (and assume that their average DPS was balanced exactly the same), and a lot of luck for both, the mage would always be on top because he simply got a larger benefit from being lucky.

you see that in current logs as well. take the theoretical crit rates and proc rates the top 10 players have raid buffed, and compare it to the actual observed proc and crit rates in their logs, and you will notice that most of them were rather lucky as well. they had more crits, or a spell with a 10% proc rate proccing 15% of the time, and their trinkets always appear to line up with damage boost phases and cooldowns active. in addition, you will nearly always see the same classes leading, because they get a much bigger bonus from that extra luck.

luck is a factor in this game, and you shouldn't discount it. blizzard balances around average DPS, because if they were to balance around maximum DPS with maximum luck, you would either have every spec play exactly the same with exactly the same stats, or the average DPS would not be the same and thus instead of disparities at the top 5%, you would have disparities on the middle 50%, where the majority of the players are.

so next time that you complain that your class is not competitive because it isn't represented in the top 100, take a step back and consider that not everyone is equally affected by luck. Blizzard can certainly balance everyone's average DPS around a certain point, but they can't balance both average and maximum DPS at the same time without massive homogenization.

If one class is not represented on top list on ANY boss fight, it has nothing to do with RNG.

There is nothing called LUCK when you're going through statistics. There is something called probability, and the probability decides the results in the long run.

And all the really good players are able to get ranked within their specc range pretty often. If you are that good, and your class/specc is still not represented on the combined top list at all, then of course you have reason to mention it.

example: Why should a good "INSERT CLASS HERE"'s theoretical "top" dps on boss X be 30% lower than the others?

[RayuEU]

1. Top 100 lists of DPS are not a fair representation of the average player. Some classes punish and reward more than others for skill or lack of skill. That is enough a reason not to look at WoL for balance.
2. RNG is not as big a factor in top DPS deviation as you make it out to be. Skill, class balance and the encounter's specifics are much more important.
3. Your analysis is poor. If you wish to make a point, post results of systematic log analysis and not a situation you came up with to back up a theory you came up with.

[Deleted]

1. Top 100 lists of DPS are not a fair representation of the average player. Some classes punish and reward more than others for skill or lack of skill. That is enough a reason not to look at WoL for balance.

Do you think classes should be balanced for top players or average players?

[Deleted]

If one class is not represented on top list on ANY boss fight, it has nothing to do with RNG.

There is nothing called LUCK when you're going through statistics. There is something called probability, and the probability decides the results in the long run.

And all the really good players are able to get ranked within their specc range pretty often. If you are that good, and your class/specc is still not represented on the combined top list at all, then of course you have reason to mention it.

example: Why should a good "INSERT CLASS HERE"'s theoretical "top" dps on boss X be 30% lower than the others?

luck is just having the results be on your side. i've mentioned that as well. if you got a 5% probability to be 7% higher than the other DPS, with so many players, the top DPS will usually be that 5%. but if you look at, say, Feng the Accursed DPS, you'll see that most specs are between 80 and 90K DPS, with 1 or 2 outliers at 100K, but there is no 30% difference.

WoL only shows the top 10 players on their rankings, but if you click through, you'll see that most player are actually pretty close together, and it's just 1 or 2 outliers with massive RNG benefits that are extremely high. but most classes are close together. for example, raidbots gives hunters as viable players, coming within 5K DPS of the second highest class (I exclude the highest class because it's another 5K up, and you should do that because statistics usually remove the top and bottom 5% because they are often skewing the results.

[RayuEU]

Do you think classes should be balanced for top players or average players?

Preferably both. I will clarify my statement; I mean that unless you are one of the few-hundred players that actually have the skill, gear and dedication to get into those lists, you have no valid point to make by pointing at them.

[Akraen]

Often, I see people complain about their class, stating that, because their class or spec doesn't appear in the top 100 of a certain fight, their spec is bad. this is a fake fallacy. people are forgetting that WoW is an RPG, and RPGs are notorious for one thing: RNG.

As someone who just made a large post regarding Frost vs. Fire, and how the gap is widening among the top 100 parses, I very nearly think your unsubstantiated analysis is a response to me.
(my 5.1 concerns thread: http://www.mmo-champion.com/threads/...rns-before-5-1)

i'm going to use a general example using bland terms. suppose you are someone of class B, and your friend is of class A. there are 100,000 people who play each class. the numbers might be slightly off here and there, but not by so much that correct numbers would invalidate it.

...

Your A vs. B thing is a fine analogy for an argument of playstyles. However, that's not the point you're trying to make. Also, if you were going to use critproc vs. channeldot analogy, then it would stand to reason that on some encounters the sustainability of the channeldot would beat the bursty RNG of the critproc. The problem with RNG though is that when balancing classes you would want to take note of the upper limit, especially when looking at DPS checks of encounters. The upper limit also takes into consideration player skill and guild skill.

The top 5 critprocs vs. the top 5 channeldots will of course favor the critprocs. However, when you're looking at damage potential and what is required in a progression guild, suddenly the argument that the bad RNG guys at the bottom are averaging it out becomes moot. It is likely that a guild will wipe if their top fire mage has horrible RNG, then succeed on the next attempt when his RNG is good. That doesn't mean suddenly frost is viable for its consistency-- it means there's too much importance on the upper limit potential of class design.

so in the average scenario, B does slightly more damage than A, about 1%. however, you can clearly see that A is much more reliant on RNG. If B had a 10% increased proc rate, he would have 108K from the dot, 80661 from the channel and 4057 from the extra ticks, for a total of 192718. however, if A experienced a 10% higher crit rate, his spell distribution would shift from 600 normal hits for 100 each, 400 crit for 200, 120 proc hits for 200, 80 proc crits for 400, 24 proc of proc hits for 200 and 16 proc of proc crits for 400. the resulting numbers are 60K, 80k, 24K, 32k, 4.8K and 6.4K, for a total of 207,200. so by giving both a 10% extra crit rate, the gap between them has switched from 1% in favor of B, to 7% in the favor of A.

I'm going to reiterate this, you're giving an example of 1%. That's not the reality. Frost vs. Fire doesn't see frost ahead in any scenario anymore. I see what you're trying to math out here, but it's too hypothetical-- anyone can make up numbers. Look for a top 25 guild fire mage that did lower DPS than a top 25 guild frost mage. It won't happen because of what I already stated: the bad RNG boss attempts are likely wipes. Some might slip through, but it's the kills that count. Even if it was so incredibly frustrating to play as fire (or whatever other spec that works like it for other classes), players will still choose the spec for progression because of the upper limit on the spec's potential output.

Say frost at ilvl 496 with 96% active time on an encounter does a lower limit (bad RNG) of 74,000 DPS and an upper limit (best RNG) of 104,000 DPS.
Fire would, at ilvl 496 and 96% active time have a lowerl imit of 80,000 DPS and an upper limit (best RNG) of 164,000 DPS.

The range between the limits reflects how much more RNG is in the fire spec, but because of the lower limit being higher and the upper limit being ridiculously higher, nearly all progression guilds will require that their mages play to the maximum output potential.

This point is what frustrates people to make these posts regarding top 100 parses. The idea here, which I think you're missing, is that you could compare the top 5 frost mages-- or even the top 1 frost mage, who had everything in the world handed to him, tricks of the trade, perfectly timed heroism, was able to tunnel boss/avoid mechanics to get a good parse, and that perfectly ideal RNG frost mage with all planets aligned will still be a good 20-30k below a fire mage who had average RNG.

So basically, you may have quite the argument on comparing top 100 parses, but when the result is the same given the top parser of one spec who had ideal RNG and could still not compare to a rank #100 or #200 of another class-- then the point is still valid and the classes are clearly in a state of imbalance, with no choice given to the raider but to be pigeon-holed.

you might think: "But Nzall, we're comparing equal gear and fight length, how could the amounts of crits be different?" that's where RNG comes in. because you never cast spells for 3000 seconds in a fight. you often cast only 1/10th of that time, which means only 100 spells and therefore much more susceptible to RNG. another factor is that your crits aren't evenly distributed. sometimes you get lucky, and 40 of your 100 spells crit. other times, you get unlucky and only 25 of your 100 spells crit. that's the devil of RNG.

RNG is alive and well in all classes. Frost might be less RNG-dependent than fire, but not by as much as you're making it out to be. Frost RNG is not based off crit nearly as much as it is based off simple procs with varying ICDs. RNG can be negated by both frost and fire rather easily with proper timing and use of addons such as TellMeWhen. This is why you don't see top 25M guilds parsing bad numbers-- they're negating a lot of bad play which is incorrectly identified as RNG.

if we assume that every distribution of crits from 20% to 40% has 5% chance of occuring across all players, that means that 5,000 players of class B will do 1% higher damage than their comrades from A, but also 5,000 players of class A will have 7% higher damage than their friends from B. but on WoL, you'll only see the 100 best players overall, so you're only going to see 100 players of class A, while their average DPS with a gear-related critrate is lower than that of B. remember, both players had the same gear, the same stats and the same practical crit rate.

You can view more than top 100 players, and the results are often consistent. For some classes the gap of one spec over another does shrink a little when you incorporate all players. Let me show you some examples:

All parses, ranking not including, from all players who submitted data to WoL in the past 60 days:
http://raidbots.com/dpsbot/Spec_Scor...default/#7vvvv

That's not top 100, dude, and the specs are still in line with people's complaints. Let's get more detailed, shall we?

Vizier 25N - All parses, NOT top 100: http://raidbots.com/dpsbot/Imperial_.../60/default/#7
---Fire has a 6149 DPS lead over frost and 16447 DPS lead over arcane
Vizier 25N - Top 100: http://raidbots.com/dpsbot/Imperial_.../60/default/#7
---Fire has a 23381 DPS lead over frost and 45555.5 DPS lead over arcane
Vizier 25N - 95th Percentile of all parses (so the top 5% of fire players, top 5% of frost players, top 5% of arcane players, to filter out the riff-raff): http://raidbots.com/dpsbot/Imperial_...l/14/60/p95/#7
---Fire has a 10096 DPS lead over frost and a 19777 DPS lead over arcane
Feng the Accursed 25H - All parses, NOT the top 100: http://raidbots.com/dpsbot/Feng_the_...4/60/median/#7
---Fire has a 4584 DPS lead over frost and a 10877 DPS lead over arcane
Feng the Accursed 25H - Top 100: http://raidbots.com/dpsbot/Feng_the_...4/60/median/#7
---Fire has a 27305.5 DPS lead over frost and a 33598.5 DPS lead over arcane
Feng the Accursed 25H - 95th Percentile of all parses
---Fire has a 4441.4 DPS lead over frost and a 18428.2 DPS lead over arcane

I could keep going. Every fight has results like this. The problem is your stance on the issue is just as bad as the people whom you are accusing of misrepresenting data. I like the point you're trying to make, because if you were right, me playing frost would be justified. In reality, I am hurting the progression of my guild a little bit by being frost (I'm hoping 4pc will be a saving grace, but I doubt it).

Take note that once you do deviate away from the top 100s, you are able to see the gap is much, MUCH smaller. The sky isn't falling quite as bad as some make it out to be-- but it's still falling. The gap between fire and the other specs is growing over time, which is the argument of my thread. Even if the gap was smaller than it is now, it's still growing. The growing gap is indicative of a favorable scaling for fire, which leads to Dragon Soul, basically. It doesn't mean one spec is bad, it just means the wide-range RNG spec needs a little less of a wide-range when current boss encounters have strict DPS checks.

These points would be moot if encounters didn't fall over with amazing DPS.

You're stating the same thing in the final 4 paragraphs of your post, so I'm going to summarize how you're coming across:
Luck & RNG are part of the game, and the top 10 players have all the best buffs and luck on top of it. Their trinkets are lined up and everything is giving them a strong bonus which helps explain why the upper limit is so high for them.

I'm countering you with one simple statement.
The same luck and RNG, with trinkets and buffs is going to happen to the other spec too.

Now, I encourage you to do a little more research. If you weren't talking about mages-- you might be correct with some other specs-- I don't claim to know them. But mages are one thing I research extremely heavily and I can say with certainty... your argument is invalid, sir.

[Deleted]

make an TL: DR at end of post.

[Deleted]

Akraen, you make some really good points. i'll try to address some of them.

1) it wasn't aimed at mages in particular. it was aimed at cross-class balance. more precisely, i was reading the hunter forums and in a thread discussing the state of the class, someone stated that hunters couldn't compete with other specs because they didn't appear in the top raid logs.
2) my channeldot and critproc were just examples of how RNG can affect 1 class more than the other. I wasn't aiming to describe an actual ingame situation. you just interpreteded that as frost and fire.
3) my point wasn't that RNG gives some classes a higher variance than others, although I did focus on that aspect. the point i wanted to make, but likely came across incomplete, is that because RNG affects some classes more than others, when balancing, you'll have to choose between having each spec do equal DPS when the stars align, or have each spec do the same damage on average DPS, or have each spec play exactly the same but with balance just right. to compare that to your mage parrallel with madeup numbers:

Frost DPS is on average 90K DPS. if they have bad RNG, they do 80K, if they have good RNG, they do 100K.
fire DPS is also on average 90, but bad RNG means 60K DPS and good RNG means 120K DPS.

in this case, both classes are balanced for 90K DPS, but if the fire mage has a good streak, he does 20K more than frost.

the other extreme, is that both do 110K DPS with maximum RNG, but frost does 100K average and fire does 80K average. if they both got good streaks of RNG, they're equal, but that's a relatively rare occurence, and most of the time they're doing average DPS, which puts fire behind in this scenario.

now, I realize that the numbers are completely different in the actual game, and that fire is currently always ahead, but they need to balance in some way. and if you look at simulationcraft, which gives accurate numbers over a large sample size and thus shows a pretty reasonable measure of average DPS, you'll see that frost and fire are in an average situation pretty close with fire slightly in the lead. however, fire mage variance is double that of frost. now, this is on a patchwerk fight, which basically never happens. the numbers are close if you can just fire on the boss at will with high skill and average luck. however, if you get high luck ratios, you're either going to need extreme homogenization to ensure that everyone is affected equally by luck, or you'll have discrepancies where 1 class gets affected by it much more than another.

and that's what i'm partially getting at: you need to choose how to balance the game. if you balance for average DPS, you'll have statistical outliers when luck is involved, which favors the RNG-heavy class. if you balance for maximum DPS, you'll have averages that are further apart, which favors the consistent classes. making this choice is like choosing what skill level to balance PvP around. either you balance around average skill, which mean highly skilled players playing high skillcap classes are benefited; or you balance around top skill, which means that depending on the skill ceiling for a class, you'll have players that can't progress because their classes skill range is bigger; or you give every class and spec have the same skill cap and skill distribution, which usually means homogenization and loss of class uniqueness.

[Akraen]

Simcraft, especially for mages, has yet to be remotely accurate in any situation in-game, even on target dummies. Please, please, please don't cite Simcraft.

Your point is valid for the very point you're trying to make - however for the subject title of your post, it is a little deceptive and really doesn't do anything to prove the point implied by the title of your thread. You're essentially asking the community not to analyze data, why? Do the threads bother you in some way?

If hunters are posting completely inaccurate analysis of data, then I recommend posting the data that contradicts them rather than asking them to stop posting data. That's what people did on my thread when they said the gap is definitely there but not as strongly as the top 100s that I posted in the OP. They were right to call me out on the extreme approach to the balance topic I initiated, but in the end it seems nearly everyone is in agreement that fire is either too high, frost is too low, or the scaling of each can be tweaked.

Blizzard has less power than they think with regard to how they wish to balance the game. They can choose to balance it around average or median DPS all they want, but it will be the upper limit outliers that get progression kills first. The spotlight is on them, then the community adjusts accordingly-- and tens of thousands of raiders are forced into alignment of what is "best."

[Rollo]

I've personally never used WoL to compare my classes dps to that of another classes dps. Some fights just favor the style if some classes, or multiply the inherent differences between. I use it more to see what I can r and should e doing with my class, in comparison to others of my class.