Is AMR's gearing tool and it's simming tool accurate and trustworthy?

[Djriff]

Hey, I'm will get back to you all on the diff's between SimC and AMR.

One quick correction @Djriff: When you run a simulation you want to target the margin of error - the number of iterations doesn't really mean much, since 15,000 can get you to 0.1% on one character or class, but 0.25% on another. For stat weights, you can set the margin of error, btw. So you can customize it to be a lower one if you'd like

Also, what makes you think S2M is broken in AMR? I know swol updated the rotation a bit, but the spell was never broken. I know SimC had some troubles, not sure if they fixed that yet or not.

Sure it's class dependent and outside of shadow I don't know what those iteration numbers would be. It's hard to trust a margin of error when the simulation isn't putting out accurate results in the first place. SimC has the same problems, but like we've discussed before I find it's results tab much more informative (minus a couple of things) than the one AMR puts out with a log. I do prefer AMR's log since it's easily filterable and searchable.

Considering the fact that I can do 100k more DPS than the Sim is giving me every single time, it's not accurate in my opinion. It may suffer from the same issues of SimC where the actor for the iteration dies early in StM resulting in a sim that runs for a long time with 0 DPS going on, resulting in a lower DPS number for it. I've mentioned it to Swol before, but nothing was done that I know of.

[LordAmbrosia]

It's been screwy for me. Telling me to change out a trinket for another one, then simulating the new setup as lower DPS than the one before (????)

[Zoopercat]

@Djriff can you link me to a log and simulation where you do 100k more DPS in the log?

[Djriff]

@Djriff can you link me to a log and simulation where you do 100k more DPS in the log?

Not atm, can't pull up AMR or logs. I'll link later in discord.

[Zoopercat]

@Vegas82, are you referring to the simulator? That's what we're talking about, and if you have anything that seems odd about it, let me know so I can look into it.
@Djriff what is your character name and realm? I'll do a search on WCL

[Swol]

Considering the fact that I can do 100k more DPS than the Sim is giving me every single time, it's not accurate in my opinion. It may suffer from the same issues of SimC where the actor for the iteration dies early in StM resulting in a sim that runs for a long time with 0 DPS going on, resulting in a lower DPS number for it. I've mentioned it to Swol before, but nothing was done that I know of.

We've never had a bug like that. The simulation always ends when the player dies.

Djriff - I don't really understand why you are insisting that our simulations are not accurate, considering how much time you and I spent working on them - together.

Here is the top ranked Heroic Ursoc kill by a shadow priest as of the time of this post (560k DPS):
https://www.warcraftlogs.com/reports...-done&source=6

And here is a simulation with that character's exact same setup using my default rotation (470k DPS with 52k std dev):
http://www.askmrrobot.com/wow/simula...2e728a11209716

Top parses will always be "lucky" outlier parses. So at the top end of the standard deviation that is 530k - the actual variance is even larger than the standard deviation, so 560k is a totally believable result for a top parse, and falls within the expectation of the simulation.

On short fights the damage you can do with surrender to madness goes way up. Fight length affects the damage you can do with S2M tremendously. That is why it is such an inherently unbalanced talent that is destroying all top parses. Your damage can vary wildly from fight to fight, on top of that.

If you need help getting your character to simulate to values that mirror the parses you are getting in game, hit me up. The simulator is working, and the default rotation is doing a very good job with a talent that depends so heavily on gear, talents, and fight length. I still have more improvements that I plan to make to help the default rotation scale up to higher item levels for the 7.1 patch as well.

[Djriff]

@Djriff what is your character name and realm? I'll do a search on WCL

<--- It's also in my sig. Stormrage (US).

We've never had a bug like that. The simulation always ends when the player dies.

Which leads to more skewed results imo. It's better to just can that iteration entirely if a player dies early, ie before the fight ends. Simulations are suppose to be perfect play.

Djriff - I don't really understand why you are insisting that our simulations are not accurate, considering how much time you and I spent working on them - together.

The sims themselves just aren't matching up to what I've been seeing is all.

Here is the top ranked Heroic Ursoc kill by a shadow priest as of the time of this post (560k DPS):
https://www.warcraftlogs.com/reports...-done&source=6

And here is a simulation with that character's exact same setup using my default rotation (470k DPS with 52k std dev):
http://www.askmrrobot.com/wow/simula...2e728a11209716

Top parses will always be "lucky" outlier parses. So at the top end of the standard deviation that is 530k - the actual variance is even larger than the standard deviation, so 560k is a totally believable result for a top parse, and falls within the expectation of the simulation.

Luck has very little to do with a 30k difference with StM. RNG will get you to a few more seconds or can save your bacon in a punch, but it's not going to put you up to 30k in difference.

On short fights the damage you can do with surrender to madness goes way up. Fight length affects the damage you can do with S2M tremendously. That is why it is such an inherently unbalanced talent that is destroying all top parses. Your damage can vary wildly from fight to fight, on top of that.

It's not that StM is too good (although admittedly with Mass Hysteria it is a bit on the OP side), the other 2 lvl 100 talents plain suck. Fight length will effect any type of parse regardless of talent, StM just makes the issue worse since you can reach unbelievable amounts of haste, especially if lust is timed in execute phase. Now that's the extreme end of the spectrum, but some fights on normal/heroic it can happen.

[Swol]

<--- It's also in my sig. Stormrage (US).
Which leads to more skewed results imo. It's better to just can that iteration entirely if a player dies early, ie before the fight ends. Simulations are suppose to be perfect play..

The DPS is always calculated based on the total length of the fight, but the simulator stops doing anything as soon as the player dies, to avoid any weird bugs from happening. That is why you get better average DPS by going a little on the short side for S2M start time, to avoid dying too early.

[Yellowfive]

Djriff, I think that there's a little misunderstanding here about how some of these statistics work...

It would be a terrible idea to throw away any iteration where the player dies a little early... that would introduce extreme bias in your simulation results. You would be biased towards trials where you used surrender too late. Unless you have a symmetrical way to throw away attempts where you used it too late, you're doing bad statistics. And even then... it's still bad to throw out all "not perfect" attempts.

Also, say you are doing 500k DPS, and you die 5 seconds before the fight ends because you hit StM too early. That's a 16k DPS loss in a short 2.5 min kill like we're seeing in these top parses. And you can't tell me that you'll get your StM timed to the second in every attempt... it can go the other way too: if you hit it too late and lose a few seconds of extra voidform time, you'll lose significant DPS. This leads to a high standard deviation for StM compared to other specs because there is an extra uncontrollable variable that is generally hard to predict (fight length) that has a larger than usual impact on this spec.

Throw on top of that just the natural variation due to the random nature of the game (critical strikes, proc chances, etc.). You can't just magically decide to ignore the standard deviation when looking at a single trial and say "there's no possible way DPS could vary like that from attempt to attempt." There is, and it does. If the std. dev is 50k, there is a very real chance of getting a value 50k higher than the mean.

Earlier you also mentioned that you can't trust our simulation results because you can't set the number of iterations for the simulator. That indicates a fundamental misunderstanding of how simulation works. You should never simulate a specific number of iterations or even care about the number of iterations. It is a mistake for a monte carlo simulation to even expose that as a setting. Margin of error and confidence interval is all you care about -- you run iterations until you reach the desired level of precision. It takes as many iterations as it takes. And you can adjust the margin of error in our simulator and simc -- NEVER use static iteration counts. ALWAYS use the margin of error options in either simulator.

[FenrizZJ]

.

btw Zoopercat, when I load up my armory and do "best in bags" it tells me to remove one of my rings and doesn't replace it with anything.

seems like an issue.

[Zoopercat]

Ok I was able to find an ursoc log you had done and matched it up to our simulator (matching gear, artifact, talents). Note that when I do a single target default run, I get 260k DPS. When I pick the ursoc script and enter the exact duration of your log I get 338k DPS. So you can see how some simple setups can dramatically change the simulation results.

AMR Ursoc script to match you: 338k DPS http://www.askmrrobot.com/wow/simula...8389da5193eb03
Your log: 321k DPS https://www.warcraftlogs.com/reports...pe=damage-done

You can look at a breakdown of each spell to see how the avg dmg and # of hits line up with our logs.

As for voidform, on the log you had 73% uptime. In the AMR simulator, we had 78% uptime on voidform. Surrender to madness seems to have gotten you last voidform in the logs to last for 84 seconds. In our simulator is lasted 111 seconds.

Hope that info helps clear some things up. As yellow/swol offered, we're happy to help walk you through the setup and analysis of comparing logs to our sims.

- - - Updated - - -
@FenrizZJ - can you give me the export string that is doing that? If one of them is a legendary, I think we know about that and have an update coming soon. If not, it's something else.

@Vegas82 - can you give me some information as to why you think SimC is more accurate? Also, if you link me to your character or a log, I can do an analysis like I just did for DJRiff. We currently have no outstanding bugs for SP, so I'd like to look at your setup.

[Djriff]

The DPS is always calculated based on the total length of the fight, but the simulator stops doing anything as soon as the player dies, to avoid any weird bugs from happening. That is why you get better average DPS by going a little on the short side for S2M start time, to avoid dying too early.

The problem is, that if the simulator can't take the actor to a 3rd Void Torrent consecutively there's an issue somewhere.

Djriff, I think that there's a little misunderstanding here about how some of these statistics work...

It would be a terrible idea to throw away any iteration where the player dies a little early... that would introduce extreme bias in your simulation results. You would be biased towards trials where you used surrender too late. Unless you have a symmetrical way to throw away attempts where you used it too late, you're doing bad statistics. And even then... it's still bad to throw out all "not perfect" attempts.

Also, say you are doing 500k DPS, and you die 5 seconds before the fight ends because you hit StM too early. That's a 16k DPS loss in a short 2.5 min kill like we're seeing in these top parses. And you can't tell me that you'll get your StM timed to the second in every attempt... it can go the other way too: if you hit it too late and lose a few seconds of extra voidform time, you'll lose significant DPS. This leads to a high standard deviation for StM compared to other specs because there is an extra uncontrollable variable that is generally hard to predict (fight length) that has a larger than usual impact on this spec.

I wasn't implying those, it's ideal to die within 5 seconds of the boss or immediately after. Anytime you're alive when the boss falls means you popped StM too late. I'm saying those iterations where your StM Voidform only lasts 70 seconds should be tossed as it brings down the overall DPS and either was an error or a glitch as no amount of RNG with AS is going to produce 20s less of a 90s from fight end StM.

Throw on top of that just the natural variation due to the random nature of the game (critical strikes, proc chances, etc.). You can't just magically decide to ignore the standard deviation when looking at a single trial and say "there's no possible way DPS could vary like that from attempt to attempt." There is, and it does. If the std. dev is 50k, there is a very real chance of getting a value 50k higher than the mean.

I'm not magically deciding anything, I'm just saying that fights where the actor dies way earlier than they should in a sim should be discarded. If that means more iterations so be it.

Earlier you also mentioned that you can't trust our simulation results because you can't set the number of iterations for the simulator. That indicates a fundamental misunderstanding of how simulation works. You should never simulate a specific number of iterations or even care about the number of iterations. It is a mistake for a monte carlo simulation to even expose that as a setting. Margin of error and confidence interval is all you care about -- you run iterations until you reach the desired level of precision. It takes as many iterations as it takes. And you can adjust the margin of error in our simulator and simc -- NEVER use static iteration counts. ALWAYS use the margin of error options in either simulator.

That was a misunderstanding in terms, I'm inclined to agree with you on that.

Ok I was able to find an ursoc log you had done and matched it up to our simulator (matching gear, artifact, talents). Note that when I do a single target default run, I get 260k DPS. When I pick the ursoc script and enter the exact duration of your log I get 338k DPS. So you can see how some simple setups can dramatically change the simulation results.

AMR Ursoc script to match you: 338k DPS http://www.askmrrobot.com/wow/simula...8389da5193eb03
Your log: 321k DPS https://www.warcraftlogs.com/reports...pe=damage-done

You can look at a breakdown of each spell to see how the avg dmg and # of hits line up with our logs.

As for voidform, on the log you had 73% uptime. In the AMR simulator, we had 78% uptime on voidform. Surrender to madness seems to have gotten you last voidform in the logs to last for 84 seconds. In our simulator is lasted 111 seconds.

Hope that info helps clear some things up. As yellow/swol offered, we're happy to help walk you through the setup and analysis of comparing logs to our sims.
I'm curious did you use my armory gear or gear I had in the log? I'm wearing a very different set of gear now.

[Zoopercat]

@Djriff I matched all of your gear from that log. If you scroll down in the report, you can expand the gear section and see that it all matches It's a handy feature so people can verify setups. https://www.warcraftlogs.com/reports...pe=damage-done

[Djriff]

No I know you can do that, I'm still at work so I can't verify that myself. I'll take a look when I get home.

[Zoopercat]

Sorry, you mentioned you were at work before, I forgot!! Let me know if there is anything else I can research until you get home, happy to help

[FenrizZJ]

ome things up. As yellow/swol offered, we're happy to help walk you through the setup and analysis of comparing logs to our sims.

- - - Updated - - -
@FenrizZJ - can you give me the export string that is doing that? If one of them is a legendary, I think we know about that and have an update coming soon. If not, it's something else.

yeah sure here you go:
$45;US;Zul'jin;Fenrìz;Deicide;9;2;110;4:745,8:768;3;.s3;27;1213211;852,1,1,1,1,2,1,1,1,1, 1,1,2,483;3,3,1,3,3,3,1,3,1,1,1,1,1,1;137407b1492b1727b1813,138226b1487b1805,136721b1492b1 727b1813;.q3;128476s16b743t15x137407m1492m1727m1813y819n1487n1805z-1505o1492o1727o1813c111;3s17t16;5165s13v110b764b306b1602t12;553s5v0b-1898b291b5b1602t4x-6499c1;2s6v0b-3375b1482b291b1604t5;39s7v0b-1915b225b81b5t6x0c1;1s10v0b-276b271b1529b80t9x25c1;219s9v0b-1910b1829b77t8;79s11v0b-1881b1805b76t10e5429;2787s8v0b-1896b1820b76t7;134s2v0b-1901b215b1610t2e8;1790s15v0b-1850b318t14e-2;33s1v0b-283b205b1610t1;48s14v0b-1855b325b1b1528t13x-25c1;415s3v0b48b1t3$g\137407\137407\\\@g\138226\138226\\\@g\136721\136721\\\@g\130222\1302 22\\150 _Mastery_\@g\130247\130247\\200 _Agility_\@e\5429\128543\191015\200 _Mastery_\124442=4,124440=35@e\5437\128551\190892\Mark of the Claw\124442=15,124441=12@e\5435\128549\190878\200 _Agility_\124442=10,124440=30,124124=3
$45;US;Zul'jin;Fenrìz;Deicide;9;2;110;4:727,8:761;1;.s1;25;0000000;;;;.q1;133644s14v110b1 507b306b1602t12;553s5v0b-1898b291b5b1602t4x130222c1;2s6v0b-3375b1482b291b1604t5;39s7v0b-1915b225b81b5t6x0c1;1s10v0b-276b271b1529b80t9x25c1;219s9v0b-1910b1829b77t8;29s11v0b-1886b281b1529b80t11x-28c1e5427;50s12v0b-1885b1805b76t10e2;2787s8v0b-1896b1820b76t7;134s2v0b-1901b215b1610t2e8;1790s15v0b-1850b318t14e-2;33s1v0b-283b205b1610t1;48s13v0b-1855b325b1b1528t13x3c1;415s3v0b48b1t3$g\130222\130222\\150 _Mastery_\@g\130247\130247\\200 _Agility_\@g\130219\130219\\150 _CriticalStrike_\@e\5427\128541\191013\200 _CriticalStrike_\124442=4,124440=35@e\5429\128543\191015\200 _Mastery_\124442=4,124440=35@e\5437\128551\190892\Mark of the Claw\124442=15,124441=12@e\5435\128549\190878\200 _Agility_\124442=10,124440=30,124124=3
$45;US;Zul'jin;Fenrìz;Deicide;9;2;110;4:727,8:761;2;.s2;26;0000000;;;;.q2;133644s14v110b1 507b306b1602t12;553s5v0b-1898b291b5b1602t4x130222c1;2s6v0b-3375b1482b291b1604t5;39s7v0b-1915b225b81b5t6x0c1;1s10v0b-276b271b1529b80t9x25c1;219s9v0b-1910b1829b77t8;29s11v0b-1886b281b1529b80t11x-28c1e5427;50s12v0b-1885b1805b76t10e2;2787s8v0b-1896b1820b76t7;134s2v0b-1901b215b1610t2e8;1790s15v0b-1850b318t14e-2;33s1v0b-283b205b1610t1;48s13v0b-1855b325b1b1528t13x3c1;415s3v0b48b1t3$g\130222\130222\\150 _Mastery_\@g\130247\130247\\200 _Agility_\@g\130219\130219\\150 _CriticalStrike_\@e\5427\128541\191013\200 _CriticalStrike_\124442=4,124440=35@e\5429\128543\191015\200 _Mastery_\124442=4,124440=35@e\5437\128551\190892\Mark of the Claw\124442=15,124441=12@e\5435\128549\190878\200 _Agility_\124442=10,124440=30,124124=3

[Deleted]

Hey, I'm will get back to you all on the diff's between SimC and AMR.

Hello, please dont forget about us