is AskMrRobot simulator reliable??

[Deleted]

I would like to know if any of you guys use their website to sim talents etc and if its accurate ?

I did a few sims but my sim dps jumps up and down each time i sim same stuff :/, just want to know if its up to date so i know im using the right dps talents

[Zoopercat]

Hey Alfredoss, I'm from Ask Mr. Robot. Of course you'd expect me to say it's reliable, right? So here is some information to give you that might help give you confidence in the simulator.

1. We check all spell data 3 ways. First, from the game data. Second, we go in-game and test it. Third, we compare sim reports to actual combat log data. These checks help find things like a spell that is supposed to be buffed by X, but isn't. Etc.

2. Our spell wiki makes the simulator very open source, and people can check on our spell implementations (a fourth method of verification, I suppose!). http://www.askmrrobot.com/wow/theory...p&version=live

This is all said with an asterisk - since 7.1.5 came out, we are still verifying data (step 2 is mostly done, step 3 (comparing to logs) will be started this upcoming week).

Can you link me to a couple reports where the DPS is jumping around? I will take a look at it and see what I can find

[NexiiTank]

I would like to know if any of you guys use their website to sim talents etc and if its accurate ?

I did a few sims but my sim dps jumps up and down each time i sim same stuff :/, just want to know if its up to date so i know im using the right dps talents

Using your own stat weights it can be reliable as far as upgrades and comparing gear. Using their stats it is unreliable at best and Id say most of the more serious WoW community doesnt use it. It seems they are trying to reinvent using methods that are inaccurate so far, and trying harder to discredit theorycrafters than trying to work with them to actually improve the work being done. Just my opinion, but I know many agree with me, including those in the hunter discord that rejected the Legion release work that was done.

[Zoopercat]

@Zenimara Can you give me a specific example of what is unreliable on AMR? Either I can walk you through the proper setup, or find and fix a bug

FWIW, we test the accuracy in 3 ways.
1. We compare out simulator wiki (where the spells are for anyone to see and edit), to in game data.
2. We do in game testing to make sure everything is confirmed (what can crit, get buffed, actual damage, etc)
3. We compare our simulation results to logs on WCL. Here's an example that I just did for a rogue, and as you can see, our sims (including the rotation), match up extremely well.

That being said, we haven't gotten to triple check everything for 7.1.5, so I'm sure there will be some adjustments over the next couple of weeks.

Also, as for stat weights, here's some info on what we do to make sure they are really good:

1. We look at 1000s of data points (SimC only looks at 2 per stat, which isn't enough to do real statistical analysis on).
2. We look at how stats interact (SimC's stat weight feature looks at each stat in isolation. However, if you use SimC's reforge plots, those DO look at stat interactions, so if you're using SimC make sure to do that).
3. We also have a new system that is a step above stat weights, Machine Learning (it's still in beta). Stat weights don't balance stats well or work well with stat interactions. For example if mastery gets better as you get more crit, stat weights don't tell you that. They say that crit is best and mastery is 2nd best. Stat weights aren't able to maintain a ratio or any other necessary interaction. With Machine Learning, we ARE able to account for stat interactions. We are also able to see what is actually acheiveable with the gear you have available to you and adjust the best setups from there. Here's a blog post that talks more about machine learning.

and trying harder to discredit theorycrafters than trying to work with them to actually improve the work being done.

Actually, over the weekend we are working with one of the people who helps with SimC hunter code on differences between SimC and AMR. We work with all theorycrafters who are interested in collaboration Robots are very friendly.

[Khrux]

@Zoopercat

Does AMR have the new increases to Kill Command or Cobra shot? I can't find any multipliers added in either the live build or build 7.1.5.23360

If the wiki only updates based on live data, then it isn't possible to sim the buffs coming tomorrow until they are live, is that right?

[Zoopercat]

@Khrux yes those changes are live on the simulator - we put them in since the update was coming so soon.

But let's say we didn't have them updated, you can actually change them yourself and sim if you wanted. To do that, on the wiki page, you see the 2 drop downs on the upper left for the live/build number and spec? Right below that is a blue clone link. Clone the database. Then you can click on any spell, click the edit button at the top of the spell page, and edit it. We have a lot of people who do that to test things out, or say things like 'what if it was buffed more?' etc.

[Firebyrd]

@Zenimara Can you give me a specific example of what is unreliable on AMR? Either I can walk you through the proper setup, or find and fix a bug

I can't speak for other people, Zooper, but I know I wouldn't trust AMR farther than I can throw it because when you guys first started, pretty much everything you guys posted for hunters was wrong. Then representatives from your site would come here and argue with people like Zeherah and tell them that they were wrong. I'm sure I'm not the only one who remembers this, so you've probably got an uphill battle here.

[Deleted]

So.

Can AMR provide stat weights to import to pawn based on your custom sim if we run it after every new item?

[Zoopercat]

@Firebyrd, do you have specific examples of what isn't working right now? When we first launched, we did a lot of work with hunters on the survival rotation and a few others to improve it in SimC (like Swol's discussion here). Not sure what else you're referring to. Also, I spend a lot of time comparing our sim results to actual logs from WCL to make sure spells match up and the rotations are matching reality. They were spot on for hunters (pre 7.1.5, I still have to verify them for Nighthold when the logs come out).

@kiltedjock We don't have an automatic pawn export yet, so you'd want to enter them directly into the addon.

[Firebyrd]

@Firebyrd, do you have specific examples of what isn't working right now? When we first launched, we did a lot of work with hunters on the survival rotation and a few others to improve it in SimC (like Swol's discussion here). Not sure what else you're referring to. Also, I spend a lot of time comparing our sim results to actual logs from WCL to make sure spells match up and the rotations are matching reality. They were spot on for hunters (pre 7.1.5, I still have to verify them for Nighthold when the logs come out).

@kiltedjock We don't have an automatic pawn export yet, so you'd want to enter them directly into the addon.

I'm talking about when the site first started back in Cata or whenever. I don't use the site and will never use the site because of how wrong it was and how representatives from it acted when people who actually knew their shit tried to help. The site gained a reputation for unreliability here then and I was pointing out that at least some people are probably still working off that information as I am.

[Deleted]

Good god, you're still against the site for how it was three expansions ago? Er... I'm not saying it is reliable or not, but at least base yourself from something more recent than 3-4 years ago.

[Firebyrd]

Good god, you're still against the site for how it was three expansions ago? Er... I'm not saying it is reliable or not, but at least base yourself from something more recent than 3-4 years ago.

So should I re-evaluate Noxxic every expansion? Pretty sure that's still run by the same people and why would AMR be run by someone different?

ETA: My point really is both bringing up a potential reason posts like this keep popping up and that it was the attitude by site representatives that caused this. Wrong can be fixed, but it was their declarations that all the established hunter theorycrafters were wrong with nothing to back it up except their site that was malfunctioning in the first place that was an issue. If they've fixed the wrong, that's great, but they did gain a reputation.

[Zoopercat]

@Firebyrd I still don't know what you're referring to about this hunter theorycrafter thing. We even worked with zeherah (who you mentioned) and helped pay for femaledwarf servers to keep it running longer. I don't recall any bad blood with him. We've done that for several sites and projects, because the more theorycrafter and tools there are, the better off the whole community is.

There are surely cases where we 'argue,' which is intended to be an interesting discussion and to delve into details and assumptions. While those discussions have the best of intentions from all sides, tone of voice is lost and people might get mad at each other. That certainly happens on the internet. We don't intend to do that, and I'm sure the other side doesn't either.

Anyway, if anyone does find a specific example that makes the simulator not work right for hunters, let me know. Bugs appear during patch times, or stealth changes slip through the cracks. We'll be doing our verification stuff over the next few weeks too.

[Vitinariy]

@Zoopercat

Just out of curiosity, are you working in collaboration with Icy-Veins?
Because they do recommend your website, however, some sims directly contradict the recommended talents. For example, Veins say that Spitting Cobra is not worth it for Survival and Expert trapper should always be taken. However in all my recent sims, using both my character and your templates, Spitting Cobra sims notably higher then Expert Trapper.

[Zoopercat]

We work with some of the people who write guides for Icy Veins. However, guides (like our default stat weights), show just one way to play. Sometimes a talent is better because of your legendary, as a possible example. Other times, some guide writers 'merge' weights or talents - meaning instead of giving you the best setup for single target OR AoE, they merge the two to give you a pretty good setup that does well in both. I personally prefer gearing for each type specifically, since there is so much gear, it's not hard to tailor a set for each.

As for Azor's guide on IV, I know he runs several types of setups. Single target, a boss that always has 2 adds, etc. I am not sure if he merges those or looks through them to make an educated recommendation. I talked to him around EN's launch a bit and I know he digs through a lot of setups and tries to make a call on what is best to recommend to people. For example, a talent might be really good only in specific situations, and therefor he might recommend a different one that is good for situations that happen much more often.

Think about these guides (and writers) like this: if your friend comes to you and have never played wow, and asks what to play, what do you suggest? Probably a class that is a bit easier to master. Another friend comes to you who is a pretty hard core raider in SWTOR, so you probably recommend a class that's hard to play but has more potential damage. Etc, etc. So these guide writers are trying to figure out the best things to suggest to a large audience.

Anyway, there are a ton of ways to come up with recommendations in a guide. We like to provide tools to help people figure out how they want to play, and not tell them what they should be doing.

Also, I'm not a hunter expert, I'm a simulation / optimization expert. So I don't have any helpful advice on specific talents, I'll leave that to the pro's here, and our simulator