[Balance] BFA Boomkin Theorycrafting -- intro to chickendb

[Slippykins]

tl;dr:
Big theorycrafting post using new analytics tool chickendb.

Link to chickendb: http://www.chickendb.com
First theorycrafting post + how-to guide: https://www.dreamgrove.gg/balance/ch..._dg_firstpost/
--------------------------------------------------------------------------------------------

Hey all!

It has been a while since I’ve been able to write up an in-depth piece of theorycrafting, and I couldn’t be more excited to share this innovative update with you all.

Iskalla and I have teamed up to create a new way to display, disseminate and document theorycrafting for Boomkin with BFA, and have put in a tremendous amount of effort to bring this in time for the expansion. Essentially, we were unhappy with the traditional methods of theorycrafting, which typically followed the pattern of:

• sim all builds,
• compare top choices,
• make justifications based on the ordering.

This style of theorycrafting, while correct in most instances, lacked further data analysis and explorative insights. In general this approach works, and the inconsistencies with top builds in WarcraftLogs can be hand-waved as “simulation error” or “sims are junk.” We strongly refute this point, and argue that sims are only as good as the model and assumptions behind it. As such, a large part of this project has been to target these deficiencies, and relax some of the constricting assumptions we have historically clung to.

Chickendb is the first major step into data analytics and data aggregation, combined with a shiny front-end with Power BI visuals and interactivity. We’re putting a whole new spin on theorycrafting, and letting the user become the one in charge of what they see. You can explore the results of over 50 separate sims with 95+ profiles per sim, and go straight to the information you need. We’ve cleaned, cut-up and reported the actual insightful items hiding in each sim report, giving you the ability to really see what makes up your DPS and how impactful your decisions really are.

We hope you all enjoy this first introduction to chickendb. We are looking for feedback on the analytics, look-and-feel and general usability of the site, so please contact us if you have any suggestions, comments or queries.


Cheers
Slip & Iska

[Tauton]

We strongly refute this point, and argue that sims are only as good as the model and assumptions behind it.

Agreed a thousand times over. I'm a coder too that works in this field and people need to understand this part so hard!

I'd regularly update the priority system on simcraft and have a spreadsheet with different weights on it based on the small difference of each boss of that tier (my role, the movement, the burst windows etc), rather than just look at single target over 3 target (for instance). Hopefully your way of doing things will look at the realities of what we are facing and not just broad assumptions.

Then again, I enjoyed being high up on logs for simming gear correctly rather than using generic weights.... you may ruin that... am I cheering for the wrong team...? :P

Good luck, I will keep track of this

[huth]

Then again, I enjoyed being high up on logs for simming gear correctly rather than using generic weights.... you may ruin that... am I cheering for the wrong team...? :P

You can't screw in a nail with a spatula. People who can't use tools correctly won't suddenly be able to do so. They'll just find a new way to misinterpret the data.

If there's anything i don't like, it's the lack of ability to zoom in on the graphs, especially the tail ends. So consider that as a feature suggestion i suppose.

Other than that, looking great!

[Rhaxus]

amazing, will use it for sure, thank you for all the work!

[Slippykins]

Hopefully your way of doing things will look at the realities of what we are facing and not just broad assumptions.

Then again, I enjoyed being high up on logs for simming gear correctly rather than using generic weights.... you may ruin that... am I cheering for the wrong team...? :P

Good luck, I will keep track of this

That's the idea! Removing some of the restrictive assumptions from simming (ST, 5T cleave only) will hopefully bring some clarity to talent and gear decisions. There's scope to work in a movement module, and to show dynamic stat scaling over a few scenarios -- no more stat weights!

And yep, we may be eroding your advantage. D:

You can't screw in a nail with a spatula. People who can't use tools correctly won't suddenly be able to do so. They'll just find a new way to misinterpret the data.

If there's anything i don't like, it's the lack of ability to zoom in on the graphs, especially the tail ends. So consider that as a feature suggestion i suppose.

Other than that, looking great!

Completely agree -- I've pitched the site as something that can be used by anyone, but I think some of the visuals could be misinterpreted if the guide isn't read.

You can zoom in on the graphs with the focus mode in the top-right corner!

https://docs.google.com/document/d/1...h.hcuitfrqqah8

amazing, will use it for sure, thank you for all the work!

No problem! We're really excited to see what we can turn this into.

[huth]

You can zoom in on the graphs with the focus mode in the top-right corner!

Tried that, but it just makes the graph go full page, which can still be far from what i need to actually make out what i'm looking at, particularly in the fight length graph. Some of our stuff is really close together.

[Slippykins]

Tried that, but it just makes the graph go full page, which can still be far from what i need to actually make out what i'm looking at, particularly in the fight length graph. Some of our stuff is really close together.

Ah gotcha. Yep you can't zoom in further than focus mode unfortunately.

Though, with things that close, it's getting to the point where sim error and in-game variance are going to muddle the distinction. The fight length graph is really interesting though! I like how we can make build choices without having to actually sim the shorter length fights.

[huth]

There's some unusual builds on top right now, as well. Don't think many expected FoN up there.

[Slippykins]

There's some unusual builds on top right now, as well. Don't think many expected FoN up there.

Yep, agreed. And I think the Shooting Stars buffs are really highlighted with the results on chickendb right now.

Today's updates:

• implemented gearsets in the database
• created (hidden) page on report to see gear being used in the profiles
• recreated sim profiles with profilesets, rather than copies (reduced sim time by 50%)
• created mobile version of site

Next updates:

• creation of different gearsets
• creation of mobile version of report
• migrate site to VM

[Tauton]

Which version of SimC are you using for this? May you add that in and keep it updated so that we can see the base APL being used. I ask because it's being updated regularly and will help me (and others that really dig into it) what assumptions SimC are initially using before you guys use the data.

edit - I believe i've fully read through your guide/info now and, whilst it explains your reasoning behind your decisions, I still can't see the exacts of where/how you are getting your data. You mention here that the standard way of simming isn't good enough (agreed), yet you don't seem to explain what you have done to get the data to back your views. Otherwise your work is effectively giving your views/elaborations on 'data shown', but with no suggestions of where that data has come from which I think is imperative for the purpose of data analysis.

Is there any chance of adding that in somewhere at some point?

(Or am I overthinking it and it is just the below input with each type of talents, for each fight type, for each number of adds, stored in a table and extracted with SQL?)
" Max_time: 300s
Vary_combat_length: 10%
Default_skill: 1
Optimal_raid: True
Fight_style: Patchwerk
Level: 120
Gear ilevel: 370")

(I found it very hard to write down what I meant without it sounding like i'm being a dick, I really do think this is great piece of work you're doing - I just can't see it's clear how you are getting your data and once I am, I can relax and shut up )

[huth]

Yep, agreed. And I think the Shooting Stars buffs are really highlighted with the results on chickendb right now.

I think they went a little to far with it. Moon Moon needs some help, it's not gonna get the thing this way.

[Slippykins]

Which version of SimC are you using for this? May you add that in and keep it updated so that we can see the base APL being used. I ask because it's being updated regularly and will help me (and others that really dig into it) what assumptions SimC are initially using before you guys use the data.

edit - I believe i've fully read through your guide/info now and, whilst it explains your reasoning behind your decisions, I still can't see the exacts of where/how you are getting your data. You mention here that the standard way of simming isn't good enough (agreed), yet you don't seem to explain what you have done to get the data to back your views. Otherwise your work is effectively giving your views/elaborations on 'data shown', but with no suggestions of where that data has come from which I think is imperative for the purpose of data analysis.

Is there any chance of adding that in somewhere at some point?

(Or am I overthinking it and it is just the below input with each type of talents, for each fight type, for each number of adds, stored in a table and extracted with SQL?)
" Max_time: 300s
Vary_combat_length: 10%
Default_skill: 1
Optimal_raid: True
Fight_style: Patchwerk
Level: 120
Gear ilevel: 370")

(I found it very hard to write down what I meant without it sounding like i'm being a dick, I really do think this is great piece of work you're doing - I just can't see it's clear how you are getting your data and once I am, I can relax and shut up )

Great questions!

I should add to the main post where I've gathered the data. What we have done is simulate multiple gearsets over multiple raid events, with some constant assumptions chucked in (the quoted parameters above).

For clarity, we iterate over all the raid_event types and parameters we detailed in the guide, which includes singletarget, cleave, dotcleave and addwave modules. This results in 19 different raid_event scenarios with a single static gearset -- this will change in the future, and we anticipate having over 100 separate sims to run.

Each sim then has a base profile without talents, 12 profiles with only a single talent selected, and then 3^4=81 profiles, one for each combination of the four talents. In total, that results in 94 profiles per sim.

We then gather the .JSON report from the sim and parse that into our backend database. We perform a talent DPS decomposition in this process, where each talent's DPS is calculated as (profile_with_only_that_talent_DPS - base_profile_DPS), and each talent combination profile's talent synergy is calculated as (profile_DPS - talent1_DPS - talent2_DPS - talent3_DPS - talent4_DPS - base).

Finally, we query our database through Power BI to create the visuals seen on chickendb. You're essentially correct in saying "the below input with each type of talents, for each fight type, for each number of adds, stored in a table and extracted with SQL," but we will be relaxing the Gear ilevel assumption in the coming weeks. Fight_style is not an assumption technically, as it just represents an empty raid_events script (to which we add our own).

At the moment, our sim run takes 22 hours to complete, and with our in-progress additions including gearset changes, stat scaling and some efficiencies, we anticipate each sim run will take roughly 50 hours. We will be looking to migrate our sim process at some point.

And I completely agree with "Otherwise your work is effectively giving your views/elaborations on 'data shown', but with no suggestions of where that data has come from which I think is imperative for the purpose of data analysis." We will only ever report on data we've produced through simming, or through raw game data. We're in close contact with the people who maintain the balance simc module, and we're constantly seeking improvements in that area.

[Slippykins]

Updates for 27/07:

• created gearset profiles for normal/heroic/mythic uldir
• created azerite profiles
• began main sim run, completion expected in 5 days
• began azerite sim run, completion expected in 3 days

[Slippykins]

Big update today!

We've now finalised the layout for chickendb, as well as included the headline profiles to display on the Results page. This includes over 2,100 profiles with a huge combination of talents, azerite and trinkets!

We have also offloaded our simming requirements to Raidbots, courtesy of Seriallos. We currently require 741,000,000 iterations per sim run, but with Raidbots doing the computation, we can squish this from 10 days of simming down to a little over a day!

If anyone has any further improvements or suggestions, let me know.

[ChrisKoe]

Are the numbers for trinkets correct? They all show the same dps for me.
Also, will there be a stat comparison or will you leave it at gear pieces?

Anyway, great project!


e: not only trinkets, but all gear pieces have identical numbers

[alkyd]

Thank you, I have bookmarked it. Gonna use it in BfA.

[Slippykins]

Are the numbers for trinkets correct? They all show the same dps for me.
Also, will there be a stat comparison or will you leave it at gear pieces?

Anyway, great project!


e: not only trinkets, but all gear pieces have identical numbers

Yep I broke the gear page -- fixing it within the hour!

Thank you, I have bookmarked it. Gonna use it in BfA.

Welcome!

[huth]

Probably on your list already, but we need some better way to quickly distinguish what build we're looking at. The long lists of initialisms aren't really helping, particularly if the part that interests you is somewhere in the middle.

[Tauton]

It might just be me but the drop downs do not work in firefox, checked in chrome and everything looks... better and the drop downs work perfectly, things make a bit more sense now

[huth]

It might just be me but the drop downs do not work in firefox, checked in chrome and everything looks... better and the drop downs work perfectly, things make a bit more sense now

They work fine for me. Do you have any blocked scripts or sites maybe?