[Frost][5.2] 10 Minute Parse / T15 4PC Problems

[Akraen]

Update: A lot more data has been gathered, please scroll down to see - the original parse consisted of a tiny amount by comparison.

Big preface/disclaimer: I recognize the element of RNG in this game. When a set bonus is released that introduces further RNG, there will be scenarios to a variety of extremes.

I had heard from a few people who realid me for frost mage stuffs that the 4pc of Tier 15, (6% FoF increase change off Frostbolt) was underwhelming. One person suggested that it might just not even be active.

So, I decided to do a 10 minute test with the following parameters:
No buttons pressed other than frostbolt, ice lance, and invocation (can't go oom).
When a FoF proc is generated, ice lance is hit immediately so that all FoF procs are tracked.

These are the results.



My major concern here is the # of frostbolts cast versus the # of FoF generated.

With a statistical time frame of 10 minutes (600 sec), I calculate the following:
Haste gain from the ilvl boost of 522 tier is 10 more frostbolts: 3.01% more frostbolts
% of FoF procs in Tier 15: 11.34%
% of FoF procs in Tier 14 (half 517, half 496, all I have sorry): 13.77%

Difference in yield: -2.43% FoF procs using T15
Difference in yield considering potential set bonus: -8.43% FoF procs using T15

So, referring back to my disclaimer, and taking RNG into consideration, it still really appears this set bonus is not active. I encourage others to run tests as well. If I find the time after raid tonight I'll try to run even more 10 minute tests to prove that this is not a case of RNG.


Sidenote: 2pc of 517 tier will really last a while, that bonus is substantial!

[Deleted]

I have recognized on PTR tonight (scaled to ilvl 522 resp. 527), that I do significant more dps with T14 4 piece than with T15 4 piece. I hope that´s on Blizzards radar, otherwise a frost mage would keep 4p till the end of T15 (or it is not active?).

[zomgDPS]

I'll try to run even more 10 minute tests to prove that this is not a case of RNG.

This is going to be key, since right now you are dealing with % ranges that can definitely be within the realms of RNG.

These kinds of things can be tricky coz of that.

With that aside, did you notice this in the current (last?) build? Or is this ever since the change was made. I ask because I didn't really dig into Frost testing this last patch (it lasted around what.. 2 days?), but I do remember making a note that it was working back in previous builds.


Either way, it would be worth testing it again with the latest build (its going up as we speak). This would allow for us to isolate the issue.

[Akraen]

This is going to be key, since right now you are dealing with % ranges that can definitely be within the realms of RNG.

These kinds of things can be tricky coz of that.

With that aside, did you notice this in the current (last?) build? Or is this ever since the change was made. I ask because I didn't really dig into Frost testing this last patch (it lasted around what.. 2 days?), but I do remember making a note that it was working back in previous builds.


Either way, it would be worth testing it again with the latest build (its going up as we speak). This would allow for us to isolate the issue.

It's also possible the set bonus wasn't active until this build. However, I wanted to get numbers up there since 4pc is a separate testing issue than your thread.

Hopefully we can square this away before release, pro or con.

---------- Post added 2013-02-28 at 01:04 AM ----------

GC read this thread and replied (I wasn't the one who tweeted it, someone else):

https://twitter.com/Ghostcrawler/sta...43385846030337

So let's get some more data. I said I'm "concerned" - I'm not necessarily throwing out blatant accusations here. In 10 minutes I figured RNG would level out, but as stats go, you can flip a coin 1000 times and it could come up heads 1000 times.

So if anyone can get on the PTR and run these tests, we can be sure.

[Deleted]

For 334 frostbolt, it will result in 70.14 proc at 21% and 50.1 proc at 15%.
You can have test it easily without T14, as it's 4 piece bonus is only about damage.

But, we can test more. For reducing rng, we need a lot more FB cast. Can we have 10 or 20 people adding here there result for the same testing.
It can be done by a 1 hours testing (2000 frostbolt) or by aggregating 6+ result for 10min testing
Here is some expected result :
1000 Frostbolt - 150 proc (15%) / 210 proc (21%)
2000 Frostbolt - 300/420 proc
3000 Frostbolt - 450/630 proc

[Akraen]

Here is my latest test. I wanted to hit 2000 but I have to get on a morning conference call. I'll do even more later.

1500 frostbolts, 198 procs
15% = 225 (-27)
21% = 315 (-117)

I know this is still subject to RNG, even at 1500 casts (1 hr of Frostbolt spam), however, I want to add some perspective here. Last night we were working on Heroic progression in 25M. We had 19 attempts, and the total amount of Frostbolt casts in 19 attempts, was 957 (excluding the icy veins triple bolts). So when I do a 1500 frostbolt spam test and see these results, it is discouraging. It is my opinion that this is not a quality set bonus because you will never feel like you are getting any type of benefit.

Here is the SS for the test:


Is there anyone out there with a statistics degree that can weigh in on the number of instances that must take place for a 21% probability to be observed? I will frostbolt until Tuesday morning if that's what it takes :P

[Deleted]

I will try to make a 1h test before raid.

It's bizarre. It feel like the 4 pieces bonus don't work. Because 198/1500 = 13.2%. You're at -37% under the 21% proc rate!!
I'm ok with +/- 10% that would have make +/-31 proc for 1500 frostbolt.

Is there a bug that have done it multiplicative : 12% * 1.03 (5.2 buff) * 1.06 (T15) = 13.1%.

[tannzenator]

Quickly using Akraen's data:
Measured Probability: 0.132 (198 procs after 1500 frostbolts)
15% z-test: 1.95
21% z-test: 7.42

Values greater than about 3 demonstrate that we are not reaching the proc rate expected beyond reasonable RNG.

[Akraen]

Here we go, I did another test. 3000 frostbolts, which is about 3 full raids worth of frostbolts:

3000 Frostbolts, 418 Fingers of Frost = 13.933%

15% = 450 (-32 which is 7.1% under proc rate)
21% = 630 (-212 which is 33.65% under proc rate)



---------- Post added 2013-02-28 at 09:43 AM ----------

Quickly using Akraen's data:
Measured Probability: 0.132 (198 events in 1500 trials)
Standard Error of the mean: 0.02

A result of 0.21 is then within 4 standard errors (much less than 1% chance of occuring). This should be sufficient data to show that we have an actual issue here and not just RNG.

Thank you so much! I was operating a little bit on intuition and thinking "man this can't be right" - but this is good stuff. I hope all this info can get back to GC.

[tannzenator]

Quickly using Akraen's data:
Measured Probability: 0.139 (418 procs after 3000 frostbolts)
15% z-test: 1.64
21% z-test: 9.50

Values greater than about 3 demonstrate that we are not reaching the proc rate expected beyond reasonable RNG.

[Akraen]

I'm on conference calls all day I'll see if I can throw 50k frostbolts.

Should I unequip the T15 set bonus for this test?

[Methusula]

I would. You can isolate the issue better that way. if It is an issue with base proc generation or simply the set bonus. As it stands, we see an issue with the set equipped. Now it's a matter of seeing if the base rate is also jacked.

[Deleted]

I think you will have a good estimation with 1k or 2k Frostbolt.

Result can change a lot :
if it's still at 5.1 proc rate, it would give around 12%. Normally, it would give you 15%. But It can also result in a 7.5% proc rate (13.7% - 6% = 7.7%)
For the last, it could be seen very quickly.

edit: I hope for a blue answers on this bug, soon.

[Akraen]

Ok, here is a 2000 frostbolt sample without T15.

Note how much higher ice lance damage is even though T14 is lower ilvl, that 2pc is so strong! Anyway:

2000 Frostbolts, 264 Fingers of Frost = 13.2%

15% = 300; -36 or -12%

So if we compare 4pc T15 yield of 13.93% to not having the tier, we have a yield of 13.2% - calling on some statistician help again to look at that. I appreciate the help you guys are giving, I just have an MIS degree and I only had two stat courses in college :P

Here's the SS, don't mind the shaman that is on my recount, he asked for a portal and he melee'd the totem when I invited him to group.

[Deleted]

Just did a test on live (5.1) with frostbolt and ice lance. The funny party was, everytime i reached full hundred, the procrate matched perfectly (100/12 200/24 and so forth)



Going to test on PTR with and without T15 4p (hopefully before PTR tests start tonight)

*Edit* Server restart : /

[Deleted]

just before server restart :
3pT15 : 400/64 (16%) but I have a string a good proc (was at 300/56 - 18.56%) followed by a lot of no-proc (100/8 - 8%).

It's only show how much of impact have RNG !!!

[tannzenator]

Quickly using Akraen's data:
Measured Probability: 0.132 (264 procs after 2000 frostbolts)
15% z-test: 2.25
21% z-test: 8.56

Values greater than about 3 demonstrate that we are not reaching the proc rate expected beyond reasonable RNG.

[Deleted]

How, I think that the under proc rate problem is part of why T14 is so good. But I can easily see that 2T41/2T15 > 4T15
especially if it's loves/shoulders that are T14 upgraded

did they restart because of a hotfix for the proc rate ?

[tannzenator]

did they restart because of a hotfix for the proc rate ?

Only way to tell would be to test but I'd be surprised if they fixed it already.

[Deleted]

Second test and second time if got even numbers. I did a test on PTR with T15 4-piece and my procrate was exactly 16%




I´m curious what frost is capable of with a working T15 4p!


*Edit* Test was done right after restart