[Holy] News from the western front: Serenity crit is broken

[Danner]

http://us.battle.net/wow/en/forum/topic/3123313738

[zakaluka]

no, he's using a sample size of 50. To get any kind of consistency you need a sample size of at least 1000.

I did such a test quite some time ago, posted it in a thread about the efficiency of holy heals somewhere back 3-4 months ago. HW:Serenity does indeed yield 25% additional crit. However, there's some kind of odd behavior with how multiple sources of crit% apply. It seems to be related to the crit cap and how high your crit already is. If I had to guess, I'd say each additional source of crit ratios against the crit cap.

You have:
~13% crit from int+crit rating.
10% crit from chakra:serenity
5% crit from leader of the pack
25% crit from HW:S

If those just added, you'd have 53% crit.
However, they seem to multiply in some way against the cap, and really you wind up with something like 37% crit.

Don't believe everything someone posts just because they back up what they're saying with math. That goes for what I say, too. You have to think it through, see if the poster forgot to think of something.

[Ynna]

I didn't read the thread, only the opening posts, but 50 casts to determine this is a terrible sample size.

[Danner]

Scroll down to Kaels' post then. Much larger sample size.

Also, 95% confidence interval says 50 samples is enough.

[zakaluka]

he also defends his small sample size then reinforces his argument with another test sample size 150.

I'm not very fluent in statistics, but I remember what my Political Science professor told me about polling. To get confidence of +/- 3 points in polling, they need 3500 responses. Seems like he should do a test this large before trying to make any kind of claim. His 'population size/confidence interval' calculator seems geared to a different scenario and taken very out of context.

But, looking at Kael's post it's possible. I still think he needs a bigger sample.

[Wuga]

he also defends his small sample size then reinforces his argument with another test sample size 150.

I'm not very fluent in statistics, but I remember what my Political Science professor told me about polling. To get confidence of +/- 3 points in polling, they need 3500 responses. Seems like he should do a test this large before trying to make any kind of claim. His 'population size/confidence interval' calculator seems geared to a different scenario and taken very out of context.

The sample sizes in question are plenty large given the enormous disparity in actual results (~30%) versus predicted (50%+) and the confidence intervals he calculates are correct. It's just a standard binomial distribution. Your polling example is a different situation.

[Deleted]

As far as I can tell it is broken. I did a test of 200 Flash Heals with Serenity on the target and got a crit of 19.5%

My base crit is 10.87% (with things that proc taken off, like power torrent staff, trinkets etc).

Since I'd expect to be getting nearer 45% if it was working, even with such a small sample size it's pretty clear that something isn't right.