[Frost][5.2] 10 Minute Parse / T15 4PC Problems

[tannzenator]

Second test and second time if got even numbers. I did a test on PTR with T15 4-piece and my procrate was exactly 16%


I´m curious what frost is capable of with a working T15 4p!


*Edit* Test was done right after restart

citizenpete's latest data:
Measured Probability: 0.16 (80 procs after 500 frostbolts)
15% z-test: 0.63
21% z-test: 2.74

Values greater than about 3 demonstrate that we are not reaching the proc rate expected beyond reasonable RNG.

[Akraen]

I'm being told that these statistics are useless by Lhivera, who is in direct contact with the devs. Part of me feels like this is just a way to see if I will spend another 6 hours casting frostbolts, I feel slightly jerked with but ho-hum we move on.

Anyway recount might not be good enough for the devs. So if everyone who is willing to help settle this once and for all could please type /combatlog in-game before they spam frostbolts, then email them to me: akraen@live.com - I will pass them along to the powers that be. If you need help with the combat logs, be sure to let me know and I'll help. Also, I am on Anasterian (US) and named Akraen, if anyone wants to come show that target dummy what true cold feels like

We need as much data as possible, in the neighborhood of 50k frostbolts. I really, really wish a GM could pop next to me on the PTR and give me a super haste buff or something to help make this go faster.

[Deleted]

They can easily test it on ptr with better tool than us.
They prefer combat log so they can see proc, not IL count, I think.

[Akraen]

Here is a combat log with 3250 frostbolts: http://sdrv.ms/YLmqol (that's on my Microsoft Skydrive, the link was shortened so I could fit it on the tweet conversation I had going).

Here's the recount to go along with it:

I wore 4pc T15 for this iteration.
3200 frostbolts, 507 fingers of frost = 15.84% (this is better, did the earlier restart bring it back up to 15% +/- deviation?)

15% = 480 (+27)
21% = 672 (-165)

[zomgDPS]

I'm being told that these statistics are useless by Lhivera, who is in direct contact with the devs.

Lolwat?
These statistics could use more data points sure, but calling the exercise "useless" is pretty 'effing retarded.

I knew the whole MvP thing went to his head, but I think its getting past the point where it was cute. What I don't get is where in the description of "MVP" was the role of 'personalized community censorship' was outlined.
I don't know why you have to first get the matter 'past' Lhivera before the devs see it in the first place. Meaning he gets to hand pick what is an issue with the class and what isn't? Yea.. thats not a good idea imho.

I think a pretty clear message needs to be sent that the mvp does not speak for the mage community. Otherwise you guys will constantly get left up sh!t creak without a paddle, just like you were in beta (where the same thing happened).


On topic though, logging it is probably a better way to go anyway. It will be easy to run the logs through a parser to get some good numbers at the end. Plus, you can go around doing your regular PTR fun-ness with frost as long as you are logging, so you won't really need to sit at a dummy and keep spamming frostbolts.

A few hours of play's worth of logging and you should have a pretty significant data set. Merge those with some of the ones others send you and then the analysis should be easy.

[Akraen]

I haven't been able to get any type of parses to upload to WoL, so that's why I was relying on recount. If Blizzard devs have a way to parse the combat logs, I'll do just that and continue to amass them

Do you know of any method to trick WoL into accepting the PTR parses? It'd make this so much less painful!!

[zomgDPS]

I haven't been able to get any type of parses to upload to WoL, so that's why I was relying on recount. If Blizzard devs have a way to parse the combat logs, I'll do just that and continue to amass them

Do you know of any method to trick WoL into accepting the PTR parses? It'd make this so much less painful!!

When you /combatlog, it spits out a regular txt file. Just pass along that file (or keep building upon it with others' data).

Either way, we could probably build you a fast, specific parser relatively quickly. If I had time, I could probably do it myself by the weekend.


For now, just get the data. Be sure to log times where you were with the set bonus separately from the times you were not. If at the end of the day we get 2 massive txt files, one with set bonus data and one without, we can then start having a good time

For funzies, PM me your first logfile when you have it. Perhaps just some dummy testing. We'll take it from there.

Edit:
Nevermind.. it seems there is a standalone parser already out there so no need to build one (tbh, I would have been surprised if there wasn't one... this game is 10 years old after all).


Link to parser.

Try using that^ and let us know how it goes.

[Akraen]

Thanks Zomg! The link to my first parse (3200 frostbolts, 4pc T15) is in my latest post, but I'll relink it here: http://sdrv.ms/YLmqol (just my public skydrive)

I'll work at getting more logs. I'll have a fair bit of time tonight and tomorrow (we don't raid Thur/Fri). Stay tuned.



Edit: That parser doesn't work. Was updated in 2011. Googled for an update, nothing found. Going to need to find a new parser. I won't worry about that right now though, I'm going to continue to gather data on the PTR. I'll rename my logs to be more detailed "3200 frostbolts 4pc T15" for example-- I think consistency is key.

[Wuga]

Something seems off with the standard errors reported earlier in this thread (they're all too large)... but anyway, if your goal is to test whether the 4p is working or not, the last parse with 3200 casts is plenty. There's not really any need for more data when the discrepancy between observed and "expected" is so large. If the real proc rate is 21%, the odds of getting 507 (or fewer) procs on 3200 casts is laughably small... something like 1 in 50 trillion (and I don't think you're that unlucky).

What more data would be useful for is determining if perhaps it's not adding 6 percentage points, but just 6% to the proc rate (i.e., 15%*1.06 = 15.9%), which is more consistent with these results. You don't really have enough data at this time to reliably distinguish the observed rate (15.8%) from 15%.

[Akraen]

Something seems off with the standard errors reported earlier in this thread (they're all too large)... but anyway, if your goal is to test whether the 4p is working or not, the last parse with 3200 casts is plenty. There's not really any need for more data when the discrepancy between observed and "expected" is so large. If the real proc rate is 21%, the odds of getting 507 (or fewer) procs on 3200 casts is laughably small... something like 1 in 50 trillion (and I don't think you're that unlucky).

What more data would be useful for is determining if perhaps it's not adding 6 percentage points, but just 6% to the proc rate (i.e., 15%*1.06 = 15.9%), which is more consistent with these results. You don't really have enough data at this time to reliably distinguish the observed rate (15.8%) from 15%.

If this is true, they may as well not even type a 4pc bonus on the item for frost.

[Methusula]

Lolwat?
These statistics could use more data points sure, but calling the exercise "useless" is pretty 'effing retarded.

I knew the whole MvP thing went to his head, but I think its getting past the point where it was cute. What I don't get is where in the description of "MVP" was the role of 'personalized community censorship' was outlined.
I don't know why you have to first get the matter 'past' Lhivera before the devs see it in the first place. Meaning he gets to hand pick what is an issue with the class and what isn't? Yea.. thats not a good idea imho.

I think a pretty clear message needs to be sent that the mvp does not speak for the mage community. Otherwise you guys will constantly get left up sh!t creak without a paddle, just like you were in beta (where the same thing happened).


On topic though, logging it is probably a better way to go anyway. It will be easy to run the logs through a parser to get some good numbers at the end. Plus, you can go around doing your regular PTR fun-ness with frost as long as you are logging, so you won't really need to sit at a dummy and keep spamming frostbolts.

A few hours of play's worth of logging and you should have a pretty significant data set. Merge those with some of the ones others send you and then the analysis should be easy.

The only defense I could see for Lhiv here is that you can skew these results if for whatever reason, you wanted to. Because these are screenshots of recount data, the possibility exists that people could be procc'ing far more FoF than they are casting to make it look like we are being starved. I honestly don't think this is a very valid argument since it can be very easily determined by blizz if this was the case. This is the only thing that comes to mind that could potentially invalidate this type of data.

[Akraen]

The only defense I could see for Lhiv here is that you can skew these results if for whatever reason, you wanted to. Because these are screenshots of recount data, the possibility exists that people could be procc'ing far more FoF than they are casting to make it look like we are being starved. I honestly don't think this is a very valid argument since it can be very easily determined by blizz if this was the case. This is the only thing that comes to mind that could potentially invalidate this type of data.

That's an incredibly valid point and I do see the reason for requesting a combat log. However, for me to test 10k-50k frostbolts takes a long time and I'm confident in the assumption that the devs have much more efficient ways of testing large iterations than I do (I calculated it would take me 33 hours of non-stop casting to get 50000 frostbolt casts).

There are two reasons I began with recount. The first was simply to prove that there is a reason to investigate. That's all-- preliminary data that brings the concern to the table in a way better than most players do. The second reason was I couldn't get my combat logs to work on World of Logs and I figured they'd be very hard to sift though.

So I figured it was a start - and it was! We're now down to two chief scenarios:

A) The bonus is multiplicative and we should be seeing 15.8% procs up from 15%, which is a 0% dps increase. RNG will never allow for this to be even a visible gain of any sort.
B) This is an error and it should be 21%, but it's not. Therefore should be corrected.

I doubt it is point A because of: https://twitter.com/Ghostcrawler/sta...43385846030337 (he mentions 21%)

Combat logs now exist, mine and also I think Leviatron is uploading one as we speak. These confirm the information on recount, which should alleviate concern of nefarious intent. So I hope the devs will weigh in on the issue to settle it. I'm not out for blood. However, I will be wearing T14 in T15 raids if this doesn't change, not because I want to - I'm excited for the new tier, it's beautiful and I love upgrading-- but I'll be wearing it because I have to, it is higher DPS.

[tannzenator]

Something seems off with the standard errors reported earlier in this thread (they're all too large)... but anyway, if your goal is to test whether the 4p is working or not, the last parse with 3200 casts is plenty. There's not really any need for more data when the discrepancy between observed and "expected" is so large. If the real proc rate is 21%, the odds of getting 507 (or fewer) procs on 3200 casts is laughably small... something like 1 in 50 trillion (and I don't think you're that unlucky).

What more data would be useful for is determining if perhaps it's not adding 6 percentage points, but just 6% to the proc rate (i.e., 15%*1.06 = 15.9%), which is more consistent with these results. You don't really have enough data at this time to reliably distinguish the observed rate (15.8%) from 15%.

I was typing up an explanation of my method when I realized, yeah, my earlier calculations were incorrect...really silly mistakes I made and I apologize for anyone I mislead and would like to thank Wuja for pointing this out.

In general we have two possibilities per frostbolt, either a fingers of frost proc or no proc. This would scream Binomial Distribution (with the mean as the expected number of procs). We are then able to look at the standard deviation (using N as the number of trials and p as the expected probability):

Std = Sqrt[N p (1 - p)]

We then compare the expected number of procs to the number of procs measured and divide by the standard deviation to find the number of standard deviations away from the expected mean our trial is. The larger this number is, the more unlikely the event was to have actually occurred if p was our true probability (particle physicists look for this value to be over 5 before confirming results, we don't need to be quite that high to demonstrate our point since blizz will perform their own calculations). As an example, if the value is 3 then it is farther away from the mean than 99.7% of the data we would expect to be meaning that we may be fairly confident that our trial does not fit the the distribution with the expected mean (if the value is 5, then the percentage grows to 99.99994%).

For Akraen's data (507 procs in 3250 frostbolts) I will compare the result to 15%, 15.9% (predicted by Wuga above) and 21% and list the number of standard deviations away the measured result is:
21%: z = 7.56
15.9%: z = 0.47
15%: z = 0.96

The only significant result here is that we are very confident that the proc rate does not match what 4p should be doing for us. I will go through my previous posts and edit with the proper analysis as well.

[Akraen]

Here is an additional log Leviatron asked me to upload for him:
4pc T15, 3200 frost bolts - to match mine
https://skydrive.live.com/redir?resi...930040FA7C!656

[pyrostorm9001]

I'm being told that these statistics are useless by Lhivera, who is in direct contact with the devs. Part of me feels like this is just a way to see if I will spend another 6 hours casting frostbolts, I feel slightly jerked with but ho-hum we move on.

Anyway recount might not be good enough for the devs. So if everyone who is willing to help settle this once and for all could please type /combatlog in-game before they spam frostbolts, then email them to me: akraen@live.com - I will pass them along to the powers that be. If you need help with the combat logs, be sure to let me know and I'll help. Also, I am on Anasterian (US) and named Akraen, if anyone wants to come show that target dummy what true cold

feels like

We need as much data as possible, in the neighborhood of 50k frostbolts. I really, really wish a GM could pop next to me on the PTR and give me a super haste buff or something to help make this go faster.

Happy to help with combat logs, ill have enough free time this weekend to do some work for this.

[Deleted]

On Lhiv, he have report the problem to dev in a way they are more open to hear. Just for that he need a thanks.
On Recount versus CombatLog, it's just a better way to check proc from Frostbolt than comparison between the umber of cast of FB and IL.

For this 2 things, there's nothing more. And Yes, Blizzard have a parser for CombatLog. I think it's one of there primary tool for Debugging. I also think that they can test really quickly the real FoF proc rate.

One of the important thing here, is that the community can found bug and give accurate data to prove it.


Now, I have some question : is the PTR frost damage tuned around the expected T15 proc rate or the implemented proc rate ?
In the second case, we can expect some "nerf" to compensate.


Finally, on yesterday restart, I too think they correct one problem with FoF base proc rate. It clearly show by Akraen number (13.7% before/15.8% after). I don't think that it's multiplicative (15%*1.06) because it was never the case before. But I think it can be part of the issue.

[Deleted]

We can´t nerfed any further! They have to seriously fix that issue

[Deleted]

Here some further calculation on 2T14/4T15 comparison.
reference number :

• base proc rate : 15%
• T15 proc rate : 21%
• T14 bonus : +12%
• IL base damage : 100k/150k/200k
• IL T14 damage : 112k/168k/224k

For a combat with 1000 Frostbolt :

• number of T14 IL : 150
• damage of T14 IL : 16,800k / 25,200k / 33,600k
• number of T15 IL : 210
• damage of T15 IL : 21,000k / 31,500k / 42,000k
• damage of T15 (bug): 15,000k / 22,500k / 30,000k

Percentage of difference between T14 and T15 (T15/T14):

• T15 / T14 = +25%
• bug / T14 = -10.7%
• T15/bug = +40%

So the correction of the bug will result in a 40% gain of IL's dps (18.2% of total dps), +7.28% of the total.

[zomgDPS]

The only defense I could see for Lhiv here is that you can skew these results if for whatever reason, you wanted to.

I don't buy it.

If his only defense is that to attack the integrity of the tester, then that is no defense at all. I don't like the fact that as a tester you are assumed guilty till proven innocent. That is no way to behave.


It would be pointless for a tester to make a bruhaha about something that he is manipulating. Eventually, the truth will come out because the numbers don't lie. Someone doesn't put in hours upon hours casting frostbolts into a dummy just so he can troll everyone for a few mins. I'm 100% positive Akraen isn't doing that.

But fuck Lhivera, he is a pointless entity for the class now. A relic and a forgotten hero. He should make his exit now, lest he turn into the villain of the tale.




On to more constructive matters.

@Akraen:
I had some time last night when I was cooking dinner to code up a quick parser for your logs (side point: coding and cooking is fun!). It is in no shape to release publicly (though I will do so soon enough), however, it is in a stable place so that we can start getting "answers"


For the first logfile you sent me, here are the results:

# of Frostbolts cast: 3200
# of FoF procs: 438

For sheer speed of coding, I went with perl, but I can probably re-write it in any major language you wish, if you want the code. I will release the code soon just so people know I'm not trolling, I just need to clean it up a bit :P I might do it all in php/python and put it online, just so people can use it.


I thought I'd at least let you know that you don't have to worry about parsing. Just get me the logs I'll take it from there

The only thing you CAN do when you make them is to ensure the filename you give the logfile doesn't have spaces. This isn't a big thing, but it will just make my life a little easier


EDIT:
I said fuck it and decided just to dump the perl code here, feel free to troll through it if you wish. For now, only the parsefile is paremeterized, but we can parameterize pretty much anything we want later.

#!/usr/bin/perl

if ($#ARGV != 0 ) {
	print "Gotta type the name of the parsefile broseph...\n";
	print "$#ARGV";
	exit;
}
$filename=$ARGV[0];
$numFrostbolts = 0;
$numFoFprocs = 0;


$event1 = "SPELL_CAST_SUCCESS";
$event2 = "SPELL_AURA_APPLIED";

$spell1 = "\"Frostbolt\"";
$spell2 = "\"Fingers of Frost\"";

$actor = "\"Akraen\"";
$testActor = "\"Spiderr-Brill(EU)\"";
$lineCount = 0;


open (FILE, $filename) or die $!;

 while (my $lines = <FILE>) {
	
	$lineCount++;
	$line = $lines;

	@first = split(',', $line);
	@second = split(' ', $first[0]);
	
	#check event first
	if( $second[2] ne $event1 && $second[2] ne $event2 ) {
		next;
	}
	
	#next check the actor (logs pick up all sorts of randomness)
	if( $first[2] ne $actor ) {
		next;
	}
	
	if($second[2] eq $event1) {
		if( $first[10] eq $spell1 ) { 
			$numFrostbolts++; 
		}
	}
	
	if($second[2] eq $event2) {
		if( $first[10] eq $spell2 ) { 
			$numFoFprocs++; 
		}
	}
	
 }
 close (FILE);

 print "---------\n";
 print "Output\n";
 print "Linecount: $lineCount\n";
 print "# of Frostbolts cast: $numFrostbolts\n";
 print "# of FoF procs:	$numFoFprocs\n";

 exit;

If you got perl on your machine (if it is a linux box or mac machine, it should have it by default), just copy paste that code into a regular txt file. Name it and give it an .pl extension. Put it in the same directory as your logfiles and run it with the logfile you want to parse as the first parameter on the command line.

It should spit out some results for you.

[Akraen]

Cool Zomg, if there's anything you'd like to email me directly, feel free to send stuff to akraen@live.com - also let me know if there's anything else you'd like me to test.

This is my public WoW skydrive: https://skydrive.live.com/?cid=4af3b2930040fa7c
- I'll always reference this location for logs that I upload.

There is a new combatlog from my frost buddy, Leviatron. He also did 3200 frostbolts with 4pc T15, his file name is labeled with his name. Please let me know if I have any errors in my sharing settings.

-

Question though, any idea why my recount # of ice lances is higher than the counted FoF procs in the combat log?

And on a side note, I really appreciate you backing me. It takes a while to do these tests, and as a contractor time is money for me-- I wouldn't trash my name by making anything up