Raider.IO Development Updates - Raids and Mythic Plus

[Dracodraco]

Functionality of the site is amazing. I can see and feel the progress as the weeks go by, but one thing has been bothering me since I found out about this a while ago ...

Visually the website is very pedestrian.
Any visual upgrade forthcoming? Something a bit more modern and sleek?

Considering the comparisons are guildox and wowprogress, this IS modern and sleek. I honestly don't see any issues with the way it looks, and even like that guilds have icons of the boss kills they have, makes it easier to spot at a glance.

As for OP; Can we check clusters like Wowprogress does? If I look at just my realms' guild, I only see 3 guilds, but we're connected to 10 other realms...

[Ulsoga]

Functionality of the site is amazing. I can see and feel the progress as the weeks go by, but one thing has been bothering me since I found out about this a while ago ...

Visually the website is very pedestrian.
Any visual upgrade forthcoming? Something a bit more modern and sleek?

I agree that our site isn't exactly on the cutting edge of modern web design, and doesn't really subscribe to any of the trends you see in cool sites these days - but I think our first and foremost priority is functionality and ease of use. I'm an artist so I totally understand the desire for visual aesthetics, but for now it's more about building up the core components to be useful and fast. Maybe down the road we can be more modern and sleek

- - - Updated - - -

Considering the comparisons are guildox and wowprogress, this IS modern and sleek. I honestly don't see any issues with the way it looks, and even like that guilds have icons of the boss kills they have, makes it easier to spot at a glance.

As for OP; Can we check clusters like Wowprogress does? If I look at just my realms' guild, I only see 3 guilds, but we're connected to 10 other realms...

We do actually have rankings by connected realm, and we should probably make them a little easier to access. You can see a realm list here: http://www.raider.io/the-nighthold/realms/us/mythic but you can also type your realm into the search field, and it should return both the realm result as well as the connected realm result.

[Deleted]

My guild has a few kills that are wrong, for example we killed Gul'dan Heroic on the 23/1 and it is down at 15/2, according to your timeline we killed krosus mythic before guldan
Also the raid setups/members universally have 1-2 mistakes, but that isn't a major issue IMO

[Reloe]

My guild has a few kills that are wrong, for example we killed Gul'dan Heroic on the 23/1 and it is down at 15/2, according to your timeline we killed krosus mythic before guldan
Also the raid setups/members universally have 1-2 mistakes, but that isn't a major issue IMO

heroic kills aren't logged through achievement dates, but rather your armory feed. So the site probably just takes the first kill registered in the feed when it started tracking things, and I think they didn't start immediatly when nighthold was released?

raid setups or rather the specs can also only be guessed from the armory, but they're planning to let you edit them yourself in the future maybe.

[Ulsoga]

heroic kills aren't logged through achievement dates, but rather your armory feed. So the site probably just takes the first kill registered in the feed when it started tracking things, and I think they didn't start immediatly when nighthold was released?

raid setups or rather the specs can also only be guessed from the armory, but they're planning to let you edit them yourself in the future maybe.

This is correct. Although we did start scanning at the beginning of Nighthold, it took us a while to populate our database with guilds. We are also still working on improving our crawler speed to ensure that guilds are more likely to be consistently scanned at least once a week. Right now it helps a lot to have players come to the site and queue up their guilds after a kill, but we don't want to have to rely on it.

[Ulsoga]

Weekly update time! We have lots of small changes on the user side, but some more significant stuff happening on the backend. There is a lot of priority going into updating our crawlers before ToS is released so that we can have more accurate heroic and normal mode rankings at the next tier.


May 5, 2017

• Added various Mythic+ links to sidebar on pages with Mythic+ data
• Added full resolution clear time to Mythic+ pages (showing milliseconds). This is on-hover on the rankings pages, and just listed out on an individual run's page.
• By default realms that are part of a connected realm will link to the connected realm when clicked. The only exception here is on the Realms listing page where you can individually click a realm to see it's page directly.
• Updated titles for all pages to represent what you are actually seeing on the page
• Trailing slashes will be automatically removed from the end of character/guild URLs
• Backend crawler speed improvements
• Added 404 "Page Not Found" Page

That's it for now - more to come soon.

[Ulsoga]

Hey all! We feel like we have a pretty good update for the Mythic+ Keystone crowd this week, and possibly an even better one next week (spoiler alert: m+ character rankings are on the way.) For now here is a list of the new changes we have pushed live to raider.io:

May 16, 2017

- Created a M+ leaderboard that merges all of the instances. This still needs a small visual update to include a column for the dungeon names, but the information is there for users to browse.
- Added the option to view the Mythic+ leaderboards in "strict" mode, which completely eliminates all runs that did not make the timer from the list.
- Added the ability to filter runs by affixes. Previously if you viewed a M+ dungeon's leaderboard you'd see all of the runs compared against eachother. You can still view it this way (it is the default setting) but if you want to drill down deeper you have the option to do so. The filters can be applied to both the default and strict leaderboards.



- We have updated tool tips in the "Run Time" column to show the amount of time a run went over or under the timer.
- You can queue up a server's m+ leaderboard for a scan on our queuing page. This is particularly useful for players that complete a run at the end of the week, close to the time that Blizzard resets their leaderboard. It's the best way to guarantee we get your run in before it's inaccessible.



- We have added better (less obtrusive) social media icons to the navigation bar.
- The FAQ has been updated with a little more information
- Fixed an issue where we weren't tracking M+ runs done by groups consisting of players entirely on Azjol-Nerub (sorry guys!)
- Fixed issue where characters with blank names sometimes showing up in M+ runs.
- Fixed issue with the dropdown navigation bars highlighting and staying highlighted in odd ways
- Fixed a bug with modal errors using white text on a white background, making the error message invisible
- Backend crawler performance optimizations


I think that is all of the updates for this week, but if I missed something I'll sneak back in here and update this post.

'Til next week!
Ulsoga
raider.io

[Emancptr]

-Created a M+ leaderboard that merges all of the instances. This still needs a small visual update to include a column for the dungeon names, but the information is there for users to browse.

Definitely needs Dungeon name. Would make it much easier to visually see what dungeon the higher keys are being done on.

- Added the option to view the Mythic+ leaderboards in "strict" mode, which completely eliminates all runs that did not make the timer from the list.
- Added the ability to filter runs by affixes. Previously if you viewed a M+ dungeon's leaderboard you'd see all of the runs compared against eachother. You can still view it this way (it is the default setting) but if you want to drill down deeper you have the option to do so. The filters can be applied to both the default and strict leaderboards.

Really good ideas and implementation.

- We have updated tool tips in the "Run Time" column to show the amount of time a run went over or under the timer.

Good idea, but poor implementation. When you mouseover the run time there is too much information being conveyed. Suggestion would be to eliminate the run timer in the mouseover bubble (it's what you're mousing over already, and only slightly more detailed).

[Aspir]

Definitely needs Dungeon name. Would make it much easier to visually see what dungeon the higher keys are being done on.

Definitely agreed. We plan to add that, but we still need to work things out from a space perspective (the row is cramped). We played with using just an abbreviation for the dungeons and it was still cramped, but we'll come up with a solution We did add a tooltip on the M+ level that includes the instance name, but this was just a temporary holdover until we figure out how to get the name onto the row.

Good idea, but poor implementation. When you mouseover the run time there is too much information being conveyed. Suggestion would be to eliminate the run timer in the mouseover bubble (it's what you're mousing over already, and only slightly more detailed).

Good catch there. We'll be removing the timer w/ milliseconds resolution from the tooltip, because now that we are showing over/under timing, you can effectively derive the MS by looking at that value.

Thanks for the feedback!

[Ulsoga]

We've been pretty active in this thread, keeping readers apprised of the Mythic Plus changes to Raider.IO and we've added a few more things this week that we are excited to share. Since our last update we also had a number of small tweaks and bug fixes, but we really want this post to highlight the new features and explain the functionality of the M+ portion of our site.

**Mythic Plus Leaderboards**

We have added a number of M+ leaderboards to the site to help players gauge their level of progression within this content. It's important to note that Blizzard does not provide an API for M+ and so all of our data is acquired by scraping Blizzard's weekly Top 100 realm leaderboards. These leaderboards can change regularly, and they are reset every week. This means once the data is gone we no longer have access to it. If you run a M+ run late in the week and worry about it not being scanned before the leaderboard reset, we recommend checking to see that your run displays on Blizzard's site, and then queuing the leaderboard for a forced scan on our site here: https://raider.io/add

So what kind of leaderboards do we have and what do they mean?

**Timed Runs Leaderboard**

This is Raider.IO's default leaderboard, using an algorithm we've created (in conjunction with a good friend and math nerd - thanks Devilnaht!) to rank only runs that have met the timer requirements of the dungeons. Our scoring calculation provides an exponential increase in points as you push higher and higher keys (above M+10), and penalizes runs that go over a dungeon's par time. Our scoring system, was designed with two primary motivations in mind:

- We wanted to reward players for successfully running higher and higher keys, and we wanted the amount of this reward to increase exponentially — just like the difficulty of the dungeons themselves.

- We wanted to deter "score farming" abuse that we've seen elsewhere, where high level keys were repeatedly run while depleted, far over the timer, just to get additional ranking points.

We want players to push keys they can actually complete, and to strive for the best time they can, so we designed our system rewards for this. Here is a screenshot showing a sample of the Timed Leaderboard across all dungeons and affixes, worldwide. I've also included a sample on hover tool tip that displays some more in depth information about each run.



**All Runs Leaderboard**

This ranks runs exactly as Blizzard does on their standard leaderboards. This means, that the runs are ranked first by the Mythic+ level of the dungeon, then by how long it look for the run to be completed. This leaderboard allows you to see all runs, even if they have been depleted. Sample screenshot of this board, this time with a hover that displays a run that went beyond the dungeons timer.



**Characters Leaderboard**

This leaderboard ranks individual characters against one another, by tallying up all of their highest scoring runs from each dungeon. In this screenshot you can see the current Top 5 M+ players in the world (big shout out to Meowchan by the way, who has provided a lot of useful feedback via our Discord) channel.) This view also presents you with a list of all of the dungeons that are currently contributing to their score, shown on the right.



A character's Mythic+ score is calculated by summing their highest scoring run from each dungeon. Characters earn a score for each of their roles, and a score across all of their roles combined.

You can see an example of this in the image below:



This character, Combatcow had individual role scores of 319.60 for Tank, and 935.54 for DPS. However, their combined Mythic+ score is 1061.32. This helps illustrate that the "All Roles" score is not just a sum of the scores you have for each role. Instead, what we do is take the highest scoring that you've done for each dungeon, regardless of your role and combine those to make your score.

**Leaderboard Filters**

Now that we have the basics of each leaderboard covered, we can drill down into how a player can adjust the data their viewing with a numbers of filters. Selecting the dungeon dropdown with either the All Runs or Timed Runs leaderboards can let you view a combination of all dungeons (this is the default) or allows the user to focus on one dungeon. You can get even more specific by choosing a certain set of affixes from the affix filter, and of course you can sort by region as well.

The Character board has it's only unique set of filters, that mainly allow someone to compare classes and roles against one another. Curious to see how you measure up against the other Resto Druids in the world? You can choose that option from the menu at see what the top healer druids are up to in M+.



**Character Page Filters**

If you head over to an individual character's page (in this case we'll use Cohumulone - another shoutout for great feedback!) you can see we provide a list of all of their top runs by dungeon. In the future we plan on adding a way to look through ALL runs, but for now the table is limited to the "best". For clarification, "best" means either "highest scoring" when on the Timed Runs tab, or highest M+ level/time if "All Runs" is selected. You can use the various tabs on the table to specify which information you'd like to see. In this case we're looking at his highest scoring Tank runs (by clicking the Tank tab) and including depleted runs (by clicking the All Runs tab).



**Character Page Rankings**

On every character page we've put a brief summary of all of their ranks at the top of the Mythic Plus section. This includes their overall rank (across all roles) within the world, region, and realm as well as class specific ranks within the world, region, and realm.



I think maybe that's it! We look forward to hearing what you guys think and thanks for sticking with us while we continue to make Raider.IO something to be proud of

Ulsoga
Raider.IO

[Emancptr]

This is Raider.IO's default leaderboard, using an algorithm we've created (in conjunction with a good friend and math nerd - thanks Devilnaht!) to rank only runs that have met the timer requirements of the dungeons. Our scoring calculation provides an exponential increase in points as you push higher and higher keys (above M+10), and penalizes runs that go over a dungeon's par time.

This is awesome! I know you probably don't want to share 100% of the formula behind this, but can you give us some ballpark ideas of what run would outscore another run. It sounds like the system gives you more leeway (over timer) the higher ilvl your key is.

Basically, what i'm looking for is an approximation like this (if you're willing to share):

17 Vault in time ~= 18 Vault 10 minutes over timer.
22 Vault in time ~= 23 Vault 20 minutes over timer.

- We wanted to deter "score farming" abuse that we've seen elsewhere, where high level keys were repeatedly run while depleted, far over the timer, just to get additional ranking points.

Are you doing anything to combat this outside of the the scoring system that you've already mentioned?

For example, what happens if I run a 25 CoS with 5 different groups, but given your scoring system (again I don't know how this works, so this may not ever happen) it would be an upgrade for the points on 3 or 4 of those groups. Are all of those people going to get rewarded for the run? Or only the group that did the best time for that key (something that could be subjectively hard to judge)?

**Characters Leaderboard**

Awesome again! Seeing the Dungeon + Level is very nice. My one complaint is that this isn't replicated anywhere else (most notably when I go to an individual character). On my individual character pane I have an option for Timed Runs and All Runs, but not an option for "Runs that were scored for character leaderboard." I would like to see what dungeons were scored in a tab on my individual character panel.

Right now if i'm on my individual character pane and total the score from either my Timed Runs or my All Runs it may not add up to the score shown on the Character Leaderboard score. This could lead to confusion as people see missing/floating points. This will also better identify what runs would improve my score. I.e. Would improving the time on the 24 DHT I just run by 2 minutes increase my score or not?

[Ulsoga]

Hey Emancptr - I'm going to leave this one for Aspyr to answer since he'll be better at it than I would. He's already gone off to bed but he'll get to it some time tomorrow

[Aspir]

Hi Emancptr,

We're happy to chat about this stuff, although we aren't going to go into all the exact details of the formula right now, but we absolutely want to hear from people like you who have questions and thoughts about all of this. At the end of the day we want to build a solution that the community is able to get behind, so please let us know where things feel wrong and where things feel right.

Here's some background and a few details:

We designed the system to focus point rewards on successful runs. At the same time, we wanted to provide a benefit to completing a run within the timer vs over time, and to discourage "score farming" where keys are repeatedly run far over the timer just to get additional points for people. We knew there is a big difference in the feelings of "success" for a run when it goes over the timer by just 3 minutes vs going over the timer by 3 hours, so a penalty seemed based on time felt necessary.

To put some specific numbers to it: as soon as a run goes over time its base point value is immediately dropped by two M+ levels, then from there it degrades at a steady clip as time continues to progress over the dungeon's par time.

Even with these penalties though, you can see that someone like Meowchan, has his highest scoring Court of Stars (as of this writing) as a +26 run that went over by 2 minutes: https://raider.io/mythic-plus/runs/2...court-of-stars (361.96 points). His next highest CoS score was a +23 completed on time, providing 345.69 points: https://raider.io/mythic-plus/runs/1...court-of-stars

Are you doing anything to combat this outside of the the scoring system that you've already mentioned?

It's an interesting situation, but we aren't doing anything outside of the scoring system to try to address this.

I would like to see what dungeons were scored in a tab on my individual character panel.

Very good suggestion. We will add this. I'm currently picturing just one more tab called "Scored Runs" on each character's page.

[Emancptr]

To put some specific numbers to it: as soon as a run goes over time its base point value is immediately dropped by two M+ levels, then from there it degrades at a steady clip as time continues to progress over the dungeon's par time.

Even with these penalties though, you can see that someone like Meowchan, has his highest scoring Court of Stars (as of this writing) as a +26 run that went over by 2 minutes: https://raider.io/mythic-plus/runs/2...court-of-stars (361.96 points). His next highest CoS score was a +23 completed on time, providing 345.69 points: https://raider.io/mythic-plus/runs/1...court-of-stars

This seems overly punishing. I'm not sure why you wouldn't just work on a sliding scale past that timer (instead of instantly dropping it by 2 levels). I would guess that 90%+ of the time this scoring system is essentially a "Runs you've done in time" system (would love for you to look at the actual data on this). With the odd outlier like Meowchan who has a dungeon 3+ levels higher, but barely over the timer. Even in that niche case, Meowchan could easily run a 24 or 25 CoS in time and drastically increase his score (just a matter of getting that key).

This system looks to be implemented mostly in a way that prevents people from sharing their high keys and doing them close to the timer. Specifically, from the perspective of NA (EU may be different). Keys that are depleted and shared around for wowprogress score are rarely done anywhere close to the timer. They're usually done 10-15+ minutes over the timer (because the group doing the key isn't capable of doing it closer).

Coming from that perspective, I would like to see something like 0-3 minutes over the timer is an ~1 level penalty and 4-8 minutes over the timer is a 2 level penalty (or something like 10% of dungeon timer = 1 level penalty). This more closely reflects the actual difficulty of the dungeon level increase. If you are only 2 minutes over the timer on a +26 CoS (Meowchan example) there is very little doubt that he would do a +25 CoS in time (if he was given the key and the same group).

[Aspir]

I'm not sure why you wouldn't just work on a sliding scale past that timer (instead of instantly dropping it by 2 levels). I would guess that 90%+ of the time this scoring system is essentially a "Runs you've done in time" system (would love for you to look at the actual data on this). With the odd outlier like Meowchan who has a dungeon 3+ levels higher, but barely over the timer. Even in that niche case, Meowchan could easily run a 24 or 25 CoS in time and drastically increase his score (just a matter of getting that key).

We started with a purely sliding scale, but opted to integrate an initial penalty for a specific reason: We wanted someone who first initially fails a dungeon but then later comes back and completes it to get a noticeable increase in points. If all we did was gradually scale down the points when over the timer then someone could run a barely out of time dungeon and feel less motivated to run it again because the potential score increase would be so minor. Our current system will always reward the player with a nice point increase if they complete a previously uncompleted M+ level on time.

Analyzing the data we are collecting (we have a LOT) in the context of our algorithm is a key part of what we are doing now. Things we learn from both the data, and from players like yourself, will impact how we consider the problem.

Do you have any runs where you felt that the out-of-time penalty was too high for the time it took to complete the run? It would be great to see specific example runs.

[Reckoniize]

I'm at work currently so the filters only allow me to look at the screenshots, but this site looks badass, I'll give it a try in a couple of hours.

[st33l]

Simpler. Please make it way, way, way simpler.

[Emancptr]

We started with a purely sliding scale, but opted to integrate an initial penalty for a specific reason: We wanted someone who first initially fails a dungeon but then later comes back and completes it to get a noticeable increase in points. If all we did was gradually scale down the points when over the timer then someone could run a barely out of time dungeon and feel less motivated to run it again because the potential score increase would be so minor. Our current system will always reward the player with a nice point increase if they complete a previously uncompleted M+ level on time.

Personally, I think your system has the opposite effect. If i'm running a dungeon at the very edge of my ability (24/25/26) the current scoring system tells me not to try a key again unless I can make the timer. Because unless i'm going to make the timer I might as well push another key or pug and try to do the 22/23 (that I know I can do in time for guaranteed easier points).

At that very edge I simply may not be able to make the timer (given current gear/affixes), but maybe I can improve my time by 5-10 minutes. Improving my time by 5-10 minutes (and barely missing the timer on these 25+ keys) is a significant improvement and something that should be rewarded. Instead of just trying to sweet-spot all my dungeon keys at 23/24 (in-time), which is what your current system will promotes. Meowchan is a very good example of this even though he's run a lot of 25/26/27, 80% of his scores on raider.io are 23/24s that he has run in time (or lower because he hasn't done that dungeon in-time on a 23/24).

Do you have any runs where you felt that the out-of-time penalty was too high for the time it took to complete the run? It would be great to see specific example runs.

I think the Meowchan example given above illustrates my point perfectly. A 26 missed by 2 minutes is almost outscored by a 23 done in-time. That 26 missed by 2 minutes is in-arguably (in my mind) a better dungeon run than the 23 that he completed. How much better is it? That's hard to answer, but in my mind it's worth ~25 keystone done in time.

I don't have other examples because I can't remember all of my individual runs and how close they were to timer (and if raider.io even picked them up).

Simpler. Please make it way, way, way simpler.

The scoring system for M+ or the website in general?

[Reloe]

We shouldn't be arguing about depleted keys anymore anyway, you can't just simply rerun them come 7.2.5 unless you have the same key twice, so any argument about giving people incentive to run the dungeon again for more score is kinda pointless as thats going away pretty soon and we can live with whatever for the next 3-4 weeks I guess.

Whats more important is how we want to handle it on the new system. I really like that going overtime immediatly punishes you by 2levels, because when you can't make +26 intime you just reset the dungeon and do it on +25 instead, though that would mean you kinda give up on the loot because when you finish it overtime you get a random +25 key :/
Especially when you only have one shot on your key the system should be rewarding you when you make it intime.

Also wondering if leaderboards should be somewhat reset, or a new season should be started because the increase per level is going from 8% to 10%.
ofcourse we will be getting to higher itemlevels and such but just look at the numbers, it'll take like a month to get back to the levels we were before 7.2.5 and thats just a guess.
live +10: 100% 7.2.5 +10: 136%
live +15: 194% 7.2.5 +15: 280%
live +20: 332% 7.2.5 +20: 512%
live +25: 534% 7.2.5 +25: 885%

[st33l]

The scoring system for M+ or the website in general?

The scoring system being complex is fine - it probably needs to be. The current scoring system used on Wowprogress at least is pretty broken.
The website is currently way too much. One of the key attractions for Wowprogress over the years has been its simplicity of design.

When I go to a rankings website during a progression raid, I don't want to have to look around to see who is winning; it should be right there, bam, one simple list.
In fact, when we had 25m/10m, that was a bit disconcerting.

We should definitely have all different types of data that can be pulled up, but at one time, usually just showing one piece of it, is desirable. The main page should be just the current world mythic rankings; if someone switches to mythic+, we should have the exact same format, same succinct information, now for world M+ rankings. Of course, as one uses knobs to drill down into specific rankings, that data changes but the simplicity of the presentation shouldn't.

The current website has way too much information, which takes a bit of time to parse. Secondly, once I have parsed it, I am not necessarily interested in all of it. How much do I care who is at 1/10m. In fact, I still can't tell what that really even means. Is that the first guild to hit 1/10? Is that guild still there at that progression? Or is it just that the list of guilds which hit that progression first are listed there?

I shouldn't have to think about any of this. The data should be *accessible* to me if I want it; in fact the way to access it should be very discoverable, just not shoved in my face.


I hope this makes sense. I have seen way too many great ideas succumb to overcomplicating things. Simplicity is elegant and it works. Even if you want to give *all* the data, always give it in the very minimal chunks that make sense.