Why bug reporting system does not work and how to fix this

[kaminaris]

Preamble: I am a developer with about 8 years of experience. I am also addon dev.

OK so I've been thinking about in-game (and forum) bug reporting system and have few thoughts:

1. This system will just not work. 500 chars limit, no way to easily attach a screenshot, movie. Does not have guidelines what to write.
2. No way to search for existing reports to not create duplicates.
3. Tons of wasted time to sort all the duplicates for blizz employees.
4. No way to provide more information at later date.

One of the most archaic system I have seen.

Honestly, this might be a nightmare for people trying to figure how to make a tasks for developers to fix this. So I just got a feeling that bug reports are actually being ignored (not just me).

Now take a look at this bug report system github has:
https://github.com/magento/magento2/issues

For starters it has labels for each issue to identify it properly ex:

• Component: Checkout
• Issue: Clear Description
• Issue: Confirmed
• Issue: Format is valid
• Issue: Ready for Work
• Reproduced on 2.2.x
• Reproduced on 2.3.x

People can react with emote to let them know they are experiencing the same issue. (Thumbs up). Or to comment if they have more to add (steps to reproduce, screenshots, movies etc.)

So instead of creating a tons of duplicated bug reports (which are impossible to sort out) they could just stick with one thread.

Additionally for WoW these bug reports could be moderated.

1. If issues was duplicate - merge with original thread
2. If it was some garbage report -> delete
3. If it is exploit -> make it non-public

Of course if someone finds exploit nothing stops them from making it public on other forums so... that's not really a problem.

Let me know what you think. I would love to report bugs but the current system does not encourage me to do so (Pet bugs on Siege of Boralus which I reported back in beta are still not fixed)

Well obviously, to fix it, we need a modern bug reporting system. Similar to github.

[Ezekeel]

Last time I used it I could not even properly describe the situation due to the character limit. I suspect its only function is to take the steam out of players. If it was used to actually track fix bugs, people who use it would try to get it improved. Maybe it is only used to make a content analysis and automatically find trending bugs, I don't know.

[FelPlague]

Last time I used it I could not even properly describe the situation due to the character limit. I suspect its only function is to take the steam out of players. If it was used to actually track fix bugs, people who use it would try to get it improved. Maybe it is only used to make a content analysis and automatically find trending bugs, I don't know.

blizzard has said many times it needs to be short and concise, they are not going to read 50 pages of a bug post to find out it was not a bug and the player was just using the system wrong...

[Lucetia]

This system will just not work. 500 chars limit, no way to easily attach a screenshot, movie. Does not have guidelines what to write.

That's because they've said multiple times it needs to be short and concise. Not spend forever describing something that took 3 words.

They've also said Screenshots, videos, etc aren't looked at because they don't use them as a viable means of reporting something due to them being able to be edited.

[cparle87]

Preamble: I am a developer with about 8 years of experience. I am also addon dev.

OK so I've been thinking about in-game (and forum) bug reporting system and have few thoughts:

1. This system will just not work. 500 chars limit, no way to easily attach a screenshot, movie. Does not have guidelines what to write.
2. No way to search for existing reports to not create duplicates.
3. Tons of wasted time to sort all the duplicates for blizz employees.
4. No way to provide more information at later date.

One of the most archaic system I have seen.

Honestly, this might be a nightmare for people trying to figure how to make a tasks for developers to fix this. So I just got a feeling that bug reports are actually being ignored (not just me).

Now take a look at this bug report system github has:
https://github.com/magento/magento2/issues

For starters it has labels for each issue to identify it properly ex:

• Component: Checkout
• Issue: Clear Description
• Issue: Confirmed
• Issue: Format is valid
• Issue: Ready for Work
• Reproduced on 2.2.x
• Reproduced on 2.3.x

People can react with emote to let them know they are experiencing the same issue. (Thumbs up). Or to comment if they have more to add (steps to reproduce, screenshots, movies etc.)

So instead of creating a tons of duplicated bug reports (which are impossible to sort out) they could just stick with one thread.

Additionally for WoW these bug reports could be moderated.

1. If issues was duplicate - merge with original thread
2. If it was some garbage report -> delete
3. If it is exploit -> make it non-public

Of course if someone finds exploit nothing stops them from making it public on other forums so... that's not really a problem.

Let me know what you think. I would love to report bugs but the current system does not encourage me to do so (Pet bugs on Siege of Boralus which I reported back in beta are still not fixed)

Well obviously, to fix it, we need a modern bug reporting system. Similar to github.

I think in this case the problem is... you're a developer and are thinking like one. Not as a gamer/consumer of this product. 500 letters is more than enough to describe an ability not working and how, an npc not behaving as expected and how, and such.

[majesta]

It's funny how I can see a hacker in a lot of games and file a bug/cheating report and the game is able to include gameplay footage around the time the incident happened. Maybe something like that should be implemented so that real bugs or straight up odd stuff can be observed first hand. I think it would be a better system than what is in place.

For screenshots, there is a solution. Encoded timestamped yadda yadda aka verifiable legit. You know the stuff that holds up in court. Maybe Blizzard can start doing that with their screenshots so they are trustworthy then?

[kaminaris]

That's because they've said multiple times it needs to be short and concise. Not spend forever describing something that took 3 words.

They've also said Screenshots, videos, etc aren't looked at because they don't use them as a viable means of reporting something due to them being able to be edited.

This is because the system is bad and they are getting thousands of bug reports about the same thing over and over again. This is the root of the problem.

Once those 1000 reports gets milled down to one by their QA team. Developer gets detailed report with screenshots, movies etc.
So they are yet again reproduce this issue, make proper report and hand it over to developer.

[Lucetia]

This is because the system is bad and they are getting thousands of bug reports about the same thing over and over again. This is the root of the problem.

Once those 1000 reports gets milled down to one by their QA team. Developer gets detailed report with screenshots, movies etc.
So they are yet again reproduce this issue, make proper report and hand it over to developer.

Except that's your guess. You have no idea how they work. Detailed reports with screenshots, videos, will simply have those links ignore. They don't want that information at all. How is that hard to understand? Just because you want to provide that information or have it provided to you doesn't mean they do.

Being short and concise is a lot better than having to trudge through a 10 minute video just to get something across that you could've wrote in shorter words.

[kaminaris]

Because I am a developer and I know how it works. That is not my guess.
And this is why most of those "short and concise" reports are being ignored.

I could write: "My pet bugs out on siege of boralus on last boss on last platform"
Will they understand what I meant? Nope
Will they even try to reproduce this? Nope

Because it will get filtered out by their QA team as nonsense. Like 999 similar reports.

And before you say they are don't want my bug reports, well, you are wrong.

Somehow blizzard fixed a hole in textures in Azjol-Nerub when I posted a bug report about this back in pandaria.
Yet they didnt fix a pet issues on boralus even tho I reported it about 3 times (first time back in beta BfA, and 2 times on live)

This is simply limitation of archaic system that doesn't let you check out if users didn't report similar bug already.
Which is most important. To see how often bug appears.

You don't want to waste your developers time on something that happened once or twice.
You want to fix issues that happens often and repeatedly.

And some facts: big blue post on blizzard bug forum how to make proper bug reports:
https://us.battle.net/forums/en/wow/topic/11307560726

With some lengthy example:

Summary: Beacon of Light does not proc off of heals cast on Tsulong.

Steps:
1. Form and raid and enter the Terrace of Endless Spring Instance with a Holy Paladin.
2. Progress through the raid instance until you reach the Tsulong Encounter.
3. Engage the boss, deal damage to it, and wait for the Day Phase.
4. Have the paladin cast Beacon of Light on a player in the raid.
5. Target Tsulong and cast Holy Light.

Outcome: The Beacon of Light heal effect will not get triggered on the player with the buff.
Expected Outcome: The Beacon of Light heal effect will trigger on the player with the buff.

So basically I am suggesting to replace the current in-game system to something similar that bug reports forum has but with the ability to search for a bugs and include screenshots.

[Lochton]

blizzard has said many times it needs to be short and concise, they are not going to read 50 pages of a bug post to find out it was not a bug and the player was just using the system wrong...

This is the truth.

[Nomads]

I reported a "bug" last month. It was more of an overlooked consequence. If not being able to get Garrison Follower rep after level 111. It was fixed a few weeks later.

[MrLachyG]

They don’t read them anyway, what’s the point? The amount of bugs that I reported during beta both in game and on the forums that made it to live was quite numerous

[valax]

It's funny how I can see a hacker in a lot of games and file a bug/cheating report and the game is able to include gameplay footage around the time the incident happened. Maybe something like that should be implemented so that real bugs or straight up odd stuff can be observed first hand. I think it would be a better system than what is in place.

For screenshots, there is a solution. Encoded timestamped yadda yadda aka verifiable legit. You know the stuff that holds up in court. Maybe Blizzard can start doing that with their screenshots so they are trustworthy then?

In 7.3.5 in my server there was a warlock clipping inside walls/pillars in orgrimmar dotting everybody and refilling constantly his HP , when I contacted blizzard they said they would not take screenshots or videos as evidence of the hack , even though i was a demon hunter and spectral vision could show exactly what the guy was doing . A whole lot of people must have reported him , still 2 days later the dude is back griefing AH and killing whoever dared to flag himself for pvp

[SirCowdog]

They don’t read them anyway, what’s the point? The amount of bugs that I reported during beta both in game and on the forums that made it to live was quite numerous

This right here is more accurate than anything else. No amount of high-tech, easy-to-use and intuitive report system will help if Blizzard doesn't bother to read the reports.

[Deleted]

Real reason why the bug reporting system does not work: All reports are directed into the recycle bin.

[kaminaris]

Real reason why the bug reporting system does not work: All reports are directed into the recycle bin.

Nope, not all of them but that is the consequence of outdated system not reason.

Now lets focus from blizzard perspective how this system works:

Lets say they have 1000 developers. (blind guess)
Lets say their QA team is 300-400 people (at most because of 1:3 ratio).
And then we have over 3 000 000 players


1. Blizzard releases patch 8.1, a tons of things broke.
2. Short burst of bug reports are being sent.
3. Tons of duplicates, some of them are written by idiots.

Now imagine yourself as QA member that has to go thru all that crap.
10 000 reports about quest giver missing, each of them has different wording
5000 reports about dungeons being broken
3000 reports about raid boss being bugged
etc

I would simply go insane

[Tiwack]

I don't think it should be in-game for one, but it pads customer satisfaction by having it be so easy and makes players think they're actively helping while blizz get to save money avoiding the intensive QA they're known for. They have to prioritize and many inconsequential bugs are left to stay as they'd rather just leave it and make secrets instead of bug-fixes with their free-time
Can't say I blame them given the age of the client. It can't be much fun or easy work to work out and fix some of these glitches in the spaghetti code.

[thebdc]

What's even more crazy is that they used exactly the same system for people to report bugs in the alphas and betas for the expansion up to (including WoD). In Legion they introduced new UI elements that popped up after a dungeon boss, or when you finished a quest, where you could directly press a button if there was a problem with it, and you could also enter comments. I imagine that provided much more useful information for them.

[Biztechcs]

Thanks for the suggestion you gave.It helped me.

[FAILoZOFF]

Yeah, like.... let's pretend they actually care about bugs anymore... Xd hahaha good one!