Excluding the bad launch period where perfomance was all over the place, the thing that broke the thing can be separated in two factors mainly (and i'm talking about Vanilla and later on, ignoring all the improvement of RoS which were very good in the short term but bad on the long term and not really effective due to people having left already to never return).
First point was the "transfer hype" i mentioned before. A lot of players were the ones that came from WoW and not from D2 - way too many of them believed the game was simply "something different", was "too repetitive" or similar reasoning; they jumped the ship pretty soon just because they were mislead by other people's hype and they just didn't like the classic ARPG formula, which is totally fine and only natural from my point of view.
The second, that hurt the late game more, was the AH - in the sense that all the itemization and difficulty system was tailored on the AH existance and the caveat that people would have bought gear instead of farming it (very likely to reap dollars from the RMAH). Most items were totally useless due to total random rolls (like skill bonuses on a weapon that class couldn't equip) and the fact only a few stats were valuable. Playing the game was useless as most of the time hitting Inferno you could do quite comfortably the first act (when difficulties were only doing 4 times the main story, there was no Torment levels/adventure mode and monster power came out only after a while aswell as paragon) but you were totally wrecked by act 2 literal first fucking bees. And i mean LITERALLY.
Act 1 inferno monsters were level 60, so they could drop up to ilvl 60 whith an abysmal chance of dropping 61. Monsters in act two were lvl 61 and they just wrecked the shit out of you until you cheesed the whole game with some fancy perma invulnerable glass cannon build.
It turned out totally awful, especially for old school fand that were used to the D2 system where you could trade, but it was complementary. In D3 you played the AH and that drove most players oput of it immediately. They did improve the situation but it was all "too little, too late" and as Kotaku has reported, the second D3 expansion was canned the day RoS released without even seeing if it was a good thing or not - all the stuff that came out shortly after were things already developed for the second xpack (including the Necro) but salvaged to avoid having them just thrown in the trash.