Why I have not been in raiding guilds is I have a late night schedule and cant always play. Guild drama was not much of a problem and I enjoyed playing with the players I raided with, just couldnt line up my play time with them anymore and moved to PuG raiding. Its a fallacy that you need a guild to see raid content. Why I moved to LFR is because it was the only way left to both see raid content and progress my character after the massive drop off of late night PuGs on my server after the introduction of LFR. If I could move life around and play during prime time then I could jump into those PuGs and have engaging group based content that fits my casual play times. Also five mans took a beating from the nerf bat and have been made almost useless in order to prop up LFR. If I could have done Challenge modes as an alternative to LFR then I would have but we cant have that because players would cry for nerfs. I had to stop playing LFR as it was just needlessly frustrating and felt far more tedious and grind like than PuG raiding normals and I ended up drinking a lot while still topping meters.
In the end I stopped playing due to the lack of engaging group based content as a casual.
I come from a high population realm so there are a lot of players and if you set your standards too high without having something to back it then you are going to fail as players will have another guild they can look at. Same thing went on with PuGs, if you set an absurd requirements while not meeting them yourselves then you was embarrassed in trade and your raid likely didnt fill. Ah the things that low pops have to deal with that might not be an issue much longer with virtual realms.