I'd still play the game because it's not the only reason I play it, but I'd feel like I'm missing quite a huge part of the story/lore/gameplay because I don't have time to play in a guild and do normal raiding, or even PU raids.
I don't think LFR has ever been an issue appart from, at times, the rewards from it that could feel mandatory to get for some higher raiders. The system in itself, giving the opportunity to more players to do the raids and get some gear, is fine.