You can't just 'remove' players either. Certain game design promotes certain behavior. LFR in its current inception is a strange place. While on paper it's supposed to act as sort of a raid safari, it's actually used for a number of different reasons by various different kinds of players. Very casual players use it to see content. Casual players looking to progress further use it as a gear springboard. Normal and heroic mode raiders use it to fill missing tier pieces and to get trinkets. PVP players might use it to get easy access to trinkets and others just want to farm pets.
This creates a big discrepancy of gear and boss mechanic knowledge in a single raid and an atmosphere where almost no one wants to be there with each other. Heroic raiders are annoyed because people are dying to content they outgear and have been farming for weeks. Casuals are annoyed because they're not in a rush and don't care about maximizing their dps, and so on.
LFR currently caters to too many people for what it's supposed to be. I personally wouldn't care if they removed it, although a solution that keeps everyone happy would obviously be best. It's good to try and get the community to work closer and better with each other, but LFR is achieving the opposite effect.