We all know that the current LFR and LFD systems aren't stellar and that's because of the people we encounter in them. I've heard LFR called a cesspool and after tanking in it I know why that is.
The vote-kick system isn't really helping that much (with it's mysterious internal cooldowns that I've not found any official information on). And in LFD the kick is limited to one person, which sometimes gets replaced by an even worse person.
AFK'rs, ninja looters, griefers and so on. We've all encountered them time and time again. And even if they do get kicked they just pop back in another instance to cause another group problems.
I've read quite a few threads on the blizz forums about the problems with the system so here is my suggestion and my open invitation to other people that want to add in their 2 cents on the matter.
Reputation points. You can downvote someone in your group. The more people downvote that person the lower their rating goes and they get paired up with people with the same rating. The ratings can be reset in a month or so to give those people a new chance.
Same goes for people with good rating - they get groups with similar rating to theirs. This would work in LFR as well as LFD.
Those ratings should be visible to the player as an encouragement.
I'm sure my suggestion has a lot of flaws but that's why I made a forum post about it, to hear opinions and suggestions on it.