Most people do not raid. Even at it's highest participation, which is ICC and DS, the participation was, comparative to the numbers of players, REALLY low, and those periods, even, were comparably enormously high.
Most people, to me, seem like they don't want to do what it takes to raid, or see it as unnecessary to get enjoyment out of a game. Can you really blame them?
What's the bizarre thing is, is Blizzard seemingly is still focusing on that few percent that do.