I'm about prestige 6, rank 44 and i hardly ever do it... why?
Casual pvp is extremely frustrating to do; either you get grouped with people who steamroll the opponents, or far more likely people who just get steamrolled... (Can't count all the times i was highest on healing meter as ret pally in legion.
Rated pvp on the other hand is extremely frustrating to get into; everyone and their grandmother keeps requesting 2k or 2.2k experience players... i just don't have the nerves to get trough them.