There are ways to compare tanks. Less damage taken, more damage done, etc. I have everything on my Skada set up for this (Skada for Tanks from Curse, just an extra few Skada meters). World of logs is great too; me and my guildies who are tanks (4 in total, a warrior, Druid, monk, and myself, another warrior) frequently go through logs to help determine if we can make ourselves better (better defensive/cooldowns usage, etc).
It's the same with DPS and heals. Sure, you could go by "who did the most DPS," but I'd rather have the guy who did 195k but had 50% of his DPS on the adds killing the healers than the 220k guy who tunneled the boss. More often than not, adds are more of a priority than the boss.