I want to bring this up as I think the community opinion hasn't adjusted properly to the changes in BfA.
There are two main components of comparing stats that need to be taken into account:
1. The quality of damage reduction provided.
What this means is that we would prefer 1 % DR from versatility over 1 % DR from crit. Why? Because versatility is reliable and crit isn't. Crit gives parry which you cannot count on as it's random.
Similarly, we would prefer 1 % DR from haste over 1 % DR from versatility. The DR haste provides comes mostly from higher SotR up-time which is not only reliable, it allows you to choose when you want to use it, providing superior control (all the DR granted by haste can be concentrated into a few seconds when you need it the most).
Finally, mastery works partially as versatility when standing in consecration and partially RNG giving block chance. That puts it just below versatility in usefulness.
That gives us the following priority: Haste > Versatility >= Mastery > Crit.
I have seen a lot of people stop right here and pronounce that as our stat priority.
Unfortunatelly, this isn't all there is to consider and we have to look at the other part as well:
2. The amount of damage reduction provided by 1 rating.
The simplest way to present this idea is this: Even though we can all agree haste is the "best" stat, getting 10 % DR from haste is not as good as getting 50 % DR from versatility (these numbers are just for illustration, the difference isn't that high).
In the past the stats were much more balanced, so this played lesser role. We knew that haste used to be weak numbers wise in Legion, but the difference was smaller so the quality of haste was more important than its "lack of numbers".
I have done some calculations about the raw strength of stats (not taking the quality into consideration) in this spreadsheet.
Coincidentally (not really because I'm convinced the developers aren't that stupid and did it on purpose), the raw strength of stats is the exact opposite than their usefulness: Crit >= Mastery > Versatility > Haste.
Another thing to consider is that the spredsheet assumes all damage is blockable, parryable and reduced by armor. Against magic damage mastery and versatility keep their full value while haste and crit are close to worthless.
So how do we come to a conclusion when we have two competing components?
It's not simple, not only is it partly a matter of opinion, the answer also varries depending on the encounter and its damage paterns.
Comparing to established Legion values.
(I will be comparing to versatility because it is a stat that hasn't been changed and was considered a solid 2nd defensive option in Legion.)
In Legion, the accepted recommendation was to prefer haste as our best stat, but still prioritize item level when the difference was more than 5 item levels.
For jewelry, rings and necks (since they don't have primary stat and armor), the recommendation used to be 15-20 item levels.
This was quoted multiple times a day on the Discord prot paladin channel, including by moderators and templars so I think it's a good place to start as a general gearing strategy good for most situations.
15-20 item levels translated to 10 - 13 % more secondary stats. Assuming roughly 50/50 split between two stats (say, haste/crit vs. vers/crit ring), we get 20 - 25 % more vers than haste if we keep equal crit. There is also a little bit of stamina, which isn't too important, but not completely worthless either, so let's bump it up to roughly 30 % difference.
So since we know that haste was valued about 30 % higher than versatility in Legion, what does it mean today?
The value of haste dropped significantly because of the nerfs to SotR:
1. The DR provided by SotR is much smaller now, likely not exceeding 40 % (depends on your strength vs. armor). In Legion it used to be much more powerful (also buffed by mastery).
2. SotR only provides armor now, which means it doesn't work agains magic damage or bleeds. You might have noticed being a lot more vulnerable against Ymiron in MoS.
3. Base up-time is lower because the cooldown of SotR has been increased to 18 seconds. That also means that increasing the up-time by the same percentage gives smaller absolute up-time increase.
Despite the value of haste dropping significantly compared to Legion, I still consider it very useful and believe it's probably slightly better, especially in raids with a lot of moderate tanking up-time encounters (typical tank-swap fights) and m+ with some downtime as well (finishing off 1-2 mobs, running between packs etc.)
That being said, the lead haste has on versatility is much smaller nowadays and rather than 30 %, I'd say it's better by a very small margin of maybe 10 %. If you tank something with 100 % tanking up-time, I wouldn't even object to considering haste worse than versatility, but such encounters are practically nonexistent.
That brings us to the second part of the comparison and that is the new mastery. Old mastery was quite bad, but the new one is amazing.
Consecration portion is reliable and is worth about 85 % of versatility by itself. Add in the block chance part and we have an extremely powerful stat. Sure, blocking isn't reliable, but it isn't as bad as pure dodge/parry in terms of smoothing. If block was reliable, the value of mastery would be almost 70 % higher than versatility.
Even discounting block by 30 - 40 %, mastery still beats versatility by about 40 %, which is arguably competitive with Legion haste, definitelly more than BfA haste.
There are some other more ways to compare the stats more dirrectly, but they aren't quite as proper.
Still, I'll include some for illustration (talking mastery vs. haste since I believe mastery to be the strongest stat and the community mostly agres on the rest of the stat priority).
1. One counterargument I have received a few times in relation to the spreadsheet was this: "with 150 extra stats, the difference in damage taken between mastery and haste is only about 1 %. That is quite low and I'd rather have the control provided by haste over 'measly' 1 % DR."
The problem is that 150 stats is a reasonably small amount. I used to have much bigger numbers in the spreadsheet for this purpose, but it was pointed to me that they were unrealistic. It's not like you can decide to reenchant a couple rings and gain thousands of stats.
What the people making this argument don't realize is how many extra SotRs you get for that 1 % DR. On a typical fight of 5-7 minutes, the answer is less than 1. You don't even get a single full SotR with only 150 haste. So the idea that there is some tangible increase in control that you can actually feel during the fight for this much stats is incorrect.
2. Comparison to Bastion of Light talent (3 SotRs per 2 minutes) = 1575 rating is equivalent to 14.69 % DR from mastery. Would you rather have BoL (7 seconds of on demand 40 % reduction per minute) or a new talent that gave permanent 15 % damage reduction at all times?
While there are a few fights (think 3tanking Argus p3) where the BoL is the difference between having and not having SotR up for every big hit you take, on the vast majority of encounters the "new" talent would the clear winner.
3. Extreme option of incredibly high amount (5k) of stats:
At 5k stats, haste gives roughly 40 % of SotR Up-time (which is 38-39 % DR), while 5k mastery gives 38.4 % at all times (effectively providing the equivalent of permanent SotR).
With all this taken into account (and a couple of things that were beyond the scope of this, such as the small increase in SotR up-time from crit), my personal recommendation for defensive stat priority is Mastery > Haste >= Versatility = Crit.
I am more than happy to discuss this on Discord, I am also perfectly happy to be proven wrong (I'm not perfect and it's quite possible that there are mistakes in the spreadsheet that I used for the raw data as well as in my reasoning).
I only ask that if you disagree, you provide your reasoning and arguments that aren't just "The quality of DR provided" mentioned at the beginning because as you can see, I have already taken it into account and used the generally accepted Legion conversion for this value.