only on fights that required it... they would actually be allowed to be used, legendaries would become situational based on fight mechanics, mobility, survivability etc instead of just "use whatever one has the strongest DPS increase special effect"
and it would be a miniscule difference in DPS compared to how it is now.or the one with the best stats for your spec.