Let me explain it in more depth:
ALL incoming damage to a tank consists of:
- melee autoattack damage
- specials
- (in 5.4) flagged specials (don't give vengeance, stopping tanks standing in fire and shit)
Melee autoattack damage goes under an algorithm to quickly give you most of the vengeance that you would expect to get if you were getting hit straight for 20 seconds. This would mean that all auto attack damage is based on averages, and not avoiding the hits, and not mitigating the hits. Have you ever noticed how your vengeance 5 sec into the pull isn't much less than your vengeance 20 sec into the pull at all?
Specials that are not flagged (so they give vengeance) are strictly converted to AP based on the original damage that would've been dealt to a naked player.
When your vengeance is updated, it goes through this formula (
source):
old veng - decay + new veng = new total veng (decay being 5% per second since last update)