I tried all three - unstable magic, nether tempest and supernova - in UBRS runs. For the boss fights as well as trash supernova seems to be performing way better than the other two. I went from being nr. 4 on recount with NT, below the tank, to nr. 1 with Supernova, doing about 40% of total dps in trash fights and 30ish on boss.
From what I have seen so far, Nether Tempest does a lot less damage than Supernova, even single target. Also it feels kind of clunky to refresh, since you should cast it as soon as you have 4 charges, but I don't often have that many arcane missile charges which means I would already have to refresh it after one AM or AB cast, because if I wouldn't, I probably wouldn't make it to the next 4 charges in time to refresh before the previous NT is already over.
So in effect I'd be casting it twice, once as soon as I have 4 charges, and once right before arcane blast to make sure I can refresh it in time before it runs out. But that means I'd be losing dps because the second time I'd cast it, it wouldn't already be in the last 3.6 seconds, more like somewhere around 6 seconds.
Not sure which is better between NT and UM, but I think I'll be sticking with Supernova for now. Between the three, Supernova excels in dps for me.