Have you tried speccing frost and having your Mirror Images/Frozen Orb/Water Elemental help you kill off the little adds? You can also get away with a lot of FFB/Lances if you're running Nether Tempest or Frost Bomb as they can't get locked on cast (no cast, wohoo!).