Hi Karshda,
Yes if you do not get DI often (or ever) it should be a small bonus to single target and multi-dot dps to settle your haste a little lower. You can keep it above 2141 if there's a chance of getting DI every now and then. How low you go depends upon keeping ES active. Experiment with various levels to find what works for you, but do not go below 1602.