I have illusionary mage and warlock, 2 phantasms. I think you are mixing up phantasms recharge time and illusion skill recharge time.As for your traits, I feel the last Major you chosen for Illusions to be a bit of a waste (because I favor Staves). The only phantasm at your disposal is Illusionary Mage. You're probably better off with compounding power with 9% damage increase versus faster skill recharge on a 20s CD spell? No brainer there.
The trait phantasmal haste, reduces the recharge time on phantasm skills (the skills phantasms use) not the actual summon phantasm.
So more retaliations + confusion with illusionary mage and more damage per second with warlock. Which I find more handy over a 9% increase in damage if i have 3 clones out, so at times it will be just 3% or 6%.
I tend to avoid traits like +50 power or +3% increase, it's just boring, they have minimal effects, and there are better utility's out there that I could afford such a low damage buff.
I'm curious to see if confusion has a stack limit. If so I can probably drop a few confusion things like mirror images.