Until you get interrupted. Congratulations, you're now locked out of casting everything for x seconds. It's why the only things that deal Chaos damage don't have cast times.
I'd get on board with an Arcon-esque change to AP though. Maybe a small power increase, but it also alters your appearance and makes you immune to interrupts or something?