Cancelling a buff while in combat is a protected function. Addons are not allowed to automatically do that. Your best bet is to make a macro for '/cancelaura Your Buff' and make a huge Weak Aura that yells at you to hit that button.
One of our tanks literally macroed `/canceraura Hand of Protection` into *every* common ability because this one time it got cast unexpectedly...
I kind of wish I could get an actionbar replacement that would let me write a "template" for abilities, so I could drop, eg, rune strike onto it and end up with a nice macro that would do some cancelauras that I need, proc that trinket, and that other ability, I want to use on CD rather than syncing with anything, and do a bit of other stuff automatically. (By generating a macro script attached to a secure button, obviously.