Nah, can't remove them. You just buy extra ones after completing the quest, for 10,000 gold each I believe.
What I don't get though is why bother allowing us to stack extra sigils if they aren't going to be used? I've got 18 sigils of power in my bags, and I figured that you could use extra sigils to get more gems (maybe not another 10 of each, but like 3 of each per extra gem or something). But no, apparently not. 10,000 gold please.
Ah well. I definitely won't bother gemming an LFR weapon if I happen to get it: my guild has killed both Tsulong and Shek'zeer on normal mode, so it's only a matter of time until I get one or the other (Gao'rei/Claws of Shek'zeer).
10,000g seems to overpriced :O But looks like Blizzard are realy going in for those gold sinks this expansions :P Good things I guess, but I dont got the gold for all this..