You don't need them. In fact there is every chance you could have killed every boss every week since the raids went live and never gotten a single item from your charm rolls.
You know what the real problem is? Blizzard can never put any gear in the game as a "bonus". Because players have developed a mentality whereby if it is possible to get something, it's mandatory. Period. Doesn't matter how minor the advantage is or how time consuming it would be to farm, if it exists it's mandatory. This is not Blizzard's fault, it's a bad mindset.
P.S. The best use for charms is obviously on Sha and Galleon! MOOOOUUUUUNNNTTTSSSS!!!!