Well, wouldn't using JUST a weapon make an Ogre an easy target for someone who can dodge it and go for any major weak spot within the blade's reach?
If we're talking about armorless combat the ability to actually land a blow is important, and ogres don't really shine in that regard if their strike doesn't connect.
Considering your comparison, while I do agree and actually know a few serious bodybuilders who are suprisingly flexible and fast, given ther looks... Garrosh is known to be quite bulky, even by Orc standarts, unlike his father, so he wouldn't be
that far away from Cairne in terms of body builds. So them being a good match with Cairne isn't something unexpected, it's just Cairne, relying on vast his battle experience, doesn't have to move that much to counter Garrosh's attacks. Technique versus brute strength. Plus... well. Difference in weapon choices, I'd bet my money on someone fighting with a spear, rather than a two-handed axe, but that's just my opinion, thinking it a more versatile weapon choice.