The magic of "The writers don't care about plot holes you could drive a truck through."
Alternatively, their military forces were decimated, but it was mostly orcs, anyhow, and now their kids are taking up the sword 4 years later since Orcs breed like rabbits in a Viagra testing center.