Each phase, and the subsequent "add ons" for each phase (anvil, alchemy table, etc.) require a set number of turn ins of the specific quests that fund them. There's no mechanic that increases that number. So no, a realm with a higher population is not handicapped.
There is however a mechanic that will auto-complete each phase after a set amount of time for those realms that aren't able to complete them within a "decent" amount of time.
What you may be seeing is a general cool down of people taking part in those quests? I can't really say why your realm may be seeing a slower progression aside from them just not completing as many quests as another realm.
Once the Armory is taken over you can begin building the Anvil, once the Anvil is complete the new badge vendor becomes available. (
Source)