There's two only somewhat related factors to consider.
You seem to be focusing on the internal economy, on a per-capita basis. While it's true that more population doesn't necessarily boost this directly, in a global economic sense, the bigger your population, the bigger market you provide, and the more weight you have in influencing that global market in your favor.
Since the OP is referring to Sweden I´m just gonna say(sorry if already said.), that employers do NOT pay you anything when on paternal-leave. So stop arguing that in regards to the OP.