The idea behind this is that eventually they'll catch up to you, especially if you continually draw attention by making your way to and from a location. Now obviously the show removes a sense of realism by "manufacturing zombies" in large quantities, common sense tells us that eventually you'd kill everything in the local area but that's hard to decisively say for ourselves. But under these (largely made up) rules, the threat would still persist, and would have to find some way to block them out long term (walls, tree forts, something). On the other hand, you cannot deny that going
toward a heavy population center isn't exactly the smartest thing to do either. More people = more zombies!
Biggest problem is that the further out you are, the further you need to go for resources, which also consumes more resources (gas, food and water for travel). Likewise the further out you are in the wildnerness, the less likely you are to have room to fortify anything. Sure you could find a cave, but those generally have one entrance/exit point. Not a great idea when a horde could come knocking at your door. Also there is a problem of light/space/etc. On the other hand, you might find a large field with good sightlines, but it would take a whole hell of a lot of work to build any kind of "fort" using hand tools alone.