I agree, but we have what we have. Until someone confronts Danuser on this and makes him change his stance and/or provide additional context and reasoning, this is canon.
I don't really think that your Bastion theory could work though because ending up in Bastion is supposed to be earned. It's reserved for those who lived a life in service. You can't just dump anyone who needs a memory wipe there.