Some of those Founders, like Jefferson, were Virgianian.
A lot of Americans like to pretend their Founding Fathers were mythic heroes who stood head and shoulders above their fellow men, ethically and morally and intellectually, but a lot of them were pretty fucking stupid, and a lot of the rest were amoral sons-of-bitches like Jefferson, the brutal slave-raping slaveowner. Who, despite writing some supposedly pro-abolition stuff, went home and raped his wife's half-sister (whom he owned, as a slave) repeatedly, fathering children on her, and then never emancipating her or his children, even though he absolutely could have.