Right now, that's what happening, except that there are some states that chose not to have a floor.
They are using the current minimum wage, which is $7.25 an hour, and therefore (40 hours a week, 50 weeks a year) $14,500 for a full-time job.
The poverty line for a single American is $12,500.
Now maybe your point is "every state should be required to set their own". In which case, yeah, we can find common ground on that one.
Until then, there are seven states with no minimum wage. They are all red states. Funny story, all seven are also "right to work" states. So if we simply abandon the federal minimum wage, those seven states not only would allow employers to pay $5/hour but would also prohibit collective bargaining to fix that.
Maybe you'll argue $15/hour is too high. We might find common ground on that one, too. But simply throwing away the federal minimum wage and making no other changes is not an option -- or, it is an option that will just flat-out kill the lower class in seven states.