Milo has been associated with the alt-right since the very beginning. In fact he wrote articles popularising the term back when it was still obscure. The Breitbart emails that leaked in 2017 revealed that he was covertly working with neo-Nazis and white supremacists the whole time.
But sure, keep lying to yourself. You're not fooling anyone else.