Good thing no one is arguing that's the only way the source material needs to be interpreted then.
Why do they have to always be evil if we're talking about an adaptation that takes tells a story that exists beyond what was in the original novels? New stories can be written, new characters introduced, without shifting the universe and creating a multicultural melting pot out of every race.
I mean the 2nd age is ripe with stories of how the Numenoreans oppressed and enslaved the men of the east and south, which lead them to distrust the men of the west and side with Sauron. Interesting stories can be told in between, like having a civil conflict between those who sought to join Sauron and those who refuse side with darkness. There's plenty that can be done with the source material without literally shoehorning people of ethnic backgrounds into white-centric roles.
I'd say the same thing about casting Asians into the role of Elves or Dwarves for the sake of diversity. Why do that when Middle Earth is absolutely open to exploring Asian culture through any other number of means? Rhûn exists. Easterlings exist. There's plenty of untapped material to work with here.