One thing is racism and other thing that none usually talk about is aesthetics.
Some people simply don't like X skin color, they could still be good friends with those but they won't as they mostly won't start words with them.
I think that part is fine as long as no racism is involved. It's like going into a gay bar with friends, you could speak with someone inside in a friendly way, but there's no point because you don't want anything with that people, yet you respect them since them since you don't have kind of problem with them.
That said, I don't have any of those issues, but that's definitely not racism
I think racism is like the internet trolls, everyone will call someone a troll even if he's not, even he simply did a bad joke. Same goes with racism, some people think in some way, yet it doesn't mean they are racists.