How many black people can the n-word and be a racist? Very few if any.
How many white people can the n-word and be a racist? At best its a coin flip. The ones that aren't being racist are probably just quoting Mark Twain.
- - - Updated - - -
I'm not claiming its rational or irrational. People's opinions and actions vary. I prefer to err on the side of not using them at all. I can usually determine someone's intent in any case.