I don't see how it could be considered racist. Physical attraction has nothing to do with ethics.
I'm a sucker for Asian women and any woman short/petite. I tend to dislike taller women. I'm 5'10" myself but I like 'em 5'2" or so. My girlfriends 5'7" but eh at least she's shorter than me!
I'm not really too picky when it comes to what I am attracted to race-wise, but if my kids turned out to not look like me I would be a little hurt inside. I kinda feel bad for thinking like that, but that is probably why I'll never be able to seriously be with a black woman/latina. I can/do find both attractive, though.