I'm sure they can tell a difference, and a girl's first cycle dictates maturity not breast size. But I doubt they walk around with boners all day, or disregard the B cups of their tribe because they find it unattractive. Whereas in Western culture, your average teenager will likely masturbate to hollywood nip slips.
Other mammals have different breasts within their own species and subspecies...The differences may be less evident to us because we judge primarily on size. A good example would be cows. Now we breed cows to have the best possible udders for rearing young and producing milk, so you may even think that they just come in 1 size/shape.
Besides a human's race, weight, and skin have a lot more to do with breast size and shape than some evolutionary "gene" dictating breast size.