Neither being a Nazi or a Fascist is a crime in the US.
Why wouldn't every major religion be considered a crime?
Why wouldn't being a Republican or Democrat be a crime? What about being a Royalist, Federalist, Socialist, Anarchist, etc?
Because the red shits won in the WW2.
There are countries where communism symbols are banned (at least on paper) and they are basically on the same level as nazis. But then again, majority of the West escaped the horror of living under communist rule, so they do not know better and thus weekly threads about how communism could actually work.
Ask Eastern Europe about free trips to Siberia, ask Chechens and you will have the answer.