Woman here, I personally like my man to be cleanly shaven. Soul patches and goatees aren't that bad tho, but a mustache and full beards are no no for me.
But this is all personal preference. You shouldn't alter your looks after what strangers think. If you want to have a beard, then grow it and sport it proudly.