Apart from the fact that it is ok to let Transgender people use the bathroom of the gender they identify with (for the reason I mentioned above, something bad coming from it is basically unheard of): It does matter in a certain way, because it is very telling regarding what kind of people they are.
Of course they do. People are all acting like assaulting/filming women/children is suddenly legal when Trans people are allowed to enter the bathroom.
Hint to all those people: It is still illegal. Men who pretend to be trans in order to gain access to a bathroom could still get reported and would still get punished if they did something in there which they should not.