The alt-right and SJWs have made this issue so hard to voice an opinion on. If you say anything that favors one side or the other you automatically get attached to everything each side says or does.
First of all, the racism and xenophobia of Japan is being blown up to ridiculous levels in this thread. It's being portrayed as the American south from 150 years ago. It's silly.
Japan is completely justified to have concerns about refugees and immigrants from the middle east. Just like the USA is. That's why the legal process for entering the USA is so damn tough and long. The Japanese see how difficult it is for them to integrate properly (especially the young males).