The EU immigration thing has fuck all to do with people from outside the EU entering the EU.
It's about movement within the EU.
Just try employing a non-EU citizen. It's a complete fucking ballache. There's a mountain of paperwork and fees involved.
The internal freedom of movement was to create a unified labour market, so that people with desirable skills could move to where the jobs in their field were. To make it easier to conduct business in the modern world, where some people might need to visit dozens of different cities every year as part of their job.
It's all very well scoffing at this from the US, where people can train in something in Boston and then get a job in California, then in their job go to Houston and New York on a semi-regular basis. Europe contains considerably more borders than North America, and the places you may need to visit may be on the other side of those borders.