That's a big part of it for me, for sure. I have two reasons I don't like no flying. The first is fairly objective, whether a person agrees it's a bad or good thing, it's an objective fact that without flying, it takes longer to do things in game. And I don't have the same amount of free time I had 10 years ago. So when I have to spend a lot more time on things that do not contribute to my in game goals, that makes me frustrated and unhappy, and I feel like my time is being wasted.
But then the second is exactly what you said. It's more subjective. It's about the feeling of the game and the world, and to me that is diminished by the lack of flight. The loss of being able to see the world from all angles and views. The feeling that my character is less of a hero, is weaker, can no longer do something it used to be able to do while random NPCs and flightmasters still can, for no story related reason whatsoever. It harms immersion for me when a mount with wings can't fly. It kills the feeling of this being a consistent world. It nags at my brain anytime I play because it just feels *wrong.* It just doesn't feel like the same game.