No, Stardew Valley isn't "challenging", but I do consider it to give me a challenge to overcome. It's not a difficult challenge by any stretch, but it's still a baseline level of engagement that allows me to get immersed in the game instead of just breezing through it unhindered.
Navigating the zone on the ground in WoW isnt challenging in that sense either, but it does require some level of thought and planning, enough to make me engage with the game. It also gives actual moments of organic gameplay that I would not get with flying.
Flying takes all those moments away and replaces it with nothing. No challenge, no planning, no engagement, just visual noise on the screen where there should have been a game.