That's not the complaint. The problem is 'Annatar' is clearly just the human guy in a wig. They could have gotten a different actor (which I believe news suggest they have for parts of it?) or done more to mask this guy clearly just being Halbrand in a wig.
Sauron is a shape changer. Him changing appearance makes sense, and his Halbrand disguise is already comprimised. There's no reason to continue resembling Halbrand.
But I guess missing the point is how one writes this show or defends it.