According to
worldofwargraphs 37.2% of players earned at least one challenge skin, and 11.6% earned all skins on at least one class (which, again, doesn't mean they didn't get more than 3 spread among multiple alts). 62.8%, although it is a majority, is nowhere near "vast", and it includes a lot of people who never knew or didn't bother, which is very different from "can't do it".
You are right that it was accessible, which was a very good thing, but people who think it makes them special are grossly exaggerating. There's always a catch when we're talking accessibility and exclusivity. I'm willing to bet that if the tower challenges remained in game, even while going through faceroll periods depending on class buffs and scaling changes, completion rates would have been much, much lower, because many complacent players would have simply pushed doing their challenge further indefinitely, and many of us who pushed 36/36 would have stopped after a couple, or maybe even 1 skin.