Well creeper's are far more dangerous and are SILENT until they're right behind you! They make me run and jump in shock and all. I hate them ambushing me! Endermen are cool though, I don't mind them and they don't mind me, we can chill together all we want and no one will explode.
Endermen are much creepier, but Creepers are more scary practically speaking. Endermen are just slightly more annoying zombies.
Neither of them compare to how threatening a Skeleton is on Hard, though, especially if you aren't on level, open ground and don't have a bow to shoot back with.
At range or groups: Skeles. My first experience with friends on a private server ended up in us unwittingly mining our way into a 3 dungeon 'mega dungeon' with all Skeles. Nightmarishly hilarious amounts of dying ensued until we got our loot.
Solo: Creepers. Silent an they pack a punch. Endermen, the few I've seen/killed (prolly no more than 5 total) seem to be just teleporting Zombie with a stickman figure that (until 1.9) carry random blocks. Not very scary. SSSssssSSSS BOOM! is.....
Skeletons are underrated.
Sorry for my bad english skills