I honestly don't find endermen to be scary. I was rather disappointed by them actually. They always either dissapear and never come back, or they get stuck.
Creepers are still the most horrible thing in the game.

Ghasts, and the nether in general still scare me...