There's three sides to Majora's Mask: the people who utterly love it, the people who dislike it, and the people who are indifferent. After my research on the Internet, I have deduced that roughly 28% of all Zelda fans have it as their favorite Zelda, 18% of all Zelda fans dislike it, and the rest disregard it or don't have it as their favorite Zelda.
That's a pretty big statistic, if you ask a sane human.
Pretty sure MM is at least second place. It recently got really popular, not to mention one of the devs (as I have said many times before) took a survey asking people's favorite Zelda games. MM had the most mentions.
I used to hate MM when I was little because I didn't like the idea of a time limit in my games. That changed when I decided to actually play the game and get immersed in it. The game went from being my most hated Zelda to being my favorite. From my personal experience, I think that this game is disliked by a lot (not all) of people because they want Ocarina of time 2 and never give Majora a fair chance
For me, it's the darkness in the game. There's a palpable sense of doom and sadness; the atmosphere is simply mystifying. I also like how everything you do has consequences, as you can see using the Bomber's Notebook. If you do one thing, it opens up some sidequests and closes down others. In that way, it's more realistic than OoT, because people act differently depending on the day, and you can influence them depending on what you do.
It's hard to exactly say why I prefer MM to OoT, but the atmosphere of the game has a lot to do with it. And I think that's why so many people like it: it's a Zelda game with a darker story and more maturity. And the three-day system is so unique and intriguing; it's one of those games with an experience you can't get anywhere else.