Dark Hyrule Castle was the most annoying for me. You have to fight three Darknuts at once, and if you don't beat them quick, then Vaati drains Zelda's life and it's game over. I was so close to beating the game without dying on my first try! That just stinks! Then, I have to go and fight Vaati twice with no time to heal.

__________________
BA characters: [Selena] [Geretta] [Latexah]
My timeline:
TMC-FS-Child OoT-MM-TP
TMC-FS-Adult OoT-TWW-PH-FSA-ALttP-LA-LoZ-AoL-OoA/OoS