One of the gossip stones in OoT said that sometimes Gerudos come to town looking for men. So...they come in and have some fun with hylian men, and return home.

So that would be where Ganondorf came from.

__________________
If the world is to be destroyed, so be it. If my fate is to die, I must simply laugh.