I have a theory that they had a sort of harem society, in which they all breed with the current Gerudo male (Ganondorf) but with him getting sealed in the Sacred Realm, they had to have Hylian mates to survive, resulting in Hylians with red hair like Malon (if you talk to Talon with the Gerudo Mask he says it looks like his wife) and Gerudos with Hylian ears (like Telma) but in the long run their blood line ran thin and they eventually became extinct. And the Kokiri? Well they are immortal, and we never see one die, so I don't think they even need to reproduce if they're around forever.

__________________
It's the end of the world! Do something!