If this is the case, that would make the Skull Kid perhaps hundreds of years old - it takes a few generations for something as grand as guardian gods to be forgotten, after all. Who knows?
Most of the humanoid people in the Zelda series - in particular, the Hylians who have died out in ALTTP and TWW, while they are still prominent in OOT - seem to be of elven descent. If this is true, they would likely have long lifespans.
As for the Skull Kid, perhaps it's a curse of immortality. Skull Kids, like the Kokiri, don't seem to grow up - in OOT, if you return to the Lost Woods as an adult and go to the Skull Kid BEFORE you begin the Biggoron's Sword quest, Skull Kid will still be there, and he hasn't aged a bit, although he seems to be terrified of adults.

Now that you mention the Deku Butler, there's something else regarding his estranged son. As you all well know, the Skull Kid changes Link into a Deku Scrub at the beginning of the game. However, as is evident with the Goron and Zora masks, these masks contain the unhealed souls of the individuals they resemble. So where did the Deku soul come from?
As you make Link makes his way to Termina in his new Deku body, he passes a withered tree that has a face - Tatl remarks that it looks as though it had been crying.
Once Link has defeated Odolwa and returned the Deku Princess to the palace, the Deku Butler, after leading you deep into the palace, remarks that Link looks remarkably like his estranged son when he wears the Deku mask.
At the games end, when the credits are nearly over, we see the Deku Butler on his knees in front of the tree in the passage to Termina. He appears to be crying.
Could it be that this is all that remains of his son? If so, that coulf mean that the Skull Kid extracted his soul to transform Link. This, in my eyes, is one of the most evil, twisted things that the Skull Kid has done.
I couldn't have been the only person that noticed this detail, surely. Anyone else agree?