In OoT, Link's father dies in a fire (don't quote me on this, but I believe he was a soldier fighting for Daphnes before he became King. I remember reading something like that... somewhere), and his mother is mortally wounded. Link, at this time I believe, is still only a few months old. She flees to the Kokiri forest and pleas to the Deku Tree with her dieing breath to raise her son. The Deku Tree senses that the boy is to play a very important role in Hyrule's future, so he agrees. Children don't usually start retaining much memory until they are about two-and-a-half to three years of age, plus Link didn't even know he was a Hylian, so he probably doesn't remember anything about them.
Wind Waker Link may remember his parents. They did live long enough to produce another child, Aryll. And I'm sure their grandma tells them stories about their parents, so WW Link may have some memories to hang on to.