IDK, it just seems more likely that Link just left everything after
TP. I'll bet that he just moved on to a different land/village, or continued to purge the land of evil. I can pretty easily picture him staying, and if he did, I think he would choose Ilia, but after what happened, it seems like he wouldn't want to be part of his old (or new for that matter) life.
If you watch the credits when he returns to Ordon, he seems to be a bit of an outcast. It could just be that no one cares about him as all of the children have just returned, but it seems to be a bit more.