Ganondorf, a Gerudo, wielded powerful magic before getting the Triforce of Power in
OoT, just before Link gets the Ocarina of Time.
Then there's the two witches Koume and Kotake, obviously Gerudos (although very old, so no red/orange hair anymore) that wielded powerful magic.
Anyway, my point is not to prove that the Twili are related to the Gerudo, I merely wish to discuss why Zant has what appears to be very similar to the Gerudo mark on his clothes.
Then there's the Sheikah symbol on the back of Midna's mask. Odd, with the marks.