I think it was some sort of time-portal thing. Somehow the Temple of Time got destroyed, and then it was restored through magic, or some other force. The Temple no longer exists in the material world, but it IS in some magical other dimension or something like that.