Yeah I did forget. Other comments reminded me of that but I won't edit my post cus it won't make sense to new readers.
Personally I'm not entirely sure if the Legion are interested in time travel... There is one Legion, no alternate timeline Legions. They reside in the Twisting Nether where they have an overview of all the timelines that they want and go to any location at any point in time in any timeline that they want. They don't need the Caverns of Time.