But it's literally not an act of aspulling. It's right there in every lore text and every book concerning Night Elf history. Zin-Azshari was the capital city of the Kaldorei Empire. It's not a ret-con, it's not fan-fic, and it's not a stretch. It's where the majority of them lived and where much of their history played out.
They then relocated to Mount Hyjal, and because the destruction of the continent separated them from basically everything but modern Kalimdor, that's where they remained and further developed their culture into something more druidic.