The fate of the High Elves was decided by a human. A human by the name of Arthas Menethil. While Garithos was definitely a racist and was a major factor in Kael becoming disenfranchised with the Alliance, he was not the reason they joined the Horde.
Garithos' actions made Kael accept aid from Lasy Vashj and the Naga, and eventually Illidan. This is circa Warcraft 3: TFT.
The reason they join the Horde plays out in The Burning Crusade, where the majoriy of Blood Elves fold into the Horde via relations to Sylvanas and her Forsaken, as she was ranking Ranger General at the time of the invasion, and helps them find a home with the Horde.
The scattered High Elves who either weren't in Quel'thalas at the time of the invasion, or those who didn't go along with the Blood Elves direction, were scattered to the wind, with the only 'major' group being a contingent of Dalaran High Elves in the Silver Covenant. Either way, High Elves as a greater society was eliminated.
Eventually we find in Outland that Kael has turned to power mongering and dealing directly with the Legion, and so the Blood Elves kill him as part of a joint venture with the draenei in the Shattered Sun Offensive, and remove the Legion attack on the Sunwell, eventually leading to its holy fueled reignition.
To try to boil it all down to Garithos and Kael were bad and that's why Alliance lost High Elves is a disservice to over a decade of storytelling across multiple games.