You're mistaking the meaning I intended with the usage of "theirs", it sounds like.
I'm not suggesting that they view it as "theirs" in the sense that it is controlled by or
belongs to High Elves, but as "theirs" in the sense that they've lived there for centuries (or perhaps even millennia) and as such have become completely naturalized into the culture. The same way it shouldn't be surprising that someone born and raised in Texas would be inculcated in the common culture therein, which could then be easily utilized to create a dichotomy of "us" and "them", with regards to people who are sufficiently Texan and people who aren't.
It's very likely that the Silver Covenant sees itself as an organization which specifically upholds the cultural values of Dalaran, and not the cultural values of the broader population which employs the demonym of High Elf. This is circumstantially corroborated by the Silver Covenant's proximity to Dalaran in virtually every confrontation, as well as by their continued interest and usage of magic (while pretty much all of the other clusters of High Elves either weaned themselves off of magic entirely, struggled with and ultimately found ways of coping with their addiction, or succumbed and either became Withered or corpses).
While on the subject of the broader population, I have to admit that I've never really been a fan of the idea of playable groups being implemented as cultural grab-bags (i.e. lumping a bunch of closely related groupings into a larger, broader categorization, like the Mag'har Orcs). This means that using all of these disparate groups (i.e. Highvale Elves, Allerian Stronghold Elves, etc.) to create some kind of hodge-podge identity doesn't really excite me -- especially when these isolated groupings are more-or-less culturally identical to modern Blood Elves but without access to the same resources, and the Silver Covenant represents the only objectively successful population of High Elves, and who have been defined as being culturally divergent (as opposed to just politically divergent, though they are that as well).