Destiny's Edge (the group you're talking about) never actually defeated a dragon -- they fought dragon champions composing of an undead sea captain, a destroyer, and a telepathic ice creature. All of which while powerful, are like insects in both power and size to an elder dragon.
Destinty's Edge spoiler:
In fact, when they did try to take on the elder dragon Kralkatorrik with the help of Glint -- not only a dragon herself but Kralkatorrik's champion though she rebelled -- it ended in disaster. And that was with Snaff subduing Kralkatorrik via arcane crystals. Without him or another Asura capable of that, taking down elder dragons got a whole lot harder.
The truce between charr and humans is tenuous at best. There's centuries of vitriol and body counts are high on both sides, so even now there is very little trust on either side. There's quite a few members on both sides who think they don't need help to take on the elder dragons, or want to wipe out the other race even if it means not standing a chance to the dragons, just for the sake of revenge.
And the asura weren't pushed out by one dragon per se, it was the minions of Primordus, the destroyers who did that. When Primordus awoke, his minions awoke with him and rampaged throughout the Depths of Tyria. The Asura had built closely to the sleeping Primordus since he emitted magical energy, so when he awoke there was very little warning for them. Most of the Asura ended dead or cut off, the dwarves were massacred before sacrificing themselves to delay the destroyers and stand watch, and a number of other species such as the skritt were pushed up to the surface.
The elder dragons are very, very powerful but they aren't invincible. Kralkatorrik was close to being taken down and Jormag had one of his teeth removed by a norn. But that was when both were newly awakened so it will take a whole lot more to kill them now. It's not impossible, but it will be one hell of a fight