The only way to permanently banish them is to destroy them in their own realm.
They will definitely soon be back. Once N'Zoth surfaces to Azeroth and make his presence physically known, then the other old gods will follow.
Do you know why old gods aren't randomly hurled at a random spot in Azeroth but they precisely seek out those sweet spots?
The well of Eternity is just a portal. In it's latent state it is a pool of arcane energies that grants bonuses to those who drink from it, use, it or reside nears it.
But do you know old gods are like the burning legion? They will respawn and return not until you make them manifest physically then kill them in their own realm
Remember the lines: I return to Nya'lotha?
It is the Black Empire's Twisting Nether and is connected to the rift of Aln.