You can find old sets at Stormwind/Orgrimmar, Gadgetzan, Dalaran (in the sewers) and Area 52.
Area 52 has some BC-era sets
Dalaran and Gadgetzan have some BC, WotLK and Cataclysm sets.
Stormwind/Orgrimmar also have some BC, WotLK and Cataclysm sets. Some of those are the same as those in Dalaran/Gadgetzan, others are not.
All those sets cost Honor.
The faction elven vendors in Dalaran also have some nice sets to sell, but they are sold with Justice.
They are adding even more old sets (as well as old Pandaria sets) in 6.0 as well. Will be worth it checking out those locations again when the expansion hits.