I know I have a massive advantage in terms of doing this quest because of gear but even at a basically NM gear level of around 525/530 it should be pretty straight forward.
You can reach this part of the quest by solely using Raid Finder. Sure, you will have some valor gear at that point, but the difference in gear might still be relatively big, depending on your luck with the runestone drops etc.
With a 510 ilvl or something like that, he definitely can be challenging.
Hit him with a SW: P and stand near the entrance of the temple. He'll come to you. As soon as he gets to you, speed boost all the way to the edge of the balcony on the other side of Yu'lon. Sometimes this will glitch the adds so they don't follow you. Then just fight him normally up on the balcony until he eventually does he mirror images. Stay on the balcony while he finishes this attack. (It can take awhile, just be patient.) If the adds find you, they move pretty slowly and they're all coming from one direction so just DoT em up while avoiding his attacks. They should all die up on the balcony. Then just move back to the entrance of the temple and continue fighting down there. When the adds wake up, DoT em up as they near you and they should die towards the entrance. Move back up to the balcony, rinse, repeat.