Well the syndicate are annoying, and there is little incentive to work for them. But I personally find Ravenholdt the most annoying. Lore-wise, they are a bunch of smug ninjas and grinding their reputation offers little in the way of incentives and rewards. Also the higher reputation you gain, the more the guards start to recognize you, including insulting you for being in the lower orders.