I just did the Belfry part on a new character this past weekend (I had an urge to play). The guys that were invading were really bad. I don't know if the good PvPers have moved on, OR if they just stick to the Arenas...but even 2v1'ing was laughably easy.
ALSO, in SotFS, the 2nd Floor Belfry has a summon spot for an NPC with a ridiculously high health pool. The summon sign is under the stairs. If you are fearful of being invaded, summon that guy. You can dash to the spot and summon well before an Invader can connect to your world.