First of all, it seems the raidleader(s) of your former raidgroup are not able to be confrontational, which is imo a first requirement to lead well. Confrontational does not mean to shout and abuse, but to be able to issue criticism, to remove people from raids if needed etc. To let leaving be up to members means you wont get anywhere, since few people wish to leave a raid by themself - and just because people are willing to does not mean they are the ones that should.
Second, I dont get the whole drama. But that is up to you and yours - why should anyone (not) be able to talk to anyone? If you have a forum you could pose that question.
Last, I dont get why you are so protective to that woman that treats you badly. Friendships should be mutual... if it is not, let it go. If people are worthy of you they show so by defending you when needed. If they let you in the mud, it is time to move on. Loyalty is a great thing for building relations- up to a point.