I am transgender woman and I live with pretty much hetero man. I am still waiting in line to get sex-change surgery. My husband clearly wants to have sex with real woman and that's why we tried to find 3rd person / another couple to fulfill some desires but when I saw how exited and lustful he became, I could not do it. I've said to him soooooo many times that if he wants to be with real woman then tell about it and we can go our own ways. I cannot give him what he wants and I cannot bear to see him go so lustful when he has a chance to be with biological woman. So going in separate ways is best thing to do if one night with woman is what he wants. If we had 3rd person with us, I would be left aside to just watch... I could not live in polyamory relationship or have 3rd person with us randomly (specially because it most likely would become something that man wants everytime). I would feel like lesser being in that group.
It really annoys me that I am not able to do that because I don't want to keep him in the leash when it comes to the sexuality, because I know that I am not able to give him everything he wants, ever... If men really just think with their downstairs brain when they get a chance like that, then it's better to be selfish and say no to polyamory.