All I will say is; learn to control your urges.
We are all humans, we have needs, we feel urges, we have the ability to control our feelings in favor of <insert something you believe in, love, companionship, faith, trust, e.g.> If you want to avoid feeling guilt then think before you act, otherwise you have to live with your mistakes, do what you see fit but keep in mind things may come to bite you in the ass so keeping quiet isn't always smart either, face your lover and prove you are good enough for them, if you hide things and act in secrecy then it won't do any good to your relationship and will most likely break down anyway. Don't let your actions harm the other person, let them at least go on with their lives...