What should I do?

So I was invited to my friend and his wife's house for a sexual encounter. During this encounter I'm on my back while she is riding me and he is behind doggy style penetrating her ass. As we get into it I start feeling fingers on my asshole. It's my friend the guy trying to finger me. I have been with a guy before and he knows I'm bi but there both strait. After a while she goes to the bathroom and outside to have a smoke break and her husband stays in the room with me and ask if I would come back after she goes to work the next morning because he has been wanting to try having sex with me. So at this point I figured I would let him since we just had a 3sum, I could let him play a fantasy out, his wife wouldn't know and everything would go back to normal. Well a year later we have went from trying it out for him to me going to his house almost every morning after his wife leaves. During sex we do anything from just sex to get off to him saying how much he loves having sex with me and being more emotional. At this point I feel bad for her not knowing but I really enjoy it and we still have 3 sums every few months together. What should I do? Should I stop having sex with him? Or just keep going like it is?