Hi,
My situation is different - I am the wife of a N who had affairs.
From what I know, my H totally lied to the other women. The first OW I found out about broke the physical affair off when she came to her senses. They tried to remain "friends" or keep a "professional relationship" (she worked for him) but he could not respect her boundaries and kept pushing her for an inappropriate relationship. She finally cut him off and then his attitude toward her changed completely - his one-time perfect soulmate became "psychobitch". He then spread misinformation about her among their mutual friends, and harassed her via telephone and email for several weeks. She is quite afraid of him now.
Since then he has moved on to at least one more inappropriate relationship. He seems to try and pick women who are susceptible to his charm and lies, maybe going through some personal difficulties that make them more vulnerable at that time. Maybe this was the case with you? How do you know that your N is not stringing other women along, too?
You can't remain friends if he is truly N, and even if he isn't, if you want to keep your marriage, staying friends is a dangerous idea. I am sure that you are at heart a good person, and that you were manipulated by him. It sounds like he is just using you - why would you want to remain friends anyway? You need to protect yourself!
Good luck.
Pandora