I also go through this. I think it is because any decison I make is cause for me to be labled as; selfish, controling, insensitive, rebellious, etc.
It has only been when there are other people involved for accountability, or when I am willing to endure the accusations that I have been able to make decisions in the face of my N, and then always with the payment that is required of listening to him tell me over and over again, in various ways and sometimes even years later why I was so unthoughtful to have shown any personal resolve.
Make the decisions. If you do, the only thing you have to lose is the voice of the N telling you that you are wrong, but if you don't you will lose so very much... the worst being your very SELF.
Take care!
Elizabeth