Unfortunately, one can't expect an narcissist to play fair, even if you have been more than fair with them. Particulary if you are dealing with a psychopath/sadist narcissist, I'm not really certain why you might have even expected him to work with you, and trusted in this enough to have begun to make your own plans on his weekend with the kids, thinking it would be fine.
Sadly, you cannot expect this in the future, and it is best not to assume that he might not mind, or might want a break, or this or that. They are completely unpredictable, and have no empathy, which is needed to have the ability to reciprocate things.
Though, that aside, even in non narcissistic cases, I'm sure the same scenario has been played out by separate or divorced couples, if there is bad feeling between them.
Really, it was his weekend and it seems that you planned something on it without telling him right off the bat either (if I have that right -perhaps I don't) based on your assmuption only, that it would be okay because he'd want a break.
I'm not sticking up for a narcissist by any means, but just looking at the situation of two people, in and of itself.
I wouldn't needlessly escalate things (particularly with psychopath/sadist narcissist) with the restraining order comment.
I would just stick to the rules in the future, and not assume any leeway from him. On your part, you can decide what kind of person you want to be in terms of working things out with him if he asks. You could continue to if it really wouldn't be a bother to you at the time he asks, or you certainly don't have to.
I guess you could consider what the kids might appreciate best at those times.
Sorry it worked out that way for you...
Take care.
