If you can identify what shaped your original desire to help people who want to use you, that might clarify the situation for you. If you were conditioned to feel selfish if you thought about your own needs or by extension your family instead of N or uncaring etc etc, you might want to try practicing saying to yourself, I do care but the best help I can give this person is the opportunity to help himself.
You may also be accepting this person's implication that you are responsible for him. You may have to shake yourself out of it. Some people spray words like tear gas and while you are crying your eyes out, they're stealing your money, loved ones, or your very persona.
Just get real about what's yours and what's his. You may need to write it down and post it in your wallet, on your phone, on your bathroom mirror until you internalize it. Good luck to you.