You don’t have to be a doormat. If someone hurts you, it is okay to let them know. It is entirely possible that the intent was not to hurt you, and that is good for you to know. And it is also good to to let them know how their words or actions affected you, as they may not realize it. Face it, a hurt will still hurt, regardless of the intent.