For the zilionth time the castle law does not come to this, his actions would be as justifiable in a duty to retreat state, since if someone is sitting on you and beating you up, you have nowhere to retreat to.
You do not have the right to "whoop" the but of anyone who is following you. You do realise that you can die from a single unlucky blow to the head right? I dont know about you, but if I were in his place I would not place my fate in his hands, hoping he would only beat me up...