Another Venus Saturn issue
What do you do with clingy people? Yeah, I hear this.
You have Saturn (responsibility) tied up with Venus (love) in your chart. I’m guessing you feel responsible for him. I’m also guessing he reinforces this erroneous idea and in effect, restricts you but it’s not real.