AITA for calling off the engagement after my fiance kept saying I will "give him a baby" once we're married?

My fiance (31M) and I (25F) have been together for 2 years, and engaged for six months. We've both wanted kids at some point, but never set a specific timeline.

Lately though, he's been making comments about how I'll "give him a baby" once we're married. The first time I let it go but when he said it another time I joked back "So that's my job now?" and he just said "Yeah, you're the one making it."

I told him that the way he was wording it was rubbing me the wrong way, and he rolled his eyes and said I was overthinking it. But he said it like that a couple more times later. I started to feel less excited about starting a family.

I told him straight up that it was making me uncomfortable after he said it like that again, later. He laughed and said "It's not that deep, that's just how it works." And in that moment, I was starting to feel done.

So I called off the engagement. He said I was being ridiculous over "a poor choice of words." His family got involved and is telling me that I misunderstood him and that he just meant he was excited to start a family with me.

I'm wondering if I overreacted. AITA?