Good ol "36 vs 40/41mm" question

Having small wrists, anyone think this is some sort of"big thing" they going with a 40mm? I have small wrists and I think the 36 MM is fine. I just don't not feel like going to the AD, I'm not exactly in a primary area for Rolex but over the bridge is a different story in Westchester county

I know there are more options with a 36MM but was just wondering if anybody here perhaps has both and prefers one over the other?

Shot in the dark question and I know I'll be shitted on for asking this question