Tell me what if anything will happen if I close this credit card before the annual fee starts up?

I opened an American Express Delta SkyMiles credit card about 4 months ago. It has an approx. $150 annual fee starting on year no. 2. I got a $500 credit about 2 months ago from a flight I booked that was pricey about 3 months ago. I also have gotten a few checked bags taken on the plane free using this credit card (as long as you fly with Delta of course). I do not plan on traveling really much over the next few years. I would like to know if I cancel this particular credit card if I will then have to pay back the $500 credit or anything? I just do not want to be billed the $150 annual fee so am thinking of cancelling in the next few months.

And for the record are there any credit cards (if so name them please) that if they have an annual fee and you cancel the card a few weeks before the annual fees they will still charge you the annual fee and/or take the credits they gave you?