Don't overlook Nantucket Island. You'll need to get a cab into town.

If you fly to the P Town Airport on Cape Cod it's about a mile walk to
P Town itself: that's an interesting place to visit as well, and easy
walking. Rent a bike, ride on the paved bike paths that wind thru the
dunes and go down to the National Seashore.

Of the various choices in the area, P Town would be high on my list.