I'm based in HPN, and we go over to BDR all the time to find some
empty pavement to land on. Why did it drive the guys in the tower
nuts?
We were keeping the tower in business.

I did the same thing when I was instructing at HPN as often times it
got too busy for ATC to work the T&G pattern with heavy airline
trafiic. At Bridgeport it wasn't so much the (then) endless T&G traffic
it was the absolutely clueless students, often with instructors
onboard, that busted the airspace and/or gave bad position reports
and/or screwed up the pattern entry instructions etc. I spent lots of
time hanging out in the tower with my pals and couldn't believe some of
what I'd seen & heard. The pilots that were ahead of the power curve
were never a hassle if they asked for a few circuits, it was the
students & the guys that fly 3 times a year that caused the headaches
for them.