That was in the letter I wrote to complain. The (written) response
was "you have to follow all instructions the controller gives you".

To which you should have returned: "What regulation states that a VFR
aircraft in class E (which I presume you were in) airspace is required
to follow all instructions given by the controller?

Well, I look at it this way. I may have been stupid enough to have
gotten into a ****ing contest with the FAA in the first place, but at
least I was smart enough to quit while I still had some dry clothes
on.

Wise decision, Grasshoppa.