Often times when ATC isn't able to give you a release they will ask
you if you can depart VFR. Perhaps you should have called them back
and say that was your intention? My airport is just under the ILS for
a large cargo airport so its not unusual to have to wait some time
before getting a slot. However, ATC will always offer me that if I
depart VFR and fly 5 miles South they gave offer me the clearance in
the air. The only downside of this is that we don't have wx on my
field so I never know what the clouds heights are.

-Robert