Forgive my ignorance, but if there is VFR traffic in the pattern, why
would cancelling IFR not be permitted?

Ops specs? VFR traffic is illegally flying in what is really IFR
conditions (not likely for a towered airport though)? Funny airspace
(such as the FRZ or DC ADIZ, where cancelling might be problematic,
though it might not be)

Jose
--
The price of freedom is... well... freedom.
for Email, make the obvious change in the address.