""weather vs. flight" ratio for several years, and it is indeed a rare VFR
flight that is cancelled because of conditions that I would feel comfortable
with flying IFR."

Jay,

As you just found out, weather many times has nothing to do with filing IFR.
It is just far simpler to file IFR and let the controllers worry about the
airspace ahead.

Usually in any kind of high density area, east coast, Los Angeles, Etc. IFR
is just way easier, faster, more relaxing and safer. Even in severe clear!

Karl