Things to use against you in your original statement:
1: "based very much on the subjectivity of "safe" and not on any clear
cut violation"
If saftey is just subjective to you, then maybe you where in
violation.
2: "since I fly a lot and use my plane for business"
Sounds like your not a commercial pilot but being paid to fly.
CFI maybe? What unsafe things are you teaching?
3: "violation on my record if I decide to get a flying job in the
future?"
Quite possibly you've already applied for a job and this has come
up before.