clamped, the drill press, or mill is running very slow, and you use a
good cutting lubricant.

One of the best lubes for cutting aluminum is WD-40. I've heard ATF works
well too, but I've never tried it.