Yes, the higher speed for the same L/D at a higher (ballasted) weight creates higher reynolds number flow and changes the amount of laminar flow on the wing (it increases). Look at the flight tests on the PIK-20B and Nimbus 2 by Dick Johnson. Both showed an apparent increase in L/D max when flown with ballast.

Steve Leonard