Once it starts moving, even slowly, the wing is producing lift. The
more lift is produced, the more difficult it is to cause a wing to
drop, as this lift distributed over the whole wingspan damps a tendency
to roll.

Eliminate this damping by opening the spoilers and now the feeble
aileron is able to cause a roll.

Try this experiment: On a day with a light wind, sit in the glider
while it's pointed into the wind. Starting from a wings level
position, start rocking the wings. With the spoilers open, you will
find that you are able to go steeper and make quicker reversals. If
the wind is very light, it may all you can do just to keep the wings
level.

-Tom