After a little more research I found batteries are aroud $300 / kg,
giving about 150 Wh / kg. This seems to be for quite a few different
brands so I would imagine that is a reasonable baseline price. This
doesn't take into account chargers or controllers for the power source.

What type of HG wing are you trying to power, and what type of harness
are you modifying? Will they accept a power-unit ?

Sorry I was a bit unlcear in my initial post. I am not actually in the
process of building this, just throwing it out as a hypothetical (at
the moment).

The HG wing would just be a typical one for these calculations. Same
with harness weight. Modifying a harness to add an engine on the back
would be an issue, i agree.

Will the purpose of the power unit be to take off and climb out, or
just as a sustainer between widely spread thermals?

Just for self-launch..

Why electric and not gasoline? Site restrictions? Noise? Objections
from other HG pilots?

Yes noise is an issue. I also feel 'solid-state' type machines like
electric engines are more reliable than petrol, due to only a single
moving part. It would also mean the fuel source could be spread out in
some interesting ways over the aircraft.

So to restate some of the calculations using the new values:

One kg of batteries will give you 150 W for an hour of use. Keeping a
250 pound hang glider fly level needs about 2000 W. You could fly
level for about 5 mins with 1 kg of battery for $300. Each kg would
give you an extra 5 mins of level time.

Don't batteries seem appealing these days with those numbers?
Especially for small engines like hang glider ones as you only need
them for a short amount of time, ie. to get equivalent height of a
cliff launch but from your paddock.