Someone may have said something similar on previous post for I have not read all.
Looking at current sports in the Olympics (
http://www.topendsports.com/events/summer/sports/) it seems to me the one thing they have in common is spectators. Maybe not televised but in the stands. IMHO, I think the committee looks at what will bring revenue into the host site. There have been great strides in helping make soaring better for spectators with tracking but until sailplane racing gets more into every household and becomes an event that can be watched and cashed in on, its a long shot to be an Olympic sport.