FLARM in Stealth Mode at US 15M/Standard Nationals - Loved It!

9B, that's a really excellent point about ADS-B. Flarm Stealth mode may be a completely moot point in 5 years. Thanks for pointing it out.

-John, Q3

On Thursday, August 6, 2015 at 2:41:21 PM UTC-4, Andy Blackburn wrote:
Ultimately, FAA action notwithstanding, you will see gliders equipped with ADS-B Out. These will offer ranges of many tens of miles, not the 3-5 miles offered by Flarm. All of that traffic information will be picked up by your PowerFLARM and go straight to your flight display and will include gliders as well as any power traffic equipped with ADS-B 1090ES. There's no stealth mode for ADS-B and ADS-B doesn't discriminate aircraft type. It does, I believe, transmit an ICAO code.

I'm guessing we don't want to be in the position of requiring pilots to turn off their ADS-B gear or banning its installation. You might be able to filter out glider ICAO codes using some sort of registration database like FlarmNet (not a current Flarm capability), but that starts to get pretty complicated and I'd bet the FAA would have something to say about it.

Not an immediate issue, but 5 years from now it could be very real.

9B

On Thursday, August 6, 2015 at 3:36:28 PM UTC-4, John Cochrane wrote:
One thing to keep in mind: Pilots can always use stealth mode on their own. If it bugs you that someone may follow you, then turn on stealth. The issue is mandating it for all.

John Cochrane BB

If you are in Stealth, and your competitors are not, you are strongly disadvantaged from my experience. I stubbornly tried it. If the Flarmleechtool is in use, one can not afford not to be using it.
And- dirty little secret- pilots are covering their antennas when they think they are in the lead so as to not give away information. One well known world champion has been doing this, according to those I believe are in the know. "Whats the aluminum foil hat for?"
UH

Flying a regionals competition last week in the UK, with my regular Flarm in open mode, even with my bat eyes I spotted gaggles or single thermalling gliders before Flarm told me about them. The best help I had was one day when I had a friend with eagle eyes in the back seat. And yet I believe that I was visible to ground tracking stations most of the time.

The main lesson that I draw from this discussion is that we Europeans should be upgrading to Powerflarm!

But from a flight safety POV I am quite happy not to be told about traffic beyond 2 km or 300 m from my position; that is the way it is for me at the moment anyway and seems appropriate for a competition.

However I don't like the idea of relative altitude being displayed with noise even if another glider is quite close. I want to be able to visually locate a closer glider as quickly as possible.

On Thursday, August 6, 2015 at 3:32:48 PM UTC-4, ND wrote:
On Wednesday, August 5, 2015 at 11:06:24 PM UTC-4, jfitch wrote:
On Wednesday, August 5, 2015 at 7:50:45 PM UTC-7, XC wrote:
I've seen a pilot in a Regional who didn't practice all year and never was very good locate a top competitor pre-start using FLARM, follow him around the course and win the day in a FAI handicapped class because he was flying a 15m glider and the top competitor was flying an 18m glider. I started with the two of them split off and did my own thing.

XC

I have to agree with JC. In fact I would prefer that the range be greatly increased. If I get a Flarm warning that I was not expecting, I consider that a failure of situational awareness.

The story about a mediocre pilot in a 15m following a top pilot in an 18 meter is quite far fetched - at least where I fly pretty much impossible as described. If performance in this sport comes down to simply aping another's actions, then it isn't quite the skill sport that we thought.

False. I've seen it too. i've seen mediocre pilots post excellent speeds because they mimed an excellent pilot's decision making. does that mean he had the right stuff? no. it means he copied someone's homework. it IS the skill sport we think it is. it's why Jerzy can go faster than everyone else by 5-10 MPH more often than seems possible. because to win over and over, tremendous skill is required.

From my point of view (quite a while out of competition, but did compete against a lot of top US pilots before....), just because you "know" a "great pilot is doing something (like diving for the trees to scrape off leeches), YOU, still have to find the thermal, center it & climb out.
;-)

I lost a good place at Newcastle, VA (when all I had to do was get home) by deciding to follow another pilot back out (I had final glide made at ~8:1 in an ASW-20 before I headed back out) hoping I could garner an even better spot.

I landed out.
:-(

Yes, leeching can help at times, but being in the "right place at the right time" does NOT mean you'll get back up.

US glider racing has a very big problem right now -- and it definitely isn't leeching! It is declining participation, fewer contest being offered, contest that are so small that they can't accommodate the several glider classes. Every change that's contemplated now should be viewed through that lens first.

It's perplexing to me that we're experiencing such a decline. The fact is that racing is really fun. But it's even more fun when you know about the presence of gliders around you and especially when you know who's in those gliders. I was certainly disappointed to read that stealth had been mandated in NY. If it were entirely my choice, I'd disallow stealth mode and mandate contest ID association.

For me, the fun factor of PowerFlarm has been a great side benefit to the outstanding safety benefit that it accords. It's just dumb to trash that.

On Thursday, August 6, 2015 at 7:40:21 AM UTC-7, Andy Blackburn wrote:

We now have at least one contest we can compare the results from to see if pilots scored significantly differently with stealth vs not - there is normally a lot of variance in the scoring even over a long contest so it may or may not show anything but at least we will have an indication if anything significant is going on.


I ran some numbers to test the theory that less skilled pilots use Flarm as a way to up their performance - and that Stealth mode helps negate this strategy. I compare the last two 15 Meter Nationals - 2014 at Montague (Flarm without Stealth Mode) and 2015 at Harris Hill (Flarm with Stealth Mode). I compared each pilot's performance as measured by final score as a percentage of winner's score to their PRL percentage, which is the longer-term average of the same metric and is used for US Team selection so it's the best available measure of overall pilot skill. The metric I used was average absolute error for each contest - on average how closely did contest performance match PRL percentage. I dropped from the analysis any pilots who withdrew from either contest.

The theory we are testing is that with unrestricted Flarm you should get more inexperienced pilots higher in the scoresheet with higher scores and potentially some of the more highly skilled pilots dropping down as a result.

The average absolute error for the 2014 15-Meter Nationals at Montague was 7.4%. That is, on average pilots scored only 7.4% off of their PRL. Four pilots were competing in Standard Class gliders at Montague without handicaps.. If I give them each a 4% handicap the average absolute error drops to 7.1%.

The 2015 15-Meter Nationals at Harris Hill had an average absolute error of 11.8%. That a significant difference and not in the direction you'd expect.. It turns out that at Harris Hill a number of highly ranked pilots under-performed their PRL and a few lower-ranked pilots out-performed their PRL.

This is 35 pilots flying a total of 259 contest flights, so it's not a super big sample size, but not terrible given the stark difference. I also looked for any potential sources of sample bias - average pilot skill, contest length or devalued days. The two contests were pretty similar. Average PRL for Montague was .899 and Harris Hill was .906. Montague was 8 days and Harris Hill was 7 days. The average daily winner's score at Montague was 869 points and Harris Hill was 844 points. Not perfect, but pretty close.

So, if the theory is that Flarm allows lower skilled pilots to leech their way to the top of the score sheet and Stealth mode puts a lid on this behavior, the data suggests the opposite. In fact a number of long-time (30-40 years) highly ranked pilots under-performed their PRL with stealth mode. One potential reason for this is that more experienced pilots are actually better at making use of Flarm information than inexperienced pilots. The other theory is that Flarm helps reduce the random landouts that tend to scramble the scoresheet and actually reinforces stability and order.

Discuss.

9B

P.S. I already said that it's only 259 data points.

On Thursday, August 6, 2015 at 8:28:24 PM UTC-4, Andy Blackburn wrote:
On Thursday, August 6, 2015 at 7:40:21 AM UTC-7, Andy Blackburn wrote:

We now have at least one contest we can compare the results from to see if pilots scored significantly differently with stealth vs not - there is normally a lot of variance in the scoring even over a long contest so it may or may not show anything but at least we will have an indication if anything significant is going on.


I ran some numbers to test the theory that less skilled pilots use Flarm as a way to up their performance - and that Stealth mode helps negate this strategy. I compare the last two 15 Meter Nationals - 2014 at Montague (Flarm without Stealth Mode) and 2015 at Harris Hill (Flarm with Stealth Mode). I compared each pilot's performance as measured by final score as a percentage of winner's score to their PRL percentage, which is the longer-term average of the same metric and is used for US Team selection so it's the best available measure of overall pilot skill. The metric I used was average absolute error for each contest - on average how closely did contest performance match PRL percentage. I dropped from the analysis any pilots who withdrew from either contest.

The theory we are testing is that with unrestricted Flarm you should get more inexperienced pilots higher in the scoresheet with higher scores and potentially some of the more highly skilled pilots dropping down as a result..

The average absolute error for the 2014 15-Meter Nationals at Montague was 7.4%. That is, on average pilots scored only 7.4% off of their PRL. Four pilots were competing in Standard Class gliders at Montague without handicaps. If I give them each a 4% handicap the average absolute error drops to 7..1%.

The 2015 15-Meter Nationals at Harris Hill had an average absolute error of 11.8%. That a significant difference and not in the direction you'd expect. It turns out that at Harris Hill a number of highly ranked pilots under-performed their PRL and a few lower-ranked pilots out-performed their PRL.

This is 35 pilots flying a total of 259 contest flights, so it's not a super big sample size, but not terrible given the stark difference. I also looked for any potential sources of sample bias - average pilot skill, contest length or devalued days. The two contests were pretty similar. Average PRL for Montague was .899 and Harris Hill was .906. Montague was 8 days and Harris Hill was 7 days. The average daily winner's score at Montague was 869 points and Harris Hill was 844 points. Not perfect, but pretty close.

So, if the theory is that Flarm allows lower skilled pilots to leech their way to the top of the score sheet and Stealth mode puts a lid on this behavior, the data suggests the opposite. In fact a number of long-time (30-40 years) highly ranked pilots under-performed their PRL with stealth mode. One potential reason for this is that more experienced pilots are actually better at making use of Flarm information than inexperienced pilots. The other theory is that Flarm helps reduce the random landouts that tend to scramble the scoresheet and actually reinforces stability and order.

Discuss.

9B

P.S. I already said that it's only 259 data points.

Weather variability is most likely a bigger factor than Flarm. To wit- SM, who never lands out got snookered multiple times at Elmira. Same with KS. Only 3 pilots completed all tasks assigned. I suspect the weather had a much bigger effect on performance variability from the expected than Stealth- or not- ever would.
It is also worth noting that FS, who has a lot of Flarm expertise relative to the rest of us, won 15M in Stealth. Would he have enhanced his advantage? I think so.
UH

On Thursday, August 6, 2015 at 5:40:12 PM UTC-7, wrote:
On Thursday, August 6, 2015 at 8:28:24 PM UTC-4, Andy Blackburn wrote:
On Thursday, August 6, 2015 at 7:40:21 AM UTC-7, Andy Blackburn wrote:

We now have at least one contest we can compare the results from to see if pilots scored significantly differently with stealth vs not - there is normally a lot of variance in the scoring even over a long contest so it may or may not show anything but at least we will have an indication if anything significant is going on.


I ran some numbers to test the theory that less skilled pilots use Flarm as a way to up their performance - and that Stealth mode helps negate this strategy. I compare the last two 15 Meter Nationals - 2014 at Montague (Flarm without Stealth Mode) and 2015 at Harris Hill (Flarm with Stealth Mode). I compared each pilot's performance as measured by final score as a percentage of winner's score to their PRL percentage, which is the longer-term average of the same metric and is used for US Team selection so it's the best available measure of overall pilot skill. The metric I used was average absolute error for each contest - on average how closely did contest performance match PRL percentage. I dropped from the analysis any pilots who withdrew from either contest.

The theory we are testing is that with unrestricted Flarm you should get more inexperienced pilots higher in the scoresheet with higher scores and potentially some of the more highly skilled pilots dropping down as a result.

The average absolute error for the 2014 15-Meter Nationals at Montague was 7.4%. That is, on average pilots scored only 7.4% off of their PRL. Four pilots were competing in Standard Class gliders at Montague without handicaps. If I give them each a 4% handicap the average absolute error drops to 7.1%.

The 2015 15-Meter Nationals at Harris Hill had an average absolute error of 11.8%. That a significant difference and not in the direction you'd expect. It turns out that at Harris Hill a number of highly ranked pilots under-performed their PRL and a few lower-ranked pilots out-performed their PRL.

This is 35 pilots flying a total of 259 contest flights, so it's not a super big sample size, but not terrible given the stark difference. I also looked for any potential sources of sample bias - average pilot skill, contest length or devalued days. The two contests were pretty similar. Average PRL for Montague was .899 and Harris Hill was .906. Montague was 8 days and Harris Hill was 7 days. The average daily winner's score at Montague was 869 points and Harris Hill was 844 points. Not perfect, but pretty close.

So, if the theory is that Flarm allows lower skilled pilots to leech their way to the top of the score sheet and Stealth mode puts a lid on this behavior, the data suggests the opposite. In fact a number of long-time (30-40 years) highly ranked pilots under-performed their PRL with stealth mode. One potential reason for this is that more experienced pilots are actually better at making use of Flarm information than inexperienced pilots. The other theory is that Flarm helps reduce the random landouts that tend to scramble the scoresheet and actually reinforces stability and order.

Discuss.

9B

P.S. I already said that it's only 259 data points.

Weather variability is most likely a bigger factor than Flarm. To wit- SM, who never lands out got snookered multiple times at Elmira. Same with KS. Only 3 pilots completed all tasks assigned. I suspect the weather had a much bigger effect on performance variability from the expected than Stealth- or not- ever would.
It is also worth noting that FS, who has a lot of Flarm expertise relative to the rest of us, won 15M in Stealth. Would he have enhanced his advantage? I think so.
UH

Well SM landed out on a 486 point day at Harris Hill and P7 landed out on a 850 point day at Montague (and scored even lower) so I'd say that was worse. Also, the landout day that caught the guys you mentioned also got the guy who won the contest, so I'd say that bit of weather variability didn't really scramble the field much and therefore wasn't much of a factor. Just to be sure, I took out the roughly 200 point benefit the finishers got over the landouts that day. The average standard error went from 11.8% to 12.9% - so it made it even worse.

There are always exceptional circumstances on any given day, but things tend to average out with enough days and the effects aren't as strong as people seem to remember.

The puzzling thing is that lower-ranked pilots out-performed with Stealth on and HH and despite a fair amount of weather randomness at Montague and a lot of low scratching in the mountains and blue holes - the middle ranked pilots weren't able to out-perform, so not only is the leeching theory not reflected in the data we have, but the data tilts a bit in the opposite direction - Stealth mode is actually associated with less consistent results.

Yes, only two contests - but if there was something seriously going wrong from using Flarm you'd not expect these results.

9B

I flew Montegue and loved it. Rex and Nowelle did a great job. If you haven't flown there go at the next opportunity!

There are several reasons why the numerical analysis presented has limited usefulness. I'll just mention two:

1) I don't believe most people last year at Montegue were using a complete FLARM view type set up. I flew the contest with FLARM (through ClearNav) as anti-collision only. This year many more pilots were outfitted with the whole set up.

2) The weather at Harris Hill was much more fickle and unforgiving than Montegue, no comparison. 7 or even 8 days were flyable but each had significant challenges that tripped up even the locals. The guys I would have picked to win all had problems, sometimes on the first leg. On Day 6 in 15m 14/21 pilots landed out on the first leg. Others chose well and scored big. This really blew the rankings out of the water but that was the reality of the weather that day.

Great time at both contests,
XC

On Thursday, August 6, 2015 at 12:32:48 PM UTC-7, ND wrote:
On Wednesday, August 5, 2015 at 11:06:24 PM UTC-4, jfitch wrote:
On Wednesday, August 5, 2015 at 7:50:45 PM UTC-7, XC wrote:
I've seen a pilot in a Regional who didn't practice all year and never was very good locate a top competitor pre-start using FLARM, follow him around the course and win the day in a FAI handicapped class because he was flying a 15m glider and the top competitor was flying an 18m glider. I started with the two of them split off and did my own thing.

XC

I have to agree with JC. In fact I would prefer that the range be greatly increased. If I get a Flarm warning that I was not expecting, I consider that a failure of situational awareness.

The story about a mediocre pilot in a 15m following a top pilot in an 18 meter is quite far fetched - at least where I fly pretty much impossible as described. If performance in this sport comes down to simply aping another's actions, then it isn't quite the skill sport that we thought.

False. I've seen it too. i've seen mediocre pilots post excellent speeds because they mimed an excellent pilot's decision making. does that mean he had the right stuff? no. it means he copied someone's homework. it IS the skill sport we think it is. it's why Jerzy can go faster than everyone else by 5-10 MPH more often than seems possible. because to win over and over, tremendous skill is required.

Well I did qualify that with "where I fly", which is out west. Perhaps true for the flatlands and hillocks - I don't know. But if Jerzy can go 5 - 10 MPH faster than everyone else, I'm doubting that a novice in a 15m will keep up with him in an 18 no matter how many Flarms are involved. Not where I fly, anyway.

There are many technologies that have changed the sport over the years. This one not as earth shattering as many others gone before. The purists among us really should be flying with wooden wings, Clark Y airfoils, and a pellet vario. But the point about ADS-B makes this almost an academic argument?