wrote
However, there should be no surprises about what the applicant is
expected to do on a test. Otherwise it becomes a guessing game.

I think this is at the core of the dispute. I also don't agree with
your position, though I certainly understand where it comes from.

In aviation testing, we are used to the "no-surprises" testing
paradigm. We take written tests where all the questions are given in
advance. We take practical tests where all the tasks are spelled out
in advance. As a result, it is possible (and actually quite common)
for someone to get a certificate or rating without actually knowing
much (or anything) about how to use it, knowing only what is going to
be on the test. I think this is a huge problem, and unfortunately I
don't know how to resolve it.

There is a classic question that every teacher hears early in his
career, and keeps hearing. "Is this going to be on the test?"

I will say this - no teacher wants to hear it, and no real student
ever asks it. By real student I mean someone who is actually
interested in learning the material, rather than someone who just
wants a piece of paper. Unfortunately real students are a very small
minority in most schools. I would wager that isn't the case when it
comes to flight training - at least I never heard it from a flight or
skydiving student. Perhaps this is because I only train older,
professionally established students who aspire to use their airplanes
for travel and understand that the real test is not the checkride but
the real-life flying that follows.

So why this insistence on no-surprises testing? Well, it makes a
statement, and the statement is "I don't trust you to test fairly and
reasonably." Unfortunately, when it comes to the FAA the distrust is
reasonable and earned. I would be the first to object to the FAA not
publishing the written test questions - and not because I think that
the training/testing paradigm of having all specific questions known
in advance is a good one. It is not. It makes it impossible to test
at a level higher than rote memorization. It is possible to study for
the tests at a higher level - and many do - but those who are most
successful study largely at the rote level. That's because a huge
percentage of the questions is downright bad. There are math
questions where the 'correct' answer is based on an approximation and
an exact solution lands halfway between a correct and an incorrect
answer, there are regulatory questions where no answer is really
correct and the best answer is a matter of opinion, and there are
absolutely irrelevant questions that would never be an issue for any
pilot, any time, anywhere.

Thus because I don't trust the FAA to do a good job of testing, I also
want no surprises. But it's important to remember that this is
inherently a bad situation. The no-surprises model is NOT a good
testing model.

Designated examiners are generally experienced pilots, but the fact is
that becoming one has more to do with having an "in" at the FSDO than
it has with being good at teaching or testing. Some good examiners do
slip through the process, but the process is such that I have only
ever sent a student to an examiner I did not know personally once, and
my experience was such that I will never do it again for any reason.
I can certainly understand the desire for a no-surprises practical
test process when going to an unknown examiner - it expresses a lack
of trust, but once again I believe the lack of trust is reasonable and
earned.

However, this kind of no-surprises testing process is very difficult
to enforce (since the examiner does have fairly broad discretion) and
is in any case not in the best interest of the aviation community for
it breeds inferior pilots.

A pilot who can only do what he has previously practiced, and has no
ability to handle the unexpected is an inferior pilot, and given that
the unexpected DOES happen, he is not long for this world. Thus I
believe that effective testing requires that at some point the
applicant be surprised.

If you insist on treating the PTS as a restrictive document, thus
limiting the authority of the examiner to what he is specifically
permitted to do, then you eliminate the possibility of any surprise.
Of course that also means that the student can practice every possible
thing the examiner could possibly do. On the other hand, if you treat
the PTS as a permissive document, allowing the examiner the authority
to test in any way that does not explicitly conflict with the PTS,
then practicing every possible variation of what the examiner could do
is impossible. On the other hand, for this to work the examiner has
to be reasonable. Otherwise, he could always flunk your student by
asking him to perform a maneuver in a manner that no mewly rated pilot
could reasonably be expected to do. That sounds bad, but in reality
he can fail your student anyway - all he needs to say is that a
maneuver was out of tolerance, and who will question him?

I certainly believe it is reasonable to fail equipment (like a moving
map) not actually required for the task at hand. Since it is not
required, and is thus providing only supplemental information, its
failure can be considered a realistic distraction. It is realistic
because such failures do happen, and it is a distraction (rather than
a true emergency) because the equipment actually required for the
procedure is still available and functioning. On the other hand, is
it reasonable to fail the GPS, LORAN, autopilot, and VSI on a partial
panel approach, and make the student perform an off-field NDB partial
panel approach with hold in lieu of PT in a slick retract like a
Mooney or Bonanza? In my opinion, it is not - not because the skill
can't be learned (it can) and not because it's impossible (it's not -
multiple point failures can happen) but because this is simply beyond
what it reasonable to require at the intial instrument pilot skill
level. I might feel differently at the CFII level - but on paper, the
maneuvers and standards are the same. That's not reasonable either.

So what's the answer? In an ideal world examiners would be highly
experienced and respected instructors, and we could trust their
judgment. Of course then we would not need a PTS at all - we would
simply train our students for the real world, and count on examiners
to effectively test their readiness to exercise the priveleges of the
certificate or rating.

In the real world, the solution is to send your students to DE's you
know and respect who will in fact test that way. Then you can forget
about the PTS and train students based on your knowledge of what is
really important in flying, and it will be OK - they will both pass
the checkride and become competent pilots. Works for me.

And forget about trying to fight it out with bad DE's - they have it
all their way.

Michael