Stuart Shieber, 1994a.

The Shieber Argument

Although allegedly established to "further the scientific understanding of complex human behavior," the Loebner prize is so restricted that it fails to serve this purpose. By restricting conversation to limited topics and using referees to ensure that no unfair questions are asked, the "open-ended, free-wheeling" nature of real human conversation is lost. As a result, contestants win by exploiting crafty tricks rather than by simulating intelligence, and hence the goal of understanding human intelligence is lost.

"Thus, it is difficult to imagine a clear scientific goal that the Loebner prize might satisfy. Turing's test as originally defined, on the other hand, had a clear goal: to serve as a sufficient condition for demonstrating that a human artifact exhibited intelligent behavior. Even this goal is lost in the Loebner prize competition. By limiting the test, it no longer serves its original purpose (and arguably, no purpose at all), as Turing's syllogism fails. It is questionable whether the notion of a Turing Test limited in the ways specified by the Loebner Prize committee is even a coherent one" (Shieber, 1994, 75-76).

"The reason that Turing chose natural language as the behavior definitional of human intelligence is because of its open-ended free-wheeling nature. 'The question-and-answer method seems to be suitable for introducing almost any of the fields of human endeavor that we wish to include' (28, p. 435).

In attempting to limit the task of the contestants through limiting the domain alone, the prize committee succeeded in doing neither" (Shieber, 1994, 76).

"In summary, the Loebner Prize competition neither satisfies its own avowed goals, nor the original goals of Alan Turing" (Shieber, 1994, 76).

Shieber, Stuart (1994). "Lessons from a Restricted Turing Test." Communications of the ACM, 37:6