About That Turing Test Win...

This past weekend, the media was all abuzz about how the Turing test had just been passed by a computer. This was an amazing achievement thatheralded a new milestone in computing history. Or was it? Let’s take a look at what happened and what it really means.

The announcement was made by the University of Reading on Sunday, June 8th. They were very proud of the achievement. From the press release:

The 65 year-old iconic Turing Test was passed for the very first time by computer programme Eugene Goostman during Turing Test 2014 held at the renowned Royal Society in London on Saturday.

The upshot is that the software program called “Eugene Goostman” was able to fool 33% of the human judges as to whether or not it was a human. We’ll talk more about this in a moment. There was a pretty amusing take down done by Tech Dirt’s Mike Masnick, which is worth a read.

I have a personal and professional interest in artificial intelligence (AI) and AlanTuring’s work. I have studied and done work in the field for a number of years. As a result, announcements like these always catch my attention. This seemed a good opportunity to talk just a little bit about the Turing test, what it means, and the relevance of any software or hardware passing it.

Turing is one of the founding fathers of the field of computer science. He was a skilled cryptanalyst, logician, mathematician, and philosopher. Anyone who has ever studied computer science in earnest should know of Turing and his work. His foundational work on discrete-state machines, general purpose state machines, and the Turing machine concept still forms the basis for nearly all computing to this day.

The Turing test comes from a proposal Turing made as a part of his paper “Computing Machinery and Intelligence,” published in 1950 in the journal Mind (volume 59, 433-460). A copy can be found here for your enjoyment. It’s worth a read and offers a glimpse into the state of computational science in 1950 as well as a better understanding how things have changed, and how they have not.

In the paper, Turing addresses the fundamental question, “Can machines think?”He writes, "I propose to consider the question, 'Can machines think?' This should begin with definitions of the meaning of the terms 'machine' and 'think.'"

He goes on to observe that the use of popular definitions, or consensus definitions, to explore the question would be foolish. He proposes, instead, a new question to consider:

The new form of the problem can be described in terms of a game which we call the 'imitation game.' It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart from the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either 'X is A and Y is B' or 'X is B and Y is A.'

This is the Turing test, in a nutshell. The idea is to have two people available who communicate with a third, the interrogator. The interrogator’s job is to determine the gender of persons X and Y, knowing in advance that there is one man and one woman who are either X or Y. He goes on to explain thatthe job of A(the male) is to attempt to fool the interrogator into believing that he is a woman. The object for B(the female) is to attempt to convince the interrogator into believing that she is the woman.

Obviously, the interrogator is unable to touch, see, hear, or otherwise interact with A and B in any way other than written or typewritten communication (chat or email in many modern tests) to avoid any clues other than the communication itself.

The test has been recast and reinterpreted over the years in many ways. It is commonly thought to test whether or not a human chatting with a computer is fooled as to whether or not the computer is a human. This is a more simplified version of the test, and, in my opinion, a much weaker one.

The true essence of the Turing test is one of rich context, of nuance. The idea was to have a competitive set-up where each participant has their own goal, which isn’t just to convince another that they are human, but rather to convince them of gender as well. Since all are aware of each other’s goals, the interrogator knows that one will attempt to trick and the other to tell the truth.

On the surface, one can see that the basic test of “am I chatting with a human or a computer” is overly simple andsignificantly less useful. It is based on the credulity of the interrogator only. If the interrogator asks poor questions, does not stray into areas of weakness for the machine, might they not come to an incorrect conclusion?

More importantly, if a plurality of interrogators accept a machine’s communications as from a human, does this mean the machine is truly thinking? Or does a failure to convince mean that the machine is not thinking?

Judging whether or not a clever program or computer is thinking is very difficult. Turing made a good attempt at a test for this, but I do not believe the Turing test is sufficient to determine whether or not a machine is an AI. Consider that chatbots, like Cleverbot, have fooled quite a few folks quite a few times. Amusingly, in fact Cleverbot got a better score (59%) than Eugene Goostman did. Heck, even good old Eliza used to snooker the odd freshman back in my days.

Personally, I am not convinced that discrete state machines, such as digital computers, will be able to achieve human-level intelligence. The human mind is a continuous phenomenon, an analog device in an analog universe. It may be the case that true AI can only be made once we have hardware that better simulates a real brain. Of course, Mr. Turing did not fully agree. He addresses this concern in his paper in section six, numberseven, “Argument from Continuity in the Nervous System.” A good argument, but I stand by my opinion, until I get more evidence, of course.

Be well.