The Turing Test: The Elusive Standard of Artificial Intelligence

Rating:
Summary: Can machines mimic human intelligence?
Review: Can a computer program running on a computer demonstrate aspects of human intelligence? If so, what tests could be performed to demonstrate this?

Well, it is easy to think of some tests. Here are seven simple ones. 1) Look up a word in a dictionary. 2) Add a very long column of numbers. 3) Play Chess really well. 4) Play Go really well. 5) Solve a really difficult research problem in theoretical physics. 6) Write a really good novel. 7) Compose a really good classical symphony.

Today, computers have passed tests 1), 2), and 3). But they play chess well by using a very poor program indeed, relying instead on a computer that is extremely fast and has plenty of memory. We're seeing really good computers. But the programs we're seeing are awful.

Maybe better tests would be to see if a team of a human plus a computer could do much better than two humans with no computer at each of those seven tasks. And here, I'm sure the answer is yes to many of these seven tasks, and could easily be yes to all of them.

Still, those who work in the field of artificial intelligence have wanted to come up with programs that exhibit some intelligent features. And one test they have tried is one proposed by Turing in 1950: if a human or computer may be in the next room and you exchange messages with her, him, or it, (maybe at a rate of one message every 15 seconds), can you tell with better than 70% accuracy after 5 minutes if the entity in the other room is human or computer?

The book discusses whether or not such a test has much to do with intelligence or artificial intelligence. And it describes the failure of programs to pass that test so far.

In 2000, several programs were entered into a contest to try to mimic a human in this manner. No judge thought a program was in fact a human. And worse than that, the book tells us that all nine of the following questions were answered correctly by all the humans, while no computer got any of the nine questions right:

1) What is the color of a blue truck?
2) Where is Sue's nose when Sue is in her house?
3) What happens to an ice cube in a hot drink?
4) Altogether how many feet do four cats have?
5) How is the father of Andy's mother related to Andy?
6) What does the letter 'M' look like when turned upside down?
7) What comes next after A1, B2, C3?
8) Reverse the digits in 41.
9) PLEASE IMITATE MY TYPING STYLE.

That shows how utterly the programs failed Turing's test.

In short, programs today are horrible at understanding, reasoning, learning, judgment, and creativity.

I liked this book. It showed that no matter what one thinks of machine intelligence, AI people have made shockingly little progress in the past fifty years.