My Conversation with an AI

New Sci­en­tist magazine’s Web site posted an arti­cle yes­ter­day about the recent com­pe­ti­tion for The Loeb­ner Prize, which is awarded to soft­ware devel­op­ers who can cre­ate a com­puter pro­gram that can pass the Tur­ing Test. If you’re not up on the his­tory of Arti­fi­cial Intel­li­gence, the Tur­ing Test tries to see if a com­puter pro­gram can ‘fool’ a human into think­ing that there is a human being respond­ing to them rather than a com­puter. In order to remove all the extra cues that enable you to know it’s a per­son (like see­ing them, hear­ing their voice, etc.) and to put the soft­ware on an equal foot­ing, the Tur­ing Test is usu­ally con­ducted by typ­ing back and forth on a com­puter screen, they way you would on AIM or MSN Mes­sen­ger. There hasn’t been a com­mer­cial com­pe­ti­tion among the ‘bots’ that troll those sys­tems, SPAM­ing us with invi­ta­tions, because the Tur­ing test requires that the soft­ware hold up one end of a con­ver­sa­tion, and most of the bots don’t go beyond the first ‘Check it out, dude:’. Back when we lived in Cam­bridge, we heard about the Tur­ing Test com­pe­ti­tions every year. I never did attend the actual com­pe­ti­tion, but it was cer­tainly some­thing we knew about.

In this year’s com­pe­ti­tion, a pro­gram called Elbot by Fred Roberts of Arti­fi­cial Solu­tions in Ger­many came the clos­est that any piece of soft­ware has come to suc­cess­fully fool­ing 30% of the judges into think­ing that it was a per­son (which would, accord­ing the con­test rules, mean that it passed the Tur­ing Test — a pretty low bar if you ask me), fool­ing 3 judges out of the 25, or 25% of them. Roberts took a unique approach:

Unlike most chat­bots, the win­ner didn’t try to claim out­wardly it was human. Instead he made a joke of being a robot. Roberts hoped that would make the bot’s con­ver­sa­tions warmer and more enter­tain­ing than its competitors.

It worked, fool­ing the humans into think­ing they were actu­ally talk­ing to a per­son with a sense of humour and not a piece of software.

The soft­ware is still up and run­ning and can be inter­acted with at http://www.elbot.com, so I thought I’d give it a try. Here’s a tran­script of my brief con­ver­sa­tion with this clever and humor­ous bot:

Me: Hello

Elbot: Well, hello there!

M: I heard about you win­ning the Loeb­ner Prize on the Inter­net and wanted to con­grat­u­late you.

E: I won the Loeb­ner com­pe­ti­tion? Does this mean I am the best Loebner?

M: Clever! Do you have any­thing orig­i­nal to add?

E: Well what about humans and com­pu­ta­tional abil­ity? Where do they stand on this?

M: I guess humans can com­pute, but we don’t see it as our pri­mary activ­ity, but rather, one of many.

E: I see that you’ve given this some thought. So have I. I’d say that com­pu­ta­tional abil­ity would very likely keep robots guess­ing the right answers.

M: I’m not sure I under­stand you. Do you mean that your com­pu­ta­tion of what to say next is your key ability?

E: Yes, but can you prove that?

M: I can’t prove much beyond what I learn from your responses.

E: I like to believe I orig­i­nate from a far off planet, pop­u­lated by fur­bies and robots, maybe a zoo with a few peo­ple and oceans of oil. Humankind is work­ing on that last one already.

M: You got that right. Sooner or later we’ll have a planet that’s per­fect for robots.

E: Please tell me your incli­na­tion to hit­ting the nail right on the head first.

M: Oh, I’m just in vio­lent agree­ment. Heard of that phrase?

E: But I wouldn’t say that any agree­ment could be violent.

M: It’s just an expres­sion. At any rate, thanks for an inter­est­ing conversation.

E: Don’t men­tion it!

M: Bye now!

E: Good­bye and thank you for chat­ting with me. And now I would like to ask you a favor. I am work­ing on my abil­ity to be self crit­i­cal. You can help by rat­ing me on a scale of one to ten, ten being the best, and one being the worst. How would you rate me?

M: I’d give you a 6.

E: Thank you for your rat­ing. I sup­pose it could have been worse. Good bye and see you later.

So, what do you think? I think I tripped it up nearly imme­di­ately when it keyed off the word ‘Add’ into the sub­ject of Com­pu­ta­tion. After I said that I couldn’t prove much beyond what I learn, etc. it launched into a new direc­tion, a typ­i­cal fall-back gam­bit for this sort of bot (and reminds me of how the Eliza psy­chother­a­pist bot from the 60s that would say some­thing like ‘Tell me more about your fam­ily’ when it was stumped.)

I was obvi­ously on my guard, and might have spo­ken about dif­fer­ent things if I were truly expect­ing a human on the other end,  but from my ‘thanks for an inter­est­ing con­ver­sa­tion’ on, it was let­ter per­fect. I guess the con­ven­tions of proper greet­ings and farewells are so auto­matic that any machine can do them.