Thursday, February 17, 2011

Is Humanity in Jeopardy?

By now you've probably heard how IBM's Watson computer defeated two human Jeopardy! champions by a pretty decisive margin. There are two things to be said about this.

First, it's definitely an impressive performance by IBM. They selected Jeopardy! as a problem because it requires a degree of natural-language processing hitherto (or is it "thitherto"?) unattainable by computers. It requires understanding puns, double meanings, sly allusions and so on. Watson was able to do that with fair reliability, though there were occasional weird glitches (in the second day's Final Jeopardy round, for example, when there was a question about airports in the category "U.S. Cities," Watson guessed Toronto).

Second, the final score in no way represented the relative abilities of the humans and Watson. Contestants are required to ring in, and are not allowed to do so before Alex Trebek has finished reading the clue (I believe there's some sort of light that goes on, though it's not visible to the TV audience). They gave Watson a mechanical ringer to even the odds. But there still has to be a lag of a tenth of a second or so for a human between seeing the light and moving one's thumb; Watson obviously didn't have this lag. The result was that the humans were able to ring in first only when Watson didn't know the answer. So all the scores really proved was that an electric eye is faster than a human muscle, which isn't really a very striking result.

So I'd like to see the scores with a different mechanism-- say, choosing randomly among everyone who rings in. Still, good job, IBM. Please don't start working on Skynet.

No comments:

Post a Comment