Team Watson Begs to Differ

A reader writes:

I’m part of the team that worked on Watson. I’d like to respond to a comment you made:

But there still has to be a lag of a tenth of a second or so for a human between seeing the light and moving one's thumb; Watson obviously didn't have this lag. The result was that the humans were able to ring in first only when Watson didn't know the answer.

In fact there were many times in the televised game in which Watson was beaten to the buzzer, that was plainly visible to viewers. On several occasions you could see Watson’s answer panel showing the correct answer yet Brad or Ken got the buzz.

 Watson gets the signal electronically and can reliably and repeatedly hit its buzzer very quickly, but not instantaneously. There is a built-in mechanical lag. And as any champion Jeopardy player knows, great players listen to Alex reading the clue and lead him -- they start to press the button just as he’s finishing saying the final syllable. In that way they can buzz in only single milliseconds after the light goes on. Watson is very fast, but Watson had no microphone and could not anticipate the onset of the light as the humans can.

I stand corrected. In my defense, (1) I've never been on  Jeopardy!, (2) I certainly saw times where Watson had the correct answer and someone else got the points, but I assumed that we were seeing the results of a calculation that had finished late. I was especially struck by the results in the first game, when Watson built up a huge lead simply by getting in first over and over. This happened even on questions where the other players surely knew the answer (or, to be precise, vice versa).  But watching on TV, it's impossible to tell to what extent the same thing happens with human contestants.

Still, I'd like to see statistics on how fast human response actually is. The New York Times said on Sunday that Watson's lag time once he had solved the problem was 10 milliseconds. That's not a lot of milliseconds, especially since I believe the light has to be turned on by a human and so is not perfectly predictable.

I stand by my comment that the difference in scores doesn't reflect the differences in cognitive abilities, but perhaps this is true of Jeopardy! in general: perhaps small differences in thumb technique play a large role. (I've certainly heard that said about Ken Jennings's streak.) Fortunately for fans like me, there are still plenty of occasions on ordinary days for shouting at the screen when no one knows the answer.

I also stand by my comment, "Good job, IBM." The impressive thing is that Watson was competitive at all.