How IBM's Watson Pattern-Matches to Answer Trivia

The way Watson plays the game also requires massive amounts of pattern matching. The supercomputer has been loaded with hundreds of millions of unconnected digital documents, including encyclopedias and other reference works, newspaper stories, and the Bible. When it receives a question, it immediately goes to work to figure out what is being asked (using algorithms that specialize in complex communication), then starts querying all these documents to find and match patterns in search of the answer. Watson works with astonishing thoroughness and speed, as IBM research manager Eric Brown explained in an interview:

We start with a single clue, we analyze the clue, and then we go through a candidate generation phase, which actually runs several different primary searches, which each produce on the order of 50 search results. Then, each search result can produce several candidate answers, and so by the time we’ve generated all of our candidate answers, we might have three to five hundred candidate answers for the clue.

Now, all of these candidate answers can be processed independently and in parallel, so now they fan out to answer-scoring analytics [that] score the answers. Then, we run additional searches for the answers to gather more evidence, and then run deep analytics on each piece of evidence, so each candidate answer might go and generate 20 pieces of evidence to support that answer.

Now, all of this evidence can be analyzed independently and in parallel, so that fans out again. Now you have evidence being deeply analyzed … and then all of these analytics produce scores that ultimately get merged together, using a machine-learning framework to weight the scores and produce a final ranked order for the candidate answers, as well as a final confidence in them. Then, that’s what comes out in the end.

What comes out in the end is so fast and accurate that even the best human Jeopardy! players simply can’t keep up. In February of 2011, Watson played in a televised tournament against the two most accomplished human contestants in the show’s history. After two rounds of the game shown over three days, the computer finished with more than three times as much money as its closest flesh-and-blood competitor. One of these competitors, Ken Jennings, acknowledged that digital technologies had taken over the game of Jeopardy! Underneath his written response to the tournament’s last question, he added, “I for one welcome our new computer overlords.”

And how it does it so fast.