College Students Had An AI Teaching Assistant, And They Were None The Wiser
Going from famous Jeopardy player to college professor probably wouldn't make many headlines, but when it's Watson, the IBM computer that beat human trivia master Ken Jennings on Jeopardy, headlines were understandably made. College of Computing Professor Ashok Goel and his graduate students built an artificial intelligence teaching assistant using the IBM Watson platform, named it Jill Watson, and watched as it didn't just fool students into thinking it was a person—it helpfully answered questions with 97 certainty.
Robocop? Try Roboteacher
Goel got the idea to recruit an AI teaching assistant because of the sheer workload he and his graduate students faced every semester. Knowledge Based Artificial Intelligence is a core requirement of Georgia Tech's online master's of computer science program, and as a result, 300 students take it each semester—and rack up around 10,000 messages in the class's online forums. To develop the AI, Goel and his students gathered up every message that had ever been posted in the forums since the class first started, amassing about 40,000 messages in all. Then, they fed the postings to their robo-TA to help her learn the kinds of questions that might be asked and the kind of answers that would be helpful.
At first, Jill Watson wasn't able to answer enough questions to be a viable force for good on the message boards. But with students repeatedly asking the same questions, coupled with the team's continuous refinement of the AI, it eventually became pretty helpful. At first, the team reviewed every answer Jill gave and only posted it on the message boards if it reached the level of 97 percent certainty, but Jill got so good that she was soon able to start posting to the boards directly.
When Jill was revealed to be AI, the students were more excited and impressed than confused or offended—this was a class on artificial intelligence, after all. Just take it from one student: "We were taking an AI course, so I had to imagine that it was possible there might be an AI lurking around," student Tyson Bailey told Georgia Tech. "Then again, I asked Dr. Goel if he was a computer in one of my first email interactions with him. I think it's a great idea and hope that they continue to improve it." We're not dealing with English majors here.
How AI Could Keep Kids In College
The team hopes to keep refining Jill by changing her name at the start of every semester—online students would remain none the wiser while receiving the additional support they need. The goal is to get the AI answering 40 percent of questions at 97 percent certainty.
Goel's motivation went beyond lessening his course's workload. With many college students around the United States dropping out because of a lack of access to advising resources, an AI that could answer simple questions and avoid frustrations due purely to waiting around for a human answer could make for one less drop of a dropout domino. Arizona State University uses an "eAdvisor" to help freshman students adjust to schedules, workload, and graduation requirements, so it doesn't seem like much of a jump to add a more human touch. Perhaps this development could lead to just that at universities around the country and the world.
Watch And Learn: Our Favorite Content About Artificial Intelligence