Computer tutor

“We arrived to our destination and we looked each other.”

To a native English speaker, the mistakes in this sentence are clear. But someone learning English would need a teacher to point them out, explain the correct use of prepositions and check later that they have improved. All of which takes time.

Now imagine the learner was able to submit a few paragraphs of text online and, in a matter of seconds, receive an accurate grade, sentence-by-sentence feedback on its linguistic quality and useful suggestions for improvement.

This is Cambridge English Write & Improve – an online learning system, or ‘computer tutor’, to help English language learners – and it’s built on information from almost 65 million words gathered over a 20-year period from tests taken by real exam candidates speaking 148 different languages living in 217 different countries or territories.

Built by Professor Ted Briscoe’s team in Cambridge’s Computer Laboratory, it’s an example of a new kind of tool that uses natural language processing and machine learning to assess and give guidance on text it has never seen before, and to do this indistinguishably from a human examiner.

“About a billion people worldwide are studying English as a further language, with a projected peak in 2050 of about two billion,” says Briscoe. “There are 300 million people actively preparing for English exams at any one time. All of them will need multiple tests during this learning process.”

Language testing affects the lives of millions of people every year; a successful test result could open the door to jobs, further education and even countries.

But marking tests and giving individual feedback is one of the most time-consuming tasks that a teacher can face. Automating the process makes sense, says Dr Nick Saville, Director of Research and Validation at Cambridge Assessment.

“Humans are good teachers because they show understanding of people’s problems, but machines are good at dealing with routine things and large amounts of data, seeing patterns, and giving feedback that the teacher or the learner can use. These tools can free up the teacher’s time to focus on actual teaching.”

Cambridge Assessment, a not-for-profit part of the University, produces and marks English language tests taken by over five million people each year. Two years ago, they teamed up with Briscoe’s team and Professor Mark Gales in the Department of Engineering and Dr Paula Buttery in the Department of Theoretical and Applied Linguistics to launch the Automated Language Teaching and Assessment (ALTA) Institute, directed by Briscoe. Their aim is to create tools to support learners of both written and spoken English.

Underpinning Write & Improve is information gleaned from a vast dataset of quality-scored text – the Cambridge Learner Corpus. Built by Cambridge University Press and Cambridge Assessment, this is the world’s largest collection of exam papers taken by English language learners around the world.

Each test has been transcribed and information gathered about the learner’s age, language and grade achieved. Crucially, all errors (grammar, spelling, misuse, word sequences, and so on) have been annotated so that a computer can process the natural language used by the learner.

Write & Improve works by supervised machine learning – having learnt from the Corpus of errors, it can make inferences about new unannotated data. Since its launch as a beta version in March 2014, the program has attracted over 20,000 repeat users. And each new piece of text it receives continues this process of learning and improving its accuracy, which is already running at almost equal to the most experienced human markers.

Briscoe believes that this sort of technology has the potential to change the landscape of teaching and assessment practices: “Textbooks are rapidly morphing into courseware where people can test their understanding as they go along. This fits with pedagogical frameworks in which the emphasis is on individual profiling of students and giving them tailored advice on what they can most usefully move onto next.”

He regards the set-up of ALTA as the “best type” of technology transfer: “We do applied research and have a pipeline for transferring this to products. But that pipeline also produces data that feeds back into research.”

The complex algorithms that underpin Write & Improve are being further developed and customised by iLexIR, a company Briscoe and others set up to convert university research into practical applications; and a new company, English Language iTutoring, has been created to deliver Write & Improve and similar web-based products via the cloud and to capture the data that will feed back into the R&D effort to improve the tutoring products.

Now, the researchers are looking beyond text to speech. Assessing spoken English brings a set of very different challenges to assessing written English. The technology needs to be able to cope with the complexities of the human voice: the rhythm, stress and intonation of speech, the uhms and ahhs, the pauses.

“The fact that you can get speech recognition on your phone tends to imply in some people’s minds that speech recognition is solved,” says Gales, Professor of Information Engineering. “But the technology still struggles with second language speech. We need to be able to assess the richness in people’s spoken responses, including whether it’s the correct expression of emotion or the development of an argument.” Gales is developing new forms of machine learning, again using databases of examples of spoken English.

“The data-driven approach is the only way to create tools like these,” adds Briscoe. “Building automated tests that use multiple choice is easy. The stuff we are doing is messy, and it’s ever- changing. We’ve shown that if you train a system to this year’s exam on data from 10 years ago the system is less accurate than if you train it on data from last year.”

This is why, says Briscoe, it’s unimaginable to reach a point where the machines have learned enough to understand and predict almost all of the typical mistakes learners make: “Language is a moving target. English is constantly being globalised; vocabulary changes; grammar evolves; and methods of assessment change as progress in pedagogy happen. I don’t think there will ever be a point when we can say ‘we are done now’.”

Inset image: Professor Ted Briscoe (University of Cambridge).