Boffins from the University of Hong Kong and Singapore’s Agency for Science, Technology, and Research devised an algorithm called TextFooler capable of deceiving an AI system without changing the meaning of a piece of text. The algorithm uses AI to suggest which words should be converted into synonyms to fool a machine.

To trick an algorithm designed to judge movie reviews, for example, TextFooler altered the sentence:

“The characters, cast in impossibly contrived situations, are totally estranged from reality”, to read: “The characters, cast in impossibly engineered circumstances, are fully estranged from reality.”

This caused the algorithm to classify the review as “positive”, instead of “negative”. The demonstration highlights an uncomfortable truth about AI—that it can be both remarkably clever and surprisingly dumb.

Researchers tested their approach using several popular algorithms and data sets, and they were able to reduce an algorithm’s accuracy from above 90 percent to below 10 percent. People generally judged the altered phrases to have the same meaning.