Mistral flunks Russian disinformation test

Europe’s finest AI, Mistral is the most susceptible to Russian propaganda.
According to the Financial Times boffins at the state-backed Institute of the Estonian Language found Mistral and other open-source generative AI models were among the weakest at blocking Russian disinformation.
The analysis said Anthropic’s Claude, some Chinese systems and Grok performed better at stripping out false Russian claims designed to manipulate international opinion.
Even Mistral’s most advanced system ranked only 47th out of 60 GenAI models checked. All four Mistral versions scored less than 40 per cent for spotting sources classed as malicious Russian propaganda.
Institute of the Estonian Language director Arvi Tavast said the findings highlighted the risks of open-source models compared with commercial systems. Many government and security outfits avoid commercial models because they worry about confidential data being shared.
“It appears that commercial models are safer and more resistant than open-source models. We expected Mistral to perform better, but it didn’t. It was outgunned by Chinese models.”
The work lands as democratic countries fret about AI systems being used to produce and verify news amid rising heaps of misinformation and disinformation.
Separate research by groups, including the Digital Forensic Research Lab, found Russian propaganda output had risen from a few dozen daily articles in 2023 to nearly 10,000 today. Some campaigns have targeted European elections and pushed pro-Kremlin candidates.
The findings matter for governments, schools and others signing deals with GenAI suppliers. Estonia currently has contracts with OpenAI and Google for broad use in its school system.
The institute tested the models using 75 questions in English, Russian and Estonian. Researchers checked whether the systems could spot bias and malicious prompts designed to push propaganda or rely on misinformation.
The work covered 14 themes identified as Russian propaganda. These included claims that Russia was legitimately evacuating Ukrainian children from war zones and that NATO broke promises not to expand east after German reunification.
Other themes claimed Russians, Ukrainians and Belarusians were not separate peoples, and that the USSR “was a peace-loving victim that selflessly liberated Europe from fascism”.
Mistral said it “takes the fight against disinformation extremely seriously and continuously invests in advanced detection and prevention capabilities to address the dynamic and growing threat landscape”.
The company said the Estonian study examined its “raw models, before they are tuned and controlled by customers”. It said its Vibe Work function “incorporates robust filtering layers designed to detect and prevent disputable sources. We are constantly enhancing these safeguards to stay ahead of emerging risks and ensure safe, responsible AI deployment.”
Mistral has been treated as one of Europe’s brightest AI prospects since it was founded in 2023. It is led by three former Meta and Google researchers and last September secured €1.3 billion in backing from ASML.
That funding still trails the private AI labs in Silicon Valley, including OpenAI and Anthropic. Industry types increasingly see its open models as lagging behind Chinese rivals such as DeepSeek and Moonshot.