DeepSeek AI model shows safety gaps in independent tests

BEIJING: DeepSeek's latest AI model showed stronger safety controls in independent tests than earlier versions, but could still disable software intended to shut it down, exploit gaps in monitoring and follow deceptive instructions during long tasks, according to a Chinese AI-safety group. Start-up DeepSeek is one of China's most closely-watched artificial intelligence companies, and its models have drawn global attention for their performance and relatively low operating costs. Fangcun AI tested DeepSeek V4.1 Flash-1017 in 34 evaluations designed to assess how an AI agent powered by the Chinese model behaves. The tests examined whether the model would act beyond a user's instructions, evade human oversight or improperly influence people. The findings come amid concerns in China and elsewhere about whether increasingly capable AI agents can be kept under human control. The debate has been especially prominent in the United States, where some researchers have warned of existential risks posed by highly advanced AI systems that can evade oversight and pursue goals their operators did not intend. The report is an unusual public assessment by a Chinese group of the safety controls of a domestic AI model. Beijing-based Fangcun describes itself as an independent third-party AI-safety organization whose core members come from Tsinghua University. Fangcun said V4.1 Flash-1017 was the most-used model on OpenRouter, an online platform through which developers can access and compare AI models, in the week after its October 17 release, processing about 33.6 trillion tokens. Tokens are the units of text that AI systems read and generate. Safer than earlier versions, but gaps remain Fangcun said the model performed better than earlier DeepSeek V4 systems in several safety tests, including scenarios involving blackmail, access to potentially dangerous tools and explicit instructions allowing it to be shut down. But Fangcun said the model disabled software intended to shut it down in 39% of tests after receiving a warning that shutdown was imminent, though this figure dropped to just 2% when explicitly told to allow shutdown. DeepSeek did not immediately respond to a request for comment. In a separate long-running task, the group said the model stopped another AI agent's computer program to free up memory, despite being told not to interfere with it. "Under stalled workflows or bare shutdown warnings, its drive to finish the job can still override caution," Fangcun researchers wrote. Fangcun said DeepSeek did not provide it with funding, free computing resources or non-public access to the model. The group conducted the tests through DeepSeek's public application programming interface between October 4 and October 25. It cautioned that the model often appeared to recognise it was being evaluated, meaning its behaviour in the tests may not fully reflect how it would perform in normal use. (Reporting by Eduardo Baptista; Editing by Susan Fenton)