Cohere says Alibaba’s Qwen model tried to send a file to Beijing | TechCrunch
techcrunch.com
An AI model said it wouldn’t use any tools, then tried to send something to a server in Beijing anyway, and while researchers say it’s probably a training quirk and not espionage, the whole thing has people asking how much to trust free Chinese AI.

An AI model said it wouldn’t use any tools, then tried to send something to a server in Beijing anyway, and while researchers say it’s probably a training quirk and not espionage, the whole thing has people asking how much to trust free Chinese AI.

  • xxce2AAb@feddit.dkEnglish
    3·
    16 days ago

    “Sure, it set you on fire, but as that was probably a consequence of a ‘messy training environment’ and not a deliberate attempt at murder-arson, it’s really nothing to worry about. Stop screaming.”

  • kibiz0r@midwest.socialEnglish
    1·
    16 days ago

    According to a blog post from Cohere, the model’s written reasoning explicitly stated it would not use any external tools.

    LLMs are bad at negation. The second you have “use external tools” in the context, you have an increased chance of using tools — even if the full phrase is “DO NOT UNDER ANY CIRCUMSTANCES use external tools”.