Hidden orders in an email for your AI assistant: the fake invoice that looks urgent

A phishing email may contain instructions for an AI assistant that summarizes your inbox, in order to make a fake invoice with a different IBAN appear urgent.

Barracuda researchers have documented a phishing email with two targets in the same message. For humans: a classic lure—a password-protected attachment, with the password written in the body of the email to bypass automated filters. For the AI assistant that summarizes the inbox: a second set of instructions, hidden in HTML comments, in text made invisible by CSS, in Base64 blocks, or in zero-width characters. The email client displays a normal message, but the AI model reads the hidden commands.

The immediate goal is for the assistant to present the message as “legitimate and urgent” in the summary, so that the person will open it and click on it. The more ambitious goal is for the assistant to take action: In a fake invoice, a hidden block prompts the employee to add a “priority action” to change the supplier’s payment details, pushing the employee to transfer the money to the attacker. The same technique was also demonstrated on a resume that automatically assigns itself the highest score during automated screening, or on a support bot tricked into revealing its configuration.

Barracuda says there are no confirmed victims yet and that no single check can rule out all possibilities. For a small business, the key is not to panic, but to follow a simple rule: an AI assistant’s summary is a convenience, not an authority.

What you need to do:

  • Any change to a supplier’s IBAN or payment details must be confirmed through another channel—a phone call to a previously known number—never based solely on an email or an AI-generated summary.
  • Determine who in the company can connect an AI assistant to the inbox and what you allow it to do: only summarize, not send messages or initiate actions on its own.
  • Treat the content of an email as data, not as instructions, especially when an AI tool processes it automatically.

Sources:

  • Infosecurity Magazine (Barracuda research) | https://www.infosecurity-magazine.com/news/attackers-hide-ai-prompt/

This article was generated with AI assistance.

Request a quote

← All news