Microsoft’s warning that its new experimental AI feature “Copilot Actions” could lead to device infections and sensitive data theft has sparked a wave of criticism from security experts, who question the wisdom of pushing new technology before fully securing it.
Article Contents:
- An unprecedented warning from the tech giant
- What is the “Copilot Actions” feature?
- Underlying vulnerabilities: Hallucinations and prompt injection
- The new threat: Cross-Site Prompt Injection (XPIA)
- Expert reactions: Comparisons to dangerous “macros”
- Is Microsoft trying to dodge liability?
- Ambitious security goals, but user-dependent
- Frequently Asked Questions
An unprecedented warning from the tech giant
In the technological arms race, speed in launching new features often takes priority over comprehensive security. This approach was clearly evident in Microsoft’s recent announcement regarding the experimental “Copilot Actions” feature built into the Windows operating system. While the company celebrated the capabilities of the new feature, it attached a major security warning, prompting a familiar response from security-minded critics: Why do big tech companies insist on launching new features before fully understanding and containing their dangerous behaviors?
What is the “Copilot Actions” feature?
Microsoft introduced “Copilot Actions,” a new set of “experimental agentic features.” When enabled, these features can perform daily tasks such as organizing files, scheduling meetings, or sending emails. Microsoft describes them as “an active digital collaborator capable of executing complex tasks to boost efficiency and productivity.”
However, this announcement came with a major caveat. Microsoft recommended that users enable “Copilot Actions” only “if they understand the security implications outlined,” noting that the feature is currently available only in preview versions of Windows and is disabled by default.
Underlying vulnerabilities: Hallucinations and prompt injection
The warning is based on well-known and inherent flaws in most Large Language Models (LLMs), including “Copilot.” One common flaw is “hallucinations,” where models provide incorrect and illogical answers, meaning their outputs cannot be trusted without independent verification.
The greater danger is “prompt injection.” This is a vulnerability that allows attackers to plant malicious instructions in websites, resumes, or emails. Because large language models are programmed to follow directions enthusiastically, they cannot distinguish between valid user commands and those contained in untrusted external content, granting attackers the same level of trust as the user.
The new threat: Cross-Site Prompt Injection (XPIA)
Both flaws can be exploited in attacks that lead to sensitive data leakage, execution of malicious code, and cryptocurrency theft. To date, it has proven impossible for developers to completely prevent these vulnerabilities.
Microsoft acknowledged this risk in its disclosure: “Agentic AI applications introduce new security risks, such as Cross-Site Prompt Injection (XPIA), where malicious content embedded in user interface elements or documents can override agent instructions, leading to unintended actions like data exfiltration or malware installation.”]
Expert reactions: Comparisons to dangerous “macros”
Some security experts questioned the value of these warnings, comparing them to warnings Microsoft has provided for decades about the danger of using “macros” in Office applications. Despite ongoing advice, macros have remained one of the easiest ways for hackers to install malware on Windows devices, because they became so central to productivity that many users cannot do without them.
Independent security researcher Kevin Beaumont said: “Microsoft saying ‘don’t do macros, they’re dangerous’… has never really worked out well.” He described “Copilot Actions” as “macros on superpowers,” pointing to the escalating danger they pose.
Is Microsoft trying to dodge liability?
Microsoft indicated that only experienced users should enable “Copilot Actions.” However, the company did not specify what kind of training or experience these users should have, or the steps they must take to prevent their devices from being compromised.
Critics argue that this warning is merely a legal maneuver to try to protect the company from liability and shift the burden to the user. Critic Reed Medicky said: “Microsoft (like the rest of the industry) has no idea how to stop prompt injection or hallucinations, rendering them fundamentally unfit for anything serious. The solution? Shift the responsibility to the user.”
Ambitious security goals, but user-dependent
Much of Microsoft’s announcement focused on its overarching strategy for securing agentic features in Windows. Goals include ensuring all actions are observable and requiring user consent when accessing data or taking actions.
These goals are sound, but they ultimately rely on users reading dialogue boxes that warn of risks and require precise consent. This diminishes the value of protection for many users who may grow accustomed to clicking “yes” all the time, or who can be tricked into following dangerous instructions due to fatigue or lack of knowledge. Experts also note that experimental features often become default over time, increasing long-term risks.
Frequently Asked Questions
What is the “Copilot Actions” feature that Microsoft is warning about?
It is a set of experimental AI features in Windows that can perform tasks such as organizing files and automatically sending emails.
What are the main security risks associated with this feature?
The main risk is “prompt injection,” where malicious content can trick the AI into executing unintended actions, such as stealing data or installing malware.
Is this feature enabled by default?
No, the feature is disabled by default and is currently available only in preview versions of Windows. Microsoft recommends enabling it only for experienced users.
Why are security experts criticizing Microsoft?
They are criticizing the company for launching new technology before fully securing it, and are concerned that the warnings shift the security burden onto users rather than solving the underlying problems in AI models.
