AI security for business: shadow AI, prompt injection, agent permissions — and AI in cybersecurity

AI security in a company is often summed up in one sentence: “be careful what you paste into the chat.” That matters, but it is only part of the picture. AI in a company needs protecting — and at the same time AI can help protect the company. This article covers both sides from a defensive perspective: what it takes to stop AI tools becoming a new way into your systems, and where AI takes load off the people responsible for security. We do not describe attack techniques.

AI security does not start with a filter on the input; it starts with the question of what AI may read and what it may do. If an agent can only reach what it needs and a person approves every important step, even a successful manipulation has limited consequences. I see AI in defence the same way: as support for a team that understands its systems, not a replacement for it.

— Dariusz Wtykło, Owner and CEO of Primessio

Why now: AI entered processes faster than security did

The AI security study by BSI and TÜV-Verband, covering German companies, puts numbers on the gap. Binding internal policies for AI use exist at 59% of companies using AI³. Dedicated processes for AI-related security incidents exist at only 11% of companies that use or plan to use AI¹. At the same time, 17% reported confirmed or suspected cyberattacks or fraud attempts involving AI in the past 12 months². The authors put it plainly: companies are introducing AI faster than they adapt their security structures¹.

Side one: securing AI in your company

Shadow AI — tools the company does not know about

Shadow AI is the use of AI tools without the company’s knowledge or approval: a private chat account, a browser extension, a free translator. In the Microsoft and LinkedIn study, 78% of AI users said they bring their own AI tools to work⁶ — a vendor study, but the direction is clear. The problem is not AI itself; it is that company data ends up where the company does not control how it is stored or reused.

A ban alone rarely works, because the need remains. BSI and TÜV-Verband recommend a maintained AI inventory — applications, models, interfaces and owners, including AI features built into existing software — together with clear usage rules, risk assessments and approvals intended to prevent shadow AI⁴. Add a company tool that genuinely replaces private accounts, and short training on what may and may not be pasted in.

Prompt injection — when data poses as instructions

A language model does not strictly separate instructions from data. Researchers put it this way: LLM-integrated applications blur the line between data and instructions⁹. When an assistant reads an email, a web page or a file, a sentence hidden in it can try to change its behaviour — OWASP calls this indirect prompt injection, when a model accepts input from external sources such as websites or files⁷.

NIST notes that there is no foolproof way yet to protect AI from misdirection¹⁰. That leads to a practical rule: assume the filter will fail at some point and design the system so the consequences stay small. External content is marked as untrusted and kept apart from instructions, an assistant that reads mail has no permission to send or pay, and every step is logged.

Excessive permissions for AI agents

An AI agent does not just answer; it acts: it sends, saves and changes records. OWASP describes this risk as excessive agency — a vulnerability that enables damaging actions in response to unexpected, ambiguous or manipulated model outputs — and recommends requiring a human to approve high-impact actions before they are taken⁸. The OECD describes the same logic for agents in organisations: they make sense where outcomes can be checked and the cost of an error is bounded or reversible¹¹.

In practice that means a separate technical account for the agent, access only to the data it needs, limits on actions, human approval for payments, messages to customers and changes in systems, a log of every step, and tests with unusual and malicious inputs before production. We cover agent design in more detail in our analysis of AI agents — the link is at the end of this article.

Data in the EU and dependence on providers

Every query to a model is a data transfer. According to BSI and TÜV-Verband, 54% of companies run AI applications mainly with external cloud providers, and 30% describe themselves as dependent on non-European AI providers³. Across the EU, data protection concerns are among the most common reasons enterprises do not use AI at all (52.7%)¹².

The answer is not to give up on AI, but to decide which data is processed where: a provider with processing in the EU, a data processing agreement, data minimisation in queries and — where justified — a model running on your own infrastructure. For high-risk systems, the EU AI Act requires an appropriate level of accuracy, robustness and cybersecurity throughout the system’s lifecycle (Art. 15)¹⁴.

Side two: AI in cybersecurity — AI as a defensive tool

The same technology helps the teams that defend systems. In the BSI and TÜV-Verband study, one in three respondents expects AI to raise their IT security, for example through early detection of cyberattacks⁵. The authors recommend checking how AI can support defence: in finding vulnerabilities in authorised environments, maintaining the IT inventory and filtering suspicious emails⁵.

Reviewing code and dependencies

AI reads large amounts of code quickly and flags places to check: unsafe patterns, missing input validation, outdated libraries. In the final of DARPA’s AI Cyber Challenge, the teams’ AI systems discovered 54 unique synthetic vulnerabilities and patched 43 of them, and also found 18 real vulnerabilities¹⁵. That is a competition result on prepared tasks, not a promise for every company — but it shows the direction.

It works the other way too: code written with AI needs the same review. In a study presented at the IEEE Symposium on Security and Privacy, approximately 40% of programs produced by a code assistant in scenarios covering common weakness classes were vulnerable¹⁶. That is why, in our work, every change — written by a person or with AI — goes through review, tests and automated guards.

Logs and anomalies

A company’s systems produce more logs than a person can read. AI helps summarise them, group similar events and flag deviations from normal: a login at an unusual hour, a sudden rise in requests, a new traffic destination. Whether it is an incident is a human decision — the model shows where to look; it does not decide. The same logs are also the basis for overseeing AI itself: without a record of an agent’s steps you cannot reconstruct what it did and why.

Prioritising vulnerabilities

In a typical company the list of known vulnerabilities is longer than the time available to patch them, so the key question is where to start. EPSS, maintained by FIRST, is a machine-learning system that estimates the probability that a published CVE will be exploited in the wild in the next 30 days¹⁷. Combined with knowledge of your own systems — what is reachable from the internet, where customer data lives — it gives a data-driven order of work rather than a severity list alone.

Email and phishing

Among companies in the BSI and TÜV-Verband study that reported AI-assisted attacks or fraud attempts, 90% named highly realistic phishing emails². On the defensive side, AI helps filter suspicious messages⁵. A simple rule matters just as much: payment instructions and changed bank details are always confirmed through a second channel.

Area	What AI does	What stays with people
Code and dependencies	flags suspicious code and outdated libraries	assessment, fix, test
Logs and anomalies	summarises, groups, flags deviations	deciding whether it is an incident
Vulnerability prioritisation	estimates the probability of exploitation	order based on how critical systems are
Email	filters and flags suspicious messages	confirming payments through a second channel

What AI does not replace in security

• Knowledge of your own systems. AI flags a suspicious event, but only the team knows which server is critical and which is a test box.
• Checking results. According to BSI and TÜV-Verband, 67% of companies check critical AI outputs³; in McKinsey’s global survey only 27% of respondents say employees review all AI-generated content before use¹³. The rest is risk the company carries itself.
• Incident processes. Existing security and incident processes need AI scenarios added: data leakage, manipulated content and unwanted agent actions.
• Accountability. The company is responsible for the decision, whoever or whatever prepared it.

Where to start — a plan for the first month

• List the AI tools your team uses — including AI features in software you already have.
• Set rules: which tools, with which data, who approves new ones.
• Give the team a company tool that replaces private accounts.
• For each agent, document its permissions, set up human approval for high-impact actions and switch on step logging.
• Add AI scenarios to your incident processes.
• Pick one defensive area for AI support — for example dependency review or vulnerability prioritisation — and measure whether it helps.

This is how we design security in our implementations — see the AI security section on our AI implementation and automation page. The Primessio AI Implementation Framework describes the whole implementation path, from process mapping to operations — a method developed by Primessio in project practice.

Frequently asked questions

What is shadow AI and how do you contain it?

Shadow AI is the use of AI tools the company has not approved — usually in good faith, but with company data in a private account. In the Microsoft and LinkedIn study, 78% of AI users say they bring their own AI tools to work. BSI and TÜV-Verband recommend a maintained AI inventory of applications, models, interfaces and owners, plus clear usage rules, risk assessments and approvals. In practice a combination works: a company assistant, a short list of approved tools and training.⁶⁴

What is prompt injection and how do you protect against it?

Prompt injection is an attempt to change a model’s behaviour through instructions hidden in content. The indirect form arrives through external data such as websites or files. Protection comes in layers: external content is treated as untrusted, the agent has only the permissions it needs, high-impact actions wait for human approval, and every step is logged. NIST notes there is no foolproof protection yet — so limiting the impact is what counts.⁷⁸¹⁰

How does AI help with a company’s cybersecurity?

With work there is too much of for people: reviewing code and dependencies, analysing logs, spotting unusual events, filtering suspicious emails and prioritising vulnerabilities. BSI and TÜV-Verband recommend checking how AI can support defence — for example in finding vulnerabilities in authorised environments, maintaining the IT inventory and filtering suspicious emails. EPSS uses machine learning to estimate the probability that a vulnerability will be exploited in the next 30 days.⁵¹⁷

Is AI-generated code secure?

Not by default. In a study presented at the IEEE Symposium on Security and Privacy, approximately 40% of programs produced by a code assistant in scenarios covering common weakness classes were vulnerable. That is why AI-written code goes through the same review, tests and dependency checks as code written by people.¹⁶

Where should company data be processed in AI solutions?

Where the company has control and a legal basis — for EU companies usually in the EU, with as little data sent to the model as possible. According to BSI and TÜV-Verband, 54% of companies run AI applications mainly with external cloud providers, and 30% describe themselves as dependent on non-European AI providers. Data protection concerns are among the most common reasons EU enterprises do not use AI (52.7%).³¹²

Sources

1. BSI und TÜV-Verband (Feldarbeit: forsa): KI-Sicherheitsstudie 2026 · 2026-10-07 · tier 2 · checked 2026-10-20
2. BSI und TÜV-Verband (Feldarbeit: forsa): KI-Sicherheitsstudie 2026 · 2026-10-07 · tier 2 · checked 2026-10-20
3. BSI und TÜV-Verband (Feldarbeit: forsa): KI-Sicherheitsstudie 2026 · 2026-10-07 · tier 2 · checked 2026-10-20
4. BSI und TÜV-Verband (Feldarbeit: forsa): KI-Sicherheitsstudie 2026 · 2026-10-07 · tier 2 · checked 2026-10-20
5. BSI und TÜV-Verband (Feldarbeit: forsa): KI-Sicherheitsstudie 2026 · 2026-10-07 · tier 2 · checked 2026-10-20
6. Microsoft und LinkedIn: 2024 Work Trend Index Annual Report: AI at Work Is Here. Now Comes the Hard Part · 2024-05-08 · tier 3 · checked 2026-10-20
7. OWASP GenAI Security Project: LLM01:2025 Prompt Injection · 2024-11 · tier 2 · checked 2026-10-09
8. OWASP GenAI Security Project: LLM06:2025 Excessive Agency · 2024-11 · tier 2 · checked 2026-10-09
9. arXiv: Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (Greshake et al.) · 2023-02-23 · preprint, not peer-reviewed · tier 2 · checked 2026-10-20
10. NIST: NIST Identifies Types of Cyberattacks That Manipulate Behavior of AI Systems · 2024-01-04 · tier 2 · checked 2026-10-20
11. OECD: Agentic AI in organisations: Early insights from practitioner interviews (OECD AI Papers No. 65) · 2026-09 · tier 2 · checked 2026-10-09
12. Eurostat: The use of artificial intelligence in the European Union — Key results · 2026-03 · tier 1 · checked 2026-10-09
13. McKinsey & Company: The state of AI: How organizations are rewiring to capture value · 2025-03-12 · tier 2 · checked 2026-10-09
14. EUR-Lex: Regulation (EU) 2024/1689 (AI Act), Article 15: Accuracy, robustness and cybersecurity · 2024-07-12 · tier 1 · checked 2026-10-20
15. DARPA: AI Cyber Challenge marks pivotal inflection point for cyber defense · 2025-08-08 · tier 2 · checked 2026-10-20
16. IEEE S&P / arXiv: Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions (Pearce et al., IEEE Symposium on Security and Privacy 2022) · 2022-05 · tier 1 · checked 2026-10-20
17. FIRST: Exploit Prediction Scoring System (EPSS) · 2026 · tier 2 · checked 2026-10-20

Related

AI implementation services →How to implement AI in business →AI agents in business: what they are, what they can do and how to deploy them safely →AI business transformation: how AI changes the way you see your company — and what it can realistically deliver →EU AI Act timeline: what businesses must do after the 2026 changes →