AI Security for Business Applications: How to Protect AI Systems
How to secure AI applications: threat model, prompt injection, tool permissions, data exposure, secrets, authorization, model and supply chain risks, runtime monitoring and AI incident response.
Quick answer
Secure AI applications by threat-modelling the new attack surface and applying layered controls. Treat all natural-language inputs, documents and tool results as untrusted; limit tools and data to least privilege and the user's own permissions; require approvals for consequential actions; validate model outputs before they reach HTML, SQL, commands or APIs; keep secrets out of prompts; vet models, packages and AI vendors; apply rate and cost limits; log and monitor runtime behaviour; and have an AI incident plan with kill switches.
Where This Fits
This is the hub for AI security. Deep dives: prompt injection prevention, AI agent guardrails, MCP security and AI data privacy. General web security is in website security checklist.
The AI Threat Model
A step-by-step method is in AI application threat modeling, and a test checklist in AI security testing.
| Threat | Example | Primary control |
|---|---|---|
| Prompt injection | A document instructs the assistant to leak data | Least privilege, untrusted-content handling, approvals |
| Sensitive data exposure | Model reveals another user's data | Permission-aware retrieval, tenant isolation |
| Excessive agency | Agent with admin rights deletes records | Narrow tools, policy checks |
| Insecure output handling | Model output rendered as HTML with scripts | Output encoding and validation |
| Supply chain | Compromised model weights or packages | Vetted sources, pinned versions |
| Secrets leakage | API keys in prompts or logs | Secrets manager, redaction |
| Unbounded consumption | Requests crafted to run up costs | Rate limits, budgets |
Securing Tools and Data Access
Most serious AI incidents involve what the AI could reach. Give each AI feature the minimum tools and data, act with the signed-in user's permissions, enforce policies inside tools, separate read and write capabilities and require confirmation for writes, payments, deletions and external messages. For tool connections through MCP, follow the specification's authorization requirements.
Shipping AI features and worried about security?
ZSpace Labs reviews and builds AI applications with threat models, least-privilege tools and runtime controls.
Runtime Monitoring and Incident Response
- Log inputs, tool calls and outputs with redaction
- Alert on unusual tool use, data volumes, policy denials and cost spikes
- Kill switches per feature, tool and model
- Playbooks for prompt injection, data exposure and runaway costs
- Post-incident tests added to evaluation sets
Vendor and Model Risk
Assess AI vendors for security practices, data retention and training use, subprocessors, regions and incident notification. Pin model versions where possible and re-evaluate when providers update models. Verify open-weight models and packages from trusted sources, and watch for hallucinated package names suggested by coding assistants.
Model provenance, safe formats and AI bills of materials are covered in AI supply chain security.
Advantages and Limitations
Layered AI security makes it possible to deploy AI features with confidence and to contain problems quickly. No control fully prevents model manipulation; the goal is limiting impact and detecting misuse. Security reviews must be part of AI delivery, not an afterthought.
How to Secure an AI Application Step by Step
- 1. Threat-model inputs, data, tools and outputs
- 2. Minimize tools and data access
- 3. Add policy checks and approvals
- 4. Validate and encode outputs
- 5. Protect secrets and vet dependencies
- 6. Add rate limits and budgets
- 7. Monitor, test adversarially and rehearse incidents
Security Testing for AI
- Adversarial evaluation cases: direct and indirect prompt injection
- Attempts to access other users' or tenants' data
- Attempts to trigger unauthorized tool actions
- Output injection tests (scripts, SQL, commands in model output)
- Cost abuse tests: very long inputs, repeated expensive calls
- Periodic red-team exercises by people outside the build team
AI in the Secure Development Lifecycle
Add AI-specific steps to existing practices: threat modelling covers model inputs, tools and outputs; design reviews check permissions and approvals; code review checks output handling and secrets; testing includes adversarial evaluation; release checks monitoring and kill switches; operations includes AI incident playbooks. Privacy and governance reviews run alongside; see AI data privacy and AI governance.
Agents and Tool Permissions
Security risk rises sharply when AI can take actions. An assistant that only answers questions can leak information; an agent with tools can change records, send messages and move money. Apply least privilege to every tool: scope credentials to the specific operations and data needed, act as the requesting user where possible and require human approval for consequential actions.
Treat all content the agent reads, including emails, web pages, documents and tool outputs, as untrusted, because it may contain instructions designed to hijack the agent. Validate tool arguments against schemas and business rules outside the model. Our guide to prompt injection prevention covers these defences in more depth.
Frameworks and Standards
Several resources help structure AI security work. The OWASP Top 10 for LLM Applications lists common vulnerability classes such as prompt injection, sensitive information disclosure and excessive agency. MITRE ATLAS catalogues adversary tactics against AI systems. NIST's AI Risk Management Framework and its Generative AI Profile (NIST AI 600-1) cover security alongside broader risks.
Use these as checklists and shared vocabulary within existing security programmes rather than as separate processes. Map controls to your threat model, test them and record results. Governance links are covered in AI governance framework.
Sources: OWASP Top 10 for LLM Applications, MITRE ATLAS and NIST AI 600-1.
Data Leakage Through Outputs
AI applications can leak data through their answers: retrieving documents the user should not see, revealing other users' data in shared contexts, exposing system prompts or reproducing sensitive training or fine-tuning data. Enforce permissions at retrieval, isolate tenants and sessions, avoid placing secrets in prompts and test for leakage with adversarial queries.
Output channels matter too. Rendering model output as HTML or markdown with external images can exfiltrate data through URLs, so sanitize output and restrict external content. Privacy controls are covered in AI data privacy.
Security Checklist for Launch
- Threat model reviewed and mitigations implemented
- Permissions enforced outside the model for data and tools
- Adversarial tests passed, including indirect prompt injection
- Secrets absent from prompts and logs
- Rate limits, budgets and kill switch in place
- Monitoring and incident runbook ready
Worked Example
An illustrative scenario, not a client case: a support assistant can read tickets and issue account credits. A red-team test plants an instruction in a ticket asking for a large credit to an attacker's account. The model attempts it, but the credit tool only applies credits to the ticket's own customer within a limit, and the attempt triggers an alert. The team adds the case to evaluations and tightens monitoring.
Common Mistakes
- Relying on prompts for security
- Over-privileged service accounts for AI
- Rendering model output without encoding
- Secrets in prompts or logs
- No kill switches or incident playbooks
Want an AI security review?
Talk to ZSpace Labs about secure AI development and application security engineering.
Conclusion
AI security is application security plus new inputs, new actors and new failure modes. Limit access, validate outputs, monitor runtime and prepare for incidents. Related: prompt injection and guardrails.
Common questions
Prompt injection, sensitive data exposure, excessive agency through over-privileged tools, insecure handling of model outputs, supply chain risks in models and packages, secrets exposure, denial of service through costly requests and system prompt leakage.