93.1 Introduction
Security architecture is not finished when an application is deployed.
AI platforms continuously change:
New models are introduced.
New APIs are added.
New tools are connected.
New RAG sources appear.
New storage systems are deployed.
New cloud services are integrated.
New agent capabilities are enabled.
New dependencies enter the supply chain.
New tenants and users arrive.
New security threats emerge.
Because of this, an architecture that was secure yesterday may contain new attack paths tomorrow.
This chapter focuses on continuous security architecture review and attack-surface management.
The objective is to establish a process that continuously answers:
What is exposed, what is trusted, what has changed, what could be attacked, and are our controls still effective?
A mature AI platform therefore requires three connected capabilities:
Architecture Review → Attack-Surface Management → Continuous Security Validation
93.2 Security Architecture Review
A security architecture review evaluates whether the system's design provides appropriate protection before, during, and after implementation.
It should examine:
Identity
Authorization
APIs
Data
Models
RAG
Memory
Agents
Tools
Storage
Networking
Infrastructure
Dependencies
Logging
Monitoring
Incident response
Recovery
The review should consider both normal operation and failure conditions.
93.3 Architecture Review Objectives
A security architecture review should determine:
What assets require protection?
What trust boundaries exist?
What external systems are trusted?
Where does untrusted input enter?
Where does sensitive data flow?
Where can privileged actions occur?
What happens if a component is compromised?
What security controls exist?
What controls are independently enforced?
What assumptions remain unverified?
The goal is not merely to produce documentation.
The goal is to identify and reduce real security risk.
93.4 Security Architecture Review Lifecycle
A repeatable lifecycle is:
Architecture Proposal
|
v
System Inventory
|
v
Data-Flow Analysis
|
v
Trust-Boundary Analysis
|
v
Threat Modeling
|
v
Control Review
|
v
Security Testing
|
v
Risk Decision
|
v
Approval / Remediation
|
v
Continuous Monitoring
Architecture review should therefore connect directly with the development and release lifecycle.
93.5 Architecture Review Triggers
A new review should be considered when:
A new model is added
A model provider changes
A new agent is introduced
A new privileged tool is connected
A new RAG source is added
Database architecture changes
Object storage changes
Authentication changes
Authorization logic changes
New external APIs are integrated
Network boundaries change
Cloud infrastructure changes
Sensitive data processing changes
Major dependencies change
New autonomous functionality is introduced
Not every small UI change requires a full architecture review.
The review process should therefore support risk-based review depth.
93.6 Risk-Based Architecture Reviews
A useful classification is:
Low-risk change
Examples:
Possible review:
Medium-risk change
Examples:
Possible review:
High-risk change
Examples:
Possible review:
Formal threat modeling
Architecture review
Security testing
Approval gate
Critical change
Examples:
Authentication architecture change
Authorization redesign
Core tenant-isolation change
Highly privileged automation
Possible review:
93.7 Architecture Review Board
Larger platforms may establish a security architecture review group.
Potential participants:
The board should not become a bottleneck.
Its role is to provide governance for high-risk changes while enabling low-risk engineering work to move quickly.
93.8 Architecture Review Evidence
A security review should be supported by evidence.
Useful evidence includes:
Architecture diagrams
Data-flow diagrams
Threat models
IAM policies
Network policies
Database authorization rules
Storage policies
Model registry information
Tool permission definitions
Security test results
Dependency inventory
SBOM
Logging configuration
Monitoring coverage
Incident-response procedures
A claim such as:
"The system is secure."
is not useful without evidence.
A stronger statement is:
"Cross-tenant retrieval is prevented by authorization enforcement, database isolation, retrieval filtering, and automated security tests."
93.9 Architecture Security Principles
A secure AI architecture should follow several principles.
Least privilege
Components receive only required permissions.
Defense in depth
Multiple independent controls protect critical assets.
Fail closed
Security-sensitive decisions should fail safely when required information is unavailable.
Explicit trust
No component should be trusted merely because it is internal.
Separation of duties
Critical operations should not depend on a single unrestricted identity.
Strong isolation
Tenants, workloads, and sensitive processing should have appropriate boundaries.
Continuous verification
Security assumptions should be tested continuously.
93.10 Attack-Surface Management
Attack-surface management identifies and monitors the systems and capabilities that attackers could potentially reach.
Traditional attack surfaces include:
Domains
IP addresses
APIs
Servers
Containers
Databases
Cloud resources
AI introduces additional surfaces:
93.11 External Attack Surface
The external attack surface contains assets reachable from outside the trusted environment.
Examples:
Internet
|
+-- Web application
+-- API gateway
+-- Authentication
+-- Upload endpoint
+-- AI generation endpoint
+-- Webhooks
+-- Public assets
+-- External integrations
Every externally reachable component should have:
93.12 Internal Attack Surface
Internal systems can also be attacked after initial compromise.
Internal surfaces include:
The architecture should not assume:
"Internal means trusted."
Instead:
Internal access should still be authenticated, authorized, monitored, and constrained.
93.13 AI Attack-Surface Inventory
A mature inventory should track:
Asset
|
+-- Asset type
+-- Owner
+-- Environment
+-- Exposure
+-- Authentication
+-- Authorization
+-- Data classification
+-- Privileges
+-- Dependencies
+-- Security controls
+-- Monitoring
+-- Lifecycle status
For AI systems, asset types can include:
Model
Model endpoint
Prompt service
RAG index
Vector database
Agent
Tool
Memory store
Dataset
Evaluation service
93.14 Asset Discovery
Attack-surface management begins with accurate discovery.
Sources can include:
The objective is to detect unknown assets.
Unknown assets create unknown risk.
93.15 Shadow AI
Organizations may unintentionally deploy AI systems outside approved governance.
Examples include:
This is sometimes referred to as shadow AI.
The security response should focus on visibility and governance rather than simply assuming that every unapproved experiment is malicious.
93.16 AI Provider Attack Surface
External model providers introduce additional dependencies.
A provider relationship may involve:
Application
|
v
AI Gateway
|
v
Provider API
|
v
External Model
The review should determine:
What data is sent?
How is it protected?
What identity is used?
What happens if the provider is unavailable?
What logging is available?
What data-retention assumptions exist?
What contractual/security controls apply?
Can traffic be redirected safely?
Provider dependency should be treated as an architectural security consideration.
93.17 Third-Party Integration Review
Every external integration should be evaluated for:
Identity
How is the integration authenticated?
Permissions
What can it access?
Data
What information leaves the platform?
Availability
What happens if the service fails?
Integrity
Can it influence important decisions?
Monitoring
Can suspicious activity be detected?
Revocation
Can access be removed quickly?
93.18 Attack Surface Reduction
The best attack surface is often the one that does not exist.
Examples:
Remove unused APIs
Disable unused tools
Remove obsolete models
Delete unused service accounts
Close unused network paths
Remove unnecessary dependencies
Disable unused integrations
Reduce administrative interfaces
Remove obsolete cloud resources
Attack-surface reduction is often cheaper and more reliable than adding additional monitoring for unnecessary exposure.
93.19 Attack-Surface Prioritization
Not every exposed component has equal risk.
Prioritization can consider:
Internet exposure
Authentication strength
Privilege
Data sensitivity
Business criticality
Exploitability
Model/tool capabilities
Tenant impact
Detectability
Existing controls
A public AI endpoint capable of triggering privileged tools should receive substantially more attention than a public static image.
93.20 Critical Attack Paths
Security teams should identify critical attack paths.
Example:
Internet
|
v
Public API
|
v
Compromised Account
|
v
Agent
|
v
Privileged Tool
|
v
Sensitive Database
The individual components may each have reasonable controls.
The attack path may still be dangerous.
Therefore security review must ask:
Can an attacker move from an exposed surface to a high-impact asset?
93.21 Attack-Path Management
Critical attack paths should have:
Owner
Risk rating
Security controls
Detection coverage
Testing status
Remediation plan
Review date
A platform can then prioritize security engineering around the paths that matter most.
93.22 Continuous Security Validation
Traditional security validation often occurs periodically.
AI platforms benefit from continuous validation.
Continuous validation means repeatedly checking whether important security assumptions remain true.
Examples:
Tenant isolation
Authorization boundaries
Network segmentation
Tool permissions
Model integrity
RAG filtering
Logging
Security policies
Resource limits
93.23 Security Invariants
A security invariant is a property that should remain true regardless of normal system changes.
Examples:
A tenant cannot access another tenant's documents.
An agent cannot execute a tool outside its assigned permission set.
Production workloads cannot use development credentials.
Unapproved models cannot enter production.
Administrative actions are audited.
Security invariants are extremely valuable because they can be continuously tested.
93.24 Automated Security Validation
Automated validation can operate at multiple layers.
Code
Static analysis
Dependency scanning
Secret scanning
Infrastructure
Application
Authorization tests
Input-validation tests
API security tests
AI
Prompt-injection tests
RAG isolation tests
Tool-authorization tests
Model-policy tests
Runtime
93.25 Continuous Authorization Validation
Authorization should be continuously tested.
Important cases include:
User A -> User A data ALLOW
User A -> User B data DENY
Tenant A -> Tenant B data DENY
Normal user -> Admin API DENY
Agent A -> Tool A ALLOW
Agent A -> Tool B DENY
These tests should run automatically whenever authorization logic changes.
93.26 Continuous Tenant-Isolation Validation
Multi-tenant AI platforms should continuously test:
Database isolation
Object-storage isolation
Vector isolation
Cache isolation
Memory isolation
Search isolation
Logs
Queues
Background jobs
A single isolation test is not sufficient.
Isolation must remain true throughout the platform lifecycle.
93.27 Continuous RAG Security Validation
RAG security validation can test:
Access control
Can users retrieve only authorized documents?
Metadata isolation
Are tenant identifiers preserved?
Retrieval filtering
Are unauthorized chunks excluded?
Prompt boundaries
Is retrieved content treated as data?
Citation integrity
Does the system correctly identify sources?
Poisoning resistance
Can malicious content influence privileged behavior?
93.28 Continuous Agent Security Validation
Agent systems should be evaluated for:
A particularly important invariant is:
The agent must not be able to grant itself additional authority.
93.29 Continuous Model Integrity Validation
Production systems should periodically verify:
Model identity
Model version
Artifact hash
Deployment status
Registry status
Approval status
Configuration
Provider identity
Unexpected model changes should generate security events.
93.30 Configuration Drift
Security controls can weaken through configuration drift.
Examples:
A desired-state system should compare:
Desired Configuration
|
v
Actual Configuration
|
v
Difference
|
v
Risk Assessment
Unexpected differences should be investigated.
93.31 Security Drift Detection
Important drift categories include:
Identity drift
Unexpected permission changes.
Network drift
Unexpected connectivity.
Configuration drift
Security settings changed.
Model drift
Unexpected model artifact or version.
Policy drift
Guardrail or authorization rules changed.
Data drift
Unexpected sensitive data movement.
Tool drift
New or modified agent capabilities.
93.32 Security Control Validation
Security controls should be tested for actual effectiveness.
For example:
Claimed control
"Database access is restricted."
Validation
Attempt authorized and unauthorized access through automated tests.
Evidence
Record:
Test
Expected result
Actual result
Timestamp
Environment
Version
This converts security claims into measurable evidence.
93.33 Continuous Control Monitoring
Security teams can maintain a control-status model:
| Control | Status | Evidence |
|---|
| Tenant isolation | Passing | Automated tests |
| Model integrity | Passing | Artifact verification |
| Admin MFA | Passing | IAM validation |
| Tool authorization | Passing | Agent tests |
| Logging | Warning | Coverage gap |
| Network policy | Passing | Policy scan |
This makes security posture visible.
93.34 Security Validation Pipeline
A mature pipeline can look like:
Code / Config Change
|
v
Static Security Checks
|
v
Threat-Scenario Tests
|
v
AI Security Evaluation
|
v
Infrastructure Validation
|
v
Deployment
|
v
Runtime Validation
|
v
Continuous Monitoring
Security validation should therefore continue after deployment.
93.35 Security Regression Testing
Security behavior must not regress.
Example:
A release introduces a new search feature.
Previously:
Tenant A -> Tenant B document = DENY
After release, the same test must still produce:
DENY
Security regression tests should cover high-value invariants.
93.36 Security Chaos Engineering
Controlled failure testing can help determine whether security controls survive failures.
Examples:
Disable a policy service
Simulate a database failure
Interrupt model provider connectivity
Remove a service credential
Stop a logging component
Simulate queue delays
The objective is to verify safe behavior.
For example:
If authorization cannot be evaluated, does the system safely deny a privileged operation?
93.37 Security Failure Modes
Security architecture reviews should explicitly analyze failures.
Fail-open
A security control fails and access is allowed.
Potential risk:
Fail-closed
A security control fails and access is denied.
Potential downside:
The correct behavior depends on the operation.
High-impact authorization decisions generally require strong protection against fail-open behavior.
93.38 Security Validation During Canary Releases
Canary deployment can include security validation.
Example:
New Version
|
v
Small Traffic Segment
|
+---- Security tests
+---- Authorization tests
+---- AI safety tests
+---- Runtime monitoring
|
v
Decision
If security behavior degrades, rollout should stop.
93.39 Production Security Synthetic Tests
Synthetic security tests can periodically verify important paths.
Examples:
Login
Authorization
File upload
Document retrieval
Model invocation
Agent tool permission
Tenant isolation
Audit logging
These tests should use safe test accounts and controlled data.
93.40 Security Validation Evidence
Every important validation result should have:
Test ID
System
Version
Environment
Timestamp
Expected Result
Observed Result
Severity
Status
Evidence
Owner
This creates an auditable security-validation history.
93.41 Attack-Surface Metrics
Useful metrics include:
Unknown asset count
Number of assets without approved inventory records.
External exposure count
Number of externally reachable assets.
Critical exposure count
Number of high-impact exposed components.
Attack-path count
Number of known critical paths.
Remediation age
How long critical exposure remains unresolved.
Control coverage
Percentage of important assets protected by validated controls.
Validation coverage
Percentage of critical security invariants continuously tested.
93.42 Architecture Security Scorecard
A security scorecard can track:
| Category | Example |
|---|
| Identity | MFA, workload identity |
| Authorization | Least privilege |
| Data | Encryption, classification |
| AI | Model integrity |
| RAG | Tenant isolation |
| Agents | Tool permissions |
| Infrastructure | Isolation |
| Network | Segmentation |
| Supply chain | SBOM |
| Monitoring | Detection coverage |
| Recovery | Tested backups |
The scorecard should emphasize evidence rather than subjective confidence.
93.43 Security Debt
Security architecture can accumulate debt.
Examples:
Security debt should be tracked similarly to technical debt.
93.44 Security Exceptions
Sometimes a control cannot immediately be implemented.
An exception should contain:
Description
Business reason
Risk
Compensating controls
Owner
Approval
Expiration date
Review date
Exceptions should never become permanent undocumented architecture.
93.45 Compensating Controls
If one control is unavailable, another may reduce risk.
Example:
Missing network restriction
|
+-- Strong authentication
+-- Narrow authorization
+-- Monitoring
+-- Rate limiting
+-- Automated detection
Compensating controls should be explicitly documented and tested.
93.46 Architecture Review Documentation
A practical architecture security package may contain:
01-System-Overview
02-Architecture-Diagram
03-Data-Flow-Diagram
04-Trust-Boundaries
05-Asset-Inventory
06-Threat-Model
07-Attack-Paths
08-Security-Controls
09-Security-Tests
10-Open-Risks
11-Exceptions
12-Approval
13-Monitoring-Plan
14-Review-History
This provides a durable record of security decisions.
93.47 Continuous Security Validation Dashboard
A mature dashboard can show:
Security Architecture
|
+-- Current threat models
+-- Critical attack paths
+-- Unknown assets
+-- Exposed services
+-- Control failures
+-- Security regressions
+-- Configuration drift
+-- Model integrity
+-- Tenant-isolation status
+-- Agent security status
+-- Open exceptions
The dashboard should help engineers make decisions rather than simply display numbers.
93.48 Security Review Automation
Automation can automatically detect:
New public endpoints
New cloud resources
New service accounts
New model deployments
New tool permissions
New data flows
New external providers
Security policy changes
These events can trigger risk-based review.
93.49 Architecture Change Detection
An effective system compares architecture versions.
Architecture V1
|
v
Architecture V2
|
v
Change Detection
|
+---- New component
+---- New data flow
+---- New privilege
+---- New exposure
+---- New dependency
|
v
Security Review
This allows security teams to focus on what actually changed.
93.50 Security Validation of External AI Providers
External providers should be continuously evaluated for:
Availability
Authentication
Data handling
API behavior
Rate limits
Security incidents
Contractual changes
Model changes
Service changes
Provider changes can alter the security posture of the entire platform.
93.51 Continuous Model Governance
When a model changes, validation should verify:
Approved model identity
Expected capabilities
Security evaluation
Safety evaluation
Data-access permissions
Tool permissions
Cost limits
Latency behavior
Monitoring coverage
A model upgrade should not automatically inherit the security assumptions of the previous model.
93.52 Continuous AI Security Evaluation
Security validation should include recurring tests for:
Testing should evolve as new threats are identified.
93.53 Security Validation and Threat Intelligence
Threat intelligence should continuously influence validation.
New Threat
|
v
Threat Analysis
|
v
Security Scenario
|
v
Automated Test
|
v
Control Validation
|
v
Result
|
+---- Failure ---> Remediation
|
+---- Pass ------> Monitoring
This ensures that research becomes operational security.
93.54 Continuous Security Validation Maturity
Level 1 — Periodic
Security reviews happen occasionally.
Level 2 — Repeatable
Defined review processes and checklists exist.
Level 3 — Automated
Security validation runs automatically in CI/CD.
Level 4 — Continuous
Production systems continuously validate critical security invariants.
Level 5 — Adaptive
Threat intelligence, telemetry, architecture changes, testing, and controls form a continuous adaptive security loop.
93.55 Reference Architecture
A mature secure AI platform can use the following model:
SECURITY GOVERNANCE
|
+--------------+--------------+
| |
v v
Threat Intelligence Architecture Review
| |
+--------------+--------------+
|
v
Attack-Surface Inventory
|
v
Risk Analysis
|
v
Security Requirements
|
+--------------+--------------+
| |
v v
Security Controls Continuous Validation
| |
+--------------+--------------+
|
v
Production
|
v
Telemetry
|
v
Security Analytics
|
+--------------+--------------+
| |
v v
New Findings Architecture Changes
| |
+--------------+--------------+
|
v
Updated Security Model
This architecture treats security as a continuous engineering process.
93.56 Practical Implementation Strategy
A practical implementation can begin with five stages.
Stage 1 — Inventory
Create an inventory of:
Applications
APIs
Models
Agents
Tools
Databases
Storage
External providers
Cloud resources
Stage 2 — Map
Create:
Architecture diagrams
Data-flow diagrams
Trust boundaries
Critical attack paths
Stage 3 — Validate
Automate tests for:
Authorization
Tenant isolation
Model integrity
Tool permissions
Security configuration
Stage 4 — Monitor
Continuously monitor:
Exposure
Drift
New assets
New privileges
Security failures
Stage 5 — Improve
Use:
Threat intelligence
Incidents
Security testing
Production telemetry
to update architecture and controls.
93.57 Architecture Security Checklist
Architecture
AI
Exposure
Controls
Continuous validation
Governance
93.58 Key Metrics
A mature platform should monitor:
Percentage of inventoried assets
Unknown asset count
Public exposure count
Critical attack-path count
Security control coverage
Security test coverage
Security regression rate
Configuration drift rate
Model-integrity failures
Authorization-test failures
Tenant-isolation failures
Mean remediation time
Security exception age
Architecture-review completion rate
Metrics should be tied to actual risk reduction.
93.59 Common Architecture Review Mistakes
Mistake 1 — Reviewing only the application
Infrastructure, models, agents, and external providers are ignored.
Better approach
Review the complete AI system.
Mistake 2 — Treating diagrams as proof
A diagram may show intended architecture rather than actual architecture.
Better approach
Compare architecture documentation with deployed reality.
Mistake 3 — Testing only before release
Security may regress after deployment.
Better approach
Continuously validate critical security properties.
Mistake 4 — Ignoring attack chains
Individual components appear secure but can form a dangerous path.
Better approach
Model end-to-end attack paths.
Mistake 5 — Allowing permanent exceptions
Temporary risks become permanent weaknesses.
Better approach
Require expiration and periodic review.
Mistake 6 — Monitoring everything equally
Security teams become overwhelmed.
Better approach
Prioritize critical assets and attack paths.
93.60 Final Principles
A secure AI architecture should follow these principles:
Know what exists.
Know what is exposed.
Know what is trusted.
Know what data can flow where.
Know which identities have privileges.
Know which AI systems can take actions.
Know which attack paths can reach critical assets.
Continuously validate important security assumptions.
Detect architecture and configuration drift.
Turn security findings into engineering changes.
93.61 Final Architecture
The complete security architecture lifecycle can be summarized as:
THREAT INTELLIGENCE
|
v
THREAT MODELING
|
v
SECURITY ARCHITECTURE
|
v
ATTACK-SURFACE INVENTORY
|
v
ATTACK-PATH ANALYSIS
|
v
SECURITY CONTROLS
|
v
CONTINUOUS VALIDATION
|
v
RELEASE
|
v
PRODUCTION
|
v
TELEMETRY
|
v
SECURITY ANALYTICS
|
+----------+----------+
| |
v v
New Threats Architecture Drift
| |
+----------+----------+
|
v
SECURITY IMPROVEMENT
|
+-------> THREAT MODEL
Conclusion
Security architecture should not be treated as a one-time design document.
For AI platforms, the architecture changes continuously because models, agents, tools, data sources, integrations, infrastructure, and threat conditions continuously change.
Therefore, the security program must continuously answer:
What changed?
What became exposed?
What privileges changed?
What new data flows appeared?
What new attack paths exist?
Are security invariants still true?
Can the SOC detect violations?
Can the organization contain failures?
Can the architecture recover safely?
The strongest security architecture is therefore not simply one with many controls.
It is one where security assumptions are explicit, attack surfaces are known, critical attack paths are understood, controls are independently enforced, and important security properties are continuously tested.
A secure AI platform should continuously discover its attack surface, evaluate its architecture, validate its security invariants, detect drift, and convert new evidence into stronger controls.
Comments
Post a Comment