67.1 Introduction
A production AI platform is not a single application.
It is usually a distributed system containing:
These components communicate continuously.
Therefore, network security becomes one of the most important parts of the overall AI security architecture.
A secure network design should answer:
Who is communicating?
What service is being contacted?
Is the connection authenticated?
Is the destination authorized?
Is the traffic encrypted?
Is the destination private or public?
Can the service make arbitrary outbound connections?
Can one tenant reach another tenant's resources?
What happens if a service is compromised?
Can suspicious traffic be detected?
The core principle is:
Network location should never be treated as proof of trust.
Being inside a private subnet does not automatically make a service trustworthy.
67.2 Traditional Network Trust vs Zero Trust
Traditional architectures often assume:
Internet
↓
Firewall
↓
Private Network
↓
Trusted Services
This creates a large implicit trust boundary.
A zero-trust architecture instead assumes:
Service A
↓
Authenticate
↓
Authorize
↓
Encrypt
↓
Service B
Every important service interaction receives explicit security treatment.
67.3 AI Application Network Model
A production AI platform may look like:
Internet
│
▼
┌─────────────┐
│ CDN / WAF │
└──────┬──────┘
│
▼
┌─────────────┐
│ API Gateway │
└──────┬──────┘
│
┌───────────┴───────────┐
▼ ▼
Application Services Auth Services
│
┌────────┼────────┐
▼ ▼ ▼
Database Queue Cache
│
▼
Workers
│
┌────────┼────────┐
▼ ▼ ▼
AI APIs Storage External APIs
Each connection represents a potential security boundary.
67.4 TLS
Transport Layer Security protects network communications from interception and tampering.
A simplified flow is:
Service A
│
│ TLS
▼
Service B
TLS should be considered for:
browser → API
API → authentication service
API → database
API → cache
worker → queue
application → AI provider
service → object storage
service → notification provider
The exact implementation depends on the infrastructure.
67.5 HTTPS
Public web traffic should use HTTPS.
The basic architecture is:
Browser
│
HTTPS
▼
Gateway
│
HTTPS / private secure channel
▼
Application
HTTP should not be used for sensitive production communication merely because the traffic occurs inside a private network.
67.6 Certificate Validation
TLS is only useful when certificate validation is performed correctly.
Applications should validate:
certificate chain
hostname
expiration
trust configuration
Disabling certificate verification is dangerous.
Avoid patterns equivalent to:
verify_tls = false
in production.
67.7 Mutual TLS
For sensitive service-to-service communication, mutual TLS can provide stronger service identity.
Ordinary TLS:
Client ────────► Server
server
authenticates
mTLS:
Client ◄──────► Server
│ │
└─ authenticate ─┘
Both sides present certificates.
This can help establish:
67.8 Service Identity
Every important service should have a distinct identity.
For example:
api-service
worker-service
billing-service
notification-service
media-service
ai-orchestrator
Avoid using one universal credential for every internal component.
If one service is compromised, unique identities reduce the blast radius.
67.9 Service-to-Service Authorization
Authentication answers:
Who are you?
Authorization answers:
What are you allowed to do?
For example:
worker-service
│
├── allowed → generation-service
├── allowed → object-storage
└── denied → billing-admin
A successful network connection should not automatically grant unrestricted application access.
67.10 East-West Traffic
North-south traffic generally refers to:
Internet ↔ application
East-west traffic refers to:
service ↔ service
Modern AI systems can have enormous amounts of east-west traffic.
Therefore, internal traffic deserves serious security controls.
A compromised worker should not automatically be able to scan or communicate with every internal service.
67.11 Network Segmentation
A useful architecture separates services into logical zones.
Public Zone
└── Gateway
Application Zone
├── API
├── Auth
└── AI Orchestrator
Processing Zone
├── Workers
└── Media Processing
Data Zone
├── Database
├── Cache
└── Queue
Storage Zone
└── Object Storage
Network rules should explicitly define permitted communication paths.
67.12 Least-Privilege Networking
Instead of:
API → everything
Worker → everything
prefer:
API
├── Database
├── Cache
└── Queue
Worker
├── Queue
├── Database
├── Object Storage
└── AI Provider
The worker should not need unrestricted access to unrelated services.
67.13 Egress Control
One of the most important controls for AI systems is outbound traffic restriction.
A compromised AI worker might otherwise attempt:
Worker
├── Internet
├── Internal services
├── Metadata endpoints
└── arbitrary external hosts
A safer design is:
Worker
│
▼
Egress Gateway
│
├── Approved AI provider
├── Approved storage
└── Approved APIs
This dramatically reduces the impact of server-side request attacks.
67.14 SSRF
Server-Side Request Forgery occurs when an attacker influences a server into making a network request to an unintended destination.
AI applications are particularly relevant because they often process:
URLs
documents
web content
image URLs
API integrations
plugins
tool calls
A dangerous architecture might allow:
User
↓
AI Agent
↓
fetch arbitrary URL
The AI system should never automatically receive unrestricted network access.
67.15 SSRF Defense
Use multiple layers:
User URL
↓
URL parser
↓
scheme validation
↓
hostname validation
↓
DNS resolution
↓
private/reserved IP rejection
↓
allowlist
↓
egress proxy
↓
request
The important point is that URL validation should not rely on a single string comparison.
67.16 Private IP Protection
Outbound fetchers should carefully restrict access to private and special-purpose network ranges.
Examples include:
An attacker should not be able to convert:
fetch URL
into:
access internal infrastructure
67.17 DNS Security
DNS is an important part of network security.
Applications should consider:
A particularly important issue is DNS rebinding.
A hostname might resolve to one address during validation and another address during the actual request.
Therefore, validation and connection handling should be designed together.
67.18 URL Allowlisting
Where practical, external integrations should use explicit allowlists.
For example:
Approved:
api.provider.example
storage.example
notifications.example
rather than:
Any Internet destination
This is especially important for:
AI tools
web fetchers
plugins
autonomous agents
document importers
67.19 AI Provider Connections
AI orchestration services often communicate with multiple providers.
For example:
AI Orchestrator
│
├── Provider A
├── Provider B
├── Provider C
└── Local Model
Each provider connection should have:
dedicated credentials
TLS
timeout
rate limit
retry policy
request-size limit
output validation
monitoring
67.20 Provider Failover
Suppose Provider A becomes unavailable.
A failover architecture might be:
Request
↓
Provider A
│
└── failure
↓
Policy Engine
↓
Provider B
However, failover should not accidentally transfer data to a provider that is not authorized to receive it.
Before failover, consider:
data classification
tenant policy
regional restrictions
model policy
provider approval
privacy requirements
67.21 Network Timeouts
Every outbound request should have bounded timeouts.
Without timeouts:
Worker
↓
external API
↓
hangs
↓
worker remains occupied
Repeated failures can eventually exhaust worker capacity.
Use separate limits for:
connection timeout
request timeout
response timeout
total workflow timeout
67.22 Connection Pooling
Connection pools improve performance but must be bounded.
An unbounded pool can create:
too many requests
↓
too many connections
↓
resource exhaustion
Configure maximum connection counts appropriate to:
service capacity
database capacity
provider limits
workload characteristics
67.23 Network Rate Limiting
Rate limiting can be applied at multiple levels:
Client
↓
Gateway
↓
Service
↓
Provider
Examples:
Distributed rate-limit state can be implemented using the architecture discussed in Chapter 66.
67.24 API Gateway
The API gateway should provide a strong perimeter.
Typical responsibilities include:
TLS termination
authentication
request validation
rate limiting
routing
request size limits
WAF integration
observability
abuse controls
But authorization should still be enforced inside the application.
The gateway should not become the only security layer.
67.25 WAF
A Web Application Firewall can help identify and block common malicious traffic patterns.
Possible protections include:
WAF rules should supplement application-level validation rather than replace it.
67.26 Internal APIs
Internal APIs should use authentication and authorization just like external APIs.
Avoid:
/internal/delete-user
being protected only by the assumption:
Only internal services can access it.
Instead:
Service identity
+
authorization policy
+
validated request
should be required.
67.27 Service Mesh
A service mesh can centralize some service-to-service controls.
Conceptually:
Application A
│
▼
Sidecar / Proxy
│
▼
Network
│
▼
Sidecar / Proxy
│
▼
Application B
Possible capabilities include:
mTLS
service identity
traffic policy
retries
observability
authorization
A service mesh can simplify large deployments but also adds operational complexity.
67.28 Kubernetes Networking
In Kubernetes, network policies can restrict which pods communicate.
Conceptually:
API Pod
│
├── allowed → DB
├── allowed → Redis
└── allowed → Queue
API Pod
└── denied → unrelated workload
Default-deny network policies are often a useful security baseline when operationally practical.
67.29 Namespace Isolation
Kubernetes namespaces can separate workloads logically.
Example:
production
├── api
├── workers
└── ai
monitoring
└── observability
security
└── scanners
Namespaces are useful organizational boundaries, but they should not be treated as complete security boundaries by themselves.
67.30 Container Network Security
Containers should have:
A compromised container should have as few paths as possible to reach sensitive infrastructure.
67.31 Database Network Access
Databases should not be publicly reachable.
Preferred:
Internet
X
│
Database
Application
│
▼
Private Database
Only authorized application services should have network access.
Database authorization still needs to be enforced at the database/application level.
67.32 Cache Network Access
Similarly:
Internet
X
Redis
Redis should reside in a private network.
Application services communicate through controlled network paths.
67.33 Object Storage Network Access
Object storage may use public cloud endpoints, but application access should still be controlled through:
Network security is one layer of the storage security model.
67.34 Private Connectivity
Cloud platforms often provide private connectivity mechanisms.
Examples conceptually include:
Application Network
│
▼
Private Endpoint
│
▼
Managed Service
This can reduce exposure to the public Internet.
67.35 Network Segmentation for Media Processing
Media processing is especially important because uploaded files are untrusted.
A strong architecture is:
Upload
↓
Quarantine Storage
↓
Scanner
↓
Isolated Processing Worker
↓
Validated Output
↓
Trusted Storage
The processing worker should have minimal network access.
It may need:
read quarantine
write output
but not:
access production database administration
access internal secrets
access unrelated services
67.36 AI Agent Network Permissions
Autonomous agents introduce a special problem.
An agent might have tools such as:
web_fetch
email_send
file_read
database_query
api_call
Each tool should have its own network policy.
For example:
Agent
│
├── web_fetch → Internet proxy
├── file_read → approved storage
├── database → restricted API
└── email → notification service
Do not give the agent unrestricted network connectivity merely because it may need several tools.
67.37 Network Access as a Tool Permission
A useful security model is:
Tool
↓
Network destination
↓
Allowed method
↓
Allowed data class
↓
Allowed rate
For example:
web_fetch
├── HTTPS only
├── approved domains
├── no private IPs
├── response size limit
└── timeout
This creates a much safer agent architecture.
67.38 Prompt Injection and Network Access
Prompt injection becomes significantly more dangerous when the AI agent has network privileges.
Consider:
Untrusted document
↓
Prompt injection
↓
AI agent
↓
network tool
The malicious document may attempt to persuade the model to:
The defense is not merely "make the model smarter."
Instead:
Model decision
↓
Policy enforcement
↓
Network authorization
↓
Tool execution
The network layer must independently enforce boundaries.
67.39 Data Exfiltration Controls
Outbound traffic should be monitored for unusual patterns.
Potential indicators:
Detection should combine:
network telemetry
+
application logs
+
identity information
+
request context
67.40 DNS and Domain Monitoring
For systems with external network access, maintain visibility into:
requested domains
resolved IPs
service identity
destination port
request volume
response volume
This can help detect compromised workloads communicating with unexpected infrastructure.
67.41 Network Encryption vs Application Encryption
TLS protects data during transport.
Application-level encryption may protect sensitive data beyond the transport layer.
For highly sensitive information, consider whether encryption should occur:
Application
↓
encrypted payload
↓
TLS
↓
Network
This provides defense in depth.
67.42 Certificate Rotation
Certificates should not live indefinitely.
A production system should support:
certificate issued
↓
deployed
↓
monitored
↓
rotated
↓
old certificate retired
Automation is strongly preferable to manual certificate replacement.
67.43 Credential Rotation
The same principle applies to service credentials.
Credentials should be:
Avoid embedding service credentials in source code.
67.44 Network Observability
Important telemetry includes:
source service
destination service
source identity
destination identity
protocol
port
bytes
latency
result
authentication status
authorization result
This helps reconstruct incidents.
67.45 Incident Response
Suppose a worker is compromised.
A strong architecture should allow:
Compromised Worker
↓
Detect
↓
Isolate network access
↓
Revoke identity
↓
Stop workloads
↓
Rotate credentials
↓
Investigate
↓
Recover
Network isolation is one of the most powerful containment mechanisms.
67.46 Kill Switch
Critical AI infrastructure should have mechanisms to disable dangerous capabilities quickly.
For example:
Emergency Policy
↓
Disable external network access
↓
Disable autonomous tools
↓
Pause workers
↓
Preserve evidence
This should be tested before an incident occurs.
67.47 Failure Containment
A secure distributed system assumes components can fail or become compromised.
Therefore:
Service A compromised
↓
Service B remains protected
↓
Database remains protected
↓
Secrets remain protected
↓
Incident contained
This is the practical goal of segmentation and least privilege.
67.48 Secure Networking Checklist
[ ] Public services use HTTPS
[ ] TLS certificate validation is enabled
[ ] Sensitive internal connections are encrypted
[ ] Service identities are unique
[ ] Service-to-service authorization exists
[ ] Internal APIs require authentication
[ ] Network zones are defined
[ ] Database is private
[ ] Redis is private
[ ] Queue infrastructure is private
[ ] Worker networks are restricted
[ ] Egress is controlled
[ ] SSRF protections exist
[ ] Private IP access is blocked where appropriate
[ ] DNS security is considered
[ ] External domains are allowlisted where practical
[ ] AI provider connections are controlled
[ ] Timeouts exist
[ ] Connection pools are bounded
[ ] Rate limits exist
[ ] Network telemetry is collected
[ ] Suspicious outbound traffic can be detected
[ ] Credentials can be rotated
[ ] Certificates can be rotated
[ ] Compromised services can be isolated
[ ] Emergency network controls exist
67.49 Reference Zero-Trust Architecture
A mature AI platform can use:
INTERNET
│
▼
┌─────────────┐
│ CDN / WAF │
└──────┬──────┘
│
TLS
│
▼
┌─────────────┐
│ API Gateway │
└──────┬──────┘
│
Identity Check
│
Authorization
│
▼
┌──────────────────┐
│ Application Mesh │
└────────┬─────────┘
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
API Workers AI Service
│ │ │
│ │ │
▼ ▼ ▼
Database Queue Egress Proxy
│ │
▼ ▼
Cache Approved APIs
│
▼
Object Storage
Each major connection should have:
identity
+
encryption
+
authorization
+
network policy
+
monitoring
67.50 Final Principle
Network security for AI systems is no longer simply:
"Put the servers behind a firewall."
A modern AI platform must assume that:
users can be malicious
files can be malicious
prompts can be malicious
AI outputs can be manipulated
workers can be compromised
internal services can fail
credentials can leak
agents can make incorrect decisions
Therefore, network controls must remain effective even when an application component behaves incorrectly.
The strongest architecture is:
Identity
↓
Authentication
↓
Authorization
↓
Encrypted Connection
↓
Network Segmentation
↓
Least-Privilege Access
↓
Egress Control
↓
Application Validation
↓
Monitoring
↓
Containment
The key principle is:
Every network connection is an authorization decision, not merely a routing decision.
This becomes especially important when AI agents, external model providers, web tools, plugins and autonomous workflows are introduced.
The next logical chapter is:
Chapter 68 — Secure AI Secrets & Key Management: API Keys, Encryption Keys, KMS, Vault Architecture, Secret Rotation, Credential Isolation, Key Hierarchies, Envelope Encryption, HSMs, Provider Credentials & Emergency Revocation.
Comments
Post a Comment