SHADOW AI

Chapter 26 created the first working AI-chat foundation.
Now we connect that prototype to real application infrastructure:
USER
↓
SIGN UP
↓
LOGIN
↓
SESSION
↓
DASHBOARD
↓
CHAT
↓
DATABASE
↓
CONVERSATION
↓
MESSAGE HISTORY
↓
AI
The goal is to make ACAI multi-user and persistent.
By the end of this chapter, ACAI should have the architecture for:
[✓] User accounts
[✓] Authentication
[✓] Sessions
[✓] Protected pages
[✓] Protected APIs
[✓] Database
[✓] User ownership
[✓] Projects
[✓] Conversations
[✓] Messages
[✓] Persistent chat history
[✓] Logout
[✓] Authorization checks
For the application layer, a relational database is a strong starting point.
Conceptually:
ACAI
↓
ORM / DATABASE CLIENT
↓
POSTGRESQL
A managed PostgreSQL service can be used in development and production.
The exact provider can be selected later.
Instead of writing raw database queries throughout the application, use a consistent database access layer.
Architecture:
API
↓
SERVICE
↓
ORM / DATABASE CLIENT
↓
POSTGRESQL
This keeps database logic organized.
The first database version needs:
User
Project
Conversation
Message
Later models can include:
File
Generation
Job
Usage
AuditEvent
Subscription
Conceptual structure:
User
--------------------------------
id
email
name
role
status
createdAt
updatedAt
Example roles:
USER
ADMIN
More roles can be introduced if required.
Project
--------------------------------
id
userId
name
description
createdAt
updatedAt
Relationship:
USER
│
├── PROJECT 1
├── PROJECT 2
└── PROJECT 3
Conversation
--------------------------------
id
userId
projectId
title
createdAt
updatedAt
Relationship:
PROJECT
│
├── Conversation A
├── Conversation B
└── Conversation C
Message
--------------------------------
id
conversationId
role
content
createdAt
Roles:
USER
ASSISTANT
SYSTEM
Additional message metadata can be introduced later.
USER
│
├──────────────┐
▼ ▼
PROJECT CONVERSATION
│ │
│ └──────► MESSAGE
│
└──────────────► CONVERSATION
The exact relationship should enforce the intended ownership rules.
Every private resource must belong to the correct user.
For example:
User A
└── Conversation A
User B
└── Conversation B
User A must not be able to retrieve:
Conversation B
by simply changing an ID.
Authentication becomes:
USER
↓
SIGN UP / LOGIN
↓
AUTH PROVIDER
↓
SESSION
↓
ACAI
The application does not need to implement cryptographic authentication from scratch.
Use a mature authentication solution compatible with the selected Next.js architecture.
The signup process:
SIGN UP
↓
VALIDATE INPUT
↓
CHECK ACCOUNT
↓
CREATE USER
↓
CREATE SESSION
↓
REDIRECT
↓
DASHBOARD
EMAIL / AUTH METHOD
↓
AUTHENTICATE
↓
CREATE SESSION
↓
DASHBOARD
If authentication fails:
INVALID CREDENTIALS
↓
USER-FRIENDLY ERROR
Do not expose unnecessary information about whether a particular account exists.
USER
↓
LOGOUT
↓
SESSION REVOKED / INVALIDATED
↓
PUBLIC PAGE
After logout, private API requests must no longer be authorized through that session.
A session connects the authenticated browser to the authenticated user.
Conceptually:
SESSION
├── userId
├── expiration
└── security metadata
The application should use secure session handling appropriate to the chosen authentication framework.
The dashboard should require authentication.
OPEN /dashboard
│
▼
AUTH CHECK
│
┌───┴────┐
▼ ▼
LOGGED NOT LOGGED
│ │
▼ ▼
DASHBOARD LOGIN
The same rule applies to APIs.
POST /api/chat
│
▼
SESSION CHECK
│
┌─────┴─────┐
▼ ▼
VALID INVALID
▼ ▼
PROCESS 401
The frontend cannot be the only protection.
Authentication answers:
WHO ARE YOU?
Authorization answers:
WHAT ARE YOU ALLOWED TO DO?
Example:
USER
↓
REQUEST PROJECT
↓
CHECK PROJECT OWNER
↓
AUTHORIZED?
├── YES → CONTINUE
└── NO → DENY
Create a centralized database module.
Conceptually:
src/lib/db/
The application services use it:
API
↓
SERVICE
↓
DB MODULE
↓
DATABASE
Do not create a new database connection unnecessarily for every request.
Use the database access pattern recommended for the selected runtime and deployment model.
The database connection belongs in server-side environment configuration.
Conceptually:
DATABASE_URL=...
Do not expose the database connection string to the browser.
Database structure will change over time.
Use migrations:
SCHEMA
↓
MIGRATION
↓
DATABASE
Example evolution:
Version 1
User
Version 2
User + Project
Version 3
User + Project + Conversation
Version 4
Messages
This gives the database a reproducible history.
Development can use seed data.
Example:
DEVELOPMENT
↓
CREATE TEST USER
↓
CREATE TEST PROJECT
↓
CREATE TEST CONVERSATION
↓
CREATE TEST MESSAGES
Never accidentally use development seed credentials in production.
The signup endpoint should validate:
Email
Password if applicable
Name
Required fields
Example rules:
Email
↓
valid format?
Password
↓
meets minimum requirements?
Name
↓
valid length?
If an email is already associated with an account, handle the situation through the authentication system's intended behavior.
Avoid leaking unnecessary account-existence information through overly specific error messages.
If password-based authentication is implemented:
PASSWORD
↓
SECURE HASH
↓
DATABASE
Never:
PASSWORD
↓
PLAIN TEXT DATABASE
Do not invent a custom hashing algorithm.
Use the authentication framework or a well-established password hashing mechanism.
For public production applications, email verification can be added:
SIGN UP
↓
VERIFICATION EMAIL
↓
USER CLICKS
↓
ACCOUNT VERIFIED
Whether verification is mandatory depends on the application's requirements.
A production account system should provide a secure recovery process:
FORGOT PASSWORD
↓
VERIFY USER
↓
TIME-LIMITED RESET FLOW
↓
NEW PASSWORD
↓
OLD SESSION HANDLING
Never send passwords through email.
Once authenticated:
SESSION
↓
USER ID
↓
DATABASE
↓
USER DATA
↓
DASHBOARD
The dashboard can display:
User name
Recent conversations
Projects
Recent generations
Usage
The flow:
DASHBOARD
↓
NEW PROJECT
↓
PROJECT NAME
↓
API
↓
AUTH
↓
DATABASE
↓
PROJECT CREATED
The server must associate the new project with the authenticated user's ID.
PROJECT
↓
NEW CHAT
↓
CREATE CONVERSATION
↓
DATABASE
↓
CHAT PAGE
The conversation should automatically inherit the correct user ownership.
Now the earlier chat system becomes:
USER
↓
CHAT UI
↓
AUTHENTICATED API
↓
LOAD CONVERSATION
↓
CHECK OWNERSHIP
↓
SAVE USER MESSAGE
↓
AI GATEWAY
↓
MODEL
↓
SAVE ASSISTANT MESSAGE
↓
RETURN
This is the first important persistent AI workflow.
Before:
USER
↓
AI
↓
SCREEN
After:
USER
↓
DATABASE
↓
AI
↓
DATABASE
↓
SCREEN
The conversation survives refreshes and future sessions.
When opening:
/chat/abc123
the server should:
SESSION
↓
USER ID
↓
CONVERSATION ID
↓
OWNERSHIP CHECK
↓
LOAD MESSAGES
↓
DISPLAY
If the conversation does not belong to the user:
403 / NOT FOUND
according to the application's chosen authorization/error strategy.
A scalable structure can be:
/chat
for the chat landing page and:
/chat/[conversationId]
for a specific conversation.
Conceptually:
/chat
│
├── New conversation
│
└── Recent conversations
/chat/abc123
│
└── Conversation abc123
The interface can become:
┌─────────────────────────────────────────────┐
│ ACAI │
├───────────────┬─────────────────────────────┤
│ Conversations │ Conversation │
│ │ │
│ + New Chat │ User: Hello │
│ │ AI: Hello! │
│ Project A │ │
│ Project B │ User: Explain AI │
│ Project C │ AI: ... │
│ │ │
├───────────────┴─────────────────────────────┤
│ Ask ACAI... [Send] │
└─────────────────────────────────────────────┘
Messages should have a reliable ordering mechanism.
Typical approach:
createdAt
or another sequence field.
Then:
MESSAGE 1
MESSAGE 2
MESSAGE 3
...
always appears in the intended order.
Some model workflows may require system instructions.
These should be separated conceptually from user-generated messages.
SYSTEM
↓
USER
↓
ASSISTANT
System instructions should not be editable through ordinary user message controls.
When generating a response:
DATABASE
↓
RECENT MESSAGES
↓
CONTEXT BUILDER
↓
AI GATEWAY
↓
MODEL
Do not blindly send unlimited conversation history.
Long conversations eventually exceed the model's context capacity.
Possible strategy:
OLD MESSAGES
↓
SUMMARY
↓
RECENT MESSAGES
↓
MODEL
This can later connect to ACAI memory.
A secure conceptual query is:
Find conversation where:
conversation.id = requestedId
AND conversation.userId = authenticatedUserId
Not:
Find conversation where:
conversation.id = requestedId
The second pattern can create an authorization vulnerability if ownership is checked elsewhere incorrectly.
The complete security boundary is:
REQUEST
↓
SESSION
↓
USER ID
↓
RESOURCE QUERY
↓
OWNER CHECK
↓
RESOURCE
This pattern should be repeated for:
Projects
Conversations
Messages
Files
Generations
Usage
Some operations involve multiple writes.
Example:
CREATE CONVERSATION
+
CREATE FIRST MESSAGE
A transaction can keep related writes consistent.
Conceptually:
BEGIN
↓
WRITE A
↓
WRITE B
↓
COMMIT
If something fails:
ROLLBACK
Suppose:
USER MESSAGE SAVED
↓
AI PROVIDER FAILS
Do not pretend the assistant generated an answer.
Instead:
USER MESSAGE
↓
AI ERROR
↓
CONTROLLED FAILURE STATE
The database can record the failed generation if useful.
Each request can eventually create:
Usage
----------------
id
userId
model
provider
inputTokens
outputTokens
latency
createdAt
This allows:
Usage tracking
Quotas
Cost estimation
Analytics
The complete persistent chat flow is now:
1. USER OPENS CHAT
↓
2. SESSION VERIFIED
↓
3. CONVERSATION LOADED
↓
4. USER WRITES MESSAGE
↓
5. API REQUEST
↓
6. AUTHORIZATION
↓
7. VALIDATION
↓
8. SAVE USER MESSAGE
↓
9. AI GATEWAY
↓
10. MODEL
↓
11. SAVE ASSISTANT MESSAGE
↓
12. SAVE USAGE
↓
13. RETURN RESPONSE
↓
14. DISPLAY
PUBLIC
├── /
├── /login
└── /signup
PRIVATE
├── /dashboard
├── /chat
├── /documents
├── /image
├── /video
└── /settings
The actual routing structure can vary, but the security boundary should remain explicit.
The first settings page can contain:
Account
Security
Appearance
AI Preferences
Usage
Later:
Billing
API Keys
Integrations
Privacy
Example:
Name
Email
Profile
Account status
Sensitive changes should require appropriate re-authentication or verification depending on the operation.
Initial:
USER
Later:
USER
ADMIN
MODERATOR
ORGANIZATION_ADMIN
Do not give ordinary users administrative permissions.
Admin API:
REQUEST
↓
AUTH
↓
ROLE CHECK
↓
ADMIN?
├── YES → CONTINUE
└── NO → DENY
The frontend hiding an admin button is not security.
The backend must enforce the role.
The AI API should know:
userId
This allows:
Usage tracking
Conversation ownership
Rate limiting
Quota
Personalization
Audit
Example:
USER
↓
QUOTA CHECK
↓
Remaining quota?
├── YES → AI
└── NO → LIMIT RESPONSE
The exact limits can be configured later.
Authentication and chat endpoints can have separate limits.
Example conceptual policy:
LOGIN
↓
strict rate limit
CHAT
↓
usage-aware rate limit
UPLOAD
↓
size + frequency limit
Do not use one global limit for every operation.
Production data should have a backup strategy.
DATABASE
↓
BACKUP
↓
SECURE STORAGE
Backups should also be protected.
A backup containing user data is itself sensitive data.
Before production migration:
BACKUP
↓
TEST MIGRATION
↓
STAGING
↓
VERIFY
↓
PRODUCTION
Avoid testing an unverified schema migration directly against the production database.
Test:
[ ] Signup works
[ ] Login works
[ ] Logout works
[ ] Invalid login rejected
[ ] Private page protected
[ ] Private API protected
[ ] Session expires correctly
[ ] User cannot access another user's resource
Test:
[ ] User creation
[ ] Project creation
[ ] Conversation creation
[ ] Message creation
[ ] Message retrieval
[ ] Ownership checks
[ ] Delete behavior
[ ] Migration
The complete test:
1. Open ACAI
↓
2. Create account
↓
3. Login
↓
4. Open dashboard
↓
5. Create project
↓
6. Create chat
↓
7. Send "Hello"
↓
8. AI responds
↓
9. Refresh page
↓
10. Conversation remains
↓
11. Logout
↓
12. Private page is blocked
If this entire sequence succeeds:
✓ AUTH
✓ DATABASE
✓ CHAT
✓ PERSISTENCE
are working together.
Create:
User A
User B
Then:
User A
↓
Conversation A
Attempt:
User B
↓
Conversation A
Expected:
ACCESS DENIED
This is one of the most important tests in the entire application.
During development, use a separate database/environment.
DEVELOPMENT
≠
PRODUCTION
Never test destructive migration experiments against real production data.
Production:
ACAI SERVER
↓
PRODUCTION DATABASE
should have:
Backups
Monitoring
Access controls
Secure credentials
Migration process
Recovery plan
After Chapter 27:
ACAI
│
┌──────────────┴──────────────┐
▼ ▼
AUTHENTICATION FRONTEND
│ │
▼ ▼
SESSION CHAT
│ │
└──────────────┬──────────────┘
▼
API
│
┌──────┴──────┐
▼ ▼
DATABASE AI GATEWAY
│ │
▼ ▼
CONVERSATIONS MODEL
│
▼
MESSAGES
With this architecture, the platform can support:
User account
↓
Login
↓
Dashboard
↓
Project
↓
Conversation
↓
Persistent messages
↓
AI response
This is a major milestone because ACAI is now structured as a real multi-user application rather than only a local demo.
The next major problem is file and document infrastructure.
We need:
USER
↓
UPLOAD FILE
↓
VALIDATE
↓
STORAGE
↓
DATABASE
↓
PROCESSING
Then:
DOCUMENT
↓
TEXT EXTRACTION
↓
CHUNKING
↓
EMBEDDINGS
↓
VECTOR DATABASE
Then:
USER QUESTION
↓
RAG
↓
AUTHORIZED DOCUMENT CONTEXT
↓
AI
The next chapter will connect the user-facing application more deeply:
LOGIN
↓
DASHBOARD
↓
PROJECTS
↓
CONVERSATIONS
↓
HISTORY
↓
FILE UPLOAD
↓
STORAGE
↓
FILE RECORD
The chapter after that will begin turning uploaded files into searchable AI knowledge.
[✓] PostgreSQL architecture
[✓] Database layer
[✓] User model
[✓] Project model
[✓] Conversation model
[✓] Message model
[✓] Authentication architecture
[✓] Signup flow
[✓] Login flow
[✓] Logout flow
[✓] Session architecture
[✓] Protected pages
[✓] Protected APIs
[✓] Authorization
[✓] User ownership
[✓] Conversation persistence
[✓] Message persistence
[✓] Usage tracking foundation
[✓] Rate-limit foundation
[✓] Database migrations
[✓] Backup strategy
[✓] User isolation testing
[✓] End-to-end account test
The ACAI request now has a real identity and persistence layer:
USER
↓
AUTHENTICATION
↓
SESSION
↓
AUTHORIZED API
↓
DATABASE
↓
CONVERSATION
↓
AI GATEWAY
↓
MODEL
↓
DATABASE
↓
USER
This means the application can now remember who the user is, which projects belong to them, which conversations belong to them, and what messages have already been exchanged.
That foundation is necessary before building secure document intelligence, RAG, agents, and advanced media workflows.
END OF CHAPTER 27
Comments
Post a Comment