DevOps Requirements Oktivo Voice Agent Platform Scaling

Priority: P0 Critical

Timeline: 1 Week

First Client: 7-8 Lakh calls/month ready to onboard

🧠 What is Oktivo? (Context for DevOps)

Oktivo is a white-label AI voice calling platform. Companies (our clients) sign up, upload a CSV of phone numbers, configure an AI voice agent, and run outbound calling campaigns. The AI agent calls each phone number via SIP, has a conversation, records it, transcribes it, and analyzes it.

3-Tier Hierarchy

SuperAdmin (us Oktivo platform)
  └── Organization (our client e.g. a real estate company)
        └── Customer (end user within that org who runs campaigns)

What Happens During a Call

1. Customer clicks "Start Campaign" in our web dashboard
2. Backend picks pending contacts from MongoDB, creates a LiveKit room per call
3. LiveKit dispatches a Python AI agent into the room
4. Agent initiates a SIP call to the phone number via LiveKit SIP gateway
5. SIP gateway routes through our SIP provider (VoiceLink) to PSTN
6. Phone rings → person picks up → AI agent talks to them
7. Call is recorded via LiveKit Egress → uploaded to S3
8. After call: transcription + AI analysis stored in MongoDB
9. Backend picks next pending contact, repeats

Each campaign can have thousands of contacts. Multiple campaigns can run simultaneously. Each call is an independent LiveKit room with 2 participants (AI agent + phone user).

🔧 What is LiveKit? (Key Knowledge for This Job)

LiveKit is an open-source WebRTC server we self-host. It has 4 separate services that all coordinate via Redis:

Service	What It Does	Port	Docker Image
LiveKit Server	Core WebRTC SFU manages rooms, routes audio between participants	7880 (API/WS), 7881 (TCP), 50000-60000 (UDP/WebRTC)	`livekit/livekit-server`
LiveKit SIP	SIP-to-WebRTC bridge converts SIP calls into WebRTC rooms	5060 (SIP signaling), 10000-20000 (RTP media)	`livekit/sip`
LiveKit Egress	Recording service captures room audio and uploads to S3	7890 (health check)	`livekit/egress`
Redis	Shared state room metadata, job queues, service coordination	6379	`redis:7-alpine`

How They Work Together

                    ┌─────────────────────────────────┐
                    │         Redis (shared state)     │
                    └──────┬──────┬──────┬────────────┘
                           │      │      │
                    ┌──────▼──┐ ┌─▼────┐ ┌▼─────────┐
                    │ LiveKit │ │ SIP  │ │  Egress   │
                    │ Server  │ │ GW   │ │ (record)  │
                    └────┬────┘ └──┬───┘ └─────┬─────┘
                         │         │           │
                    WebRTC rooms   SIP calls   S3 upload

Critical: All 4 services MUST share the same Redis instance. When you add more LiveKit Server nodes, they auto-discover each other via Redis and distribute rooms across nodes. Same for SIP and Egress they pick up work from Redis queues.

LiveKit Scaling Model

LiveKit Server: Add more nodes → rooms auto-distribute. Each node needs host networking (not bridge/NAT). One node per EC2 instance.
SIP Gateway: Add more instances → they coordinate via Redis. SIP provider needs to send traffic to all IPs (or use NLB).
Egress: Add more instances → they pick up recording jobs from Redis. This is the most CPU-hungry service each Room Composite recording uses ~1.5 CPU cores. Audio-only Track Egress uses ~0.3-0.5 CPU.
Redis: Single instance is fine for our scale, but needs HA (ElastiCache Multi-AZ).

LiveKit Networking Requirements

LiveKit requires direct network access (host networking). It cannot run behind NAT or in Fargate. In ECS, this means EC2 launch type only, one LiveKit task per EC2 instance.

Ports that must be open:

TCP 7880 LiveKit API + WebSocket (clients connect here)
TCP 7881 WebRTC TCP fallback
UDP 50000-60000 WebRTC media (audio/video)
TCP 5060 SIP signaling (restrict to SIP provider IP: 103.20.104.213)
UDP 10000-20000 SIP RTP media
TCP 7890 Egress health check

Official docs: https://docs.livekit.io/realtime/self-hosting/distributed/

📊 Scale Requirements

Volume

10 Lakh (1 Million) calls/month across all tenants
First client: 7-8 Lakh calls/month
Calling window: 9 AM - 9 PM IST (12 hours)

Concurrency Math

10 Lakh/month ÷ 26 working days = ~38,500 calls/day
38,500 ÷ 720 minutes = ~53 calls/minute
Avg call duration ~2 min → ~106 concurrent calls at steady state
Peak (2x): ~200 concurrent calls

First client (7-8 Lakh/month):
  ~30,000 calls/day → ~84 concurrent at steady state → ~150 peak

Target: Handle 200 concurrent calls with headroom

🏗️ Current Infrastructure (What Exists Today)

LiveKit EC2 (voice.oktivo.com)

Instance: t3.xlarge (4 vCPU, 16 GB RAM) ap-south-1
IP: Elastic IP → A record in DNS
Running: Docker Compose with: Redis, LiveKit Server, LiveKit SIP, LiveKit Egress
Problem: ALL 4 services on ONE machine. Egress needs 1.5 CPU per recording → can only record ~2 calls simultaneously on 4 cores.

Backend EC2 (backend.oktivo.com)

Instance: t3a.large (2 vCPU, 8 GB RAM) ap-south-1
IP: Elastic IP → A record in DNS
Running: Docker Compose with: Node.js backend (port 3000), Python Agent (LiveKit worker), Python Analysis (FastAPI port 8000), Nginx (SSL termination), Certbot
Problem: Single Node.js process, single Agent container. Cannot scale horizontally.

Other Services

MongoDB: Atlas (managed) current tier unknown, likely M10/M20
S3: oktivo-recordings bucket in ap-south-1
Frontend: Cloudflare (no changes needed)
SIP Provider: VoiceLink (TCP, port 3300 for trunk, port 5060 for signaling)

Current Docker Compose Files

livekit_selfhosted/docker-compose.yml LiveKit stack
oktivo_backend/docker-compose.yml Backend stack
Both use network_mode: host for LiveKit services
Images are on DockerHub: dheerajoptimusprime/oktivo:*

🎯 Target Architecture

                         ┌──────────────┐
                         │   Route 53   │
                         │  DNS + Health │
                         └──────┬───────┘
                                │
                    ┌───────────┼───────────┐
                    │           │           │
              ┌─────▼─────┐ ┌──▼───┐ ┌─────▼─────┐
              │    NLB    │ │ ALB  │ │ Cloudflare│
              │(LiveKit)  │ │(API) │ │(Frontend) │
              └─────┬─────┘ └──┬───┘ └───────────┘
                    │          │
         ┌──────────┼──────────┼──────────┐
         │          ECS Cluster (ap-south-1)          │
         │                                            │
         │  ┌────────────┐  ┌────────────┐           │
         │  │ LiveKit x2 │  │  SIP GW x2 │           │
         │  │ (host net)  │  │ (host net)  │           │
         │  └────────────┘  └────────────┘           │
         │                                            │
         │  ┌────────────┐  ┌────────────┐           │
         │  │ Egress x3-5│  │ Agent x3-5 │           │
         │  └────────────┘  └────────────┘           │
         │                                            │
         │  ┌────────────┐  ┌────────────┐           │
         │  │Backend x2  │  │Analysis x1 │           │
         │  └────────────┘  └────────────┘           │
         │                                            │
         └────────────────────────────────────────────┘
                    │              │
         ┌──────────▼──┐  ┌───────▼───────┐
         │ ElastiCache │  │ MongoDB Atlas │
         │ Redis (HA)  │  │    M30        │
         └─────────────┘  └───────────────┘

📋 Tasks Day by Day

Day 1-2: Foundation

Task 1: AWS ElastiCache Redis

Replace the Docker Redis container on LiveKit EC2 with managed ElastiCache.

Why: Redis is the backbone of LiveKit coordination. If it dies, all calls drop. Currently it's a single Docker container with no replication.

Spec:

Engine: Redis 7.x
Node type: cache.r6g.large (2 vCPU, 13 GB)
Multi-AZ: Yes (1 primary + 1 replica)
Encryption: In-transit + at-rest
VPC: Same VPC as ECS cluster
Security group: Allow port 6379 from ECS cluster SG only

bash

aws elasticache create-replication-group \
  --replication-group-id oktivo-redis \
  --replication-group-description "Oktivo LiveKit + Backend Redis" \
  --engine redis \
  --engine-version 7.1 \
  --cache-node-type cache.r6g.large \
  --num-node-groups 1 \
  --replicas-per-node-group 1 \
  --automatic-failover-enabled \
  --multi-az-enabled \
  --at-rest-encryption-enabled \
  --transit-encryption-enabled

Validation: All LiveKit services start and can create rooms. Run a test call.

Task 2: ECS Cluster Setup

Create an ECS cluster to run all services.

Spec:

Cluster name: oktivo-prod
Region: ap-south-1
Launch type: EC2 (NOT Fargate LiveKit needs host networking)
EC2 instances managed via Auto Scaling Group
ECR repositories for images (or continue DockerHub pull)

Instance types needed in the ASG:

Purpose	Instance Type	Count	Notes
LiveKit Server + SIP	c6i.xlarge (4 vCPU, 8GB)	2-4	Host networking, 1 LiveKit task per instance
Egress workers	c6i.xlarge (4 vCPU, 8GB)	3-5	CPU-heavy, host networking
Agent + Backend + Analysis	c6i.xlarge (4 vCPU, 8GB)	3-5	Can co-locate multiple tasks

Networking:

VPC with public + private subnets across 2 AZs
LiveKit/SIP/Egress instances need public IPs (WebRTC requires direct access)
Backend/Agent/Analysis can be in private subnets behind ALB

Task 3: Load Balancers

NLB (Network Load Balancer) for LiveKit:

Target: LiveKit Server instances on port 7880
Protocol: TCP (WebSocket upgrade needs TCP passthrough)
Health check: TCP 7880
DNS: voice.oktivo.com → NLB
Also route SIP traffic (TCP 5060) if SIP provider supports it, otherwise use separate Elastic IPs per SIP instance

ALB (Application Load Balancer) for Backend API:

Target group 1: Backend instances on port 3000 (path: /client/*)
Target group 2: Analysis instances on port 8000 (path: /analysis/*)
HTTPS termination with ACM certificate
Health check: GET /health on each service
DNS: backend.oktivo.com → ALB

Day 2-3: LiveKit Services on ECS

Task 4: LiveKit Server (ECS Service)

Task Definition:

Image: livekit/livekit-server:latest
Network mode: host
CPU: 4096 (4 vCPU), Memory: 8192 (8 GB)
Essential: true

Environment (LIVEKIT_CONFIG):

yaml

port: 7880
keys:
  <API_KEY>: <API_SECRET>
redis:
  address: <ELASTICACHE_ENDPOINT>:6379
  use_tls: true
rtc:
  port_range_start: 50000
  port_range_end: 60000
  tcp_port: 7881
  use_external_ip: true
logging:
  level: info
webhook:
  urls:
    - https://backend.oktivo.com/client/livekit-webhook
  api_key: <WEBHOOK_API_KEY>
room:
  auto_create: true
  empty_timeout: 300

Service config:

Desired count: 2
Placement: spread across instances (one task per instance)
No auto-scaling initially (add later based on room count metric)

Task 5: LiveKit SIP Gateway (ECS Service)

Task Definition:

Image: livekit/sip:latest
Network mode: host
CPU: 2048, Memory: 4096

Environment (SIP_CONFIG_BODY):

yaml

api_key: <API_KEY>
api_secret: <API_SECRET>
ws_url: ws://localhost:7880
redis:
  address: <ELASTICACHE_ENDPOINT>:6379
  use_tls: true
sip_port: 5060
rtp_port: 10000-20000
use_external_ip: true
logging:
  level: info

Important: SIP gateway connects to LiveKit Server via ws://localhost:7880 if co-located, or via the NLB endpoint if on separate instances. If on separate instances, change ws_url to ws://<NLB_DNS>:7880.

Service config:

Desired count: 2
Security group: Allow TCP 5060 + UDP 10000-20000 from SIP provider IP (103.20.104.213)

SIP Provider Config:

Contact VoiceLink to add all SIP instance IPs as destinations
Or point VoiceLink to NLB IP for SIP traffic

Task 6: LiveKit Egress (ECS Service)

Task Definition:

Image: livekit/egress:latest
Network mode: host
CPU: 4096, Memory: 8192

Environment (EGRESS_CONFIG_BODY):

yaml

api_key: <API_KEY>
api_secret: <API_SECRET>
ws_url: ws://<NLB_DNS>:7880
redis:
  address: <ELASTICACHE_ENDPOINT>:6379
  use_tls: true
health_port: 7890
insecure: true
logging:
  level: info
storage:
  s3:
    access_key: <AWS_ACCESS_KEY_ID>
    secret: <AWS_SECRET_ACCESS_KEY>
    region: ap-south-1
    bucket: oktivo-recordings
session_limits:
  file_output_max_duration: 1h
cpu_cost:
  room_composite_cpu_cost: 1.5
  track_composite_cpu_cost: 0.5
  track_cpu_cost: 0.3
  web_cpu_cost: 1.5

Service config:

Desired count: 3 (minimum)
Auto-scaling: Target CPU 70%, max 8 instances
Each c6i.xlarge (4 vCPU) handles ~8-13 audio-only recordings concurrently

Note: We are working on switching from Room Composite (1.5 CPU) to Track Egress (0.3-0.5 CPU) on the code side. Until then, plan for 1.5 CPU per recording.

Day 3-4: Backend Services on ECS

Task 7: Node.js Backend (ECS Service)

Task Definition:

Image: dheerajoptimusprime/oktivo:client-backend-prod-latest
Network mode: awsvpc (runs behind ALB)
CPU: 2048, Memory: 4096
Port mapping: 3000
Health check: GET /health

Environment variables: (from current env/client_backend.env)

MONGO_URI, DB_NAME, PORT=3000, NODE_ENV=production
LIVEKIT_URL=wss://voice.oktivo.com (points to NLB)
LIVEKIT_API_URL=https://voice.oktivo.com
LIVEKIT_API_KEY, LIVEKIT_API_SECRET
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, S3_RECORDING_BUCKET
All other env vars from current .env
NEW: REDIS_URL=rediss://<ELASTICACHE_ENDPOINT>:6379 (for campaign state + BullMQ job queues already implemented in code)

Service config:

Desired count: 2
ALB target group: path /client/*
Auto-scaling: Target CPU 70%, max 4 instances

Task 8: Python Agent (ECS Service)

Task Definition:

Image: dheerajoptimusprime/oktivo:agent-prod-latest
Network mode: awsvpc (agents connect to LiveKit via WebSocket, no host networking needed)
CPU: 2048, Memory: 4096
No port mapping (agent connects outbound to LiveKit, no inbound traffic)

Environment variables: (from current env/agent.env)

LIVEKIT_URL=wss://voice.oktivo.com
LIVEKIT_API_KEY, LIVEKIT_API_SECRET
MONGO_URI
All STT/TTS/LLM API keys (OPENAI, DEEPGRAM, CARTESIA, etc.)
BACKEND_URL=http://<BACKEND_ALB_DNS>/client (or use service discovery)
ANALYSIS_URL=http://<ANALYSIS_SERVICE_DNS>:8000

Service config:

Desired count: 3 (minimum)
Auto-scaling: Target CPU 60%, max 8 instances
Each agent container handles multiple concurrent calls via LiveKit's job distribution

Important: The agent registers itself with LiveKit via Redis. When a room is created and a dispatch is sent, LiveKit assigns the job to an available agent worker. More agent containers = more concurrent call capacity.

Task 9: Analysis Service (ECS Service)

Task Definition:

Image: dheerajoptimusprime/oktivo:analysis-prod-latest
Command: uvicorn app:app --host 0.0.0.0 --port 8000
Network mode: awsvpc
CPU: 1024, Memory: 2048
Port mapping: 8000

Service config:

Desired count: 1 (low traffic only called after each call ends)
Auto-scaling: Target CPU 70%, max 3

Task 10: Remove Nginx from ECS

Nginx is currently used for SSL termination and routing. With ALB handling both, Nginx is no longer needed. ALB does:

SSL termination (ACM certificate)
Path-based routing (/client/* → backend, /analysis/* → analysis)
Health checks
Load balancing

Day 4-5: Monitoring & Observability

Task 11: CloudWatch

EC2/ECS Metrics (automatic):

CPU utilization, Memory utilization, Network I/O
ECS service metrics: running task count, pending task count

Custom CloudWatch Alarms:

Alarm	Threshold	Action
LiveKit CPU > 80%	5 min sustained	SNS → Slack
Egress CPU > 80%	5 min sustained	SNS → Slack + auto-scale
Agent CPU > 70%	5 min sustained	Auto-scale
Backend CPU > 70%	5 min sustained	Auto-scale
Redis CPU > 70%	5 min sustained	SNS → Slack
Redis Memory > 80%	5 min sustained	SNS → Slack
ALB 5xx errors > 10/min	1 min	SNS → Slack
ECS task count = 0 (any service)	Immediate	SNS → Slack (critical)

CloudWatch Logs:

All ECS tasks should log to CloudWatch Logs (awslogs driver)
Log group per service: /ecs/oktivo/backend, /ecs/oktivo/agent, etc.
Retention: 30 days

Task 12: Prometheus + Grafana (Optional but Recommended)

LiveKit exposes Prometheus metrics at http://<host>:7880/metrics. Key metrics:

livekit_room_count          active rooms (= active calls)
livekit_participant_count   active participants
livekit_packet_loss_total   call quality indicator
livekit_egress_active       active recordings

Setup:

Deploy Prometheus as ECS task (or use Amazon Managed Prometheus)
Scrape LiveKit metrics from all server instances
Deploy Grafana for dashboards (or use Amazon Managed Grafana)
Create dashboard: Active Calls, Egress Queue, CPU per service, Error rates

Day 5-6: Load Testing

Task 13: Load Test Infrastructure

We need to validate the setup handles 100-200 concurrent calls before onboarding the client.

Staged approach:

10 concurrent smoke test, verify all services communicate
50 concurrent identify first bottlenecks
100 concurrent target for first client
200 concurrent peak capacity validation

What to monitor at each stage:

[ ] Call success rate > 98%
[ ] Call setup time < 3 seconds
[ ] Recordings saved to S3 > 99%
[ ] CPU < 80% on all instances
[ ] Memory < 85% on all instances
[ ] No Redis connection errors
[ ] No MongoDB connection errors
[ ] No orphaned LiveKit rooms

Tools:

LiveKit CLI (lk) for creating test rooms
SIPp for SIP load testing
Custom Python script (we will provide)

Task 14: SIP Provider Capacity

Contact VoiceLink to confirm they support 200+ concurrent channels
Verify trunk configuration supports multiple destination IPs
Get their capacity limits in writing

Day 6-7: Hardening

Task 15: Security

All inter-service traffic over private VPC (no public IPs on backend/agent/analysis)
LiveKit/SIP/Egress need public IPs (WebRTC requirement)
Redis: VPC-only access, no public endpoint
Security groups: principle of least privilege
Secrets in AWS Secrets Manager or SSM Parameter Store (not env files)
Rotate LiveKit API keys after migration

Task 16: CI/CD Pipeline

GitHub Actions workflow:
1. Build Docker image
2. Push to ECR
3. Update ECS service (force new deployment)
Blue/green deployment for zero-downtime updates
Rollback: revert to previous task definition revision

Task 17: Auto-Scaling Policies

Service	Min	Max	Scale Trigger
LiveKit Server	2	5	Manual (add nodes as orgs grow)
SIP Gateway	2	3	Manual
Egress	3	8	CPU > 70%
Agent	3	8	CPU > 60%
Backend	2	4	CPU > 70%
Analysis	1	3	CPU > 70%

Task 18: Disaster Recovery Checklist

[ ] ElastiCache: Multi-AZ failover tested
[ ] MongoDB Atlas: Backups enabled, tested restore
[ ] S3: Versioning enabled on recordings bucket
[ ] ECS: Services restart automatically on task failure
[ ] LiveKit: Draining mode works on SIGTERM (built-in)
[ ] DNS: Low TTL (60s) on voice.oktivo.com and backend.oktivo.com for fast failover

💰 Estimated AWS Costs (Monthly)

For 100 Concurrent (First Client)

Service	Spec	Count	Monthly (USD)
EC2 (LiveKit+SIP)	c6i.xlarge (4 vCPU)	2	~$250
EC2 (Egress)	c6i.xlarge (4 vCPU)	3	~$375
EC2 (Agent+Backend)	c6i.xlarge (4 vCPU)	3	~$375
ElastiCache Redis	r6g.large (Multi-AZ)	1	~$200
ALB + NLB		2	~$50
S3	~200 GB recordings		~$5
Data Transfer	~1 TB		~$90
CloudWatch	Logs + Metrics		~$50
TOTAL			~$1,395/month

(MongoDB Atlas M30 ~$400/month billed separately)

Cost optimization: Use Reserved Instances (1-year) for 30-40% savings. Use Spot Instances for Agent workers for 60-70% savings.

For 200 Concurrent (Full 10 Lakh/month)

Service	Spec	Count	Monthly (USD)
EC2 (LiveKit+SIP)	c6i.xlarge (4 vCPU)	4	~$500
EC2 (Egress)	c6i.2xlarge (8 vCPU)	4	~$1,000
EC2 (Agent+Backend)	c6i.xlarge (4 vCPU)	5	~$625
ElastiCache Redis	r6g.large (Multi-AZ)	1	~$200
ALB + NLB		2	~$50
S3	~500 GB		~$12
Data Transfer	~2 TB		~$180
CloudWatch + Grafana			~$100
TOTAL			~$2,667/month

📎 Reference Files in Codebase

livekit_selfhosted/
├── docker-compose.yml          # Current LiveKit stack (reference for config values)
├── .env.example                # LiveKit env vars template
└── scripts/init-server.sh      # Current Nginx + SSL setup (will be replaced by NLB)

oktivo_backend/
├── docker-compose.yml          # Current backend stack (reference for service configs)
├── nginx.conf                  # Current Nginx config (will be replaced by ALB)
├── scripts/init-ssl.sh         # Current SSL setup (will be replaced by ACM)
├── Agent/Dockerfile            # Agent Docker build
├── Agent/.env.example          # Agent env vars
├── Analysis/requirements.txt   # Analysis service deps
├── client_backend/Dockerfile   # Backend Docker build
└── client_backend/.env.example # Backend env vars

✅ Acceptance Criteria

[ ] ElastiCache Redis running (Multi-AZ, encrypted)
[ ] ECS cluster with all 6 services running
[ ] NLB routing to LiveKit Server instances
[ ] ALB routing to Backend + Analysis
[ ] LiveKit multi-node: rooms distribute across 2+ server nodes
[ ] SIP gateway: 2 instances, failover tested
[ ] Egress: 3+ instances, auto-scaling configured
[ ] Agent: 3+ instances, auto-scaling configured
[ ] Backend: 2 instances behind ALB
[ ] CloudWatch alarms for CPU, memory, errors
[ ] CloudWatch Logs for all services
[ ] Load test: 100 concurrent calls with >98% success rate
[ ] CI/CD: GitHub Actions → ECR → ECS deploy
[ ] Security groups: least privilege
[ ] Secrets in SSM/Secrets Manager
[ ] DNS updated: voice.oktivo.com → NLB, backend.oktivo.com → ALB
[ ] Old EC2 instances kept as fallback for 1 week, then terminated

DevOps Requirements Oktivo Voice Agent Platform Scaling ​

Priority: P0 Critical ​

Timeline: 1 Week ​

First Client: 7-8 Lakh calls/month ready to onboard ​

🧠 What is Oktivo? (Context for DevOps) ​

3-Tier Hierarchy ​

What Happens During a Call ​

🔧 What is LiveKit? (Key Knowledge for This Job) ​

How They Work Together ​

LiveKit Scaling Model ​

LiveKit Networking Requirements ​

📊 Scale Requirements ​

Volume ​

Concurrency Math ​

Target: Handle 200 concurrent calls with headroom ​

🏗️ Current Infrastructure (What Exists Today) ​

LiveKit EC2 (voice.oktivo.com) ​

Backend EC2 (backend.oktivo.com) ​

Other Services ​

Current Docker Compose Files ​

🎯 Target Architecture ​

📋 Tasks Day by Day ​

Day 1-2: Foundation ​

Task 1: AWS ElastiCache Redis ​

Task 2: ECS Cluster Setup ​

Task 3: Load Balancers ​

Day 2-3: LiveKit Services on ECS ​

Task 4: LiveKit Server (ECS Service) ​

Task 5: LiveKit SIP Gateway (ECS Service) ​

Task 6: LiveKit Egress (ECS Service) ​

Day 3-4: Backend Services on ECS ​

Task 7: Node.js Backend (ECS Service) ​

Task 8: Python Agent (ECS Service) ​

Task 9: Analysis Service (ECS Service) ​

Task 10: Remove Nginx from ECS ​

Day 4-5: Monitoring & Observability ​

Task 11: CloudWatch ​

Task 12: Prometheus + Grafana (Optional but Recommended) ​

Day 5-6: Load Testing ​

Task 13: Load Test Infrastructure ​

Task 14: SIP Provider Capacity ​

Day 6-7: Hardening ​

Task 15: Security ​

Task 16: CI/CD Pipeline ​

Task 17: Auto-Scaling Policies ​

Task 18: Disaster Recovery Checklist ​

💰 Estimated AWS Costs (Monthly) ​

For 100 Concurrent (First Client) ​

For 200 Concurrent (Full 10 Lakh/month) ​

📎 Reference Files in Codebase ​

✅ Acceptance Criteria ​

🔗 Useful Links ​

DevOps Requirements Oktivo Voice Agent Platform Scaling

Priority: P0 Critical

Timeline: 1 Week

First Client: 7-8 Lakh calls/month ready to onboard

🧠 What is Oktivo? (Context for DevOps)

3-Tier Hierarchy

What Happens During a Call

🔧 What is LiveKit? (Key Knowledge for This Job)

How They Work Together

LiveKit Scaling Model

LiveKit Networking Requirements

📊 Scale Requirements

Volume

Concurrency Math

Target: Handle 200 concurrent calls with headroom

🏗️ Current Infrastructure (What Exists Today)

LiveKit EC2 (voice.oktivo.com)

Backend EC2 (backend.oktivo.com)

Other Services

Current Docker Compose Files

🎯 Target Architecture

📋 Tasks Day by Day

Day 1-2: Foundation

Task 1: AWS ElastiCache Redis

Task 2: ECS Cluster Setup

Task 3: Load Balancers

Day 2-3: LiveKit Services on ECS

Task 4: LiveKit Server (ECS Service)

Task 5: LiveKit SIP Gateway (ECS Service)

Task 6: LiveKit Egress (ECS Service)

Day 3-4: Backend Services on ECS

Task 7: Node.js Backend (ECS Service)

Task 8: Python Agent (ECS Service)

Task 9: Analysis Service (ECS Service)

Task 10: Remove Nginx from ECS

Day 4-5: Monitoring & Observability

Task 11: CloudWatch

Task 12: Prometheus + Grafana (Optional but Recommended)

Day 5-6: Load Testing

Task 13: Load Test Infrastructure

Task 14: SIP Provider Capacity

Day 6-7: Hardening

Task 15: Security

Task 16: CI/CD Pipeline

Task 17: Auto-Scaling Policies

Task 18: Disaster Recovery Checklist

💰 Estimated AWS Costs (Monthly)

For 100 Concurrent (First Client)

For 200 Concurrent (Full 10 Lakh/month)

📎 Reference Files in Codebase

✅ Acceptance Criteria

🔗 Useful Links