Helm chart

Production install on Kubernetes — the same containers as Docker Compose, with autoscaling, rolling updates, and secret manager integration.

The Helm chart at infra/helm/livechat is the recommended production deployment for workloads beyond a single-VPS scope. It ships every service as its own Deployment + Service + Ingress, with HPA, PDB, default-deny NetworkPolicy, and optional Prometheus ServiceMonitor.

Info — Same containers as docker-compose

The images are identical between docker-compose and Helm; only the orchestrator differs. Going from compose to Helm doesn't require any re-architecture — just re-deploying the same artifacts on K8s.

Prerequisites

Parameters

Install

helm upgrade --install livechat ./infra/helm/livechat \
  --namespace livechat --create-namespace \
  --set global.image.tag=$(git rev-parse --short HEAD) \
  --set global.domain=chatly.example \
  --set secrets.mode=externalSecrets \
  --values values.prod.yaml

Update with new container tags:

helm upgrade livechat ./infra/helm/livechat \
  --reuse-values \
  --set global.image.tag=v0.43.2

Rolling update applies — services drain via terminationGracePeriodSeconds: 60 and the PDB ensures at least 1 replica stays up.

Values (the important ones)

Top-level values

The full reference is at infra/helm/livechat/values.yaml. Annotate your values.prod.yaml with only the overrides you need.

Secret modes

External Secrets Operator (recommended)

The chart generates ExternalSecret resources that pull from your backend (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, HashiCorp Vault, etc.) on a schedule:

secrets:
  mode: externalSecrets
  externalSecrets:
    secretStoreRef:
      name: aws-secrets
      kind: ClusterSecretStore
    refreshInterval: 1h

ESO fetches secrets, writes them to Secret resources, and rotation is automatic.

HashiCorp Vault Secrets Operator

Uses VaultStaticSecret to inject from Vault:

secrets:
  mode: vault
  vault:
    auth: kubernetes
    namespace: chatly
    path: secret/data/chatly/prod

Pair with the @livechat/secrets runtime resolver in code if you want code-level access to the resolved bundle.

Native K8s Secrets (simplest)

You create plain Secret resources yourself, and the chart references them by name:

kubectl create secret generic livechat-core \
  --from-literal=DATABASE_URL='postgresql://...' \
  --from-literal=REDIS_URL='redis://...' \
  --from-literal=MFA_ENCRYPTION_KEY="$(openssl rand -base64 32)" \
  --from-file=JWT_PRIVATE_KEY_PEM_B64=/tmp/jwt-priv.b64 \
  --from-file=JWT_PUBLIC_KEY_PEM_B64=/tmp/jwt-pub.b64

No rotation automation; you manage it.

Autoscaling

Each service ships an HPA configured for its workload:

Parameters

Network policies

With networkPolicy.enabled: true the chart applies:

• Default deny on the namespace.

• Allow API → DB + Redis explicit egress.

• Allow ingress → API/realtime/help-center/marketing/widget from the ingress controller's namespace only.

• Allow worker → DB + Redis + external HTTPS (for webhook delivery).

• Allow voice → LiveKit + DB + Redis; deny voice ↔ other internal services.

• Deny inter-pod traffic that isn't on the explicit allow list.

Production deployments should keep this on. It's the second line of defense after RBAC at the database.

Observability

observability.enabled: true creates:

• ServiceMonitor for each service (Prometheus operator)

• PodMonitor for the worker (no Service to monitor)

• OTel collector sidecar (optional) to forward to Tempo + Loki

Bundled Grafana dashboards (under infra/helm/livechat/dashboards/):

• API health — RPS, p50/p95/p99 latency, error rate by route.

• Realtime health — connections, fan-out latency, dropped events.

• Worker health — queue depths, job latency, retry rate.

• AI usage — calls, tokens, $ per workspace.

• Voice — call concurrency, audio quality, recording lag.

Database migrations

The chart includes a Job that runs drizzle-kit migrate on every helm upgrade, gated by a pre-upgrade Helm hook:

migrations:
  enabled: true
  ttlSecondsAfterFinished: 600
  backoffLimit: 3

Migration failures abort the helm upgrade — the existing pods keep serving while you fix the migration.

Upgrading

Standard helm upgrade flow:

git fetch origin && git checkout main && git pull
helm upgrade livechat ./infra/helm/livechat \
  --reuse-values \
  --set global.image.tag=v0.43.2

The chart's Deployment strategy is RollingUpdate with maxUnavailable: 25% and maxSurge: 25%. Combined with the PDB, there's no downtime during normal upgrades.

For migrations that aren't backwards-compatible with the previous release (rare), use the two-phase upgrade: deploy the migration job separately, then bump the tag.

Multi-region

For multi-region deployments, run one Helm release per region with its own values + database. Use a Cloudflare / Route 53 latency-based routing policy to direct workspaces to their closest region. Cross- region data sync is not automatic — workspaces are region-pinned.

Cost ballparks

A production workload at ~50k MAU on EKS:

• API: 2 replicas × 0.5 CPU × 1 GiB = ~$30/mo

• Realtime: 2 replicas × 0.5 CPU × 1 GiB = ~$30/mo

• Worker: 2 replicas × 1 CPU × 2 GiB = ~$60/mo

• AI: 1 replica × 1 CPU × 2 GiB = ~$30/mo

• Voice: 1 replica × 1 CPU × 1 GiB = ~$25/mo

• Postgres (RDS db.t4g.medium): ~$50/mo

• Redis (ElastiCache cache.t4g.small): ~$25/mo

• Misc (NAT, ALB, S3, CloudWatch): ~$50/mo

• Total infra: ~$300/mo before BYOK customer costs

(BYOK costs are downstream of the customer — LLM, Twilio, email — and don't hit your bill.)

Troubleshooting

Warning — Migration job stuck pending

The migrator's PVC or service-account is wrong — check the pod events: kubectl describe pod -l app=livechat-migrate.

Info — Helm upgrade hangs at 'pre-upgrade hook'

The migration job is running. Tail logs: kubectl logs -f job/livechat-migrate. If it takes more than 5 minutes, your migration is probably the issue, not Helm.