DevOps & CI/CD for Python
Deploy Python applications with Docker, GitHub Actions CI/CD, environment configuration, and production best practices.
Getting code from your laptop to production reliably requires automation. DevOps practices and CI/CD pipelines make deployments repeatable and safe.
Dockerizing a Python App
# Dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
Build and run:
docker build -t myapp .
docker run -p 8000:8000 --env-file .env myapp
Multi-Stage Builds
Keep production images small:
FROM python:3.12-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt --target /deps
FROM python:3.12-slim
COPY --from=builder /deps /usr/local/lib/python3.12/site-packages
COPY . /app
WORKDIR /app
CMD ["python", "main.py"]
GitHub Actions CI Pipeline
Create .github/workflows/ci.yml:
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install pytest flake8 mypy
- name: Lint
run: flake8 src/
- name: Type check
run: mypy src/
- name: Test
run: pytest tests/ -v --cov=src
Environment Configuration
Separate config from code using environment variables:
import os
from dataclasses import dataclass
@dataclass
class Config:
debug: bool = os.getenv("DEBUG", "false").lower() == "true"
database_url: str = os.environ["DATABASE_URL"]
secret_key: str = os.environ["SECRET_KEY"]
config = Config()
Deployment Strategies
| Strategy | Description | Risk |
|---|---|---|
| Rolling | Replace instances one at a time | Low |
| Blue-Green | Switch traffic between two environments | Very low |
| Canary | Route small % of traffic to new version | Lowest |
Production Checklist
- Use a process manager (gunicorn, uvicorn workers)
- Set
DEBUG=Falsein production - Configure logging (structured JSON logs)
- Set up health check endpoints (
/health) - Monitor with Prometheus, Datadog, or Sentry
- Automate deployments via CI/CD
- Run database migrations as a separate step
- Back up databases regularly
Example: Deploy FastAPI with Docker Compose
# docker-compose.yml
services:
web:
build: .
ports:
- "8000:8000"
environment:
- DATABASE_URL=postgresql://user:pass@db:5432/myapp
depends_on:
- db
db:
image: postgres:16
environment:
POSTGRES_USER: user
POSTGRES_PASSWORD: pass
POSTGRES_DB: myapp
volumes:
- pgdata:/var/lib/postgresql/data
volumes:
pgdata:
Automating build, test, and deploy pipelines lets you ship Python applications with confidence.
Production WSGI/ASGI Servers
Never use Flask/Django/FastAPI dev servers in production:
# FastAPI / ASGI
pip install uvicorn gunicorn
gunicorn main:app -k uvicorn.workers.UvicornWorker -w 4 --bind 0.0.0.0:8000
# Django / WSGI
gunicorn mysite.wsgi:application -w 4 --bind 0.0.0.0:8000
# Flask
gunicorn app:app -w 4 --bind 0.0.0.0:8000
-w 4 runs four worker processes. Scale workers to (2 × CPU cores) + 1 as a starting point.
Health Check Endpoints
Load balancers and orchestrators need a health endpoint:
# FastAPI
@app.get("/health")
def health():
return {"status": "ok"}
# With database check
@app.get("/health")
def health(db: Session = Depends(get_db)):
db.execute(text("SELECT 1"))
return {"status": "ok", "database": "connected"}
Configure your load balancer to remove unhealthy instances automatically.
Structured Logging in Production
JSON logs integrate with CloudWatch, Datadog, and ELK:
import logging
import json
class JSONFormatter(logging.Formatter):
def format(self, record):
return json.dumps({
"level": record.levelname,
"message": record.getMessage(),
"logger": record.name,
})
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logging.getLogger().addHandler(handler)
See Logging for full coverage.
.dockerignore
Speed up builds by excluding unnecessary files:
.git
.venv
__pycache__
*.pyc
.env
tests/
htmlcov/
.pytest_cache/
CD Pipeline — Deploy on Merge
Extend CI to deploy after tests pass:
deploy:
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- name: Build and push Docker image
run: |
docker build -t myregistry/myapp:${{ github.sha }} .
docker push myregistry/myapp:${{ github.sha }}
- name: Deploy to server
run: |
ssh deploy@server "docker pull myregistry/myapp:${{ github.sha }} && docker compose up -d"
Use secrets for registry credentials and SSH keys — never hardcode them.
Environment-Specific Config
# config.py
import os
ENV = os.getenv("APP_ENV", "development")
class DevelopmentConfig:
DEBUG = True
DATABASE_URL = "sqlite:///dev.db"
class ProductionConfig:
DEBUG = False
DATABASE_URL = os.environ["DATABASE_URL"]
config = ProductionConfig() if ENV == "production" else DevelopmentConfig()
Monitoring and Alerting
| Tool | Purpose |
|---|---|
| Sentry | Error tracking and stack traces |
| Prometheus + Grafana | Metrics and dashboards |
| Uptime Robot / Pingdom | External availability checks |
| CloudWatch / Azure Monitor | Managed cloud monitoring |
Set alerts on error rate spikes, high latency, and failed health checks.
Database Migrations in CI/CD
Run migrations as a separate deploy step — not inside app startup:
# Django
python manage.py migrate --noinput
# Alembic (SQLAlchemy)
alembic upgrade head
Run migrations before switching traffic to the new version.
Related
- Security — production security checklist
- Virtual Environments — reproducible dependencies
- Packaging — publish Python packages
- Serverless Deployment — AWS Lambda CI/CD
- Full-Stack ML Capstone — end-to-end deploy example