Productify

Appearance

Docker Compose Deployment ​

Complete guide for deploying Productify using Docker Compose.

Developing against the platform?

This page is the way to run the platform locally. A one-command boot-and-seed (pfy dev up && pfy dev seed) is planned but not implemented — see Known Issues.

Prerequisites ​

Before starting, ensure your system meets these requirements:

System Requirements ​

  • Operating System: Linux (Ubuntu 20.04+, Debian 11+), macOS (12+), or Windows 10/11 with WSL2
  • Docker Engine: 20.10+ installed and running
  • Docker Compose: 2.0+ (usually bundled with Docker Desktop)
  • Memory: Minimum 4 GB RAM (8 GB recommended)
  • Storage: Minimum 10 GB free disk space for images and database
  • Network: Internet connection for downloading dependencies (first run)

Verify Installation ​

bash
# Check Docker version
docker --version
# Should show: Docker version 20.10.x or higher

# Check Docker Compose version
docker compose version
# Should show: Docker Compose version 2.x.x or higher

# Verify Docker is running
docker ps
# Should list running containers (or empty list if none running)

Quick Start ​

Minimal Setup ​

yaml
services:
  postgres:
    image: postgres:16
    environment:
      POSTGRES_DB: productify
      POSTGRES_USER: productify
      POSTGRES_PASSWORD: ${DB_PASSWORD:-changeme}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U productify"]
      interval: 10s
      timeout: 5s
      retries: 5

  manager:
    image: ghcr.io/productifyfw/manager:latest
    ports:
      - "8080:8080"
    environment:
      PFY_RUN_MODE: both
      PFY_DB_HOST: postgres
      PFY_DB_PORT: 5432
      PFY_DB_USER: productify
      PFY_DB_PASSWORD: ${DB_PASSWORD:-changeme}
      PFY_DB_NAME: productify
    depends_on:
      postgres:
        condition: service_healthy
    restart: unless-stopped

volumes:
  postgres_data:

Run:

bash
docker compose up -d

Step-by-Step Setup Guide ​

1. Clone the Repository ​

First, download the source code for the components you need:

bash
# Clone individual components:
git clone https://github.com/ProductifyFW/manager.git
git clone https://github.com/ProductifyFW/proxy.git
git clone https://github.com/ProductifyFW/autoscaler.git

2. Start Manager Component ​

The manager component includes the GraphQL API, database, and frontend application.

bash
cd manager

# Start all services (postgres DB + backend + frontend)
docker compose up -d

# View logs (optional)
docker compose logs -f

# Check service status
docker compose ps

The manager will be available at http://localhost:8080.

Important Ports:

  • 8080: Manager backend API and frontend
  • 8081: Manager health check endpoints (/healthz, /readyz)
  • 5173: Frontend dev server (Vite)
  • 5432: PostgreSQL database (localhost only)

Development Mode

In development mode, the manager runs without authentication. For production, always use the proxy for authenticated access.

3. Start Proxy Component (Optional) ​

The proxy provides unified entry point, authentication, and routing.

bash
cd proxy

# Configure Pocket ID
cp .env.pocketid.example .env.pocketid
# Edit .env.pocketid and set POCKETID_PUBLIC_URL=http://pocketid.localhost

# Start proxy and Pocket ID
docker compose up -d

# Check status
docker compose ps

# Open Pocket ID setup in browser
open http://pocketid.localhost/setup

Available Services:

  • http://pocketid.localhost: Pocket ID authentication service
  • http://manager.localhost: Manager application (with authentication)
  • http://proxytest.localhost: Proxy test page

Note: Add the hostnames to your /etc/hosts file:

bash
sudo tee -a /etc/hosts <<EOF
127.0.0.1 pocketid.localhost
127.0.0.1 manager.localhost
127.0.0.1 proxytest.localhost
EOF

4. Start Autoscaler Optimizer (Optional) ​

The optimizer service can run standalone for testing:

bash
cd autoscaler/optimizer

# Option 1: Using Poetry
poetry install
poetry run web

# Option 2: Using Docker
docker run --rm -p 8015:8015 ghcr.io/productifyfw/optimizer:latest

# Test optimizer
curl http://localhost:8015/health

Autoscaler Limitations

Full autoscaler functionality requires Nomad. See Nomad Deployment for complete setup.

Production Deployment ​

Complete Stack ​

yaml
services:
  # Database
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: ${DB_NAME:-productify}
      POSTGRES_USER: ${DB_USER:-productify}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_INITDB_ARGS: "-E UTF8 --locale=en_US.UTF-8"
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USER:-productify}"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped
    networks:
      - backend

  # Manager API instances
  manager-api:
    image: ghcr.io/productifyfw/manager:${VERSION:-latest}
    deploy:
      replicas: 3
    environment:
      PFY_RUN_MODE: api
      PFY_PORT: 8080
      PFY_HEALTH_PORT: 8081
      PFY_ENV: ${ENV:-production}

      # Database
      PFY_DB_HOST: postgres
      PFY_DB_PORT: 5432
      PFY_DB_USER: ${DB_USER:-productify}
      PFY_DB_PASSWORD: ${DB_PASSWORD}
      PFY_DB_NAME: ${DB_NAME:-productify}
      PFY_DB_SSLMODE: ${DB_SSLMODE:-disable}

      # PocketID
      PFY_POCKET_ID_HOST: ${POCKET_ID_HOST}
      PFY_POCKET_ID_API_KEY: ${POCKET_ID_API_KEY}

      # Cron
      PFY_CRON_BACKEND_HEARTBEAT_TIMEOUT: 2m
      PFY_CRON_TRIGGER_CHECK_INTERVAL: 1s
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      test:
        [
          "CMD",
          "wget",
          "--no-verbose",
          "--tries=1",
          "--spider",
          "http://localhost:8081/healthz",
        ]
      interval: 10s
      timeout: 5s
      retries: 3
    restart: unless-stopped
    networks:
      - backend
      - frontend

  # Manager Executor (single instance)
  manager-executor:
    image: ghcr.io/productifyfw/manager:${VERSION:-latest}
    environment:
      PFY_RUN_MODE: executor
      PFY_ENV: ${ENV:-production}

      # Database
      PFY_DB_HOST: postgres
      PFY_DB_PORT: 5432
      PFY_DB_USER: ${DB_USER:-productify}
      PFY_DB_PASSWORD: ${DB_PASSWORD}
      PFY_DB_NAME: ${DB_NAME:-productify}
      PFY_DB_SSLMODE: ${DB_SSLMODE:-disable}

      # Logging
      PFY_LOGGER_LEVEL: ${LOG_LEVEL:-info}
    depends_on:
      postgres:
        condition: service_healthy
    restart: unless-stopped
    networks:
      - backend

  # Optimizer Service
  optimizer:
    image: ghcr.io/productifyfw/optimizer:${VERSION:-latest}
    ports:
      - "8015:8015"
    environment:
      CONFIG_PATH: /app/config.ini
    volumes:
      - ./optimizer-config.ini:/app/config.ini:ro
    restart: unless-stopped
    networks:
      - backend

  # Proxy (custom Caddy build)
  proxy:
    image: ghcr.io/productifyfw/proxy:${VERSION:-latest}
    ports:
      - "80:80"
      - "443:443"
      - "2112:2112" # Prometheus metrics
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
    depends_on:
      - manager-api
    restart: unless-stopped
    networks:
      - frontend

volumes:
  postgres_data:
  caddy_data:
  caddy_config:

networks:
  backend:
    internal: true
  frontend:

Environment File (.env) ​

bash
# Version
VERSION=1.0.0

# Database
DB_NAME=productify
DB_USER=productify
DB_PASSWORD=<secure-random-password>

# PocketID Integration
POCKET_ID_HOST=https://auth.example.com
POCKET_ID_API_KEY=<your-api-key>

# Environment
ENV=production

# Logging
LOG_LEVEL=info

Caddyfile ​

nginx
{
  admin off
  email admin@example.com
}

manager.example.com {
  reverse_proxy manager-api:8080 {
    lb_policy least_conn
    health_uri http://manager-api:8081/healthz
    health_interval 10s
    health_timeout 5s
  }
}

Pocket ID Authentication Setup ​

Pocket ID provides OpenID Connect (OIDC) authentication for the Productify platform. Follow these steps for complete configuration.

1. Initial Setup ​

After starting the containers, access the setup page:

  1. Navigate to http://pocketid.localhost/setup
  2. Create an administrator account:
    • Username
    • Email address
    • Password
  3. Configure email server (optional but recommended for production)
  4. Set up Passkey (optional, enhances security)

2. Generate API Key for Manager ​

The manager needs an API key to communicate with Pocket ID:

  1. Log in to Pocket ID at http://pocketid.localhost
  2. Go to Settings > Admin > API Keys
  3. Click "Create New API Key"
  4. Enter a name (e.g., "Manager Service")
  5. Copy the generated key (it will only be shown once!)
  6. Add it to the manager's config.yml:
yaml
env: production
port: 8080
db:
  host: postgres
  port: 5432
  user: productify
  password: <secure-random-password>
  dbname: productify
  sslmode: disable
pocket_id:
  host: http://pocketid:1411
  api_key: <paste-your-api-key-here>

3. Register OIDC Client for Proxy ​

To enable login through the proxy:

  1. Go to Settings > Admin > OIDC Clients
  2. Click "Create New Client"
  3. Fill in the details:
    • Client Name: Manager Proxy (or any name)
    • Redirect URIs: http://*.localhost/auth/oauth2/generic/authorization-code-callback
    • Scopes: openid, email, profile
    • Grant Types: Authorization Code
  4. Save and copy the Client ID and Client Secret
  5. Update the proxy Caddyfile:
caddyfile
{
  security {
    oauth identity provider generic {
      realm generic
      driver generic
      client_id <paste-client-id-here>
      client_secret <paste-client-secret-here>
      scopes openid email profile
      # ... rest of config
    }
  }
}

4. Development Login (No Email) ​

If Passkey is not set up and email is not configured, use one-time access codes:

bash
# In Docker Compose environment
docker compose exec pocketid /app/pocket-id one-time-access-token <username-or-email>

# The output URL can be opened in a browser to log in directly

Security Best Practices ​

Production Security

  • Always enable email verification in production
  • Use strong passwords and MFA (Multi-Factor Authentication)
  • Never store API keys or client secrets in version control
  • Use environment variables or encrypted configuration files
  • HTTPS is mandatory in production environments
  • Regularly rotate API keys and secrets

Development Setup ​

Development Compose ​

yaml
version: "3.8"

services:
  postgres:
    image: postgres:16
    ports:
      - "5432:5432"
    environment:
      POSTGRES_DB: manager_dev
      POSTGRES_USER: dev
      POSTGRES_PASSWORD: dev
    volumes:
      - postgres_dev:/var/lib/postgresql/data

  manager:
    build:
      context: ./manager
      dockerfile: Dockerfile.backend
    ports:
      - "8080:8080"
      - "8081:8081"
    environment:
      PFY_RUN_MODE: both
      PFY_DB_HOST: postgres
      PFY_DB_USER: dev
      PFY_DB_PASSWORD: dev
      PFY_DB_NAME: manager_dev
      PFY_LOGGER_LEVEL: debug
    volumes:
      - ./manager:/app
    depends_on:
      - postgres
    # The Dockerfile.backend image runs air for hot reload

  optimizer:
    build:
      context: ./autoscaler/optimizer
      dockerfile: Dockerfile
    ports:
      - "8015:8015"

volumes:
  postgres_dev:

Advanced Configuration ​

Secrets Management ​

Keep secrets out of version control: store them in a .env file (git-ignored) or a mounted config.local.yml, and generate strong values:

bash
# Generate a random database password
openssl rand -hex 32

The Manager reads its configuration from config.local.yml/config.yml (or PFY_* environment variables); it does not support Docker secrets *_FILE indirection.

Resource Limits ​

yaml
services:
  manager-api:
    image: ghcr.io/productifyfw/manager:latest
    deploy:
      resources:
        limits:
          cpus: "2"
          memory: 2G
        reservations:
          cpus: "0.5"
          memory: 512M

Logging ​

yaml
services:
  manager-api:
    image: ghcr.io/productifyfw/manager:latest
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

Custom Networks ​

yaml
networks:
  frontend:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/16
  backend:
    driver: bridge
    internal: true
    ipam:
      config:
        - subnet: 172.21.0.0/16

Monitoring ​

Prometheus ​

yaml
services:
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
    networks:
      - backend

volumes:
  prometheus_data:

prometheus.yml:

yaml
global:
  scrape_interval: 15s

scrape_configs:
  # Executor-mode Manager exposes Prometheus metrics on port 9090
  - job_name: "manager-executor"
    static_configs:
      - targets: ["manager-executor:9090"]

Grafana ​

yaml
services:
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD}
    volumes:
      - grafana_data:/var/lib/grafana
      - ./grafana/dashboards:/etc/grafana/provisioning/dashboards:ro
      - ./grafana/datasources:/etc/grafana/provisioning/datasources:ro
    depends_on:
      - prometheus
    networks:
      - backend

volumes:
  grafana_data:

Troubleshooting ​

Port Conflicts ​

If you see port already in use errors:

bash
# Check what's using the port (Linux/macOS)
sudo lsof -i :8080
sudo lsof -i :5432

# Find and stop the conflicting process
kill -9 <PID>

# Or change the host port in docker-compose.yml
ports:
  - "8082:8080"  # Use 8082 instead of 8080

Docker Network Issues ​

If containers cannot reach each other:

bash
# Restart Docker network
docker compose down
docker network prune -f
docker compose up -d

# Check container connectivity
docker compose exec manager ping postgres

Database Migration Errors ​

If the database schema is incorrect:

bash
# Migrations run automatically at Manager startup; restart the service
docker compose restart manager

# Or recreate everything (WARNING: deletes all data)
docker compose down -v
docker compose up -d

Container Logs ​

View logs for debugging:

bash
# All services
docker compose logs -f

# Specific service
docker compose logs -f manager

# Last 100 lines
docker compose logs --tail=100 manager

# With timestamps
docker compose logs -f -t manager

Health Check Failures ​

If health checks keep failing:

bash
# Check service health
docker compose ps

# Inspect specific container
docker inspect <container-id>

# Check health check logs
docker compose logs manager | grep health

# Manually test health endpoint
docker compose exec manager wget -O- http://localhost:8081/healthz

Permission Issues ​

If you encounter permission errors with volumes:

bash
# Fix volume permissions (Linux)
sudo chown -R $USER:$USER ./postgres_data

# Or run with correct user in docker-compose.yml
services:
  postgres:
    user: "${UID}:${GID}"

Memory Issues ​

If containers are being killed due to OOM:

bash
# Check Docker resource usage
docker stats

# Increase Docker Desktop memory limit (Mac/Windows)
# Docker Desktop > Settings > Resources > Memory

# Add resource limits in docker-compose.yml
services:
  manager:
    deploy:
      resources:
        limits:
          memory: 2G

Clean Restart ​

For a complete clean restart:

bash
# Stop all containers
docker compose down

# Remove volumes (deletes all data!)
docker compose down -v

# Remove images
docker compose down --rmi all

# Clean up Docker system
docker system prune -af

# Start fresh
docker compose up -d

Environment Variables ​

Create a .env file in the same directory as docker-compose.yml:

bash
# Database
DB_NAME=productify
DB_USER=productify
DB_PASSWORD=your-secure-password-here

# Manager
VERSION=latest
ENV=production

# Pocket ID
POCKET_ID_HOST=http://pocketid:1411
POCKET_ID_API_KEY=your-api-key-here

# Logging
LOG_LEVEL=info

WARNING

Never commit the .env file to version control. Add it to .gitignore:

bash
echo ".env" >> .gitignore

Next Steps ​

Backup & Recovery ​

Database Backup ​

yaml
services:
  backup:
    image: postgres:16
    volumes:
      - ./backups:/backups
      - postgres_data:/data:ro
    environment:
      PGHOST: postgres
      PGUSER: ${DB_USER}
      PGPASSWORD: ${DB_PASSWORD}
      PGDATABASE: ${DB_NAME}
    command: >
      sh -c "pg_dump -Fc -f /backups/backup_$(date +%Y%m%d_%H%M%S).dump"
    networks:
      - backend

Cron job:

bash
0 2 * * * docker-compose run --rm backup

Restore ​

bash
docker-compose exec postgres pg_restore \
  -U productify \
  -d productify \
  -c \
  /backups/backup_20250101_020000.dump

Automated backup and restore-and-verify drills ​

The single-database sketch above is superseded by the versioned scripts in deployments/ops/backup/, which back up both databases (manager-db and Pocket ID) and — crucially — verify the backups by restoring them:

  • backup.sh — pg_dump -Fc both databases to timestamped dumps, fails loudly on an empty/truncated dump, and prunes past RETENTION_DAYS (default 30).
  • restore-drill.sh — the drill: restores each dump into a throwaway scratch Postgres container and asserts the expected tables exist, core tables are non-empty, and a smoke query succeeds. A corrupt dump fails the drill (non-zero exit). This is what turns a dump-and-forget cron into a backup you can trust.
  • backup.nomad — a periodic (daily 02:00 UTC) Nomad batch job running backup.sh; DB URLs come from Nomad variables, never committed.
bash
# Back up both databases.
MANAGER_DB_URL=postgres://productify:$PW@localhost:5432/productify \
POCKET_ID_DB_URL=postgres://pocketid:$PW@localhost:5432/pocketid \
  deployments/ops/backup/backup.sh

# Prove they restore (boots an ephemeral scratch Postgres, verifies, tears down).
deployments/ops/backup/restore-drill.sh

Pocket ID storage

The shipped compose runs Pocket ID on SQLite by default. To bring it under this unified pg_dump backup, point Pocket ID at Postgres via its DB_CONNECTION_STRING; otherwise back up its /app/data volume separately.

Deployment Commands ​

Initial Deployment ​

bash
# Create environment file
cp .env.example .env
nano .env

# Generate a strong database password for .env
openssl rand -hex 32

# Pull images
docker-compose pull

# Start services
docker-compose up -d

# Check status
docker-compose ps
docker-compose logs -f

Updates ​

bash
# Pull new images
docker-compose pull

# Recreate containers
docker-compose up -d

# View logs
docker-compose logs -f manager-api

Scaling ​

bash
# Scale API instances
docker-compose up -d --scale manager-api=5

# Do NOT scale executor (must be 1)

Health Checks ​

bash
# Check all services
docker-compose ps

# Health check endpoints
curl http://localhost:8081/healthz   # Manager (health server)
curl http://localhost:8015/health    # Optimizer
curl http://localhost:2112/metrics   # Proxy (metrics endpoint)

# Database connection
docker-compose exec postgres pg_isready -U productify

Production Troubleshooting ​

Container Won't Start ​

bash
# View logs
docker-compose logs manager-api

# Check environment
docker-compose config

# Inspect container
docker inspect productify_manager-api_1

Database Connection Issues ​

bash
# Test connection from Manager
docker-compose exec manager-api sh -c 'nc -zv postgres 5432'

# Check database logs
docker-compose logs postgres

# Verify credentials
docker-compose exec postgres psql -U productify -c '\l'

Network Issues ​

bash
# List networks
docker network ls

# Inspect network
docker network inspect productify_backend

# Test connectivity
docker-compose exec manager-api ping postgres

Performance Issues ​

bash
# View resource usage
docker stats

# Check container limits
docker inspect --format='{{.HostConfig.Memory}}' productify_manager-api_1

Production Checklist ​

  • [ ] Use specific image versions (not latest)
  • [ ] Set strong passwords in .env
  • [ ] Configure backup strategy
  • [ ] Enable TLS/SSL on Proxy
  • [ ] Set resource limits
  • [ ] Configure log rotation
  • [ ] Set up monitoring (Prometheus + Grafana)
  • [ ] Test disaster recovery
  • [ ] Document deployment process
  • [ ] Set up health checks
  • [ ] Configure firewall rules
  • [ ] Enable auto-restart policies

See Also ​