Manager Deployment

Complete deployment guide for the Productify Manager component.

The Manager's own job renders from install.yaml

Since the Wave 3 release train the platform stack — including the Manager's own Nomad job / Compose service — renders from a single install.yaml via pfy platform render|deploy (the bootstrap exception: platform-core artifacts render locally, not from the Manager they start). This page documents the configuration knobs the install.yaml drives.

Deployment Overview

The Manager supports three run modes:

• API Mode - Handles HTTP/GraphQL requests (stateless, scalable)
• Executor Mode - Runs trigger execution loop (stateful, single instance)
• Both Mode - Combined API + Executor (development only)

Run Modes

The Manager supports three run modes configured via PFY_RUN_MODE environment variable or run_mode in config.yml:

API Mode

run_mode: api

Or with environment variable:

export PFY_RUN_MODE=api
./manager

Characteristics:

• Stateless
• Horizontally scalable
• Load balanced
• No trigger execution

Executor Mode

run_mode: executor
cron:
  metrics_port: 9090 # Prometheus metrics endpoint

Or with environment variable:

export PFY_RUN_MODE=executor
export PFY_CRON_METRICS_PORT=9090
./manager

Characteristics:

• Stateful (cron loop)
• Single instance only (database-level locking)
• Not load balanced
• Handles trigger execution
• Exposes Prometheus metrics on port 9090

Prometheus Metrics:

When running in executor mode, metrics are exposed at :9090/metrics for autoscaler integration:

• pfy_executor_queue_all_total{app="<app-id>"} - Total triggers queued per app
• pfy_executor_queue_processed_total{app="<app-id>"} - Successfully processed triggers per app
• pfy_executor_queue_waiting{app="<app-id>"} - Current waiting triggers per app
• pfy_executor_queue_process_time_seconds{app="<app-id>"} - Processing duration histogram per app
• pfy_executor_active_triggers - Currently active/enabled triggers
• pfy_executor_backend_dispatch_total{status="success|error"} - Backend dispatch results

Database-Level Locking:

The Executor uses database-level locking to ensure only one instance processes triggers at a time across the cluster. This prevents duplicate trigger executions even if multiple executor instances are accidentally started.

Both Mode (Development)

run_mode: both

Or with environment variable:

export PFY_RUN_MODE=both
./manager

Characteristics:

• Combined API + Executor
• Not recommended for production
• Convenient for development

Docker Deployment

API Instance

docker run -d \
  --name manager-api \
  -p 8080:8080 \
  -p 8081:8081 \
  -e PFY_RUN_MODE=api \
  -e PFY_DB_HOST=postgres \
  ghcr.io/productifyfw/manager:latest

Executor Instance

docker run -d \
  --name manager-executor \
  -p 9090:9090 \
  -e PFY_RUN_MODE=executor \
  -e PFY_CRON_METRICS_PORT=9090 \
  -e PFY_DB_HOST=postgres \
  -e PFY_DB_PASSWORD=secret \
  ghcr.io/productifyfw/manager:latest

Accessing metrics:

curl http://localhost:9090/metrics

Docker Compose

version: "3.8"

services:
  manager-api:
    image: ghcr.io/productifyfw/manager:latest
    deploy:
      replicas: 3
    ports:
      - "8080-8082:8080"
    environment:
      PFY_RUN_MODE: api
      PFY_DB_HOST: postgres
      PFY_DB_PASSWORD: ${DB_PASSWORD}
    depends_on:
      - postgres
    restart: unless-stopped

  manager-executor:
    image: ghcr.io/productifyfw/manager:latest
    ports:
      - "9090:9090"
    environment:
      PFY_RUN_MODE: executor
      PFY_CRON_METRICS_PORT: 9090
      PFY_DB_HOST: postgres
      PFY_DB_PASSWORD: ${DB_PASSWORD}
    depends_on:
      - postgres
    restart: unless-stopped

  postgres:
    image: postgres:16
    environment:
      POSTGRES_DB: postgres
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    restart: unless-stopped

volumes:
  postgres_data:

Nomad Deployment

A complete reference job specification is available at manager/nomad/manager.nomad. The example below shows the API and Executor groups:

job "manager" {
  datacenters = ["dc1"]
  type        = "service"

  # API instances (scalable)
  group "api" {
    count = 3

    network {
      port "http" {
        to = 8080
      }
      port "health" {
        to = 8081
      }
    }

    task "manager-api" {
      driver = "docker"

      config {
        image = "ghcr.io/productifyfw/manager:latest"
        ports = ["http", "health"]
      }

      env {
        PFY_ENV         = "production"
        PFY_RUN_MODE    = "api"
        PFY_PORT        = "8080"
        PFY_HEALTH_PORT = "8081"
        PFY_DB_HOST     = "postgres.example.com"
        PFY_DB_PORT     = "5432"
        PFY_DB_USER     = "postgres"
        PFY_DB_PASSWORD = "<secure-password>"
        PFY_DB_NAME     = "productify"

        # PocketID Configuration
        PFY_POCKET_ID_HOST    = "http://pocketid:1411"
        PFY_POCKET_ID_API_KEY = "<your-api-key>"
      }

      resources {
        cpu    = 1000
        memory = 512
      }

      service {
        name = "manager-api"
        port = "http"

        check {
          type     = "http"
          port     = "health"
          path     = "/healthz"
          interval = "10s"
          timeout  = "2s"
        }

        check {
          name     = "readiness"
          type     = "http"
          port     = "health"
          path     = "/readyz"
          interval = "10s"
          timeout  = "2s"
        }
      }
    }
  }

  # Executor instance (single)
  group "executor" {
    count = 1

    network {
      port "health" {
        to = 8081
      }
      port "metrics" {
        static = 9090
      }
    }

    task "manager-executor" {
      driver = "docker"

      config {
        image = "ghcr.io/productifyfw/manager:latest"
        ports = ["health", "metrics"]
      }

      env {
        PFY_ENV               = "production"
        PFY_RUN_MODE          = "executor"
        PFY_HEALTH_PORT       = "8081"
        PFY_CRON_METRICS_PORT = "9090"
        PFY_DB_HOST           = "postgres.example.com"
        PFY_DB_PORT           = "5432"
        PFY_DB_USER           = "postgres"
        PFY_DB_PASSWORD       = "<secure-password>"
        PFY_DB_NAME           = "productify"
      }

      resources {
        cpu    = 500
        memory = 256
      }

      service {
        name = "manager-executor"
        port = "metrics"

        tags = [
          "prometheus",
          "metrics"
        ]

        check {
          type     = "http"
          port     = "health"
          path     = "/healthz"
          interval = "30s"
          timeout  = "5s"
        }
      }
    }
  }
}

Future Support

Kubernetes deployment manifests will be added in a future release.

Configuration

See Manager Configuration for complete configuration reference.

Minimum required environment variables:

# Database
PFY_DB_HOST=postgres.example.com
PFY_DB_PORT=5432
PFY_DB_USER=manager
PFY_DB_PASSWORD=<secure-password>
PFY_DB_NAME=manager_prod
PFY_DB_SSLMODE=require

# PocketID Integration
PFY_POCKET_ID_HOST=https://auth.example.com
PFY_POCKET_ID_API_KEY=<your-api-key>

# Server Configuration
PFY_ENV=production
PFY_RUN_MODE=api  # Options: api, executor, both
PFY_PORT=8080

Health Checks

The Manager runs a separate health server (default port 8081, configurable via PFY_HEALTH_PORT or health_port) with liveness and readiness endpoints:

curl http://localhost:8081/healthz

Response:

{ "status": "healthy" }

The readiness endpoint additionally verifies the database connection:

curl http://localhost:8081/readyz

Monitoring

Metrics

API Mode Metrics (Future):

• Request rate (per endpoint)
• Response latency (p50, p95, p99)
• Error rate
• Database connection pool usage

Executor Mode Metrics (Available Now):

Prometheus metrics exposed at :9090/metrics:

• Queue Metrics - Triggers queued, processed, and waiting (per app)
• Processing Time - Histogram of trigger execution duration (per app)
• Active Triggers - Count of enabled triggers
• Backend Dispatch - Success/error counts for backend callbacks

Prometheus Scrape Configuration:

scrape_configs:
  - job_name: "manager-executor"
    static_configs:
      - targets: ["manager-executor:9090"]
    scrape_interval: 15s

Example PromQL Queries:

# Triggers waiting per application
pfy_executor_queue_waiting{app="app-uuid"}

# Trigger processing rate (per second)
rate(pfy_executor_queue_processed_total[5m])

# 95th percentile processing time
histogram_quantile(0.95, rate(pfy_executor_queue_process_time_seconds_bucket[5m]))

# Backend dispatch error rate
rate(pfy_executor_backend_dispatch_total{status="error"}[5m])

Logs

Structured JSON logging:

{
  "level": "info",
  "msg": "Request completed",
  "method": "POST",
  "path": "/query",
  "status": 200,
  "duration_ms": 45,
  "timestamp": "2025-12-01T10:00:00Z"
}

Scaling

API Instances

Scale horizontally based on:

• CPU utilization (target 70%)
• Request rate
• Response latency

Nomad scaling policy:

scaling {
  min = 2
  max = 10

  policy {
    evaluation_interval = "30s"
    cooldown = "60s"
  }
}

Executor Instance

DO NOT SCALE - Must be single instance only.

Troubleshooting

API Not Responding

Check:

• Container is running
• Port is accessible
• Database connection is valid
• Health check passes

Executor Not Running Triggers

Verify:

• Mode is set to executor or both
• Database connection is valid
• Triggers are enabled
• Cron schedules are valid

Database Connection Issues

Debug:

• Connection string is correct
• Database is accessible from Manager
• Credentials are valid
• SSL/TLS configured properly

See Also

• Production Setup
• Docker Compose Deployment
• Configuration
• Architecture