Productify

Appearance

Productify Autoscaler ​

The autoscaler component for the Productify FrameWork — primarily for use with Nomad. This repository holds 2 components. Each has its own README with detailed usage and configuration.

Overview ​

The autoscaler uses a second-level time-based caching mechanism for improved accuracy:

  1. The optimizer service forecasts resource requirements at second-level granularity
  2. It returns a list of desired replica counts for the next N seconds (configurable via cache_size, default: 10)
  3. The nomadscaler plugin caches these values with a timestamp
  4. Each cached value corresponds to one second of elapsed time
  5. The plugin selects the appropriate cached value based on seconds elapsed since cache update
  6. After using the cache 2 times, the plugin requests fresh predictions (with fallback to cache on failure)

This approach allows for granular, second-by-second scaling decisions based on the optimizer's MILP-calculated resource requirements over time.

Components ​

Prediction Horizon ​

The prediction horizon is configurable:

  • Default: 10 seconds (returns 10 values)
  • Configure via cache_size parameter in the scaling policy
  • Optimizer uses SARIMAX time-series forecasting for predictions
  • Historical metrics (minutes) are used to forecast future demand (seconds)

Architecture ​

┌───────────────────────────────────────────────────┐
│         Nomad Autoscaler Framework                │
│  ┌────────────────────────────────────────────┐   │
│  │   Nomadscaler Plugin (Strategy Plugin)     │   │
│  │   - Receives policy check evaluations      │   │
│  │   - Manages cache (second-level)           │   │
│  │   - Returns desired replica counts         │   │
│  └────────────────────────────────────────────┘   │
└───────────────────────────────────────────────────┘
                    │ HTTP API
                    ▼
┌───────────────────────────────────────────────────┐
│         Optimizer Service (Python)                │
│  - SARIMAX time-series forecasting                │
│  - MILP optimization                              │
│  - Returns N second predictions                   │
│  - Historical metrics analysis                    │
└───────────────────────────────────────────────────┘
                    │
                    ▼
          ┌───────────────────┐
          │    Prometheus     │
          │ (app + Nomad      │
          │  alloc metrics)   │
          └───────────────────┘

How It Works ​

1. Prediction Request ​

Nomadscaler plugin requests predictions from optimizer service:

json
{
  "check": {
    "metric_app_name": "my-app"
  },
  "current_replicas": 3,
  "min_replicas": 1,
  "max_replicas": 10,
  "cache_size": 10
}

2. Time-Series Forecasting ​

Optimizer analyzes historical metrics and forecasts demand for next N seconds.

3. MILP Optimization ​

For each second in the prediction horizon, optimizer calculates optimal replica count considering:

  • Forecasted resource demand
  • Min/max constraints
  • Cost optimization
  • Performance requirements

4. Second-Level Caching ​

Optimizer returns array of desired replica counts:

json
{
  "desired": [3, 3, 4, 4, 5, 5, 6, 6, 7, 7]
}

Each value corresponds to one second in the future.

5. Cache-Based Scaling ​

Plugin caches predictions and selects value based on elapsed time:

  • Second 0: Use desired[0] = 3 replicas
  • Second 1: Use desired[1] = 3 replicas
  • Second 2: Use desired[2] = 4 replicas
  • ...and so on

6. Automatic Refresh ​

After using cache 2 times, plugin requests fresh predictions (with fallback to cache on failure).

Use Cases ​

Traffic Spike Mitigation ​

The optimizer forecasts demand for the next cache_size seconds, so replicas can be added before an incoming increase fully arrives, instead of reacting only after resource metrics saturate.

Cost Optimization ​

When forecast demand drops, the MILP weighs replica cost against the SLA penalty and scales down as far as the configured min allows. With min = 0, an idle service can be scaled down to zero replicas.

Benefits ​

Accuracy ​

  • Second-level predictions vs. minute-level in traditional autoscalers
  • MILP optimization for mathematically optimal scaling decisions
  • Historical pattern recognition for accurate forecasting

Performance ​

  • Proactive scaling reduces reaction lag
  • Smooth transitions with gradual scaling (bounded scale-up/down per step)
  • Reduced scaling oscillations due to the prediction cache

Cost Efficiency ​

  • Minimize over-provisioning with accurate predictions
  • Optimize resource usage with MILP
  • Scale-down during forecast low-demand periods

Reliability ​

  • Fallback mechanisms maintain service during optimizer outages
  • Cache redundancy ensures continuous operation
  • Constraint enforcement prevents under/over-scaling

Requirements ​

  • Nomad cluster with the Nomad Autoscaler
  • Go 1.25+ (for building nomadscaler)
  • Python 3.12+ (for optimizer service)
  • Prometheus (metrics source for the optimizer; a built-in test mode is available)

Next Steps ​

  1. Read the Architecture overview
  2. Follow the Quick Start guide
  3. Configure Scaling Policies
  4. Deploy to Nomad