Scaling Policies

A scaling policy is the scaling block in a Nomad job's task group. It tells the Nomad Autoscaler how often to evaluate, which metric to read, and which strategy decides the new count. The block below is what the productify-scaler strategy accepts; the rest of the page covers tuning it. For what each key does inside the plugin, see Nomadscaler Plugin and Configuration.

pfy nomad render / pfy nomad deploy render a scaling block of this shape into the application job from the deployment spec's scaling {enabled, min, max} (see Generated Artifacts): source = "nomad-apm" with query = "avg_cpu-allocated", metric_app_name set to the application UUID, group and task set to the rendered job name, the strategy's min/max copied from the scaling block, and optimizer_url from the spec's scaler endpoint. It emits no cache_size and no optimizer_token; the token comes from the agent config or OPTIMIZER_TOKEN.

Policy structure

scaling {
  enabled = true
  min     = <minimum count Nomad allows>
  max     = <maximum count Nomad allows>

  policy {
    evaluation_interval = "<how often the check runs, e.g. 10s>"
    cooldown            = "<minimum time between scaling actions, e.g. 30s>"

    check "<check name>" {
      source = "nomad-apm"          # or "prometheus"
      query  = "<APM query>"        # e.g. "avg_cpu" for nomad-apm

      strategy "productify-scaler" {
        metric_app_name = "<app>"           # required: the `app` label on the pfy_* series (an application UUID for anything pfy renders)
        group           = "<task group>"    # recommended: the Nomad task group this scaling block lives in
        task            = "<task>"          # recommended: the Nomad task inside it, for the nomad_client_allocs_* queries
        min             = <min replicas>    # optional, default 1
        max             = <max replicas>    # optional, default 10; 0 means no ceiling
        cache_size      = <seconds>         # optional, default 10, max 600; above step_seconds gives a multi-step ramp
        optimizer_url   = "<url>"           # optional if set in the agent config / OPTIMIZER_URL
        optimizer_token = "<token>"         # optional if set in the agent config / OPTIMIZER_TOKEN
      }
    }
  }
}

Two sets of bounds appear here. scaling { min max } is enforced by Nomad on every action; strategy { min max } is what the plugin clamps its decision to and sends to the optimizer as min_replicas / max_replicas. Neither overrides the other, so set them to the same values.

Strategy parameters

Parameter	Required	Default	Description
metric_app_name	yes	none	Must equal the app label on the pfy_* metrics (or an app_aliases entry on the optimizer). For anything pfy nomad render emits this is the application UUID. It is not a Nomad identifier; see task
optimizer_url	yes*	none	Optimizer base URL
optimizer_token	yes*	none	Bearer token for the optimizer
group	no (recommended)	the check block's own group attribute, else empty	The Nomad task group the scaling block lives in; the optimizer uses it as the task_group label when reading the task's allocated CPU/memory (pfy nomad render sets it to the rendered task group). Empty is allowed: the optimizer then matches by task name only and the plugin logs no `group` configured for check once
task	no (recommended)	empty; the optimizer falls back to metric_app_name	The Nomad task whose allocated CPU/memory the optimizer reads, used as the task label of the nomad_client_allocs_* queries. A separate identity from metric_app_name: those series are exported by the Nomad client and carry the job's own names, never the app label. There is no SDK last resort the way group has one, because a scaling policy attaches to a task group that may hold several tasks, so only the policy can say which one is being sized. Left empty, the plugin logs no `task` configured for check once and the optimizer warns once per app
min	no	1	Lower bound. Must be >= 0 and, when max > 0, <= max. 0 permits zero replicas here, but scale-to-zero needs more than this key; see min and max
max	no	10	Upper bound; 0 means no ceiling (only min is enforced); negative values are rejected
cache_size	no	10	Seconds of decisions per optimizer call; clamped to 600. At or below step_seconds the response is a single value, so the count moves at most one scale step per call; see cache_size

* May instead be set in the autoscaler agent's strategy "productify-scaler" { config = { ... } } block or the OPTIMIZER_URL / OPTIMIZER_TOKEN environment variables. min, max, group and task may also be set in the agent block as global defaults, though a single global task only makes sense where every check sizes the same task name.

Keys the plugin does not know are logged as unknown key in check strategy config (once per check and key) and ignored.

Complete example

autoscaler/nomadscaler/config/test.hcl (the optimizer URL and token come from the agent config in config/autoscaler.hcl). It is a development fixture. Its min = 0 does not give a working cold-start cycle: with this policy's source = "nomad-apm" the app is not evaluated at all once it reaches zero (Known Issues).

job "webapp" {
  datacenters = ["dc1"]

  group "demo" {
    count = 3

    network {
      port "webapp_http" {}
    }

    scaling {
      enabled = true
      min     = 0
      max     = 10

      policy {
        evaluation_interval = "3s"
        cooldown            = "10s"

        check "productify-scale-check" {
          source = "nomad-apm"
          query  = "avg_cpu-allocated"

          strategy "productify-scaler" {
            min             = 0
            max             = 10
            # The `app` Prometheus label: prometheus.yml in autoscaler.hcl
            # scrapes the optimizer's own test metrics under app="test".
            metric_app_name = "test"
            # The Nomad identifiers, which are different values again: this
            # group holds one task, named "webapp".
            group           = "demo"   # the Nomad task group this scaling block lives in
            task            = "webapp" # the task inside it, i.e. the task block below
            cache_size      = 10 # Number of cached replica values (default: 10)
          }
        }
      }
    }

    task "webapp" {
      driver = "docker"

      config {
        image = "hashicorp/demo-webapp-lb-guide"
        ports = ["webapp_http"]
      }

      env {
        PORT    = "${NOMAD_PORT_webapp_http}"
        NODE_IP = "${NOMAD_IP_webapp_http}"
      }

      resources {
        cpu    = 100
        memory = 16
      }
    }
  }
}

This fixture shows the identities being genuinely different values. metric_app_name = "test" matches the app: 'test' label the reference Prometheus config attaches to the optimizer's test metrics endpoint; group = "demo" and task = "webapp" are Nomad identifiers naming the task group and the task "webapp" block further down the same file. All three differ, and none of them can be derived from the others.

The check query

The strategy uses the check's samples for two things: to decide whether demand is present when it must fall back, and to forward them to the optimizer, which appends them to the request-rate series it forecasts only when the check's query names that series (request_rate_metric, default pfy_total_requests). The plugin sends the check's query verbatim, so a query that returns the request rate is the best match:

check "requests" {
  source = "prometheus"
  query  = "sum(rate(pfy_total_requests{app=\"my-app\"}[30s]))"
}

With source = "nomad-apm" the valid task-group queries are <sum|avg|max|min>_<cpu|cpu-allocated|memory|memory-allocated>, e.g. avg_cpu. These values are CPU/memory figures, not request rates; the optimizer forecasts from Prometheus history and ignores the forwarded samples.

Tuning

cache_size

The optimizer plans in steps of step_seconds (15 by default), so the effective cached span rounds up to whole steps: cache_size = 10 gives 1 value (15 s), cache_size = 45 gives 3 values, cache_size = 60 gives 4 values.

The MILP pins the first step of every plan to the count the job is already running and plans milp_startup_delay + ceil(cache_size / step_seconds) steps, returning the actionable tail, so even a one-value response is a decision. What cache_size buys is the shape of the ramp:

• Longer cache (e.g. 60): fewer optimizer calls and a multi-step plan the count can follow between calls, but new metrics take longer to influence it.
• Shorter cache (e.g. the default 10): one actionable value per call, so the count moves by at most milp_max_scale_up / milp_max_scale_down (2 each) before the next call. An idle app at 3 replicas that is worth shrinking answers [1], not [0]; reaching zero takes a second evaluation. Whether it is worth shrinking is a separate question; see min and max.

strategy "productify-scaler" {
  metric_app_name = "my-app"
  min             = 2
  max             = 20
  cache_size      = 60
}

evaluation_interval and cooldown

• evaluation_interval must be short enough to hit every step boundary you care about; with a 15-second step, 3 to 10 seconds is typical. Each evaluation is served from the cache unless it is time-exhausted, so a short interval is cheap.
• cooldown is enforced by Nomad after each scaling action. A cooldown longer than step_seconds means some cached per-step values can never be applied. Start around 2 × step_seconds (30 s) and lengthen it if the count oscillates.

min and max

• min = 0 permits zero replicas in the strategy's own clamp; on its own it does not give you a working cold-start cycle. The MILP charges the hard SLA cost for every step that has positive demand and zero replicas, so it only chooses zero when the forecast demand for the step is zero, and only when releasing the replica pays for itself (next bullet; at the shipped fallback_cpu_alloc = 100 MHz it does not in one step). What is missing around it: on the Manager-managed path min: 0 cannot be set at all, and with source = "nomad-apm" an app that reaches zero is never evaluated again, because the APM returns metric not found with no running allocation. Read Known Issues and the APM blocker before relying on it.

• Whether a replica is released is an economic question, not a horizon one. Releasing one costs milp_shutdown_cost (0.3 in the committed config.ini) once, and saves replica_cost on every step of the actionable horizon, so the count only falls when the saving over that horizon reaches the shutdown cost. Over a single actionable step, the default cache_size, that is replica_cost >= 0.3.

  replica_cost = cpu_alloc × cost_cpu_mhz_hour + (mem_alloc / 1024³) × cost_mem_gb_hour. The memory term is negligible at the shipped coefficients, so in practice the threshold is 300 MHz of allocated CPU at cost_cpu_mhz_hour = 0.001. Below that, an idle app holds its count over one actionable step and needs a longer cache_size to shrink. With the shipped optimizer/config.ini at the shipped fallback_cpu_alloc = 100 MHz / fallback_mem_alloc = 16777216 bytes (replica_cost = 0.100000156), an idle app at 1 replica answers [1] at cache_size = 10 and [1, 1] at 30, and reaches [0, 0, 0] at cache_size = 45, three actionable steps, where 3 × 0.100000156 first clears 0.3. At 500 MHz it answers [0] at cache_size = 10; the boundary is exactly 300 MHz ([1] at 299, [0] at 300). See Known Issues for the full table.

• max bounds cost. The MILP also limits the change per step to milp_max_scale_up / milp_max_scale_down (default 2), so reaching a high max from a low count takes several steps.

Optimizer-side knobs

Reaction speed and cost balance live in the optimizer's config.ini, not in the policy: cost_sla_soft / cost_sla_hard (how expensive unmet demand is), milp_startup_cost / milp_shutdown_cost (how expensive changing the count is), milp_max_scale_up / milp_max_scale_down, risk_aware. See Configuration.

Troubleshooting

Symptom	Things to check
Count never changes	Agent log for missing required field, optimizer request failed, or APM returned no metrics; cooldown too long; strategy and scaling-block bounds disagree
Frequent up/down oscillation	Lengthen cooldown; increase cache_size; raise milp_startup_cost / milp_shutdown_cost on the optimizer
Count never falls, even when idle	replica_cost below milp_shutdown_cost over the actionable horizon, the common case under 300 MHz of allocated CPU. See min and max. Lengthen cache_size, raise cost_cpu_mhz_hour, or lower milp_shutdown_cost
Count never rises from zero	The app is not being evaluated at all: source = "nomad-apm" cannot produce a sample with no running allocation; see Known Issues
Scales up too eagerly / costs too much	Lower cost_sla_soft / cost_sla_hard or raise cost_cpu_mhz_hour on the optimizer; lower max
Scales up too late / latency spikes	Raise cost_sla_hard; shorten cache_size so decisions are re-taken more often; make sure risk_aware=true; check milp_max_scale_up allows a fast enough ramp
503 metrics_unavailable in the log	The optimizer cannot read Prometheus, or nomad_client_allocs_*{task_group, task} returns nothing while replicas are running: task must equal the Nomad task name, group the task group, and Prometheus must scrape Nomad's /v1/metrics. A policy with no task line is a common cause: it falls back to metric_app_name, a UUID no task is named after. At zero replicas the optimizer uses the last-known or configured fallback allocation instead
no `task` configured for check in the agent log	Add task = "<task name>" to the strategy block. Without it the optimizer selects allocation metrics by metric_app_name, which matches a real series only where the Nomad task happens to be named after the app label; see Known Issues
no `group` configured for check in the agent log	Add group = "<task group>" to the strategy block; without it the optimizer matches allocations by task name only

See also

• Configuration
• Nomadscaler Plugin
• Optimizer Service
• Architecture