Managed Data Dependencies
A managed data dependency is a postgres or redis instance the platform provisions, runs, credentials, injects and backs up for an application. You declare that the application needs a database; the platform owns everything between that declaration and a DATABASE_URL inside the container.
The operational procedures (configuring backup targets, taking a backup out of schedule, restoring) are in the Data Store Backup & Restore runbook.
Volumes are NEVER deleted automatically
Deleting a data store, through pfy data delete or the Manager UI, removes its job and the credentials generated for it. The underlying host volume is left untouched, always. The data stays on disk, and once the record is gone nothing in the platform describes what that volume held. Reclaiming it is a deliberate act by a human on the node.
The same rule governs restores: nothing in the restore path ever deletes a dump.
Declaring a store
pfy data create main --app shop \
--kind postgres --version 16 \
--volume shop-main-pg --size 20G \
--cpu 500 --memory 1024
pfy data create cache --app shop --kind redis --version 7
pfy data list --app shop| Field | Flag | Notes |
|---|---|---|
slug | positional | Identifies the store within the application. Appears in the job name, the database name and the injected env var. Cannot be changed after create; delete and recreate instead. |
kind | --kind | postgres or redis. Immutable after create: changing it would rename the job and orphan the volume. |
version | --version | The image tag line, e.g. 16 or 7. Not a full image reference; the renderer composes postgres:16. A value containing : or / is refused. |
persistence.volume | --volume | The Nomad host volume the data lives on. Empty means the derived default <job>-data (for example shop--main-data). |
persistence.size | --size | A capacity hint, e.g. 20G. Documentation, not an enforced quota. |
resources.cpu / resources.memory | --cpu / --memory | Reservations, in MHz and MB. |
backup.schedule, backup.target, backup.retention | --backup-schedule, --backup-target, --backup-retention | See Backups. |
A store is declared once on the application and instantiated once per (environment, dedicated tenant). A dedicated tenant's store is a different job on a different volume with its own credential. Nothing is shared with the shared instance except the declaration.
The GraphQL surface is dataDependencies(applicationId), createDataDependency, updateDataDependency and deleteDataDependency(id, confirm). There is no status field on the type: a store's run state is read live from Nomad by pfy data status, not stored.
Instances and names
| Shared | Dedicated tenant | |
|---|---|---|
| Job and Nomad service | shop--main | shop--main--acme |
| Nomad variable | nomad/jobs/shop--main | nomad/jobs/shop--main--acme |
| Backup job | shop--main--backup | shop--main--acme--backup |
| Postgres database | shop_main | shop_main_acme |
| Postgres role | shop | shop |
The environment is not in the name. Environments are separate Nomad namespaces, so an instance is identified by (namespace, job name), exactly as application jobs are.
For postgres, the database name carries the whole instance triple and the role is the application slug. That redundancy is deliberate: the instances are already isolated by being separate jobs on separate volumes, so a shared database name would cost nothing in correctness, but it would mean an operator restoring a dump into the wrong instance gets no error.
The credentials model
The Manager never sees a data store password. There is no field on DataDependency for one, no field on any create or update input, and nothing in the Manager's database, audit rows or GraphQL surface that could hold one.
The lifecycle:
Seeding.
pfy nomad deploy --stackgenerates a password locally and writes it into a Nomad variable atnomad/jobs/<job>, before the store's job is submitted. The store reads the variable at start-up through atemplatestanza, so a variable written afterwards would only take effect on the next restart. The password exists in the CLI process only between generation and the write. It is never returned, logged, printed, or passed on a command line.Existing credentials are left alone. The apply path checks the variable first and writes with a create-only check-index (
-check-index=0), so two applies racing each other cannot rotate a live database's password out from under its consumers.Resolution. Both the store job and every consuming job carry a
templatestanza that dereferences the variable at start-up. The rendered artifacts contain a template dereference, never a value, which is what makes it safe to commit rendered jobs to a repository.Rotation.
pfy data regenerate-credentialsreplaces the value, then restarts the store first and waits for it to come back healthy (up to two minutes) before restarting the consumers. A consumer restarted early would reconnect with a password the database has not adopted yet. It does not read the old value first: nothing about replacing a credential requires seeing the one being replaced.Redis re-reads its password on every start. Postgres would not on its own (
POSTGRES_PASSWORDapplies only when it first initialises its data directory), so the rendered postgres job wraps its entrypoint in a reconcile step that syncs the running database's password to the variable's current value on every start, over the image's trusted local socket. That is what makes a postgres rotation take effect rather than locking every consumer out. The reconcile is a no-op when the password already matches.
pfy data regenerate-credentials main --app shop --env prod
pfy data regenerate-credentials main --app shop --env prod --tenant acme # a dedicated tenant's storeWhat this buys you
The audit trail, the Manager database, the GraphQL API, the rendered job files and your terminal scrollback are all secret-free by construction, not by redaction. There is no path that could put a password in any of them, so there is nothing to remember to scrub.
Injection
Every service that has not opted out receives a connection URL per store, as an environment variable:
| Store | Injected as |
|---|---|
| Oldest postgres | DATABASE_URL |
| Every later postgres | DATABASE_URL_<SLUG> |
| Oldest redis | REDIS_URL |
| Every later redis | REDIS_URL_<SLUG> |
Stores are sorted by creation time before naming, and that sort is load-bearing: only the first store of a kind gets the conventional name, so the order decides which store an application's DATABASE_URL points at.
Creation order is what makes adding a store safe. A new store is necessarily the newest, so it can never take the conventional name from an existing one, whatever its slug sorts as: adding a store called analytics alongside an existing main leaves DATABASE_URL on main. Ties (two stores created in the same instant) fall back to slug order, so the result is still deterministic across applies of an unchanged specification. pfy data list lists stores in creation order.
Only the owner gets the bare name; it has no _<SLUG> alias. A service that must not care which store is oldest should read the suffixed variable of a non-owner store, or opt out of injection and wire its own.
The injected value is itself a template: the address comes from Nomad service discovery (nomadService) and the password from the Nomad variable (nomadVar), so the rendered job never contains a literal host or password. A store with no running instance renders no line at all rather than a URL that points at nothing.
Postgres URLs carry sslmode=disable. The connection never leaves the cluster's bridge network, the store terminates no TLS, and a client defaulting to prefer would negotiate and fall back on every connect.
Opting out
pfy service create worker --app shop --no-data-env
pfy service update worker --app shop --no-data-env
pfy service update worker --app shop --data-env # opt back inA service with injectDataEnv = false receives nothing and renders byte-identically to a service from before data dependencies existed. Use it when the service does its own connection management.
Note the consequence: an opted-out service is also skipped by regenerate-credentials and by the restore path's stop/start list, because it never received the URL. A service that reaches the store through a hand-set environment variable is therefore not covered by either.
Backups
Three fields on the store:
| Field | Meaning |
|---|---|
schedule | Five-field cron expression, evaluated in UTC. Empty means backups are off. |
target | The name of a backup target declared in install.yaml under backup_targets:. |
retention | How many dumps to keep. Unset (0) means the default, 7. |
Postgres stores get a daily schedule (0 3 * * *) when created without one; redis backups are opt-in. A postgres store whose schedule was explicitly cleared is reported in the warnings field of the GraphQL type on every read, so an application does not quietly stop being backed up.
target is a name, not a location
target is a bare name matching [a-z0-9][a-z0-9_-]{0,62}. It is not a path and not a URL, and it cannot contain :, @, /, ? or #.
That is a security property, not a style choice. The Manager stores the value and copies it verbatim into the before/after snapshot of every audit row for the store, and the audit trail is long-lived and widely readable. A field that accepted s3://KEY:SECRET@host/bucket would eventually contain one. A name has nowhere to put a secret.
Where the dumps actually land, and any object-store credentials needed to put them there, are configured by the operator in install.yaml under backup_targets:. The CLI resolves the name at render time; nothing about the target travels through the Manager. Object-store credentials reach the backup job through a Nomad variable named by the target, by the same mechanism the store's own password uses.
An unknown target name fails the stack apply, naming the bad value and listing the configured ones. A schedule with no target is a warning and renders no backup job. That is the normal state between creating a store and declaring the operator's targets, but it is said out loud, because the alternative is a database whose dashboard reads "backups: daily" and whose dumps do not exist.
Full configuration syntax, retention semantics, pfy data backup-now and the restore procedure: Data Store Backup & Restore.
Checking state
pfy data status --app shop --env prod
pfy data status --app shop --env prod --tenant acmeReports each store's instance, image, job, run state, backup policy and backup-job state, plus any warnings the Manager attached. A store with no schedule reads backups OFF (no schedule) in words rather than as a blank column: the difference between "no backups" and "I could not tell" is the whole point of looking.
Applying
Managed stores are applied by the stack path, first, with a health gate before any service that consumes them is submitted:
pfy nomad deploy --stack --env prod --yes
pfy nomad deploy --stack --env prod --only-data --yes # stores + backup jobs onlySee Anatomy of a Stack.
Deleting
pfy data delete main --app shopThe CLI prints which volume will survive, then asks you to type the store's qualified name (<app>/<store>) back before it proceeds; --yes skips the prompt and is required under --ci. Deleting the record removes the job and the generated credentials. The data on disk is untouched and is no longer described by anything in the platform.