Environment variables

Set non-sensitive runtime configuration on an app, and change it on a running one.

Introduction

Environment variables carry the configuration your app reads at runtime and is safe in plain text: a cache directory, a log level, a feature flag, a model revision.

runware serverless deploy ./model.py --id my-app --gpu-type rtx-pro-6000 \
  --env-var HF_HOME=/root/.cache/huggingface \
  --env-var LOG_LEVEL=info

Anything that would hurt if it leaked belongs in a secret instead.

--env-var sets a variable and --env picks the environment the command addresses. The CLI tells the two apart, so a KEY=VALUE passed to --env is answered rather than deployed.

Changing them on a running app

An app's environment is part of the version snapshot the worker is rendered from. Changing a variable records a new version with the same image and rolls the app, so a running worker picks the value up. A stopped app records the version and rolls it on resume. Writing the value a variable already has records nothing.

A deploy does not replace them. Your latest write is what the next version is recorded from, so a rollback leaves the current values in place rather than restoring the deployed version's set.

A write while a rollout is still in progress returns 409 and is discarded. Setting several variables one at a time is that many rollouts with 409s in between, so change them together in one update of the app.

If the rollout fails, the previous environment keeps serving and listing shows that environment, so retrying the same value is a real change.

runware serverless apps env-vars list my-app

One namespace with secrets

Plain variables and attached secrets land in the same environment on the worker, so their names have to be unique across both.

A variable whose name collides with an attached secret's injected name is rejected with a 422, and attaching a secret over an existing variable is rejected the same way. Neither silently wins, which is deliberate: a credential quietly overridden by a plain variable would slip past everyone.

An app holds at most 100 bindings in total, counting plain variables and attached secrets together. Overwriting an existing one is always allowed, and adding one past the ceiling returns a 422.

Names the platform keeps

The platform sets some variables on the serving container itself. Those names are reserved. Using one returns a 422, which keeps the runtime's own values intact.

An attached OIDC connector reserves the names it sets in the same way, such as AWS_ROLE_ARN, for as long as it stays attached. GOOGLE_APPLICATION_CREDENTIALS is reserved whether or not one is attached, so an app that points it at a service-account file of its own is refused even with no gcp connector on it.

Reading them

Nothing special. They are environment variables.

import os

@serve
class FluxDev:
    def load(self) -> None:
        cache = os.environ.get("HF_HOME", "/tmp/hf")
        self.pipe = load_pipeline(cache_dir=cache)

Pointing a library's cache at a volume through its environment variable is the most common use of this, and the one worth getting right first: it is what makes a cold start reuse your weights.

Keeping values out of your shell

A value passed with --env is visible in the process list while the command runs and stays in your shell history afterwards. For anything you would rather not leave there, use a file.

runware serverless deploy ./model.py --id my-app --gpu-type rtx-pro-6000 \
  --env-file .env.deploy

.env files are never uploaded with your source, so one sitting in your project directory configures the deploy without traveling into the build.