Environment variables

Set non-sensitive runtime configuration on an app, and change it on a running one.

Introduction

Environment variables carry the configuration your app reads at runtime and is safe in plain text: a cache directory, a log level, a feature flag, a model revision.

runware serverless deploy ./model.py --id my-app --gpu-type l40s \
  --env HF_HOME=/root/.cache/huggingface \
  --env LOG_LEVEL=info

Anything that would hurt if it leaked belongs in a secret instead.

Changing them on a running app

An app's environment is part of the version snapshot the worker is rendered from. Changing a variable records a new version with the same image and rolls the app, so a running worker picks the value up. A stopped app records the version and rolls it on resume. Writing the value a variable already has records nothing.

A write while a rollout is still in progress returns 409 and is discarded. Setting several variables one at a time is that many rollouts with 409s in between, so change them together in one update of the app.

If the rollout fails, the previous environment keeps serving and listing shows that environment, so retrying the same value is a real change.

runware serverless apps env list my-app

One namespace with secrets

Plain variables and attached secrets land in the same environment on the worker, so their names have to be unique across both.

A variable whose name collides with an attached secret's injected name is rejected with a 422, and attaching a secret over an existing variable is rejected the same way. Neither silently wins, which is deliberate: a credential quietly overridden by a plain variable would slip past everyone.

An app holds at most 100 bindings in total, counting plain variables and attached secrets together. Overwriting an existing one is always allowed, and adding one past the ceiling returns a 422.

Names the platform keeps

The platform sets some variables on the serving container itself. Those names are reserved. Using one returns a 422, which keeps the runtime's own values intact.

Reading them

Nothing special. They are environment variables.

import os

@serve
class FluxDev:
    def load(self) -> None:
        cache = os.environ.get("HF_HOME", "/tmp/hf")
        self.pipe = load_pipeline(cache_dir=cache)

Pointing a library's cache at a volume through its environment variable is the most common use of this, and the one worth getting right first: it is what makes a cold start reuse your weights.

Keeping values out of your shell

A value passed with --env is visible in the process list while the command runs and stays in your shell history afterwards. For anything you would rather not leave there, use a file.

runware serverless deploy ./model.py --id my-app --gpu-type l40s \
  --env-file .env.deploy

.env files are never uploaded with your source, so one sitting in your project directory configures the deploy without traveling into the build.