---
title: Compute and scaling | Runware Docs
url: https://runware.ai/docs/serverless/compute-and-scaling
description: Choose a GPU, set the bounds the platform scales between, and change any of it on a running app without a rebuild.
relatedDocuments:
  - https://runware.ai/docs/serverless/pricing
  - https://runware.ai/docs/serverless/monitoring
  - https://runware.ai/docs/serverless/versions
---
## Introduction

An app's `configuration` says **what hardware its workers run on and how many of them there may be**. The platform stays inside those bounds and decides the rest.

```json
{
  "gpuType": "l40s",
  "maxWorkers": 4,
  "idleTtlSecs": 300,
  "scalingDelaySecs": 30,
  "minWorkers": 0
}
```

**Four of these are required**: `gpuType`, `maxWorkers`, `idleTtlSecs` and `scalingDelaySecs`. The rest have defaults.

## Choosing a GPU

`gpuType` is a lowercase catalog code such as `l40s` or `h100`. It must match an id from `GET /v1/gpu-types` **that currently has capacity**, so the list you get back is what you can actually deploy on rather than everything that exists.

Every app must pin a GPU type. Omitting it **fails with a `422` before a build starts**, because an unpinned app would fall through to whatever default the deployer renders.

`fallbackGpuType` names a **substitute GPU type** for the app. Placement uses it when the pool behind `gpuType` has no recorded inventory, or when no complete block is free there. It stays unused when placement is merely contending for a busy pool, when a scan reaches its limit, or when your credit does not cover `gpuType`.

A worker placed on the substitute **reports it as its own `gpuType`**, and one app's workers can sit on both types at once. **The substitute bills at its own catalog rate**, including when that rate is higher than the type you preferred, and a commitment bought for `gpuType` does not cover it.

The value has to **differ from `gpuType` and resolve to capacity in the same cluster**, and it has to grant the `gpusPerWorker` you asked for. Send an empty string to clear it.

## More than one GPU per worker

A worker holds its GPUs as **one group the cluster grants indivisibly**, so `gpusPerWorker` is a group size: `1`, `2`, `4` or `8`.

The size also has to be one the cluster behind your `gpuType` grants. A size it does not grant answers **`422` naming `/configuration/gpusPerWorker`**, because the pod would have nowhere to run.

Not every GPU type can hold a group at all. `GET /v1/gpu-types` reports **`multiGpuSupport` on each entry**, and a type that reports `false` holds exactly one GPU per worker.

> [!WARNING]
> Raising it above `1` needs a redeploy, not only a new number: **an image built for a single rank holds every GPU it was granted and uses one**.

> [!NOTE]
> GPU is the only compute type. `computeType` defaults to it and **naming `cpu` is a `422`**.

## Bounds, not instructions

The platform starts and stops workers for you. You set the edges.

<table><tbody><tr><td>Field</td><td>What it does</td><td>Default</td></tr><tr><td><code>minWorkers</code></td><td>The floor. <code>0</code> allows the app to release every worker when nothing is arriving</td><td><code>0</code></td></tr><tr><td><code>maxWorkers</code></td><td>The ceiling, and your spend limit. <strong>Required</strong></td><td>none</td></tr><tr><td><code>idleTtlSecs</code></td><td>How long an idle worker survives before it is reclaimed. <strong>Required</strong></td><td>none</td></tr><tr><td><code>scalingDelaySecs</code></td><td>The damping window before a scaling decision acts. <strong>Required</strong></td><td>none</td></tr><tr><td><code>minAvailableWorkers</code></td><td>Idle workers held above current demand, so a burst lands on workers that are already up. Must be lower than <code>maxWorkers</code></td><td>none</td></tr><tr><td><code>availableWorkersPct</code></td><td>The same buffer as a percentage of current demand, rounded up. When both are set, the larger applies</td><td>none</td></tr><tr><td><code>startupTimeoutSecs</code></td><td>How long readiness has before the pod is failed, from 1 to 2400</td><td>platform</td></tr><tr><td><code>requestTimeoutSecs</code></td><td>How long the platform waits on one forwarded request, from 1 to 900. Container apps only</td><td>platform</td></tr></tbody></table>

> [!NOTE]
> **`requestTimeoutSecs` bounds the worker, not your HTTP wait.** `invoke-sync` still answers `504` at the platform deadline so you can go on polling the task. A code app stores and returns the field but does not apply it, because it keeps the platform read ceiling.

> [!WARNING]
> On a container app **a later deploy overwrites both from `container.yaml`**, out of its `timeouts.requestSeconds` and `timeouts.startupSeconds`. Setting them through the API and then deploying that document puts the document's values back.

> [!NOTE]
> The buffer only holds **while work is queued**. An idle app still scales down to `minWorkers`, so the floor is what holds capacity through a quiet stretch. While demand holds steady, the platform skips a buffer below about a tenth of current demand.

> [!NOTE]
> **Your credit balance can lower the ceiling.** When your organization's credit cannot back every worker up to `maxWorkers` at deploy time, the platform applies a lower ceiling and reports it on the app as `effectiveMaxWorkers`. The app scales only up to it. `maxWorkers` keeps the value you set, and a top-up raises a running app's ceiling back without a redeploy.

## The trade you are actually making

Scaling to zero is the reason this product is cheap, and the reason a cold request is slow. **A request arriving at an app with no workers waits for one to start and for your model to load.**

The levers are few and they pull against each other.

**`minWorkers` above zero** keeps capacity ready and removes the cold start for the traffic that fits inside it. You pay for that GPU whether or not anything arrives.

**A buffer** keeps spare workers ahead of a burst while traffic is flowing, and releases them when the queue empties.

**A longer `idleTtlSecs`** keeps a worker alive through gaps in traffic, so the second request after a quiet minute is fast. You pay for the idle time you did not use.

**A longer `scalingDelaySecs`** stops a brief spike from starting workers you will not need. It also delays the response to a real one.

Caching weights on a [volume](https://runware.ai/docs/serverless/volumes) attacks the same problem from the other side: **a worker that already has the weights on disk starts faster than one that has to fetch them**, and that is where most of a cold start goes.

## Changing it on a running app

Configuration lives on the app, **separate from your source**, and can be changed while the app is serving.

```text
runware serverless apps scale <appId> --max-workers 16
```

Every field except `computeType` can be patched this way. **A field you omit is left alone**, so a patch touches only what you name.

> [!WARNING]
> Clearing a value by sending `null` is not supported. **Omit the field instead**, which leaves it unchanged. The one exception is `fallbackGpuType`, which an empty string clears.

Two consequences worth knowing. **Changing `gpuType` affects only workers created after the change**, so the ones already running finish on the hardware they started on. And a patch is rejected with a `422` if the new type has no capacity on offer, which is the same check that applies at create.

On a running app, **a change that raises `minWorkers` or moves to another `gpuType` is checked against your credit balance** and returns `402` when the balance cannot back it. A change that lowers capacity is never refused that way, though the rollout it starts reads the balance again like any other.

## What it costs you in versions

A configuration change **produces a new version but not a new build**. The image is carried forward untouched, so tuning your scaling is fast and cheap, and the version history still records exactly what changed.

That is also why **your scaling survives a redeploy of your code**: the values live on the app.