---
title: Embeddings | Runware Docs
url: https://runware.ai/docs/serverless/examples/embeddings
description: Embed many texts in one request, and batch them inside your handler because the platform will not do it for you.
relatedDocuments:
  - https://runware.ai/docs/serverless/writing-a-model
  - https://runware.ai/docs/serverless/compute-and-scaling
  - https://runware.ai/docs/serverless/volumes
---
## Introduction

A text embedding model that takes a list of texts and returns one vector per text.

`bge-small-en-v1.5` is MIT and ungated, so you deploy it with **no token**.

## The app

**model.py**:

```python
"""Many inputs in one request: a list of texts in, a list of vectors out."""

from __future__ import annotations

from pydantic import BaseModel, ConfigDict, Field

from runware_serverless import serve

MODEL = "BAAI/bge-small-en-v1.5"
"""MIT and ungated, 384 dimensions, small enough that the GPU is never the limit."""

CACHE = "/weights"
"""Mount a volume here. Small as this model is, a checkpoint captures the worker's
filesystem, so weights left outside a volume are copied into every one."""

MAX_TEXTS = 256
"""The largest list this app accepts.

Measured: the platform checks the types it published and rejects a wrong one in
under half a second without waking anything, but it does not check a list's
length, so a request carrying more than this reaches the worker and fails there.
This app enforces the value constraint itself, and a request that breaks one
wakes a worker before it is refused."""

ENCODE_BATCH = 64
"""How many texts cross the GPU at once inside one request. Unrelated to how many
the request carries: the platform gives a worker one task at a time, so batching
is something you do inside your handler or not at all."""

class Request(BaseModel):
    # Without this pydantic ignores a field it does not know, so a caller who
    # misspells one gets a silent success. An unknown field is an error here.
    model_config = ConfigDict(extra="forbid")

    texts: list[str] = Field(
        min_length=1,
        max_length=MAX_TEXTS,
        description="The texts to embed. One request, one pass over the GPU.",
    )
    normalize: bool = Field(
        default=True,
        description="Return unit vectors, so a dot product is a cosine similarity.",
    )

class Embedded(BaseModel):
    vectors: list[list[float]] = Field(description="One vector per input, in order.")
    dimensions: int
    count: int

@serve
class Embeddings:
    requirements = [
        "torch==2.14.1",
        "transformers==5.21.0",
        "sentence-transformers==6.2.0",
    ]

    def load(self) -> None:
        if hasattr(self, "model"):
            return

        # Here and not at the top of the file. The build imports this file with the
        # platform's own dependencies, not yours, so nothing in requirements exists
        # until a worker runs. A `with imports():` block at the top of the file does
        # the same, and is what to use when a name has to be visible outside a method.
        from sentence_transformers import SentenceTransformer

        self.model = SentenceTransformer(MODEL, cache_folder=CACHE, device="cuda")
        self.model.encode(["warmup"])

    def predict(self, request: Request) -> Embedded:
        vectors = self.model.encode(
            request.texts,
            batch_size=ENCODE_BATCH,
            normalize_embeddings=request.normalize,
        )

        return Embedded(
            vectors=vectors.tolist(),
            dimensions=int(vectors.shape[1]),
            count=len(request.texts),
        )
```

## Deploying the app

embeddingsrtx-pro-6000[source](https://github.com/Runware/runware-serverless-examples/tree/main/embeddings)

[One-click deploy](https://runware.ai/serverless/apps?template=embeddings)

**From a URL**:

```bash
runware serverless deploy --from https://runware.ai/docs/serverless/examples/embeddings.zip
```

**From a clone**:

```bash
runware serverless deploy model.py \
  --id embeddings \
  --gpu-type rtx-pro-6000 \
  --volume /weights
```

The model is 130MB, small enough that you might skip the volume. Mount it anyway: a checkpoint captures the worker's filesystem, so an unmounted download is copied into every one.

## Batching inside one request

A worker takes **one task at a time**. There is no setting that lets several requests share a worker, so the only batching available to you is the batching you put inside one request.

That is why the request takes a list rather than a string.

Measured on an RTX PRO 6000, against a worker already serving:

<table><tbody><tr><td>Texts in one request</td><td>Time on the worker</td><td>Per text</td></tr><tr><td>1</td><td>0.62s</td><td>620ms</td></tr><tr><td>8</td><td>0.41s</td><td>52ms</td></tr><tr><td>32</td><td>0.54s</td><td>17ms</td></tr><tr><td>128</td><td>0.80s</td><td>6.2ms</td></tr><tr><td>256</td><td>0.65s</td><td>2.5ms</td></tr></tbody></table>

**The time barely moves between one text and 256**, and eight came back faster than one, because on a warm worker almost all of it is fixed overhead rather than model work.

Sending those 256 texts one per request would pay 256 queue hops and 256 times that fixed cost for the same work.

## What the platform checks

Check this before you rely on a schema to keep bad requests away from your worker. Measured against this app:

<table><tbody><tr><td>Request</td><td>What happened</td></tr><tr><td><code>texts</code> as a string</td><td><strong>422 in 0.4s</strong>, nothing woken</td></tr><tr><td><code>texts</code> holding integers</td><td><strong>422</strong>, with a pointer per bad index</td></tr><tr><td><code>normalize</code> as a string</td><td><strong>422 in 0.1s</strong></td></tr><tr><td>An empty list, against <code>min_length</code></td><td><strong>reached the worker</strong> and failed there</td></tr><tr><td>257 texts, against <code>max_length</code></td><td><strong>reached the worker</strong> and failed there</td></tr></tbody></table>

A type is checked before anything wakes. **A value constraint is enforced by the app, not the platform**, and a request that breaks one wakes a worker before it is refused.

> [!WARNING]
> Pydantic **ignores an unknown field** by default, so a caller who writes `normalise` instead of `normalize` gets a silent success and the wrong behavior. `model_config = ConfigDict(extra="forbid")` turns that into an error, and the platform publishes and enforces it: the same misspelled request went from a completed task on a woken worker to **a 422 in half a second with nothing woken**.

## Calling the app

The class declares no `@endpoint`, so `@serve` exposes `predict` at the default path.

**TypeScript**:

```typescript
import { createClient } from '@runware/sdk'

const client = await createClient({ apiKey: process.env.RUNWARE_API_KEY })

const task = await client.invoke({
  appId: 'embeddings',
  endpointPath: 'predict',
  payload: {
    request: {
      texts: [
        'a red bicycle',
        'a blue car'
      ]
    }
  }
})
```

**Python**:

```python
import asyncio
import os

from runware import Runware

async def main():
    client = Runware(api_key=os.environ["RUNWARE_API_KEY"])
    try:
        task = await client.invoke(
            "embeddings",
            "predict",
            {
                "request": {
                    "texts": [
                        "a red bicycle",
                        "a blue car"
                    ]
                }
            },
        )
    finally:
        await client.close()

asyncio.run(main())
```

**cURL**:

```bash
curl https://api.serverless.runware.ai/v1/apps/embeddings/invoke-async/predict \
  -H "Authorization: Bearer $RUNWARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "taskId": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
    "payload": {
      "request": {
        "texts": [
          "a red bicycle",
          "a blue car"
        ]
      }
    }
  }'
```

**CLI**:

```bash
cat <<'EOF' | runware serverless apps invoke embeddings predict -f -
{
  "request": {
    "texts": [
      "a red bicycle",
      "a blue car"
    ]
  }
}
EOF
```

**JSON**:

```json
{
  "taskId": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
  "payload": {
    "request": {
      "texts": [
        "a red bicycle",
        "a blue car"
      ]
    }
  }
}
```

The vectors come back at `output.predictions[0].vectors`, one per input and in the same order. They are unit length unless you ask for `normalize: false`, so a dot product between two of them is their cosine similarity.

Measured on an RTX PRO 6000: a **2m57s** build, **112s** for the first call with an empty volume, 14 seconds for a restart that finds the weights already there, and 0.6 seconds on a worker already serving.