Writing a model

Declare endpoints with the decorator, load your weights once, and let the build derive the request schema from your handler signatures.

Introduction

A code application is one Python class in one file. You write the class, mark the methods you want callable, and mark the class itself. The platform builds the image, starts the worker, and routes each request to the method that answers it.

from runware_serverless import endpoint, serve


@serve
class ImageTools:
    def load(self) -> None:
        self.pipe = load_my_pipeline()

    @endpoint
    def generate(self, prompt: str, steps: int = 4) -> dict:
        steps = 4 if steps is None else steps
        return {"image": self.pipe(prompt, steps).encode()}

    @endpoint
    def upscale(self, image: str, scale: int = 2) -> dict:
        scale = 2 if scale is None else scale
        return {"image": self.pipe.upscale(image, scale).encode()}

That file serves two endpoints, generate and upscale, on one worker.

load

load runs once per worker, before any request reaches it. Put your weights, your pipeline and anything else expensive here, so the cost lands at startup and callers reach a warm worker.

Everything you assign to self in load is available to every handler for the life of that worker.

Import your heavy dependencies inside load or inside a handler, not at the top of the file. The build imports your file to find your endpoints, and it does that without those dependencies installed, so a module-level import torch fails the build. Every example that touches a real model imports this way.

load is required, even when there is nothing to load. The build identifies your class by looking for one that defines load alongside its handlers, so a class missing it stays invisible to the build, which fails with no deployment class found. Write def load(self) -> None: pass when you have nothing to put in it.

There is no matching shutdown hook. And load is retried when the GPU is out of memory at startup, waiting for a larger allocation each time. The retry re-instantiates your class, so both __init__ and load have to be safe to run twice.

The endpoint decorator

@endpoint marks a method as callable. Bare, the path is the method name with each underscore replaced by a hyphen, so remove_background answers at remove-background. That keeps a path identifying its handler on sight, straight from the URL to the method that runs.

It takes two optional keyword arguments, and nothing else:

path names a route the method name cannot spell, such as @endpoint(path="v1/chat/completions").
default=True marks the endpoint a root invocation reaches, one that names no endpoint path. An app marks one at most.

A positional argument such as @endpoint("upscale"), a keyword other than those two, a second default=True, or a path that is not legal fails the build, before a deployment exists.

A class with no @endpoint methods falls back to predict, so a single-endpoint model needs no decorator at all. An app with exactly one endpoint is its own default, so you can invoke it without naming the path. The moment you declare one endpoint, only declared paths are served, and predict returns 404 like any other path you did not declare.

One app declares at most 20 endpoints. Going over fails the build rather than producing something the platform would refuse later.

Endpoints you inherit

Decorated methods are found through the whole inheritance chain, so a shared base class serves its endpoints to every model built on it.

class ImageBase:
    @endpoint
    def upscale(self, image: str, scale: int = 2) -> dict: ...


@serve
class ImageTools(ImageBase):
    def load(self) -> None: ...

    @endpoint
    def generate(self, prompt: str) -> dict: ...

ImageTools serves both generate and upscale. An override serves its own implementation whether or not you decorate it again, so removing the decorator from an override keeps the path served. The alternative would drop a served endpoint silently, with nothing to report it.

The schema comes from your signature

The build reads each handler's signature and records one manifest entry per endpoint.

The rules are short. A parameter without a default becomes required. A parameter with a default becomes optional. And the schema is closed, so a body carrying a field outside the signature is rejected rather than ignored.

For the class above, a request to generate must carry prompt and may carry steps. Sending image to generate fails with a 422 before it reaches your worker, because image belongs to upscale.

An omitted optional field arrives as an explicit null, not as a missing key, so a Python default alone will not fire. Apply the default inside the handler, as steps = 4 if steps is None else steps does above.

Every parameter has to be fillable by name, because each request is dispatched as handler(model, **fields). A positional-only parameter, *args or **kwargs fails the build rather than shipping an endpoint that cannot be called.

Typed requests with Pydantic

A parameter annotated with a Pydantic model is validated into that model before your handler runs, and the model's own JSON Schema becomes that field's schema in the contract.

from pydantic import BaseModel
from runware_serverless import endpoint, serve


class Transcribe(BaseModel):
    audio_url: str
    language: str | None = None


@serve
class Whisper:
    def load(self) -> None:
        self.pipe = load_my_pipeline()

    @endpoint
    def transcribe(self, req: Transcribe) -> dict:
        return {"text": self.pipe(req.audio_url, language=req.language)}

req arrives as a Transcribe instance, not a dictionary, so the annotation is a contract the platform enforces. It behaves the way the same signature does on a FastAPI route.

This is the most precise contract you can publish. A bare dict leaves the caller guessing, and every field you declare on a model is a field the platform can reject before your worker is woken.

An annotation the build cannot resolve degrades to "anything" rather than failing. A type that only exists at serving time, imported under if TYPE_CHECKING, is the usual cause: the build reads your file without your heavy dependencies installed, so the name is missing at build time. The endpoint still works, and its schema stops describing that field.

Returning a model declares your response shape in the same way. Only the request side is coerced, though: the value your handler returns goes out as it is, and nothing checks it against the declared type. A response type that is not JSON needs encode_response below.

Coercion reaches a model, or a model that may be null. It does not reach a model inside a container: a parameter annotated list[Item] arrives as a list of plain dictionaries, not a list of Item. Declare a wrapper model with the list as a field when you want a batch validated.

serve

@serve registers the class and hands it straight back, so the file stays importable for the class it defines. Your own tests can import ImageTools and call its methods directly without any of the serving machinery running.

It reads as a decorator above the class, and serve(ImageTools) after the class body does the same thing if you prefer that shape.

Endpoints are resolved when the module is imported, not on the first request, so a class with nothing to serve fails in the build pod, with the reason in its log.

A model file where serve registered no class fails the build too, before the manifest is written, so a missing decorator surfaces in the build log rather than at the first request.

Returning something your signature does not describe

The schema is derived from your handler's return type, which is a problem the moment your wire format differs from it. An image model that returns a PIL image cannot declare that, and encoding inside the handler would make the derived contract lie.

encode_response is the seam for exactly that. It maps the handler's results to what goes on the wire, and leaves the signature describing what the handler really returns.

@serve(encode_response=to_base64_png)
class FluxDev:
    ...

Reach for it when your response needs a transport encoding. Leave it alone otherwise, because every use of it is a place where the declared contract and the wire diverge on purpose.

Endpoints are not units of compute

Every endpoint on an app shares one queue and one worker pool. Hardware and scaling are set once, on the app, and apply to all of them.

The platform enforces that at three points: the decorator accepts only path and default, the create and update routes reject an endpoints field, and a container configuration carrying a hardware key is a 422.

If two endpoints genuinely need different hardware, they are two apps.

Dependencies

List what your model imports beyond the standard library. There are three places to put that list, and the build reads the first one it finds: a requirements attribute on your model class, then whatever you passed as --requirement on the deploy, then a pyproject.toml and uv.lock in your source directory.

The attribute keeps the list beside the code that needs it, so a file that moves between projects carries its own dependencies.

@serve
class MyModel:
    requirements = ["torch==2.5.1", "diffusers>=0.31"]

    def load(self) -> None:
        ...

It has to be a list or tuple of strings. The name is reserved, so a handler called requirements is refused and has to be renamed. And declaring the list on the class and on the deploy at the same time fails the build rather than picking one, because picking one would silently drop the other.

Leave runware-serverless out of it. The platform's own client libraries are installed into the image at build time and again when the container starts, so a version pinned in your requirements would drift from the services it talks to.

What the build actually ships

The build ships the importable code and leaves the rest behind: a .py, .so or .pyd file, or a directory carrying an __init__.py. A folder of loose files beside your model file is not a package, so it stays behind.

Each entry it does take is capped at 100 MiB. Weights belong on a volume or in the mounted cache, fetched in load.

A model file that reads a data file next to itself, say a prompts.txt at the root, will not find it in the worker. Move it inside a package that ships, or read it at runtime from a mounted path.

Every entry the build skips is named in the build log with the reason, so a missing file is something you can look up.

For a whole file rather than the pieces, the examples are working apps you can deploy as they are.