Model Router

Let the model router choose the model for a task, guided by your own priorities, price ceiling and requirements.

Introduction

Sending "model": "auto" asks the model router to choose the model for you. The request is otherwise the request you would have sent anyway. The task type says what you want made, the parameters say how, and the only thing left open is which model does it.

You never name a model. What you state instead is what the output has to be, in routing, and that is read as conditions a model has to satisfy. A model released next month that meets them can be chosen for the same request, and a model that is retired takes nothing with it. Your code says what the result has to be, once, and does not name a model again.

The simplest case states nothing at all and takes the best answer the catalog has for that task.

Request

Send the task exactly as you otherwise would, with auto in place of an AIR identifier.

import { createClient } from '@runware/sdk'

const client = await createClient({ apiKey: process.env.RUNWARE_API_KEY })
await client.connect()

const [result] = await client.run<'image'>({
  taskType: 'imageInference',
  model: 'auto',
  positivePrompt: 'a ceramic coffee mug on a marble counter',
  width: 1024,
  height: 1024
})

import asyncio
import os

from runware import Runware


async def main():
    async with Runware(api_key=os.environ["RUNWARE_API_KEY"]) as client:
        results = await client.run({
            "taskType": "imageInference",
            "model": "auto",
            "positivePrompt": "a ceramic coffee mug on a marble counter",
            "width": 1024,
            "height": 1024
        })


asyncio.run(main())

curl https://api.runware.ai/v1 \
  -H "Authorization: Bearer $RUNWARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[
    {
      "taskType": "imageInference",
      "taskUUID": "50836053-a0ee-4cf5-b9d6-ae7c5d140ada",
      "model": "auto",
      "positivePrompt": "a ceramic coffee mug on a marble counter",
      "width": 1024,
      "height": 1024
    }
  ]'

runware run auto \
  --task-type imageInference \
  positivePrompt="a ceramic coffee mug on a marble counter" \
  width=1024 \
  height=1024

{
  "taskType": "imageInference",
  "taskUUID": "50836053-a0ee-4cf5-b9d6-ae7c5d140ada",
  "model": "auto",
  "positivePrompt": "a ceramic coffee mug on a marble counter",
  "width": 1024,
  "height": 1024
}

Everything you send is kept exactly as you sent it. The router only fills in what you left out, so a width of 1024 stays 1024 and a prompt is never rewritten. A model that cannot accept one of the values you sent is not chosen.

The response has the shape that task always has, and the task runs the moment a model is chosen. There is nothing extra to poll and no second request to make.

Supported task types

Task type	Task type
`imageInference`	`caption`
`videoInference`	`upscale`
`audioInference`	`removeBackground`
`textInference`	`vectorize`
`3dInference`	`imageMasking`
`promptEnhance`	`controlNetPreprocess`

The router chooses between the models the catalog has for that task, and that is anywhere from one to well over a hundred. Where there is one, it sends the request to it and there is nothing to weigh.

The routing object

Every model left standing is one that can serve the request, so something has to choose between them. That choice is made on measured evidence, several signals weighed together, among them how the model scores on public benchmarks, how long its runs take and what it costs. The optional routing object changes how they are weighed, and what the result has to be.

import { createClient } from '@runware/sdk'

const client = await createClient({ apiKey: process.env.RUNWARE_API_KEY })
await client.connect()

const [result] = await client.run<'video'>({
  taskType: 'videoInference',
  model: 'auto',
  positivePrompt: 'a drone shot over a harbor at sunrise',
  duration: 5,
  routing: {
    preferences: {
      quality: 5,
      speed: 2,
      cost: 1
    },
    maxPrice: 0.4,
    requirements: 'with audio, 1080p or better'
  }
})

import asyncio
import os

from runware import Runware


async def main():
    async with Runware(api_key=os.environ["RUNWARE_API_KEY"]) as client:
        results = await client.run({
            "taskType": "videoInference",
            "model": "auto",
            "positivePrompt": "a drone shot over a harbor at sunrise",
            "duration": 5,
            "routing": {
                "preferences": {
                    "quality": 5,
                    "speed": 2,
                    "cost": 1
                },
                "maxPrice": 0.4,
                "requirements": "with audio, 1080p or better"
            }
        })


asyncio.run(main())

curl https://api.runware.ai/v1 \
  -H "Authorization: Bearer $RUNWARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[
    {
      "taskType": "videoInference",
      "taskUUID": "50836053-a0ee-4cf5-b9d6-ae7c5d140ada",
      "model": "auto",
      "positivePrompt": "a drone shot over a harbor at sunrise",
      "duration": 5,
      "routing": {
        "preferences": {
          "quality": 5,
          "speed": 2,
          "cost": 1
        },
        "maxPrice": 0.4,
        "requirements": "with audio, 1080p or better"
      }
    }
  ]'

runware run auto \
  --task-type videoInference \
  positivePrompt="a drone shot over a harbor at sunrise" \
  duration=5 \
  routing.preferences.quality=5 \
  routing.preferences.speed=2 \
  routing.preferences.cost=1 \
  routing.maxPrice=0.4 \
  routing.requirements="with audio, 1080p or better"

{
  "taskType": "videoInference",
  "taskUUID": "50836053-a0ee-4cf5-b9d6-ae7c5d140ada",
  "model": "auto",
  "positivePrompt": "a drone shot over a harbor at sunrise",
  "duration": 5,
  "routing": {
    "preferences": {
      "quality": 5,
      "speed": 2,
      "cost": 1
    },
    "maxPrice": 0.4,
    "requirements": "with audio, 1080p or better"
  }
}

preferences object

Relative importance of each dimension when models are compared. Weights, not thresholds.

Properties3 properties

preferences » quality quality integermin: 0max: 5: Importance of one dimension. 0 leaves it out of the decision.

preferences » speed speed integermin: 0max: 5: Importance of one dimension. 0 leaves it out of the decision.

preferences » cost cost integermin: 0max: 5: Importance of one dimension. 0 leaves it out of the decision.

maxPrice floatmin: 0: Upper bound on what this request may cost, in USD. A model whose cost cannot be bounded is not chosen.

requirements stringmax: 3000: Conditions the result has to meet, in plain words, such as a style or a feature. A model that cannot meet them is not chosen. Output size is set with width and height rather than described here.

Leaving preferences out weighs the three evenly.

A preference orders the models that fit, a requirement decides which ones fit at all.

requirements describes the result, the prompt describes the subject, so "with audio, 1080p or better" belongs in one and the scene in the other. A request that carries no prompt has only the one, and the prompt is then written from the requirements as well.

How a price ceiling is read

maxPrice is measured against the request you sent, not against a model's headline rate. A model billed by the second is costed over the duration you asked for, one billed per image over the number of results you asked for, so the same number means the same thing whichever model is being considered.

Models often publish several rates for one unit, one per quality tier or resolution. The ceiling is checked against the dearest of them, because which tier a request lands on is settled when it runs. A model that would have come in under your ceiling on a cheaper tier can be turned away by it. Raise the number and it comes back.

Some models cannot be costed ahead of the run at all. While a ceiling is set those are not chosen, since there is no figure to check it against. Send the request without maxPrice to reach them, and weigh cost in preferences instead.

When nothing is chosen

The router refuses rather than guessing. Every refusal names what stopped it, so the request can be corrected without having to read the catalog.

modelRouterUnsupportedTaskType: Routing is not available for that task type. The message names the current task type name when the one sent has been superseded.

noEligibleModel: No model can serve the request as sent. The reason names what ruled the rest out, such as a parameter no model of that kind accepts, a value outside every model's range, or a price ceiling nothing meets.

modelRouterNeedsDescription: The request carries media but nothing that says what to make, and the chosen model cannot run without it. Describe the result in routing.requirements, or send the parameter the message names.

invalidParameter: A parameter no model declares, or an unknown key inside routing. The message names the parameter, and a near miss such as inputs.strength for strength is named with it.

modelRouterUnavailable retryable: Routing is temporarily unavailable. Retry, or send the request with a specific model.

The router chooses from the official models, the ones in the model catalog. Community models and your organization's private models are not routed to, and stay available by naming their AIR identifier.