Search across models, guides, examples and the rest of the docs.
↑↓ Navigateenter Openesc Close
Dashboard

Qwen3.6 35B A3B

A mixture-of-experts model that activates a fraction of its weights per token, in FP8.

Text generationrtx-pro-6000AlibabaApache-2.0Qwen/Qwen3.6-35B-A3B-FP8

What runs

This is the whole app. The entry names the model and what serves it, and the build turns that into these files: there is nothing else to write.

Dockerfile
# Generated from a declarative template. Nothing here asks for an integration:
# the platform reads the label the image carries and attaches its serving layer
# to the `vllm serve` below.
FROM vllm/vllm-openai:v0.28.0

ENV MODEL=Qwen/Qwen3.6-35B-A3B-FP8 \
    MODEL_REVISION=95a723d08a9490559dae23d0cff1d9466213d989 \
    MAX_MODEL_LEN=8192 \
    GPU_MEMORY_UTILIZATION=0.85 \
    MAX_NUM_SEQS=256 \
    HF_HOME=/weights \
    PORT=8080

EXPOSE 8080

ENTRYPOINT ["/bin/sh", "-c", \
  "exec vllm serve \"$MODEL\" --revision \"$MODEL_REVISION\" --port \"$PORT\" --max-model-len \"$MAX_MODEL_LEN\" --gpu-memory-utilization \"$GPU_MEMORY_UTILIZATION\" --max-num-seqs \"$MAX_NUM_SEQS\""]
container.yaml
configVersion: 1

port: 8080

probes:
  readiness: /health
  liveness: /live

timeouts:
  requestSeconds: 300
  # The first boot pulls the weights into the volume. Later boots read them back.
  startupSeconds: 2400

endpoints:
  - path: v1/chat/completions
    input:
      type: object
      additionalProperties: false
      # The server requires both, and a schema that called `model` optional
      # would accept a request the server then answers with a 400.
      required: [model, messages]
      properties:
        model: { type: string }
        messages: { type: array }
        max_tokens: { type: integer }
        temperature: { type: number }
        top_p: { type: number }
        seed: { type: integer }
runware.yaml
configVersion: 1
id: qwen3-6-35b-a3b
name: Qwen3.6 35B A3B
source:
  type: container
compute:
  gpuType: rtx-pro-6000
volumes:
  - /weights
envVars:
  VLLM_USE_DEEP_GEMM: "0"

Deploying

Deploy it as it is, or take the files above and deploy your own copy. Nothing you change reaches back to this page.

qwen3-6-35b-a3brtx-pro-6000source
One-click deploy
From a URL
runware serverless deploy --from https://runware.ai/docs/serverless/examples/qwen3-6-35b-a3b.zip
From a clone
runware serverless deploy --container . \
  --id qwen3-6-35b-a3b \
  --gpu-type rtx-pro-6000 \
  --volume /weights \
  --env-var VLLM_USE_DEEP_GEMM=0

Calling it

One request that works, against the app you just deployed. The fields are the ones the entry declares above, and anything it does not declare is refused rather than ignored.

import { createClient } from '@runware/sdk'

const client = await createClient({ apiKey: process.env.RUNWARE_API_KEY })

const task = await client.invoke({
  appId: 'qwen3-6-35b-a3b',
  endpointPath: 'v1/chat/completions',
  payload: {
    model: 'Qwen/Qwen3.6-35B-A3B-FP8',
    messages: [
      {
        role: 'user',
        content: 'Name three uses for a serverless GPU.'
      }
    ]
  }
})
import asyncio
import os

from runware import Runware


async def main():
    client = Runware(api_key=os.environ["RUNWARE_API_KEY"])
    try:
        task = await client.invoke(
            "qwen3-6-35b-a3b",
            "v1/chat/completions",
            {
                "model": "Qwen/Qwen3.6-35B-A3B-FP8",
                "messages": [
                    {
                        "role": "user",
                        "content": "Name three uses for a serverless GPU."
                    }
                ]
            },
        )
    finally:
        await client.close()


asyncio.run(main())
curl https://api.serverless.runware.ai/v1/apps/qwen3-6-35b-a3b/invoke-async/v1/chat/completions \
  -H "Authorization: Bearer $RUNWARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "taskId": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
    "payload": {
      "model": "Qwen/Qwen3.6-35B-A3B-FP8",
      "messages": [
        {
          "role": "user",
          "content": "Name three uses for a serverless GPU."
        }
      ]
    }
  }'
cat <<'EOF' | runware serverless apps invoke qwen3-6-35b-a3b v1/chat/completions -f -
{
  "model": "Qwen/Qwen3.6-35B-A3B-FP8",
  "messages": [
    {
      "role": "user",
      "content": "Name three uses for a serverless GPU."
    }
  ]
}
EOF
{
  "taskId": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
  "payload": {
    "model": "Qwen/Qwen3.6-35B-A3B-FP8",
    "messages": [
      {
        "role": "user",
        "content": "Name three uses for a serverless GPU."
      }
    ]
  }
}

What it costs

This entry is pinned to rtx-pro-6000, which is the card it was tested on rather than the only one it runs on: --gpu-typedeploys it somewhere else. You pay for that card by the second while a worker holds it, and for nothing else. Pricing covers what meters, and the rates are published per GPU-second.

Ask AI
Send feedback