Model Schemas

Rules between parameters

Most parameters are valid whatever inputs are sent. H3 has 20 parameters outside inputs, and only three depend on anything else: width, height and resolution. What ties them to the inputs is a set of rules under allOf, at the root of the task schema.

The rules are standard JSON Schema, and nearly all of them take one of five shapes.

This is the H3 rule that removes the size parameters once a frame is sent.

{
  "if": {
    "properties": { "inputs": { "required": ["frameImages"] } },
    "required": ["inputs"]
  },
  "then": {
    "not": {
      "anyOf": [{ "required": ["width"] }, { "required": ["height"] }]
    }
  }
}

The condition is a presence check. if matches when inputs.frameImages is sent, and then rejects the request if width or height came with it. With a frame, the output takes its aspect ratio from the image and resolution sets its size.

Dimensions

Many models accept a closed list of width and height pairs instead of any size. The list is a oneOf, and each entry carries a title you can show as a label. This is the start of the H3 list.

{
  "oneOf": [
    { "title": "768p (~16:9)", "properties": { "width": { "const": 1344 }, "height": { "const": 768 } } },
    { "title": "768p (4:3)", "properties": { "width": { "const": 1024 }, "height": { "const": 768 } } },
    { "title": "768p (1:1)", "properties": { "width": { "const": 768 }, "height": { "const": 768 } } }
  ]
}

A size outside the list is rejected, and on H3 that includes 1920 × 1080.

Limits on the media you send

JSON Schema can check that referenceVideos is a list of URLs. It cannot check how long each clip runs. Limits that depend on the file itself are in x-constraints, at the root of the schema.

[
  {
    "operation": "duration",
    "parameters": ["inputs.referenceVideos", "inputs.referenceAudios"],
    "minimum": 2,
    "maximum": 15,
    "scope": "each"
  },
  {
    "operation": "duration",
    "parameters": ["inputs.referenceVideos", "inputs.referenceAudios"],
    "maximum": 15
  }
]

On H3 each reference clip runs between 2 and 15 seconds, and the clips of one request add up to 15 seconds at most. Every entry is built from the same fields.

operation takes one of these values.

A JSON Schema validator skips x-constraints, so a request can pass validation and still break one of these limits. Check them in your own code before uploading a file.

Validating a request

A JSON Schema validator applies every rule at once, so an interface can check a request as the user fills it in and before anything is sent.

import Ajv from 'ajv/dist/2020.js'

const response = await fetch('https://runware.ai/docs/models/minimax-h3/schema.json')
const document = await response.json()
const task = document.components.schemas.RequestBody.items

const compile = (schema) => new Ajv({ strict: false, validateFormats: false }).compile(schema)
const validate = compile(task)

const base = {
  taskType: 'videoInference',
  taskUUID: crypto.randomUUID(),
  model: 'minimax:h3@0',
  positivePrompt: 'The worker lifts the mature wasabi root smoothly into view.',
}

const firstFrame = {
  frameImages: ['https://assets.runware.ai/assets/inputs/93bc5eb1-8f3d-4c56-9232-06f3da7dce28.jpg'],
}

validate({ ...base, width: 1344, height: 768 })                      // true
validate({ ...base, resolution: '768p', inputs: firstFrame })        // true
validate({ ...base, width: 1344, height: 768, inputs: firstFrame })  // false

One schema accepts the request with a frame and the request without one, and rejects the size parameters that do not fit the inputs sent. The rejected call reports schemaPath, the location of the rule that failed. Here it is #/allOf/3/then/not, the fourth rule of the task. An interface can use it to disable the control, and an agent can read the rule and correct the payload.

A few models tie one value to another, so validate the final request with every value in place. A model can offer durations of 5, 8 and 10 seconds and sizes up to 1080p, and still take 5 or 8 seconds only at 1080p.

The TypeScript and Python SDKs run this validation for you, on every request, before it leaves your machine.

Published examples

Every model also publishes examples, each with the request that produced it. info["x-examples"] links to them, and they are there when you would rather adapt a request that ran than write one from an empty object.

The published requests leave out taskUUID, so add one before sending.

The modes of a model

info["x-modes"] lists every valid combination of inputs. The list is shortened here.

curl -s https://runware.ai/docs/models/minimax-h3/schema.json | jq '.info["x-modes"]'

[
  { "id": "text", "inputs": [] },
  { "id": "reference-images", "inputs": ["referenceImages"] },
  { "id": "first-frame", "inputs": ["frameImages"], "frames": ["first"] },
  { "id": "first-last-frames", "inputs": ["frameImages"], "frames": ["first", "last"] },
  { "id": "reference-videos", "inputs": ["referenceVideos"] },
  { "id": "reference-images+reference-videos", "inputs": ["referenceImages", "referenceVideos"] }
]

The list already applies the rules between inputs. H3 has no mode that mixes frameImages with a reference input, and no reference-audios mode on its own, because an audio reference needs an image or video reference next to it. A combination missing from the list is one the model rejects.

Choosing which modes to expose

Every entry is a valid request, so the full list is the whole menu. One item per mode is the predictable choice: each item maps to one id and shows exactly the inputs it sends.

For fewer items, keep the modes whose inputs are not contained in a larger one, and treat their inputs as optional.

const modes = document.info['x-modes']

const widest = modes.filter((mode) => (
  !modes.some((other) => (
    other.inputs.length > mode.inputs.length
    && mode.inputs.every((input) => (other.inputs.includes(input)))
  ))
))

On H3 that leaves three: first-frame, first-last-frames, and reference-images+reference-videos+reference-audios.

The list keeps growing as models accept more combinations, and each new one is another item to add. The interface described in Building on the schema takes them all and stays the same.

Parameters of a mode

Most of the schema is the same in every mode. A few parameters are ruled out, and those few are all that separates one mode from the next.

On H3 the three parameters named in a rule are the only difference between its nine modes.

Many models have no rule at all, and each of their modes takes the full schema. For the rest, collecting the names that appear under allOf gives the short list to resolve. Not every name on it changes with the mode, and the test in the next section settles which ones do.

The snippets in this section reuse document, task, compile, base and firstFrame from Validating a request.

const ruleNames = (node, names = new Set()) => {
  if (Array.isArray(node)) {
    node.forEach((item) => ruleNames(item, names))
  } else if (node && typeof node === 'object') {
    for (const name of node.required ?? []) { names.add(name) }
    for (const name of Object.keys(node.properties ?? {})) { names.add(name) }
    Object.values(node).forEach((value) => ruleNames(value, names))
  }

  return names
}

const changing = [...ruleNames(task.allOf)]
  .filter((name) => (name !== 'inputs' && name in task.properties))

// ['resolution', 'width', 'height']

A schema per mode

Adding one constraint to the task schema narrows it to a single mode: the inputs of the mode become required and every other input is ruled out. The result is a regular JSON Schema, so a product with one item per mode can store one of these next to each item.

const modeSchema = (mode) => {
  const pinned = {}

  const count = (mode.frames)
    ? { minimum: mode.frames.length, maximum: mode.frames.length }
    : mode.frameCount

  if (count) {
    pinned.frameImages = { minItems: count.minimum, maxItems: count.maximum }
  }

  for (const [path, value] of Object.entries(mode.values ?? {})) {
    const [input, field] = path.split('.')
    pinned[input] = { items: { required: [field], properties: { [field]: { const: value } } } }
  }

  return {
    allOf: [task, {
      required: (mode.inputs.length) ? ['inputs'] : [],
      properties: {
        inputs: { required: mode.inputs, maxProperties: mode.inputs.length, properties: pinned },
      },
    }],
  }
}

const firstFrameMode = modes.find((mode) => (mode.id === 'first-frame'))
const validateMode = compile(modeSchema(firstFrameMode))

const request = { ...base, inputs: firstFrame }

validateMode({ ...request, resolution: '768p' })           // true
validateMode({ ...request, width: 1344, height: 768 })     // false
validateMode({ ...base, width: 1344, height: 768 })        // false

The first two calls settle the size parameters of first-frame: it takes resolution and rejects width and height. The third is a valid text request, and the mode schema rejects it because the frame is missing. Run the same test for each changing parameter against each mode and you have the form of every mode, with no rule read by hand.

A working request per mode

The mode schema also picks the published example that fits the mode, which gives each item a request that already ran.

const { examples } = await (await fetch(document.info['x-examples'])).json()

const starting = examples.find((example) => (
  validateMode({ ...example.request, taskUUID: crypto.randomUUID() })
))

// starting.request: { taskType, model, positivePrompt, resolution, duration, inputs }

A mode with no published example starts from the closest one, with its inputs swapped for the inputs of the mode.