ControlNet tools

Enhance your image generation with ControlNet. Get detailed instructions on integrating and using ControlNet with Runware's different APIs.

Introduction

ControlNet offers advanced capabilities for precise image processing through the use of guide images in specific formats, known as preprocessed images. This powerful tool enhances the control and customization of image generation, enabling users to achieve desired artistic styles and detailed adjustments effectively.

Using ControlNet via our API simplifies the integration of guide images into your workflow. By leveraging the API, you can seamlessly incorporate preprocessed images and specify various parameters to tailor the image generation process to your exact requirements.

Request

Our API always accepts an array of objects as input, where each object represents a specific task to be performed. The structure of the object varies depending on the type of the task. For this section, we will focus on the parameters related to the ControlNet preprocessing task.

The following JSON snippet shows the basic structure of a request object. All properties are explained in detail in the next section.

[
  {
    "taskType": "imageControlNetPreProcess",
    "taskUUID": "3303f1be-b3dc-41a2-94df-ead00498db57",
    "inputImage": "ff1d9a0b-b80f-4665-ae07-8055b99f4aea",
    "preProcessorType": "canny",
    "height": 512,
    "width": 512
  }
]

taskType string required: The type of task to be performed. For this task, the value should be imageControlNetPreProcess.

taskUUID string required UUID v4

When a task is sent to the API you must include a random UUID v4 string using the taskUUID parameter. This string is used to match the async responses to their corresponding tasks.

If you send multiple tasks at the same time, the taskUUID will help you match the responses to the correct tasks.

The taskUUID must be unique for each task you send to the API.

outputType "base64Data" | "dataURI" | "URL" Default: URL

Specifies the output type in which the image is returned. Supported values are: dataURI, URL, and base64Data.

base64Data: The image is returned as a base64-encoded string using the guideImageBase64Data parameter in the response object.
dataURI: The image is returned as a data URI string using the guideImageDataURI parameter in the response object.
URL: The image is returned as a URL string using the guideImageURL parameter in the response object.

outputFormat "JPG" | "PNG" | "WEBP" Default: JPG: Specifies the format of the output image. Supported formats are: PNG, JPG and WEBP.

outputQuality integer Min: 20 Max: 99 Default: 95: Sets the compression quality of the output image. Higher values preserve more quality but increase file size, lower values reduce file size but decrease quality.

includeCost boolean Default: false: If set to true, the cost to perform the task will be included in the response object.

inputImage string required

Specifies the input image to be preprocessed to generate a guide image. This guide image will be used as a reference for image generation processes, guiding the AI to generate images that are more aligned with the input image. The input image can be specified in one of the following formats:

An UUID v4 string of a previously uploaded image or a generated image.
A data URI string representing the image. The data URI must be in the format data:<mediaType>;base64, followed by the base64-encoded image. For example: data:image/png;base64,iVBORw0KGgo....
A base64 encoded image without the data URI prefix. For example: iVBORw0KGgo....
A URL pointing to the image. The image must be accessible publicly.

Supported formats are: PNG, JPG and WEBP.

preProcessorType string

The preprocessor to be used.

View list of preprocessors

canny
depth
mlsd
normalbae
openpose
tile
seg
lineart
lineart_anime
shuffle
scribble
softedge

Learn more ⁨1⁩ resource

Creating consistent gaming assets with ControlNet Canny
ARTICLE

height integer

If the inputImage height dimension is larger than this value, the output image will be resized to the specified height in this parameter.

If the parameter width is not set, the output image will maintain the aspect ratio.

width integer

If the inputImage width dimension is larger than this value, the output image will be resized to the specified width in this parameter.

If the parameter height is not set, the output image will maintain the aspect ratio.

lowThresholdCanny integer Min: 0 Max: 255 Default: 100

Defines the lower threshold when using the Canny edge detection preprocessor.

The value must be less than highThresholdCanny.

Learn more ⁨1⁩ resource

Creating consistent gaming assets with ControlNet Canny
ARTICLE

highThresholdCanny integer Min: 0 Max: 255 Default: 200

Defines the high threshold when using the Canny edge detection preprocessor.

The value must be greater than lowThresholdCanny.

Learn more ⁨1⁩ resource

Creating consistent gaming assets with ControlNet Canny
ARTICLE

includeHandsAndFaceOpenPose boolean Default: false: Include the hands and face in the pose outline when using the OpenPose preprocessor.

Response

Results will be delivered in the format below.

{
  "data": [
    {
      "taskType": "imageControlNetPreProcess",
      "taskUUID": "3303f1be-b3dc-41a2-94df-ead00498db57",
      "guideImageUUID": "b6a06b3b-ce32-4884-ad93-c5eca7937ba0",
      "inputImageUUID": "ff1d9a0b-b80f-4665-ae07-8055b99f4aea",
      "guideImageURL": "https://im.runware.ai/image/ws/0.5/ii/b6a06b3b-ce32-4884-ad93-c5eca7937ba0.jpg",
      "cost": 0.0006
    }
  ]
}

taskType string: The API will return the taskType you sent in the request. In this case, it will be imageControlNetPreProcess. This helps match the responses to the correct task type.

taskUUID string UUID v4: The API will return the taskUUID you sent in the request. This way you can match the responses to the correct request tasks.

inputImageUUID string UUID v4: The unique identifier of the original image used as input for the preprocessing task.

guideImageUUID string UUID v4: The unique identifier of the preprocessed image.

guideImageURL string: If outputType is set to URL, this parameter contains the URL of the preprocessed image to be downloaded.

guideImageBase64Data string: If outputType is set to base64Data, this parameter contains the base64-encoded data of the preprocessed image.

guideImageDataURI string: If outputType is set to dataURI, this parameter contains the data URI of the preprocessed image.

cost float: if includeCost is set to true, the response will include a cost field for each task object. This field indicates the cost of the request in USD.