Kling VIDEO 4.0

Introduction

Kling VIDEO 4.0 generates speech with lip sync in Chinese, English, Japanese, Korean, Spanish, Portuguese, German, French and Hindi, and it renders readable text inside the frame. Both come from the prompt. A spoken line is written in quotation marks, and a sign or a subtitle is spelled out exactly as it should appear.

Request shape

There is no separate speech parameter. The line goes inside positivePrompt, next to the description of the shot:

import { createClient } from '@runware/sdk'

const client = await createClient({ apiKey: process.env.RUNWARE_API_KEY })
await client.connect()

const [result] = await client.run({
  model: 'klingai:kling-video@4.0',
  positivePrompt: 'A vertical selfie-style video of the founder of a small skincare brand standing in her bright packing studio. She looks into the lens and says in warm, natural American English, at a relaxed pace: "We make every batch in small runs, so it\'s always fresh when it reaches you." She smiles at the end. Audio: her voice clear and close, quiet room tone, no music, no subtitles.',
  width: 1080,
  height: 1920,
  duration: 8
})

import asyncio
import os

from runware import Runware


async def main():
    async with Runware(api_key=os.environ["RUNWARE_API_KEY"]) as client:
        results = await client.run({
            "model": "klingai:kling-video@4.0",
            "positivePrompt": "A vertical selfie-style video of the founder of a small skincare brand standing in her bright packing studio. She looks into the lens and says in warm, natural American English, at a relaxed pace: \"We make every batch in small runs, so it's always fresh when it reaches you.\" She smiles at the end. Audio: her voice clear and close, quiet room tone, no music, no subtitles.",
            "width": 1080,
            "height": 1920,
            "duration": 8
        })


asyncio.run(main())

curl https://api.runware.ai/v1 \
  -H "Authorization: Bearer $RUNWARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[
    {
      "taskType": "videoInference",
      "taskUUID": "2b3c4d5e-6f7a-4b8c-9d0e-1f2a3b4c5d6e",
      "model": "klingai:kling-video@4.0",
      "positivePrompt": "A vertical selfie-style video of the founder of a small skincare brand standing in her bright packing studio. She looks into the lens and says in warm, natural American English, at a relaxed pace: \"We make every batch in small runs, so it's always fresh when it reaches you.\" She smiles at the end. Audio: her voice clear and close, quiet room tone, no music, no subtitles.",
      "width": 1080,
      "height": 1920,
      "duration": 8
    }
  ]'

runware run klingai:kling-video@4.0 \
  positivePrompt="A vertical selfie-style video of the founder of a small skincare brand standing in her bright packing studio. She looks into the lens and says in warm, natural American English, at a relaxed pace: \"We make every batch in small runs, so it's always fresh when it reaches you.\" She smiles at the end. Audio: her voice clear and close, quiet room tone, no music, no subtitles." \
  width=1080 \
  height=1920 \
  duration=8

{
  "taskType": "videoInference",
  "taskUUID": "2b3c4d5e-6f7a-4b8c-9d0e-1f2a3b4c5d6e",
  "model": "klingai:kling-video@4.0",
  "positivePrompt": "A vertical selfie-style video of the founder of a small skincare brand standing in her bright packing studio. She looks into the lens and says in warm, natural American English, at a relaxed pace: \"We make every batch in small runs, so it's always fresh when it reaches you.\" She smiles at the end. Audio: her voice clear and close, quiet room tone, no music, no subtitles.",
  "width": 1080,
  "height": 1920,
  "duration": 8
}

[
  {
    "taskType": "videoInference",
    "taskUUID": "2b3c4d5e-6f7a-4b8c-9d0e-1f2a3b4c5d6e",
    "videoUUID": "7e8f9a0b-1c2d-4e3f-8a4b-5c6d7e8f9a0b",
    "videoURL": "https://vm.runware.ai/video/os/a14d18/ws/2/vi/7e8f9a0b-1c2d-4e3f-8a4b-5c6d7e8f9a0b.mp4"
  }
]

The quotation marks inside the prompt are escaped in JSON. Whatever sits between them is what the character says, word for word.

Writing a spoken line

A line works best when the prompt answers three questions: who says it, the exact words, and how it sounds. The speaker can be named by role or position ("the founder", "the woman on the left"). The words go in quotes. The delivery is where most of the character comes from: age, tone, pace and emotion, as in "warm, natural American English, at a relaxed pace".

Match the line to the clip. An eight-second clip comfortably holds one or two short sentences. A speaker who has to rush a paragraph loses the natural pauses that make the delivery believable, and a line much shorter than the clip can be padded with invented words to fill the time. For longer copy, raise duration rather than packing more words into the same clip.

Languages and accents

To get speech in another language, write the line in that language and name the language in the sentence that introduces it. The same greeting from a hotel front desk, in four versions:

Accents belong in the delivery note. English takes American, British and Indian accents, and Chinese takes regional accents and dialects such as Cantonese. Name the one you want in the same place you name the language.

Each take above is a separate generation from text, so the receptionist herself changes between them. To keep one presenter across every language, pass her in as a reference image or save her as an element, and change only the line.

Two or more speakers

Conversations need two extra pieces of information: which person says which line, and when. Assign every line to someone the viewer can identify, put each turn on the clock, and say that people take turns:

The cuts in the prompt follow the dialogue: the camera is on whoever is talking. State "only one person speaks at a time" and "each person has a distinct voice" outright, since they keep lines from overlapping and voices from blending.

Burned-in subtitles

The model can draw subtitles straight into the frame. This is the opposite of the "no subtitles" exclusion from Prompting: when you do want them, specify the position, the style, and the timing of every line, and make each subtitle match its spoken line word for word.

The timestamps do double duty: they schedule the speech and tell the model when each subtitle appears. Keep the panel inside the title-safe area and to one or two lines, the same rules a human editor would follow for a vertical platform.

Text in the scene

Signs and packaging follow the same principle as dialogue. Spell the text exactly, in quotes, and say where it sits. If the camera moves, ask for the text to stay fixed to its surface:

The store name and the price both held their spelling through the dolly. Keep in-scene text to strings that short, and add longer copy in post.

Tips

1. Quote every line exactly. The words between the quotation marks are what gets spoken.

2. Describe the voice, not just the words. Age, tone, pace and emotion turn a line into a performance.

3. Write the line in its own language. Name the language and, if it matters, the accent in the same sentence.

4. Schedule conversations. Assign each line to a speaker and a time slot, and state that only one person speaks at a time.

5. Ask for subtitles with a full spec. Position, style, and a time range per line that matches the speech.

6. Keep in-scene text short. A name and a price held their spelling through a camera move.