P-Video-2

Introduction

Native audio has the model invent the soundtrack. inputs.audio inverts that: you supply the track and the picture is generated to fit it. That matters whenever the audio is not yours to regenerate, which covers a licensed song, a recorded voiceover, a brand sonic identity, or a line a legal team has already signed off.

Three inputs, one output. The face comes from the still, the voice and the phrasing come from the song, and the prompt only says what the performance does. Nothing in that request describes a melody, because the melody was supplied. This guide covers the request, the length rule that trips people up, what the picture actually takes from music, and how imported speech compares with letting the model generate its own.

The request

inputs.audio is a single string, not an array. It takes a URL, a UUID from an earlier task, a data URI or base64.

{
  "inputs": {
    "audio": "https://am.runware.ai/audio/os/a14d18/ws/2/ai/c81f3a95-7d24-4e60-be15-2a7c9d413f80.mp3"
  }
}

It combines with a first frame, which is the shape most production work wants: your track, your approved face, and a prompt that only describes the performance.

import { createClient } from '@runware/sdk'

const client = await createClient({ apiKey: process.env.RUNWARE_API_KEY })
await client.connect()

const [result] = await client.run({
  model: 'prunaai:p-video@2',
  positivePrompt: 'She delivers the line to camera, her mouth matching the words, with small natural head movement and one open-handed gesture, then settles. The camera holds still.',
  inputs: {
    audio: 'https://am.runware.ai/audio/os/a14d18/ws/2/ai/5c2e8a71-4b93-4d06-9f18-7a3c1d5e2b84.mp3',
    frameImages: [
      {
        image: 'https://im.runware.ai/image/os/a14d18/ws/2/ii/1d7a4c92-8e35-4b60-9f27-5c8a2d1f7e93.jpg',
        frame: 'first'
      }
    ]
  },
  resolution: '720p',
  seed: 7729481,
  deliveryMethod: 'async'
})

import asyncio
import os

from runware import Runware


async def main():
    async with Runware(api_key=os.environ["RUNWARE_API_KEY"]) as client:
        results = await client.run({
            "model": "prunaai:p-video@2",
            "positivePrompt": "She delivers the line to camera, her mouth matching the words, with small natural head movement and one open-handed gesture, then settles. The camera holds still.",
            "inputs": {
                "audio": "https://am.runware.ai/audio/os/a14d18/ws/2/ai/5c2e8a71-4b93-4d06-9f18-7a3c1d5e2b84.mp3",
                "frameImages": [
                    {
                        "image": "https://im.runware.ai/image/os/a14d18/ws/2/ii/1d7a4c92-8e35-4b60-9f27-5c8a2d1f7e93.jpg",
                        "frame": "first"
                    }
                ]
            },
            "resolution": "720p",
            "seed": 7729481,
            "deliveryMethod": "async"
        })


asyncio.run(main())

curl https://api.runware.ai/v1 \
  -H "Authorization: Bearer $RUNWARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[
    {
      "taskType": "videoInference",
      "taskUUID": "d47b1c85-3e92-4a06-8f51-2c7d9a5e1b30",
      "model": "prunaai:p-video@2",
      "positivePrompt": "She delivers the line to camera, her mouth matching the words, with small natural head movement and one open-handed gesture, then settles. The camera holds still.",
      "inputs": {
        "audio": "https://am.runware.ai/audio/os/a14d18/ws/2/ai/5c2e8a71-4b93-4d06-9f18-7a3c1d5e2b84.mp3",
        "frameImages": [
          {
            "image": "https://im.runware.ai/image/os/a14d18/ws/2/ii/1d7a4c92-8e35-4b60-9f27-5c8a2d1f7e93.jpg",
            "frame": "first"
          }
        ]
      },
      "resolution": "720p",
      "seed": 7729481,
      "deliveryMethod": "async"
    }
  ]'

runware run prunaai:p-video@2 \
  positivePrompt="She delivers the line to camera, her mouth matching the words, with small natural head movement and one open-handed gesture, then settles. The camera holds still." \
  inputs.audio=https://am.runware.ai/audio/os/a14d18/ws/2/ai/5c2e8a71-4b93-4d06-9f18-7a3c1d5e2b84.mp3 \
  inputs.frameImages.0.image=https://im.runware.ai/image/os/a14d18/ws/2/ii/1d7a4c92-8e35-4b60-9f27-5c8a2d1f7e93.jpg \
  inputs.frameImages.0.frame=first \
  resolution=720p \
  seed=7729481 \
  deliveryMethod=async

{
  "taskType": "videoInference",
  "taskUUID": "d47b1c85-3e92-4a06-8f51-2c7d9a5e1b30",
  "model": "prunaai:p-video@2",
  "positivePrompt": "She delivers the line to camera, her mouth matching the words, with small natural head movement and one open-handed gesture, then settles. The camera holds still.",
  "inputs": {
    "audio": "https://am.runware.ai/audio/os/a14d18/ws/2/ai/5c2e8a71-4b93-4d06-9f18-7a3c1d5e2b84.mp3",
    "frameImages": [
      {
        "image": "https://im.runware.ai/image/os/a14d18/ws/2/ii/1d7a4c92-8e35-4b60-9f27-5c8a2d1f7e93.jpg",
        "frame": "first"
      }
    ]
  },
  "resolution": "720p",
  "seed": 7729481,
  "deliveryMethod": "async"
}

[
  {
    "taskType": "videoInference",
    "taskUUID": "d47b1c85-3e92-4a06-8f51-2c7d9a5e1b30",
    "videoUUID": "3a8c5f21-9d47-4b60-8e13-2c7a9d1f5b48",
    "videoURL": "https://vm.runware.ai/video/os/a14d18/ws/2/vi/3a8c5f21-9d47-4b60-8e13-2c7a9d1f5b48.mp4"
  }
]

Notice what is missing from that prompt. There is no Audio: clause, because the soundtrack is no longer a thing the model decides. A sound description left in an audio-conditioned request is prompt length spent on an instruction that cannot be followed.

The track sets the length

This is enforced, not advisory. duration is rejected outright when inputs.audio is present. A request carrying both fails validation.

That inverts the usual workflow. The way to control a clip's length is to cut the audio before you send it, and the same track at two lengths gives two different clips with nothing else changed:

Both are cuts of one instrumental track, taken from the same offset, and it is the same track you can hear in the next section. Neither request said anything about length.

Two practical consequences follow. Trim to the beat, not to a round number, because the clip ends where the audio ends and a track cut mid-phrase gives you a video that stops mid-phrase. And trim before you experiment: iterating on a fourteen-second track costs three and a half times what iterating on a four-second cut does, so find the prompt on a short excerpt and run the full length once.

What the picture takes from the music

The model reads the track for energy and pace, not just length. Same prompt, same seed, two tracks:

One prompt, two performances. The fast track produces sharp movement that changes direction often, the slow one produces sustained movement that holds. The prompt asked for a routine and the track decided what kind, which is why "matching the energy of the music" is a clause worth writing: it tells the model the audio is a director and not just a length.

What this is not is beat detection. Do not expect a hit on a specific downbeat. The correspondence is at the level of feel over seconds, so a cut that has to land on a particular accent still belongs in your edit. Plan the audio-driven clip as the raw material and keep the frame-accurate work where you have frames.

Speaking with your own audio

A recorded or synthesised line plus a portrait gives you a spokesperson whose words you fully control, which is the case for anything with a number, a price or a legal term in it.

Both go to the same portrait, which is what keeps the presenter identical across the set:

A woman in her thirties with dark hair tied back, wearing a slate blue blazer, standing in front of a plain warm white wall with a leafy plant softly out of focus to one side, hands relaxed, looking straight into the lens with a neutral friendly expression and her mouth closed. Soft even daylight from the front left. Photoreal corporate spokesperson photography, medium shot, chest-up, no text.

Feeding both tracks to that still, with the same prompt and the same seed, gives a localized pair:

This is the pattern for a localized announcement set. One approved portrait plus one track per market gets you the same presenter across every language, with no chance of the face drifting between versions, because the face is a file rather than a generation.

Two things make it work in practice. The still should start with the mouth closed and a neutral expression, so the first frame is a plausible position to begin speaking from. And the prompt has to say she is speaking, since the model needs to know the audio is a voice to be lip-synced rather than a soundtrack to move to.

Imported audio or native speech

Both routes produce a talking clip and they are not equivalent. The difference is where the lip sync comes from.

Native speech is generated with the picture, so the model chooses the pacing and shapes the mouth around a delivery it is authoring. That is why the lip sync is at its strongest here, and why the pause-line-rest shape is so consistent.

Imported audio arrives fixed. The model has to fit a mouth to timing it did not choose, and it favors a stable, clean face over exaggerated articulation. Faces hold together well and the sync reads as natural at conversational distance, but do not expect the pronounced mouth shapes that a generated line produces.

The choice follows from what is actually constrained:

• The words must be exact, or the voice is a brand asset, or you need multiple languages. Import the audio. Nothing else guarantees the script.
• The words are yours to write and one clip is the deliverable. Generate natively. Fewer moving parts and better sync.
• A licensed or recorded track has to be in the clip. Import it, and accept that the length comes with it.
• The clip needs a music bed under a spoken line. Generate natively, since one imported file cannot be separated into a voice and a bed the model can treat differently.

Tips

1. Cut the audio to set the length. duration is rejected with inputs.audio, so the trim is your only length control, and it decides the bill.

2. Trim on a phrase, not a round number. The video stops when the audio stops, so a track cut mid-bar gives you a clip that ends mid-bar.

3. Iterate on a short excerpt. Find the prompt on four seconds, then run the full track once.

4. Drop the Audio: clause. With a track supplied there is nothing for it to direct, and it competes with the picture for prompt length.

5. Say "matching the energy of the music". It tells the model to treat the track as direction rather than as a length, and the movement changes with the tempo.

6. Do not expect beat-accurate hits. The correspondence is feel over seconds. Frame-accurate cuts belong in your edit.

7. Start portraits with the mouth closed. A first frame mid-expression is a bad position to begin a line from, and it shows in the first half second.

8. Say the subject is speaking. Without it the model may read a voice track as ambient sound and move the subject to it instead of syncing.

9. Reuse one portrait across every language. One still plus one track per market keeps the presenter identical, which no prompt-only approach can promise.

10. Section long audio on musical phrases. Past 20 seconds you are cutting anyway, so cut where the edit wants to be.