HeyGen Voice 1.0

Introduction

HeyGen Voice 1.0 speaks 39 languages, and any voice can speak any of them: one of the public voices or one cloned from your own recording. The language comes from speech.language when you set it, and is detected from the text when you don't.

The fitness coach below recorded an introduction in English. The clip is a Spanish workout cue in her cloned voice, from one request.

A fitness app can record its coach once, in one language, and ship every cue in every market it sells in.

Detected or declared

The request below is the one behind the clip above. The recording is in English, the text is in Spanish, and speech.language says which language to speak.

import { createClient } from '@runware/sdk'

const client = await createClient({ apiKey: process.env.RUNWARE_API_KEY })
await client.connect()

const [result] = await client.run({
  model: 'heygen:voice@1.0',
  inputs: {
    audio: 'https://example.com/recordings/coach-intro.mp3'
  },
  speech: {
    text: 'Muy bien, última serie. Baja despacio durante tres segundos, empuja fuerte hacia arriba y mantén el abdomen firme. Diez repeticiones más y terminamos por hoy.',
    language: 'es'
  }
})

import asyncio
import os

from runware import Runware


async def main():
    async with Runware(api_key=os.environ["RUNWARE_API_KEY"]) as client:
        results = await client.run({
            "model": "heygen:voice@1.0",
            "inputs": {
                "audio": "https://example.com/recordings/coach-intro.mp3"
            },
            "speech": {
                "text": "Muy bien, última serie. Baja despacio durante tres segundos, empuja fuerte hacia arriba y mantén el abdomen firme. Diez repeticiones más y terminamos por hoy.",
                "language": "es"
            }
        })


asyncio.run(main())

curl https://api.runware.ai/v1 \
  -H "Authorization: Bearer $RUNWARE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[
    {
      "taskType": "audioInference",
      "taskUUID": "5e2b7d9c-1a46-4f83-b8c0-3d6e9f1a2c75",
      "model": "heygen:voice@1.0",
      "inputs": {
        "audio": "https://example.com/recordings/coach-intro.mp3"
      },
      "speech": {
        "text": "Muy bien, última serie. Baja despacio durante tres segundos, empuja fuerte hacia arriba y mantén el abdomen firme. Diez repeticiones más y terminamos por hoy.",
        "language": "es"
      }
    }
  ]'

runware run heygen:voice@1.0 \
  inputs.audio=https://example.com/recordings/coach-intro.mp3 \
  speech.text="Muy bien, última serie. Baja despacio durante tres segundos, empuja fuerte hacia arriba y mantén el abdomen firme. Diez repeticiones más y terminamos por hoy." \
  speech.language=es

{
  "taskType": "audioInference",
  "taskUUID": "5e2b7d9c-1a46-4f83-b8c0-3d6e9f1a2c75",
  "model": "heygen:voice@1.0",
  "inputs": {
    "audio": "https://example.com/recordings/coach-intro.mp3"
  },
  "speech": {
    "text": "Muy bien, última serie. Baja despacio durante tres segundos, empuja fuerte hacia arriba y mantén el abdomen firme. Diez repeticiones más y terminamos por hoy.",
    "language": "es"
  }
}

[
  {
    "taskType": "audioInference",
    "taskUUID": "5e2b7d9c-1a46-4f83-b8c0-3d6e9f1a2c75",
    "audioUUID": "b7c1e4a8-6d29-4f50-9a3e-2c8f1d7b5e96",
    "audioURL": "https://am.runware.ai/audio/os/a14d18/ws/2/ai/b7c1e4a8-6d29-4f50-9a3e-2c8f1d7b5e96.mp3",
    "outputs": {
      "voiceId": "8d4f1b7e2c9a46d3b5e0a7c2f9d1e684"
    }
  }
]

Without speech.language, the language is detected from the text. When the text alone is not conclusive, a cloned voice falls back to the language of its recording, as long as the text uses that language's script, and the last resort is the usual language of the script, English for Latin letters. Setting the code skips the guess, so send it whenever you know it.

speech.language takes base codes such as es or pt. There is no regional variant to choose, so one code covers every market that speaks the language.

One voice for every market

A storefront that sells in four countries can keep one narrator for every product video. Translate the text, set the matching code and send the same voice ID each time.

All four clips use Iona, a Bright & Energetic public voice. Only speech.text and speech.language change between them, which is what makes the brand voice the same in every storefront.

Product names can stay in their original spelling. In the Spanish and German clips, Aerolite is pronounced the way it is in English. An English name inside a translated sentence also leaves language detection alone: in our tests, a Spanish line with three English product names returned the same audio with speech.language set to es and with no code at all.

Tips

1. Send speech.language whenever you know it. Detection reads the text and picks the likeliest language. A code is used exactly as sent.

2. Translate the text and keep the voice. The same public voice ID or the same cloned outputs.voiceId in every language keeps one voice across markets.

3. Clone from a recording in any supported language. The recording and the text do not have to match, so a coach recorded in English can lead workouts in Spanish.

4. Leave product names untranslated. A brand name keeps its pronunciation inside a Spanish or German sentence, so the same spelling works in every market.