Multi-language VOD Dubbing Guide

This guide shows how to use the MK.IO AI dubbing pipeline to generate Spanish, German, and French audio tracks from English-language video content. The completed source asset streams with the original English track and three dubbed tracks that viewers can select.

For a full list of supported languages and more detailed information about the available transforms, visit the transcription and translation reference page.

Prerequisites

• Active MK.IO project access
• API token: available in Organisation Settings → API Tokens
• English-language MP4 video (2 to 10 minutes recommended for testing)
• Azure Storage account connected to MK.IO

Starting point: If you already have an MP4v4 asset with a .mpd manifest, skip to Step 2. If your asset is in the legacy HSS format (it has a .ismc manifest), see Convert HSS assets for VOD dubbing before continuing.

Step 1: Upload and encode source video

1.1 Create an asset

1. In the MK.IO dashboard, navigate to Assets → + Add Asset.
2. Select your storage location.
3. Enter the asset details:
   • Asset name: english-source-video
   • Container: videos
   • Storage account: select your Azure Storage account
4. Upload your MP4 file. This guide uses a file named english-video-demo.mp4.
5. Select Upload and wait for it to complete.

1.2 Create an encoding transform and job

1. Navigate to Video Processing → Transforms → + Create Transform.
2. Configure the transform:
   • Name: encode-streaming
   • Type: Encoding
   • Preset: H.264 Multiple Bitrate 1080p
3. Select Create.
4. Navigate to Video Processing → Jobs → + Create Job.
5. Configure the job:
   • Name: encode-english-source
   • Transform: encode-streaming
   • Input asset name: english-source-video. Select english-video-demo.mp4.
   • Output asset name: english-encoded
6. Select Create and monitor the job status.

7. Wait for the job status to show Finished.

Step 2: Create multi-language dubs

2.1 Create a dubbing transform

A dubbing transform defines the source language and the target languages for the AI dubbing pipeline.

{
  "properties": {
    "description": "AI dub: English to Spanish, German, French",
    "outputs": [
      {
        "preset": {
          "@odata.type": "#MediaKind.AIPipelinePreset",
          "pipeline": {
            "name": "Predefined_ACSVodSpeechToSpeech",
            "arguments": {
              "VodSpeechToSpeechTranslation": [
                {
                  "name": "language",
                  "value": "en-US"
                },
                {
                  "name": "targetLanguages",
                  "value": [
                    "es-ES",
                    "de-DE",
                    "fr-FR"
                  ]
                },
                {
                  "name": "speakerCount",
                  "value": "auto"
                },
                {
                  "name": "personalVoice",
                  "value": false
                }
              ]
            }
          }
        }
      }
    ]
  }
}

Notes on the configuration:

• Target languages: A single dubbing job generates all three language dubs simultaneously.
• Personal voice: Setting personalVoice to false uses a synthetic voice. Set to true to attempt to preserve each speaker's voice characteristics across languages.

2.2 Create a dubbing job

{
  "properties": {
    "description": "Generate Spanish, German, French dubs",
    "priority": "Normal",
    "input": {
      "files": [
        "english-video-demo_320x180_400k.mp4"
      ],
      "@odata.type": "#Microsoft.Media.JobInputAsset",
      "assetName": "english-encoded"
    },
    "outputs": [
      {
        "@odata.type": "#Microsoft.Media.JobOutputAsset",
        "assetName": "dubbed-audio"
      }
    ]
  }
}

Input file: Specify any single encoded bitrate variant from the source asset (for example, english-video-demo_320x180_400k.mp4). All variants contain the audio track required for dubbing, so you can use the lowest bitrate file.

Monitor progress: Navigate to Video Processing → Jobs and wait for the job status to show Finished.

Output files: When the job completes, the dubbed-audio asset contains three files:

• english-video-demo_320x180_400k.mp4_es-ES.mp4: Spanish
• english-video-demo_320x180_400k.mp4_de-DE.mp4: German
• english-video-demo_320x180_400k.mp4_fr-FR.mp4: French

Step 3: Insert audio tracks

This step adds the dubbed audio tracks to the encoded video asset, making all languages available to viewers.

3.1 Create track insertion transforms

Create three transforms: one per language. Each transform defines the track name, display name, and language code for the inserted audio.

Spanish insert transform:

PUT https://app.mk.io/api/v1/projects/<YOUR_PROJECT_NAME>/media/transforms/<TRANSFORM_NAME>

Path parameters

• <YOUR_PROJECT_NAME>: Your unique project identifier
• <TRANSFORM_NAME>: Transform name (for example, spanish-insert)

Request body

{
  "properties": {
    "description": "Insert Spanish audio",
    "outputs": [
      {
        "preset": {
          "tracks": [
            {
              "@odata.type": "#MediaKind.AudioTrack",
              "trackName": "audio-spanish",
              "displayName": "Español (AI Dubbed)",
              "languageCode": "es-ES"
            }
          ],
          "@odata.type": "#MediaKind.TrackInserterPreset"
        },
        "relativePriority": "Normal"
      }
    ]
  }
}

Repeat for German and French:

• German transform: set trackName to audio-german, displayName to Deutsch (AI Dubbed), and languageCode to de-DE
• French transform: set trackName to audio-french, displayName to Français (AI Dubbed), and languageCode to fr-FR

3.2 Create track insertion jobs

Create one job per language. Each job inserts the corresponding dubbed audio file into the english-encoded asset.

{
  "properties": {
    "input": {
      "files": [
        "english-video-demo_320x180_400k.mp4_es-ES.mp4"
      ],
      "@odata.type": "#Microsoft.Media.JobInputAsset",
      "assetName": "dubbed-audio"
    },
    "outputs": [
      {
        "@odata.type": "#Microsoft.Media.JobOutputAsset",
        "assetName": "english-encoded"
      }
    ],
    "priority": "Normal"
  }
}

Monitor progress: Wait for all three jobs to show Finished.

Verify: Navigate to Assets → english-encoded and open the Tracks section. You should see three audio tracks for Spanish, German, and French.

Step 4: Configure streaming

4.1 Create a streaming endpoint

1. Navigate to Streaming Endpoints → + Create Streaming Endpoint.
2. Configure the endpoint:
   • Name: production
   • Base URL: content
   • Type: Dedicated
3. Select Create, then Start.

4.2 Create a streaming locator

1. Navigate to Assets and select english-encoded.
2. Select the production endpoint created in step 4.1.
3. Add a streaming locator:
   • Name: live
   • Policy: Predefined_DownloadAndClearStreaming
4. Copy the playback URLs provided.

4.3 Test multi-language playback

1. Select the embedded player in the asset details.
2. Use the audio track selector to switch between languages.
3. Confirm that Spanish, German, and French audio tracks are selectable alongside the original English track.