Transfer the voice, expressions and head motion of a talking video to an avatar (Create from selfie)

Start from the track editor with an avatar clip (the bar that holds the picture of the presenter) for the avatar that should perform, in the timeline in the lower half of the screen. No speech has been entered yet.

1. 1Check that there is an avatar clip (Ryuto in this example).
2. 2Click Selfie in the toolbar, the row of buttons just below the preview.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check Check that there is an avatar clip (Ryuto in this example).
2. 2
   Action Click Selfie in the toolbar, the row of buttons just below the preview.

Check The Create from selfie dialog opens. Four stages are shown at the top: Input, Import, Choose and Place.

In the first stage you specify the source video. In this example we choose a video file prepared in advance.

1. 1Check that Video file is selected.
2. 2Click Choose file… and pick the video on your computer. The dialog says you can also drag and drop it into the box.
3. 3Read the price and the conditions for a video that transfers well.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check Check that Video file is selected.
2. 2
   Action Click Choose file… and pick the video on your computer. The dialog says you can also drag and drop it into the box.
3. 3
   Check Read the price and the conditions for a video that transfers well.

Check Record with camera, Audio only and From history next to it are for recording on the spot, using audio only and reusing an earlier import. They are not demonstrated in this guide. From history cannot be pressed in a project with no earlier import.

After you choose a video, a player for checking it appears in the upper half and the file name is shown below it. This screenshot shows the part below the player.

1. 1Check the name and size of the chosen file. To use another video, remove it with the bin at the upper right of the player.
2. 2Click Import.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check Check the name and size of the chosen file. To use another video, remove it with the bin at the upper right of the player.
2. 2
   Action Click Import.

Check As the dialog says, check in the player that the face is large enough before importing.

When the import starts, the video is prepared and then the face motion is analysed.

1. 1The current process and its progress are shown.
2. 2The length of the video and the credits it costs are shown. This example reads 8 seconds, 30 credits.
3. 3The note says that processing continues even if you close this dialog and that a failed import is not charged.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check The current process and its progress are shown.
2. 2
   Check The length of the video and the credits it costs are shown. This example reads 8 seconds, 30 credits.
3. 3
   Check The note says that processing continues even if you close this dialog and that a failed import is not charged.

Check For this 8-second example the next stage appeared after about 30 seconds. The wait depends on the length of the video and how busy the service is.

When the import finishes, you choose how the voice, the head motion, and the expressions and mouth are transferred to the avatar. Start with the voice. You can keep the original voice, but this example replaces it. Selecting Choose voice… in the Change voice row opens this list.

1. 1You can filter by voice name. In this example we typed 大地.
2. 2Click the row of the voice you want. This example uses 大地カケル.
3. 3We left the speaking style as it was when the list opened.
4. 4Click Use this voice.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Enter You can filter by voice name. In this example we typed 大地.
2. 2
   Action Click the row of the voice you want. This example uses 大地カケル.
3. 3
   Check We left the speaking style as it was when the list opened.
4. 4
   Action Click Use this voice.

Check Preview at the right end of each row lets you listen to a voice before choosing it (not demonstrated in this guide).

After choosing a voice you return to the Choose screen. Choosing alone does not replace the voice yet.

1. 1Check that the chosen voice name (大地カケル) is shown to the right of Choose voice…, then click the blue Confirm this voice button. The button states the credits and the length.
2. 2The note says listening is free and that confirming consumes credits according to the length of the voice.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Action Check that the chosen voice name (大地カケル) is shown to the right of Choose voice…, then click the blue Confirm this voice button. The button states the credits and the length.
2. 2
   Check The note says listening is free and that confirming consumes credits according to the length of the voice.

Check In the account used for this example the button read that nothing is consumed because of an unlimited plan, 8 seconds. What it shows depends on your plan. Before confirming you can also check with Listen and Try 5 seconds (free), which are not demonstrated in this guide.

The voice is replaced within a few seconds and a Changed voice row is added.

1. 1The player in the row labelled Changed voice lets you listen to the replaced voice (not played in this guide).
2. 2The message says the voice was changed to 大地カケル and that this voice will be placed on the track.
3. 3Use Back to original voice if you want the original voice again.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check The player in the row labelled Changed voice lets you listen to the replaced voice (not played in this guide).
2. 2
   Check The message says the voice was changed to 大地カケル and that this voice will be placed on the track.
3. 3
   Check Use Back to original voice if you want the original voice again.

Check In this screenshot the faint text saying the voice is being changed remains where the confirm button was, although the replacement has finished.

Further down the same screen are the settings for head motion and for expressions and mouth. This example keeps all of them as they were.

1. 1When Head motion is ticked, the head motion in the video is transferred.
2. 2Motion adjustment is Standard. The range of the motion that was read is shown below it.
3. 3When Expressions and mouth is ticked, expressions and mouth motion are transferred.
4. 4Transfer the mouth shapes of the video is selected. It transfers how the mouth opens and its shapes as they are.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check When Head motion is ticked, the head motion in the video is transferred.
2. 2
   Check Motion adjustment is Standard. The range of the motion that was read is shown below it.
3. 3
   Check When Expressions and mouth is ticked, expressions and mouth motion are transferred.
4. 4
   Check Transfer the mouth shapes of the video is selected. It transfers how the mouth opens and its shapes as they are.

Check Further down are Expression smoothing and Motion strength (100% means as read). When you have checked the settings, scroll to the bottom of the dialog and click Next to go to the next stage (Next is not shown in this screenshot). Try mouth opening is not demonstrated.

The last stage lists what is about to be placed on the timeline.

1. 1Check the destination track, the voice, the head motion, the expressions and mouth, the motion strength and the length.
2. 2Click Place on tracks.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check Check the destination track, the voice, the head motion, the expressions and mouth, the motion strength and the length.
2. 2
   Action Click Place on tracks.

Check The note at the bottom says the avatar clip then needs generation and that Generate pending applies it to the video. In this demonstration the avatar clip stayed at 5 seconds and was not extended automatically. We therefore played the project once so that the playhead stopped at the end of the voice (8.40 seconds), right-clicked the avatar clip and chose the item that extends its end to the playhead, which lengthened it to 8.40 seconds. The background image was extended in the same way (there is no screenshot of this operation). The next screenshot was taken afterwards.

After placing, three clips appear below the avatar track: voice, head motion and expression. The screenshot was taken after saving and reopening the project.

1. 1The speech clip with the replaced voice is on the voice track. The words read from the video are shown as text.
2. 2The head motion track has a clip labelled Selfie (strength 1.0).
3. 3The expression track also has a Selfie clip (strength 100%).
4. 4The avatar clip has not been generated yet. Click Generate pending to create the video.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check The speech clip with the replaced voice is on the voice track. The words read from the video are shown as text.
2. 2
   Check The head motion track has a clip labelled Selfie (strength 1.0).
3. 3
   Check The expression track also has a Selfie clip (strength 100%).
4. 4
   Action The avatar clip has not been generated yet. Click Generate pending to create the video.

Check The words turned into text in this example were a short Japanese greeting. Because the voice is based on the audio of the video, editing this text does not change the voice (the speech clip settings say that the audio does not change, only captions and the clip label).

When generation finishes, the avatar clip shows Preview generated. Play it and check that the avatar speaks like the video. The screenshot was taken during playback.

1. 1Check that the avatar clip now shows Preview generated.
2. 2Click the Play button at the far left of the row just below the preview. It shows the Pause mark here because playback is in progress.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check Check that the avatar clip now shows Preview generated.
2. 2
   Action Click the Play button at the far left of the row just below the preview. It shows the Pause mark here because playback is in progress.

Check Head tilts and eye and mouth motion follow the delivery in the source video. Where you can play sound, also listen to whether the replaced voice matches the mouth motion.

Finally, turn it into an MP4 with Output → Export video at the top. When the export finishes, a player with the finished video opens. The screenshot was taken after it played to the end.

1. 1The time reads 0:08 / 0:08, which tells you the video played to the end.
2. 2Click Close when you have finished checking.

↘Red arrows and white labels are guide annotations ＋ Enlarge with callouts

1. 1
   Check The time reads 0:08 / 0:08, which tells you the video played to the end.
2. 2
   Action Click Close when you have finished checking.

Check Export settings and downloading are covered in the guide on exporting an MP4. The finished video contains only the avatar, the background and the replaced voice, not the picture of the source video.