AI-Made Music Videos: The Dance Moves Are Generated While Listening to the Song, So the Moves Stay in Time with the Beat
Sep 27, 2026 · Updated Sep 28, 2026 · Seen on YouTube · 0 views
A two-minute music video: the song, the characters, and the choreography were all created by AI. Whether the dance is in time with the beat depends on whether the AI was able to hear the song while generating it.
I started by saying, “I’m not suggesting that one option replace another; I just want to go through all the possibilities first… to calculate the cost, time, and results.” After making the comparison, I asked, “Could we combine the video generated by Suno to create a two-minute music video?”
How was it done? Who did it?
- Grok, draw a full-body illustration of the character with a green background so it will be easy to cut the character out later.
- Suno created a song called “HIBANA,” using their free credit
- The dance videos are divided into 12 segments: 7 were generated using MiniMax H3 on a home graphics card, each lasting about 5 minutes; 5 were generated using a Grok subscription plan.
- Codex writes the implementation code, and Claude Code writes the specifications and performs the final acceptance testing.
- Codex reviewed it one more time from the beginning and found four actual errors.
The AI initially recommended using the paid version of H3, which costs about $9.60 for two minutes. I chose to use my home graphics card along with a Grok subscription, so I didn’t spend any extra money on the MV itself.
How does the image move?
It has two layers:
- Dance Steps: It wasn't created by a program; it's a video generated directly by H3. When it was generated, that part of the song was included as well.
- Special Effects and Editing: Python calculates frame by frame, at 24 frames per second. First, cut out the character, identify 33 points on the body, then draw lines and sparks that move with the body, and switch shots in time with the beat.
The song's tempo gradually increases from 134.5 beats per minute to 137.5 beats per minute, so the program identifies the 271 beats one by one, rather than using a fixed grid.
Three Ways to Generate Dance Moves from the Same Image
| Methods | Music for the dance | Is there a beat to the dance? |
|---|---|---|
| Grok (pay-as-you-go) | Our song—Grok didn't hear it when it was created | None |
| Grok (Subscription Plans) | Our song—Grok didn't hear it when it was created | None |
| MiniMax H3 | Our song—let’s give it a name together when it’s created | Yes (4 out of 7 sections are obvious) |
| Grok, with its built-in soundtrack | Grok created it on its own | It depends on luck; you'll have to try a few more times before finding one (1 out of 10). |
Both types of Grok come with their own background music; the only difference is that the music videos are set to our songs. For the column on built-in background music, see “Grok’s built-in background music can also be used as source material” below. Whether the music is in time with the beat is measured using statistical methods, not by visual inspection.
How Far Has AI Come?
It can be done
- Handled the entire process from specifications, creation, character editing, and adding special effects to final approval
- The Codex review identified four actual errors, such as a 10-second silence being measured as 139.5 beats per minute.
- Before the music video, we’ll first create a 10-second test clip: using the same specifications, we’ll have Codex and Claude Code each generate one, and both will pass acceptance testing.
I still need some help
- A dance generated without hearing the song can't be salvaged afterward: Claude Code and Codex each wrote a program to sync the dance moves to the beat, and while both passed their own tests, the dance still wasn't in time with the music.
- Even in "H3," only 4 out of the 7 sections in the music video clearly hit the beat.
- H3 I didn't follow the instructions to use a green background, so I had to use AI to remove the background instead.
- A metronome that measured 136 beats per minute as 74 beats per minute was later repaired.
- I noticed that "some of the track's skeletal elements weren't turned off," so I changed it so that the first two beats of each line in the chorus light up.
Grok's built-in soundtrack can also be used as source material
When Grok generates videos—whether via the CLI or the API—it includes background music by default, though each clip is only about ten seconds long (we’ve generated both 10-second and 15-second versions). If you generate multiple clips, the background music in each one is separate and doesn’t transition smoothly from one to the next.
But it's good enough to use as source material:
- Videos under 15 seconds: Just take a clip with a “green screen and built-in soundtrack,” remove the background, and you’ve got a video. Gorden Sun made one this way: Grok generated the 15-second raw footage, and Opus 5.5 handled the background removal, layout, and editing.
- Longer clips: Upload an audio clip from one of the segments into Suno to expand it into a full song, then use that song as the timeline for the entire video.
Whether the dance moves happen to sync up with the built-in soundtrack is a matter of luck. If you want a segment that’s in time with the beat, you’ll have to generate a few more and pick the right one—it’s like drawing cards. When we were making the music video, those five clips of Grok actually all came with their own background music. Later, we generated four more 15-second clips, bringing the total to ten along with an earlier test clip. When we synced each clip to its own background music, only one clearly matched the beat. Gorden Sun’s original footage, when timed, was in sync with the beat.
The demo video uses that one segment where I tap the beat: the background was removed, and a computer-generated background, text cards, and hand-drawn lines were layered on top. The text cards dance to the beat of their own soundtrack—the whole thing lasts 10 seconds.
Each segment is only 10–15 seconds long, and with Grok’s tracks, it’s often hard to tell whether the tempo is fast or slow (For the same segment, some tools measured 87 beats per minute, while others measured 175 beats per minute), so I tried two different measurement methods and three different tempos, but these figures should still be treated as a rough guide only.
How much did it cost?
When comparing the various methods, the actual cost was approximately $2.25 (as of September 2026). MV itself did not pay anything extra.
There is a very brief freeze frame at the 50-second and 80-second marks; since the final cut was already approved, no further edits were made.
Recipe from Sankakuten (Tweet,Sequel) and aicreataro| Song generated by Suno
How to Make a Short Video with Its Own Music:Tweet by Gorden Sun
September 27, 2026