Video Glitch: When You Show Reference Videos to AI, the Timing Actually Gets Off

When I show the AI a reference video, the lip-synced video it produces looks more like the original, but the timing is off. "Syncing to the beat" refers to changing the visuals in time with the music. It all started with a set of prompts shared by Gorden Sun: let the AI take the lead, make the visuals look like they’re from editing software, shoot it in one take, and create your own electronic music track to sync with the beat. How was it done? Who did it? The lead AI watched the original video and created a Mid-Autumn Festival version. Find 3 more brand-new AI models and have each create a version: one without watching the original video at all, one without watching the original video but with a Halloween twist, and one using the original video as a reference. I chose the direction: “Let’s just go with Halloween,” but then I added, “Mid-Autumn Festival would work better—we might still need to use the video as a reference.” I request the following localization changes: the author's name should be written as "Joneshong," simplified Chinese characters should be changed to traditional Chinese characters, the typing interface should switch from Pinyin to Zhuyin, and Taiwanese terminology should be used. How does the image move? The image is drawn using the web's canvas (Canvas 2D). The program asks, "What frame number is it now?" and then renders that frame at 30 frames per second. The music was also synthesized from scratch using a program, at 120 beats per minute. A “timing sheet” synchronizes both the visuals and the music, so the timing was synchronized from the very beginning—not adjusted afterward. I opened six browser tabs at the same time to work on the illustration simultaneously. It was a 58-second clip with 1,755 frames, and I finished it in five minutes. The same prompt was run four times Version How well you keep time Mid-Autumn Festival: I've seen the original film 8.9 times Mid-Autumn Festival—I didn't watch the original movie 6.6 times Halloween—I haven't seen the original movie 12 times On Halloween, I watched the original movie. 6 times "Multiplier" refers to the visual changes in time—specifically, how many times more changes occur between two beats. The higher the number, the more precisely the visuals follow the beat. Create only one version of each type. Those two Halloween videos are the best for comparison: I was given the original footage—the robot looks different (full screen, brightly colored, wearing headphones)—but the timing is only half as accurate. How Far Has AI Come? It can be done Creating an entire video from scratch: handling the visuals, music, and shot list all on my own Two AIs that didn't know each other chose exactly the same approach: 120 beats and the same beat chart. I built my own sync-checking tool and intentionally offset the audio by 0.2 seconds; the tool was able to detect it. I still need some help The default settings use simplified Chinese characters, Pinyin, and mainland terminology; they only change them when I point it out. Here's a reference video; it learns what people look like, not when to move. Original prompt:Tweet by Gorden Sun September 27, 2026

Version	How well you keep time
Mid-Autumn Festival: I've seen the original film	8.9 times
Mid-Autumn Festival—I didn't watch the original movie	6.6 times
Halloween—I haven't seen the original movie	12 times
On Halloween, I watched the original movie.	6 times