I think previous video generator used to generate the video frame by frame sequentially, so by generating a first frame, then generating the second frame based on the first, and the next one on the two previous ones, etc... This is prone to drift as the AI model will usually not pay as much attention to distant frames compared to the ones close to the frame that is being generated, which might cause drift. But for short videos, the model will be able to stay coherent even by working sequentially. One of the latest video generator, Lumiere , generates the whole video at once using a method called Diffusion. Basically Diffusion works by iteratively removing noise from the data until you get crisp data. In this case the diffusion is applied to all frames at the same time. There is some randomness since the noise you start from is completely random, but the reverse diffusion process is guided by the prompt and makes the data more and more likely, which eliminates lots of unlikely possibilities. The model effectively proposes what it thinks is a likely video given the prompt you gave, and according to the data that was used for its training. Answer from Cosmolithe on reddit.com