Kling AI 1.6 Image to Video Body Movement Testing
I’ve recently started testing AI image to video tools as a hobbyist to have fun. I find it fascinating how these AI models are able to take still images and give them life using text prompts. For my latest test I took this photo taken by a student of me during a triple jump track and field event in high school. If you’re not familiar with triple jump, it’s a tricky event where a person takes three successive jumps to get the most distance possible. There are several different styles that can be used for performing this event. Below is a video so you can get an idea of what the body mechanics look like.
The above video shows the world record holder Jonathan Edwards who jumped 18.29 meters back in 1995. You can see the style showing the first jump followed by a second jump using the same leg and then a third jump using the opposite leg. Needless to say I was new to this event in high school and not very good at it but truly enjoyed learning and getting better over the course of the season.
A student was able to capture this photo of me during the third jump during this event.
I thought it would be fun to try and use an image to video model to see if I could write a prompt to create a video to continue the body motion of this third jump. I’ve primarily been testing Kling and Hailuoa video models. Until recently I consistently was having better luck generating videos with Hailuoai IV2-01 over Kling 1.5 models. However I recently got access to Kling 1.6 and the quality improvement has been dramatic.
A few things to keep in mind as I go through what I tested here. I only used 2 models for this test and there are many more out there to try including Sora, Veo 2, Runway, Luma, Pika and many more. Also, each model has their own strenghts in certain areas and they keep improving quickly. So you need to test all models depending on your use case to see which one performs best for you.
Another thing to keep in mind is that you can generate a video using a prompt and then generate it again with the exact same prompt and it will generate something different. So again, there is quite a bit of trial and error necessary if you are trying to achieve something very specific as I was trying to do here.
Below are the videos generated by Kling 1.5 and Hailuo IV2-01. I felt that Kling did a terrible job and although the Hailuoa did a little better with more realistic movements, it didn’t adhere to the prompt well.
When I heard about the Kling 1.6 release and learning that it adheres to prompts better I had to run the same test again and was impressed at how much better this model performed. I realized that my prompt wasn’t detailed enough and kept iterating to try and describe the body movements better with hopes of achieving what I wanted. It didn’t work, so I decided to use this Chatgpt Kling Prompt Creator to see if it improved. I wasn’t able to get an accurate depiction of the third jump similar to what you see in the in the video of John Edwards I shared. I think this test demonstrates the difficulty in prompting body movements for a very specific activity. And in this case these are pretty uncommon movements which probably make it harder to describe and generate. In any case it was fun watching the different clips it generated.
If you compare the background animation of the people and objects between 1.5 and 1.6 you can also clearly see what a better job it does without natural movements without artifacting as much. It’s also interesting that it always created a slow motion video regardless of whether I included that in the prompt. I will note that the level of consistency and challenges with prompt generation will take time before the models will infer what we want and fill in the gaps to provide the results we want.
Here’s a thorough test I came across from Heather Cooper using the same image and prompt for all the top models.
Please share any of your experiences and learnings testing video models in the comments. And if you enjoyed this post be sure to check out my AI Resource Guide for News, Tools, Tutorials and more.