Exit
  • Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
  • 한국 커뮤니티
0

Firefly Video Identical First and Last Frame Issue

Community Expert ,
Jan 22, 2025 Jan 22, 2025

I'm not sure if this is eaxctly a bug or just a deficiency in what Firefly video can do at this moment, but here is what I've experienced.


The version of Firefly Generate Video as of today (23 Jan 2025) allows you to add images to act as the first and last frame. I have been experimenting with having the same image for both the first and last frame. My intension is to create a short loop with a subject that animates with a consistent visual character. I have not had much success on this front.

Attempt 1 Steps:
1. Upload an image to first and last frame in Firefly video. Image is of a colourful animal created as a static image with Firefly Text to Image.
2. Add the prompt "Colourful bright animal talking".

Result (after multiple attempts with different images but always first and last frame having matching image)
A 5 second video with nothing happening. 
I did get an attempt with a subtle animation of the ears, another of the fur.

Expected result
The animal in the image moving its mouth with the background remaining the same.

Other attempts
I tried the same thing (same image and prompt as above) with just the first frame having a reference image.

Result
The video shows animation of the animal opening its mouth - not a talking type movement.
Plus the fur and other parts of the animal animating (waving fur, growing, changing colour).
Another attempt had the head turning and changing into a slighly different version of the animal.

Expected result
My intension was for only the mouth to animate.


Further attempts

Back with the images the same for first and last frame, I tried 'blinking' or 'winking' in the prompt instead of 'talking'. I did get some animation but a very distorted result.


From my testing, it I feel that Firefly video is lacking of controls/settings to ensure it knows when to remain consistent vs when to change things (and what to change). If you have a first frame reference image, you can't gurantee your subject will have consistency as it animates or the background will remain unchanged. If you have both first and last frames the same, Firefly video has trouble animating, particularly with the term "talking".

Bug Unresolved
TOPICS
Audio and video
517
Translate
Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines

correct answers 1 Pinned Reply

Adobe Employee , Jan 23, 2025 Jan 23, 2025

Thanks for sharing your experience with us, @DeanUtian.

 

Our Keyframes workflow is designed to have different images as the first frame and last frame, with the prompt describing how to transition between the two. When the image is the same for both, the model doesn't (yet) recognize that a change needs to happen and ignores the prompt.

I understand your goal of wanting to have a motionless character with only their mouth or a body part moving. This is the type of control we'd love to give you a

...
Translate
2 Comments
Adobe Employee ,
Jan 23, 2025 Jan 23, 2025

Thanks for sharing your experience with us, @DeanUtian.

 

Our Keyframes workflow is designed to have different images as the first frame and last frame, with the prompt describing how to transition between the two. When the image is the same for both, the model doesn't (yet) recognize that a change needs to happen and ignores the prompt.

I understand your goal of wanting to have a motionless character with only their mouth or a body part moving. This is the type of control we'd love to give you and we're currently exploring ways to achieve this.

Translate
Report
Community Expert ,
Jan 23, 2025 Jan 23, 2025
LATEST

Thanks @warren_trezevant 
This is a case where users intend to do things beyond what Adobe had originally designed the product to do.Adobe products should always allow users to push the product into new and unexpected possibilities.What I tried to do as well, which Firefly video did not allow me to do, is set the final frame but not the first one. This would be to define where you wnat the animation to end but have a new beginning.

Another part of my request (beyond simply being able to have an animated loop) is to control the visual consistency of the animation. Firefly video has changed from having a reference image to images for first and last frames. I discussed that a reference image is good but need flexibility in what the image controls in anotehr post, link below.
https://community.adobe.com/t5/adobe-firefly-ideas/firefly-video-shot-size-and-camera-angle-with-ref... 

Translate
Report