Images generated are irrelevant to prompt

[AkshayeeShetty]

I am experiencing an issue where the images generated are completely irrelevant to the prompt I enter. Even when I use very simple and concrete prompts (for example, “Studio Ghibli style illustration, a man offering candy to a child in a park” or “Studio Ghibli style illustration, a man in a park running fast), Firefly often produces images that do not match the description at all.

I have tried simplifying the language, reducing adjectives, and using common nouns and actions, but the results are still inconsistent and unrelated to the instructions. This makes it very difficult to use Firefly for educational/storytelling purposes, where clarity and accuracy are essential.

Could you please investigate why prompts are not being followed correctly and suggest any best practices or fixes?

[droopydog500]

Hello @AkshayeeShetty,

Thank you for your message. 

So, the term "Studio Ghibli" style will not do what you want as the model is not trained on styles related to trademarked or copyrighted entities or characters. It might ignore the terms, generate an error, or produce unpredictable results.

It appears based on my testing that those words are causing the strange results. When you remove those words, the generated image matches the prompt.  It just does not have the style you are seeking. 

The closest you will be able to come to that is by describing the style in the prompt and/or using existing styles. In general, Firefly is not a good tool to use when you are trying to imitate the style of a specific artist or studio.

Thanks,
    droopy