Firefly can't count

[holtwebbart]

Why can't Firefly count? It's been over a year with hundreds of complaints about how Firefly can't count. Is counting really all that hard?
Another attempt... I typed "eight wheeled car" and EVERY image was a four wheeled car. Over and over again. Firefly couldn't grasp a car with more than four wheels. So, I tried fewer than four. I typed in "one wheeled car" and it gave me four-wheeled cars again.

I tried a different approach. I typed in "eight birds". It gave me six.

Seriously, why can't Firefly count?

[Kremsnita]

If you want to generate per example 6 ladies it generates 4 or 5 of them

[CMass]

@Kremsnita Can you share the prompt and you used and an example?

[CMass]

Here is an example of what I did: https://community.adobe.com/t5/user/viewprofilepage/user-id/6845948

But I also asked for 10 people, and it only gave me 6: 

https://firefly.adobe.com/public/t2i?id=urn:aaid:sc:US:d23246b8-5286-4455-be31-63298f5fea45

[J E L]

Interesting! It doesn't seem to know how to count, lol.

[Matthias.G.ermany]

es muss Begrifflichkeiten ergeben die als 100% Bedingung von firefly akzeptiert werden.

z.B. wenn ich 5 Glasschälchen oder transparten Gefäße wünsche
sollten dies auf allen Vorschlägen zus ehen sein.
90% der gezeigten Bilder zeigen nicht transpartene Gefäße (in einer anderen Anzahl)

[Matthias.G.ermany]

why is my sugesstion mowe to the number of objects?

my sugesstions was created because of wrong physicial attributes.
i want tranparent bowls and get majority non tranparent bowls. thats why was my sugesstion
to implement 100% actionable keywords wihich define fix rules for the KI

[holtwebbart]

Why can't Firefly count? It's been over a year with hundreds of complaints about how Firefly can't count. Is counting really all that hard for AI?
Another attempt... I typed "eight wheeled car" and EVERY image was a four wheeled car. Over and over again. Firefly couldn't grasp a car with more than four wheels. So, I tried fewer than four. I typed in "one wheeled car" and it gave me four-wheeled cars again.

I tried a different approach. I typed in "eight birds". It gave me six.

So, what's going on?

[jane-e]

@holtwebbart 

Duplicate threads posted one minute apart merged to one thread. Forum guidelines ask that you post once. Thank you!

Jane

[Monika Gause]

Here's how AI works and how it's trained. And that's why it cannot do that.

https://www.youtube.com/watch?v=24yjRbBah3w 

[holtwebbart]

I appreciate you trying, but that video is about *hands*. Do you have a
video explaining why Firefly can't *count*? The video you sent only
mentions counting by saying we're *biased* in how important counting
fingers is to us. How did that unrelated response get so many upvotes?

Regardless of the excuses for hands, my post was actually about how Firefly
can't count (as you can see in the post headline), not why Firefly has
problems with hands.

I think there is a very large difference between hand generation and
counting. What does hand generation have to do with this image of "eight"
birds (hint... there are only six birds).


--
-Holt
Artist, Photographer, Creative Soul

(PII removed)

[holtwebbart]

Okay, Jane-e... Did anyone get hurt? No. good.

One post was in the "bugs" section, one in the "community" section. Two different groups where I was hoping to get two different responses. Like telling the city there is a pothole on your street and also telling your neighbors. 

[jane-e]

@holtwebbart wrote: Do you have a video explaining why Firefly can't *count*?

Did you watch the part where he asked for five freckles?

Jane

[holtwebbart]

Of course I did, Jane. Watched the entire video (though I don't think the people upvoting your post have watched it). It says the number of freckles isn't important. Does that mean the number of wheels on a car is unimportant? Does that mean the number of birds I asked for is unimportant? What if the number IS important?

[Monika Gause]

How did that unrelated response get so many upvotes?

By @holtwebbart

I do not know. Probably because it is not unrelated.

The exact number of things played no role in the "learning" of the AI (generally obviously, it's not only Firefly). It does not count things. It has no relation to counting. 

Most of the time when you photograph a car you do not even see 4 wheels. You see 2 wheels, sometimes 3. The AI doesn't even know that.

If you want more wheels, use the background replacement, mark the place where you want a wheel and have it generate the wheel.

[holtwebbart]

I'm sorry if this response pops up twice. My page crashed right after posting and I don't see my latest reply...

Monika Gause, are you saying that quantity isn't important?
So, if I (hypothetically) have a client who wants an image with eight birds and Firefly will only give me six birds, that should be good enough and I can add the rest of the birds myself?

[holtwebbart]

This is an image of "six monkeys winning money at casino". 

[Monika Gause]

Monika Gause, are you saying that quantity isn't important?

By @holtwebbart

I think you need to re-read my post.

Counting was obviously not a thing in the training of the generative AI. There is no documentation about what those gen AI models do on the inside and how they work. But counting obviously is something they did not learn. None of them. It's not how they see the world or analyze things.

[jane-e]

Why can't Firefly count?

By @holtwebbart

I expect a rebuttal, but I found this for you anyway. It doesn't mention Firefly — only three other Generative AI tools that can't count.

https://theconversation.com/if-ai-image-generators-are-so-smart-why-do-they-struggle-to-write-and-co...

"Generative AI tools such as Midjourney, Stable Diffusion and DALL-E 2 have astounded us with their ability to produce remarkable images in a matter of seconds. Despite their achievements, however, there remains a puzzling disparity between what AI image generators can produce and what we can. For instance, these tools often won’t deliver satisfactory results for seemingly simple tasks such as counting objects and producing accurate text."

(italics mine)

If you are not happy with Generative AI, you can always use traditional methods in Photoshop and Illustrator to composite your image. Or you can wait for it to get better. It's learning all the time.

Jane

[holtwebbart]

That article might give me some insight. I'll take a look. Thank you. 
All I really want is to understand why AI can't count. It's frustrating when it can create incredibly realistic and artistic images of landscapes and animals and people doing so much, but can't give me the correct number of items.
If I understand "why" AI can't count, I might not get so frustrated when creating things that require counting. 

[holtwebbart]

I read it and re-read it. That's why I asked. Clarification. 🙂
It sounds like you're saying the developers of AI don't consider counting to be important. Why is that? In addition to the many other ways we see the world, we humans see the world and analyze it through numbers and we consider that incredibly important. Would you be able to help me understand why counting isn't important in AI development? (Genuine question. Trying to learn as much as I can so I understand it all better.) Thanks.

[holtwebbart]

Well, creating from scratch when AI doesn't work takes us outside the scope of the AI discussion. That's just doing things "old school". 😄
Good article. Thanks for digging that out. It talks about how AI can’t make out letters or numbers because it’s trained on image data only. Would it make sense to incorporate OCR (optical character recognition) into the AI platforms? Technology has had the ability to recognize text and numbers for decades.

AI can accurately reproduce a life-like animal in a variety of realistic scenes but has no idea how many animals there are in that scene. I wonder why the two can’t work together. AI should be able to use OCR software.

I understand that it doesn’t work that way, but I don’t understand why. Numbers and letters are so integral to human development that I’m baffled why the creators never addressed it.

[jane-e]

It sounds like you're saying the developers of AI don't consider counting to be important.

By @holtwebbart

I couldn't find where either of us used that word, but I found 10 places where you used it. As for me, I don't think it. All I know is counting is not there (yet).

• Both of us are volunteers who don't work for Adobe. 
• We are not developers.
• Counting is not working in any generative AI. It's not just Firefly.
• The hope is that it will work someday and that it's being worked on.
• Developers rarely share want they are working on. They wait until they have something that works.

I hope this conversation has helped. If not, I hope someone else will step in. I can't think of anything else to add without repeating myself.

Jane

[Maciej MaciekPaK]

It is very easy answer to why AI that create images can not count.

That was aparently not in the trained set of data. When people describe the images in text, it is very rare that they count number of items. We usually use other quantitative words like a lot, a few, a croud, etc. Therefore without a special training for that particualr use case the AI will never be able to count or understand that less precise words for quantity.

- Independent software developer and artist.

[holtwebbart]

I appreciate everyone answering my question about AI not being able to count. And I appreciate the explanations of how developers didn't include counting in AI training. It helps me understand that AI wasn't programmed to include counting. And, I suppose if you understand how/why AI works, you understand why counting wasn't included in the training.

For what it's worth, I have been trying to clarify your statements to make sure I understand you. That's why I used phrases like, "it sounds like you're saying". So, when we say counting wasn't in the training, it sounds to me like the trainers didn't consider it "of significance or consequence" to include it at that time. Or... maybe it was too difficult to include. Or maybe too expensive. Either way, the end result is the same. Counting wasn't included.

And I thank everyone for helping explain that.

However... and you knew this was coming... I still don't understand "why" it wasn't included. And if you don't know "why" either, that's cool. We're in the same boat.

Counting is of base-level importance to human development, so my human brain naturally assumes it would be important to an artificial intelligence designed to help humans (assuming that's what AI is here to do). That's where I am right now. I don't get it. I get that it wasn't included in the training, but I don't get why it wasn't included.

And that may not be a question anyone here can answer, since we're not AI developers.

I'm cool with that. Maybe one day an AI developer will help shed some light. 🙂

[Maciej MaciekPaK]

If you prefer in simple terms. That would be far too expensie and require yet years to gather data and completly new model to include such simple for humans and yet difficult for AI model thing to do. I would estimate a milions of budget and at least 3 years of reserach.