P: Generated images violate user guidelines

[Hematita]

So as you can see, it's a PG-13 relatively inoffensive image of a woman in a bunny outfit. The top worked fine, and I was able to complete the top ear, which is cool. When I tried to extend the bottom with generative fill, though, I got this warning. They're just a pair of legs wearing stockings, and I wanted to extend it.

It feels like a false flag - though I could be wrong? I find myself thinking it would do the same for women in swimsuits.

Figured I'd share here.

[zvi_t]

Prompt "3+1=4 written in the sand" violated user guidelines. I'm so sorry Adobe, I didn't see anywhere in the guidelines that we can't generate letters in sand.

[Gaspar30]

I can not remember a time in the last 5 years that I was more frustarted than I have been in the last few weeks trying to use the Generative Fill tool in Photoshop. Every second thing I try gives the "violates terms and conditions" message when I can easily do the exact same prompts and keywords in other platforms (for example MidJourney). Here are a couple of examples:

1. I have a young family friend who loves all things military and army related so I was doing up a cool poster of him dressed in military uniform with a cool epic battle scene behind him (rubble, military tanks, explosions, etc.) In the end I gave up and just put him against a bunch of trees in the background because I was ready to pull my hair out with (cursing removed) Photoshop!!! Every single word I tried was banned (tank,military, historical military vehicles, war scene, etc.)

2. I was making a funny photo, sort of like a meme, with 2 groups of teachers fighting in a classroom (like a battle scene) and every (cursing removed) prompt I tried failed, even things like (destruction, hole in wall, fire, explosion, damaged building, etc.)

Adobe, for the love!!! get your stupid act together and figure out how to moderate prompts properly!!! AARGGGHHH! When MidJourney thinks you've prompted something inapporpriate it actually lets you send the prompt to 2nd tier moderation bot and further checks it for problems. 95% of the time the prompt then gets through with no further problems.

SOOOO FRUSTRATED! 😞

[Arrrggghhh]

I share your frustration. I'm a Product Designer and a big part of my work is Halloween costumes. I have to modify or add details based on what the client wants. Sometimes, when I work on any areas that show skin, like a wrist, a neck, or legs, I get the “violating guidelines” message. It’s so annoying that I have to use other AI sources to achieve the design I want.

Note: I noticed that using the word “model” in the Generative Fill can help avoid those “violation” alerts. For example: “Add a leather bracelet around the model’s wrist” usually works. More precise prompt descriptions tend to make Photoshop loosen its grip on censoring a little. 

[Encartauk]

I've not used any other AI art 'generators' apart from trying a few free web-based ones and Bing AI chat. I have seen results in other programs like Stable Diffusion and questioned the quality of PS results myself - but I was thinking, do any of those other programs allow users to select areas of images and generate areas based off text prompts and/or just AI's assessment if left blank?

I assumed that in the others that if the user wants to change things they have to do it all by text prompts without selecting certain areas that they want modifying first e.g. 'move right arm above head, change dress to red, replace shoes with sneakers' etc.

Also, do any of them allow to work in layers where you can mix things up and then chop and change easily during the process? And/or incorporate your own images, real or AI?

So, I'm not stanning for Adobe here, but coming from a place myself where I've realised that Adobe and PS may have a unique tool here - but I could be wrong, which is why I'm asking.

[Graham24508943nobd]

I've tried a few including mid journey and stable diffusion. Mid journey humans are far better, but, IMO adobe AI generated one have a far more natural look. Even stable diffusion have some terribly distorted  limbs & faces.

[Encartauk]

But can you select specific areas with a selection tool in those program's images and make amendments? Or do you have to use a text prompt to tell them where to make the change, and it has to correctly identify that area e.g. a left arm, or right ear, floor? 

[Jiri31574162k5gw]

I am under the impression that the generative filler feature is only for children between the ages of 4 and 12. I really don't understand this completely nonsensical restriction by Adobe. I have experienced that sometimes just typing a dot instead of a prompt will generate the fill.

[xandrwrld]

It appears that the word "symmetrical" is a prompt that violates the guidelines. I am simply trying to put in a set of steps into this image, but that keyword seems to flag it every time. This has been happening a lot recently with benign, simple things that do not go against anything in the guidelines. I think it's time for a review of the PSGF's censorship algorithm.   

[Graham24508943nobd]

I don't find it slow, but it's far worse at removing objects and adding objects that just aren't anything if you know what I mean ?  I asked for a patio and it added some gawdawful monstrosity that would NOT move even after deleting the layer. I had to scrub the project I was working on 

[Graham24508943nobd]

Red, try using swimming costume instead of swimwear. Works for me 

[Encartauk]

I've been trying to add a fringe (what we in the UK call them - in the US it's 'bangs' I think) to a woman's hair. I've tried in the web version of text to image, then tried in the web version of generative fill, and now I'm using PS Beta 25.0.

If I select the forehead and ask for 'fringe' it gives me all sorts of weird ornaments, straps, bows and indecipherable things, but if I ask for 'bangs' it gets blocked as a violation! Not my fault the US decided to call a part of a hairstyle 'bangs' (no logic to that name that I can see whatsoever). Bangs is either violent as a gun or bomb 'bangs' or is sexual in Adobe's mind it would seem, and not possibly used in any other innocent context.

I've tried 'hair fringe', 'hair bangs', 'straight bangs' and more - only the words using bangs has something resembling hair and not objects, but they are usually parted and don't cover the forehead.

This is going to be confusing and unintuitive as heck if words don't give an obvious result (I'd say it should be obvious if the AI is smart enough to recognise that the area I've selected is on someone's head at least, and surrounded by hair) or the words we need to use are getting blocked because they can also be used in an 'inappropriate' context. 

Solution: don't block so many words. 

Better solution: don't block anything and let people create what they want and then they can later suffer the potential consequences with wherever they use the image next - just as they would with painting, drawing, sclupting, photography or any other form of art that springs to mind.

[simplicity05]

Attempted to use the generative expand function to increase the size of an image of billowing smoke so I could have it match the size of another file I wish to use it in. Using the crop tool to expand the background size and hitting "generate" does not give satisfactory results. Then I used the generative prompt field in the properties tab that pops up, and the word "smoke" triggers a TOS violation message. I suppose this is due to an association with the violence and/or illegal activities or goods portions of the policies, yet there's plenty of reasonable cases where an artist would want to use smoke in a project that would not violate TOS. Examples that come to mind off the cuff would be anything relating to cars or auto sports, use for a dramatic movie-style poster, or artwork involving fire fighters and other first responders. Could also be used to provide a base texture to be significantly modified through filters and other artistic expressions.

1. Version of the app: Photoshop Beta 25.0.0
2. OS: Windows 11 Home, Version: 22H2, OS build: 22621.2134, Experience: Windows Feature Experience Pack 1000.22659.1000.0
3. Basic steps to reproduce the problem: use generative expand prompt "smoke" as shown in the attached image
4. Expected result and actual result: allow the feature to function with the prompt provided

[Graham24508943nobd]

Do a google image search for smoke with transparent background, simply save the pic, copy & paste into your image.

[zvi_t]

Baby fist. Baby fist hand. Baby's hand like a fist.  VIOLATION for any way I use the word "fist."

1. I do not want a fist punching anyone!

2. I do not want a fist killing anyone!

3. I do not want a fist committing a crime!

Al I want is a fist of a baby!!!! How long will it be until Adobe fixes this ridiculous issue?

 

[Kevin Stohlmeyer]

@zvi_t first, thank you for being a tester in this beta. Second, I've found it helpful to Google synonyms for violating terms. Instead of "fist" why not try "closed hand"?

[zvi_t]

@Kevin Stohlmeyer  Thanks for the suggestion, and sorry for the outburst. Yes, I'm struggling with "closed hand" which gives me all kinds of open hands, which I then have to fix part by part to close them.

Closed fingers is also something that's better, but it would be much easier to just being able to write fist.

[Kevin Stohlmeyer]

As with any Beta, it will continue to improve with user feedback and constructive input. Thanks!

[zvi_t]

This post titled "Generated Images Violate User Guidelines" was published on May 23, 2023. It's been three months and 43 pages of discussion, yet there have been no changes regarding these incorrect warnings. People can only be patient with the "beta" excuse for so long.

[Kevin Stohlmeyer]

@zvi_t but it is a beta, meaning this is all on a voluntary basis and by definition is a work in progress. No one is forcing anyone to participate. Honestly three months to develop something as complex and nuanced as an AI interface in such a robust imaging software package is short-term IMO. Expecting Adobe to be able to roll out something that is this complicated in such a short time is rediculous.

[daniellei4510]

The neural filter "Depth Blur" has been in beta since, if I recall, 2021. Live Gausian fill was once in beta, then introduced in Adobe Photoshop proper, and has since been removed even from the beta version of Photoshop. Apple's operating systems, beginning with Developers and progressing to public betas last nearly a year before being introduced to the general public. It's a long and painstaking process and 3 or 4 months is a very short time in comparison.

[zvi_t]

I'm not being naive, guys. As a software engineer myself, I don't expect GF to go from beta to stable in three months. GF, which uses FireFly, was not developed within three months. There was FireFly out there before PS incorporated it as GF beta.

I won't debate this, I'm entitled to my opinion. If GF started generating inappropriate images, Adobe would have the means to fix that sooner than you can say "generative fill." It's not that they don't have the means to fix the daily reported false violation alerts, it's just that it isn't their priority. Beta neural filters are in beta because Adobe is just too busy with other things to have time to fix them.

Also, if they were smart, they'd give users the option to generate just one generation instead of three. This would reduce their server usage significantly (which is very expensive), and at the same time, reduce the generation time for the user by a third.

[rayek.elfin]

@Encartauk 

Better solution: don't block anything and let people create what they want and then they can later suffer the potential consequences with wherever they use the image next - just as they would with painting, drawing, sclupting, photography or any other form of art that springs to mind.

Adobe cannot allow their users to generate whichever image they want, because (as far as I can see) that would open the floodgates to potential legal consequences. Just imagine if it is proven in court that their generative AI tools were used in a fraud case. Potentially (in the US especially so) this could lead to Adobe being hauled into court for enabling and assisting. There is a reason why Adobe Photoshop refuses to open and save images of dollar bills. It's merely a giant company protecting itself.

I sincerely doubt this will ever change. If a user wants to generate anything with a generative AI, the only option is to opt for other alternatives outside the Adobe ecosystem of software. With the added benefits of qualitative better results and much more options. And running generative AI tools on your own local machine opens the path to full freedom in regard to whatever images you'd like to generate. Although this does require more expensive hardware and a bit more knowledge to install and setup.

It is what it is. Other options exist if Adobe's generative AI turns out to be too restrictive for your work.

[Encartauk]

If someone uses a brand of car to mow people down, does the car company get called into court for enabling and assisting? If someone paints an inappropriate picture of a public figure on a canvas with oil paints, do those companies that make those tools get called into court for enabling and assisting? If someone uses a brand of laptop running a certain OS and hacks into a bank stealing billions, do those software companies get the same? 

People have had the means (including using Photoshop) to create fake images that could've been used in fraud/defamation for decades - has Adobe had to go to court previously for any of those reasons?

As I said, once the image is created, it's then down to whatever happens after that and what laws it breaks in the environment it will exist within. If it's for someone's personal use, not a problem unless it still breaks laws that would already apply e.g. ch!ld pron. If it was to be violent, and was on public display then it could fall into public decency violations or whatever. If it was used to defame someone, (and I'm not saying it's OK to do so, but how would an AI filter this out correctly) then the person that created it would fall foul of the law, why would it be anything to do with Adobe? Again, Adobe has already been providing means to manipulate images for decades, why is this any different?

[Jason Toemmes]

@Encartauk

Excellent points! Fully agree with your rebuttal.

[rayek.elfin]

@Encartauk  If all of that is indeed the case as you say... Ask yourself why Adobe added a piece of code to avoid counterfeiting money in Photoshop. 

https://helpx.adobe.com/photoshop/cds.html

The difference is that full-blown limitless generative AI is akin to providing your users with an endless library of images that include a wide range of potential offending ones. In classic image editing and compositing users import existing images or paint/draw their images. Adobe is not reponsible for what their users import or create with their own hands. With generative AI a model built by Adobe gains  access to an built-in facility to generate a sheer limitless library of images within Photoshop itself/via Adobe's servers. The context is completely different compared to users importing external images or drawing their own pictures. Users do not generate these AI images out of the void: the AI models are based on the library of images Adobe fed it.

Ask yourself: does Adobe's stock image library include offending imagery? No, it does not. So why would they allow their generative AI to act differently?

Also, with generative AI in the news and a "hot item" I doubt if Adobe would want their software mentioned on the evening news in the same sentence with various forms of celebrity nude fakes, fraude, crime, and abuse because, whether we like it or not, generative AI lowers the threshold to creating believable faked imagery to a level where even novices can pull these off relatively easily. It is a potential marketing nightmare waiting to happen.

Btw, I am all for freedom of information accessible to all. But it has been proven again and again that a majority of humans living on Earth cannot be trusted with total freedom (yet). Which is unfortunate, and until humanity has proven itself to be a trustworthy lot and be kind to each other, I'd say that it is probably wise that Adobe limits their generative AI. So I understand and empathize with their point of view, even if I disagree with Adobe on the principle of full freedom of information and agree in principle with you @Encartauk. But with the current state of humanity Adobe is sort-of forced to play "nanny" in this particular case. Nothing PG rated or above 😉

Besides, (this is an important point) allowing Photoshop users to generate whichever image they want would also be extremely problematic in regard to their young user base: Adobe cannot be make itself responsible for and allow kids to use Photoshop for generating lewd imagery. Parents, schools, and various religious institutes would not be amused!

Then Photoshop would become an adult-only software? That's just not viable. Adobe would have to release different versions for different age groups? How is that going to work? Total nightmare on so many levels to deal with. Have you thought about young users and schools at all and how they add up in this equation? I am sure you will agree with me that generative AI should be limited when kids use it to generate images. Or within a school environment.

Nope, yet another reason to integrate a kid-friendly generative AI and avoid all that potential trouble for Adobe. Definitely do not want to open that can of worms.

Just playing the devil's advocate here: Adobe'd be stark mad to allow for full artistic generative AI freedom at this point in time (or ever!).

Besides --I repeat once more-- the tools for ultimate freedom in generative AI are out there if you need them.