Exclude Certain Objects

Firefly seems to choose certain objects to appear in the images. Would it be possible to exclude certain objects? For example sometimes when a prompt is created like attorney talking with client, the images include a gavel. Could your prompt ask to exclude certain objects from appearing?

When the prompt was "attorney talking with client without gavel" every image had a gavel in it.