How do I downsize a photo in Lightroom 2023 without losing too much quality?

The answer is right there in the Export dialog box. You will find a file size preview at the bottom, similar to what Photoshop export dialog boxes have. But Adobe did it with that low contrast text color that is very difficult to read, so it’s easy to miss.

As you change settings, the File Size estimate changes, so that you can get it right the first time. Like your example, mine is set to JPG at 80% quality, and it reports an estimated export file size of 96 KB, well below the 5MB limit. 96 KB equals just 0.096 MB, so if you thought it was too high before, make sure you were not confusing kilobytes (1000 bytes) and megabytes (1,000,000 bytes).

5 MB is actually very generous for a JPEG file. To exceed 5 MB, a JPEG file needs to have rather large pixel dimensions, and be set to a high Quality value. A lot like Rob’s example. It would have to be something like a photo from a high megapixel camera that was not downsampled.

Because the file size of a 1024 px JPG image can be as low as the 96 KB as in my example, make sure your Long Side value meets the requirements of the competition. For example, your screen shot shows 1024 pixels but I just looked at the requirements for a competition website I have used, and it requires a minimum of 1200 pixels on the long side.

You also said you were concerned about maintaining quality. Notice that as you change the settings like pixel dimensions and Quality, the visual preview on the left changes. So you can always preview if visual quality is good enough at the current settings. Differences might be easier to see if you zoom in. That 80% Quality setting we keep repeating is important because it’s a good balance between low file size and high quality. As you raise it above 80% to 100%, the quality improvement is hard to see, but the file size increases a lot more than below 80% so if the file size estimate is too high, the first thing to do is drop Quality to around 70–80%.

And by the way…this ability to preview export file size and visual image quality is something Lightroom has that is not in Lightroom Classic!

In your Custom Resize with 1024 pixels you are bound to get a file under 1MB. (My test with an original 6000x4000px file using your exported settings creates a file 412KB)

Try varying your options-  You probably want a larger pixel dimension- even full-size, and at 80% quality the JPG should be sufficient quality. (My test with a 6000x4000px file using 'Full-size' and 80% creates a file 4.2MB).

So 'Trial & Error' is your workflow!