How do I make a field optional for prefill, but mandatory (if not filled) for the Participant?

You can't do this with one field. A field can only have one role assigned, either you assign the prefill role or you assign the field to a participant/signer, or the anyone role.

you could do it using 2 fields.

one field (optional( assigned a prefill role

then a second field (required) for the signer, but set to show if there's nothing in the prefill field.