I've also made some explorations, such as trying to look into the data in the cross-reference stream of the last incremental update, aiming to find the root of the problem. Initially, I suspected the issue might be due to a cross-reference stream error, but unfortunately, I couldn't find any suspicious points when parsing the cross-reference stream with iText。
Thank you for your analysis of the issue, I'm really looking forward to definitively identifying what caused it. Thanks again
I had another look at your file, and I think I found the cause of the issue: In the final revision of test.pdf (the revision with the troublesome signature) all stream Length entries are incorrect, in some cases they even claim stream lengths that would make the stream exceed the end of the file.
I added an annotation with Acrobat and then fixed the stream Length entries in a hex editor; now Acrobat shows "Document has been altered or corrupted since it was signed" - yes, of course, my changes did alter the document bytes. To compare with that I applied some arbitrary but harmless changes after adding an annotation; in this case, even though the hash of the signed bytes also was changed, Acrobat continues showing "The signature byte range is invalid". Thus, fixing the stream lengths makes a difference!
Effectively those streams with the incorrect Length entries are broken, so your test.pdf document (and all similarly signed documents) will show that error in Acrobat after adding an incremental update.