Issues with GREP Selecting Long Text Blocks in InDesign

[Robert at ID-Tasker]

Is there a limit on how long the found GREP result can be??

For shorter blocks of text - it will select all lines:

If I start removing some words from the first block - it will select bigger part - more paragraph(s)??

Also, what should be the GREP query to find everything in between:

REFERENCES

...

...

CHAPTER

but excluding those words - so the found result will be like on the 2nd screenshot.

Because, even something like this doesn't work??

Nor this:

<Title renamed by MOD>

[Robert at ID-Tasker]

I've copied whole text into a Notepad - to strip all formatting - and now, it can select few pages of text:

or:

So there is no limit on the length?

[Robert at ID-Tasker]

After deleting space before "http":

"." means "any character", right??

And change to ".+" from ".*" - finds even less??

[Peter Kahrel]

To match everything between those two words, do this:

(?s)REFERENCES\r\K.+(?=CHAPTER)

The default is that the dot doesn't match paragraph breaks. That can be overruled by (?s).

As for the amount of text that can be matched, there was a restriction a few versions ago of 32K characters. What is the limit in your tests?

[Robert at ID-Tasker]

Thanks @Peter Kahrel.

Unfortunately, it doesn't work all the time...

For some reason, there is one place where more is selected than should be?

And I've just fond something else - it completely skips this and few other: 

Also, there can be ":" after "REFERENCES" - of course I don't count those as skipped.

There are also Tables in between, so it looks like I'll have to implement it - finding & formatting block of text - differently in my IDT - which isn't a big deal, but I was hoping blocks can be found using GREP.

I can always do two searches:

and then process results by going through the list, selecting Paragraph after checked line and up to next visible and unchecked line - and ItemByRange().

Or make it even more universal - I'll select from-to and then unselect 1st and last Paragraphs of the selection.

[Peter Kahrel]

Sorry, that should be

(?s)REFERENCES\r\K.+?(?=CHAPTER)

(add a question mark after .+)

[Robert at ID-Tasker]

@Peter Kahrel 

Thanks.

Much better, after I've removed extra space in one place and ":" in another - found almost all - but still skips one block - and I don't see anything wrong there:

I've copied missing block to a Notepad to clear all formatting and junk - didn't help, still skipping it.

[Peter Kahrel]

Can you show the context of 'there'?

[Robert at ID-Tasker]

Can you show the context of 'there'?

By @Peter Kahrel

I just played a bit with it - by removing some paragrpahs - from the start of the block and the end of the block:

The top 4x paragraphs - there is more on the previous page - if they are there - this whole block won't be found...

BUT - if I'll remove ANY of them - 1st or 2nd or 3rd or 4th - but only one - this block will be found?!?!

So it's not like any specific paragraph affects the result - removing ANY ONE will find the block?!?!

[Robert at ID-Tasker]

Something is really wrong with GREP in InDesign...

I've spend last 20 minutes playing with it - and after I trimed it down a bit - removed few Paragraphs from the end of the block - then I can remove ANY additional paragraph - and whole block will be found by GREP?!?!

If I hit Ctrl+Z / Undo - block not found, remove ANY other random Paragraph - block will be found...

[Peter Kahrel]

I now remember that there is indeed a bug in inDesign's grep engine: the dot doesn't match the footnote marker. To get around that, use this one:

(?s)REFERENCES\r\K(~F|.)+?(?=CHAPTER)

[Robert at ID-Tasker]

Thanks @Peter Kahrel - but there are no Footnote markers in this block of text.

[Marie_D]

I have experienced the same problems. There is definitely a bug with long grep searches in InDesign.

[Peter Kahrel]

A simple experiment shows that a Grep search matches at most 20,000 characters. This restriction appeared a few years ago and was fixed, and now it's back apparently.

[Marie_D]

Long grep search has been broken in several versions of InDesign since that time it was fixed. Really frustrating as it is an extremely useful way to format features in book text.

[Robert at ID-Tasker]

@Marie_D

My solution will always work. 

Do you work on a Mac or PC? 

[Robert at ID-Tasker]

A simple experiment shows that a Grep search matches at most 20,000 characters. This restriction appeared a few years ago and was fixed, and now it's back apparently.

By @Peter Kahrel

The block in question is way shorter. 

At one point I even suspected "opening single quote" in the middle of the last paragraph in the block - but after reloading document - it wasn't a problem. 

So something is really wonky with the GREP. 

[Peter Kahrel]

> The block in question is way shorter. 

There could be all kinds of hidden characters (discr line-breaks, text anchors). They all count as characters.

[Robert at ID-Tasker]

> The block in question is way shorter. 

There could be all kinds of hidden characters (discr line-breaks, text anchors). They all count as characters.

By @Peter Kahrel

Even if I go through the Notepad - doesn't help.