How to Identify Missing or Incorrect Cross-References in a Manual Index

[bine_ro]

I am currently working on manually indexing over 6,000 entries in a book that contains countless cross-references within the index. Since I'm doing this chronologically and many of the referenced entries don't exist yet, I have to manually type or paste the cross-reference entry. Unfortunately, I’ve noticed that the editor who provided me with the file (a PDF with comments) occasionally made typos or omitted parts of the entry names.

Is there a way to identify cross-references in the index where the corresponding entry doesn't exist?

<Title renamed by MOD>

[Robert at ID-Tasker]

If you work on a PC - even free version of my ID-Tasker should help you with browsing those.

[Peter Kahrel]

> Is there a way to identify cross-references in the index where the corresponding entry doesn't exist?

Not sure I understand this. It's not possible to have a cross-reference to a non-existing topic: when you create a cross-reference for a new topic, the referenced topic is created automatically.

[bine_ro]

The topic is generated, but not a reference, since there needs to be an index marker for that. If I create an index and a cross-reference points to an incorrect or missing reference, it will be displayed as though the reference exists. The only way to verify this seems to be by manually checking each entry. And that is very time consuming, prone to error and since this index has throusands of entries, it's not really manageable.

What I probably need is a way to list the index entries with cross-references to unused topics?

I’ve attached an image to illustrate what I mean.

[Peter Kahrel]

That means that the cross-references are part of the topic names, correct? So you see something like the first screenshot, while it should be something like the second one.

What you're after is something that if there's a 'manual' cross-reference that points to a non-existing topic, the topic is created. Correct? If that's what you want it's an easy script.

[bine_ro]

No, I create the reference like in the second screenshot (using see [also]). The referenced topic "Rückzug" is created. Then, I hope that somewhere along the many index entries I am going to add in the process, the topic "Rückzug" will be referenced in the text. But if it's not, I need a way to find a way to single the unreferenced topics out since the cross-reference goes to nowhere—or rather, I need to single the cross-references out in which the references lead to an unused topic.

I hope this is making it more clear. I'm sorry if it's confusing.

[Peter Kahrel]

Aha, I see. You want a list of unused topics, that is, topics without any page references.

If the book is in one document it's easy. If the book is in two or more documents it's slightly more complicated because a topic may be unused in one document but used in another one.

Which is it?

[bine_ro]

I had to use the book function since its very large. There are 27 files.

[Robert at ID-Tasker]

@Peter Kahrel

What happened to your nickname??

[Robert at ID-Tasker]

@bine_ro

Can you share a sample document? On priv if you prefer - please click my nickname.

[bine_ro]

I'm afraid your ID-tasker won't be allowed from our IT-department. Anyway, I quickly made a sample document with correct and incorrect cross-references withing the index and attached them. Since the Adobe Community won't let me upload an indb file, you'd have to put them in a book file yourself.

[Robert at ID-Tasker]

@bine_ro 

There is nothing in the 1st document - but there is in the 2nd:

There are "links" to the original locations of your documents - column "Dest Doc Path".

There is not much available directly in your 2nd document:

ID-Tasker is extremely powerful tool - the free version, gives you access to the complete structer of your document(s) - in paid version ... well ... it's a completely different universe ...

[Robert at ID-Tasker]

@bine_ro 

Those are your Index Entries:

Fully clickable - double click in the TYPE column - will switch to the correct document and select entry in the text:

[bine_ro]

Thank you, I appreciate it. But I am not looking for a list of existing index markers but of the cross-references referencing missing index markers.

[Robert at ID-Tasker]

Thank you, I appreciate it. But I am not looking for a list of existing index markers but of the cross-references referencing missing index markers.

By @bine_ro

There are no Cross-References in your documents.

[bine_ro]

There are no Cross-References in your documents.

By @Robert at ID-Tasker

I'm sorry, it's confusing since the terms are the same. I don't mean cross-references as in hyperlinks. I mean the cross-references within an index that reference other index entries. Like Refererence. See cross-reference

[Peter Kahrel]

Since your IT dept. won't allow tools from different universes, maybe they'll allow a script.

The idea is to force unused topics to print in the index. The easiest way to do that is to create a page reference for an unused topic and set its type to 'Suppress page range'.

The below script does that. In each document, if there are unused topics, the script creates a small text frame on the first page of the document and creates that page reference in there. Those frames are labelled 'Unused topic' on the Layers panel.

d = app.documents.everyItem().getElements();

function processUnusedTopics (doc, topics) {
  
  var frame;
  
  function getFrame () {
    var f = doc.textFrames.item('Unused topics');
    if (!f.isValid) {
      doc.pages[0].textFrames.add ({
        geometricBounds: [0,0,'10mm','10mm'],
        name: 'Unused topics',
      });
    }
    return f;
  }
    
  for (var i = 0; i < topics.length; i++) {
    if (topics[i].pageReferences.length === 0) {
      frame = getFrame();
      topics[i].pageReferences.add (frame.insertionPoints[0], {
        pageReferenceType: PageReferenceType.SUPPRESS_PAGE_NUMBERS,
      });
    }  
  }
}


for (i = 0; i < d.length; i++) {
  if (d[i].indexes.length > 0) {
    t = d[i].indexes[0].allTopics;
    if (t.length > 0) {
      processUnusedTopics (d[i], t);
    }
  }
}

[Peter Kahrel]

I forgot to add (maybe unnecessarily) that you can find the unused topics by generating the index and checking topics without a page reference. The script could mark those topics to make them easier to find or the script could list them as well as adding them.

[bine_ro]

Thank you so much. This script is already step in the right direction! You wrote that "the script could also mark the topics". How would that work?

Right now as solution to my problem I am trying to find a way to just color the topics without a number behind it (to make them visible). And that way, I could search each and find where they were referenced. I came up with (?<!.)([\l\u]+( | |, )?\l)+?(\)|\")?(?=\r)

Unfortunately, there is also a factor that I previously hadn't throught of. There is no way to know, if it really is an unused topic or if it's a main topic without a reference but with a subtopic. So that way (and as far as I see the same problem exists within the ID-Tasker) there are many false-positives.

Would there maybe a way, rather than listing the unsused topics, to just mark the concerning cross-references and edit them to have some glyphes to grep-search for? Like Refererence. See cross-reference becomes Refererence. See **cross-reference** or something like that?

[Robert at ID-Tasker]

@bine_ro

Have you seen my last screenshot - with light red backgrounds in the Index Topic column?

Those are your "blanks". 

This list can be easily exported - blanks only. 

Or if you wish - I can make it clickable - straight to your Index Story. 

[Robert at ID-Tasker]

@bine_ro 

Attached.

[Peter Kahrel]

Marking up is not such a good idea after all, because a marked topic is a different topic, so you may end up with bboth a marked and an unmarked topic.

Another possibility would have been to create unused topics with a page reference and mark up the page reference with an override style, so that you could look for that style in the index. But because of an InDesign bug dating back to at least CS6 those character styles don't make it to the index in book jobs.

The Grep expression you used seems overcomplicated. With this one:

^[^\d]+?\r

you find all entries that don't end in a number. That finds parent topics without a page number as well, so you get some false positives, but at least you won't get any false negatives, which is more important.

[bine_ro]

Another possibility would have been to create unused topics with a page reference and mark up the page reference with an override style, so that you could look for that style in the index. But because of an InDesign bug dating back to at least CS6 those character styles don't make it to the index in book jobs.

Oh, I didn't know that yet. Another thing that works wrong in this project... (index entries in tables need to have bold numbers) Well I at least found out that if I open the file with the index and edit the entries there afterwards, the style will stay. Of all the things that will delay this project, this is probably the least worrying one.

Do you know if the bug was reported in ID UserVoice or elsewhere where I can give my support? I couldn't find anyone mentioning this.

Thank you for your Grep expression but it's too general since it also finds the parts with the crossreferences and I just want to single out the lone-standing entries. My expression didn't work that well either but I found another one. (For some reason it excluded the topics when there wasn't a hit in a first line of a aphabetic section).

But I found a way that will work for me:

1. I use your script (In copies of the original files so I won't have the addictional entries in the end file)
2. Then use the GREP ^(?!.*[.\d]).*$ and color all entries without a digit or cross-reference
3. I cut out all irrevelant (black) entries and edit the list so the entries will be separated by | (or)
4. I then use grep to search for all entries with see or in my case siehe in it followed by the list from #3. and color it in another color or mark it with a style for the editor to quickly see which are wrong. (This I would do in the original file)

Like this: See( also)? \K(ciaspe|Dolorerunt|...)|(\: (ciaspe|Dolorerunt|...)+?

That way, I can single out only the references that are wrong. And I have exactly what I need.

Thank you so much for your help Mr. Kahrel!

Also @Robert at ID-Tasker . I will keep your tasker in mind. It's just that our IT-department currently won't allow this kind of third party program.

[Robert at ID-Tasker]

@Peter Kahrel 

My IDT loads ALL index entries - so "blanks" should be included as well.

OK, I need to implement this for the top level:

if (topics[i].pageReferences.length === 0)

Thanks.

[Peter Kahrel]

Those are the 'Unused topic' text frames I mentioned.