Generated index is NOT in alphabetical order

[Solo46]

Dear community,

To my surprise InDesign generates incorrect indexes. Se example below, every line starting with the surname “Dahl” should of course be kept together and be listed before any name with the same base/syllable. In this case before “Dahlbom”!

What to do?

Similar problem:

Wrong alphabetical order in index

I couldn't get @Peter Chater Kahrels script to work in InDesign 14.0.1.

[Barb Binder]

Hi Solo46,

Yes, and I remember another (different) post on this within the last few months with the same issue. Since Peter Kahrel has a script that addresses this, let's see if we can get him to swing by to help. 

~Barb

[Peter Kahrel · Accepted solution]

Solo -- InDesign's index sorting is indeed pretty awful much of the time, especially with name indexes. Please go to Adobe InDesign Feedback and tell Adobe about the problem and submit a request to improve index sorting (and tell them how).

In the meantime, your name sample can be sorted correctly by setting each topic's sort order to the topic's name with the separating comma+space replaced with any character that's naturally sorted before any letter. Digits do that, so let's use the zero:

if (app.documents[0].indexes.length == 0) exit();
topics = app.documents[0].indexes[0].allTopics;
for (i = topics.length-1; i >= 0; i--) {
  topics.sortOrder = topics.name.replace (', ', '0');
}

P.  (Thanks for the ping, BarbBinder​)

[Solo46]

Thanks Peter and Barb!

Will that script edit the index data for just the active file?

Should the script file be named .JS or .JSX?

Just to see if I understand correctly. Will "name.replace (', ', '0')" change the sort order field from:

Smith, John

to

Smith0John

?

And one more detail. I have already used manual sorting for several topics by adding text to sort order field. It would be elegant if the script first checked if any data is present in sort order field. And if so, edits that string. If not, takes topic filed string and adds it to sort order field with comma space replaced with zero.

It seems like InDesign ignores punctuation marks when sorting. I.e. sort order field name "Smith0John" or "Smith,0John" gave the same result. So maybe the script can be edited to "name.replace (', ', ',0')".

Is it possible to add more strings to be replaced with the same replace command? Names that have no known first name in my index are instead separated with colon and then followed by title. In other words, could colon + space also be added to the replace command?

I am a bit anxious, because the index takes 30 min to generate, it contains over 7'000 lines/topics.

[Peter Kahrel]

> Will that script edit the index data for just the active file?

Yes.

> Should the script file be named .JS or .JSX?

Doesn't matter, both work.

> It would be elegant if the script first checked if any data is present in sort order field.

Well, you didn't mention that, I went by your sample and description. This changed code ignores topics that already have a sort field:

if (app.documents[0].indexes.length == 0) exit();  
topics = app.documents[0].indexes[0].allTopics;  
for (i = topics.length-1; i >= 0; i--) {
  if (topics.sortOrder == '') {
    topics.sortOrder = topics.name.replace (/[,:]\s*/, '0');
  }
}

> And if so, edits that string.

Edits -- how?

> If not, takes topic field string and adds it to sort order field with comma space replaced with zero.

That's not possible. You can have only one sort-order string.

> Is it possible to add more strings to be replaced with the same replace command?

You can now add characters. In line 5 you see [,:] and you can add characters to that list in brackets.

> I am a bit anxious, because the index takes 30 min to generate, it contains over 7,000 lines/topics.

Try it on a small sample.

P.

[Solo46]

Thanks Peter!

Now I see that the replace function uses regex, great. I am familiar with regex, but not so good at writing JS.

> Edits -- how?

If a sort order field string already is present, then the script should prioritise sort order field string over topic field and replace the comma/colon and space with zero for sort order field string. And if sort order field string is empty the script should take the data from topic filed and replace comma/colon and space with zero and add it to sort order field (just as before).

[Peter Kahrel]

Was it 'Duh!' or 'Doh!' ?

(function () {
	if (app.documents.length === 0 || app.documents[0].indexes.length == 0) {
		exit();
	}
	var re = /[,:]\s*/g;
	var topics = app.documents[0].indexes[0].allTopics;
	for (var i = topics.length-1; i >= 0; i--) {
		if (topics[i].sortOrder == '') {
			topics[i].sortOrder = topics[i].name.replace (re, '0');
		} else {
			topics[i].sortOrder = topics[i].sortOrder.replace (re, '0');
		}
	}
}());

P.

[Solo46]

Thank you so much for the new version with sortOrder check!
This is how InDesign should sort names alphabetically from the start.

I've tried you script on some test documents and it seems to solve the problem. I will be working with the index more during the week.

When cleaning up the index I found some unused topics and your excellent Index statistics script could count them. But is it possible to add a listing of all unused topics? That would be very useful as it takes quite a long time to find the unused topics. Maybe another script could delete all unused topics, or with a checkbox in the stat. script?

Out of curiosity:
I noticed that you used "i--", is it better to step backwards when scripting indexes in InDesign?

[Peter Kahrel]

Yes, a list of unused topics can be added. Looking through the code I notice that the script marks as unused a topic that is used as a cross-reference, that needs to be changed.

As to removing unused topics, doesn't InDesign's function work for you? It's in the Index panel's fly-out menu.

I used i-- because at some stage I got weird results iterating forward. But in addition, stepping backwards is usually quicker.

P.

[Solo46]

Great.

Looking through the code I notice that the script marks as unused a topic that is used as a cross-reference, that needs to be changed.

That explains the contradictory and odd results I've got when working with the index stat script. It would be fantastic if that could be fixed.

Re: removing unused topics.

Haha, yes the fly-out menu works fine. I forgot it and was thinking about JS. When your only tool is a hammer …

Edit: No, the fly out menu remove unused topics is greyed out even when there are unused topics present. But show unused topics in the same fly out panel was handy. Although not as handy as a condensed list with only unused topics.

Thanks for explaining about iteration.

[Peter Kahrel]

> the fly out menu remove unused topics is greyed out even when there are unused topics present.

Select any topic and 'remove unused topics' is available.

[Solo46]

Select any topic and 'remove unused topics' is available.

Thanks, yes that solved it. Odd interaction for such well made software, since there is only one index for each file.

[Peter Kahrel]

> But is it possible to add a listing of all unused topics?

I added a button to the window so that you can print the unused topics to a file. The file is saved on your desktop and opened.

P.

[Solo46]

Thanks. This will be very useful when troubleshooting.

> I added a button to the window so that you can print the unused topics to a file. The file is saved on your desktop and opened.

Superb. But I couldn't find it. Did you upload it to your site?

http://www.kahrel.plus.com/indesign/index_statistics.html

> the script marks as unused a topic that is used as a cross-reference

Did you look into this also?

[Peter Kahrel]

> But I couldn't find it. Did you upload it to your site?

I didn't, but now I did. Sorry.

> Did you look into this also?

Yes, I did. If a topic is used only as a reference topic, it's not considered unused (as InDesign does and the script did earlier).

P.

[Solo46]

Great news Peter. Thanks. Your script is an elegant solution for working with large indexes!

I tested it on a large document and the new version seems to calculate correct.

I really appreciate your help in this subject.
And it's a bit of a mystery to me that these tools are note available as standard tools in InDesign 2019.

[Solo46]

Well well, as usual it's more to this.

There where more special cases ahead so it's probably better to make a script that solves "all" index sorting problems. Previously I had some manual edited sortOrders (especially for W/V) that I wished to keep. Hence the previous care taken of topics already with sortOrder present, but it's much easier to accomplish it all in a script.

But I am beginner at scripting. My script draft only results in line 7 being noticeable affected in the document. And my addition to the script can probably be much more elegantly done.

Something is wrong, is topics.sortOrder is been overwritten for each line? A new variable might be needed or all regexes might be performed during a single replace call? But I don't know how it should be done.

if (app.documents[0].indexes.length == 0) exit();  
topics = app.documents[0].indexes[0].allTopics;  
for (i = topics.length-1; i >= 0; i--) {  
  topics.sortOrder = topics.name.replace      (/(?-i)W/, 'V');  // Fix Swedish alphabetical order. Treat W as V
  topics.sortOrder = topics.sortOrder.replace (/(?-i)w/, 'v');  // Fix Swedish alphabetical order. Treat w as v
  topics.sortOrder = topics.sortOrder.replace (/['´]/, '');     // Exclude punctuation that InDesign sorts as normal characters
  topics.sortOrder = topics.sortOrder.replace (/[,:]\s*/, '0'); // Fix alphabetical order for names, in format "Smith, John"
}  

And this reported as error 21: undefined is not an object.

if (app.documents[0].indexes.length == 0) exit();  
topics = app.documents[0].indexes[0].allTopics;  
var myIndex;
for (i = topics.length-1; i >= 0; i--) {  
  myIndex = topics.name;
  myIndex.replace (/(?-i)W/, 'V');  // Fix Swedish alphabetical order. Treat W as V
  myIndex.replace (/(?-i)w/, 'v');  // Fix Swedish alphabetical order. Treat w as v
  myIndex.replace (/['\´]/, '');    // Exclude punctuation that InDesign sorts as normal characters
  myIndex.replace (/[,:]\s*/, '0'); // Fix alphabetical order for names, in format "Smith, John"
  topics.sortOrder = myIndex;
} 

[Solo46]

Ok, I've been reading and testing.
This seems to work.

if (app.documents[0].indexes.length == 0) exit();  
topics = app.documents[0].indexes[0].allTopics;  
for (i = topics.length-1; i >= 0; i--) {  
  topics.sortOrder = topics.name.replace (/W/g, 'V')      
                                      .replace (/w/g, 'v')      
                                      .replace (/[\'\´]/g, '')  
                                      .replace (/[,:]\s*/, '0');
} 

Any feedback?

It can probably be done in some much more clever way.

[petern20010557]

Hello,

May be a little late but I am working already for some years on a book using Indesign CS 6 (about 800 pages). My index counts about 4000 entries and that all is still functioning. But the alphabetical order is - as we know - totally wrong as soon as spaces and "," are involved.

Example:

Braune, Werner (1909-1951)

Braun, Magnus Freiherr von (1878-1972)

Braun, Otto (1872-1955)

It shoud be:

Braun, Magnus Freiherr von (1878-1972)

Braun, Otto (1872-1955)

Braune, Werner (1909-1951)

There is a script on this site. But I know nothing about scripts ... is someone able to give me a "turn key" script I only have to paste in the scripts-menu. I can just handle that :(.

I am willing to give a few money for someones efforts. The script must be able to "eat" the 4000 entries.

Thanks, Peter.

[Laubender]

Hi together,

in the meanwhile Peter's scripts moved to creativepro.com. So the updated script can be found at:

https://creativepro.com/files/kahrel/indesign/index_statistics.html

Regards,
Uwe Laubender

( ACP )

FWIW: No idea why this thread moved to the top of the list of InDesign discussions some days ago.

Seems, this all entirely took place in February 2019.

[Troy Deckert]

I signed up for InDesign as a book publisher to be able to use a decent index. Looks as if the InDesign index is buggy, and hard to refine the order.  Is there another program out there that has an index that actually works well, right now, without adding code on our own?

[Laubender]

Hi troyd71007371,

you could look into Marc Autret's IndexMatic script:

https://www.indiscripts.com/category/projects/IndexMatic

Regards,
Uwe Laubender

( ACP )

[Troy Deckert]

Thank you, I will look at IndexMatic. The good news is that I did follow the InDesign instructions to get the index formatting working well, and it looks OK so far. I will start with a non-fiction book at 200 pages, which will have perhaps a hundred items to index, but so far so good with InDesign. As the alphabeticizing problem was in 2019, I hope that's been fixed. Rather than code it as you folks are doing, which I haven't done yet, if any index entries are out of order, and it's not too many entries, I could manually edit it by by various means, (like entering a white space where something should be and placing a single text frame over the white space, while also placing another text frame over any resulting extra index entrys that are out of order , but I hope I don't have to.)