What is this line in between the words?

[Sourabh Shivaji Khutwad]

Can any one help understanding what the below line is? This is invisible when text is not selected and only appears when the word is selected. This is causing the words/paragraph to break in SDL studio when wherever this line appears. Doesnt look like any marker. Never seen this before.

[Sourabh Shivaji Khutwad]

Hi Joel, attached is the file containing the segments that are broken studio view

[Joel Cherney]

Well, now that's quite odd! There are some very small formatting changes that ought not cause this kind of tagging; changes from 13.5 points of leading to 14 points, for example. That seems to me to imply something going on in the InDesign file that is causing the Trados import filter to choke on it. Simple document corruption in InDesign is the most likely culprit, although I don't know if a Save As or a roundtrip through IDML would fix it for you. Perhaps the CleanUp Tasks method would be the most reliable in the future? 

[Peter Spier]

I've just downloaded and opened this sample. Not only don't I see the breaks on my screen, it comes through with no paragraph styles assigned, which I thought was impossible in current versions, so I had a look at the file history:

Converted - yes
Open As Copy - yes
Open Missing Plug-ins - no
Save As - yes
Created - yes
Transfered - no
Converted from QuarkXPress - no
Converted from PageMaker - no
Book - sync - no
Book - repaginate - no
Opened from InDesign Interchange - yes
Most recent Save - yes
Book - update cross reference - no
Open a Clone - no
Used Content Dropper - no
Used Layout Adjustment - no
Exported EPUB2 - no
Exported EPUB3 - no
HTML-Fixed-Layout export - no
Document created in Touch Workspace - no
Document sent to ID from Adobe Comp - no
Macintosh - no

Created on Windows 6.2 in app version 8.0.0.370 (FS InDesign Roman) build 370 on Wednesday, June 21, 2023 at 1:01 AM
Opened from InDesign Interchange on Windows 6.2 in app version 8.0.0.370 (FS InDesign Roman) build 370 on Wednesday, June 21, 2023 at 1:01 AM
Save As on Windows 6.2 in app version 8.0.0.370 (FS InDesign Roman) build 370 on Wednesday, June 21, 2023 at 1:01 AM
Open As Copy on Windows x64 10.0 in app version 16.0.0.77 (FS InDesign Middle Eastern) build 77 on Wednesday, June 21, 2023 at 2:51 AM
Converted on Windows x64 10.0 in app version 16.0.0.77 (FS InDesign Middle Eastern) build 77 on Wednesday, June 21, 2023 at 2:51 AM
Save As on Windows x64 10.0 in app version 16.0.0.77 (FS InDesign Middle Eastern) build 77 on Wednesday, June 21, 2023 at 2:51 AM
Open As Copy on Windows x64 10.0 in app version 16.0.0.77 (FS InDesign Middle Eastern) build 77 on Wednesday, June 21, 2023 at 2:52 AM
Save As on Windows x64 10.0 in app version 16.0.0.77 (FS InDesign Middle Eastern) build 77 on Wednesday, June 21, 2023 at 2:52 AM
Most recent Save on Windows x64 10.0 in app version 16.0.0.77 (FS InDesign Middle Eastern) build 77 on Wednesday, June 21, 2023 at 5:49 AM
Open As Copy on Windows x64 10.0 in app version 18.3.0.50 (FS InDesign Roman) build 50 on Tuesday, June 27, 2023 at 3:09 AM
Converted on Windows x64 10.0 in app version 18.3.0.50 (FS InDesign Roman) build 50 on Tuesday, June 27, 2023 at 3:09 AM
Save As on Windows x64 10.0 in app version 18.3.0.50 (FS InDesign Roman) build 50 on Tuesday, June 27, 2023 at 3:12 AM

Since this was opened from an interchange file we don't have the whole history, but I do see it's gone through a lot of conversion between versions, starting out on an un-patched CS6 install on Windows 8 and moving from Roman to  an unpatched ME version 16 of InDesign while being opened in Win 10. There's a further reference to opening and converting this morning (before I opened it) with  version 18.3 (Roman)  (I'm still at 18.2). Oddly this is not listed as the most recent save.

I understand that perhaps some of this conversion was necessary to use the translation software, but I wonder if all of this back and forth between versions might be part of the problem.

[Sourabh Shivaji Khutwad]

Hi Peter, Like I said yesterday replying to George, this might be version issue. I received this file yesterday morning and I have the latest version of InDesign. The page I extracted is saved from the latest version. I am not sure how many times client have made changes at their end before sending over to me. By the way how did you find the history of the file?

[Peter Spier]

To get the file history go to the Help menu and hold down the Ctrl key as you click About InDesign

[Sourabh Shivaji Khutwad]

Thank you! I didn't know this. 

[Peter Spier]

Very useful and poorly, if at all, documented.

Learned about it here myself about twenty years ago.

[Sourabh Shivaji Khutwad]

yeah, learnt something new today, thanks! 🙂

[Joel Cherney]

I am feeling fairly certain that they are not markers, because your InDesign screenshot shows that you have hidden characters visible, and there is just nothing there. If it were some kind of invisible character, there'd be something non-printing and blue, there, like the blue of the space or the Indent to Here characters that are visible in your InDesign screenshot.  

But, even if they're not markers, they're something, right? That something that is causing a formatting change in Trados. That something is a formatting change that seems to be invisible. These are not too uncommon in multilingual InDesign workflows; you'll find e.g. settings for Japanese ruby causing a formatting override, when you're in the middle of working on a German document that has no Japanese in it whatsoever. You might find something like that yourself, if you go back to your original broken file and mouse over the overridden paragraph styles. The mouseover popup will tell you how the formatted text diverges from the paragraph style, and in my personal experience it's always something weird that I am certain that I didn't turn on, like Arabic language settings on a Burmese project, or ruby settings in German, as above. 

That's one of the reasons that Peter suggested saving out an .idml and reopening that; sometimes that clears out problematic corruption in a document that can cause formatting hijinks like this. In some cases, I might suggest throwing out your preferences for InDesign entirely, as that's solved similar issues for me in the past. 

However, since you've fixed the overrides manually, that's probably not really called for here. Similarly, if you were the one making the source files, I (and the other forums regulars) could suggest some document construciton best-practices that would help prevent similar problems from happening in the future. However, I'd bet that you are actually working with your clients' unreliable files, just as I am. 

A few final notes - apparently, these problems have been happening for a long time. I found lots of discussion, on proz and elsewhere, talking about spurious cf tags that look exactly like what you were experiencing. The frequently suggested solution was not to go hunting for the problem in the souce file, but instead to use CleanUp Tasks to resolve the unnecessary tagging. I have never used it myself, but it looks like an automated version of TagEditor to me. Seems much easier than cleaning up in TagEditor, to be honest. However I've not used it myself, and unless I wind up working in an SDL shop again, I'm not likely to be exposed to it. 

[Sourabh Shivaji Khutwad]

Hi Peter, thank you for your inputs. Unfortunately I do not create these documents as these come in just for translation and post DTP to me and explaining such issues to client is like casting pearls before swine. Instead we just move ahead finding alternate solution to run the project smoothly like for this I had to re-style everything affected. And yes, I have experienced the same formatting changes when working with Asian languages specially.

It was a quite nightmare for me yesterday to fix all these issues in a total of 10 files with 10,15 pages in each document. We have a 24 hours turn around time to deliver a quote to client as that's what we have agreed with client in SLA and there are no exceptions  unless there's huge amount of work.

Using third party apps needs persmission and a lot of hussle to get these approved, therefore it is better we find a solution by ourselves or offer client to completely recreate the file from scratch which is what we will offer here onwards if such issues ever occur.

[Brad @ Roaring Mouse]

Two observations to add:

I created a sample file of my own, and was able to see the same thing (btw: did it in an older version of ID to eliminate versioning as the issue). In my case I went back to CC2020.

a) in most (but not all) cases, the lines occur between changes of colour: in the your file, the bullet is blue, the SPACE between (and any Indent to Here inerts) is Pink and the text is Grey)... and also in changes in line spacing, as there's plenty of that going on for some reason. It could be your software is "seeing' these differences as places to break copy.

b) the lines appear or disappear if I toggle between CPU and GPU, so this points to a screen draw issue.

That being said, there are no 'hidden' characters in any way.

Regardless, you need to work with paragraph styles, especially with your indented bullet items instead of using spaces and "Indet to Here"... and make sure any overrides are eliminated if they aren't required.

[Peter Spier]

Regardless, you need to work with paragraph styles, especially with your indented bullet items instead of using spaces and "Indet to Here"... and make sure any overrides are eliminated if they aren't required.

By Brad @ Roaring Mouse

Unfortunately @Sourabh Shivaji Khutwad is not the author of the original document here, so that seems not likely.

[Joel Cherney]

When I get a damaged or poorly constructed InDesign doc that I have to throw into an app like SDL Trados Studio, I never know how much preparatory work is going to be necessary before it's ready to go to translation. Sometimes I just have to fly through and remove some soft returns with a regex; other times I have to basically rebuild it from the ground up. But my target isn't a well-constructed InDesign document, it's merely one that is not so broken that it gets stuck in the hopper. However, folks in Sourabh's shoes (like me) do expect in most cases to need to do at least some preparatory work before a given InDesign doc gets processed. Honestly, the quality of work we talk about here on these forums is found in maybe 2-3% of the docs I handle, and that's being generous. Files from many people with the "graphic designer" job title are just absolute spaghetti inside.

Sourabh's problem here is that these minor style changes discussed by Brad are leading to major tagging problems, when they ordinarily wouldn't. I sometimes go through and correct a client's styling before dropping the file into the TM hopper, as such automatic text-harvesting tools tend to work better with well-groomed styles. But this case, where some minor color changing is leading to very fragmented text in Trados, is an outlier. If I had to clear "all overrides" on all InDesign docs before dropping 'em into my TM workflow, well, I'd be billing for twice as many hours.... or, more likely, my social service clients would only be able to afford half as much translation work.

[Brad @ Roaring Mouse]

Not knowing how the Studio workflow works, I had suspected that the text in InDesign might have been in some way "exported" in RTF format/XML when saved as the IDML that Studio requires. To that end, I exported one of the problematic paragraphs of copy to RTF, and on analyziung that, all the minor formatting changes indeed broke the copy up into separate blocks, hence upsetting tagging.

[Sourabh Shivaji Khutwad]

I am neither expert in Trados nor in InDesign but I have learned a lot in past couple of days and I thank all of you coming forward to help resolve this. @Brad the issue is just not around the bullet or colored text but in between the paragraphs as well where there's no bullet point or two colors used, here's one screenshot.

Instead of wasting more time I had to recreate every style affected and push it ahead. The only way to resolve this is to have clean and clear styles for all text layouting and not to override any of styles especially if it involves colors.

[Brad @ Roaring Mouse]

Yes, I see that within the paragraphs as well, but as was already mentioned, and you can notice for yourself, the type specs before and after the "lines" are different (different line spacings. e.g. 13.9 before the line and 13.5 after the line). This is happening all through your document (why, who knows, but not using and applying proper paragraph styles is not helping you out here), and this is causing the text to break up in XML, which is probably breaking up your tagging.

This is what your above paragraph sample looks like in XML:

 

[Sourabh Shivaji Khutwad]

yeah!

[Joel Cherney]

So, I think I might actually have The Answer to Sourabh's question. 

So, we expect these translation tools to segment the text on sentences. Each one has "segmentation rules" that are in my experience usually regular expressions, that allow the tool to present only complete sentences to the translator. Chopped-up sentences like this are a problem for your localization engineers and DTP operators, because they will never match what you already have in your translation memories, and so your translation costs will be higher, and your translators will complain.

So why did it happen here? Usually, changes in font size and color don't force a new segment to start. The rules will say "force a new segment to start after a period or question mark followed by a space" not "force a new segment to start when the font size changes." So Trados is finding something in the XML output and assuming that it means "start a new segment." But looking at the XML of the story you posted, Brad, I see only proper segmentation! Whole sentences enclosed in Content tags, that's like a best case scenario for a TM tool. 

But if you look at the screenshots from Trados that Sourabh posted, there's no reason to segment after just a single letter, right? There must be something there in the XML export that is causing a segmentation where there wouldn't ordinarily be one. I see this all the time in XML output from AEM Forms Designer/Livecycle Designer; a sentence stored as HTML with some emphasis will enclose that emphasis in its own <div> tag, and the XML parser sees a DIV tag as a paragraph tag, and forces a segmentation. This is Bad from a saving-time-and-money-with-TM-tools point of view.

So, because Sourabh had to get that thing assigned right away and didn't have time to spend days chatting on the Internet with old guys trying to figure it out, the best solution was just to restyle everything as quickly as possible to get it assigned. But I think that Peter actually may have found the reason that there was spurious tag stuff in the source English document... he looked at the document history. When I see a document history like that, I assume that I have to rebuild from the ground up. It's been revised and resaved so many times that I would no longer think of it as a trustworthy document. But before I give up and try what Sourabh tried, I start with:

1) export IDML and re-open, and/or

2) drag all the pages to an empty fresh document, and/or

3) perform a diagnostic recomposition

The third one has saved me from being forced to re-style a client document numerous times, and I don't really know why it didn't occur to me the minute that I first found this thread. 

[Peter Spier]

Worth noting that the sample file actually did go through .idml first thing.

[Sourabh Shivaji Khutwad]

haha! I wouldn't say old but vastly experienced, 🙂

Unfortunately none of the above works, even the third one; I noticed that only the kerning and tracking are set to default and or I am missing something here. Post that I run the IDML in Trados and the issue still persists. 

[Geоrge]

I don't see any error or a bug with InDesign. This is Trados problem. Open ticket somewhere on Trados site and eat their mind, stop eat our!)))

And don't forget to tell us if they do any real and helpfull answer to you. 

[Peter Spier]

We really are old.

[Pickory]

Hello,

As another poster noted there is a difference with the leading 13.5 - 14.  When the substittuded check box is selected, the pink you see is drawn by a GlobalTextAdnorment routine. Some times the boundery of the highlight regions overlap and you will see a darker area where they overlap. I suspect this is just a visual effect.

P.

Edited to sya..

Here is one of my GTA going wrong.