Skip to main content
Inspiring
September 19, 2008
Pregunta

Getting rid of unwanted precision

  • September 19, 2008
  • 14 respuestas
  • 1321 visualizaciones
I'm generally enjoying InDesign after many years of holding out with PageMaker, but one thing I haven't found convenient about it is its apparent tendency to give the user more precision than he wants, needs, or can use. If I want a tab at 7mm and try to place one there, for example, ID will give me one at 7.188 or something like that, and then I'll have to go into the dialog box and manually delete the unwanted fractional value.

Now I'm starting with ID scripting, and right at the beginning, in one of the first specimen scripts of the ID scripting tutorial - namely "ImprovedHelloWorld.applescript" - I'm getting this in an apparently excessive degree, with for example 27.699999999946 and 17.999999999994 as results, rather than the normal, desirable, legible and manageable 27.7 and 18.

Can anyone give me any advice on how to weed out this extraneous and unwanted precision, perhaps in the program at large as well as on the scripting side of it?

Thanks,

Roy McCoy
Rotterdam, NL
ID CS3, OS X 10.5.5, MacBook Pro
Este tema ha sido cerrado para respuestas.

14 respuestas

Jongware
Community Expert
Community Expert
September 21, 2008
Also note that the measurements you see on the screen might be re-calculated from the internal measurements (I seem to recall ID uses points internally, but that hardly matters).

The issue of a wanted "exact" precision might bite you back in some unexpected ways. For example, if you place an object at "2cm,2cm", it may appear in the document as "2.000001cm,2.000001cm" because of internal rounding. Now, if you want to check if it's still there some time later, hey, there is nothing at "2cm,2cm" (fortunately, there are other ways to identify objects
i not
based on their exact position).
The same happens all the time with Word imported documents, where in the UI you can see a text indent value of "24pt". Well, that's already rounded, as you cannot
i search
for this 'exact' indent of 24pt -- checking with a script reveals it to be, for example, at 23.9947658(etc.).
Inspiring
September 20, 2008
The precision is a result of base 2 (binary) numbers and base 10 (decimal) numbers not mapping "precisely" on to each other. They're something you have to live with and accommodate in your scripts.

Ah, I realize now that when I first saw your name on this topic, I mistook you for Roy Marshall who does use JavaScript.

If you're comfortable with AppleScript then stay with it. I switched to JavaScript partly because I could never come to grips with AppleScript whose statements and partly because JavaScript is cross-platform.

Nice to see you again (now I've worked out who you are!).

Dave
Inspiring
September 20, 2008
Hi Dave, it's great knowing you're here. I did open ESTK (for the first time ever - I wasn't even aware of it before), and I looked at all the Number Instance Elements. But I didn't see what you meant, and anyway I'm sticking with AppleScript for the time being, since I haven't found the time even to get back into that lately, after years of laziness just using my old stuff with PageMaker.

Forgetting scripting for the moment, perhaps I can ask you if you've suffered the same general annoyance with all the multiple decimal places. To mention my previous example again, you would click on the tabs ruler in PageMaker and get a tab at an even or a half millimeter, and if you for some reason in an exceptional case wanted something more precise than that you could enter it manually. Is there any reason why it shouldn't be that way now? Or have I missed something, like you should click with the shift key or something if you want an even increment.

Getting back to scripting, I'm sure I can do rounding in AppleScript as well as JavaScript, and I did it with PM scripting to get dimensions on graphics down to a level I would care to resize to in Photoshop. Looking back at the old scripts, ha, I see a "Dave Saunders" folder with a Height Calculating PM script from ten years ago. I also see "Folios Ahoy" and various other things. I guess we'll be in touch again if you're hanging out here and this is a main place for InDesign scripting, but it will be sad for me if you're mainly into JavaScript... though I suppose I could switch, as I've barely started.

Roy McCoy
Rotterdam, NL
Inspiring
September 20, 2008
Roy,

Use ESTK's Help menu to look at the Core JavaScript object model. Take a look at the Number.toString() method.

Dave