Is XML and InDesign workflow a thing of the past?

[Sparrowhawk-AES]

I would like to explore XML importing into InDesign.  I remember taking a course on LinkedIn a few years ago but that course seems to have discontinued.  Is XML import into InDesign no longer a workflow option? Our company is looking for new workflow options for some of our dynamic content. Dynamic data publishing for individual records with extensive details as well as reports with image content. 

I found an old book that seems to be considered the bible of XML, indesign and dynamic publishing. It seems worth getting, reading and learning from it to explore this workflow. My main question is the XML workflow and InDesign past the EOL cycle?

If it is still a workflow solution, are there any tutorials on how to build on this old technology?

[James Gifford—NitroPress · Accepted solution]

All things considered, I'd look at XML into/out of FrameMaker. It was completely recast as an XML/SGML/DITA sort of tool about ten years ago. And while it's not as "artsy" as InDesign, it was more than capable of doing elaborate publication design even before the revision.

[Sparrowhawk-AES]

I have put FrameMaker on the development track for our team. It seems like a robust tool that can tackle many of our challenges.  Thanks @James Gifford—NitroPress for the recommendation. 

[m1b]

Hi @Sparrowhawk-AES, XML is still used quite extensively. It is often compared with JSON, because many modern APIs use JSON for configuration and data exchange. See this article for a reasonable comparison.

But Indesign doesn't read JSON, so JSON is only useful to Indesign if you have a plug-in or script/system that wants it. If you are using Indesign's native capabilities, with no scripts or plug-ins, I think XML is probably still your best bet (or use another tool altogether, as @James Gifford—NitroPress suggested). I'm no expert in this area—just adding what I know.

- Mark

[James Gifford—NitroPress]

I'm not sure XML and JSON are equivalents, not in the general sense of documentation content and publication. Both can be used for data exchange, but XML can describe and encompass entire publications, not just data blocks.

Unless I've missed something, you can have an XML data record, or a JSON data record, or an entire XML document... but there's no such thing as a JSON document.

[m1b]

You're spot on that XML and JSON are not equivalent at all. They come up together sometimes because systems that eschew XML often use JSON. Given a similar schema they can do similar things, but Indesign natively has no JSON capabilities (except via scripting) so it's a no go.

But yes you can have an entire JSON document—IF your app can read and write JSON documents. There is no reason why Indesign couldn't, if it had a mind to, read and write entire documents to JSON just like it does to IDML (which is XML).

- Mark

[James Gifford—NitroPress]

Well, you can have "a document" in any form including carved rocks and Excel. (Seriously, I once had a client who had built all their customer correspondence in Excel. It was the only 'structured doc' model they understood...)

There are umpty variations and flavors of vaguely XML markup, yes, and all, pretty much, can be used to do anything any of the others can. But JSON is a tool for universally accessible data chunks and I don't think I've ever seen a plug-in, extension or app that used it much beyond that scope. So "JSON + InDesign" could be a thing, but a thing unlikely ever to be raised from the undead. 🙂

Nearly all structured and packaged docs — including IDML and the MSO files — are either XML or a close variant. So it makes much more sense to talk about modular and structured doc work in ID with XML, even if it's something of a yesterday's format there. Unless, of course, you want to suggest SGML. 😄

[Bevi Chagnon - PubCom.com · Accepted solution]

It's about 10 years since we dropped XML/InDesign services from our studio, but it was working well enough back then. I doubt it is any better, and I also doubt it's worse! There are a quite a few scripts/plug-ins available to make the task faster and easier.

If I were to start an XML driven InDesign project, the first person I'd reach out to is Dorothy J. Hoskins. She was the most noted guru of XML + InDesign back then and wrote many books on the subject. She worked at Atypon for many years before branching out with her own firm Textenergy. Contact her at https://www.linkedin.com/in/dhoskins/ and via this LinkedIn group, https://www.linkedin.com/groups/2996628/

[Sparrowhawk-AES]

Thank you for providing a direction towards resources. I will look into the various paths and resources. It seems the Adobe Experience seems to have all the resources to provide products and services that we provide to our customers.

I appreciate the discussion and prespectives presented by all contributors. This short discussion has provided many paths to consider.  I am grateful for all insights and perspectives. Thank you.

[Robert at ID-Tasker]

Importing XML is only part of the "story"...

Either you need post-processing and reformat "raw" imported result - or use XML only as a source data and dedicated tool that will build document(s) based on this data. 

[Sparrowhawk-AES]

I am assuming the tasker is your product.  It does seem quite powerful.

[Robert at ID-Tasker]

Yes, extremely. You can either use it as a 3rd helping hand - when working on a current selection - or automate whole workflows - including interacting with Illustrator, Photoshop, Word, Excel, etc - anything that can be controlled using VB / VBA.

Then you can connect to a local or online databases, FTP, CMS, send emails, etc. - anything and whatever you need.

Everything that's built-in in the InDesign - is on steroids. 

[Dirk Becker]

InDesign has (at least) 4 distinct XML technologies.

XML import is one of them - very XML-ish, it even supports an antique version of XSLT, and DTDs (I never tried). Problem is that you better use it to fill in exactly structured templates with exactly tuned XML. Your XML will have to follow strict rules on whitespace - every space will show in document, and every return will produce a paragraph. Pretty-formatted (indented and wrapped) XML, ouch. If you have a complicated schema you may add post processing via scripts – they can navigate the imported data with XPath expressions and apply any changes. Insert comma separators into a list, or turn it into bulleted paragraphs? Have a table of umpteen sub-records, or better turn the single one into styled text? Choose a different template for the record with two images, or the extra long headline.

There are other import formats with schemas close to the InDesign internal formatting (e.g. specify a font), starting with tagged text – likely not your choice, too limited.

More recent formats map the scripting object model to XML elements and attributes, so you can address all aspects of the document with an outside processor.: ICML with a focus on text, IDMS snippets for page items, and IDML describes the entire document.

ExtendScript's E4X - an abandoned standard so unfortunately Adobe does not support it with the recent UXP Javascript. It is still very useful to access arbitrary XML with a smaller size, where all processing to bring the data into the document is performed in a script e.g. you bypass the XML "structure" features of the document.

UXP has DOM operations similar to modern web browsers, so you can use e.g. css selectors rather than XPaths. Besides, similar to ExtendScript you can perform any kind of post processing on the direct XML import.

From a workflow point of view, one big challenge can be to find a balance between all the processing stages especially if you intend to round trip.

Is your incoming data an export from a database? Is it using your own schema, or a specific standard? Is it XHTML or a subset?

You might need outside preprocessing e.g. to fill in text macros for elements that are only reference in your original data. An element that represents a status might turn into a paragraph within one template, or if you decide to switch to a different template it might turn into an image. Different text or image for different locales. Such choices may require layout information (does it fit the page) so you better provide all the data on import and display only a subset.

The post processing in the document may still not be sufficient, requring manual edits - tracking to squeeze a table cell, or fix a hyphenation mishap. You may want to preserve/manage those manual changes for later iterations, or return changes in the actual data (a fixed typo - thanks to the spell checker, the rewritten free text, the missing field as you finally see them in the layout) into the original data.

Even larger edits may destroy the document structure – the elements and their order. InDesign is no validating XML editor. I've seen projects add structure editors on top of InDesign, or limit edits dependent on the structure by use of plug-ins. Store snippets out of heavily modified text ranges. All doable.

[Sparrowhawk-AES]

Thank you, @Dirk Becker. This information is quite helpful. Our products and services include a varitey of different reports and printed products that follow consistent patterns and elements. Some are purely data reports, while others have linked or embedded images. Still all this information is quite helpful on our current track of streamlining current manual processes toward complete automation. Some reports may bypass InDesign entirely. 

The main challenge I have encountered is instruction and guidance to leverage to utilize InDesign with XML as well as resources for scripting.

My XML research has led me to purchase 2 books, 'XML & InDesign', by Dorothoy J. Hoskins and 'A Designer's Guide to Adobe Indesign and XML', by Jim Maivald. I went through a Lynda course by Maivald that has since been retired. It was very comprehensive. Unfortunately, at the time I took the course, our team was unwilling to develop and employ the XML process for those projects. Hence my question, 'Is XML and InDesign a thing of the past?'.

Scripting in InDesign is another area I have found little information on other than it exists and is useful. It does seem like this will be an essential part creating new processes in InDesign with XML. I have used JavaScript to automate some of our processes in Acrobat. The specifications for InDesign as far as calls and syntax to author my own scripts, I have not found that reference yet. I believe a reference to scripting wiht InDesign exists and can be found. 

Our new projects going forward seem like XML and InDesign could provide a workflow that would be efficient and deliver handsome reporting and products - quickly, effeciently and most importantly, accurate.

Framemaker seems like a great solution to many of our challenges with many different projects.

I am very grateful to all the contributions made on this string.

It has been very enlightening. It has also been encouraging to keep InDesign in the mix. 

Thank you.

[Robert at ID-Tasker]

But do you really need to work with XML - all the time?

Maybe you can export your data in chunks and in much more "simpler" form - like CSV/Tab delimited text - and use DataMerge - either native or 3rd party?

Or custom tagging and post-processing?

Unless you need and can afford InDesign Server...

Scripting specification:

https://www.indesignjs.de/extendscriptAPI/indesign-latest/#about.html

[Bevi Chagnon - PubCom.com]

Before you make your decision to go with Framemaker, note that it does not have design tools and flexibility like InDesign.

It's rare that I have found an InDesign production artist willing to tie both of their hands behind their back and move to Framemaker.  It's a coding tool, not a design tool.

But in the hands of the right person using it on the right job, it's incredible.

So be sure to give it a trial period to decide whether Framemaker is the right tool for your publication.

[James Gifford—NitroPress]

I went from the first wobbly gen of page layout tools to FM, and ended up all but bypassing Quark and the other middle-gen tools. I mostly did long-form work, at which FM excels, of course, but I did some pretty fancy-dancy stuff too. You're correct that it's not nearly as "artistic" a tool as ID, but if you're doing multipage work and need only graceful design and accents, it's more than capable.

That is, it's not just a technical word processor. It can do very elegant, very stylish multi-page, mostly-text stuff, more so than any third tool I can think of.

Or it can be a relentless DITA factory for miles of reference material. 🙂