Structured FM: Slow for really long documents with many elements?

[nils-wkg]

Hi folks,

well, I have a few little questions and would be glad to hear what the experts out there have to say about 'em. 😉

Basically, we are producing huge telephone directories using FrameMaker (all tests in this message have been done with a demo version of 2015 and 2017). Think about several hundred pages, each having two or three columns and very small text, plus a lot of graphics. Traditionally, we've been using an application written by us that creates a MIF file, which we then open in FrameMaker, save as a native MF file, and using pre-created master pages, various custom api clients / extensions, as well as manual formatting, thus finally making it look like a real phone book.

Now, we've decided to have a look at structured FrameMaker in order to decide if that would offer a worthwhile route to switch to. After all, the data we work with is highly structured, and we even already have native XML capabilities in our database, so we thought: Let's see if instead of going the MIF route, we could also get our data into FM by means of XML.

As it turned out, that did work. But ... well ... it worked very slowly! And the main question is, basically, if this is expected and the route via XML / structure is probably not the right way for us to look into, or if we are doing something wrong.

To be more precise, one has to take into account that the XML we feed into FrameMaker consists of very many elements with very little content. So, we don't have a <paragraph> construct that contains several thousand characters, but instead what we have looks more like:

<person>

     <first_name>Jeff</first_name>

     <initial>P.</initial>

     <last_name>Test</last_name>

     <address>

          <street>Test Street</street>

          <houseno>10</street>

          <zip>12345</zip>

          <city>Test City</city>

     </address>

</person>

Now picture about 80000 of such "person" entries, plus some additional stuff, in a 40 MB big schema-based XML file. In order to open the XML, FrameMaker needs 30 - 45 seconds. The same data, without structure in a MIF file we've generated, opens in less than ten seconds.

There's even more trouble: First, the good news is that once the XML data has been saved as a native structured FM file, that FM file can be opened much more quickly, almost instantly! But when we import the formatting information from a different document into that structured FM file, well, it's wait time for about 30 - 45 minutes. In contrast, importing the same formatting information into an unstructured FM document that "started its life" via our traditional MIF route only takes a few seconds.

Of course, such big wait times are not really what we'd want. It's entirely possible that the structured route is not the right one in our case anyway - in fact, we don't need structure per se and we just thought we'd experiment with structured FM / XML a bit. It's entirely possible that we will continue our traditional MIF route, which has worked well for us for years. But I'd still like to hear what the pros think: Is such a behavior concerning the delays / slowness expected with XML data of the kind we are using? Or should things in theory work much faster and we are probably only doing something wrong?

Thanks in advance and greetings to you all!

Nils

[Arnis_Gubins]

Hi Nils,

This is basic database publishing. I've been using FrameMaker with Miramo for years to create directories (1000+ pages), catalogs, etc. There's no real need to get into structured FM for this. Have a look at the Miramo Personal Edition (currently a freebie) at www.miramo.com ​

From my experience and your description of this directory, it shouldn't take Miramo+FM more than several minutes per production run to create a press-ready PDF from an XML file once you've configured the application for your needs. You can use either XSLT to wrap your XML in the MiramoXML or use the mmpp (macro pre-processor) application to wrap the XML (in a more conventional programming sense). Alternatively, you can use any sort of "tagging" (not necessarily strict XML) from database output and then run a Miramo pre-processing app to  wrap the data for publishing.

[Russ Ward]

Hi Nils,

I am one of the biggest structured FM fans on this block. However, I think that maybe your assessment is correct, that XML is not the best format for this.

XML import is slow... that's just the way it is. I imagine the delay is caused by the need to apply the EDD formatting rules element-by-element, which has to be an expensive process. This would be especially true for an EDD with lots of context rules. Perhaps Abode could fine-tune the process, but I would not wait around for that and it may not be feasible anyway.

I imagine that MIFs and binary FM files import more quickly because the formatting information is explicitly present element-by-element, paragraph-by-paragraph. It's when you need to reapply the rules, element-by-element, that the delay occurs. Incidentally, that must be what is happening when you import formats too.

The great advantages of structured FM are the superior authoring environment, automated formatting, and content reuse. It seems that none of these really apply in your case. If you have a MIF process that works, I can't think of any great reason that XML would be superior.

Russ