Format of huffman compressed data in PRC compressed tessellation?

New Here ,
Apr 19, 2012 Apr 19, 2012

Copy link to clipboard

Copied

Hi there,

I've been looking to see if I can make our application create 3D pdf files using the PRC file format. I downloaded the PRC file format document from ISO and tried to read a few example files I found on the web to make sure I understood how it worked. It's all fine until I try to read huffman compressed data in a PRC_TYPE_TESS_3D_COMPRESSED section. The standard says that I should be able to read the huffman compressed data by using the opposite of the 'WriteUnsignedInteger' but if I do that I get invalid integers. If I try to read the unsigned integers by just reading 32bits directly the data doesn't seem to make sense if I try to decode the leaf data for the fuffman tree.

Does anybody know if there is any documentation on this or any examples? I'm at the point of giving up...

I'm a bit bewildered that there is an ISO standard but the example files that I have just don't seem to match it...

Any help would be be fantastic.

Thanks

Miles Thornton

TOPICS
Rich media and 3D

Views

6.0K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
May 18, 2012 May 18, 2012

Copy link to clipboard

Copied

Dear Miles,

 

while I can only share your bewilderment (I have also tried and failed to decode PRC_TYPE_TESS_3D_COMPRESSED),

may I ask you to clarify a bit what "PRC file format document from ISO" did you deal with - the old one avialable for free at

SC2N570-PRC-WD.pdf or the one currently being sold at ISO 14739-1:2014 ?

If you have got your hands on the latter, please, tell us if it is worth its price.

Is it an much improved relative to SC2N570-PRC-WD.pdf?

 

                               Sincerely, Michail

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
May 21, 2012 May 21, 2012

Copy link to clipboard

Copied

Michail,

I actually got the standard from BSi although I assume it is the same as the one at ISO. Here's the link to where I got it from. http://shop.bsigroup.com/en/ProductDetail/?pid=000000000030208200

The short answer is, no it's not worth the price at all. There are still errors in it from what I can see and (very annoyingly) it has 'DRAFT 2011' stamped across each page which makes it virtually impossible to select and copy any text from it using Acrobat.

On a different note I think that I am actually making some progress at reading the compressed tessellation. However I can only spare the odd day here and there to look into this so progress is quite slow...

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Jun 30, 2012 Jun 30, 2012

Copy link to clipboard

Copied

Miles,

the number of pages of the current ISO draft is different from the one at bsigroup, so the text may not be the same.

Your text is called "BS ISO 14739-1", while ISO draft is "ISO/DIS 14739-1.2", that "2" may indicate the next iteration.

On the other hand, it seems a new dratf (1.3) is coming in August:

http://www.techsoft3d.com/developers/article/prc-iso-update

http://www.pdfa.org/2012/06/iso-standards-pdf-week-in-mississauga/

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Aug 15, 2012 Aug 15, 2012

Copy link to clipboard

Copied

Well, there is some progress going on.

ISO draft is not being sold now and there are more details on what took place within ISO at

http://3dpdfconsortium.org/news-and-events/3-the-3d-pdf-standard/37-pdf-week-may-2012-update.html

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Oct 16, 2012 Oct 16, 2012

Copy link to clipboard

Copied

More news - ISO draft is being sold again, it is now "ISO/DIS 14739-1.3" (was "ISO/DIS 14739-1.2")

ISO 14739-1:2014

If there is someone here who has got his hands on the new draft - please, report if it is worth its price.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Dec 22, 2012 Dec 22, 2012

Copy link to clipboard

Copied

More news.

Status of the draft at the ISO page indicates voting takes place on it.

Vote was to be terminated by 2012-12-16.

Anyone in the know of the results here?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Jan 28, 2013 Jan 28, 2013

Copy link to clipboard

Copied

Got my hands on "ISO/DIS 14739-1.3".

Dissapointed. Not that much improvement relative to the free public draft from AIIM.

"Caveat emptor".

The following pseudocode (the same in both drafts),

does not set pcArray[0] to anythimg and uses its junk value in the second "for" cycle.

An evident bug.

I am sure there are more gems like that, hidden in less obvious ways.

I am afraid the text travels thru ISO process without being read.

And Adobe/TechSoft just want an ISO stamp on a for propaganda purposes.

(Anyone remembers Windows NT POSIX compliance joke?)

------------------------------------------

void WriteCompressedIntegerArray(int* piArray, unsigned uIntArraySize,bool bWriteCompressStrategy=true)

{

unsigned u;

char* pcArray=new char[uIntArraySize];

for( u=1; u<uIntArraySize; u++)

pcArray = (char) GetNumberOfBitsUsedToStoreInteger(piArray);

WriteCharacterArray(pcArray,uIntArraySize, 6, bWriteCompressStrategy);

for( u=0;u < uIntArraySize; u++)

WriteIntegerWithVariableBitNumber(piArray, pcArray);

delete [] pcArray;

}

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Jan 29, 2013 Jan 29, 2013

Copy link to clipboard

Copied

On a different note I think that I am actually making some progress at reading the compressed tessellation.

Did you succede?

The current ISO draft has changed the pseudocode description of Huffman compressed data storage, but the new description refers to undocumented subroutines and variables. At least their names seem to imply something.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Feb 02, 2013 Feb 02, 2013

Copy link to clipboard

Copied

OK, on one hand the current ISO draft turned out to be good enough to allow recovery of Huffman encoded data from a PRC file, but on the other I was able to cope with the large amount of errors and omissions in the description of how the compression results are packed into bit fields only due to the fact that both the compression input and output were predictable, so "reverse engineering" was easy.

Since the error ratio is likely to be the same in other parts of the specification  (and not much lower then in the free pre-ISO draft), attempts to reproduce the results of vaguely defined process of highly compressing tessellation and/or to look into other murky areas of PRC format seem risky.

BTW, PDF3D team claims to produce, "independent of Adobe", highly compressed PRC tessellations in PDF output of their products. So reverse-engineering of PRC may be possible, given enough motivation and resourses, but for a casual developer the current ISO draft of the PRC standard is an investment of questionable value.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Oct 28, 2014 Oct 28, 2014

Copy link to clipboard

Copied

Hi,

did anyone find out any more information about decoding the highly compressed tesselation? I am currently working on this as well, and everything seemed fine until I came to the (supposedly) huffman encoded part, where I seem to end up getting a lot of zeroes for my values. Besides, as you guys mentioned before, there seem to be a lot of bugs and omissions in the spec regarding that huffman encoded data. Would really appreciate any hints on how to put together those triangles...

thanks and best regards,

Bjoern

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Oct 28, 2014 Oct 28, 2014

Copy link to clipboard

Copied

Dera Bjoern,

as I wrote before, I was able to uncompress one triangle, but not two.

Huffman compression itself was undone, high compression of triangles - not.

I can share the code, if you wish.

I have stopped trying to understand the charade due to general distaste of the current state of 3D PDF.

On the other hand the fact that it _is_ a charade also tells a lot.

Michail

PS. ISO is about to publish the PRC standard.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Oct 28, 2014 Oct 28, 2014

Copy link to clipboard

Copied

Hi Michail,

thanks for your reply, yeah I was able to get the first triangle too (based on the formula for calculating the first three vertices), but also stuck afterwards, playing with this for ages now, and it's getting really frustrating. Everything else, up to the point of the "highly compressed 3D tesselation" I was able to read just fine (or so it seems).

I really need to get this going at some point, so that I can render PRC 3D data, so any help is welcome. If I find out more I am also willing to share some information (though I cannot directly share the code as it belongs to my client), meanwhile I would be happy to look at your code too, thanks.

How do you know that they are about to publish the standard? And will it be more useful than what is already out there?

best regards,

Bjoern

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Oct 28, 2014 Oct 28, 2014

Copy link to clipboard

Copied

> How do you know that they are about to publish the standard?

The ISO link I already referred to, ISO 14739-1 - Document management -- 3D use of Product Representation Compact (PRC) format -- Part 1..., shows the stage as "60.00 International Standard under publication".

> And will it be more useful than what is already out there?

Out where?

I have no information on what is happening in ISO.

3D PDF Consortium is secretive, Tech Soft 3D and Adobe did not leak anything.

The ISO public draft did not provide "much improvement relative to the free public draft from AIIM".

"I am afraid the text travels thru ISO process without being read."

There were crude typesetting errors in the draft I have got from ISO (tables did not fit into pages).

So I do not place much hope in the final version.

> so I can render PRC 3D data

From what direction comes demand for making your own PRC viewer?

I am looking for answer in most general terms - CAD data interchange, digital archiving or what?

BTW, as far as I know, both Adobe PRC/U3D viewer in Reader/Acrobat and TTF PRC viewer (in 3D Reviewer) do not implement all features of the PRC format? And the implemented subset of PRC is not described in any public docs.

I never thought about reading PRC properly.

My idea was to export PRC for immediate embedding into 3D PDF (code was used in MathGL, Asymptote, forks of rgl R package).

The reader just provided ad-hock text representation of the PRC data.

It was used when the PRC docs were not clear enough and some reverse-engineering of how Adobe PRC writer works was required.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Oct 28, 2014 Oct 28, 2014

Copy link to clipboard

Copied

Hi again, Michail,

and thanks for your quick reply.

By "out there" I was referring to the SC2N570-PRC-WD.pdf document, which I am using currently too.

> So I do not place much hope in the final version.

yeah I kinda do not place much hope either..

> From what direction comes demand for making your own PRC viewer?

Basically providing a library for rendering 3D stuff, to be used by my client's software (which does some PDF rendering etc), I already got U3D implemented (and that was a pain too), now need to do PRC.

> I never thought about reading PRC properly.

I see... well I need to be able to read it, rather than write it, for testing I create some sample projects for writing data (at least partly), but the huffman part is not described very well, and for example completely fails to say where it puts the actual data (seems to encode just the huffman tree). If you happen to have any simple output files in PRC format, that would be very helpful too (like just a handful of triangles, where we know what the positions of the vertices etc should be like), then could use that for "reverse engineering"..

thanks and best regards,

Bjoern

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Participant ,
Oct 28, 2014 Oct 28, 2014

Copy link to clipboard

Copied

Dear Bjoern,

> If you happen to have any simple output files in PRC format,

I used Acrobat 9 Pro Extended to convert simple models from OBJ format to PRC.

Now one can use Tetra 4D converter (trial) for that.

There were also are 2 sample PRC output C++ projects in the Acrobat 9 SDK, one for tessellated data and the other for a non-linear surface.

Michail

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Oct 28, 2014 Oct 28, 2014

Copy link to clipboard

Copied

I see, have to check that out, thanks for the info. If I have any success anytime soon I will let you know.

cheers,

Bjoern

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Dec 15, 2014 Dec 15, 2014

Copy link to clipboard

Copied

Oh I almost forgot to reply to this... a short while after we talked I found how to decode the huffman tree data:

1. read "isCompressed" (bool) [except in the case as mentioned where it is implied whether it is compressed or not]

2. if not compressed, just read the array directly (size, entries..)

3. if compressed you read the "arraySize" of the huffman array (U32)

4. then you read arraySize _uncompressed_ U32s (ie if array size == 4, then you read 4 x 32 = 128 bits)

5. then you read numUsedOnLastU32, ie the number of bits that are actually used in the last U32 that you just read

then we come to the actual decoding of the huffman tree (where you pass in the "numBits" variable, which depends on what you read, for example that's 6 for a typical integer array)

6. those uncompressed U32's that you just read translate into a bit array, from which you read all the huffman tree related stuff

7. read "numberOfLeaves" which is numBits+1 bits long

8. read "codeEntryLength" which is 8 bits long and telling you how many bits to read to find the lenght of the leaf-huffman-code

9. then you iterate through all the nodes, ie for( i = 0; i < numberOfLeaves; ++i ) ...

and for each node you

10. read "node.value" which is numBits long

11. read "node.codeLength" which is codeEntryLength long

12. read "node.huffmanCode" which is node.codeLength bits long

and repeat for all nodes which you can add for example to a std::map, and after iterating through all leaves you

13. read "outArraySize" which is 32 bits long and tells you the size of the resulting character array

14. iterate through the outArraySize'd number of entries of huffman codes to get the actual stored values, ie along the lines of for( i = 0; i < outArraySize; ++i ) { ... read huffman codes ... }

15. from those huffman codes you just read you can recreate the outArray and return the character array, which then for example might be used to read an integer array, or in another case might already be the completed edge status array.

So far for all my example files this seems to work well.

Traversing the mesh, however, is another story. While I got a good number of the meshes correctly, there is still a load that are wrong. While it seems I can finally read through (most of the) files correctly, using up all the bits, the actual reconstruction of the triangles is still problematic. There seem to be a couple of extra conditions not mentioned in the spec which need to be checked, and which I am working on finding currently 😕 Any input on that is of course much appreciated!

Any questions to this, please reply to the post of PM me, I can try to help.

regards,

Bjoern

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Dec 17, 2014 Dec 17, 2014

Copy link to clipboard

Copied

Ironically I just found that that official ISO standard for this has just been released and is available for CHF ~200 (:-O)

While it does confirm what I said above about the huffman tree, in overall I wouldn't call it a whole lot more useful than previous drafts... to me definitely not worth the price. I have only briefly looked at the document so far (only just got it) and compared the sections which gave me trouble when reading PRC data before, and while there is some improvement, I think it will still leave MANY questions to people starting to work with reading PRC files.

if you do work on it, I wish you good luck, and would like to hear if you were successful in reading all the PRC data from the various 3D PDF sample files, or other files that you might have created yourself.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Sep 26, 2017 Sep 26, 2017

Copy link to clipboard

Copied

Hey Bjoern is there any way to get in contact with you, have you any further information in how to traverse the triangles in PRC after decoding the huffman tree. Sometimes there a some constellations in models we can't interpret correctly otherwise we have decoded an rendered almost all of the compressed 3D part.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Oct 30, 2013 Oct 30, 2013

Copy link to clipboard

Copied

Hi,

Are there any news about this topic  ??

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Feb 18, 2020 Feb 18, 2020

Copy link to clipboard

Copied

LATEST

Maybe it's too late for you, but you should take a look at CrossCad/Ware SDK: CrossCad/Ware

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines