P: support X-Trans properly

[pinkypunk35]

This is getting crazy now - Lightroom is the only RAW editor that still messes up Fuji X-Trans files. Why?

We have smaller, less finaced businesses and even individuals producing RAW convertors that can do this, why is Adobe struggling so badly?

Iridient Developer

Photo Ninja

LightZone

Capture One

SilkyPix

Raw Therapee

Aperture

All of these produce much better results and leave Lightroom looking very under par.

I can't see any reason for this. I have invested a lot of money in Lightroom (and the Creative Suite set of Adobe tools) over my entire professional life, and I did this becasue I came to expect Adobe to be at the forefront of developing up to date tools with innovative features and supporting the latest hardware. But sadly, this seems to no longer be the case and they are left looking third rate compare to far smaller developers.

[Todd Shaner]

ssprengel wrote:

The X-Trans sensor has a different color layout than most other sensors.

It could be that Adobe does understand their demosaicking is less than others’ but they cannot base their methods on others’ due to licensing agreements and have not discovered their own method in the amount of time the bureaucracy has allowed them to work on it so far.

It's more likely due to the pending Fuji X-Trans patent application:

http://www.fujirumors.com/fuji-patent-x-trans-evolution-sensor-with-enlarged-green-pixels/

I couldn't find any announcement concerning the patent being awarded yet to Fuji. Any information Fuji releases to other non-Fuji corporate entities (i.e. Adobe) concerning X-Trans design details (including demosaicing) jeopardizes approval of this patent application. Of course we know there's nothing prohibiting reverse-engineering of X-Trans demosaicing, which even DCRAW now supports quite well. I opened an RAF file with RawDigger, did a PNG screen capture, processed it with LR5.71, and the image detail is pretty darn good compared to the RAF inside LR. Both have the same LR Sharpening settings applied. BTW- I don't own an X-Trans camera, don't intend on purchasing one anytime soon, and have no axe to grind either way.

[ssprengel]

The 6x6 pixel layout from Fuji has been around for several years and isn’t what is documented in the patent application do I doubt Adobe’s lack of support is due to any future sensor capabilities.

You can save a TIF out of RawDigger. Your screen capture is much smaller resolution than the original RAF.

[Todd Shaner]

ssprengel wrote:

The 6x6 pixel layout from Fuji has been around for several years and isn’t what is documented in the patent application do I doubt Adobe’s lack of support is due to any future sensor capabilities. 

Then perhaps we will see a better X-Trans implementation in LR6.

ssprengel wrote:

You can save a TIF out of RawDigger.  Your screen capture is much smaller resolution than the original RAF.

I only have the RawDigger Exposure edition with no Tiff Export capability. The screen capture is a 100% view "crop" and 100% view LR Compare with monitor profile conversion to sRGB.

[ssprengel]

Ok, I understand the limitations and the difference in overall size. 

Here is a side-by-side that illustrates the difference of Adobe vs a free Raw Converter, RawTherapee.  Notice how smudged the leaves against the house are at the righthand side in Lightroom version:

The downside of the more detailed treatment is the false color edges which you can see around the curved antenna and along the top shingles on the roof in the RawTherapee version that is not in the Adobe version:

I don't really know how to use RawTherapee so maybe I could get rid of those false color edges, and if I open the RawTherapee output in ACR, I can remove a lot of the problem by using a little Color NR:

If you'd like to see the full size JPGs, here they are along with the original RAF:

https://www.dropbox.com/sh/v6c9fdq6xl32cz1/AAD2QET6jWDnpnoEg3i77J0ra

[panta]

Iridient v3 beta 5 ....No sharp

Dropbox - Iridientv3.jpg

vs

Last camera raw...No sharp

Dropbox - Adobe.jpg

Adobe please resolve this issue

[tschroder1]

As a Lightroom lover and Fuji X-Trans fan, I am disappointed that this issue has yet to be resolved. I am not a fan of the painterly look that LR applies to my Fuji RAW files. I have no plans to switch to a CC subscription without satisfactory support for my beloved Fuji camera.

[Cambora]

Same here, Adobe needs to fix this in LR6. I'm a paying CC customer but if this isn't resolved soon I'm leaving for Iridient or CaptureOne.

[dougryan]

I'm totally with you guys on the issue at hand. It's one thing to be an Adobe apologist, but it's something all together different to be a pompous ass about whether or not the issue is validated. I'm speaking about you, Keith.. obviously. You have zero interest in whether or not support improves for X-Trans sensors, so why not just take your nose out of the bowl? All you're doing is being combative in a place where it's not warranted or wanted. Yeah, it's the internet, you can say what you want... but you're really just being an ass, and if you didn't know it, I'm being honest and telling you so.

That said, many photographers *myself included* are ditching huge rigs for mirrorless, on photoshoots. The appeal for street shooting is obvious, and even in a controlled environment, such as a studio, it's also a benefit to use lighter/smaller gear. The results need not be inferior when comparing against DSLR gear, if the lighting is done well, and if the output processing is on point. To wit: LR is absolutely the cause of needing to switch workflows. The X T1 is one heck of a capable camera for portraits, amongst other things. The answer is not to switch back to a D800/600 and carry tons of heavy glass, but to spend far less, on other RAW processors until Adobe stops being Adobe about things, and gets their asses in gear. And since the SDK is now out, I'd expect that Adobe would want to at least quell the yammering on its forums from us folk. I'd be happy enough with them fixing it for that reason alone.

Doug

[Keith Reeder]

Ah - I see. Because I'm not lining up to give you a hug and blindly agree that Adobe are Bad People, putting you Fuji-using victims through such trauma, you're (to use a splendidly appropriate Americanism) butthurt, and I'm pompous.

If that's what it takes for you to sleep at night - although you don't seem to know the difference between "pompous" with "cynical", which last I CERTAINLY am as far as this thread is concerned.

The fact remains: you (in particular, despite your indignant, inflated-sense-of-entitlement yammering) have not made the case that Lr is flawed here. Agreeing with the the bleating on the thread doesn't make you right, doesn't make me (or more importantly, those who have provided examples or cited pictorial evidence from "out there" to demonstrate that the problem isn't nearly the problem you insist it is) wrong.

You niche cameras users sure seem to love playing the victim, I've noticed. I know damn' well that if I was as convinced as you (note - in the absence of any evidence from you to rule you out as the source of your problems - just lots of self-righteous pontification) that Lr was broken, I'd have kicked it into touch ages ago: Capture One Pro and Photo Ninja apparently do a fantastic job with Fuji files out of the box, so why are you still even here?

Lucky you're not a DxO Optics Pro user. They don't support your sensors at all.

But you lot love your hair shirts, don't you? I'll bet you call your snapshots "art", too.

Here's the thing: I have no horse in this race, I'm looking completely objectively at the available evidence (and for the avoidance of any doubt here, yes, I'm saying you're not). And it does not prove your case.

It might prove that some of you need to learn how to drive Lr properly, but that's about all...

Think of me as the Adobe engineer you need to convince that urgent and significant work still needs to be done.

If you haven't convinced me, why expect Adobe to agree with you? They'll be looking at much the same evidence that the rest of us is seeing.

Yours pompously.

[pinkypunk35]

You're not pompous Keith, you're just a troll.

You have no horse in this race, yet constantly reply here provoking those who do.

So what if you can't see any evidence? You don't own a Fuji camera so that opinion is no surprise, you need to own a Fuji to qualify for an opinion at the very least -  in fact you probably don't even own any camera. It seems the vast majority of your time is spent provoking people you don't know about a subject you don't understand.

Internet Trolls Are Narcissists, Psychopaths, and Sadists | Psychology Today

[dougryan]

Ah - I see. Because I'm not lining up to give you a hug and blindly agree that Adobe are Bad People, putting you Fuji-using victims through such trauma, you're (to use a splendidly appropriate Americanism) butthurt, and I'm pompous.

Not only are you a pompous ass, but you are also a self defeating, immature person. The level of sarcasm you toss around in lieu of adult conversation is disrespectful to anyone attempting such a thing. And please don't take my calling you out on your nonsense, as me being butthurt. You're not important to me, only to yourself. But it's cute you'd think so I guess. BTW, no one said that the people at Adobe are "bad" people. And really, there's no trauma. Maybe a bit of drama.. But that's barely the same thing.

"The fact remains: you (in particular, despite your indignant, inflated-sense-of-entitlement yammering) have not made the case that Lr is flawed here."

Me in particular eh? Because what, I made such a compelling argument, that I now qualify as someone to put on your troll attack roster? I'm honestly laughing at your lack of good judgment.  And, do you know what the definition of indignant is? Im not angry, nor do I feel that anyone is being treated unfairly by Adobe. I'm concerned, and that's what I sound like, versus being angry. I think you're  deflecting, personally speaking.

"Agreeing with the the bleating on the thread doesn't make you right"

Of course it doesn't. That's not even a qualifier for what would be right or wrong. It's called sympathizing via shared experience. Which, by the way, as has already been pointed out several times, you're not capable of doing since you have zero experience with X Trans files. And yes, it makes a difference given the context. Comparing any other RAW file type is redundant.

"doesn't make me (or more importantly, those who have provided examples or cited pictorial evidence from "out there" to demonstrate that the problem isn't nearly the problem you insist it is) wrong."

Well, actually..... It kind of does! Specifically, if we're looking at this from a black and white POV it does. Pictorial evidence from "out there". Show it to us. You at least owe us that much, through all of your accusations.

"You niche cameras users sure seem to love playing the victim, I've noticed."

You're just all sorts of sunshine, ain't ya?  lol. It's okay, I've already gone and assumed that your a tea bagger, which would explain your posturing.  For the record. I'm also a Nikon user. I have very little to gain by lying about the results I physically see. And very little to lose by voicing my observations on an official Adobe forum. One might even say that this is how progress may be achieved. But as far as what you're doing? It's really a mystery. Unless you really are a shill, trained to try and derail the topic into such an utter mess, that it's no longer a sustainable conversation. You must be a real party favorite! lol!

"just lots of self-righteous pontification" there's that deflection again... I simply stated what I know to be true. You simply spouted totally un-educated opinion.

"that Lr was broken, I'd have kicked it into touch ages ago: Capture One Pro and Photo Ninja apparently do a fantastic job with Fuji files out of the box, so why are you still even here?"

Silly rabbit. Trix are for kids! No one, not one person is claiming that LR is "broken". You really do love mincing words..  You must have been terrible at those books where you fill in the blanks with verbs, adverbs, adjectives and nouns etc... You've also got your assumption engines on full throttle. 

LR is lovely with my other RAW files. It's even great with my scanned film. You know, that other niche thing... But here's the thing, Skippy... And this too has been pointed out ( and yet you insist on ignoring the FACT), that Adobe fully admitted to not having things right with X Trans.

Funny thing is, it's also the fault of Fuji, for not having given Adobe the proper SDK tools in advance. So I'm not even fully blaming Adobe. I just found it interesting that a one man show, can get the algorithms right, and Adobe not so much. I'm even sure they'll get it right, eventually.

The question really is, what are YOU still doing here? Oh, right... No one puts baby in a corner.

Take care, baby.  Oh, and have fun trolling!

[dougryan]

Alright, so that little distraction aside...

I just downloaded and am trying out something called RAWTherapee. It's open source, and available for Mac, Win and Linux. Thankfully, I have always used a folder hierarchy system for my photographs. I've got a few different cameras, including film, and so it's always been important that a file system for them was established, even outside of database type programs. It's also how I work with LR, since I only keep my working files on external drives. Anyway, RT (therapee) basically sees those folders and allows you to start from which ever one you want upon its boot. The interface is actually pretty intuitive, but is also a bit on the loaded side, meaning that there are more things to play with and adjust than I personally am used to seeing or using. It's really fully featured.  As far as the file browsing goes, Adobe Bridge has a slight advantage semantically, given its ability to use filters and collections. This could be advantageous if looking to do away with LR for an all in one.

I also downloaded Iridient Developer, and am waiting for an email from Photo Ninja to get a free trial. I've got Iridient open next to LR right now, on the same dng file (I had imported to LR as dng at one point because of the lack of support for said RAF files) and I'm seeing some interesting things going on. I decided to nix the sharpening and noise reduction for both ID and LR, and ID still has a very clear advantage. I also set LR's color profile for the Provia Standard, so at least it's not going for the Adobe profile, which should be closer to what Iridient attempts.

A few of you were asking for a RAW file, well... here you go, three for ya. Have fun. I took these off the coast of Montezuma, Costa Rica btw. https://copy.com/vaLUCWBEwG8HOPo7

LR renders the foliage sloppily, with no definition and mushes everything into its self. And it really doesn't matter if I increase the sharpness (I always increase the masking, but in this case I even turned it off in order to get as much in range as possible) and even turning up the clarity to about 15-16 doesn't do it justice. It simply continues to keep it mushy. Even with ID's sharpness turned off, it looks better than LR with sharpness on.

With the portrait, you can absolutely see superior resolution near the lip area, and skin flaws in general. Pushing clarity or any type of sharpness with LR only exacerbates the underlying issues, as far as I can tell. Please prove me wrong. I'd be thrilled to have LR start up with a preset configuration which renders everything the way ID does. Oh, and yeah.. I won't even bother moaning about the decimated color and skin tones.. lol

The third one is indoors, and as you guessed it, not my wife lol. The lighting wasn't that great, lots of florescent bulbs, and nothing else. But this might be a better file to work on, as far as what I can give away, since it's not work and just a random snap shot.

I'll say this, however.. This phenomenon is not noticeable with every type of shot. In my case, it's a lot less obvious when dealing with portraits, especially on the skin of a baby. LR seems to do pretty well for me there. Still, the differences are there.

I'm curious to try Photo Ninja now, and see how it holds up next to ID.

Doug

[Just Shoot Me]

When you use the Adobe DNG converter to convert RAF files to DNG files you are applying what Adobe thinks the file should look like.

So you should do all testing with REAL RAF files.

[john beardsworth]

Shootistbond007 wrote:

When you use the Adobe DNG converter to convert RAF files to DNG files you are applying what Adobe thinks the file should look like.

It's nothing to do with what Adobe thinks the file should look like.

In LR the RAF and the DNG are identical. In other apps any differences arise because of the way the other app interprets DNGs - Capture One, for example, fails to treat a DNG's raw data as coming from a specific camera and fails to apply its C1 goodness.

What would be even more useful, Doug, is if you could post the best settings you've achieved (eg embedded in the DNGs) and something a bit more specific than mushy - eg screenshots of problem areas.

[pinkypunk35]

Even better would be if one of the Adobe apologists processes the Raw and provides the settings and a jpeg,because there is no shortage of people demonstrating multiple problems but not a single solution from the many apologists.

[Keith Reeder]

That's because nobody's "apologising" for Adobe - we're saying (and have been saying, very clearly, for ages) that you still haven't objectively defined or proved the "problem".

That's why John suggests that complaints about something being "mushy" aren't moving the discussion on - it's a subjective, immotive, meaningless term, which can be said about many of the contributions from "people demonstrating multiple problems".

And I'll say this again: Lr's lack of micro-detail isn't unique to Fuji files - I see it in my Canon conversions too, and it's a fundamental of Lr's demosaicing algorithm, not a Fuji-specific failing.

That's why some of us use several converters...

pinkypunk35 wrote:

but not a single solution from the many apologists.

Now that's not true, is it?

I've personally posted several links to to websites demonstrating (what I would call) "solutions" to (what you would call) "problems"...

[pinkypunk35]

If your not articulate enough to understand what the word "mushy" may mean when spoken in regards to photographic detail there is no hope of you comprehending what's being discussed, be we knew that already didn't we.

There are plenty of visuals backing up written descriptions from those having problems but not a single person has provided a solution despite many claims.

Your previous "solution" was debunked a long time ago.

Suffice to say there is a huge problem at Adobe a d they cannot match the other raw developers.

[john beardsworth]

pinkypunk35 wrote:

Even better would be if one of the Adobe apologists processes the Raw and provides the settings and a jpeg,because there is no shortage of people demonstrating multiple problems but not a single solution from the many apologists.

Please cease the personal insults.

Your most recent example looks just like operator error. Maybe it isn't, but one can't tell for sure as you fail to post the problem raw file and your best settings.

[dougryan]

Shootistbond007    "When you use the Adobe DNG converter to convert RAF files to DNG files you are applying what Adobe thinks the file should look like.

So you should do all testing with REAL RAF files."

lol. Come on now.... dng files were created by Adobe to be a universal standard. They are not interpreted to be anything other than RAW data. No conversion is actually done. Whether or not a DAM (data asset manager) is able to read a dng, has everything to do with whether or not the developer wrote the extension in to it, and nothing else. It's not an algorithmic issue.

john beardsworth‌ "What would be even more useful, Doug, is if you could post the best settings you've achieved (eg embedded in the DNGs) and something a bit more specific than mushy - eg screenshots of problem areas."

Are you asking me to post a screenshot of two DNG's, or to cook them up as jpg's?  As far as the "mushiness" goes, it's an inherent issue with the way that Lr's demosaicing (sp?) algorithm handles large quantities of specific colors, and that's honestly the best way I know how to describe it. If you look at the opposing shot done by Iridient, you'll see that it's not a matter of fine micro contrast etc.  I can play with Lr all day and never achieve what ID does from the get-go, no matter what settings I use. And thus, I've given you gentleman the ability to prove this to be wrong, so please humor me and do just that. Also, others have pointed out an actual loss of information due to Lr decimating detail, and you'll also see that example on my wife's face, as well as with the guy's face on the third.

@keith_Reeder   "Lr's lack of micro-detail isn't unique to Fuji files - I see it in my Canon conversions too, and it's a fundamental of Lr's demosaicing algorithm, not a Fuji-specific failing."

This isn't exactly true for me with my Nikon DSLR's, nor either my Nikon film scans or Canon film scans.  I've never seen this phenomenon until looking at my RAF files. But I should make clear, it's true that Lr does indeed exhibit lower levels of micro contrast from RAW files of most any type, and that this is never a problem after post processing. But this is NOT the actual issue being brought to light with how RAF files are being treated. Scrutinize the files I linked to.

Okay, you guys want more. Fine. I've got plenty of files which exhibit this proof. Objective if I ever did see an example of, proof. But alas, I've got family stuff to do today.. so this will have to wait until I get home later.

Doug

[DdeGannes]

I did a small test on one of your files and attach a screen capture comparing Qimage Ultimate (which uses DC Raw a program written by David Coffin and is utilized by many raw conversion software packages) and Lightroom 5. I have used the basics from both programs with some moderate sharpening. I have two comments concerning the file in question. (a.) The lens used is a manual lens so that the aperture used is not in the info. (b.) The background are hills and trees are not in the best focus for optimizing detail.

Left is the Qimage rendition. Just my 2c.

[Todd Shaner]

DdeGannes wrote:

I did a small test on one of your files and attach a screen capture comparing Qimage Ultimate (which uses DC Raw a program written by David Coffin and is utilized by many raw conversion software packages) and Lightroom 5. I have used the basics from both programs with some moderate sharpening.

I ran a similar test with RawDigger, which also uses DCRAW for raw conversion:

Re: Will Adobe ever support X-Trans properly? Everyone else does...

The results are notably better with DCRAW over LR5.7.1. But DCRAW does introduce color artifacts around specific high-contrast edges, so it's not perfect either.

RawDigger (DCRAW) 200% View                                    

LR5.7.1 200% View

[ssprengel]

The deficiency in LR's processing is obvious if you look at the foreground trees at the bottom right.  There is an ugly gray halo around each leaf and if you look at the rain streaks they are finer on the non-LR version:

Original raw file and various conversions available at:

https://www.dropbox.com/home/XTransAdobeVsOther

[DdeGannes]

@trshaner the point I was trying to get across with my post, was that the background area is out of focus and I am not about to try and use sharpening tools to extract detail from that area.

In essence what we are trying to test here is the equipment used to capture the raw data, sensor, camera and one of the most important components the optics.

I would be interested to see one of the Fuji users, post the results achievable with the software supplied by the Camera Manufacturer for comparison.

It is my belief that software applications for the processing of raw file data from digital cameras, are still in a period of rapid development and not yet mature and what we are seeing that improvements from different developers take place every few months. Adobe is at the year 2012 now hopefully we will see improvement in 2015.

[Todd Shaner]

DdeGannes wrote:

@trshaner the point I was trying to get across with my post, was that the background area is out of focus and I am not about to try and use sharpening tools to extract detail from that area.

Agreed, the point I was making is that when evaluating the performance of a raw converter check image areas with 1) subtle tonal detail and 2) high contrast areas with fine detail.

The DSCF0296.RAF - Box file posted here Re: Will Adobe ever support X-Trans properly? Everyone else does... is a good test image. It has green foliage and red flowers with moderate tonal detail; and high-contrast areas with fine detail (roof, antenna, etc.). When testing your raw converter with this image file makes sure to also check the roof and antenna as shown in my post #162 here for color artifacts. A well performing X-Trans raw converter will faithfully render both image types.

[dougryan]

DdeGannes‌: *

"the point I was trying to get across with my post, was that the background area is out of focus and I am not about to try and use sharpening tools to extract detail from that area.

In essence what we are trying to test here is the equipment used to capture the raw data, sensor, camera and one of the most important components the optics"

It almost sounds like you're trying to gather an objective conclusion, but at the same time, I feel that you're either making way too many excuses so that you don't admit what you are actually seeing, or that you don't quite understand what the actual issue/problem is. Yes, the lens was an Nikkor AIS manual focus lens. But if you're into using logic at all, you'd notice that the entire frame is in focus, and that is due to having used an aperture of 5.6 or above. And really, the background area is not OOF enough to render this test useless. It isn't sharpness we are looking at. It's not even micro contrast. Both those things are easily fixed in post if not in the original render, as has been stated over and over and over. As trshaner‌ has shown in the box link, it's in how the algorithm renders specific patterns or colors. And this really has zero to do with sharpness in general.

Now, one good point you bring up is the hardware. Specifically, the optics. But not for testing critical sharpness, but rather how it renders fine lines and tonal contrast. I'm going to link more photos and start with a test portrait that I took at B&H with the 56 f1.2   I asked my wife to come over and look at both the LR and ID 1:1 zooms, and asked what differences she saw, without coaching. She knows nothing about photography, and doesn't care to know. She stated the obvious, that the ID image was sharper, smoother and showed more detail. Then, I asked her to take a closer look, and ignore exposure, tone and sharpness and see if there was anything with one of them, that she didn't see on the other. Her inexperience had her scratching her head for a minute, until she finally saw it. And you'll see it in just 5 seconds. Her layman description was: "I see brackets!"  I'm like.. Uuuh.. Ookay? And then she started pointing and dragging her finger along the artifacts, confirming what I was hoping she'd see.

The ID version is smooth like butter, and yet sharper with more detail. No artifacts as those shown in the LR version. The "brackets" are most prominent when looking at the curve of the nostril moving down towards the clef on the upper lip. I'd be most happy to post more images.