P: Video Cache is out of control

[jamesr33423836]

I recently upgraded to Lightroom 6.2.1 So far there has been a lot of fuss over the import dialog... ok, well sure, it has a few issues, but they can be either worked around or just revert back to 6.1.1

I would like to report and ACTUAL bug in 6.2.1

My D drive recently ran out of space... I tried deleting some stuff I didn't need, but still it kept running out of space... So I finally ran windirstat and had a look at it.. Nearly HALF of my drive was FULL of D:/temp/Adobe Local/Lightroom/caches/video/media Cache Files. Now here's the kicker... NONE of these video files are even on the hard drive that has my catalog on it. My lightroom database is on drive L:\ all my lightroom photos and very few lightroom videos are all on L:\ I have a few scans on drive K:\ that I import to lightroom... but these videos are ALL on Drives E:\ and F:\

here's another thing.. I knew Adobe wasted space caching videos, and I do have a GOPRO and I record a little video,.. .VERY LITTLE, maybe 1 video a year... so I don't really want ANY of the video cached.. I will wait for it to load... SO a long time ago, I set my video cache in Preferences>File Handling to the minimum of 1GB (It's was always a BUG to not allow 0GB) I don't want to cache any videos!!!!!!!!!! but here it is.. NEW BUG It's STILL SET TO 1GB!!!!!!!!!!! How is it gobbling up almost 1TB of space when I have it set to 1GB??????????????? So.... here's another problem... I have Cache files that are 4GB for a movie that only takes up 2GB on my hard drive.. what's up with THAT??????????????? I randomly checked a few, and the worst one was a 5GB cache file for a movie that only takes up 750MB on my hard drive... two things, first of all... you NEVER EVER EVER need to cache an entire movie... caching more than 1 minute of a video is useless... if you're going to watch the video in lightroom, it can just play the video on the hard drive fast enough that it doesn't need further caching.. second... DON'T CACHE THE VIDEO AT ALL!!!! just capture the thumbnail of it so you don't have a blank square on the catalog and call it a day... if someone wants to play the video, it will load and play plenty fast enough, Lightroom Can't work with video files, only play them, so WHY BOTHER TO CACHE THE WHOLE THING?? come on Adobe, some COMMON SENSE please!!!!

so now ANOTHER BUG, it's already set to limit video cache to 1GB, so I figure, the new import thing must have built this RIDICULOUS Cache, so I will just Purge it, that will bring it down to 1GB right??/ WRONG!!! It pops up a message saying "Video cache is being purged, this message will be dismissed when the purge is finished... I wait 5 Seconds, and the message disappears, I have NO hard drive activity, and well... I STILL have 1GB of Video Cache files!!!!!!!!!!! Ok, maybe it didn't take.. let me set it to 2GB, then purge.. NOPE! let me set it back to 1GB then purge, NOPE..

ok, so I suspect how to fix it will be, (I Hope) click import, and select the movies folder and then select Ignore source... by the way, how do I add my E:\scans folder as a legitimate source.. I just removed a source, how do I add a nice shortcut button for a new one? ok, now that source is removed, I hope it won't scan it again... now that it will HOPEFULLY not scan that hard drive again, I'll manually delete the offending Video cache.. ok manual delete complete, Ahh my drive can breathe again.. Lets open light room and see what happens.... ok.. open.. lets open the Import dialog because eventually I'll need to import something... oh-o..... What's this??? Scanning Common Locations... OH-NO!!!!!!!!!! first of all E:\Moves and F:\TV are NOT COMMON LOCATIONS FOR PHOTOS Second, I REMOVED THOSE LOCATIONS.. It has a valid location listed, WHY IS IS LOOKING FOR SOMEWHERE ELSE????? Yes it's now scanning VIDEOS and the only reason for it to be taking so long is it went back to my E and F drive... lets look at the cache folder... oh yes, MediaCache already has 3,876 files in it.... not looking very good. Why is it caching files that are not even imported into lightroom??? and why is the cache for each file taking up more space than the entire video??? If you want to make a video cache.. just cache the THUMBNAIL ONLY!!!!! we don't need or want anything else cached.. a thumbnail is all we need cached for our few relevant videos so we don't have a black square in our catalog.

Crap, I'm going to be FORCED to revert to 6.1.1 even though I had every intention of working around the other 6.2.1 issues. TEST TEST TEST Test your software!!! the monkeys you have testing it are not doing a good enough job! send it to me, I'll test it for you, It needs to be tested on a real computer, one that is used for other things.. that way you would KNOW it's finding stupid things like CD cover art and every movie and tv show on my hard drive. I seem to find glaringly obvious issues within one day of using the product. I've been developing software for the last 28 years, I know how it should be done, and how things should be tested, and this is NOT being done AT ALL. Your programmers are sub-standard and are missing the mark, and your non-existent SQA department is NOT testing even the simplest of functions.

[TomFidd]

Evidently more than 2. Foolish on Adobe's part - when an application has hidden perfomance drags, users will be dissatisfied without knowing exactly why, and they'll quit using it without giving a clear signal as to the cause. Just look around photo forums - what's the one thing LR users complain about (aside from having to actually pay for something)? Performance!

[jasonm72359636]

Is this ever going to get fixed?!?

[jasonm72359636]

For everyone who struggles with this bug, you can use this as a workaround:

https://www.howtogeek.com/201930/how-to-automatically-delete-files-in-your-download-folder-on-a-sche...

Just change the download folder to the video cache file folder.

[RzzB]

Many thanks for this - very helpful for all sorts of things.
One question - what happens if this runs when Lightroom is open and using this cache?

[RzzB]

I just spent ages recovering from a full system C: SSD drive. Turned out to be this problem.

In preferences, Lightroom Cache is set to N: - and video cache is set to 3G.

Video cache on C: = over 60G 😞

Two years! Is there any development work happening on Lightroom, other than new lenses and mobile stuff - in which I have no interest?

Things seem to be at a standstill - and I'm paying out monthly.

Should I be looking elsewhere for my photo editing? 

Not impressed, not impressed at all. 

[TomFidd]

There is a new version, "Classic," that improves performance in a few areas, particularly previews and the grid view.

I'm not sure if it changes anything about the Dynamic Link Server behavior though. My guess is not.

[TomFidd]

Classic is much improved in some areas, particularly thumbnail and preview generation.

Video is, if anything, worse. It's status quo for the dynamiclinkmediaserver - it still seems to respect the new cache limit, at least between sessions. But performance loading large videos is poor and I've seen a few glitches, like failing to read .mov files until I restarted LR. Also, video display still consumes massive amounts of memory.

My guess is that the overall strategy is still just completely mismatched for the application. Pre-Classic, cached video files were uncompressed, making them ridiculously large on disk and in memory, and that doesn't seem to have changed. That might be useful for a video editor, but it's useless for LR. It would be far better to create a lo-res cache file, perhaps using just keyframes, with full audio but minimal video content.

[dkfidler]

Here we go again...

Affected Lightroom Version: 7.0.1  (LR Classic)
Computer:  Surface Pro 3
Computer Disks:

• C: 256GB SSD; system disk
• D: 128GB Micro SD card in the built-in SD card reader; disk containing videos organized in folders by genre

LR Video Cache settings:

• Limit video cache:  Checked
• Maximize Size:  3GB

Issue Description:

Copies of videos from D: drive are added to the LR video cache directory when a user tries to import images.  This inevitably ends up filling up the system volume (C:) requiring manual intervention to delete the cached files, as the video cache size limit is not honoured.

Video Cache Folder: C:\Users\***** \AppData\Local\Adobe\Lightroom\Caches\Video\Media Cache Files

Steps to Reproduce:

1. Per above configuration, copy video files onto Micro SD card and put into card reader
2. Create directory c:\windows\temp\_deleteme_
3. Copy some large files (in excess of 5 GB) into c:\windows\temp\_deleteme_
4. Open Lightroom CC Classic (7.0.1)
5. Click Import
6. Windows Explorer: Open video cache directory; files are being created, including copies of the videos themselves (as CFA files - some of which are several gigs large)
7. CMD Prompt:  tasklist | find "dynamiclinkmedia" ; this shows that dynamiclinkmediaserver.exe is running.  
8. Click Cancel
9. Click File / Exit  (close LR)
10. Windows Explorer:  Video cache flies are still being created and all of the videos are being copied into the cache
11. Task Manager: Sort processes by Disk and see that dynamiclinkmediaserver.exe is doing 100MB/s or more of disk activity - WTF?  LR is closed...
12. Wait until disk fills up; sometimes, windows will tell you that your disk is full
13. Monitor disk space; it drops down to about 128KB free, and then it appears to delete a file from the video cache to make more space for the next file.  Therefore, the process is thrasing writes to my SSD disk (thereby reducing the MTBF of it, effectively costing me money!)
14. Delete the files from c:\windows\temp\_deleteme_; dynamiclinkmediaserver.exe consumes all remaining disk space with more cached files
15. Monitor dynamiclinkmediaserver.exe until it dies (and the disk stops thrashing).  In my last test, when it was finally done, I was left with 31.2 MB free on C:
16. Start Lightroom; wait 5 minutes - the video cache is eventually cleared (unless there is a hung dynamiclinkmediaserver.exe process, which does appear to happen sometimes).

Issue Details:

There are so many different things going wrong here it's amazing.  They are as follows:

1. LR starts caching any files that are on the SD card immediately, even though I have not browsed into any directory that holds a video (they are all in subfolders).  I'm presuming that this is because D: is in a card reader and, for convenience purposes, card reader/external disks are displayed first because that's how many people get photos into their systems.
2. dynamicmedialinkserver.exe does not honour the "LR Video Cache Size Limit" of 3.0 GB that I've set in Lightroom.  I don't care that LR deletes the cache the next time that it starts (provided that dynamiclinkmediaserver.exe has not hung)
3. Adobe software is scanning and making copies of my files WITHOUT MY PERMISSION.  It should not be copying the videos.  I can understand LR wanting to make thumbnails of the videos - but I DO NOT GRANT ADOBE SOFTWARE PERMISSION TO PROACTIVELY SCAN MY DISKS FOR FILES AND MAKE COPIES OF THE FILES THAT IT FINDS
4. Adobe software is actively decreasing the MTBF of my SSD disks because it is making unnecessary writes to my SSD.  I am not alone here - using SSD's for system volumes is now a ubiquitous configuration and so Adobe's software is actively causing unnecessary wear-and-tear on it's customer's hardware.  This happens in two ways:  1)  By filling up the disk, Adobe software is performing at least one write to every remaining sector of the disk.  2) By filling up the disk, it reduces the pool of sectors that other programs can write to, thereby more rapidly decreasing their MTBF
5. Adobe software is thrashing both my CPU and my disks with unnecessary writes, causing slow performance impacts to other software on my computer.  Adobe software is, effectively, performing a denial of service attack on my computer.  This happens even AFTER I have closed Ligthroom!!!!!
6. With legislation increasingly focusing on data protection and end user privacy, having programs that indiscriminately scan users disks for files is going to be an increasingly dangerous legal position to be in - not to mention that it is clearly antagonizing your user base.
7. !!!!!ALL OF THESE THINGS HAPPEN EVERY TIME I USE LIGHTROOM!!!!!

Bugs like this have presented themselves in both Ligthroom and Premiere, going back as far 2014 (and possibly earlier).  The first post, alone, in this thread is over two years old. Come on guys, this has gone on long enough.

To date, I have not seen a single good explanation, from Adobe, as to why your software is performing a denial of service attack on my computer and why your software is actively reducing the effective lifetime of my hardware.

Adobe has clearly made a bad architectural decision and has implemented a very destructive process.  I can forgive bad architectural decisions (being an architect, my bad decisions have cost companies millions of $ over the years).  But this one has gone on for long enough.

!!!!!!!!!!!!!! PLEASE STOP CACHING VIDEO FILES !!!!!!!!!!!!!!
!!!!!!!!!!!!!! PLEASE STOP CACHING VIDEO FILES !!!!!!!!!!!!!!
!!!!!!!!!!!!!! PLEASE STOP CACHING VIDEO FILES !!!!!!!!!!!!!!
!!!!!!!!!!!!!! PLEASE STOP CACHING VIDEO FILES !!!!!!!!!!!!!!

Quite frankly, if someone were to contact me saying that a class action lawsuit against Adobe were underway seeking damages for this bug, I would add my name to the list in a heartbeat.

[dkfidler]

If it runs while dynamiclinkmediaserver.exe is encoding some of the files, the deletes of the files will fail.  This will only work if dynamiclinkmediaserver.exe has already completed copying files into the cache already.

[urixl]

sigh
I delete Dynamic Link Manager every time I upgrade or install Lightroom.
That's the only workaround here.

[dkfidler]

Yep, I've read that in former posts. 

I don't know how much you know about the software industry so forgive me if I'm covering old ground here.  The job of any support person is to work on a problem until there is a workaround and then file a bug.  In the bug, they list the workaround and Product Management/Engineering assess each bug and assign it a priority.  Bugs that have "workarounds" are given the lowest priority because... well... there is a workaround.

This isn't a minor annoyance with an easy workaround. 

This is a bug that will impact a high percentage of LR users (potentially millions of users), and which causes significant, and unnecessary, wear and tear to their hardware.  And many of those people won't ever know it.  

A headline like that could be very damaging to Adobe.

That is the context that Product Management/Engineering need to see this bug under - not that it's just some annoyance with a workaround.

[urixl]

Unfortunately, it's not a bug, it's an (unnesessary) feature.

Nobody in sane mind won't edit video in Lightroom, but The Marketing told that video import will boost sales.

And here we are.

[dkfidler]

Jury - This isn't about importing videos - this is about LR scanning your entire disk and making copies of each video it finds, then deleting them - completely ignoring the cache max size.  

This is not a feature. This is a bug.

I understand your frustration but sarcasm sends the wrong message to product management. 

This is a bug.  

Say it with me...

This is not a feature. This is a bug.

[TomFidd]

Whether it's a bug or a feature, it's bad for users. But they probably get relatively few complaints because it happens behind the scenes.

For some of us, video preview is helpful because there are frequently video and stills of the same event. That makes disabling dynamiclinkmediamanager a very unnatractive workaround.

The real insanity of it is that there's no reason to cache video at all - you only need that for editing. VLC and 1000 other programs can play video right off the source file. Why can't Lightroom? LR's playback performance is worse than most, probably due to the cache lag.

[TomFidd]

What they should really do is:
- create a still thumbnail for each video in the catalog (already happens)
- ditch the dynamiclinkmediamanager entirely
- open videos for playback in the system default or user-designated external player
If it takes more than a day to code and test that, something's wrong.

[TomFidd]

Incidentally, this thread is marked "in progress," so I think this is ticketed as a bug somewhere.

[FeedbackCommunityMember]

same problem with version 7.1

[nils42197491]

I've encountered a huge issue with the Lightroom cache today. I was sorting through my personal movie library on a network drive when my Mac freaked out about having no disk space left. When I looked into what was filling up my previously half empty drive, I saw that the Lightroom Video Cache took up 160GB (!!!). I didn't even open Lightroom today (!!!)



As if that wasn't enough, most of the original Files aren't even as big as the corresponding cache files!
Why is Lightroom doing that?! I now terminated every Adobe service in the background out of fear of it destroying my SSD! This ist absolutely unacceptable! Lightroom is on the Brink of being unusable with this issue!

[TangCanada]

You find it curious.... I find it ridiculous.

[FeedbackCommunityMember]

I'm running LR version 8.2.1 which I think just updated to that version over a week ago.  I got hit by the Video Cache problem as 30GB of video on my data drive got cached to the OS drive despite me not doing anything.  I never had this issue with previous versions of LR (I go all the way back to LR 3).  It's clear that the bug has not been fixed.  Fortunately, I have a 500GB OS drive running Win10 so I didn't run out of drive space. 

[TomFidd]

Don't hold your breath waiting for a fix. This ticket has been open for years, and the problem is replicable and understood AFAIK. Clearly Adobe thinks the cache size limit is a good enough solution (it's not).

[Rikk Flohr: Photography]

Lightroom Desktop should have some major improvements in this area with the 2.3 release. 

[TomFidd]

Excellent!

I assume you mean Lightroom Classic & 8.3 release?