Thank you for your donation!


Cloudsmith graciously provides open-source package management and distribution for our project.


Thumbnail generator question
#1
Hi @Tim Curtis,

I've 2740 albums on a NAS. When I regenerate the thumbnails, moodeutl says:
20250504 140948 thumb-gen: Done: 2770 folders scanned, 2740 thumbs created, 30 already in cache.

I've no artwork in my flac files. I've written some software, making sure the tags are consistent. One of those checks is about embedded artwork.

Every album has a folder.jpg in the album folder, totalling 2740 folder.jpg files on my nas.

So, it looks all OK on the NAS.

I've the same behavior on my 2 moOde instances (9.3.3).

I've no idea where the 2770 folders are coming from as it should be 2740, and how it is related to the "30 already in cache"?

Any guidance?

Thanks


Attached Files Thumbnail(s)
   
Reply
#2
Do you have any artist covers? They would be in the folder upstream from the album folder. Also check for hidden files.

The input list for the thumbnail generator is output from

Code:
/var/www/util/list-songfiles.sh
Enjoy the Music!
moodeaudio.org | Mastodon Feed | GitHub
Reply
#3
No, I don't have any artist covers.

The NAS is only used for music. I removed 2 albums since my post, but the behavior is the same:
Done: 2768 folders scanned, 2738 thumbs created, 30 already in cache.

I just rechecked some things.
total files on the nas: 32872
jpg: 2753 (=2738 album jpgs +15 playlist jpgs)
flac: 30099
dsf: 5
m3u: 15 (15 playlists)

list-songfiles.sh says 30104 wich is correct (30099 flacs + 5 dsf)

In /var/local/www/imagesw/thmcache I've 5476 files. This devided by 2 gives 2738, which is indeed the number of albums I've.

So, everthing looks ok, but why does moOde says 2768 albums and 30 already in cache? A DB issue?
Reply
#4
Hi @Tim Curtis,

Found my issue! 2 similar albums: One called "BenWebster's finest Hour", the other one "Ben Webster's finest Hour".

I looked-up the code that does the thumb generation. Studied the code and edited /var/www/util/thumb-gen.php to log the file and folder when it encounters a duplicate. ;-)

Not sure how I should have figured out which albums it concerned, if I couldn't modify the source code.

Fixed!


Code:
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/02. Ben Webster - Hotter than 'Ell.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/02. Ben Webster - Hotter than 'Ell.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/03. Ben Webster - The Ghost of Dinah.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/03. Ben Webster - The Ghost of Dinah.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/04. Ben Webster - Cotton Tail.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/04. Ben Webster - Cotton Tail.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/05. Ben Webster - Honey Hill.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/05. Ben Webster - Honey Hill.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/06. Ben Webster - Linger Awhile.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/06. Ben Webster - Linger Awhile.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/07. Ben Webster - Star Dust.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/07. Ben Webster - Star Dust.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/08. Ben Webster - Do Nothin' Till You Hear from Me.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/08. Ben Webster - Do Nothin' Till You Hear from Me.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/09. Ben Webster - My Greatest Mistake.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/09. Ben Webster - My Greatest Mistake.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/10. Ben Webster - La Rosita.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/10. Ben Webster - La Rosita.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/11. Ben Webster - Sunday.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/11. Ben Webster - Sunday.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/12. Ben Webster - While We're Dancing.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/12. Ben Webster - While We're Dancing.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/13. Ben Webster - Blues for Mr. Broadway.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/13. Ben Webster - Blues for Mr. Broadway.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/14. Ben Webster - Chelsea Bridge.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/14. Ben Webster - Chelsea Bridge.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/15. Ben Webster - Roses of Picardy.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/15. Ben Webster - Roses of Picardy.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/16. Ben Webster - Single Petal of a Rose.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/16. Ben Webster - Single Petal of a Rose.flac
Reply
#5
(05-04-2025, 05:09 PM)ils1974 Wrote: No, I don't have any artist covers.

The NAS is only used for music. I removed 2 albums since my post, but the behavior is the same:
Done: 2768 folders scanned, 2738 thumbs created, 30 already in cache.

I just rechecked some things.
total files on the nas: 32872
jpg: 2753 (=2738 album jpgs +15 playlist jpgs)
flac: 30099
dsf: 5
m3u: 15 (15 playlists)

list-songfiles.sh says 30104 wich is correct (30099 flacs + 5 dsf)

In /var/local/www/imagesw/thmcache I've 5476 files. This devided by 2 gives 2738, which is indeed the number of albums I've.

So, everthing looks ok, but why does moOde says 2768 albums and 30 already in cache? A DB issue?

I assume everything is ok in the WebUI Library views?

IIRC there can be some discrepancies in the numbers reported in the moode log but I'd have to walk through the code to  resurrect that scenario.

Whats interesting is that the "REGENERATE Music library" option should delete /var/local/www/imagesw/thmcache/* before starting afresh.

Maybe try ls -l on that directory and look at the timestamps?

If I end up doing that I'll post something.
Enjoy the Music!
moodeaudio.org | Mastodon Feed | GitHub
Reply
#6
(05-04-2025, 06:41 PM)ils1974 Wrote: Hi @Tim Curtis,

Found my issue! 2 similar albums: One called "BenWebster's finest Hour", the other one "Ben Webster's finest Hour".

I looked-up the code that does the thumb generation. Studied the code and edited /var/www/util/thumb-gen.php to log the file and folder when it encounters a duplicate. ;-)

Not sure how I should have figured out which albums it concerned, if I couldn't modify the source code.

Fixed!


Code:
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/02. Ben Webster - Hotter than 'Ell.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/02. Ben Webster - Hotter than 'Ell.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/03. Ben Webster - The Ghost of Dinah.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/03. Ben Webster - The Ghost of Dinah.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/04. Ben Webster - Cotton Tail.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/04. Ben Webster - Cotton Tail.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/05. Ben Webster - Honey Hill.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/05. Ben Webster - Honey Hill.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/06. Ben Webster - Linger Awhile.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/06. Ben Webster - Linger Awhile.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/07. Ben Webster - Star Dust.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/07. Ben Webster - Star Dust.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/08. Ben Webster - Do Nothin' Till You Hear from Me.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/08. Ben Webster - Do Nothin' Till You Hear from Me.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/09. Ben Webster - My Greatest Mistake.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/09. Ben Webster - My Greatest Mistake.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/10. Ben Webster - La Rosita.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/10. Ben Webster - La Rosita.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/11. Ben Webster - Sunday.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/11. Ben Webster - Sunday.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/12. Ben Webster - While We're Dancing.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/12. Ben Webster - While We're Dancing.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/13. Ben Webster - Blues for Mr. Broadway.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/13. Ben Webster - Blues for Mr. Broadway.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/14. Ben Webster - Chelsea Bridge.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/14. Ben Webster - Chelsea Bridge.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/15. Ben Webster - Roses of Picardy.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/15. Ben Webster - Roses of Picardy.flac
20250504 203243 Dup: NAS/Music/Ben Webster/BenWebster's finest Hour file: NAS/Music/Ben Webster/BenWebster's finest Hour/16. Ben Webster - Single Petal of a Rose.flac
20250504 203243 Dup: NAS/Music/Ben Webster/Ben Webster's Finest Hour file: NAS/Music/Ben Webster/Ben Webster's Finest Hour/16. Ben Webster - Single Petal of a Rose.flac

Post the code and as long as you grant an open source license for me to use it I'll work it into the thumbnail generator. The logging could help in troubleshooting :-)
Enjoy the Music!
moodeaudio.org | Mastodon Feed | GitHub
Reply
#7
In /var/www/util/thumb-gen.php I added the 2 lines in red.

Simple reasoning was the following: If you regenerate the thumbs, you cannot have some already in cache. If moOde lists a number already in cache, let's detail which ones, as those may cause the issue.


--------------------------------------------------

while ($line) {
    $fileA = explode(': ', $line, 2)[1];
    $dirA = dirname($fileA);

    $line = strtok("\n");

    $fileB = explode(': ', $line, 2)[1];
    $dirB = dirname($fileB);
    if ($dirA != $dirB) {
        phpSession('open');
        $_SESSION['thmcache_status'] = 'Scanning folder ' . ++$folderCnt . ' ' . $dirA;
        phpSession('close');

        if (!file_exists(THMCACHE_DIR . md5($dirA) . '.jpg')) {
            createThumb($fileA, $dirA, $searchPri, $thmW, $thmQ);
        } else {
            $msg = 'Dup: ' . $dirA . ' - file: ' . $fileA;
            workerLog($msg);
            $cachedThms++;
        }
    }
}
Reply
#8
Just be careful, as that code is the only one that gets executed; whether you UPDATE the thumbnail cache, or you REGENERATE it.
Since the album cover is generated on a directory basis, the code just checks that while scanning the files, if "we already got one file parsed and hence a thumbnail generated for this folder, then skip all other files here...".
This works well for single audio-files + track-index file, ala FLAC + CUE, for example, and on a REGENERATE thumbnail cache scenario.
It will spit a ton of "Dup: blablabla" while simply updating the library...

maybe there are some thumbnails, generated who-knows-how, that cannot be deleted; hence moOde finds them still there after the cleaning of the thumbnail directory... just suppositions, that is...
Reply
#9
Agree. It only allows to detect anomalies in case of a full regen. After I found my issue, I removed the code.
Reply


Forum Jump: