[PROBLEM] Update library

[TheOldPresbyope]

@alexarz

There is nothing special about a USB drive from moOde's point of view.

Whether a SD/uSD card, a USB drive, or a PCIe SSD drive, they each contain one or more partitions. Each partition contains a file system. 

Once mounted in Raspberry Pi OS (which takes care of details like accessing the drives and dealing with different types of file systems), all those individual file systems are integrated into one unified file system which is rooted at "/".

For convenience, the mount points used in moOde are given names signicant to humans, like /mnt/OSDISK, but those names don't convey any meaning to RPiOS or moOde. Since this is a Linux file system, there can a question of the ownership and access rights for the directories and files, but usually it can be answered by a simple examination (using ls -lR, for example).

I've never had any difficulty mounting mechanical or solid-state USB drives to my moOde players. Yes, USB 2.0 throughput is somewhat slow; pure USB 3.0 throughput is significantly faster. However, neither causes slowdowns like what you report.

I still believe the crux of your problem lies in the nature of the files you have on those drives.

Pointing to "me too" posts by other users isn't helpful if they don't provide information which can lead to a diagnosis.

I won't have more to say unless and until I find some minimum reproducible example that generates the few symptoms which have been reported.

Regards,
Kent

PS - the translation software being used these days is really quite good.

[philrandal]

** post deleted **

[philrandal]

I find this frustrating. I moved home in March, so haven't had much chance to explore. I have a weird library update issue too on my 9.3.7 moode instance. Not sure when the issue started.

Just plugged in my other moode box which runs 9.2.4 and it updates the library perfectly.

Will do an in place update on that box and see what happens.

Still works after update. Investigating further.

Wondering if I have SD Card corruption on my original moode instance.

Edit: it helps to have a 3A PSU attached to a pi with a USB disk...

Fingers crossed that my issue is resolved.

[alexarz]

@TheOldPresbyope
You still haven't written whether you tried updating the folder twice in a row when you have a disk USB 3.0 with 1000-1500 albums connected?

[TheOldPresbyope]

I don't have such a test rig.

However, I do have a NAS share mounted now. It contains 500+ albums. No errors were generated while adding it to moOde's library after my test described above. 

Now, when I update just the OSDISK folder, I see an unexpected slight time delay before I can update it again.

Aha.

With debug logging enabled in moOde, I see the following snippet:

Code:

20250627 150109 mpdindex: Cmd (update "OSDISK")
20250627 150109 worker: Update thumbnail cache
20250627 150109 thumb-gen: Start
20250627 150109 thumb-gen: Scan opt: Default
20250627 150109 thumb-gen: Priority: Embedded cover
20250627 150109 thumb-gen: Res,Qual: 600px,60
20250627 150109 thumb-gen: Px ratio: 1
20250627 150109 thumb-gen: Th width: 600
20250627 150109 thumb-gen: Thm qual: 60
20250627 150109 thumb-gen: Scanning: NAS, NVME, OSDISK, SATA, SDCARD
20250627 150112 thumb-gen: Done: 545 folders scanned, 0 thumbs created, 545 already in cache.
20250627 150112 mpdindex: Done: indexed 769 artists, 546 albums, 6079 songs
20250627 150112 worker: Job update_library done
20250627 150122 DEBUG: genLibrary(): jsonErrorMessage(): No errors

MPD is scanning only the OSDISK folder but thumb-gen is scanning everything. In your case, that would include your USB drives. We already know from a previous post of yours that there are some dodgy files on them which cause problems for thumb-gen.

Regards,
Kent

[Nutul]

1. your drives might be awfully slow
2. your album-art pictures might be awfully big
but...
why would you perform a library update very few seconds? Just do what is logically useful:
1. add files / folders to your library
2. do ONE update library thing.
3. WAIT.

Every library uopdate DOES scan your drives, and fires ALSO a thumbnail scan... it's not that trivial, so to speak.

[alexarz]

...
MPD is scanning only the OSDISK folder but thumb-gen is scanning everything. 
...

That's the whole problem. Why scan everything if we specified a selected folder?

[TheOldPresbyope]

...
MPD is scanning only the OSDISK folder but thumb-gen is scanning everything. 
...

That's the whole problem. Why scan everything if we specified a selected folder?

No, that's not the whole problem for the rest of us, so it's not high on the priority list.

In the meantime, you're free to work up modifications to the appropriate routines in moOde, demonstrate they work, and submit a pull request to the repo, just like the rest of us are.

I'm done here.

Regards,
Kent

[Tim Curtis]

...
MPD is scanning only the OSDISK folder but thumb-gen is scanning everything. 
...

That's the whole problem. Why scan everything if we specified a selected folder?

The thumbnail generator was not designed to operate on an individual directory because after the initial generation of thumbnail images it only spends time on new albums and so is very fast. It runs in the background and doesn't block anything in the WebUI.

From the old thread https://moodeaudio.org/forum/showthread....0#pid64520 your stats from a complete library regen showed that it took about 10 minutes for thumb-gen to create 2320 images. If you were to do a Library update instead of a regen the thumbnail generator would prolly take < 1 minute to complete.

Some info that might be helpful is to see the process flow for "Library update"

NOTE: After "Update library" is submitted another one cannot be submitted until PART 3 below is complete.

PART 1 (Menu > Update library or Update this folder)
- Display busy spinner
- Set GLOBAL.libLoading = true
- MPD database update -------+ These run in parallel. Either one may finish first.
- Thumbnail cache update ----+
- MPD database update done: Clear busy spinner, start PART 2
Note: thumb-get may still be running but thats ok

PART 2
- Read MPD database and...
- Generate library tag cache json file (for Tag/Album views)
- File generated, start PART 3

PART 3
- Sort/group/filter tag cache json
- Render tag cache into Tag/Album view
- Set GLOBAL.libLoading = false

The bottleneck on your end is probably in PART 2 and/or PART 3 but its not clear whats causing the bottleneck other than possibly a lot of malformed music files, maybe causing a crash.

It sounds like you are only interested in Folder view to navigate your collection and so the extra PART 2 and PART 3 processing for Tag/Album views is not really needed for your usage scenario. Is that correct?

There are other users that only use Folder view because of the structure of their collections so maybe its worth exploring an option to "Enable only Folder view" which would avoid the processing for Tag/Album view.

[alexarz]

...
There are other users that only use Folder view because of the structure of their collections so maybe its worth exploring an option to "Enable only Folder view" which would avoid the processing for Tag/Album view.
...

Yes, it would be very nice to have such an opportunity.