MM5 caching

To discuss development of addons / skins / customization of MediaMonkey.

Moderators: jiri, drakinite, Addon Administrators

Barry4679
Posts: 2398
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

MM5 caching

Post by Barry4679 »

MM5 is surprisingly fast considering the technology that it is based upon ... it seems to do a lot of caching.

I don't know if this a problem for anybody, but a F5 refresh no longer seems to trigger a re-read of the MM5 db.

ie. If an external application updates a tag in the MM5 db (eg Date last Played), then MM5 does not see the change until MM5 is stopped and restarted. .. ie. F5 does not work, neither does navigate elsewhere, & then return, nor does close tab and then open new tab & re-navigate? ... These things did work in MM4.

Is this WAD? ... I am asking mostly out of curiosity. It is suboptimal from my POV, but OTOH it is good that MM5 is not slow..

One other question about caching. The File|ManageDatabase|Rebuild option is a welcome new addition to MM5. It keeps track IDs (thankfully), but assigns new ID to other entities like Album< artist, etc.

How does this impact cached art for album, artist, etc .... Are the caches invalidated? Or is the connection maintained?
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
drakinite
Posts: 965
Joined: Tue May 12, 2020 10:06 am
Contact:

Re: MM5 caching

Post by drakinite »

I can't give any answers to your other questions, but as for album art caching, there is currently no caching done for art thumbnails. However, I'm hoping we can change that soon, as the act of forcing album art to reload from disk causes performance hitches in the grid views.
Image
Student electrical-computer engineer, web programmer, part-time MediaMonkey developer, full-time MediaMonkey enthusiast
I uploaded many addons to MM's addon page, but not all of those were created by me. "By drakinite, Submitted by drakinite" means I made it on my own time. "By Ventis Media, Inc., Submitted by drakinite" means it may have been made by me or another MediaMonkey developer, so instead of crediting/thanking me, please thank the team. You can still ask me for support on any of our addons.
rusty
Posts: 8393
Joined: Tue Apr 29, 2003 3:39 am
Location: Montreal, Canada

Re: MM5 caching

Post by rusty »

Hi Barry,

Re. MM failing to update the tracklist in response to an externally updated tag, are you referring to a case in which the file monitor is set to continuously monitor a directory and it fails to 'notice' the change?

If so, I'm able to replicate that. Tracked at https://www.ventismedia.com/mantis/view.php?id=17782

The workaround is to manually initiate a scan of the directory, that causes the change to be recognized by MM and it's reflected in the tracklist right away. My suspicion is that you've configured MM to scan your directory continuously and at startup, and that the continuous scan is failing while the startup scan isn't. Please confirm.

Thanks!

-Rusty
Barry4679
Posts: 2398
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: MM5 caching

Post by Barry4679 »

rusty wrote: Sat Apr 24, 2021 9:11 pm Re. MM failing to update the tracklist in response to an externally updated tag, are you referring to a case in which the file monitor is set to continuously monitor a directory and it fails to 'notice' the change?
No, not what I meant.
In MM4 I could update the MM database, with tag changes, using an external application, while MM4 was open and idle.
When I pressed refresh in MM4 it noticed the database change.

The same thing in MM5 only notices the change if I stop and then restart MM5. .. ie. MM5 does not re-read the database for a F5 refresh ... nor when a tab is closed and reopened ... nor when i navigate to a different collection, and then return to the collection that was open and idle when the database was changed.

An example is:
  • MM5 is idle, and open at EntireLibrary node in List By Album mode, and is sorted by Last Played column
  • an external program updates Last Played column for some new track plays
  • go to MM5 and press F5, the newly played tracks do not raise to the top ... filter the display to one the albums which have just been updated .. tracks do not show the new play date
  • close & reopen MM5, and everything is now updated
I have 47,000+ tracks, so I don't suppose that it is a memory cache .. ? You have a shadow database?


I am not saying this is a problem, because this practice is not recommended given that Sqlite is not multi-process DBMS. So I put this thread in the Developers' forum as a heads up, because this is a change from MM4 behaviour.



drakinite wrote: Sat Apr 24, 2021 9:59 am I can't give any answers to your other questions, but as for album art caching, there is currently no caching done for art thumbnails.
Yes, I should not have asked about the Album art ... Now that i think of it, this is a special case ... but what about the artist & composer art that has been downloaded and cached by MM5? ... Is the art still connected to correct composers and artists?
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
Ludek
Posts: 4945
Joined: Fri Mar 09, 2007 9:00 am

Re: MM5 caching

Post by Ludek »

Album art is cached, see the /temp/thumbs/ folder where individual album arts are cached to various thumbs variants (mostly 200px/200px for grids).

As for the caching the whole lists. Yes, there is also caching. Every track picked from MM5.DB has only one instance in the memory shared for various lilsts, so until all lists (and references to the track) are destroyed then the track is kept in memory and only edits/changes are written back to MM5.DB
Note though that we cache data sources for last three views for 1 minute and always cache data source for the last used view (i.e. the tracklist in case of List view) .
drakinite
Posts: 965
Joined: Tue May 12, 2020 10:06 am
Contact:

Re: MM5 caching

Post by drakinite »

Ludek wrote: Sun Apr 25, 2021 3:06 pm Album art is cached, see the /temp/thumbs/ folder where individual album arts are cached to various thumbs variants (mostly 200px/200px for grids).
Sorry, you're right about that. There are so many different levels to cache - I was referring to a memory cache; I should have specified as such. It's definitely true that the thumbnails are cached on disk after they're generated.
Image
Student electrical-computer engineer, web programmer, part-time MediaMonkey developer, full-time MediaMonkey enthusiast
I uploaded many addons to MM's addon page, but not all of those were created by me. "By drakinite, Submitted by drakinite" means I made it on my own time. "By Ventis Media, Inc., Submitted by drakinite" means it may have been made by me or another MediaMonkey developer, so instead of crediting/thanking me, please thank the team. You can still ask me for support on any of our addons.
Barry4679
Posts: 2398
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: MM5 caching

Post by Barry4679 »

Barry4679 wrote: Sun Apr 25, 2021 8:28 am I have 47,000+ tracks, so I don't suppose that it is a memory cache .. ? You have a shadow database?
After some more thought I suppose what I am seeing is due to a change in MM's SQL transaction management, which has affected isolation.
SQL documentation: wrote:"An example: Suppose there are two database connections X and Y. X starts a read transaction using BEGIN followed by one or more SELECT statements. Then Y comes along and runs an UPDATE statement to modify the database. X can subsequently do a SELECT against the records that Y modified but X will see the older unmodified entries because Y's changes are all invisible to X while X is holding a read transaction. If X wants to see the changes that Y made, then X must ends its read transaction and start a new one (by running COMMIT followed by another BEGIN.)"
If this is the cause, would you consider starting a new transaction during a F5 refresh?
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
Post Reply