Use of SQL Editor

Post a reply

Smilies
:D :) :( :o :-? 8) :lol: :x :P :oops: :cry: :evil: :roll: :wink:

BBCode is ON
[img] is ON
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: Use of SQL Editor

Re: Use of SQL Editor

by sonos » Thu Jan 13, 2022 5:34 am

@ Barry4679 Thanks for your comments (I missed a comment from the MM5 makers)

I did some more tests and found that the Played table does not give useful results. Not only that data and times are not always correct, but also how often the song was played (Played#). There are big inconsistencies. So there are big deviations e.g. Played# = 56 but in the table there are only 29 entries. As a conclusion one must say this table is not usable. So there is still a lot of potential left at least for the "played history".

Now to say something positive:
nevertheless MMW is for me the best music database (according to my limited knowledge). I also like the new concept of MM5. Thank you for the valuable work. I use MMW almost daily.

Re: Use of SQL Editor

by Barry4679 » Wed Jan 12, 2022 9:00 pm

sonos wrote: Fri Jan 07, 2022 7:35 am What is the deeper reason why users can set the Played# date?
I doubt that there is any deep reason, other than it is a non-static non-critical attribute that you you can "correct" if you see the need.

Played Date in MM5 is just an informed guess, because MM cannot see plays that you make with any tool that is not MMW or MMA, and as Ludek has informed us, even MMA is unable to see collect dates from some wretched Apple devices for some reason.

MM doesn't really "collect" play history as such. I see the "data" in the Played table as being some workings that they didn't throw away, just in case they are of use to any Addon developers in the community, rather than any serious attempt to collect all your play history.

Evidence is:
  • the data in the Played table is not visible in MM, nor is it easily exported from MM
  • the database has no referential integrity measures to keep the visible Played stats in sync with the Played table
  • and, and you say, they offer facility to alter the "answer" without affecting the components from which it was built

Re: Use of SQL Editor

by sonos » Fri Jan 07, 2022 7:35 am

What is the deeper reason why users can set the Played# date?

Another oddity is:
That Added date/time after ripping a CD is for the first track 2021/03/02 12:28 and for track 2 2021/03/02 12:29. This seems reasonable.
However according to query (see above) for track 1 2021/03/02 20:37 and for track 2 2021/03/02 17:31.
The date is correct but the times are very different and the order is reversed.
When will the PLAYDATE field be updated?

Re: Use of SQL Editor

by Ludek » Tue Jan 04, 2022 6:44 pm

Hi,
just reading this thread and I can confirm that Songs.playCounter value does not correspond to the sum in the Played table.

The reasons are:
1) Played # is editable field in MM5 so users can set any integer value as they like
2) The play count is synced with devices like iPods where we don't know the exact history, but just how many times the track has been played in iPod from the last sync operation (to increase in MM5)

Re: Use of SQL Editor

by sonos » Tue Jan 04, 2022 4:16 pm

Peke wrote
Hi,
Are you sure that Played # was not increased during Sync as that way it will not showed in played table when using SQL?
I have done a few tests more and do not understand the discrepancies. I have compared the data:
Song added vs. first entry of the query and
Played# count vs listed number of query.
Song added vs. first entry of query and
Played# count vs listed plays of query.

Song ADDED : first date of entry : PLAYED# : listed number of Plays of query
21.09.2011 : 30.10.2011 : 15 : 11
24.04.2011 : 05.08.2012 : 14 : 21
24.01.2020 : 24.01.2020 : 8 : 6
20.04.2021 : 20.04.2021 : 27 : 27
no rule or trend can be observed

SQL query applied:
SELECT Strftime('%Y/%m/%d %H:%M:%S', ( Julianday(playdate) + 2415018.5 ), 'localtime')
AS plays
FROM played AS p
JOIN songs AS s
ON s.id = p.idsong
WHERE album = 'Name of Album'
AND songtitle = Name of song';

Re: Use of SQL Editor

by IanRTaylorUK » Tue Jan 04, 2022 5:31 am

Good discussion. Perhaps we can put together a few additions to Ludek's Custom Nodes add-on?

Under Ludek's track node of the files CustomNodesDefintion.js (found in C:\Program Files(x86)\MMW5 Debug\Scripts\Custom Nodes in my setup) I have previously added some simple SQL queries:

Code: Select all

title: 'Tracks with Artist same as Title',
sql: 'Songs.artist = Songs.SongTitle'
}, {
title: 'Tracks with Album Artist same as Title',
sql: 'Songs.albumArtist = Songs.SongTitle'
}, {
title: 'Tracks with BPM of 1 or Less',
sql: 'Songs.BPM <= 1'
}, {
title: 'Tracks with Bit Rate of 128k or less',
sql: 'Songs.Bitrate <= 128000'
}, {
title: 'Tracks with No Lyrics',
sql: 'Songs.Lyrics = ""'
I think it would be useful to have a few nodes to query when the PlayCount is at odds with the LastPlayedDate. Also when there is disparity between the LastPlayedDate and the Play History. But I think these are more complex SQL queries than I am comfortable with!

Re: Use of SQL Editor

by Barry4679 » Tue Jan 04, 2022 3:16 am

sonos wrote: Mon Jan 03, 2022 12:21 pm After a few tests I sometimes see inconsistencies, e.g.
the main panel for a selected song and album shows: Played# = 15 with Added date = 2012/09/21.
Whereas the SQL query for this song and album shows only Played# = 13 entries and as first played date 2013/08/29.
Obviously some entries have been lost in the last 10 years. :o
Yes, like I said it is a midden pit ... ie, interesting to archaeologists, so that they can estimate historic track plays, but it is a long way short of being an official census.

They collect the data. They don't use it, and they make no attempt to validate nor protect it.

I have relieved MM from the task of collecting my play stats, somewhat because of the above issue, but mostly because it does not know what I play on my Sonos units, and it does not know what I stream when away from the house.

I have an application that collects the real data, and overwrites the play data in the MM database.

I only care about play count and date last played, because I use these as criteria in my playlists.
But I have also store all the individual plays into MM, because I have it, and one day I may get or make something that makes pretty pictures from the history.

Re: Use of SQL Editor

by Peke » Mon Jan 03, 2022 9:08 pm

Hi,
Are you sure that Played # was not increased during Sync as that way it will not showed in played table when using SQL?

Re: Use of SQL Editor

by sonos » Mon Jan 03, 2022 12:21 pm

After a few tests I sometimes see inconsistencies, e.g.
the main panel for a selected song and album shows: Played# = 15 with Added date = 2012/09/21.
Whereas the SQL query for this song and album shows only Played# = 13 entries and as first played date 2013/08/29.
Obviously some entries have been lost in the last 10 years. :o

Re: Use of SQL Editor

by sonos » Mon Jan 03, 2022 10:28 am

Hi Barry4679
This is exactly what I was looking for!

Thanks a lot :D

Re: Use of SQL Editor

by Barry4679 » Sun Jan 02, 2022 9:44 pm

sonos wrote: Sun Jan 02, 2022 1:38 pm I was able to create a table from the DB with all PLAYDATE values applying e.g.
SELECT PLAYDATE FROM Played WHERE IDSong = (SELECT ID FROM Songs WHERE Songtitle = 'The Wedding' AND Album ='Kristallen')
or
SELECT PLAYDATE + 2415018.5 FROM Played WHERE IDSong = (SELECT ID FROM Songs WHERE Songtitle = 'The Wedding' AND Album ='Kristallen')

However I'm struggling to 'feed in' this table into

SELECT strftime('%Y/%m/%d %H:%M:%S',(julianday(PLAYDATE Table) + 2415018.5),'localtime')

to finally create a table with the individual playdate data (in readable form)
I don't understand exactly what you are trying to achieve, ie what do you mean by " create a table from the DB", and also "feed in this table"?

The last sql fragment I posted only showed how to get the last play date for a album's track.
You are after all play dates I think.

To achieve that you need to join the Songs table (to get the track or album that you are interested in), and the Played table (to get all of the play dates).

eg:

Code: Select all

SELECT Strftime('%Y/%m/%d %H:%M:%S', ( Julianday(playdate) + 2415018.5 ),
       'localtime')
       AS plays
FROM   played AS p
       JOIN songs AS s
         ON s.id = p.idsong
WHERE  album COLLATE nocase = 'Citizen of Glass'
       AND songtitle COLLATE nocase = 'Trojan Horses';
or if you are only running this inside the SQL Editor running in the MM5 addon, it is a bit easier:

Code: Select all

SELECT Strftime('%Y/%m/%d %H:%M:%S', ( Julianday(playdate) + 2415018.5 ),
       'localtime')
       AS plays
FROM   played AS p
       JOIN songs AS s
         ON s.id = p.idsong
WHERE  album = 'Citizen of Glass'
       AND songtitle = 'Trojan Horses';
There is a limit to how much automation you can achieve just with SQL.
To achieve automation and operational flexibility you need to call the SQL from inside some programming languauge.

Re: Use of SQL Editor

by sonos » Sun Jan 02, 2022 1:38 pm

Hi Barry4679

Thanks for your reply!

I was able to create a table from the DB with all PLAYDATE values applying e.g.
SELECT PLAYDATE FROM Played WHERE IDSong = (SELECT ID FROM Songs WHERE Songtitle = 'The Wedding' AND Album ='Kristallen')
or
SELECT PLAYDATE + 2415018.5 FROM Played WHERE IDSong = (SELECT ID FROM Songs WHERE Songtitle = 'The Wedding' AND Album ='Kristallen')

However I'm struggling to 'feed in' this table into

SELECT strftime('%Y/%m/%d %H:%M:%S',(julianday(PLAYDATE Table) + 2415018.5),'localtime')

to finally create a table with the individual playdate data (in readable form)

Any hint is gratefully welcome

Re: Use of SQL Editor

by Barry4679 » Sat Jan 01, 2022 10:19 pm

Hi Sonos .. Hi Ian

yes MM database is a a SQLITE database.
and the PLAYED table contains details of every play, but there needs to be some caution taken before relying on it.
MM does not appear to use this data except for in the Statistics report AFAIKS.
It if a bit like a midden pit where they dump their waste and by-products.

The database appears to have no referential integrity measures that ensure the sum of Played plays bears any relation to the counts in the Songs table. The User Community tools that import external plays into MM, or allow play history corrections, typically ignore the Played table.

MM date formats in the database are a bit of a jumble for historic reasons. The date played fields are in Julian Date format.
SQLITE has function that will deal with Julain Dates: https://www.sqlite.org/lang_datefunc.html

eg.
select strftime('%Y/%m/%d %H:%M:%S',(julianday(LastTimePlayed) + 2415018.5),'localtime')
from songs
where album collate nocase = 'Citizen of Glass'

It is easier to read them, than write them.
I add my Sonos plays, and my away-from-home plays from my Last.fm account, into MM, just for the sake of completeness.
So I now have 340,000 records of bloat sitting in my Played table, accumulated over 12+ years ... but maybe you are creating some purpose for them?

Re: Use of SQL Editor

by IanRTaylorUK » Thu Dec 30, 2021 5:36 pm

Hi Sonos,

I think you are on the right track:

viewtopic.php?p=131973#131973

This may also help a little - but does not mention / define UTCOFFSET:

https://www.mediamonkey.com/wiki/Databa ... _%28MM4%29

Re: Use of SQL Editor

by sonos » Thu Dec 30, 2021 12:20 pm

And not to forget UTCOFFSET

Top