Last.fm Playcount Import (2010-03-28) [MM3/4]

Download and get help for different MediaMonkey for Windows 4 Addons.

Moderators: Peke, Gurus

Funkafonik
Posts: 58
Joined: Fri Feb 13, 2009 12:33 pm

Re: Last.fm Playcount Import [MM3]

Post by Funkafonik »

zombiefly wrote:hi,
The way i see it, If i run this once, retrieve ALL the history, apply it, then run this for the last 10 pages only automatically without prompts, it should keep my latest plays up to date, which in turn I can use to exclude latest plays from the playcount, the options are endless for the autoplaylist now.

this wouldnt happen before i modified the script. Was it working for anyone else?
I'm curious to see what Psyker says when he returns from his travels :)
Totally agree with this!

First, version 2 using the weekly static charts is flawed as these charts don't update for any modification done to the library, and they are cut short if you listen to lots of different tracks in a week...

Version 3 using the Recently Played/History is the way to go, as we get our complete Last.FM stats, and pin point accuracy for the Last Played date (which needs to be fixed as it updates to the FIRST played date right now)
This script would work best like explained by zombiefly, 1st run downloads the complete history, and after that, only the most recent plays.

A very nice addition to this script would be to also update MM's internal plays, since we now have access to them all in our Recently Played/History!
zombiefly
Posts: 190
Joined: Fri Jun 08, 2007 3:33 pm

Re: Last.fm Playcount Import [MM3]

Post by zombiefly »

i've spent some further time on this and identified a large number of tracks that can't be found in the database. It appears this is due to capitalization, as the script gets a list of tracks by each artist (which appears to have capitalization) then tries to compare this with a lowercased version of the track it's dealing with, which fails.
I've been able to replicate this behaviour and was just at the point of implementing a fix when I decided I should go to bed!... now i'm in work, so will have a quick look at lunchtime. I hope to have this fixed entirely this evening (UK)

All that remains then is some way of controlling the first run of the script, i.e. the full scrape vs daily updates. Unless of course you don't mind editing values directly in the script when you want to do a full refresh.
MM5 latest [Gold] / MMA latest | OnePlus 8 PRO | Noise I make: Pushkin on Soundcloud
Image
zombiefly
Posts: 190
Joined: Fri Jun 08, 2007 3:33 pm

Re: Last.fm Playcount Import [MM3]

Post by zombiefly »

Funkafonik wrote:
zombiefly wrote:hi,
Version 3 using the Recently Played/History is the way to go, as we get our complete Last.FM stats, and pin point accuracy for the Last Played date (which needs to be fixed as it updates to the FIRST played date right now)
What do you mean by this? I'm ordering my "recent played tracks" playlist by last played and they seem to tally with last.fm?
Funkafonik wrote:
zombiefly wrote: A very nice addition to this script would be to also update MM's internal plays, since we now have access to them all in our Recently Played/History!
again can you expand?

It's worth noting that I can fully change anything now i'm familiar with the scripts workings. I'm a bit hesitant about posting up a modified version without permission from the author though!
MM5 latest [Gold] / MMA latest | OnePlus 8 PRO | Noise I make: Pushkin on Soundcloud
Image
Funkafonik
Posts: 58
Joined: Fri Feb 13, 2009 12:33 pm

Re: Last.fm Playcount Import [MM3]

Post by Funkafonik »

zombiefly wrote:
Funkafonik wrote:
zombiefly wrote:hi,
Version 3 using the Recently Played/History is the way to go, as we get our complete Last.FM stats, and pin point accuracy for the Last Played date (which needs to be fixed as it updates to the FIRST played date right now)
What do you mean by this? I'm ordering my "recent played tracks" playlist by last played and they seem to tally with last.fm?

Funkafonik wrote:
zombiefly wrote: A very nice addition to this script would be to also update MM's internal plays, since we now have access to them all in our Recently Played/History!
again can you expand?

It's worth noting that I can fully change anything now i'm familiar with the scripts workings. I'm a bit hesitant about posting up a modified version without permission from the author though!
Hmm that's strange because I've run the V3 numerous time with my MM library being empty (playcounts & last played) and the dates in my Last Played were all updated to 2004-2005, when I started using Last.FM... instead of having the proper 2008-09-10.
So for the past few months, I've had to first run V2 to get the "more accurate" Last Played dates, and then run V3 to get the right Playcounts.

About MM's internal plays, since we now have access via LastFM History to all our plays with a datestamp for each, a script could update all these individual plays for each songs, not only the total playcount and Last Played date. That way we could use AutoPlaylists to do pretty much anything. (like find songs played on 24/11/2006 for example)
zombiefly
Posts: 190
Joined: Fri Jun 08, 2007 3:33 pm

Re: Last.fm Playcount Import [MM3]

Post by zombiefly »

Funkafonik wrote: About MM's internal plays, since we now have access via LastFM History to all our plays with a datestamp for each, a script could update all these individual plays for each songs, not only the total playcount and Last Played date. That way we could use AutoPlaylists to do pretty much anything. (like find songs played on 24/11/2006 for example)
I swear this is happening already! (unless i'm going mad)

while i was testing last night i did the following:

1. played a track on my pc. checked that it appeared in the "recent played tracks" playlist. Confirmed it was there played Today 19:45
2. checked last.fm, the same track had been scrobbled Today, 19:45
3. played a different track on my phone
4. checked last.fm, the phone track had been scrobbled Today, 19:47
5. Ran my modified script, told it to only get page 1 (most recent)
6. Refreshed the "recent played tracks" playlist in MM, The track i played on the phone is now the most recent, "last played date" is now 19:47 in MM, playcount has been incremented by 1.

It's worth noting that before I started modifying the script, this scenario didn't work as detailed above, it seems when ALL data is applied from last.fm using the original v3 script, this doesn't appear to work properly.

Am I missing your point here?
MM5 latest [Gold] / MMA latest | OnePlus 8 PRO | Noise I make: Pushkin on Soundcloud
Image
Funkafonik
Posts: 58
Joined: Fri Feb 13, 2009 12:33 pm

Re: Last.fm Playcount Import [MM3]

Post by Funkafonik »

You might have found the bug, I guess V3 applies FIRST played dates only when you first run it on an empty MM database...
I sure would love to try your updated script though! :)

About updating MM internal plays, I know there's already a script to add/remove them manually, only problem is you cannot add a playdate that is older than the file's Added Date, (which in turn cannot be changed to a date older than the file's Creation Date). So to allow a script to download and update ALL internal plays for every songs, the Creation & Added dates of all music files would have to be changed to 2004 (when Last.FM started) or to whatever year you started scrobbling.
zombiefly
Posts: 190
Joined: Fri Jun 08, 2007 3:33 pm

Re: Last.fm Playcount Import [MM3]

Post by zombiefly »

ok, my previous ramblings about errors with case turned out to be me misinterpreting the log file! :)
still, it caused me to understand more of the script.

so....

here's a screenshot of my lastfm page
http://yfrog.com/mypxpp

and here's a screenshot of my mediamonkey, after sync
http://yfrog.com/m916qp

I have PM'd you a link to my modified version
MM5 latest [Gold] / MMA latest | OnePlus 8 PRO | Noise I make: Pushkin on Soundcloud
Image
H2o
Posts: 7
Joined: Tue Aug 31, 2010 3:23 am

Re: Last.fm Playcount Import [MM3]

Post by H2o »

AWSOME SCRIPT!!!!!!


I always wanted this!

Working very well, whit no errors! :D :D :D :D :D
Psyker7
Posts: 255
Joined: Sat Mar 03, 2007 8:00 am

Re: Last.fm Playcount Import [MM3]

Post by Psyker7 »

zombiefly wrote:ok, my previous ramblings about errors with case turned out to be me misinterpreting the log file! :)
still, it caused me to understand more of the script.

so....

here's a screenshot of my lastfm page
http://yfrog.com/mypxpp

and here's a screenshot of my mediamonkey, after sync
http://yfrog.com/m916qp

I have PM'd you a link to my modified version
Any chance of letting me know the changes you made too?
Got back from overseas a little while ago but still been lazy, figured I should have another look at this, and see if I can get version 3 to a workable state with caching. XML parsing to combine the XML documents while avoiding duplicates isn't fun with the way last.fm exports their live data =(
Barry4679
Posts: 2398
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: Last.fm Playcount Import [MM3]

Post by Barry4679 »

Psyker7 wrote: Got back from overseas a little while ago but still been lazy, figured I should have another look at this, and see if I can get version 3 to a workable state with caching. XML parsing to combine the XML documents while avoiding duplicates isn't fun with the way last.fm exports their live data =(
Good to see you back Psyker7. I find your script really useful.

Is there any point pushing on with the version 3 variant? The version 2 approach does it for me.

If you had more energy for this project I would like to see version 2 somehow cache the information that it has accumulated, so that on a subsequent run it just needed to read the new weeklies, rather than go back to year dot every time. It seems that it already does something of this when Last.fm times out on your script, ie. a rerun quickly jumps to where LFM stopped playing ball, and then continues on from there. Of course you would also need an option to clear the local cache, and start afresh just as it does now.

I did try version 3, but it seemed to be giving me unreliable results. IIRC when my MM library had no play dates it seemed to work ok, but from then on it was erratic. Sometimes no update to the last played date, sometimes a datetime when I wasn't even playing anything. ... The other thing is it feels wrong to hammer LFM so hard for such a little gain with respect to play dates.

One final thing. The script stops fairly regularity with SQL errors. Sometimes it just says there was a problem executing the statement, other times it specifies the problem as the execution of a delete or insert call. I press the retry button, and it continues, and the final result seems ok. Do others see this problem? Maybe it it is just a problem with my version of the BDE, or whatever is handling the DBMS. I do a quick optimise fairly regularly to try and combat this.

thanks again for the script
Barry
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
Psyker7
Posts: 255
Joined: Sat Mar 03, 2007 8:00 am

Re: Last.fm Playcount Import [MM3]

Post by Psyker7 »

Barry4679 wrote: Is there any point pushing on with the version 3 variant? The version 2 approach does it for me.
I'm undecided. If I can get caching working then running it would be much quicker and you'd get more accurate play times obviously.
If you had more energy for this project I would like to see version 2 somehow cache the information that it has accumulated, so that on a subsequent run it just needed to read the new weeklies, rather than go back to year dot every time. It seems that it already does something of this when Last.fm times out on your script, ie. a rerun quickly jumps to where LFM stopped playing ball, and then continues on from there. Of course you would also need an option to clear the local cache, and start afresh just as it does now.
The times you see it rerun quickly is actually last.fm caching it's xml documents. Caching locally is something I'll work on, but it is going to be a little complicated as the divisions between xml files will be different each time.
One final thing. The script stops fairly regularity with SQL errors. Sometimes it just says there was a problem executing the statement, other times it specifies the problem as the execution of a delete or insert call. I press the retry button, and it continues, and the final result seems ok. Do others see this problem? Maybe it it is just a problem with my version of the BDE, or whatever is handling the DBMS. I do a quick optimise fairly regularly to try and combat this.
Haven't had much in the way of SQL errors myself. The older versions used to throw them up occasionally with non standard characters, but I've got a catchall there now. Letting me know a specific error message would help to track these down - or try enable logging.

Glad it's been useful :)
Last edited by Psyker7 on Tue Oct 12, 2010 7:46 pm, edited 2 times in total.
heartofhate
Posts: 190
Joined: Wed Jun 18, 2008 1:43 pm
Location: berlin

Re: Last.fm Playcount Import [MM3]

Post by heartofhate »

hm, with version 3 i get an xml-error after a while.
"Ein Element muss ein Element der obersten Ebene enthalten" which means "An element needs to be an elemend of the highest layer/level"
i have got absolutly no clue what this means -.-*
TJOHO
Posts: 112
Joined: Tue Oct 31, 2006 8:00 pm
Location: A Norwegian in Seoul, Korea

Re: Last.fm Playcount Import [MM3]

Post by TJOHO »

I run the import script about once a week to get plays scrobbled to Last.fm from other sources (XBMC, AudioSurf etc) into MM, but recently I have noticed tracks getting too high play counts!
I haven't been checking the log file nor paid that close attention, but sometimes I just notice that a song has way too high play counts. An example I just noticed:
  • "Dirty Deeds Done Dirt Cheap" by AC/DC had a play count of 148, which I immediately knew was too high.
  • On Last.fm it says I've scrobbled the song 27 times.
  • I used the Add/Remove Playstat script to reset the play count to only count HistoryPlays. The play count was then reset to 34! That is a lot closer to what I figured was the real play count.
Only counting HistoryPlays should list only plays that we are 100% certain have been played in MM itself, right? Any other plays must be added by some script or other. Well, the only automatic play count modifying script I ever use is the Last.fm Playcount Import script. Logically, this script must then be the culprit.

The Add/Remove Playstat script I only use manually on individual tracks to fine tune play counts.

By the way, this is with v2 of the script, v3 never worked for me.
Psyker7
Posts: 255
Joined: Sat Mar 03, 2007 8:00 am

Re: Last.fm Playcount Import [MM3]

Post by Psyker7 »

TJOHO wrote:I run the import script about once a week to get plays scrobbled to Last.fm from other sources (XBMC, AudioSurf etc) into MM, but recently I have noticed tracks getting too high play counts!
I haven't been checking the log file nor paid that close attention, but sometimes I just notice that a song has way too high play counts. An example I just noticed:
  • "Dirty Deeds Done Dirt Cheap" by AC/DC had a play count of 148, which I immediately knew was too high.
  • On Last.fm it says I've scrobbled the song 27 times.
  • I used the Add/Remove Playstat script to reset the play count to only count HistoryPlays. The play count was then reset to 34! That is a lot closer to what I figured was the real play count.
Only counting HistoryPlays should list only plays that we are 100% certain have been played in MM itself, right? Any other plays must be added by some script or other. Well, the only automatic play count modifying script I ever use is the Last.fm Playcount Import script. Logically, this script must then be the culprit.

The Add/Remove Playstat script I only use manually on individual tracks to fine tune play counts.

By the way, this is with v2 of the script, v3 never worked for me.
You're correct - HistoryPlays will only find the plays inside MM.
Does the play count get incrimented on that track once you run the script again (after resetting the playcount back to 34). It shouldn't - as the 27 on last.fm is lower than the 34 you say MM has. If you can reproduce it then let me know your last.fm username and I'll have a look.
firefoxsilver9
Posts: 18
Joined: Wed May 27, 2009 2:49 pm

Re: Last.fm Playcount Import [MM3]

Post by firefoxsilver9 »

Hello everyone. i have a few questions about this add-on.

First, is there a way to modify the script to have it override the playcount in MM? Also can the script be modified to allow a less-strict tag matching algorithm cause i noticed tracks that i have in my last.fm won't update the MM playcount if the tags aren't 100% identical, especially with music that has foreign characters. thanks in advance
Post Reply