1209: Unorganized tracks, again BUG#5184

[Mizery_Made]

There's a few things that seem off about this feature. The biggest problem I'm running into is performance. I just ran a scan of this node while DebugView was open and it took seven minutes to complete. That's for 10,715 tracks. I've had this scan run faster than this in the past, and even if I hadn't, seven minutes seems a bit much. I raised an issue with the performance of this node way back in the original 3.0 test cycle I believe, comparing it to Trixmoto's UnAutoOrganised script, pointing out that the built in feature took somewhere between two and three minutes I believe (and this was on no more than 2,000 less files than my current total) while his script could finish in under a minute.

I believe an entry was made into Mantis for this report. (Found it, #4090) I can understand pointing out that the built in feature uses a more complex method than his script, but when it performs nearly the identical function, seven minutes is a bit much in comparison to roughly thirty five seconds.

I'm attaching a debug log snippet from the point I clicked the node until it stopped cycling through the tracks. Not sure it'll help in finding a way to optimize, but you can see yourself it took an awful long time. However, there's a couple things in the log I want to point out separately.

Code: Select all

00013652	13.94685650	[5448] Auto-Organizer : Track matches rule:Genres:Country,Instrumental & Acapella,Pop,R&B,Rap,Rock; C:\My Music\<Genre>\$if(<Grouping>=--Album--,<Album Artist>\$if(<Album>=Hip-Hop Weekly,<Album>,<Year> - <Album>),$if(<Genre>=Instrumental & Acapella,,!<Grouping>!))\$if(<Grouping>=--Album--,<Track#> - {<Artist>} - <Title>,$if(<Title>=It's Okay [One Blood][Remix],$left(<Artist>,8) - <Title>,$if(<Title>=Won't Let You Down [Extended Texas Remix],$left(<Artist>,14) - <Title>,<Artist> - <Title>)))	
00013653	13.94739151	[5448] Auto-Organizer : Begin checking track C:\My Music\Rap\Matic Lee\0000 - Infinity\03 - {Matic Lee} - Chalk it up!!.mp3	
00013654	13.94751549	[5448] Auto-Organizer : Checking rule:Genres:Country,Instrumental & Acapella,Pop,R&B,Rap,Rock; C:\My Music\<Genre>\$if(<Grouping>=--Album--,<Album Artist>\$if(<Album>=Hip-Hop Weekly,<Album>,<Year> - <Album>),$if(<Genre>=Instrumental & Acapella,,!<Grouping>!))\$if(<Grouping>=--Album--,<Track#> - {<Artist>} - <Title>,$if(<Title>=It's Okay [One Blood][Remix],$left(<Artist>,8) - <Title>,$if(<Title>=Won't Let You Down [Extended Texas Remix],$left(<Artist>,14) - <Title>,<Artist> - <Title>))) for genres	
00013655	13.94757366	[5448] Auto-Organizer : Genre Rap matches	
00013656	13.94761467	[5448] Auto-Organizer : Track matches rule	
00013657	13.94765377	[5448] Auto-Organizer : Track matches path	
00013658	13.94872189	[5448] Auto-Organizer : Track is in: C:\My Music\Rap\Matic Lee\0000 - Infinity\03 - {Matic Lee} - Chalk it up!!.mp3	
00013659	13.94877148	[5448] Auto-Organizer : Should be in: C:\My Music\Rap\Matic Lee\0000 - Infinity\	
00013660	13.94904327	[5448] Auto-Organizer : Track name is: 03 - {Matic Lee} - Chalk it up!!.mp3	
00013661	13.94909000	[5448] Auto-Organizer : Track name should be: 03 - {Matic Lee} - Chalk It Up!!.mp3	

Code: Select all

00055745	159.43266296	[5448] Auto-Organizer : Track matches rule:Genres:Country,Instrumental & Acapella,Pop,R&B,Rap,Rock; C:\My Music\<Genre>\$if(<Grouping>=--Album--,<Album Artist>\$if(<Album>=Hip-Hop Weekly,<Album>,<Year> - <Album>),$if(<Genre>=Instrumental & Acapella,,!<Grouping>!))\$if(<Grouping>=--Album--,<Track#> - {<Artist>} - <Title>,$if(<Title>=It's Okay [One Blood][Remix],$left(<Artist>,8) - <Title>,$if(<Title>=Won't Let You Down [Extended Texas Remix],$left(<Artist>,14) - <Title>,<Artist> - <Title>)))	
00055746	159.43276978	[5448] Auto-Organizer : Begin checking track C:\My Music\Rap\Matic Lee\0000 - Infinity\14 - {Matic Lee} - Love And Happiness.mp3	
00055747	159.43281555	[5448] Auto-Organizer : Checking rule:Genres:Country,Instrumental & Acapella,Pop,R&B,Rap,Rock; C:\My Music\<Genre>\$if(<Grouping>=--Album--,<Album Artist>\$if(<Album>=Hip-Hop Weekly,<Album>,<Year> - <Album>),$if(<Genre>=Instrumental & Acapella,,!<Grouping>!))\$if(<Grouping>=--Album--,<Track#> - {<Artist>} - <Title>,$if(<Title>=It's Okay [One Blood][Remix],$left(<Artist>,8) - <Title>,$if(<Title>=Won't Let You Down [Extended Texas Remix],$left(<Artist>,14) - <Title>,<Artist> - <Title>))) for genres	
00055748	159.43287659	[5448] Auto-Organizer : Genre Rap matches	
00055749	159.43290710	[5448] Auto-Organizer : Track matches rule	
00055750	159.43392944	[5448] Auto-Organizer : Track matches path	
00055751	159.43400574	[5448] Auto-Organizer : Track is in: C:\My Music\Rap\Matic Lee\0000 - Infinity\14 - {Matic Lee} - Love And Happiness.mp3	
00055752	159.43426514	[5448] Auto-Organizer : Should be in: C:\My Music\Rap\Matic Lee\0000 - Infinity\	
00055753	159.43432617	[5448] Auto-Organizer : Track name is: 14 - {Matic Lee} - Love And Happiness.mp3	
00055754	159.43440247	[5448] Auto-Organizer : Track name should be: 14 - {Matic Lee} - Love and Happiness.mp3	

In the first snippet, it's ran through a track who has a filename with some lowercase letters that have since been fixed in MediaMonkey. ("Chalk it up!!" -> "Chalk It Up!!") While in the second example, I took a file that was named correctly and lowercased a word to see if it worked in reverse. ("Love And Happiness" -> "Love and Happiness") The Auto-Organizer will show a difference between the cases in the log, but neither of these tracks are displayed in the main window. Running Auto Organize on either of these files will result in it correcting the case, so why isn't the Auto-Organizer showing these as not correct?

I also thought I would note that while it was reported that a progress bar was added to this node in Mantis entry #5015, switching to the Monkey from DebugView while running a test on this node, there was never a progress bar present.

Can anyone confirm what appears to be slow performance of this node, the case issues, or the lack of functioning bar?

EDIT: Forgot to set up the "Upload attachment," then I realized the log was 19mb... so I had to RAR it, which cut it down to like a tenth of the size.

[MoDementia]

I tested the node in 3.0.5.1187 and while it didn't take 7 mins it is substantially slower than the script.

I didn't add any criteria (filters, genres, playlists) and besides, this is only 1 more select on top of the rule checking


I added 1 filter (about half my library) and it just bogged the process down (over a min)

There is something wrong with the node code for sure.

[EDIT] I haven't tested the case problem but the progress bar is there but uninformative, just reading tracks
Even the most basic scripts report X of Y progress
Ah supposedly added after this build

[chrisjj]

> took seven minutes to complete. That's for 10,715 tracks.

FWIW, 14secs for a similar qty here (P4 3.2Gz CPU)

[Mizery_Made]

There certainly seems to be something up with the use of Genre as a rule. I have two rules set, one for a "Wrestling" genre while another set for all my other genres (Pop, Rock, Country, Rap, R&B, etc.) I unchecked the genre on the single genre rule, and replaced the genres from the second rule with an auto playlist that only has the criteria of "is Accessible." Running this node with modified match criteria results in the process finishing in a considerably smaller ammount of time. I didn't time it this time, but a minute or so is a good guess. Also, with these modified rules, the Progress Bar showed up.

So I'm seeing two reproducable errors, the case issue and poor performance using Genre as a matching criteria.

[Mizery_Made]

I've been trying to run some more tests on this node to figure out where the performance problem, but most of them are running fine now, even when I went back to matching on Genre. However, using Auto-Playlists which match genres appears to perform poorly now. And the bar has gone back into hiding. I get varying results from this node, I think I'm just going to leave it alone before it drives me crazy.

The only thing that has stayed consistent is the Capitalization part.

[Lowlander]

The topic split in 2 discussions, see the How Unorganize Tracks works topic here.

[chrisjj]

Thanks LL.

[Mizery_Made]

I have entered the Case issue into Mantis as Bug#5184

[PetrCBR]

Unorganized node should be much faster in 1219.

[Mizery_Made]

I'm glad to hear there's maybe been some improvements in it's performance. The runtime seemed to vary between runs, even with the same settings, so I couldn't put my finger on a cause. Look forward to seeing how it works in the next build.

[Teknojnky]

its super fast now, ~150k tracks in 2ish minutes and works correctly with my complex mask
previously, it didn't work right and took 3 hours

Petr got it working great.

[Mizery_Made]

Indeed, looks like much improvement has been made in this area. The bar shows up pretty much the moment I select the node, while I've seen in the past it sometimes didn't like to show up until it found the first track not matching the mask. It has went from taking a couple minutes one my 10,000+ library to an estimated 10 seconds or so. I shall go edit a few tracks to make sure it finds them correctly, as I actually ran the node and fixed the errors I had not long before PetrCBR announced some improvements in this node.

EDIT: Looks to be fully functional. Awesome.

[MoDementia]

Great News, I might even have to install it just to check it out