Value of ID3v2 POPM field and MM's Rating

To get bugs in the current release fixed, please report them here.

Moderator: Gurus

MrSinatra
Posts: 41
Joined: Fri Dec 14, 2012 9:21 pm

Re: Value of ID3v2 POPM field and MM's Rating

Post by MrSinatra » Thu Jan 24, 2013 2:26 pm

maybe i'm missing something or not following you...

but i know for sure that windows explorer writes 255 for 5 stars, (as per the chart above). mm writes 252.

i am also pretty sure that most if not all versions of WMP writes 255 for 5 stars as well.

winamp uses 255, and i think foobar and most others do as well.

so if MM was going to comply with either the so-called de facto standard, or the WMP method, either way, it should use 255. as far as i know, it is the only one to use 252.

imo, it only makes sense to follow the spec, esp since thats what all the other apps seem to already be doing.

do you agree with this?

assuming you do, do you also agree that since MM supports granularity in the form of half stars, that 31 should be one full star? (keeping it compliant with windows explorer)

chrisjj
Posts: 4933
Joined: Wed Feb 14, 2007 5:14 pm
Location: UK

Re: Value of ID3v2 POPM field and MM's Rating

Post by chrisjj » Thu Jan 24, 2013 2:49 pm

MrSinatra wrote:imo, it only makes sense to follow the spec, esp since thats what all the other apps seem to already be doing.

do you agree with this?
I do.
MrSinatra wrote:assuming you do, do you also agree that since MM supports granularity in the form of half stars, that 31 should be one full star? (keeping it compliant with windows explorer)
I don't see why half-star granularity requires 31 == one full star, nor why 31 == one full star is required for Windows Explorer compatibility.

Note the standard does not require 31 == one full star. It doesn't require anything of stars values, except the max and min.
Chris

jiri
Posts: 5368
Joined: Tue Aug 14, 2001 7:00 pm
Location: Czech Republic
Contact:

Re: Value of ID3v2 POPM field and MM's Rating

Post by jiri » Thu Jan 24, 2013 6:36 pm

First of all, does it really matter whether 5 stars is represented as 252 or 255? I'm asking since I'd like the know why this actually started, whether there's any incompatibility, etc.

As for the reasoning for the choice of mapping we made in the past - let's look at POPM value 1. It's 1 star according to WMP. Ok, sounds like a good choice, at least as long as you don't consider other applications that would like to use half-stars (as MM does). Then it doesn't make sense anymore - MM can't write POPM value 1 for some below-1-star rating, since upon reading such value, it wouldn't be clear whether it's 1 star (from WMP) or e.g. a half star (from MM). So, this is why the mapping from MM starts at POPM value 3. And the 5 stars? Ok, the same reasoning doesn't apply there, since MM doesn't support something like 5.5 stars, so we really could have chosen to map 5 stars to POPM value 255. We didn't, but really, is it that important?

Jiri

chrisjj
Posts: 4933
Joined: Wed Feb 14, 2007 5:14 pm
Location: UK

Re: Value of ID3v2 POPM field and MM's Rating

Post by chrisjj » Thu Jan 24, 2013 6:52 pm

jiri wrote:We didn't, but really, is it that important?
If you think complying with the standard is unimportant, then no.
If you think complying with the standard is important, then yes.

The point of a standard is to allow interoperability. If I write a standards-compliant program to read rating and test if the value is at maximum, then it will not interoperate properly with the current MM, due to MMs non-compliance. If I kluge the program to fix this, then this will not necessarily interoperate on some other compliant program.

If MM was changed to be compliant, the problem is solved.
Chris

MrSinatra
Posts: 41
Joined: Fri Dec 14, 2012 9:21 pm

Re: Value of ID3v2 POPM field and MM's Rating

Post by MrSinatra » Thu Jan 24, 2013 6:57 pm

chrisjj wrote:
MrSinatra wrote:imo, it only makes sense to follow the spec, esp since thats what all the other apps seem to already be doing.

do you agree with this?
I do.
having now reread your post, i should have known you did, i can be thick. ;)
chrisjj wrote:
MrSinatra wrote:assuming you do, do you also agree that since MM supports granularity in the form of half stars, that 31 should be one full star? (keeping it compliant with windows explorer)
I don't see why half-star granularity requires 31 == one full star, nor why 31 == one full star is required for Windows Explorer compatibility.

Note the standard does not require 31 == one full star. It doesn't require anything of stars values, except the max and min.
well, thats true regarding the spec. part of the whole POPM problem is that it gave a range without specifying how or what to display or what sub-ranges one should have to create a display.

however, i have given this a lot of thought, and the answer is that all the apps should basically be compliant with windows. thats how a de facto standard can be established. so basically thats what apps have already done. they have conformed their scales, more or less, to what windows does.

lets look again at my chart:
MrSinatra wrote:here is what windows explorer uses:

>>
224-255 = 5 stars when READ with windows explorer, writes 255
160-223 = 4 stars when READ with windows explorer, writes 196
096-159 = 3 stars when READ with windows explorer, writes 128
032-095 = 2 stars when READ with windows explorer, writes 64
001-031 = 1 stars when READ with windows explorer, writes 1
>>

windows explorer does not support granularity for DISPLAY. it does support the ranges above though.
when READING a POPM value, windows explorer will see anything valued 1-31 as one full star. if it is 32 or higher, it would be MORE THAN one full star to windows.

so if you want granularity in MM or whatever app, you need to establish the highest number for each star windows will do. for one star, its 31. that leaves an app 1-30 "points" of granularity under one.

so say you had 5 stars for display, but your app wanted to do not just half stars, but varied fractions of stars. the chart would look like this:

-----255 = five stars
224-254 = points of granularity between four and five stars
-----223 = four stars
160-222 = points of granularity between three and four stars
-----159 = three stars
096-158 = points of granularity between two and three stars
-----095 = two stars
032-094 = points of granularity between one and two stars
-----031 = one star
001-030 = points of granularity under one star
0 = 0 stars

so you'd want an app to write the highest number that windows would see/read as a full star for each star, i.e. 31,95,159,223,255

do you agree with that?

this is problematic however.

first of all, the ranges are odd. the high and low ends only have about 30 points of granularity while the middle ranges have about 60+

furthermore, what windows writes also has to be taken into account. windows writes:

255
196
128
64
1

for each star. in an app that provides granularity, the question then is how to account for this? if 95 is two full stars, 64 would be around 1.5

but 64 is two stars to windows, and users of MM or winamp or any app will want that app to show what windows shows.

that makes it a difficult issue. i mean, how could an app like MM reconcile windows writing 1 for 1 star, with some system that displays 0.5 stars?

MrSinatra
Posts: 41
Joined: Fri Dec 14, 2012 9:21 pm

Re: Value of ID3v2 POPM field and MM's Rating

Post by MrSinatra » Thu Jan 24, 2013 7:08 pm

jiri wrote:First of all, does it really matter whether 5 stars is represented as 252 or 255? I'm asking since I'd like the know why this actually started, whether there's any incompatibility, etc.
there is incompatibility. winamp [currently] will not see a MM rated 5 star file as 5 stars. this is b/c winamp [currently] will ONLY see 255 as = to 5 stars. anything below 255, like 252, is not 5 stars to winamp. thats just one example i know of. thx to my efforts at winamp, they are likely to copy the windows explorer scale for future versions. but that doesn't totally resolve the issue, since if i sort a ratings column, files will go 255, then 252, even though both are meant to be 5 star.

but this is why i've been asking about if anyone has done WMP testing, b/c i believe it does granularity, and i believe WMP sees only 255 as 5 star, and anything under that as not 5 star.
jiri wrote:As for the reasoning for the choice of mapping we made in the past - let's look at POPM value 1. It's 1 star according to WMP. Ok, sounds like a good choice, at least as long as you don't consider other applications that would like to use half-stars (as MM does). Then it doesn't make sense anymore - MM can't write POPM value 1 for some below-1-star rating, since upon reading such value, it wouldn't be clear whether it's 1 star (from WMP) or e.g. a half star (from MM). So, this is why the mapping from MM starts at POPM value 3. And the 5 stars? Ok, the same reasoning doesn't apply there, since MM doesn't support something like 5.5 stars, so we really could have chosen to map 5 stars to POPM value 255. We didn't, but really, is it that important?

Jiri
right, thats what i was posting about above.

what i am trying to do, figure out, is a way an app like MM can thread the needle so to speak, so that it can have as much granularity as it wants, without being non-compliant with windows explorer, WMP, and other apps.

i did the testing on windows explorer myself, but i don't have similar tests / exp with WMP. once we establish the facts on what WMP does, we can better approach how to thread that needle, if possible.

thats complicated though, since all WMP versions might not behave the same exact way.

chrisjj
Posts: 4933
Joined: Wed Feb 14, 2007 5:14 pm
Location: UK

Re: Value of ID3v2 POPM field and MM's Rating

Post by chrisjj » Thu Jan 24, 2013 7:14 pm

MrSinatra wrote:having now reread your post, i should have known you did, i can be thick. ;)
No harm in checking :)
MrSinatra wrote:part of the whole POPM problem is that it gave a range without specifying how or what to display or what sub-ranges one should have to create a display.
It is not the job of a file format spec to dictate metadata display resolution. That would be like it dictating audio playback quality.
MrSinatra wrote:i mean, how could an app like MM reconcile windows writing 1 for 1 star, with some system that displays 0.5 stars?
WMP stars disallow 0.5 star. If MM wants to store rating in WMP stars, then MM can't store 0.5 stars. MM can't have its cake and eat it.
Chris

MrSinatra
Posts: 41
Joined: Fri Dec 14, 2012 9:21 pm

Re: Value of ID3v2 POPM field and MM's Rating

Post by MrSinatra » Thu Jan 24, 2013 9:08 pm

chrisjj wrote:
MrSinatra wrote:part of the whole POPM problem is that it gave a range without specifying how or what to display or what sub-ranges one should have to create a display.
It is not the job of a file format spec to dictate metadata display resolution. That would be like it dictating audio playback quality.
i totally agree that how many stars, or half stars, or whatever should not be dictated by the spec.

however, it would have been helpful if they had said something like "if using a 0-10 scale, then we recommend using these values for each point in that scale," and doing the same thing for 0-5 and 0-100 and making sure that all 3 shared the same POPM 0-255 for each full star, eg 255 is always 5 stars, or 10 or 5 or 100 respectively.

by not doing so, adoption of POPM, and emergence of a de facto standard, has been painfully slow to happen.
chrisjj wrote:
MrSinatra wrote:i mean, how could an app like MM reconcile windows writing 1 for 1 star, with some system that displays 0.5 stars?
WMP stars disallow 0.5 star. If MM wants to store rating in WMP stars, then MM can't store 0.5 stars. MM can't have its cake and eat it.
i haven't done proper WMP testing yet. its starting to look like that what i'm going to have to do, b/c i can't find answers on google. i don't know enough about how or what it displays, or what POPM values it uses to create those displays.

as you know, its a real pita, b/c mp3tag won't let popularimeter work separately from its "RATING WMP" type columns, which just irritates me to no end.

jiri
Posts: 5368
Joined: Tue Aug 14, 2001 7:00 pm
Location: Czech Republic
Contact:

Re: Value of ID3v2 POPM field and MM's Rating

Post by jiri » Fri Jan 25, 2013 3:53 am

It is not the job of a file format spec to dictate metadata display resolution. That would be like it dictating audio playback quality.
I disagree. It then results in the current situation, where WMP makes an 'obvious' choice of 1 star = POPM 1, which means that other apps either can't be fully compatible to WMP or ID3v2. Well, you can say that a developer can choose either way, but users expect that _both_ will be supported and we certainly try to implement what users want and expect.
there is incompatibility. winamp [currently] will not see a MM rated 5 star file as 5 stars
Interesting, I wasn't aware of this. I would bet that it worked well years ago with older Winamps.

I'm definitely open to discussions re. improvements in MMW in this area. Note however, that what might look like a trivial change actually isn't, since we want to take into account compatibility (full, reading and writing) with WMP, hopefully WinAmp, but old older MM versions.

I wonder, would the WinAmp problem be fixed by writing 5 stars (i.e. exactly 100 internal value in MM) as 255 in POPM? Or are there also other problems wrt other ratings?

Jiri

chrisjj
Posts: 4933
Joined: Wed Feb 14, 2007 5:14 pm
Location: UK

Re: Value of ID3v2 POPM field and MM's Rating

Post by chrisjj » Fri Jan 25, 2013 6:35 am

jiri wrote:
It is not the job of a file format spec to dictate metadata display resolution. That would be like it dictating audio playback quality.
I disagree.
The ID3 spec. does not do that job - if that is unacceptable to you, then perhaps you should not have chosen to use it in the first place.
jiri wrote:It then results in the current situation, where WMP makes an 'obvious' choice of 1 star = POPM 1, which means that other apps either can't be fully compatible to WMP or ID3v2.
That is false. Other apps can easily be fully compatible to WMP and ID3v2. What results in the current situation is poor design in MM. MM is incompatible only because it is trying to be compatible with five levels of stars while providing ten levels of stars in a representation that cannot handle those ten.

The standard says nothing about stars. The decision to display the standard value in reduced resolution in star form is entirely a decision of the application developers'. Since you application developers have not agreed a standard on this amongst yourselves, and MM is trying to be better than but compatible with one of the others, then MM should expect the trouble it is getting.
Chris

jiri
Posts: 5368
Joined: Tue Aug 14, 2001 7:00 pm
Location: Czech Republic
Contact:

Re: Value of ID3v2 POPM field and MM's Rating

Post by jiri » Fri Jan 25, 2013 7:23 am

chrisjj,

this is not the first time I find discussing with you hard, because it looks like you decided sometimes in the past that there's only one truth in this world - and it's your opinion. I'm not trying to asses whether we made the best or worse decision in the past, but the fact is that MM _is_ fully compatible with WMP ratings, that MM _does_ support half-star ratings and that MM probably is compatible with several other apps using ID3v2. Talking about rating, try to compare this state of things with other apps, e.g. Winamp - it doesn't even understand POPM value of 253 as 5 stars? Even though WMP does? Talking about poor design, I see one here. And you call a poor design that MM is compatible with many apps, even though supporting half-star ratings? Seriously?

I think that it makes more sense to try to find the best solution. I suggested a small change in the previous post that seems to help and wouldn't break anything. Looking for more info from MrSinatra and others in order to confirm whether this really is the best way to go.

Jiri

rovingcowboy
Posts: 14153
Joined: Sat Oct 25, 2003 7:57 am
Location: (Texas)
Contact:

Re: Value of ID3v2 POPM field and MM's Rating

Post by rovingcowboy » Fri Jan 25, 2013 7:55 am

I'm with chrisjj on this, subject there are bad non complyent programs out there. I'm also with jiri mm is working with several others in the way the ratings are used.

But i think you both are reading from different diconarys, and getting different meanings of the words because your always different then each other when explaining things its both funny and confusing to us non programmers. 8)
roving cowboy / keith hall. My skins http://www.mediamonkey.com/forum/viewto ... =9&t=16724 for some help check on Monkey's helpful messages at http://www.mediamonkey.com/forum/viewto ... 4008#44008 MY SYSTEMS.1.Jukebox WinXp pro sp 3 version 3.5 gigabyte mb. 281 GHz amd athlon x2 240 built by me.) 2.WinXP pro sp3, vers 2.5.5 and vers 3.5 backup storage, shuttle 32a mb,734 MHz amd athlon put together by me.) 3.Dell demension, winxp pro sp3, mm3.5 spare jukebox.) 4.WinXp pro sp3, vers 3.5, dad's computer bought from computer store. )5. Samsung Galaxy 5 Android 5) 6. Proscan tablet Android 4.3 ) 7. amd a8-5600 apu 3.60ghz mm version 4 windows 7 pro bought from computer store.

chrisjj
Posts: 4933
Joined: Wed Feb 14, 2007 5:14 pm
Location: UK

Re: Value of ID3v2 POPM field and MM's Rating

Post by chrisjj » Fri Jan 25, 2013 9:34 am

jiri wrote:it looks like you decided sometimes in the past that there's only one truth in this world - and it's your opinion.
It is my opinion that there very often is only one truth on questions about specifications such as this - because the whole purpose of a such a specification is to provide for each point a single truth which compliant applications can then rely on. Only if applications agree on the truth can they interoperate properly.
jiri wrote:the fact is that MM _is_ fully compatible with WMP ratings
That is untrue. MM does not allow the user to set the maximum WMP rating value of 255. That is why users including myself who used MM and WMP together ended up with a mixture of different values for maximum, causing trouble in other programs.
jiri wrote:try to compare this state of things with other apps, e.g. Winamp - it doesn't even understand POPM value of 253 as 5 stars? Even though WMP does? Talking about poor design, I see one here.
I think you have misunderstood the design objective of WinAmp. I see nothing to suggest the design objective was full compatibility with WMP. I see everything to suggest the design objective was fully compatibility with ID3V2.3. I see everything to suggest that design objective was achieved..
jiri wrote:And you call a poor design that MM is compatible with many apps, even though supporting half-star ratings? Seriously?
You misrepresent what I said. I said MM ill-designed programming is the likely cause of non-compliance with the ID3v2 standard.
jiri wrote:I think that it makes more sense to try to find the best solution.
I suggest you start by withdrawing from the MM documentation the false claim that MMs Rating implementation is compatible with the WMP POPM implementation. Then decide which is more important to you for the Ratings implementation - being compatible with the standard and other apps, or giving the impression of being better than other apps.
Chris

MrSinatra
Posts: 41
Joined: Fri Dec 14, 2012 9:21 pm

Re: Value of ID3v2 POPM field and MM's Rating

Post by MrSinatra » Sat Jan 26, 2013 9:46 am

i would urge everyone to remain open minded and not get upset. if we establish facts, i'm sure we can figure out a better solution.

now, it may not be a perfect solution, but even an improvement would be a worthy goal.

i have already done the research and posted it on windows explorer (via win7). i am now planning to do tests on win7 WMP12. i will post those results in this thread, (assuming i am confident i got valid results). btw, i expect that most recent versions of win exp, be it xp, 7 or 8, share the same values. i'm not as confident WMP never made changes between versions, but i don't know either way. also, i would expect that WMP does not contradict win exp, and vice versa.

let me clarify a few things:

ratings in winamp via tags are new. they have only been in winamp for about a year and a half. prior to that, they existed ONLY in the winamp DB, (and frequently then were "lost" by users who had DB issues, moved computers, etc). i basically instigated the feature and i have been working with them to perfect their implementation ever since.

afaik, all apps, all of them, write 255 as 5 stars, EXCEPT MM. its not only the current winamp that will not see 252 as = to 5 stars. i simply gave winamp as an example.

i recently got winamp to change how it reads POPM, to emulate the chart i gave for windows explorer. that change should be in the next release of winamp. their current implementation was based on how they thought WMP used POPM. but they had somehow borked their math on it, b/c while 128 was three stars, 129 would only show two! obvious bug there.

also, other incompatibilities get raised when you consider the id string for POPM doesn't always mean anything. mp3tag for instance has a column called RATING MM which is based on MMs use of no@email as the POPM id string. the problem is that just because that string is present, doesn't mean MM wrote the value! an app is not supposed to overwrite that id string, and winamp for example, doesn't. so if u have a file rated in MM at 4 stars, then later use winamp to rate it 5, it now has an unexpected value to mp3tag, of 255, b/c mp3tag sees no@email and expects nothing higher than 252. this invalidates the RATING MM column, it won't show up there any more.

(the id string is supposed to identify the user, not the app. if apps properly supported the spec, multiple POPM frames could be in a file, and each would use the users' email addys to differentiate ratings from one another in the same file. in any case, rating@winamp.com is what they use by default, but the next release of winamp, will allow the user to specify their own email, or id string, in the prefs. and winamp will continue to, properly, not overwrite existing id strings by default)

in my next post i'll go over what i think i know about WMP and try to give some recommendations. for reference, here are some links of interest:

http://id3.org/id3v2.3.0#sec4.18

http://en.wikipedia.org/wiki/ID3#ID3v2_Rating_tag_issue

http://www.ventismedia.com/mantis/print ... ug_id=3143

and some winamp ones, although please note that the info in them evolved over time, so later posts are much more likely to have correct info, but even then they may not represent the best of my current understanding:

http://forums.winamp.com/showthread.php?t=315444

http://forums.winamp.com/showthread.php?t=324512

MrSinatra
Posts: 41
Joined: Fri Dec 14, 2012 9:21 pm

Re: Value of ID3v2 POPM field and MM's Rating

Post by MrSinatra » Sat Jan 26, 2013 11:50 am

before i begin, i need to report something totally unexpected. afaict, no version of WMP ever supported half stars or fractions of a star, as set by users.

WMP only supports fractional stars when displaying stars IN BLUE, which indicates that the ratings came from AMG or some internet based metadata source. if a user rates the file, the rating is gold/yellow in WMP. i do not know if blue ratings can be inserted into POPM or not, or if they are, if they use a different id string.

here are my WMP12 (win7) results:

222-255 = 5 stars when READ with WMP12, writes 255
160-221 = 4 stars when READ with WMP12, writes 196
096-159 = 3 stars when READ with WMP12, writes 128
032-095 = 2 stars when READ with WMP12, writes 64
001-031 = 1 stars when READ with WMP12, writes 1

the first thing to notice, is that the only difference between the wmp chart and the win exp chart, is the cutoff between 4 and 5 stars. win exp uses 223/224, while wmp uses 221/222. why? god only knows. (assuming i didn't make any mistakes in my testing, which isn't a given b/c the tools i have to use are crude) having said that, i would appreciate it if anyone were to doublecheck my work, and validate it or catch errors in it.

so anyway, now we know that win exp, and wmp, are in near complete agreement, and that wmp doesn't support fractional stars. we also know that winamp will be compliant with win exp in future versions, and that only makes sense, since winamp, MM, and so on should comply with the OS they run on, over a competing media player that happens to come with that OS. (although the only slight difference in reality anyway, is the 4/5 cutoff)

given all these facts above, i think its safe to say the following:

MM should display/read/write for full stars exactly what both WMP and Win Explorer do. that means 1,64,128,196,255 must be respected as is.

...but that doesn't mean half stars can't be implemented by MM in a sensible way.

here is the chart i would urge MM to use:

(0 is always 0 stars, essentially the same as unrated/no POPM frame, nothing is displayed)

--

......255 = 5 stars when READ with MM, writes 255

197-254 = 4.5 stars when READ with MM, writes 226 (so rounded up to 5 by win exp/wmp, based on how they read values)

......196 = 4 stars when READ with MM, writes 196

129-195 = 3.5 stars when READ with MM, writes 162 (so rounded up to 4 by win exp/wmp)

......128 = 3 stars when READ with MM, writes 128

065-127 = 2.5 stars when READ with MM, writes 98 (so rounded up to 3 by win exp/wmp)

......064 = 2 stars when READ with MM, writes 64

032-063 = 1.5 stars when READ with MM, writes 47 (so rounded up to 2 by win exp/wmp)

......001 = 1 star when READ with MM, writes 1

002-031 = 0.5 stars when READ with MM, writes 15 (so rounded up to 1 by win exp/wmp)

--

obviously it is odd to have any POPM values higher than 1 = less than one star in MM, but some kind of workaround like this is needed so that there can be both half stars in MM, and interoperability between the values of POPM regardless of whether or not MM or windows or something else rated the file.

now, i know it is not trivial to make these changes or convert your system, however it is necessary something be done, if only to be sensical and compatible.

i think what winamp did could help you here. as they implemented ratings in beta, they used improper scales for data ranges, both in POPM and in Vorbis, i think of 0-5 for both. since some people had already rated their files using those scales, when they "fixed" it internally in the app, they released a new version of winamp.

the new version, when installed over the older ones, would run what i assume was a script, to convert all the 0-5 values to the proper 0-255 values for POPM, or 0-100 range for Vorbis. this happened as part of the install process. MM could do a similar conversion, you would just need to know what old ranges to map to the new chart above. (for instance, all 252 values would be converted to 255, etc)

i hope this info is helpful, and i welcome any and all corrections, feedback. (i am somewhat paranoid i made a typo, observational error, or mistake in logic re: my charts) thanks.

EDIT: while i tried to pick halfway points for the half stars, i also made them a couple points above the win/wmp reading cutoff for those ranges. but something else you could do would be to make all the half stars values for writing, end in 7, so 17 instead of 15, etc... that way when looking at the raw POPM, you'd know a value was a half star if it ended in 7 (since no full star ends in 7). they'd be 17, 47, 97, 167 and 227. 97 cuts it close, but still works.
Last edited by MrSinatra on Tue Jan 29, 2013 9:32 am, edited 2 times in total.

Post Reply