Error IUNICODE

To discuss development of addons / skins / customization of MediaMonkey.

Moderators: jiri, drakinite, Addon Administrators

philaupatte
Posts: 24
Joined: Thu Oct 01, 2015 2:52 am

Error IUNICODE

Post by philaupatte »

Hi,
I have read lot of information regarding IUNICODE and currently I am a bit lost on what should be done.

I ran a very simple test powershell :

Code: Select all

Import-Module PSSQLite

try
{
 $dbMM5 = "C:\Users\phili\AppData\Roaming\MediaMonkey5\MM5.DB" 
 $query = "SELECT ID, SongPath FROM Songs"
 $sqlConn = @{Datasource = $dbMM5}
 [pscustomobject[]]$songsFetched = Invoke-SqliteQuery @sqlConn -Query $query
} 
I get an error

Code: Select all

Invoke-SqliteQuery : Exception lors de l'appel de «Fill» avec «1» argument(s): «SQL logic error or missing database
no such collation sequence: IUNICODE »
Au caractère C:\powershell\test.ps1:13 : 35
+ ... stomobject[]]$playListsDB = Invoke-SqliteQuery @sqlConn -Query $query
+                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Invoke-SqliteQuery
This is due to SongPath column and constraint collate. Meanwhile in SQLITE Studio the same query runs just magic !!!

Is there a solution and/or bypass?


Many thanks
Barry4679
Posts: 2398
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: Error IUNICODE

Post by Barry4679 »

I suspect that even Sqlite Studio will get upset if you try and do anything with string colums other than just mention them in the Select clause.

try the query like this
SELECT ID, SongPath collate nocase FROM Songs;

Otherwise you can load MM's extension which supplies the missing collation.
The name of the dll is SQLite3MMExt.dll. I have a copy of it, but no longer know where they have it available for download.
The problem with the dll is that they have only shipped a 32 bit version, so you may or may not be alble to load it.

The other workaround that I have used is to create your own collation, and get SQLite to load it.
In Python all that I needed was define a function like this:
def iUnicodeCollate(s1, s2):
#return cmp(s1.lower(), s2.lower())
#cmp gone in P3
return s1.lower().__lt__(s2.lower())
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
philaupatte
Posts: 24
Joined: Thu Oct 01, 2015 2:52 am

Re: Error IUNICODE

Post by philaupatte »

Hola Barry,
Many thanks. I try already the collate nocase without any success.

I manage to get the DDL and I copy it into C:\Program Files (x86)\MediaMonkey 5 but what is the next move it. Request still fails.

What should I do, did not find much information within http://www.ventismedia.com/mantis/view.php?id=11083

Meanwhile I found an ugly bypass "SELECT * FROM Songs". This query processes successfully. I think, if I am not wrong, this is due to the fact indexes are not involved in the path access.
drakinite
Posts: 965
Joined: Tue May 12, 2020 10:06 am
Contact:

Re: Error IUNICODE

Post by drakinite »

Interestingly enough, when I search for your exact error, one of the first results is related to MediaMonkey: https://stackoverflow.com/questions/470 ... quence-iun

I don't personally know much about the database, but maybe this StackOverflow post can help, if you're still not sure about your problem?
Image
Student electrical-computer engineer, web programmer, part-time MediaMonkey developer, full-time MediaMonkey enthusiast
I uploaded many addons to MM's addon page, but not all of those were created by me. "By drakinite, Submitted by drakinite" means I made it on my own time. "By Ventis Media, Inc., Submitted by drakinite" means it may have been made by me or another MediaMonkey developer, so instead of crediting/thanking me, please thank the team. You can still ask me for support on any of our addons.
Barry4679
Posts: 2398
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: Error IUNICODE

Post by Barry4679 »

philaupatte wrote: Sun Sep 19, 2021 12:09 pm Hola Barry,
Many thanks. I try already the collate nocase without any success.

I manage to get the DDL and I copy it into C:\Program Files (x86)\MediaMonkey 5 but what is the next move it. Request still fails.

What should I do, did not find much information within http://www.ventismedia.com/mantis/view.php?id=11083

Meanwhile I found an ugly bypass "SELECT * FROM Songs". This query processes successfully. I think, if I am not wrong, this is due to the fact indexes are not involved in the path access.

Are you sure that the errors is coming from that SQL statement? Do you have other sql statements?
Is it just a warning message that you can ignore.

In the past I have only had problems when I try ang get SQLite to do a comparison with a text column, ie. like sort the result, or do a select with a comparison clause, and in those case the nocase collation has been a successful workaround
philaupatte wrote: Sun Sep 19, 2021 12:09 pm What should I do, did not find much information within http://www.ventismedia.com/mantis/view.php?id=11083
Look at the first post from petr in that Mantis thread.
It is of use to only if:
  • your SQLite interface allows you load a new collation: see point #7 here ... you need to be able to register a new collation which you named IUNICODE ... the documentation shows how it is done in C, and the function is implemented in the Python interface for SQLite ... you need to check your own development environment
  • and you need to be aware that Petr has only made available a 32bit version of his collation DLL
IT is PIA that MediaMonkey did not address this issue with the MM5 rewrite. IUNICODE has been tripping up, and turning off, Developers in the MM community since forever.
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
philaupatte
Posts: 24
Joined: Thu Oct 01, 2015 2:52 am

Re: Error IUNICODE

Post by philaupatte »

Hola Barry4679 & Drakinite & all of you,
Many thanks for help.
I manage to get collate DDL and to load it using ".load" Sqlite function and to process it via start-process

Code: Select all

param()
$ErrorActionPreference = "Stop"
[int]$returnCode = 0
[int]$maxReturnCode = 0

Import-Module PSSQLite

try
{
 $command = "C:\Program Files (x86)\sqlLite\sqlite3.exe"
 $parameters = "/-init C:\PowerShell\test1.txt `"`""

 Start-Process -FilePath $command -ArgumentList $parameters -RedirectStandardOutput "C:\PowerShell\output.txt" -RedirectStandardError "C:\PowerShell\error.txt" -WindowStyle Hidden
 
} 
catch
{
 Write-Error $_.Exception.ToString()
}
return $maxReturnCode
where load is in test1.txt

Code: Select all

.load /PowerShell/SQLite3MMExt.dll
.open "/Users/phili/AppData/Roaming/MediaMonkey5/MM5.DB
SELECT ID FROM Songs where SongPath = ":\$Musique-$PlayLists\Salsa\Various-I Love Salsa-122-Sal A Bailar [by Víctor Manuelle].mp3"
Still the same no way to bypass IUNICODE problem and DLL is correctly loaded !!!

I did additional test and by the end the simpler query I can make returning one row with ID column associated to the song is:

Code: Select all

param()
$ErrorActionPreference = "Stop"
[int]$returnCode = 0
[int]$maxReturnCode = 0

Import-Module PSSQLite

try
{
 $dbMM5 = "C:\Users\phili\AppData\Roaming\MediaMonkey5\MM5.DB" 
 $playListSourceFileName = "C:\PowerShell\Bailes Primera.m3u" 
 $recordsArray = [IO.File]::ReadAllLines($playListSourceFileName)
 for ($i=0;$i -lt $recordsArray.count;$i++) {$songPath = $recordsArray[$i].split(":")
  $songPath = ":"+$songPath[1].replace(".mp3",".mp_") 
  $songPath = $songPath.replace("'","''") 
  $query = "SELECT ID, Extension FROM Songs WHERE SongPath LIKE '"+$songPath+"'"
  $sqlConn = @{Datasource = $dbMM5}
  [pscustomobject[]]$songsFetched = Invoke-SqliteQuery @sqlConn -Query $query
  foreach($item in $songsFetched) {write-host($SongPath)
   write-host($item.ID)}
 } 
} 
catch
{
 Write-Error $_.Exception.ToString()
}
return $maxReturnCode
I am pretty sure that this query works because of DB relational path not using primary and/or unique index that have the collate constrainte. I don't see other explanation. Now about IUNICODE error, well I know I can get around it. Not the best for sure but at least it allows to go forward.

Thanks
Barry4679
Posts: 2398
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: Error IUNICODE

Post by Barry4679 »

I don't know enough about Powershell, or whatever you are using, to understand why it is failing for you.

I do know that the dll gets around the IUNICODE error as illustrated by:
  • download the open source DB Browser for SQLite
  • open the MM5 database
  • Run this query ... Select Album from Songs where album = 'xxx';
  • you get the missing collation issue
  • use menu option Tools|LoadExtension to load Petrs dll
  • rerun the query ... runs OK now
I remain surprised that you are getting the collation error from a query which selects a text column, but does nothing which should require collation.
eq. close and reopen the open source browser ... the following query runs OK without the collation
select album from songs LIMIT 1

Does this offer you any assistance?

You could try sending a PM to PetrCBR. He is the MM Dev who works in this area, although he should have noticed this thread already.

See his post below, where he suggested that another approach is to use their SQL dll, instead of the vanilla SQL library, because theirs already already has some of their customisation baked into it, although I am not sure whether that includes IUNICODE.
viewtopic.php?p=480908#p480908
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
philaupatte
Posts: 24
Joined: Thu Oct 01, 2015 2:52 am

Re: Error IUNICODE

Post by philaupatte »

Hola Barry

From SQLITE Studio browser all my requests work perfectly. It is only when performing requests from my scripts.
Before including a request in script, I always test it under SQLITE Studio Browser. It is more convenient.

However, I finally make it work. In fact, in my text file .LOAD statement must be placed after open database statement and not before.

So now it works perfect no more IUNICODE.

Many thanks to all of you for helping me so much.
Post Reply