Win Unicode Bug in most Addons - Printable Version +- Kodi Community Forum (https://forum.kodi.tv) +-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32) +--- Forum: Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=26) +--- Thread: Win Unicode Bug in most Addons (/showthread.php?tid=125492) |
Unicode Bug in most Addons - sphere - 2012-03-14 Hi, while reading the forum I found this thread: http://forum.xbmc.org/showthread.php?tid=124921&pid=1039311#pid1039311 I did a few tests and can confirm that it is very easy to reproduce and effects most addons - in a not so unrealistic scenario: - You just need an unicode letter in the xbmc path (which contains the windows-username on windows7). Not sure on the other OS's... You can easy reproduce this (or test your addons) with starting xbmc in portable mode (add "-p" to you xbmc start shortcut) from a xbmc folder which contains an unicode letter, "ä" or "é" for example. So just download xbmc again, install it to "C:\folder-with-unicode-ä-é\xbmc" and start "C:\folder-with-unicode-ä-é\xbmc\xbmc.exe -p". You can also test it with starting xbmc with a local windows username which contans unicode-letters. I cloned the eden plugins repo and got ~200 line matches without a real unicode decoding: Code: getAddonInfo('path') This effects all path's returned by getAddonInfo (because it returns string - not unicode) - you ALWAYS have to decode it properly. In most cases the addons are using this path for icons, something like: Code: ICON = os.path.join(ADDON.getAddonInfo('path'), 'icon.png') More bad is something like: Code: f = open(os.path.join( __settings__.getAddonInfo('profile'), "allmovies"),'wb') But it can even avoid some addons from running with something like this: Code: sys.path.append(os.path.join(Addon(ADDON_ID).getAddonInfo("path"), "resources", "lib" )) Maybe a better solution would be that xbmc would return unicode-instances (and not str)... Until that we have to do something like: Code: Addon.getAddonInfo('path').decode('utf-8') Don't get me wrong - my Addons are also affected... regards, sphere RE: Unicode Bug in most Addons - jmarshall - 2012-03-14 I don't believe it affects anything other than windows at this point, though that may possibly be due to most *nix users not having special-characters in their usernames. RE: Unicode Bug in most Addons - Martijn - 2012-03-14 Hmm haven't had any issues with Artwork Downloader concerning this. Did have some issues with show text in the dialog that threw that error and the JSON requests. But nothing in the cases you described EDIT: Ok i see your point. But this only happens in portable mode? One add-on even made from this: XBMC_Edén This: XBMC_Edén Seems lilke my add-on doesn't even care about this without any code intervention RE: Unicode Bug in most Addons - sphere - 2012-03-14 (2012-03-14, 00:26)Martijn Wrote: Ok i see your point. But this only happens in portable mode? Sorry to say this - but you are wrong. 1. No, this is not only happening in portable mode. This happens at least in one of these scenarios: a: You are on windows in portable mode with having xbmc installed in any path which contains a unicode letter b: You are on windows in "normal" mode - but your windows username contains any unicode username (think about "Renè, André, ...") 2. Of course is your addon "Artwork downloader" affected! It is even crashing in some situations because of this bug: Code: 07:44:19 T:3044 ERROR: Error Type: <type 'exceptions.WindowsError'> regards, sphere RE: Unicode Bug in most Addons - Martijn - 2012-03-14 Well at least I had no one complaining so far I had to do this: PHP Code: __addonpath__ = __addon__.getAddonInfo('path') for it not to fail. What about this one? PHP Code: THUMBS_CACHE_PATH = xbmc.translatePath( "special://profile/Thumbnails/Video" ) Should have the same problem I think. Would be nice if this could be fixed in XBMC core instead of every addon is gonna need to fix this RE: Unicode Bug in most Addons - amet - 2012-03-14 this is what I did in XBMC Subtitles(thx to chadoe), but it brought in more issues ... that chadoe fixed again testing it out now on OSX to see if it breaks anything EDIT: all good on OSX with above changes RE: Unicode Bug in most Addons - Popeye - 2012-03-14 What I have found my addons is that xbmc always returns utf-8 from e.g. keyboard and dialogs. By always converting to unicode for internal code use and then convert back to utf-8 for output problems are solved... If i were to use PHP Code: ICON = os.path.join(ADDON.getAddonInfo('path'), 'icon.png') PHP Code: ICON = os.path.join(unicode(ADDON.getAddonInfo('path'), 'utf-8'), 'icon.png') Also urllib.qoute and urllib.quote_plus is tricky since they don't handle uincode. RE: Unicode Bug in most Addons - jmarshall - 2012-03-14 Correct. ALL strings that XBMC gives python are currently utf8. I suspect XBMC can make it much easier on plugins by returning unicode strings instead, but this is obviously something that will break API (i.e. post Eden). RE: Unicode Bug in most Addons - Martijn - 2012-03-14 trac it so won't be forgotten and there can be some thought about this? RE: Unicode Bug in most Addons - Popeye - 2012-03-14 (2012-03-14, 22:40)jmarshall Wrote: Correct. ALL strings that XBMC gives python are currently utf8. An idea, why not a xbmc return encoding tag in the addon settings.xml defaulting to utf-8 with unicode as the option? RE: Unicode Bug in most Addons - jmarshall - 2012-03-15 It doesn't only affect addon settings though (and an add-on needn't have any settings) - basically every string we pass to python is utf8. RE: Unicode Bug in most Addons - sphere - 2012-03-16 So what is the "official recommended way"? - Should we (addon-authors) fix this? (80% of all addons are at least affected by not working picture-path-joins) - Ignore and add a FAQ entry to refer to "do not use windows user-names with special characters on win7 or use portable-mode instead"? - Wait for a core fix? (I guess only for post eden) regards, sphere RE: Unicode Bug in most Addons - jmarshall - 2012-03-16 Core fix is only post-eden, yes. Remember that it only affects paths per-se, so if you're writing to or reading from the filesystem then you'll want those paths handled. Depending on your add-on this may be minimal. RE: Unicode Bug in most Addons - sphere - 2012-03-17 Sure, for my 6-8 add-ons it is easy and I know what to do - I was just thinking about the other add-ons. I guess most devs still don't know about this. thanks |