UTF-8 vs. UTF-8-MAC support in XBMC for Mac OS X?

  Thread Rating:
  • 2 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
stillness2 Offline
Junior Member
Posts: 9
Joined: Apr 2008
Reputation: 0
Location: Russia, Moscow
Question  UTF-8 vs. UTF-8-MAC support in XBMC for Mac OS X?
Post: #1
Hello..
I did not understand why Russian letter 'Й' ('I shortly' or [ij]) displayed as 'И' and square, when I see file list in XBMC .. I found this..
From wikipedia : "The Mac OS X Operating System uses canonically decomposed Unicode, encoded using UTF-8 for file names in the filesystem. This is sometimes referred to as UTF-8-MAC. In canonically decomposed Unicode, the use of precomposed characters is forbidden and combining diacritics must be used to replace them."
And link http://developer.apple.com/qa/qa2001/qa1173.html


How to solve this problem described here http://developer.apple.com/qa/qa2001/qa1235.html

If you wish I can make patch.. Smile
find quote
Halligalli Offline
Junior Member
Posts: 14
Joined: Apr 2008
Reputation: 0
Location: Sweden
Post: #2
Have you tried the latest version? I experienced myself a similar issue before, where the Swedish letter Ä would be displayed like A and the two dots beside it, not above.
find quote
stillness2 Offline
Junior Member
Posts: 9
Joined: Apr 2008
Reputation: 0
Location: Russia, Moscow
Post: #3
0.5.0b2 Fix this !
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,230
Joined: Oct 2003
Reputation: 177
Post: #4
One thing to watch here:

We use UTF8 inside our XML files (smartplaylists for instance, plus many more) which may or may not be UTF8-MAC.

It won't matter if they're written by XBMC (as I presume they're written in UTF8-MAC) but it might make a difference for some of the stuff not written from XBMC, such as strings.xml and so on.

Cheers,
Jonathan

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
stillness2 Offline
Junior Member
Posts: 9
Joined: Apr 2008
Reputation: 0
Location: Russia, Moscow
Post: #5
UTF8-MAC only used for encoding filename in VFS (OS X file system API)!!!!

"In Mac OS X's VFS API file names are, by definition, canonically decomposed Unicode, encoded using UTF-8" from http://developer.apple.com/qa/qa2001/qa1173.html .

In another cases everything must be ok.
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,230
Joined: Oct 2003
Reputation: 177
Post: #6
A grep through the source indicates UTF8-MAC is being used in all utf8 -> wchar conversions, which is happened whenever something is displayed in the UI, as we store all strings internally as utf8.

From my understanding, it appears we only want to use UTF8-MAC to be used when we are grabbing filenames from the osx API. Thus, we actually want to do a UTF8-MAC -> UTF8 conversion at that point, and leave everything else in normal UTF8.

I'll mail elan and let him know in case he doesn't see it here.

Cheers,
Jonathan

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
stillness2 Offline
Junior Member
Posts: 9
Joined: Apr 2008
Reputation: 0
Location: Russia, Moscow
Post: #7
Yes, you are right. Simply, I did not see last sources. Smile
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,230
Joined: Oct 2003
Reputation: 177
Post: #8
Have chatted with Elan, and he's informed me that UTF8-MAC takes care of both cases, so things should be rosy with the current setup Smile

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
adinatha Offline
Junior Member
Posts: 31
Joined: Apr 2009
Reputation: 0
Location: Biatorbágy, Hungary
Post: #9
What about folder names in OS X? It seems to me that folder names are in decomposed Unicode while filenames are in precomposed form: diacritics are misadjusted (or don't display at all eg. "ö" -> "o ", depending on the specific GUI font) in folder names when they look perfect in filenames.

Is there any simple solution to this issue?

Thx
find quote
davilla Offline
Retired-Team-XBMC Developer
Posts: 11,508
Joined: Feb 2008
Reputation: 64
Post: #10
file a trac ticket with info on how to reproduce the error and this will be addressed.


MediaInfo : http://mediainfo.sourceforge.net/
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
find quote
kimp93 Offline
Aeon Group
Posts: 157
Joined: Mar 2004
Post: #11
@adinatha

Have you checked recent build of XBMC?
there was a fix for decomposed unicode. http://trac.xbmc.org/ticket/5658
find quote