State of the translation

  Thread Rating:
  • 2 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
viljoviitanen Offline
Member
Posts: 57
Joined: Apr 2010
Reputation: 1
Location: Jyväskylä, Finland
Post: #1
Hey all.

Browsing through the recent threads, it seems making things easier for translators is not going anywhere. And the state of the translation issues in the wiki ain't looking good either (a translation tool done in dotnet needing wine on on linux? c'mon.)

Especially with the different addons, there is a need for a centralized database from which common strings can be automatically fetched so tons of duplicate work and using different words in different addons for same things can be avoided.

XBMC project active members, devs, users, all.

Please.

Me, I'm not up for this task but I realise it is a _lot_ of work to set up this thing. I'm just fiddling with a single addon + few translations, being a native speaker of one satanic backwards language, Finnish.

We have "core" (xbmc itself) + tons of different utilities (addons), although we're of course not that big as the Gnome and KDE projects, but we still have the same issues those guys have, and we could also benefit a lot from taking a look how they do things.

To get things started, though, here's a tool written in python in a couple of minutes that can help checking the state of the translations of a single addon:

https://github.com/viljoviitanen/Simple-...ation-Tool

The tool parses all the files (assumed to be strings.xml files) given as arguments, loops through the files and stores all ids it finds, then loops through the ids and checks if all files have all the ids and complains if an id is missing.

Ideas for the tool are welcome, ideas with code are even more welcome! Smile
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #2
nice.

for starters i would love to use that tool to generate a wiki page.
another useful functionality would be the ability to register who translates a particular addon, with autogen'd mails when/if a translation changes.
find quote
Popeye Offline
Posting Freak
Posts: 909
Joined: Aug 2009
Reputation: 26
Location: Sweden
Post: #3
What about a web service listing all the official addons and their translations. The service could offer translators the option to add a new language and after translation the xml could get downloaded for a pull request to the addon authour (or automated...)...

sverigesradio | Pneumatic | SABnzbd | XBMC that just works - openelec
find quote
viljoviitanen Offline
Member
Posts: 57
Joined: Apr 2010
Reputation: 1
Location: Jyväskylä, Finland
Post: #4
Popeye Wrote:What about a web service listing all the official addons and their translations. The service could offer translators the option to add a new language and after translation the xml could get downloaded for a pull request to the addon authour (or automated...)...

Would be totally awesome, but I'm guessing quite a lot of work to implement... But many many things can be automated with github with some wizardry. Github even has a tool: http://hubot.github.com/ that's useful for this kind of things.

Still there'd be the need to check for common translations - I think all the translations in the addons have a LOT of common words.
find quote
viljoviitanen Offline
Member
Posts: 57
Joined: Apr 2010
Reputation: 1
Location: Jyväskylä, Finland
Post: #5
spiff Wrote:nice.

for starters i would love to use that tool to generate a wiki page.
another useful functionality would be the ability to register who translates a particular addon, with autogen'd mails when/if a translation changes.

Generated wiki page would list missing translations (for addons in official repo, so you can easily run it on the checked out git repo)? I can work on that, it should be easy.

As for the second thing, I think that's totally doable, and as I said in the other post, github even has a tool for github-based automation, hubot, which probably has even ready made stuff that could be used (github mentions it's doing at least translations with it!). But that's too much work for me. I'm a total newbie with git and github.
find quote
viljoviitanen Offline
Member
Posts: 57
Joined: Apr 2010
Reputation: 1
Location: Jyväskylä, Finland
Post: #6
I've been pointed (thanks ppic) to an awesome looking translation-making addon waiting for a dev to adopt it:

http://forum.xbmc.org/showthread.php?tid=117536

Combine this with an easy way to post modifications somewhere the addon dev(s) can pick it up.

Think about this: an user installs an addon of which translation not available in the language the user is using. Display a gentle plea that you can help translating the addon by installing the translation addon and translate a few strings... we'd have complete translations of all popular addons in no time.
find quote
viljoviitanen Offline
Member
Posts: 57
Joined: Apr 2010
Reputation: 1
Location: Jyväskylä, Finland
Post: #7
Ok, now my tool is more suited for real work.

It reports extra strings in translation and provides templates for missing strings.

First argument is "base" (English) to which the rest of the arguments are compared to.

Like this (example from fresh checked out github repo):[noparse]
viljo@dell:~/git/xbmc/addons/skin.confluence/language$ ~/git/Simple-XBMC-Translation-Tool/translationtool.py English/strings.xml Italian/strings.xml
Using base: English/strings.xml
Italian/strings.xml
<string id="31006">View Options</string>
<string id="31008">Fullscreen</string>
<string id="31050">Sort: Ascending</string>
<string id="31051">Sort: Descending</string>
<string id="31131"></string>
<string id="31132">Lyrics Add-on</string>
<string id="31133">Subtitle Add-on</string>
<string id="31134">Home Page Videos Submenu</string>
<string id="31135">Home Page Music Submenu</string>
<string id="31136">Home Page Pictures Submenu</string>
not in base:31250
not in base:31251
not in base:31252
<string id="31331">Album Details</string>
<string id="31390">Skin default</string>
<string id="31391">Skin default with no Caps</string>
<string id="31392">Arial based</string>
<string id="31900">Weather Maps</string>
<string id="31901">36 Hour Forecast</string>
<string id="31902">Hourly Forecast</string>
<string id="31903">Weekend Forecast</string>
<string id="31904">10 Day Forecast</string>
<string id="31905">Forecast</string>
<string id="31906">Maps & Video</string>
<string id="31907">Video Forecast [color=grey2](Fullscreen Playback)[/color]</string>
<string id="31908">Chance of Precipitation</string>
<string id="31909">Fetching forecast info...</string>
[/noparse]


It's suggested to run it like this to see the status of all languages:

translationtool.py English/strings.xml */strings.xml
(This post was last modified: 2012-01-08 18:51 by viljoviitanen.)
find quote
mad-max Offline
Posting Freak
Posts: 1,476
Joined: Jul 2010
Reputation: 55
Location: Germany
Post: #8
Just for you as a hint...there was an attempt to host all translations in an svn...
See here:
http://forum.xbmc.org/showthread.php?tid=93426

The project had 3 members...only translating into German...so I would say: FAIL!

But hey...maybe someone else picks up the idea again and we get this thing rolling...

@viljoviitanen:
If you didn't check the Confluence-Thread, you got a solution in there ;-)

cheers,
Max

[Image: obi3o55x.jpg]
[Image: 7c8w3bu8.png]

- - - Tribute to Metallica - - -

If I managed to help you, please click my reputation
find quote
Roman_V_M Offline
Fan
Posts: 514
Joined: Jun 2011
Reputation: 13
Location: Kyiv, Ukraine
Post: #9
As a localization professional I would like to share some thoughts. I'm not sure that all this can be implemented but still I'd want to describe a general concept.
What we need is basically an online CAT (Computer Assisted Translation) tool. It does not have to be as advanced as commercial tools like SDL TMS or Liox TWS, but it should be able to do at least the following:

1) Process strings.xml files, allowing to edit meaningful text while preserving xml tags.
2) Display source and target (translated) strings in one view.
3) Maintain a centralized translation database. Such database is called Translation Memory ™. This would allow to re-use existing translations, thus simplifying the process for new translations, and to maintain translation consistency between various modules, skins, plugins etc.
4) Include at least a basic search feature throughout a working TM.

The workflow, as I imagine it, should be the following:
1) A translator loads a source (English) strings.xml into the tool. This strings.xml can from be any module - the main GUI, a skin, a plugin etc.
2) The tool auto-populates existing translations from a TM for their language.
3) The translator adds new translated strings and/or edits existing ones.
4) He/she commits completed translation into the TM. Or translated string are committed in the process, allowing to re-use similar strings throughout the file.
5) He/she saves the target strings.xml for their language.

Also it would be useful if this (imaginary Big Grin) tool had the following features:
1) At least basic QA features like spellchecking.
2) Similarity (concordance) analysis with suggesting possible translation. E.g. if there is a new string "Press LEFT to see a trailer" and a TM already includes translation for "Press RIGHT to see a trailer", the tool would suggest this translation with marking the difference.

AFAIK there are at least a couple free opensource CAT tools and, I don't know, maybe they can be used for XBMC too.

AMD A4-3300 + AsRock A75M-ITX
XBMC-DSplayer 12.3 (custom build) on Windows 7
find quote
viljoviitanen Offline
Member
Posts: 57
Joined: Apr 2010
Reputation: 1
Location: Jyväskylä, Finland
Post: #10
Roman_V_M Wrote:As a localization professional I would like to share some thoughts. I'm not sure that all this can be implemented but still I'd want to describe a general concept.
...

I'm quite sure it could be implemented, it just requires hours and hours of work...

Unfortunately it would seem almost nobody is interested in getting the translation situation better (for evidence, see the number of people participating in this thread). Which sort of surprises me, because the core would seem quite well translated, but even the default skin (confluence) is much worse, and other skins and addons it's just random.

I think a centralized system like above combined with an in-xbmc-translation tool (there's already work done for such an addon, see my above post) would be a winning combo, provided there would be a gentle reminder when using an addon in a language it's not been translated to so we could get the masses of the users involved. I think just a tool for "professionals" will not be enough. It would at least have to be a website, and it would also be REALLY simple to use, and not require going thru many hoops before you can start contributing (e.g. registration like this forum).

Thinking about this, maybe google translation toolkit (http://support.google.com/translate/toolkit/ , http://translate.google.com/toolkit/TOS.html) could be used. Google has some experience on making things easy to use... Of course it would need some serious intregration to the xmbc strings.xml files.

But anyway I don't think any of this will happen unless some core xbmc dev starts to at least coordinating this. And they seem to have their hands full or work already, or rather, they are more interested in other parts of xbmc.
find quote
alanwww1 Offline
Team-XBMC Member
Posts: 1,353
Joined: Nov 2008
Reputation: 33
Location: Hungary
Post: #11
Guys !

I am taking over the handling of translations for xbmc. Tickets, pull requests, github stuff basically, but I am planning to write a completely new QT based multiplatform translator tool which can handle also context explanations for each strings.

With first step I start to make context comments for most of the strings in the English xml file (It will be a PITA :-( ). These comments will be handled of course with the new QT based translator tool.

At the end and if I have some help I plan to build a GIT client into the Utility so that it can handle things directly using GIT.

I'll keep you updated with further plans builds etc.

Cheers, Attila

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
viljoviitanen Offline
Member
Posts: 57
Joined: Apr 2010
Reputation: 1
Location: Jyväskylä, Finland
Thumbs Up   
Post: #12
alanwww1 Wrote:Guys !

I am taking over the handling of translations for xbmc. Tickets, pull requests, github stuff basically, but I am planning to write a completely new QT based multiplatform translator tool which can handle also context explanations for each strings.

With first step I start to make context comments for most of the strings in the English xml file (It will be a PITA :-( ). These comments will be handled of course with the new QT based translator tool.

At the end and if I have some help I plan to build a GIT client into the Utility so that it can handle things directly using GIT.

I'll keep you updated with further plans builds etc.

Cheers, Attila

Excellent news, thanks!

For sure the tool will handle addon translations as well as core xbmc?

But are you sure it's worth it to create a new tool, as opposed to using existing tools? Google translation tool looks pretty powerful to me... And if existing tools are for sure ruled out, if it was up to me, I'd probably try to make a web translator thing, not a qt one - modern javascript ui libraries makes creating even complex ui's pretty easy (I'm thinking http://jqueryui.com/). I'd prefer the web thing because it'll lower the barrier for contributing, and it would probably make git integration easier too, as a translator would not need to set up/understand git at all, it would be handled server side. As a bonus it would be possible to integrate the currently orphan xbmc translation addon to the central system quite easily - you could even allow completely anonymous "suggestions", and then have a logged in "moderator" accept them. (I still think it would be best if creating/correcting translations would require as little effort from the potential translator as possible...)

And if you were to do it the web way, you'd have at least one developer willing to help :-) I do python, php and even perl (I'm so old I still think perl is cool, like it was 10 years ago :-)). Java, C++/C and ruby I don't do. Javascript/node.js... I'm not sure.

OR you could have the qt client contact (optionally?) some kind of cental web system so fetching and updating translations would be super easy. Hmm... I think this is necessary, because forcing a translator to use git is not cool. Git is powerful, github even more so, but grokking all that is not easy, even if you have some version control system experience. (I started using cvs in 1995, and lately subversion, but still...)

Anyway the context idea is very good, and obviously necessary to create quality translations easily. Otherwise you'll need to be very familiar with the whole thing that's being translated. Maybe if you do some "context" strings at first and commit them in git, then other people can help when they have seen what it's about, so it'll be a smaller PITA to deal with.
find quote
alanwww1 Offline
Team-XBMC Member
Posts: 1,353
Joined: Nov 2008
Reputation: 33
Location: Hungary
Post: #13
Hi viljoviitanen !

It is really good to know that I can expect a help on this. The ideas and points are really good what you wrote. The QT way was just because I know C++ but absolutely no experience with web programming languages. I will reconsider the best way to go for. I'll leave this discussion open so that we can really find the best possible solution.

I am really count on the help you offered and really appreciate it.

About the context commenting. I am sure I'll need all the help I can get. From team members and as well as users, translators.

Thanks, again, we'll keep in touch !

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
viljoviitanen Offline
Member
Posts: 57
Joined: Apr 2010
Reputation: 1
Location: Jyväskylä, Finland
Big Grin   
Post: #14
Ok, I think I have a winning idea here (Charlie Sheen type winning, dunno Smile)

I'm still liking the javascript, 'cos with modern html5 thingies you can probably work also with local files. So we can (at first) work with a website, but later also add local support. (I'm not an html5 expert, but you can do wonders with modern browsers.) For the "local" use, we could probably just target chrome/chromium, it's popular and cross platform enough.

Then, the app. My great idea is to generate a really really big "translation memory" from freely available gnome, kde, firefox.. translations in addition to existing xbmc translations. Store the "where" part in the database, as to provide reference. Set up the search so that xbmc specific strings are displayed first.

Use json as internal format, only convert to xbmc xml when "exporting" the translated string.

And since we're talking about xbmc, changes in the core translations could even probably make an automated pull request, http://develop.github.com/p/pulls.html .

Make all this configurable so that other projects than xbmc can use the tool as well, and join developing it.

Winning!

Or maybe there is such a tool already, but at least the gnome guys, who have "some" experience in translating, do not seem to have such a tool: http://live.gnome.org/TranslationProject...ranslating

---

Also, alanwww1, I suggest to look if the "context" part in xbmc could be an attribute, like so:

Code:
<string id="0" contexthint="main menu">Programs</string>
then parsing the xml to the tool can be done more easily than if it's a comment. I just hope xbmc accepts such xml.

Alternative would be e.g.

Code:
<contexthint id="0">main menu</contexthint>
or if even that's not possible, then maybe just store the context hints in a separate file, using the same format. Anyway xml comments probably is not a good idea because of the parsing problem.
find quote
alanwww1 Offline
Team-XBMC Member
Posts: 1,353
Joined: Nov 2008
Reputation: 33
Location: Hungary
Post: #15
Thanks. The idea is great. I found extjs pretty impressive. TVHeadend uses it and it is great. http://www.sencha.com/products/extjs/examples/

What do you think ?

Except I would keep the xml format for the internal format.

To have the context hints as attributes was also in my mind. I will talk to the team members about this.

Thanks

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
Post Reply