[WIP] Addon system refactor for multiple programming languages - Overview/Status
#16
D-Bus might be present under Linux but not on OSX and Windows. Any solutions involving addons must be compatible with all three platforms, Linux, OSX and Windows.
Reply
#17
Just a quick update since I said I'd have xbmcgui done by the end of the weekend.

I spent my time over the weekend looking at SWIG.

alcoheca is right. It's perfect for this. I'm in the process of moving SWIG in place of my custom generator. Since I've refactored the Addon API into a non-addon-language specific piece (that is, pure native C++) it seems to fit readily into SWIG. I should be back up to where I was previously in short order, at which point I will check in the changes and then finish the xbmcgui refactor which I had left hanging.

I'll post a quick note when it's up there.

As always, please let me know what you guys think.
Reply
#18
I think itś great jfcaroll.... You bring XBMC to the next level
Reply
#19
davilla Wrote:D-Bus might be present under Linux but not on OSX and Windows. Any solutions involving addons must be compatible with all three platforms, Linux, OSX and Windows.

D-Bus is available for OS X and Windows, although it's less likely to be present (but it's not guaranteed on Unix either).

When the language-agnostic bindings are complete there may be bindings for languages not present on some systems, unless the plan is to include every language that can be bound? I hope not, because I'd prefer to see more "external" libraries moved out of the tree, but if it is, might as well throw in a D-Bus implementation too.

To jfcarroll: If, at some future time, more language bindings are added and an add-on is written in a language not supported on a client, how would that be indicated? Would the add-on show as gray in the browser with status "Language 'x' is required for this add-on.", or not show up at all? Doing either of those seems user-hostile ("I don't care what language it's written in, I just want to be able to install and play Merlin's Amazing Party Tetris!") Or is the plan to just include every language in the tree?
Reply
#20
dbrobins Wrote:To jfcarroll: If, at some future time, more language bindings are added and an add-on is written in a language not supported on a client, how would that be indicated? Would the add-on show as gray in the browser with status "Language 'x' is required for this add-on.", or not show up at all? Doing either of those seems user-hostile ("I don't care what language it's written in, I just want to be able to install and play Merlin's Amazing Party Tetris!") Or is the plan to just include every language in the tree?

If by "not supported" you mean that language is not available for that platform at all (is Tcl available for Windows?) then the facility that allows addon's to identify the platforms they are targeted for will specify that.

If you mean the particular computer doesn't have that scripting language installed (e.g. a Jvm isn't currently installed) then there's more than one way to solve it. I don't think that, for example, requiring the installation of a Jvm is all that hostile - but if that doesn't work for people, there are ways to include it as a prerequisite and have it installed automatically as a step in the process of installing the plugin.

I had been thinking about this and thought it might be a good idea for "language support modules" to be themselves "plugins" that if not installed, get automatically installed (or they can all be installed as part of the xbmc install). Then Python (as well as a Jvm, etc.) would be installed along with the SWIG bridge code, in shared libraries.

In any case the issue here is not that there isn't a possible solution, it's that there's several that we would need to pick from.
Reply
#21
Jfcarroll

Excellent work. If you don't mind can you post a patch with your changes for SWIG support.
The normal XBMC log IS NOT a debug log, to enable debug logging you must toggle it on under XBMC Settings - System or in advancedsettings.xml. Use XBMC Debug Log Addon to retrieve it.
Reply
#22
Thanks. I'm not quite there yet. I should be by the end of the week. I'll post back here with an update when it's available.
Reply
#23
jfcarroll Wrote:I had been thinking about this and thought it might be a good idea for "language support modules" to be themselves "plugins" that if not installed, get automatically installed (or they can all be installed as part of the xbmc install). Then Python (as well as a Jvm, etc.) would be installed along with the SWIG bridge code, in shared libraries.

This is how I see it too. Probably more useful on windows that any other OS, as I believe there was plans to support running the current scripts through the distro's libPython, rather than custom compiling and loading our own SO - might be out of the loop on that though..

The bridge code libraries would certainly be addons though

On this subject, won't your C++ API have binary compatibility issues with these 'bridging libs' loaded on-demand by c-pluff?
Reply
#24
jfcarroll Wrote:If by "not supported" you mean that language is not available for that platform at all (is Tcl available for Windows?) then the facility that allows addon's to identify the platforms they are targeted for will specify that.

If you mean the particular computer doesn't have that scripting language installed (e.g. a Jvm isn't currently installed) then there's more than one way to solve it. I don't think that, for example, requiring the installation of a Jvm is all that hostile - but if that doesn't work for people, there are ways to include it as a prerequisite and have it installed automatically as a step in the process of installing the plugin.

You misunderstood - what I was calling user-hostile was if a user couldn't get some addon they'd heard about because the language isn't installed, and installing it automatically (with user approval) as a dependent addon of its own seems a good solution (whereas not showing addons for unsupported languages seems bad because they'd wonder where they were if their friend can see them). It's fine to tell a developer that they're missing a prerequisite language and should install it, but since the system aims for ease of use, and doing that is beyond many users, more automation is called for. Which seems to be the same way you were thinking.

Quote:I had been thinking about this and thought it might be a good idea for "language support modules" to be themselves "plugins" that if not installed, get automatically installed (or they can all be installed as part of the xbmc install). Then Python (as well as a Jvm, etc.) would be installed along with the SWIG bridge code, in shared libraries.

In any case the issue here is not that there isn't a possible solution, it's that there's several that we would need to pick from.

Definitely. I was thinking along similar lines. Can a plugin be attached to multiple extension points (as c-pluff calls them)? E.g., a Python (or D-Bus) scraper would be classified both as a metadata extension and a Python (or D-Bus) extension.
Reply
#25
alcoheca Wrote:This is how I see it too. Probably more useful on windows that any other OS, as I believe there was plans to support running the current scripts through the distro's libPython, rather than custom compiling and loading our own SO - might be out of the loop on that though..

The bridge code libraries would certainly be addons though

On this subject, won't your C++ API have binary compatibility issues with these 'bridging libs' loaded on-demand by c-pluff?

Ack, that's a perilous minefield to tread. How many binary versions will an addon repository have to keep (of the paired bridge library and language library) to match all the relevant architectures and OS versions (and distributions)? Maybe it is better for some of the language dependencies to be pulled from known good sources (e.g., for Windows+perl, grab the ActivePerl 5.x installer and run it, if their license allows for that - but then there's a million Linux distros, even though Ubuntu? is the only officially supported one IIRC).

In cases where XBMC can't download and run the installer - probably unsupported Linux distros, where users can do for themselves - it can fall back to telling the user what they need to do ("Install perl 5.10+ and restart XBMC", hypothetically presuming that XBMC will detect the installation of the shared library on next start).
Reply
#26
dbrobins Wrote:Ack, that's a perilous minefield to tread. How many binary versions will an addon repository have to keep (of the paired bridge library and language library) to match all the relevant architectures and OS versions (and distributions)? Maybe it is better for some of the language dependencies to be pulled from known good sources (e.g., for Windows+perl, grab the ActivePerl 5.x installer and run it, if their license allows for that - but then there's a million Linux distros, even though Ubuntu? is the only officially supported one IIRC).

let us worry about this part. it's mostly planned, it will be a combination of static builds and package-kit usage for linux.
Reply
#27
Quote:On this subject, won't your C++ API have binary compatibility issues with these 'bridging libs' loaded on-demand by c-pluff?

I'm not sure what you're asking but maybe it's related to the issues I'm having.

I can't figure out why XBPython*.* is structured the way it is, rather than relying on the system's ability to link to the shared libraries. Why is the C Python API (PyObject*, etc) include a level of indirection through XBPython*.* when they are loaded? Is this legacy or a requirement of c-pluff?

Originally I tried to get everything running using configuring for external python and then linking with -lpython2.x but, while that seemed to work a bit, it was terribly unstable (subsequent calls to simply test scripts went from working, to failing, to crashing).

So I started moving all of the new calls into XBPython*.* which just doesn't seem right to me - but for now I'm continuing down that path.

Anyone want to suggest a different approach or clarify?

Thanks
Jim
Reply
#28
legacy. cpluff and python does not interact in any way currently. in an ideal world, we'd use the system interpreter. problem is we have to inject our vfs and currently this is done on link-time. ceros is working on a runtime solution, dunno how far he has gotten.
Reply
#29
OK. I got SWIG working. It was a pain in the ass and so far I only have the xbmc module done.

The biggest problem was there's a bug in SWIG. If anyone is interested, I reported it here:

https://sourceforge.net/tracker/?func=de...tid=101645

It's totally hacked together right now but you can take a look if you want. You'll need to get from my repository.

Code:
[email protected]:jimfcarroll/xbmc-multi-language-addons.git

I recreated the repository from scratch and ported my changes to the trunk. This repo has no branches. The master/origin should work fine.

Before this will even compile you'll need to install SWIG 2.0.0. If you want to do more than compile it you will need to patch it (you can compile without patching but you wont be able to run a script and have xbmc not SEGV). For now I stuck a patched version of 'pyrun.swg' in the root of the xbmc project. You'll need to take that and drop it in the swig-2.0.0/Lib/python directory of the SWIG source distro prior to building it.

In order to build you need to run configure like this:

Code:
./configure --enable-external-python --enable-newswiggyaddons

Then ... the build will fail in the link. :-) Cut-n-paste the last link line and add -lpython2.x (whatever version you have on your system) and run the modified link command.

Then you need to look in the SWIG generated code for the xbmc.py file and move that into a place your scripts will pick it up. I created a metadata python module "script.module.newaddon" and symbolic linked the file into the lib for the script.

Then it should run. No scripts will work that depend on more than the xbmc module but I got a test script working using the Player.

Now - if anyone is still reading... I needed to remove the chdir call in xbp_chdir in exports_python_linux.cpp. It was only compiled in there when USE_EXTERNAL_PYTHON was set but it doesn't seem to work as intended. It seems to move the directory of the entire xbmc process space and therefore once you navigate away from the script you're running, xbmc can't find anything. I have a comment in the file. Can someone let me know if they think I should file a bug report? Or if I this explanation is wrong.

Now I'm back to working on xbmcgui.

As usual, feedback helps.

Thanks
Jim
Reply
#30
I got callbacks working in SWIG. Tested them on the Player (still don't have xbmcgui finished). Take a look if you get a chance - it uses a class template carrying a templated pointer to a member function (VoidCallbackFunction in AddonCallbackFunction.cpp) :-)
Reply

Logout Mark Read Team Forum Stats Members Help
[WIP] Addon system refactor for multiple programming languages - Overview/Status1