WebKit HTML Layout Engine + python-webkit modules for (python) addon developers?
#1
Lightbulb 
Hate to quote a rival project as an example, but Plex uses webkit as it's main tool for video plugins. Its a little cpu intensive, but drasticly reduces development time for javascript/AJAX heavy pages and the majority of video plugins. It seems to have also allowed plex addon development to have very quickly caught up with XBMC.

afaik, its all totally open-source?
i know there is already a project to build a browser in xbmc using webkit.
http://forum.xbmc.org/showthread.php?tid=71465

im a little confused about all the various incarnations (webkit, webkitqt4, webkitgtk)
and the various python modules that go with them.....
but these links seem to be relevant.
http://www.gnu.org/software/pythonwebkit/
http://webkit.org/
http://www.riverbankcomputing.co.uk/stat...ebkit.html
http://wiki.plexapp.com/index.php/Creati...ite_Config

also:
https://trac.webkit.org/wiki/PythonGuidelines says:
Quote:WebKit's Python scripts require Python 2.5 or higher to run.
but that won't be a problem in Eden (XBMC 11.0) Smile



Does anyone else think this would be a really helpful tool for video addon development? It seems to make complex scraping much more simple.

If so, how would it be implemented? Would it require installing the webkit engine to an OS directory or XBMC directory? Could this be done throught the new binary addon system? Webkit has versions for each major platform (Linux, OSX, Windows), so maybe it ought to be installed to an OS directory (outside of XBMC paths).

I expect the python modules that provide bindings to webkit engine could just be packaged as an installable (ie. similiar to this: script.module.mechanize.zip)

I came across webkit when searching for ways to implement a resolver for 2shared's javascript-heavy download link that didn't require hours of committed coding. It seems the perfect way to get round the lengths that video sites go to avoid people scraping/finding the direct vid links;
simulate a fully-functional browser and use webkit to find and pass the resulting file link to xbmc.
Reply
#2
webkit is good if you want to render entire webpages (or a part of them). i.e. you can use its plugins to be a video player instead of xbmc (which is what the flash players are in boxee etc, they use the webbrowser and flash to do video playback). XBMC is fully able to play the file if they can get ahold of the video (as seen on youtube etc. where we do not use flash or anything to play).

As such, you need to ask yourself what you really want. Do you want to browse the web or do you want to playback the content, the latter is what I personally want and for that you really don't need webkit. Webkit would arguably help with it but its not a requirement.

Reason against using webkit is that it drags in an enormous amounts of dependencies we really don't need, android for example uses webkit and to not drag in the dependencies they have ported the core to use their guitoolkit and their specific libs which android already depend on. This could obviously be done for XBMC also but I cant even begin to guess at the amount of workmonths it would be to do that (note that I say months but it may very well be years of workload...) And at that point you may ask yourself, can't that time be spent better elsewhere?

Plex just plain live with the dependencies as does boxee, we want to minimize dependencies so we can port more easily because at this point its frankly a pain to deal with all the dependencies and adding more would just hinder development of XBMC core.

That being said, addons might come to the rescue where webkit actually at some point might be able to be an addon and those wanting it could live with the dependencies.

All in all, for video playback it really is not needed and you could have other dependencies to deal with parsing js (spidermonkey for example) and the various other stuff which are needed to get ahold of the actual video.

Cheers,
Tobias
If you have problems please read this before posting

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.

Image

"Well Im gonna download the code and look at it a bit but I'm certainly not a really good C/C++ programer but I'd help as much as I can, I mostly write in C#."
Reply
#3
thanks for the in-depth answer, i have noticed that xbmc does seem to be a bit more zippy than boxee and plex, and i do appreciate the careful coding choice of avoiding too many dependencies and using xbmc's internal player instead of the webkit plugins.

one of the things that won me over to xbmc was it's lack of bloat. it runs smoothly on Atom based netbooks!

i think then that the main option would be to package python modules as addons and have webkit engine live on the OS as a separate entity from xbmc since it has packages for all platforms. (not included in xbmc directories or run inside xbmc etc.) i imagine this could form a last resort scraping tool (to get vidlinks, not to pass video) for addon developers.

i wonder if this is possible...
Reply
#4
//Do you want to browse the web or do you want to playback the content, the latter is what I personally //want and for that you really don't need webkit. Webkit would arguably help with it but its not a //requirement.

hi,topfs2
if i only want to play back the content.which software can help me?
maybe it is helpfule to explain my problem.I want to scraw some page from a site(use scrapy). But the content of this site is create by js,so i need someone to help me create the real html.
The site is http://www.talkcc.com
Reply

Logout Mark Read Team Forum Stats Members Help
WebKit HTML Layout Engine + python-webkit modules for (python) addon developers?1