script.module.urlresolver development

t0mm0 · 2011-07-17 18:27

hi,

i have just pushed an initial version of urlresolver to github. it is NOT ready to be used and not in a repo, but just for developers at the moment.

it is inspired by the work previously done by anarchintosh and unbehagan and includes anarchintosh's megaupload code.

the api is currently much simpler than their version - i don't know if all the rest of the stuff is actually needed - comments welcome.

look in test.py to find out how to use it. it is currently not tested from within xbmc.

included are plugins for

megaupload
novamov
putlocker
seeon.tv (to show it can be used for live stuff too)

assorted stuff to do includes

handle delays for megaupload
find a way of managing settings/login details for plugins so that each addon doesn't have to do that
lots more error handling
extend api (what would be useful)
write lots more plugins (best to hold off until api stabilised?)
lots of nice documentation
probably lots of other stuff i haven't thought of!

please let me know what you think, as if nobody is going to actually find this useful there is little point in continuing. all comments, ideas and pull requests are most welcome!

t0mm0

t0mm0 · 2011-07-25 02:06

in case anyone is interested, work is progressing in git.

i shuffled the interfaces a bit, so there are now separate interfaces to implement if the plugin requires login (SiteAuth) or settings (PluginSettings)

i have added a method for resolver plugins to have settings (such as megaupload username and password for example) which will be global and so users will only have to set them once for all addons that use the resolver. i don't know if the method i've used is the best way of doing this or if it is sensible, but it seems to work. please take a look and let me know what you think.

delay code (from anarchintosh) has been added so that megaupload works properly.

it is now tested in xbmc and working nicely. i have written a basic addon to test it including some basic tubeplus.me support just to have something visible to test with but this is not yet committed to git as it is in a bit of a mess. will get that sorted soon so that people can easily see it in action.

there is now an extra plugin to resolve videos from tubeplus.me episode pages. if there is more than one playable link it offers a choice. it is interesting because the tubeplus plugin uses the other plugins to resolve links it finds which shows the flexability of the system.

the novamov plugin was fixed as it broke due to site changes.

developers please let me know what you think of the progress so far and don't be shy about telling me i've done it all wrong (preferably if you can tell me how to do it better or send a patch!)

t0mm0

ps. i promise i will add lots of comments and write some docs once it gets more stable Wink

mrdally204 · 2011-07-25 20:30

I'm surprised there is little interest on this. Seems like it could make everything a bit easier going down the road of future add ons. Keep up the work my friend, maybe this bump will find someone (:

Temhil · 2011-07-26 23:44

Yes, that is a great job you are doing and I am pretty sure it will be reuse in a lot of addons in the future.
Thank you for developing this module!

anarchintosh · 2011-07-27 13:54

hey t0mm0 this looks great! if you have any queries or questions you can PM me.

i'm currently sitting on an updated megaup resolver (i wrote a few months ago) that i'd like to commit to your repo, could you add me with write access? cheers.

EDIT: don't worry, i will submit a pull request

t0mm0 · 2011-07-27 14:14

hi anarchintosh!

anarchintosh Wrote:hey t0mm0 this looks great! if you have any queries or questions you can PM me.

i'm currently sitting on an updated megaup resolver (i wrote a few months ago) that i'd like to commit to your repo, could you add me with write access? cheers.

EDIT: don't worry, i will submit a pull request

glad you like it so far! i simplified the interface quite a bit from your previous stuff as i wasn't sure it was really needed to expose all that stuff to people - my plan is to make it as simple to use as possible to encourage more people to have a go at writing stuff.

i also added the settings stuff which i think might come in useful, and the priority stuff is half done (the idea is that there will be a setting that says 'use the resolver i've given the highest priority rather than ask me which link to play')

i'm also planning to change the login stuff so that the addons can call a 'log_in_to_everything()' type function when they first load rather than the resolvers logging in every time they are run as it does at the moment.

a pull request would be great - it's all still in a bit of flux at the moment. i have a few changes i've made locally and not pushed to github yet but nothing major at the moment. having your megaupload and delay popup code was really great to get it going quickly so i could focus on the structure and plugin stuff. (can i just confirm that code you've just put 'copyleft' on without a specific licence is ok to use as GPLv3?)

hopefully once i release a test addon using it then it will be easier to try out and there will be more feedback from people. and once the interfaces are stable it should be really easy for people to add loads of resolver plugins very quickly.

thanks,

t0mm0.

cloom · 2011-07-31 09:26

Hello t0mm0,

I have a bunch of ideas for plugins ongoing and I can't wait to have your module working to publish my plugins.
I can't find the file test.py you are talking about, is it in GIT?
Can you post it here please? I would like to give it a try.

Thank you.

anarchintosh · 2011-07-31 15:15

@t0mm0

yeah GPLv3 is great. (better than v2 IMO). my later code i began to put GPLv3 in.
i think you were right about a simple structure. me and unbehagan went with a slighty overkill interface previously.

'log in to everything' sounds good. could maybe even execute this on XBMC start up using autoexec.py (Dharma) / Services (Eden)

Bear in mind that some things (ie. hotfile premium) have an API which just asks for the user + pass in the request url.
For those type resolvers it would still be good to rig together a simple html form filling (see megaup resolver) login routine (just as a way of verifying whether the user + pass are vaild)

once it is basically up and running i reckon it will be very popular and quickly attract a lot of resolver contributers. it's also possible to manually translate the jdownloader java plugins' source code for things like fileserve/sonic etc

there were some problems with the updated megaup resolver... so i'll have to hsve another look at it.

t0mm0 · 2011-07-31 20:10

cloom Wrote:Hello t0mm0,

I have a bunch of ideas for plugins ongoing and I can't wait to have your module working to publish my plugins.
I can't find the file test.py you are talking about, is it in GIT?
Can you post it here please? I would like to give it a try.

Thank you.

it is no longer in git as it is out of date.

at the moment, assuming you're using isPlayable on your listitems, you just need to do something like the following to resolve urls:

Code:

import urlresolver

stream_url = urlresolver.resolve(web_url)

if stream_url:

    xbmcplugin.setResolvedUrl(plugin_handle, True, 

                              xbmcgui.ListItem(path=stream_url))

else:

    xbmcplugin.setResolvedUrl(plugin_handle, False, 

                              xbmcgui.ListItem())

be aware that this module is currently in flux and stuff may change without warning! once it is more stable there will be proper docs available.

anarchintosh Wrote:@t0mm0

yeah GPLv3 is great. (better than v2 IMO). my later code i began to put GPLv3 in.

cool, i prefer v3 too Wink

anarchintosh Wrote:i think you were right about a simple structure. me and unbehagan went with a slighty overkill interface previously.

'log in to everything' sounds good. could maybe even execute this on XBMC start up using autoexec.py (Dharma) / Services (Eden)

interesting idea, although we would have to think about what would happen when people leave xbmc running (how often do these sites need new login cookies?)

anarchintosh Wrote:Bear in mind that some things (ie. hotfile premium) have an API which just asks for the user + pass in the request url.
For those type resolvers it would still be good to rig together a simple html form filling (see megaup resolver) login routine (just as a way of verifying whether the user + pass are vaild)

good plan

anarchintosh Wrote:once it is basically up and running i reckon it will be very popular and quickly attract a lot of resolver contributers. it's also possible to manually translate the jdownloader java plugins' source code for things like fileserve/sonic etc

yes i hope to do some solid work on this over the next few days as i have some time off work, and i hope once it is stable people will contribute resolver plugins as you say (it is pretty easy and i've tried to make it so that you need to write as little code as possible). like you say there is loads of code for various sites floating around that can be adapted, like jdownloader, pyload, or even get-flash-videos if anyone can understand that perl stuff Wink

it should make some of the requests for help with addons that have been popping up on the forums really easy too.

anarchintosh Wrote:there were some problems with the updated megaup resolver... so i'll have to hsve another look at it.

cool, no problem, the existing stuff seems to work well. the only thing i changed was the method names (you added __ in front of the public methods which meant they are 'private' and so not (easily) accessible from outside the module)

t0mm0

rogerthis · 2011-07-31 21:35

Would it be possible to add a flag for working as each module could stop working at any time and need to get updated?

So in my addon I could have something like this

Code:

if urlresolver.novamov.working == true and urlresolver.novamov.exists == true

    novamov=re.compile('href="http://www.novamov.com/video/(.+?)"').findall(html)

I could have all links ready to be scrapped, but it would only show working links.

mrdally204 Fan Posts: 388 Joined: Nov 2010 Reputation: 1	2011-07-25 20:30 Post: #3 I'm surprised there is little interest on this. Seems like it could make everything a bit easier going down the road of future add ons. Keep up the work my friend, maybe this bump will find someone (:
	find quote

Temhil Skilled Python Coder Posts: 395 Joined: Apr 2008 Reputation: 1 Location: Canada	2011-07-26 23:44 Post: #4 Yes, that is a great job you are doing and I am pretty sure it will be reuse in a lot of addons in the future. Thank you for developing this module! _____________________________ Repositories Installer: select and install unofficial repositories / TAC.TV: watch videos on TAC.TV Installer Passion-XBMC: Download and Install Add-ons (pre-Dharma only)
	find quote

anarchintosh Fan Posts: 550 Joined: Jul 2010 Reputation: 4	2011-07-27 13:54 Post: #5 hey t0mm0 this looks great! if you have any queries or questions you can PM me. i'm currently sitting on an updated megaup resolver (i wrote a few months ago) that i'd like to commit to your repo, could you add me with write access? cheers. EDIT: don't worry, i will submit a pull request (This post was last modified: 2011-07-27 13:56 by anarchintosh.)
	find quote

cloom Junior Member Posts: 3 Joined: Apr 2011 Reputation: 0	2011-07-31 09:26 Post: #7 Hello t0mm0, I have a bunch of ideas for plugins ongoing and I can't wait to have your module working to publish my plugins. I can't find the file test.py you are talking about, is it in GIT? Can you post it here please? I would like to give it a try. Thank you.
	find quote