Hulu Plugin Development Thread - Developers only!

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
rwparris2 Offline
Team-XBMC Python Developer
Posts: 1,333
Joined: Jan 2008
Reputation: 2
Location: US
Post: #46
Locutus73 Wrote:Here’s a simple PERL script to get auth parameter from the decrypted pid
Code:
#!/usr/bin/perl
#
use Digest::MD5 qw(md5_hex);
$pid=”@ARGV”;
$pid_suffix=’****************************************************************​’;
$extended_pid=”$pid$pid_suffix”;
$auth=md5_hex($extended_pid);
print $auth;
The asterisks has to be changed with a “secret string” that is embedded in http://www.hulu.com/player.swf
As highlandsun said Flare is a useful tool to decompile swf files and getting stuff…

does the auth string change frequently or...?

Always read the XBMC online-manual, FAQ and search and search the forum before posting.
For troubleshooting and bug reporting please read how to submit a proper bug report.

If you're interested in writing addons for xbmc, read docs and how-to for plugins and scripts ||| http://code.google.com/p/xbmc-addons/
find quote
jonm42 Offline
Senior Member
Posts: 177
Joined: Feb 2009
Reputation: 0
Location: Portland, OR
Post: #47
rwparris2 Wrote:does the auth string change frequently or...?

As soon as they read this thread. As soon as they read it. I think the path of embedding a (very) lightweight browser like framework would be better in the long run -- they can't break the user experience for running in a browser like environment. Can we do that with AIR/Webkit?
find quote
rwparris2 Offline
Team-XBMC Python Developer
Posts: 1,333
Joined: Jan 2008
Reputation: 2
Location: US
Post: #48
jonm42 Wrote:As soon as they read this thread. As soon as they read it. I think the path of embedding a (very) lightweight browser like framework would be better in the long run -- they can't break the user experience for running in a browser like environment. Can we do that with AIR/Webkit?

I'm not saying it's a bad idea, I think webkit is a great idea, but Hulu regularly breaks boxee & plex who both use webkit to display hulu content.

Always read the XBMC online-manual, FAQ and search and search the forum before posting.
For troubleshooting and bug reporting please read how to submit a proper bug report.

If you're interested in writing addons for xbmc, read docs and how-to for plugins and scripts ||| http://code.google.com/p/xbmc-addons/
find quote
jonm42 Offline
Senior Member
Posts: 177
Joined: Feb 2009
Reputation: 0
Location: Portland, OR
Post: #49
rwparris2 Wrote:I'm not saying it's a bad idea, I think webkit is a great idea, but Hulu regularly breaks boxee & plex who both use webkit to display hulu content.

Do we know if it's malicious, i.e., on purpose?
find quote
rwparris2 Offline
Team-XBMC Python Developer
Posts: 1,333
Joined: Jan 2008
Reputation: 2
Location: US
Post: #50
jonm42 Wrote:Do we know if it's malicious, i.e., on purpose?

The Hulu thread on the plex forums says they're '100%' sure. Really it doesn't make sense that they would constantly be changing back-end stuff on the site if it wasn't to purposely stop people from leeching their content (even if they leech the commercials at the same time :rolleyesSmile.

Always read the XBMC online-manual, FAQ and search and search the forum before posting.
For troubleshooting and bug reporting please read how to submit a proper bug report.

If you're interested in writing addons for xbmc, read docs and how-to for plugins and scripts ||| http://code.google.com/p/xbmc-addons/
find quote
n9mjg Offline
Junior Member
Posts: 3
Joined: Jan 2009
Reputation: 0
Post: #51
jonm42 Wrote:Has anyone checked out the Front Row plug in (appliance) called Understudy? May have some potential. Go see http://code.google.com/p/understudy/sour...trunk/Hulu ...

Yes, and several of us have been encouraging the developer to port it to the apple tv. Understudy works well on my MacBook Pro.
find quote
jonm42 Offline
Senior Member
Posts: 177
Joined: Feb 2009
Reputation: 0
Location: Portland, OR
Post: #52
rwparris2 Wrote:The Hulu thread on the plex forums says they're '100%' sure. Really it doesn't make sense that they would constantly be changing back-end stuff on the site if it wasn't to purposely stop people from leeching their content (even if they leech the commercials at the same time :rolleyesSmile.

That makes no sense to me either. I've worked on large-scale websites before (WebMD anyone?) and you generally cannot justify the IT overhead to just go fiddle for that percent of your traffic. Made no economic sense then, makes even less now.

Ahem. So has anyone managed to get the secondary key stuff wrapped up in a package we can deploy? Or is it still too much of a moving target?
find quote
jonm42 Offline
Senior Member
Posts: 177
Joined: Feb 2009
Reputation: 0
Location: Portland, OR
Post: #53
I am in the midst of trying to simplify the plugin by writing some common routines that stuff can call like addEpisodes, etc. Right now I'm looking at the RSS (_rss.py) module and am having some very odd results.

This snippet works for each and every RSS feed I give it:

Code:
xmlsoup = BeautifulStoneSoup(common.getHTML( common.args.url ))

and this one only works for highest rated, and just waits for a while and then returns nothing at all:

Code:
content=common.getHTML(common.args.url)

This particular snippet has been used elsewhere to grab non-RSS pages and works like a champ.

I've tried working with different URLs to get to the feed (see examples in common.py for the various forms) but no luck. I've tried wrapping the second form as an argument to another call. No luck. Nothing in the log. It just terminates the python thread as being idle, and nothing comes back.

Ideas, debugging thoughts, etc. quite welcome. Thanks.
find quote
rwparris2 Offline
Team-XBMC Python Developer
Posts: 1,333
Joined: Jan 2008
Reputation: 2
Location: US
Post: #54
That doesn't really make any sense, are you sure the url is what you actually want? As I'm sure you can see, the first snippet you posted does exactly the same as the second one, plus a little more.

jonm42 Wrote:I am in the midst of trying to simplify the plugin by writing some common routines that stuff can call like addEpisodes, etc. Right now I'm looking at the RSS (_rss.py) module and am having some very odd results.

This snippet works for each and every RSS feed I give it:

Code:
xmlsoup = BeautifulStoneSoup(common.getHTML( common.args.url ))
and this one only works for highest rated, and just waits for a while and then returns nothing at all:

Code:
content=common.getHTML(common.args.url)
This particular snippet has been used elsewhere to grab non-RSS pages and works like a champ.

I've tried working with different URLs to get to the feed (see examples in common.py for the various forms) but no luck. I've tried wrapping the second form as an argument to another call. No luck. Nothing in the log. It just terminates the python thread as being idle, and nothing comes back.

Ideas, debugging thoughts, etc. quite welcome. Thanks.

Always read the XBMC online-manual, FAQ and search and search the forum before posting.
For troubleshooting and bug reporting please read how to submit a proper bug report.

If you're interested in writing addons for xbmc, read docs and how-to for plugins and scripts ||| http://code.google.com/p/xbmc-addons/
find quote
jonm42 Offline
Senior Member
Posts: 177
Joined: Feb 2009
Reputation: 0
Location: Portland, OR
Post: #55
rwparris2 Wrote:That doesn't really make any sense, are you sure the url is what you actually want? As I'm sure you can see, the first snippet you posted does exactly the same as the second one, plus a little more.

I'm passing the same URL to the one that works as to the one that doesn't.
find quote
Locutus73 Offline
Junior Member
Posts: 18
Joined: Jan 2007
Reputation: 0
Post: #56
Hi, I just wrote a set of scripts useful to retrive Hulu’s pid and auth parameters.
These tools can be used as helpers for applications like get_iplayer http://linuxcentre.net/getiplayer/
List of contents:
hulu-get-keys.c : Based on decswf.c by hyc@highlandsun.com, this program must be compiled against
swfdec-0.9.2 and is useful to retrive pid encryption keys from enc.swf file
downloaded from Hulu.
hulu-get-keys.pl : This script first downloads player.swf from Hulu, decompiles it using flare (you
must download the command line version from http://www.nowrap.de/flare.html),
retrieves the auth encryption key from the decompiled source and stores it in
hulu.auth.keys file; then it downloads enc.swf from Hulu, uses hulu-get-keys to
retrieve pid encryption keys and store them in hulu.pid.keys file.
hulu-get-pid.pl : This script, based on a script by Andrej Stepanchuk, accepts Hulu’s encrypted
pid as parameter and, using hulu.pid.keys files, returns decrypted pid on stdout.
hulu-get-auth.pl : This script accepts Hulu’s decrypted pid as parameter and, using hulu.auth.keys
files, returns auth parameter on stdout.
hulu-get-pid-auth.pl : This script accepts Hulu’s decrypted pid as parameter and, using both
hulu-get-pid.pl and hulu-get-auth.pl, returns both decrypted pid and auth
parameter on stdout. This script can be used as get_iplayer helper using its
--hulu-decrypt-pid parameter.
You can download the package from http://www.megaupload.com/?d=G1ZKIOHQ

P.S.: the scripts are in perl, but it can be useful as a base for python scripts or called by python scripts.
find quote
highlandsun Offline
Senior Member
Posts: 152
Joined: Mar 2009
Reputation: 0
Post: #57
Cute. Just a small note - Andrej's original script had two bugs in it; the version you're using is the one I fixed.
find quote
jonm42 Offline
Senior Member
Posts: 177
Joined: Feb 2009
Reputation: 0
Location: Portland, OR
Sad   
Post: #58
Anyone? The actual GET call looks like

Code:
def getHTML( url ):
    print 'HULU --> common :: getHTML :: url = '+url
    cj = cookielib.LWPCookieJar()
    if os.path.isfile(COOKIEFILE):
        cj.load(COOKIEFILE)
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    opener.addheaders = [('Referer', 'http://hulu.com'),
                         ('Content-Type', 'application/x-www-form-urlencoded'),
                         ('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14')]
    usock=opener.open(url)
    response=usock.read()
    usock.close()
    if os.path.isfile(COOKIEFILE):
        cj.save(COOKIEFILE)
    return response

which looks quite reasonable to me, and, as mentioned, works all over the places for other web content.
find quote
rwparris2 Offline
Team-XBMC Python Developer
Posts: 1,333
Joined: Jan 2008
Reputation: 2
Location: US
Post: #59
jonm42 Wrote:Anyone? The actual GET call looks like

Code:
def getHTML( url ):
    print 'HULU --> common :: getHTML :: url = '+url
    cj = cookielib.LWPCookieJar()
    if os.path.isfile(COOKIEFILE):
        cj.load(COOKIEFILE)
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    opener.addheaders = [('Referer', 'http://hulu.com'),
                         ('Content-Type', 'application/x-www-form-urlencoded'),
                         ('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14')]
    usock=opener.open(url)
    response=usock.read()
    usock.close()
    if os.path.isfile(COOKIEFILE):
        cj.save(COOKIEFILE)
    return response
which looks quite reasonable to me, and, as mentioned, works all over the places for other web content.

something else in your code is going wrong. what you're describing doesn't make sense. If foo() causes problems bar(foo()) will too, no matter what.

do this to prove it to yourself.
Code:
content=common.getHTML(common.args.url)
print 'getHTML successful'

Always read the XBMC online-manual, FAQ and search and search the forum before posting.
For troubleshooting and bug reporting please read how to submit a proper bug report.

If you're interested in writing addons for xbmc, read docs and how-to for plugins and scripts ||| http://code.google.com/p/xbmc-addons/
find quote
jonm42 Offline
Senior Member
Posts: 177
Joined: Feb 2009
Reputation: 0
Location: Portland, OR
Sad   
Post: #60
Here's the actual code -- plug it in as the _rss.py module in Hulu/resources/lib and alternate which method is called. Tell me what the results are. The only one that works for me in both is highest rated, the rest, not so much...

Code:
import xbmcplugin
import common
import sys
import re

from BeautifulSoup import BeautifulStoneSoup, BeautifulSoup

class Main:

    def __init__( self ):
        if common.args.mode == 'RSS_Shows':
            self.activateShowPage()
        else:
            xbmcplugin.setContent(int(sys.argv[1]), 'tvshows')
            self.addListings()
            #self.addShowList()
            xbmcplugin.endOfDirectory( handle=int( sys.argv[ 1 ] ))


    def activateShowPage( self ):
        import xbmc, urllib
        print 'executing--> %s?url="%s"&mode="TV_Seasons"&name="%s"&fanart="%s"&plot="%s"&genre="%s")' % (sys.argv[0], urllib.quote_plus(common.args.url), common.args.name, urllib.quote_plus(common.args.fanart), common.args.plot, common.args.genre)
        xbmc.executebuiltin('XBMC.activatewindow(MyVideoLibrary,%s?url="%s"&mode="TV_Seasons"&name="%s"&fanart="%s"&plot="%s"&genre="%s")' % (sys.argv[0], urllib.quote_plus(common.args.url), common.args.name, urllib.quote_plus(common.args.fanart), common.args.plot.replace(',',' ').replace('"',''), common.args.genre))


[color=red]    def addShowList( self ):[/color]
        print "\n\n adding show list \n\n"
        
        content=common.getHTML(common.args.url)
        
        # name, url, thumb, plot -- plot can have embedded newlines, which do not compute...
        items=re.compile('<item>.+?<title>(.+?)</title>.+?<link>(.+?)#.+?<img src="(.+?)".+?</a><p>(.+?)</p>', re.DOTALL).findall(content)
        mode=('RSS_Play', 'RSS_Shows')[common.args.name=='Recently Added Shows']
        genre=common.args.name
        
        for name, url, thumb, plot in items:
            fanart='http://assets.hulu.com/shows/key_art_' + name.split(':')[0].replace('-','_').replace(' ','_').replace('\'','').replace('"','').replace(',', '').lower() + '.jpg'
            # display show plot if asked for; we've already extracted it from the page above, so no overhead
            plot=('', plot.replace('\n', '\r'))[common.settings['get_show_plot']]

            common.addDirectory(common.cleanNames(name), url, mode, thumb, thumb, fanart, plot, genre)


[color=red]    def addListings ( self ):[/color]
        print "\n\n adding listings \n\n"

        xmlsoup = BeautifulStoneSoup(common.getHTML( common.args.url ))

        items = xmlsoup.findAll('item')
        for item in items:
            name = item.title.contents[0]
            url  = item.guid.contents[0]
            try:
                try:
                    p = re.compile('&lt;p&gt;(.+?)&lt;/p&gt;')
                    plot = p.findall(item.description.contents[0])[0]
                except:
                    p = re.compile('<p>(.+?)</p>')
                    plot = p.findall(str(item.description))[0]
            except:
                plot = 'Unavaliable'
            try:
                p = re.compile('media:thumbnail.+?url="(.+?)"')
                thumb = p.findall(str(item))[0]
            except:
                thumb = ''
            try:
                fanart = 'http://assets.hulu.com/shows/key_art_'+name.split(':')[0].replace('-','_').replace(' ','_').replace('\'','').replace('"','').lower()+'.jpg'
            except:
                fanart = ''

            genre = common.args.name

            if common.args.name == 'Recently Added Shows':
                common.addDirectory(name, url, 'RSS_Shows', thumb, thumb, fanart, plot, genre)
            else:
                common.addDirectory(name, url, 'RSS_play', thumb, thumb, fanart, plot, genre)

Thanks for reading this far. Any hints would really really be appreciated, as I'm seeing this behaviour elsewhere now, too. Maybe I have a ghost in the machine Frown.
find quote
Post Reply