urllib2/cookielib problem
#1
hi!

i've been trying to write a plugin for a resource that is protected by shibboleth authentication.

my code works fine when run outside of XBMC, but behaves differently when run as a plugin. i'm guessing this is due to the old version of python used in xbmc?

i request a page which sets a cookie and sends a 302 moved header but for some reason when run in xbmc the cookie is not sent back to the server after the redirection.

Code:
18:47:12 T:2858212208 M:735911936  NOTICE: reply:
18:47:12 T:2858212208 M:735911936  NOTICE: 'HTTP/1.1 302 Moved Temporarily\r\n'
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Date: Sat, 12 Mar 2011 18:47:21 GMT
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Set-Cookie: _idp_authn_lc_key=d6678451-3378-4de7-ab56-ed4551916fb0; Version=1; Path=/idp; Secure
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Location: https://idp.sussex.ac.uk:443/idp/Authn/UserPassword
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Content-Length: 0
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Connection: close
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Content-Type: text/plain
18:47:12 T:2858212208 M:735911936  NOTICE: send:
18:47:12 T:2858212208 M:735911936  NOTICE: 'GET /idp/Authn/UserPassword HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: idp.sussex.ac.uk:443\r\nConnection: close\r\nUser-agent: Python-urllib/2.4\r\n\r\n'
18:47:12 T:2858212208 M:735911936  NOTICE: reply:
18:47:12 T:2858212208 M:735911936  NOTICE: 'HTTP/1.1 200 OK\r\n'
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Date: Sat, 12 Mar 2011 18:47:21 GMT
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Set-Cookie: JSESSIONID=528520DAF1A3AC87C0FC23145CC36CE0; Path=/idp; Secure
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Content-Type: text/html
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Content-Length: 0
18:47:12 T:2858212208 M:735911936  NOTICE: header:
18:47:12 T:2858212208 M:735911936  NOTICE: Connection: close
18:47:12 T:2858212208 M:735911936  NOTICE: <_MozillaCookieJar.MozillaCookieJar[<Cookie PHPSESSID=6b8c067872ca04870ec083a29bb5145a for bobnational.net/>, <Cookie JSESSIONID=528520DAF1A3AC87C0FC23145CC36CE0 for idp.sussex.ac.uk/idp>]>

same code running in standalone python:
Code:
reply: 'HTTP/1.1 302 Moved Temporarily\r\n'
header: Date: Sat, 12 Mar 2011 18:32:11 GMT
header: Set-Cookie: _idp_authn_lc_key=3a08fe13-298c-4c7f-b278-42d6a1871979; Version=1; Path=/idp; Secure
header: Location: https://idp.sussex.ac.uk:443/idp/Authn/UserPassword
header: Content-Length: 0
header: Connection: close
header: Content-Type: text/plain
send: 'GET /idp/Authn/UserPassword HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: idp.sussex.ac.uk:443\r\nCookie: _idp_authn_lc_key=3a08fe13-298c-4c7f-b278-42d6a1871979\r\nConnection: close\r\nUser-Agent: Python-urllib/2.6\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Sat, 12 Mar 2011 18:32:12 GMT
header: Set-Cookie: JSESSIONID=560A3FE1FBE1E227767A5FEBA01C7566; Path=/idp; Secure
header: Content-Type: text/html
header: Connection: close
header: Transfer-Encoding: chunked
<_MozillaCookieJar.MozillaCookieJar[<Cookie PHPSESSID=2e267cd8cb42f85d2dfc29acd7d66e72 for bobnational.net/>, <Cookie JSESSIONID=560A3FE1FBE1E227767A5FEBA01C7566 for idp.sussex.ac.uk/idp>, <Cookie _idp_authn_lc_key=3a08fe13-298c-4c7f-b278-42d6a1871979 for idp.sussex.ac.uk/idp>]>

here is the code:
Code:
cj = cookielib.MozillaCookieJar(COOKIE_FILE)
if os.access(COOKIE_FILE, os.F_OK):
    cj.load(ignore_discard=True)
    
opener = urllib2.build_opener(urllib2.HTTPHandler(debuglevel=0), urllib2.HTTPSHandler(debuglevel=1), urllib2.HTTPCookieProcessor(cj))
urllib2.install_opener(opener)

response = urllib2.urlopen(URL)
print cj

the url i'm trying to GET is:
Code:
https://idp.sussex.ac.uk/idp/profile/Shibboleth/SSO?providerId=https%3A%2F%2Fbobnational.net%2Fsaml%2Fmetadata&shire=https%3A%2F%2Fbobnational.net%2Fbin%2Fopenathenssp_isapi.dll%3Ftype%3Dukfed%26dir%3Drev&target=http%3A%2F%2Fbobnational.net%2Fmybob.php

you can see that when run in xbmc, the _idp_authn_lc_key cookie doesn't get sent back to the server.

does anyone have any idea why this doesn't work in xbmc and how to work around it?

do you need any more info to help?

thanks in advance!
Reply
#2
ok, figured it out!

here's what i did in case ayone else has this issue.

it seems the default cookie policies must have changed. the 'Version=1' means it is a RFC 2965 cookie and they are discarded by default in the cookielib that ships with xbmc.

to fix it you need to set a custom cookie policy like this:

Code:
policy = cookielib.DefaultCookiePolicy(rfc2965=True, strict_rfc2965_unverifiable=False)    
cj = cookielib.MozillaCookieJar(COOKIE_FILE)
cj.set_policy(policy)

this was obvious once i worked out how to enable logging in cookielib:
Code:
import logging
logger = logging.getLogger("cookielib")
logger.addHandler(logging.StreamHandler(sys.stdout))
logger.setLevel(logging.DEBUG)

now on with writing the rest of the plugin Big Grin
Reply
#3
Hi t0mm0,

Did you get anywhere with the bobnational plugin? Will
Reply

Logout Mark Read Team Forum Stats Members Help
urllib2/cookielib problem0