How do you send cookies with urllib?

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
SofaKng Offline
Member+
Posts: 385
Joined: Mar 2005
Reputation: 0
Post: #1
i need to send cookies to a website that i'm connecting to and i was told that i could use cachedhttp.py to accomplish this. i've tried searching and looking at the documentation but i still can't figure it out.

can anybody help me? all i want is to send some cookies (like 4) every time i connect with urllib. (or i could switch to urllib2 or whatever....)
find quote
Bernd Offline
Senior Member
Posts: 211
Joined: Mar 2005
Reputation: 0
Post: #2
i encourage every xbmc script author to use cachedhttp.py!!!
its an easy and ready to use lib which provides lots of functionality ready to use incl cookies, user agent and timeouts.

bernd
find quote
SofaKng Offline
Member+
Posts: 385
Joined: Mar 2005
Reputation: 0
Post: #3
is there any documentation for this? (or really basic examples?)

i've taken a look at it but couldn't figure out how to use it...
find quote
BigBellyBilly Offline
Skilled Python Coder
Posts: 959
Joined: Feb 2005
Reputation: 2
Location: UK
Post: #4
it can be done with urllib.urlopener()

Quote:cookies = [
['cookie', 'cfid=768572'],
['cookie', 'cftoken=45453733'],
['cookie', 'person_id=1686446'],
['cookie', 'token=1ff4ddc946758852308e4fc869486fc2']
]

agent = urllib.urlopener()
for cook in cookies:
name, value = cook
agent.addheader(name, value)

f = agent.retrieve(url, file)
find quote
SofaKng Offline
Member+
Posts: 385
Joined: Mar 2005
Reputation: 0
Post: #5
after that would i just do:

f = agent.urlopen('http://www.myurl.com')
data = f.read()
...blah...

?
find quote
hattmall Offline
Junior Member
Posts: 32
Joined: Nov 2005
Reputation: 0
Post: #6
can anyone post an updated code example. I am unable to get this to work. This is my first attempt at a python script and I am piecing together from tutorials, etc.

Quote:import xbmc, xbmcgui, urllib, urllib2 , re
from string import split, replace, find


Base_URL = "http://ftp.iinet.net.au/pub/games/movies/Red_Vs_Blue/Season1/"

LinkDescription = []
LinkURL = []
cookies = [
['cookie','user=XXXXXX-%59%44%73%45%46%65%4B%4C%76%53']
]

class MyClass(xbmcgui.Window):
def __init__(self):
self.list = xbmcgui.ControlList(200, 150, 300, 400)
self.addControl(self.list)
self.list.addItem('Item 1')
self.list.addItem('Item 2')
self.list.addItem('Item 3')
self.setFocus(self.list)


def onAction(self, action):
if action.getButtonCode() == 275:
self.close()

if action.getButtonCode() == 257:
self.list.addItem('1')
WebSock = urllib.urlopen(Base_URL) # Opens a 'Socket' to URL
WebHTML = WebSock.read() # Reads Contents of URL and saves to Variable
WebSock.close()

self.list.addItem('2')

Temp_Web_URL = re.compile('<a href=["](.*)[.]zip["]>', re.IGNORECASE).findall(WebHTML) # Using find all mentions of stuff using regexp to use wildcards
Temp_Web_Desc = re.compile('<a href=["].*[.]zip["]>(.*)</a>').findall(WebHTML) # find it

self.list.addItem('3')

for urls, desc in zip(Temp_Web_URL,Temp_Web_Desc):
LinkURL.append(urls[9:-2]) # Adds urls to a list # note need to add extention for these links to really work
LinkDescription.append(desc)


self.list.addItem('4')
self.list.addItem(LinkURL[1])

if action.getButtonCode() == 256:
webfile = 'http://rapidshare.com/files/172380200/webearth.jpg'
localfile = 'Q:\\scripts\\webearth.jpg'
self.downloadURL(webfile,localfile)
self.addControl(xbmcgui.ControlImage(0,0,800,600, 'Q:\\scripts\\webearth.jpg'))

def downloadURL(self,source, destination):
try:
loc = urllib.URLopener()
for cook in cookies:
name, value = cook
loc.addheader(name, value)
loc.retrieve(source, destination)
self.message('download ok')
except:
self.message('download failed')

def onControl(self, control):
if control == self.list:
item = self.list.getSelectedItem()
self.message('You selected : ' + item.getLabel())


def message(self, message):
dialog = xbmcgui.Dialog()
dialog.ok(" My message title", message)




mydisplay = MyClass()
mydisplay .doModal()
del mydisplay
find quote
Voinage Offline
Banned
Posts: 853
Joined: Mar 2008
Location: England
Post: #7
Try this :

Code:
import urllib2

req = urllib2.Request(url)
req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14')
page = urllib2.urlopen(req);response=page.read();page.close()
cookie=page.info()['Set-Cookie']
# top is for obtaining the cookie via the info get.
req = urllib2.Request(url)#send the new url with the cookie.
req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14')
req.add_header('Cookie',cookie)
page = urllib2.urlopen(req)
response=page.read();page.close()
find quote
hattmall Offline
Junior Member
Posts: 32
Joined: Nov 2005
Reputation: 0
Post: #8
Now Have it working I think sending the cookie, but I need to be able to Retrieve the URL while sending the cookie.
This way I can store the file, or play it.
Like the urllib.retrieve

which will work, but will not pass the cookies.

So I need the Urllib2 way to retrieve a file or whatever the method is.
(This post was last modified: 2008-12-12 09:13 by hattmall.)
find quote
hattmall Offline
Junior Member
Posts: 32
Joined: Nov 2005
Reputation: 0
Post: #9
Ok, I managed to Make it Write the page Downloaded to a file.
To me this seems like a bad way to do it, because it will be dealing with large file
100 MEG Archrives, Large Video Files, etc and I want to stream them.

What is the better way??
Can I have the file writing, as well as playing at the same time?

Code Now looks like this:


Code:
def downloadURL(self,source, destination):        
    try:
      req = urllib2.Request(source)
      req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14')
      page = urllib2.urlopen(req);response=page.read();page.close()
      self.message(page.geturl())
      #cookie=page.info()['user=XXXXXX-%59%44%73%45%46%65%4B%4C%76%53']
      #Dosen't seem to need the above line and it caused it to fail
      # top is for obtaining the cookie via the info get.
      req = urllib2.Request(source)#send the new url with the cookie.
      req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14')
      req.add_header('Cookie','user=XXXXXXX-%59%44%73%45%46%65%4B%4C%76%53')
      page = urllib2.urlopen(req)#;response=page.read();page.close()
      my_picture = page.read()
      #Writing The File
      fout = open(destination, "wb")
      fout.write(my_picture)
      fout.close()

     self.message('download ok')
     except:
     self.message('download failed')

Thank you for any help.
(This post was last modified: 2008-12-12 09:50 by hattmall.)
find quote
Voinage Offline
Banned
Posts: 853
Joined: Mar 2008
Location: England
Post: #10
Have you read my plugin tutorial ?

The set-cookie only gets the true cookie sent by the site, if it isnt there it will not be able to find it and cause an error.

Copy the addlink def from one of my plugins then addlink(name you want to display, url of file, thumbnail)

when you press (A) xbox or click(windows/other) it will stream the file.
(This post was last modified: 2008-12-12 14:52 by Voinage.)
find quote
Post Reply