Use requests instead of urllib!
#1
I just came on site which sent gzip encoding pages without being requested. Urllib is too stupid to decode gzip. I fixed it only by replacing urllib with requests. It is much more high level, by using it you don't have to write ****** like parameter encoding for post etc.
Reply
#2
I just use urllib and urllib2 so I don't clutter up my projects with more modules. It's not that hard, imo, although the docs suck eggs. Your mileage may vary.

-Jerimiah

From stackoverflow:

This checks if the content is gzipped and decompresses it:
Code:
from StringIO import StringIO
import gzip

request = urllib2.Request('http://example.com/')
request.add_header('Accept-encoding', 'gzip')
response = urllib2.urlopen(request)
if response.info().get('Content-Encoding') == 'gzip':
    buf = StringIO( response.read())
    f = gzip.GzipFile(fileobj=buf)
    data = f.read()
Reply
#3
(2013-11-25, 09:49)jerimiah797 Wrote: I just use urllib and urllib2 so I don't clutter up my projects with more modules. It's not that hard, imo, although the docs suck eggs. Your mileage may vary.

jerimiah797, the idea is wrong. It's not that hard to use external modules - just put them into dependencies section of your xml config. requests already has done what you are doing - and that solution is well tested. It also has much more features - post params as dict/pairs list, sessions, converting to unicode with authomatic page encoding detection, more on their site. It's not just about requests - don't use low level shit at all, like parsing html with regexp, which ******* after " is replaced with '.
Reply
#4
Not trying to invalidate you assessment that the requests lib is great. It is. Use what you want. I don't work with html, I work with json, so there you go.
Reply
#5
(2013-11-25, 17:59)jerimiah797 Wrote: Not trying to invalidate you assessment that the requests lib is great. It is. Use what you want. I don't work with html, I work with json, so there you go.

You work with http, right? Ololo.
Reply
#6
I'm building a script for a streaming audio service that has a json api served over http and https. Everything comes to me formatted exactly how I need it for my script so it's pretty easy.
Reply
#7
requests is a http library. It has nothing to do with http content. It can make your code readable too because of easier configuration (post params, cookies, sessions). It makes your code better readble. Sooner or later your plugin will break (later if you use json, but it's still possible), and maybe someone else decides to fix your plugin because you'll do it too slow.
Reply
#8
Well, I don't know why you keep criticizing everything I say, but.... good luck with everything. I hope your all-powerful knowledge of what is right and wrong about python code, combined with your ability to predict the future, totally works out for you. I'm not here to argue the merits of various techniques. Cheers!

-Jerimiah
Reply
#9
watching your language on family friendly forum would be nice, thanks
Reply
#10
(2013-11-25, 19:11)amet Wrote: family friendly forum
Quote:opensource
development forum
/0

(2013-11-25, 19:09)jerimiah797 Wrote: Well, I don't know why you keep criticizing everything I say, but.... good luck with everything. I hope your all-powerful knowledge of what is right and wrong about python code, combined with your ability to predict the future, totally works out for you. I'm not here to argue the merits of various techniques. Cheers!

-Jerimiah

One option may be that I know python and software development better that you.

It's just a recomendation, but it's worth to follow it.
Reply
#11
(2013-11-25, 20:08)ololoe Wrote:
(2013-11-25, 19:11)amet Wrote: family friendly forum
Quote:opensource
development forum
/0

(2013-11-25, 19:09)jerimiah797 Wrote: Well, I don't know why you keep criticizing everything I say, but.... good luck with everything. I hope your all-powerful knowledge of what is right and wrong about python code, combined with your ability to predict the future, totally works out for you. I'm not here to argue the merits of various techniques. Cheers!

-Jerimiah

One option may be that I know python and software development better that you.

It's just a recomendation, but it's worth to follow it.
Oh no, ololoe is back...Confused

Got banned some months ago because of things like this, but still writing arrogant/abusive posts.
I'm wondering how long it will take until you will be banned again. Some people may be happy about it...
Reply
#12
AddonScriptorDE, I fixed your bestofyoutube plugin for a while 2 days before you done it. Still parsing html with regexps? Smile What was the flaw? " was replaced by ' in the html code or something?
Reply
#13
(2013-11-25, 20:08)ololoe Wrote:
(2013-11-25, 19:11)amet Wrote: family friendly forum
Quote:opensource
development forum
/0

that was not a request... if I edit your posts ^^ will be your last contribution here... Thanks
Reply
#14
Ar you threating me? Smile
Reply
#15
(2013-11-25, 21:28)ololoe Wrote: Ar you threating me? Smile

no, but I have asked you nicely twice... thats more than I did for anyone else.
Reply

Logout Mark Read Team Forum Stats Members Help
Use requests instead of urllib!0