[EnderW]

thanks for you answer.

i think i perhaps get some things wrong. i'm horrible at reading others code (and coding myself), but from the short time i spent looking on it i assume that every time a block of data is received it will call ondataretrieved(). my problem then would be getting the parameters out from there, as i would have to modify the module. that would make it harder to update the module in the future as the added lines made specifically for my script would have to be added. i could of course be way off here, but if you have the time it would be nice if you could explain how i would do it most easily.

[Phunck]

i can see that the name "bytesread" is confusing. this is the number of bytes read, not the actual data. if you look at the parameters for ondatareceived:

ondataretrieved(self, bytesread, totalsize, url, localfile)

then you can see that you get bytesread and totalsize. if you want to calculate a percentage (note totalsize might be none if there is no "content-length" header):

percent=(bytesread*100.0)/totalsize

if you look at the code for cachedhttpwithprogress then you'll notice that i create the progressbar here if it isn't already created and then i update the percentage. in ondownloadfinished i just close the progressbar.

[EnderW]

thanks for the clarification, seems like i have to edit the module then as i suspected. will make it more troublesome for users to update, but i can't see any other way you could handle it really...unless you could pass on a function from "my script" to your module which your module then execute in my script sending those same parameters...i suck at explaining, but i hope you understand. that way you could update the module at any time without needing to edit it.

[Phunck]

no you shouldn't have to edit the module...

you can write your own class that inherits from cachedhttp like this (if you don't want the default progressbar):

import cachedhttp

class mycachedhttp(cachedhttp.cachedhttp):
       def ondataretrieved(self, bytesread, totalsize, url, localfile):
               if (not (bytesread is none))&(not (totalsize is none)):
                       pct=int(bytesread*100.0/totalsize)
                       print(str(pct))
               return true # if you return false, then it means cancel the download
       
       def ondownloadfinished(self,success):
               pass

c=mycachedhttp()
data=c.openurl('http://www.example.com')

however, most people would simply use the 2 builtin classes:

c=cachedhttp.cachedhttp()
-or-
c=cachedhttp.cachedhttpwithprogress()

however, i just found out that the x drive is unavailable from python and can not be used. this is quite critical, so i have to fix this problem fast...

[EnderW]

ah, of course...thanks again. i got a lot left to learn about python and even though i knew about inheriting, it didn't occour to me to use it :\

xbmcscripts.com is down at the moment, but should be back up soon...no one can download your script at the moment though.

[fluidman]

instead of using urllib to getting the data you could use cachedhttp.py from http://www.xbmcscripts.com . there is an example of how to use it in the package. the main problem with urllib is that it can freeze if it the site is down. (there is no timeout)

that site doesnt come up can someone just send me the cachedhttp.py files to fluidmanatgmx.de?

thanks in advance

[Bernd]

i never did understand the greediness stuff

greediness is quite easy.
it just means the re parser tries to find the longest possible match.
the pattern: "a.*a"
on a string "abba is great!"
would yield:"abba is grea" and not "abba" !
the pattern: "a.*?a" <- don't be greedy added! stops on the first possible match.
would yield: "abba"

so when parsing html and looking for a link you should use:
<a href="(.*?)".*?>(.+?)</a>

if you hadn't added the ? and the html contains more than one link you will only find one match

bernd

[Bernd]

phunck, with your latest changes this really is the ultimate http fetch class!

it really rocks :kickass: and is a great step towards better and more reliable scripts.

thanks man

bernd

[Asteron]

is there any word on getting this module into cvs? would that be reasonable first of all?

[Phunck]

i have close to zero experience with cvs. but i'm 100% for sharing the source with everybody. so, somebody else must decide if it is appropriate to put it there.

note: as i said the x drive is not available from xbmc and so i've chosen to use 'q:\\scripts\\httpcache\\'
instead. this is not optimal as bernd pointed out, but i see no better alternative. if anybody has any better suggestion for the cache location i'm willing to change it.

[bakman]

phunck, i am trying to implement it in my new studentfilms script (soon to be released ) this is great, enormous performance boost, i was wondering: can you also send http headers when fetching ? some websites have to be spoofed with headers else they won't work.

** edit: found it.! great stuff !***

[Asteron]

if x: - z: is not doable it should probably go in something like
q:\scripts\temp\cache rather than q:\scripts\httpcache

[EnderW]

just tested the module for real, and i must say it's really really great! saves me a lot of work.

just a little note though, the example in your script has a little error - you wrote openurl instead of urlopen on the custom progress example. nothing a python programmer wouldn't figure out though, but just so you know.

anyhow, again, keep up the excellent work

[morte0815]

is there a way to learn your module to replace spaces in urls with '%20'
it would be great if you could do that.

tia

morte

[Bernd]

try using urllib.quote, urllib.quote_plus, urllib.urlencode