scraper is VERY slow w/ generating thumbs

background info: i'm using windows XP, XBMC babylon, 512mb ram

Whenever I scrape for Movies or TV shows, it takes a very long time to download thumbnails for Posters or Fanart... it can be several minutes PER thumb. If I close XBMC and test the corresponding website, such as Themoviedb.org, the site loads quickly so I don't believe it's a server issue.

Here is a FULL debug of a few scraping attempts:

http://pastebin.com/m721ba06c

I've tried using different skins but they are all slow so I don't think that is the issue (I'm sticking with PM3.hd)

If it is taking too long and I don't want to wait for previews to generate, the Fanart can still download if I choose one - however the Posters will not download without a preview thumbnail first.

Does anyone else have issues with being this slow?

As far as I can tell from the log, it seem like XBMC opens only one two threads to themoviedb.org at any one time even when it is downloading many more images, which of course is not a very good way to make use of the systems resources. Certainly when you see that most posters and backdrops seem to download in just a 2 or 3 seconds one wonders why there are not more threads used to fetch these reletively small files.

Lines like this: 13:34:32 T:2344 M:125603840 ERROR: DIRECTORY::CDirectory::Create - Error creating C:\ are some what disturbing... at least as far as I know the last thing ayone should want to do is create a new C: drive on most windows machines.

Besides those two things an the initial issue with two threads failing to connect to themoviedb.org I see no issues with in this log. The error message might actually be a simple execution speed consideration. It is easier to simply create a existing directory and fail then to first check if it exists and then only when it does not exist create it.

So the only real thing I can see wrong here is that the system does not use more then one thread to fetch the rather small files. This might be because themoviedb.org simply asked not to do this as their servers will not be very pleased with 5 times more threads banging on the door asking for files, or it might be because on the original xbox threads are not possible.

Please update to the latest RC1 release and report back if the issue still exists.

vdrfan Wrote:Please update to the latest RC1 release and report back if the issue still exists.

I have ‘Babylon’ final release updated with T3CH XBMC 2009-05-09 9.04-Babylon. And it still takes about 10 minutes to generate the thumbs for eg. Desperate Housewives Season 5. Interestingly fanart takes much less, but still 3-5 times more then with earlier releases.

I have: windows XP, XBMC babylon, 512mb ram

I know this thread is several months old but it's the only one I can find related to the extremely slow scraping. I am having that issue right now. I am using IMDB for movie scraping and it looks like themoviedb is what it uses for posters/fanart and it takes anywhere from 10 to 30 minutes PER MOVIE when new movies are added. I am using pre 9.10 r22528. That just seems way too long.

My scrapper was fine yesterday on my xbox but today it is slow to download the fanart when you select get fanart. Going to themoviedb site on my PC and it does seem to be running slow although I have never checked out the site before so don't really know how fast the site is normally.

Fetching images from themovie.db org is quite slow lately. Debug log will tell you (most likely) that the fanart download ran into an timeout.