Movie scraper is slow?
#1
Hi guys...

I'm working at scraping all my movies in the new htpc I built, and while it's working, it's incredibly slow.

I am set up as follows:

IMDB for scraper

[ ]scan automatically
[ ]folder contains a single video
[x]scan recursively
[ ]use folder names for lookups

The folder being scanned is on another windows computer across the network, and both are connected via cat-5. The fileshare is Samba.

Each movie is in it's own directory with xml-nfo files generated via XBMC companion. Each also has it's own locally stored fanart and .tbn, as well as a folder.jpg file.

It is moving along...scanning one movie every 25 seconds or so. Core CPU usage doesn't seem to get over 50%, and memory usage is only showing at about half of total.

So I guess...is this the average scan speed, or should it be going faster? I thought that by putting all the info in the .nfo files, it would find everything a lot faster...
Reply
#2
The process is entirely I/O bound so you CPU shouldn't be working hard at all. Speed will depend on your internet connection and the load of the site(s) being scraped. It may be in your interest to scrape less crap (actor images, fanart, choose less cover sources, etc). AFAIK we still scrape and cache the images locally so long as they're enabled in the scraper settings regardless of you having a local one chosen.
Reply
#3
althekiller Wrote:The process is entirely I/O bound so you CPU shouldn't be working hard at all. Speed will depend on your internet connection and the load of the site(s) being scraped. It may be in your interest to scrape less crap (actor images, fanart, choose less cover sources, etc). AFAIK we still scrape and cache the images locally so long as they're enabled in the scraper settings regardless of you having a local one chosen.

How do you define the scraper settings? Using media companion, I really don't need to scrape anything...just have xbmc read the tags in the .nfo.
Reply
#4
"settings" button at the bottom of the "set content" dialog.
Reply
#5
Saw that...

Is there any way to just have it scrape the stuff locally w/o downloading anything? I'm talking about not even having it poll IMDB, just have it read the .nfo's.
Reply
#6
digitalhigh Wrote:Saw that...

Is there any way to just have it scrape the stuff locally w/o downloading anything? I'm talking about not even having it poll IMDB, just have it read the .nfo's.

i have not tried this at all so i have no idea if it works or not, but what happens if you disconnect from the internet and try it? maybe if it cannot find a connection it will default to the information provided, should it be available
Reply

Logout Mark Read Team Forum Stats Members Help
Movie scraper is slow?0