Help understanding scraping
#1
Hi All,

I am looking for a little assistance on how the scrapers/library woek.

1) I have many movies, all ripped in the following format:
C:\Movies\Movie_Name\
C:\Movies\Movie_Name\poster.jpg
C:\Movies\Movie_Name\fanart.jpg
C:\Movies\Movie_Name\VIDEO_TS\VIDEO_TS.nfo

When I add a movie to XBMC, I use a separate instance of XBMC to scrape for the new movie, then I "Export Video Library - individual files," which creates the appropriate poster, fanart, .actors and NFO file. Then I copy the movie files over to my main instance of XBMC storage array, and "Scan for new content." The hope is that XBMC will just identify that the single movie has been added, and one NFO file has changed, and update the library. Instead, it appears to rescrape all my movies. I believe it is still using the local NFOs, but it does appear to scrape every movie, since it takes a very long time.

Is there any way to add a single movies without scraping the entire library over again? I was thinking that if I don't select a scraper then it will be forced to use everything locally. But if I don't select a scraper, the option to "Scan for new content" is not presented.

It is even worse for TV episodes, in which I have many. Rescraping all takes about 2 hours.

Additionally, here is something else I noticed: When I select a local fanart, then export the library (individual files), the resulting NFO seems to make no reference of the local fanart. So if I remove the movie, then add it back, it will not pick up the local fanart, but use the one referenced in the existing NFO.

I guess I am looking for a way to isolate the NFOs so there is not any reference to the outside world. I want to scrape once, select my poster and fanart, then never scrape again. Even if I have to rebuild library from scratch- having the NFOs on the machine, I do not want to scrape. I would like everything to be referenced locally, since I maintain everything locally (except for trailers).

Thanks!
Reply
#2
Use "update library" rather than "scan for new content". You can also press "i" or bring up "info" from the context menu on an individual movie and it will prompt a scan of that specific movie, if the parent folder has content already set.

"Scan for new content" will re-look in all locations of whatever is selected (per source, per folder, etc), while "Update library" will skip previously scanned items.

XBMC always checks for the existence of local artwork before going to the internet, even if you have no NFO file. That's why there's no reference in the NFO file.
Reply
#3
Thanks, but results were unexpected. As a test, I have 2 folders in Video-Files.
1) TV_Series
2) New_Movies

TV_Series has content set to TVdb, and has been previously scraped. So there are no movies in the library, but there are TV episodes. New_Movies does not have content set, but has all of the appropriate NFOs and artwork in its directories. I right click on New_Movies and "Update Library" (Scan for new content option in not present). The status area shows the TV database being updated, but no new movies are added.

Also, if I go to Videos-New_Movies and an individual movie directory, and I try to bring up movie information, it is unavailable as an option. If I update library for that new single movie, it again updates the TV series database. So when I set the content to TMdb, it automatically starts "Scanning movies using The Movie Database." Considering how long this takes, I can only assume it is rescraping.
Reply
#4
After the scraping finishes, don't remove the TMdb from the source. Then after you added a single movie use the 'Update Library' option.

(2013-08-12, 13:15)WannaTheater Wrote: TV_Series has content set to TVdb, and has been previously scraped. So there are no movies in the library, but there are TV episodes. New_Movies does not have content set, but has all of the appropriate NFOs and artwork in its directories. I right click on New_Movies and "Update Library" (Scan for new content option in not present). The status area shows the TV database being updated, but no new movies are added.
Obviously you will need to scrape the information into the database the first time if it doesn't exist yet. This is not rescraping, but just scraping. This will use the NFO files you presented, if you have them named correctly, in the correct spot and they have the correct contents.

In case you are in doubt about how you did this, show us the complete contents of one movie folder, the contents of the NFO and the way you set up the source. (You can make screenshots from XBMC by using the PrtScr button.)
Reply
#5
You have to set a dummy scraper in v12 even if you have local NFO files. XBMC will still use your local files, though. In the upcoming XBMC v13, there will be an explicit "use local files" option, so it's not as confusing.
Reply
#6
(2013-08-12, 10:31)Ned Scott Wrote: Use "update library" rather than "scan for new content". You can also press "i" or bring up "info" from the context menu on an individual movie and it will prompt a scan of that specific movie, if the parent folder has content already set.

"Scan for new content" will re-look in all locations of whatever is selected (per source, per folder, etc), while "Update library" will skip previously scanned items.

XBMC always checks for the existence of local artwork before going to the internet, even if you have no NFO file. That's why there's no reference in the NFO file.

This is the most informative thread I have read in a while - thanks. I never understood those two options. Now it seems to me they are named exactly backwards. Have I got this right?:

1. If you just added a few new movies (i.e., "new content") and want them "scanned", you do NOT use "Scan for new content", you use "Update library".
2. I guess I still don't understand what "Scan for new content does". It re-looks in all locations (which sounds like updating the library). Does it rescrape all locations? Just like removing the content and resetting it for a source? Under what circumstances would you use it?
LibreELEC 10.0.4 * ViMediaManager or TinyMediaManager | Raspberry pi 4b
Sharing media from NAS via NFS (optical out to receiver, HDMI to TV) | TV remote with CEC / Bluetooth keyboard
Reply
#7
"Scan for new content" - scans only the selected folder/source (and any subfolders), and forces XBMC to look in all of those folders even if XBMC thinks there are no new changes.

"Update library" - scans everything in, but XBMC will skip folders it thinks have no new changes.

I'm not sure what XBMC does to determine if changes have been made or not. Maybe folder modification date?

Neither option will re-scrape content that has already been scraped.
Reply

Logout Mark Read Team Forum Stats Members Help
Help understanding scraping0