There's obviously been a slight change in akas.imdb.com
#16
um there already is a way to test NfoUrl in scraperXML Editor..
you just load the file in the tester.

however, ScraperXML Editor will load the Setting so its not a good representation of how XBMC acts.

Are you sure you understand the whole concept of NfoUrl?
ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

Image
Reply
#17
I've been using an XML editor for the "development" of the scraper. Perhaps I should try your editor then Smile

No I'm pretty sure I don't fully understand all about internal workings of scrapers, but I got all to work except IMDB no's from NFO files and fanart (had it working shortly). Fanart kills the scraper tester and doesn't show any fanart in XBMC, so no idea what's wrong there.

I tried again to fix XBMC but gave up again hehe. No idea how to get the settings for NfoUrl loaded/passed around... Will have to wait for Spiff.

Will give Fanart a crack again if I can get it to test properly inside your editor...

Tnx again!
Reply
#18
NfoUrl searches a local file for a URL and takes that URL and jumps to GetDetails, All the info compared in NfoUrl is done locally, it's just to get a USER specified URL to download data for use with GetDetails Function. This is to facilitate a user being able to specify the URL to use in the getdetails Function not having to go through createSearchUrl and GetSearchResults. There is no searching done from NfoUrl, the only function that runs from NfoUrl is itself and GetDetails

and from all i've gathered, you're trying to run a search from NfoUrl.. and that's not gonna happen. Again to do this you want to use CreateSearchUrl & GetSearchResults which DO use settings
ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

Image
Reply
#19
Then I have understood it correctly and just need my settings provided and swapped in the buffers Smile
Reply
#20
well you are throwing "command=SearchIMDBId" at it so i figured you were trying to do a search

I have no idea what you're TRYING to do, and no idea about how the page you are refering to handles IMDBId search so i can't help any further.

in anycase ScraperXML itself has no handling for NfoUrl, that's something that's going to be dependant on whatever program is using the library - as a method for setting the buffers has been allowed for. I can't be expected to do all the work for the programs. I'm just setting up a framework to be used.
ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

Image
Reply
#21
Well, my understanding is that when searching, the url is coming from GetSearchResults and used in GetDetails.
NfoUrl works similar and the url it outputs is also used in GetDetails.

Now, my problem is that a imdb id is not unique. There can be 0-many movies in the database with the same imdb id. This is because the database does not only contain film info, but also DVD, HD-DVD and Blu-Ray, not to mention different editions of same movie (take Terminator 2 for example Wink.
This is not a problem when searching since you can choose the version of the movie you like, however automated scan just takes the best match - not looking at data really.
What I need to do with NfoUrl is, look up the titles with this imdb no and chose one to use as url for GetDetails.
I guess a setting for default movie type would help me out, but it would not be perfect since you can have different versions in your library. Mine is a mix of films (rips with no relation to source), iso's of dvd's (or video_ts folder), iso's of blu-rays. It would ofcourse be ideal if XBMC would tell the type of move to the NfoUrl function (hello Spiff Wink).

The better way to make a perfect match (at least with my scraper) would be to use the DiscID. This can be obtained from iso's of dvd's/blu-ray's/hd-dvd's. Movies with no disc id (films with no relation) could use imdb no and film info (not currently in database, but will be soon).
Here is a request for DiscID support in XBMC: http://trac.xbmc.org/ticket/6800

So, I think we have a bit of road to travel to truely get "correct" metadata for our movies in our libraries. Smile
Reply
#22
ultrabrutal Wrote:Well, my understanding is that when searching, the url is coming from GetSearchResults and used in GetDetails.
NfoUrl works similar and the url it outputs is also used in GetDetails.

Now, my problem is that a imdb id is not unique. There can be 0-many movies in the database with the same imdb id. This is because the database does not only contain film info, but also DVD, HD-DVD and Blu-Ray, not to mention different editions of same movie (take Terminator 2 for example Wink.
This is not a problem when searching since you can choose the version of the movie you like, however automated scan just takes the best match - not looking at data really.
What I need to do with NfoUrl is, look up the titles with this imdb no and chose one to use as url for GetDetails.
I guess a setting for default movie type would help me out, but it would not be perfect since you can have different versions in your library. Mine is a mix of films (rips with no relation to source), iso's of dvd's (or video_ts folder), iso's of blu-rays. It would ofcourse be ideal if XBMC would tell the type of move to the NfoUrl function (hello Spiff Wink).

The better way to make a perfect match (at least with my scraper) would be to use the DiscID. This can be obtained from iso's of dvd's/blu-ray's/hd-dvd's. Movies with no disc id (films with no relation) could use imdb no and film info (not currently in database, but will be soon).
Here is a request for DiscID support in XBMC: http://trac.xbmc.org/ticket/6800

So, I think we have a bit of road to travel to truely get "correct" metadata for our movies in our libraries. Smile

So if i'm understanding correctly you want to use mymovies.dk with a IMDBId lookup and it returns multiple copies of the same movie (with different formats)

once again - if i'm understanding correctly, sounds like a job for GetSearchResults, nfoUrl should really only be used if you know the EXACT page you want to scraper info from... now if all these results are contained in the same page then nfoUrl could still be used, in GetDetails you'd just have to use RegExp's to filter out the content you wanted, for instance, copy the section for BluRay version into another buffer as a first RegExp to run, then set that buffer to pull the rest of your details. but if they are all on seperate pages, there is no avoiding going through CreateSearchUrl and GetSearchResults.
ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

Image
Reply
#23
nope, searching for imdb no should return just one result which GetDetails will be able to process. I'll be working with mymovies author to make this happen and make a setting for specifying the type of media to scrape from (film,dvd,hd-dvd, blu-ray).
hopefully in a few days I can wrap this up (except nfourl) but that should start working as soon as spiff have fixed the missing passing of settings to the function
Reply
#24
ultrabrutal Wrote:nope, searching for imdb no should return just one result which GetDetails will be able to process. I'll be working with mymovies author to make this happen and make a setting for specifying the type of media to scrape from (film,dvd,hd-dvd, blu-ray).

That makes more sense

Quote:hopefully in a few days I can wrap this up (except nfourl) but that should start working as soon as spiff have fixed the missing passing of settings to the function

I'd like to say i could help by taking some of the load of spiff and submitting some patches, but C++ while alot easier for me to read now, is still a foreign language to me.
ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

Image
Reply
#25
well I'm currently a professional C++ programmer, however only been at it for little over a year, so I'm still learning. My problem is the XBMC source which I think is always the problem. Understanding how a framework works and how the source is built upon it and how other developers do stuff. I've found the problem why NfoUrl does not work, but I cannot figure out how to get the scraper settings passed to that function Smile
Reply
#26
ultrabrutal Wrote:well I'm currently a professional C++ programmer, however only been at it for little over a year, so I'm still learning. My problem is the XBMC source which I think is always the problem. Understanding how a framework works and how the source is built upon it and how other developers do stuff. I've found the problem why NfoUrl does not work, but I cannot figure out how to get the scraper settings passed to that function Smile

where we read the scraper db info, add a member CScraperSettings class to CNfoFile, load the settings and use them if the currently tested scraper matches whatever scraper the settings were stored for. you use them by passing the class to the CScraperParser:Tonguearse() functions, see CIMDB for an example or two.
Reply
#27
Cool I've written in a NfoUrl handler for Each scraper type, which i've not included for the reason that XBMC doesn't handle Settings in NfoUrl, if this gets changed, then I'll unlock that function

Trying to keep the library loyal to XBMC's functionality
ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

Image
Reply
#28
Nicezia Wrote:Cool I've written in a NfoUrl handler for Each scraper type, which i've not included for the reason that XBMC doesn't handle Settings in NfoUrl, if this gets changed, then I'll unlock that function

Trying to keep the library loyal to XBMC's functionality

nicezia, I've fixed this! you can get a patch from trac Wink
Reply
#29
Soon as its reviewed and approved, i'll unlock the NfoUrl functions
ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

Image
Reply

Logout Mark Read Team Forum Stats Members Help
There's obviously been a slight change in akas.imdb.com0