ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work...

Here's a idea of how i want the library to interface with programs



As you can see i want the programs to have to do very little in the way of actually processing any scraper information

Because i want to account for all the behaviours of the XBMC parser...

There's something i've run across with the TVdb scraper which is a RegExp without an input buffer reference, i'm assuming if there is no input reference it automatically assumes buffer $$1?

there is also an issue that I didn't know how to handle with the allocine scraper:


I just changed the actual XML file to read "true" so it could be parsed correctly. Is this intentional and needs to be coded around? If so, what exactly are we doing here?

yeah also there's scrapers that use dest="5+" on the first RegExp to be executed, (and for some reason my recursive coding goes over the process twice (and i haven't been able to nail down why, although if you saw in the source there was a place that i had a suspicion was the cause) so with (TVdb scraper for example) it ends up twice as long as it should be.

Okay I'm down to the final little bit of getting my code completely working (all the simpler ones work, its the IMDB one i'm working on now) and this is what i'm coming up with (before processing custom functions... so what exactley is <url cache> for? what does it do or tell XBMC to do?

Nicezia,

I've been following the XMLParser work by Spiff followed by the library work by yourself. I'm very interested in seeing something like this implemented for MeediOS in order to avoid issues of scrapers breaking plugins, etc. My question for you is this (and know that I didn't look at the code yet):

Is there a mechanism for the parser library to just fetch the top search result without returning a list of selections for the user to choose the best match?

The reason I'm asking this is while I personally prefer user initiated matching, some people like to setup importers to auto-import and then manually correct within the UI if necessary.

Finally, if we end up determining that some particular fields of interest are missing in an XMLScraper file how are you guys managing modifications or versioning to those scripts?

Thanks.

Shawn

Yes I'm working on that as an alternative

I've run into situations where there is an EXACT match only, so i'm trying to code in the ability to return only the First match that is returned

I think however that will be set up as a "Return Only Exact Match" setting, naturaully that would b dependant on supplying the right information on a search.

The imdb especially is bad about not getting the exact match as the first, so i'm working on a routine that will sort the matches (don't know if they have that in the XBMC scraper system, but i plan to implement it in mine.

cache is a local filename. we cache the url to that file. it is used for speeding up running several functions on the same page

sbass, that's the beauty of xml. if a field is available, you parse it. if not, you do not. the scrapers have no clue what they are returning as such, to them all is text.

if these guys gets adopted more widely, we should definitely add a version field to them. since it's all xml, again no problem

and yes, we have sorting etc in our system. but that is all external to the scraper stuff