MovieMeter.nl (Dutch Movies) Scraper development...

spiff Wrote:you got the search thing working i assume?

I've got it as far as downloading the main page and recovering the hash value needed for the search. I've been on vacation so haven't worked on it much.

What it does now is request the main page and retrieve the hashcode. Next i loose the moviename/search string as $$1 gets overwritten. I need both to build the complete search string. Next problem is to get the scraper to request the actual search url. Which is where chaining (is what you mentioned) comes in i guess?

I've very little time to figure everyting out. Mostly i skip my lunch breaks at work to work on this, so please excuse me if it takes me a long time. Hope to have it figured out and complete in a few weeks if i can find time.

pseudo;

<CreateSearchUrl clearbuffers="no" dest="3">
store inputstring in,say, $$9
return <url>mainpage</url>

<GetSearchResults dest="3">
fetch hash
return <url function="realgetresults">someurl</url> based on hash and $$9

<RealGetResults>
return the parsed results

important points being;

1) the clearbuffers=no will make sure you don't clear the buffers between subsequent calls. this means you can pass the searchstring further on.
2) the url with a function parameter is what i'm referring to as chaining

Thanks spiff that clears some things up.

I made/found some time yesterday to work on the scraper and think i got it at a stage that it can retrieve the basic details from moviemeter. I'm testing it using scrap.exe but it seems to keep crashing. I also tried the imdb.xml with scrap.exe which also fails. So i'm guessing that scrap.exe differs from xbmc in functionality and that i should test the scraper with xbmc. Hopefully i will find time this weekend to make a stable "basic" version of the scraper after which i will be looking at adding impawards, movieposterdb support for posters and imdb support for additional movie details.

scrap is outdated and we lost the source code for parts of it :/

Thanks for all the help you are providing spiff.
I think it might be usefull to update the wiki with the fact that scrap.exe is outdated because the current wiki page doesnt notify you of this. Or am i missing something?

good point

Did this scraper got any progress I'd still love my movie information in dutch.

Sadly i can't report much progress. As i said in my latest post in this thread, i think i have the basic functionality done but need to test it in xbmc. I just haven't found the time this last weeks to do this. I'm still determined to finish the scaper, and add support for impawards, movieposterdb, fanart and additional imdb info. But sadly i can't give you a timeframe for this all. I'm working on it in the (very) little spare time that i have.

Trazer Wrote:Sadly i can't report much progress. As i said in my latest post in this thread, i think i have the basic functionality done but need to test it in xbmc. I just haven't found the time this last weeks to do this. I'm still determined to finish the scaper, and add support for impawards, movieposterdb, fanart and additional imdb info. But sadly i can't give you a timeframe for this all. I'm working on it in the (very) little spare time that i have.

Sounds very ambitious I have plenty of time, just nice to hear your planning on finishing it in the future, hopefully someone pops up to offer you help to speed everything up a bit, there are plenty of dutch/flemish xbmc users out there.

Missaar Wrote:Sounds very ambitious I have plenty of time, just nice to hear your planning on finishing it in the future, hopefully someone pops up to offer you help to speed everything up a bit, there are plenty of dutch/flemish xbmc users out there.

True!

Maybe you can ask for help in the Dutch community topic.