Scraper Debug

olum · 2011-01-09 00:48

Hello

I'm trying to edit the OFDB Scrapler becouse it doesn't find all my Movies.
It is my first work with XBMC and also with a Scrapler.

First I was debugging with a tool I found in the Wiki but it seem to be outdated.
Now I'm working with the Scraper Editor I found in the forum.

My Problem is that it all works fine with the Editor but if I try to get it work with XBMC it doesn't work.

This is a part of the debug file.

Quote:23:05:11 T:1008 M:1588563968 DEBUG: CVideoDatabase::GetMovieId (D:\Medien\Videos\Filme\Pulp Fiction (1994)\VIDEO_TS.IFO), query = select idMovie from movie where idFile=2
23:05:11 T:1008 M:1588563968 DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'D:\Medien\Videos\Filme\Pulp Fiction (1994)\VIDEO_TS.IFO'
23:05:11 T:1008 M:1588527104 DEBUG: CIMDB::InternalFindMovie: Searching for 'pulp fiction' using OFDB scraper (path: '?xxx?\XBMC\addons\metadata.ofdb.de', content: 'movies', version: '1.0.16')
23:05:11 T:1008 M:1588527104 DEBUG: scraper: CreateSearchUrl returned http://www.ofdb.de/view.php?page=fsuche&...%C3%BChren
23:05:11 T:1008 M:1588527104 ERROR: ADDON::CScraper::Run: Unable to parse xml

I think everything can help me, because I'm quit new.

Thank you
olum

**vdrfan** · 2011-01-09 13:00

Log snippet looks strange. You've modified the ofdb scraper? Please post the entire log and tell us about movies that are unable to scrape for you.

olum · 2011-01-09 14:27

Hello

There a some movies that the scrapler cann't find.
For Examble "Hunde von Riga" http://www.ofdb.de/film/65063,Hunde-von-Riga

I found out that the scrapler works with the bing such engine and there is the problem.
Now I was trying to use the ofdb search. And I think it should work.

This is the code I changed.

Code:

<CreateSearchUrl dest="3">

        <RegExp input="$$1" output="http://www.ofdb.de/view.php?page=fsuche&amp;Typ=N&amp;AB=-&amp;Titel=\1&amp;Genre=-&amp;Note=&amp;HLand=-&amp;Jahr=$$4&amp;Regie=&amp;Darsteller=&amp;Wo=-&amp;Wer=&amp;Land=-&amp;Freigabe=-&amp;Cut=A&amp;Indiziert=A&amp;Info=&amp;Submit2=Suche+ausf%C3%BChren" dest="3">

            <RegExp input="$$2" output="\1" dest="4+">

                <expression clear="yes">(.+)</expression>

            </RegExp>

            <expression noclean="1" />

        </RegExp>

    </CreateSearchUrl>

    <GetSearchResults dest="8">

        <RegExp input="$$5" output="&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; standalone=&quot;yes&quot;?&gt;&lt;results&gt;\1&lt;/results&gt;" dest="8">

            <RegExp input="$$1" output="&lt;entity&gt;&lt;title&gt;\2&lt;/title&gt;&lt;year&gt;\3&lt;/year&gt;&lt;url cache=&quot;ofdb-\1.html&quot;&gt;http://www.ofdb.de/film/\1,&lt;/url&gt;&lt;/entity&gt;" dest="5+">

                <expression repeat="yes" trim="1">&lt;a href=&quot;film/([0-9]*),[^&lt;]*&lt;b&gt;(.[^\(;]*) \(([0-9]*)\)</expression>

            </RegExp>

            <expression noclean="1" />

        </RegExp>

    </GetSearchResults>

I hope that will help.

Thank you
olum

**vdrfan** · 2011-01-09 14:47

Search URL must be wrapped in <url> tags. Make it output="<url>http://www.ofdb.de/view.php?..</url>" and it should work.

The reason for the bing search was the imo utterly broken search function on ofdb.de when it comes to umlauts and other special chars. I'll check the site again in order to verify.. In case it's fixed now we can revert back to the built-in search function.

olum · 2011-01-09 15:38

Thank you.

This was the important hint.
Now it works good.

Maybe we can change it.

Thank you
olum

dizzy72b · 2011-03-23 11:10

olum Wrote:Now it works good.

Maybe we can change it.

Thank you
olum

hi there!

i kicked my me with IMDB scraping with 9.11
and since 10.0 beta i've been scraping all my (german) movies with OFDB scraper. something like 3000+ german dubbed movies.

found 2 real problems...

1st:

the OFDB scraper only finds 85% of the movies - rest must have an "bla.nfo" with the exact http://www.ofdb.de/823820842 ...

COME ON! 15% NOT SCRAPABLE? THIS NUBER IS TOO HIGH!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2nd:

the OFDB info is missing in ~ 20% a PLOT SUMMARY - any chance there could be a fallback to eg. CINEFACTS? or IMDB?

someone allready fixed the scraper somehow?

olum! is your version working "way" better?

rest is like this - like that "running slick"!

bb!dc.

**spiff** · 2011-03-24 16:38

note how the forum description says absolutely nothing to indicate that whiny, demanding posts are ok.

dizzy72b · 2011-04-19 10:05

yes.

can someone pls post the code-snippet to replace the bing search with ofdb search? reason is, that i'm very unhappy with bing search. bing results are most of the time right - but a high amount IS false.

so: please post the "ofdb-search" code-snippet.

thanks!

dc.

olum Junior Member Posts: 6 Joined: Jan 2011 Reputation: 0	2011-01-09 15:38 Post: #5 Thank you. This was the important hint. Now it works good. Maybe we can change it. Thank you olum
	find quote

spiff Grumpy Bastard Developer Posts: 12,181 Joined: Nov 2003 Reputation: 82	2011-03-24 16:38 Post: #7 note how the forum description says absolutely nothing to indicate that whiny, demanding posts are ok. Always read the XBMC online-manual, FAQ and search the forum before posting. Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules. For troubleshooting and bug reporting please make sure you read this first.
	find quote

dizzy72b Junior Member Posts: 18 Joined: Sep 2010 Reputation: 0	2011-04-19 10:05 Post: #8 yes. can someone pls post the code-snippet to replace the bing search with ofdb search? reason is, that i'm very unhappy with bing search. bing results are most of the time right - but a high amount IS false. so: please post the "ofdb-search" code-snippet. thanks! dc.
	find quote