Scraper Debug

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
olum Offline
Junior Member
Posts: 6
Joined: Jan 2011
Reputation: 0
Post: #1
Hello

I'm trying to edit the OFDB Scrapler becouse it doesn't find all my Movies.
It is my first work with XBMC and also with a Scrapler.

First I was debugging with a tool I found in the Wiki but it seem to be outdated.
Now I'm working with the Scraper Editor I found in the forum.

My Problem is that it all works fine with the Editor but if I try to get it work with XBMC it doesn't work.

This is a part of the debug file.
Quote:23:05:11 T:1008 M:1588563968 DEBUG: CVideoDatabase::GetMovieId (D:\Medien\Videos\Filme\Pulp Fiction (1994)\VIDEO_TS.IFO), query = select idMovie from movie where idFile=2
23:05:11 T:1008 M:1588563968 DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'D:\Medien\Videos\Filme\Pulp Fiction (1994)\VIDEO_TS.IFO'
23:05:11 T:1008 M:1588527104 DEBUG: CIMDB::InternalFindMovie: Searching for 'pulp fiction' using OFDB scraper (path: '?xxx?\XBMC\addons\metadata.ofdb.de', content: 'movies', version: '1.0.16')
23:05:11 T:1008 M:1588527104 DEBUG: scraper: CreateSearchUrl returned http://www.ofdb.de/view.php?page=fsuche&...%C3%BChren
23:05:11 T:1008 M:1588527104 ERROR: ADDON::CScraper::Run: Unable to parse xml

I think everything can help me, because I'm quit new.

Thank you
olum
find quote
mkortstiege Offline
Team-XBMC Developer
Posts: 2,919
Joined: Jan 2008
Reputation: 8
Location: Germany
Post: #2
Log snippet looks strange. You've modified the ofdb scraper? Please post the entire log and tell us about movies that are unable to scrape for you.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules
For troubleshooting and bug reporting please make sure you read this first.
find quote
olum Offline
Junior Member
Posts: 6
Joined: Jan 2011
Reputation: 0
Post: #3
Hello

There a some movies that the scrapler cann't find.
For Examble "Hunde von Riga" http://www.ofdb.de/film/65063,Hunde-von-Riga

I found out that the scrapler works with the bing such engine and there is the problem.
Now I was trying to use the ofdb search. And I think it should work.

This is the code I changed.
Code:
<CreateSearchUrl dest="3">
        <RegExp input="$$1" output="http://www.ofdb.de/view.php?page=fsuche&amp;Typ=N&amp;AB=-&amp;Titel=\1&amp;Genre=-&amp;Note=&amp;HLand=-&amp;Jahr=$$4&amp;Regie=&amp;Darsteller=&amp;Wo=-&amp;Wer=&amp;Land=-&amp;Freigabe=-&amp;Cut=A&amp;Indiziert=A&amp;Info=&amp;Submit2=Suche+ausf%C3%BChren" dest="3">
            <RegExp input="$$2" output="\1" dest="4+">
                <expression clear="yes">(.+)</expression>
            </RegExp>
            <expression noclean="1" />
        </RegExp>
    </CreateSearchUrl>
    <GetSearchResults dest="8">
        <RegExp input="$$5" output="&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; standalone=&quot;yes&quot;?&gt;&lt;results&gt;\1&lt;/results&gt;" dest="8">
            <RegExp input="$$1" output="&lt;entity&gt;&lt;title&gt;\2&lt;/title&gt;&lt;year&gt;\3&lt;/year&gt;&lt;url cache=&quot;ofdb-\1.html&quot;&gt;http://www.ofdb.de/film/\1,&lt;/url&gt;&lt;/entity&gt;" dest="5+">
                <expression repeat="yes" trim="1">&lt;a href=&quot;film/([0-9]*),[^&lt;]*&lt;b&gt;(.[^\(;]*) \(([0-9]*)\)</expression>
            </RegExp>
            <expression noclean="1" />
        </RegExp>
    </GetSearchResults>

I hope that will help.

Thank you
olum
find quote
mkortstiege Offline
Team-XBMC Developer
Posts: 2,919
Joined: Jan 2008
Reputation: 8
Location: Germany
Post: #4
Search URL must be wrapped in <url> tags. Make it output="&lt;url&gt;http://www.ofdb.de/view.php?..&lt;/url&gt;" and it should work.

The reason for the bing search was the imo utterly broken search function on ofdb.de when it comes to umlauts and other special chars. I'll check the site again in order to verify.. In case it's fixed now we can revert back to the built-in search function.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules
For troubleshooting and bug reporting please make sure you read this first.
find quote
olum Offline
Junior Member
Posts: 6
Joined: Jan 2011
Reputation: 0
Post: #5
Thank you.

This was the important hint.
Now it works good.

Maybe we can change it.

Thank you
olum
find quote
dizzy72b Offline
Junior Member
Posts: 21
Joined: Sep 2010
Reputation: 0
Question   
Post: #6
olum Wrote:Now it works good.

Maybe we can change it.

Thank you
olum


hi there!

i kicked my me with IMDB scraping with 9.11
and since 10.0 beta i've been scraping all my (german) movies with OFDB scraper. something like 3000+ german dubbed movies.

found 2 real problems...

1st:

the OFDB scraper only finds 85% of the movies - rest must have an "bla.nfo" with the exact http://www.ofdb.de/823820842 ...

COME ON! 15% NOT SCRAPABLE? THIS NUBER IS TOO HIGH!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


2nd:

the OFDB info is missing in ~ 20% a PLOT SUMMARY - any chance there could be a fallback to eg. CINEFACTS? or IMDB?



someone allready fixed the scraper somehow?

olum! is your version working "way" better?

rest is like this - like that "running slick"!



bb!dc.
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #7
note how the forum description says absolutely nothing to indicate that whiny, demanding posts are ok.
find quote
dizzy72b Offline
Junior Member
Posts: 21
Joined: Sep 2010
Reputation: 0
Post: #8
yes.

can someone pls post the code-snippet to replace the bing search with ofdb search? reason is, that i'm very unhappy with bing search. bing results are most of the time right - but a high amount IS false.

so: please post the "ofdb-search" code-snippet.

thanks!

dc.
find quote