Scraper Debug
#1
Hello

I'm trying to edit the OFDB Scrapler becouse it doesn't find all my Movies.
It is my first work with XBMC and also with a Scrapler.

First I was debugging with a tool I found in the Wiki but it seem to be outdated.
Now I'm working with the Scraper Editor I found in the forum.

My Problem is that it all works fine with the Editor but if I try to get it work with XBMC it doesn't work.

This is a part of the debug file.
Quote:23:05:11 T:1008 M:1588563968 DEBUG: CVideoDatabase::GetMovieId (D:\Medien\Videos\Filme\Pulp Fiction (1994)\VIDEO_TS.IFO), query = select idMovie from movie where idFile=2
23:05:11 T:1008 M:1588563968 DEBUG: VideoInfoScanner: No NFO file found. Using title search for 'D:\Medien\Videos\Filme\Pulp Fiction (1994)\VIDEO_TS.IFO'
23:05:11 T:1008 M:1588527104 DEBUG: CIMDB::InternalFindMovie: Searching for 'pulp fiction' using OFDB scraper (path: '?xxx?\XBMC\addons\metadata.ofdb.de', content: 'movies', version: '1.0.16')
23:05:11 T:1008 M:1588527104 DEBUG: scraper: CreateSearchUrl returned http://www.ofdb.de/view.php?page=fsuche&...%C3%BChren
23:05:11 T:1008 M:1588527104 ERROR: ADDON::CScraper::Run: Unable to parse xml

I think everything can help me, because I'm quit new.

Thank you
olum
Reply
#2
Log snippet looks strange. You've modified the ofdb scraper? Please post the entire log and tell us about movies that are unable to scrape for you.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not PM or e-mail Team-Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
#3
Hello

There a some movies that the scrapler cann't find.
For Examble "Hunde von Riga" http://www.ofdb.de/film/65063,Hunde-von-Riga

I found out that the scrapler works with the bing such engine and there is the problem.
Now I was trying to use the ofdb search. And I think it should work.

This is the code I changed.
Code:
<CreateSearchUrl dest="3">
        <RegExp input="$$1" output="http://www.ofdb.de/view.php?page=fsuche&amp;Typ=N&amp;AB=-&amp;Titel=\1&amp;Genre=-&amp;Note=&amp;HLand=-&amp;Jahr=$$4&amp;Regie=&amp;Darsteller=&amp;Wo=-&amp;Wer=&amp;Land=-&amp;Freigabe=-&amp;Cut=A&amp;Indiziert=A&amp;Info=&amp;Submit2=Suche+ausf%C3%BChren" dest="3">
            <RegExp input="$$2" output="\1" dest="4+">
                <expression clear="yes">(.+)</expression>
            </RegExp>
            <expression noclean="1" />
        </RegExp>
    </CreateSearchUrl>
    <GetSearchResults dest="8">
        <RegExp input="$$5" output="&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; standalone=&quot;yes&quot;?&gt;&lt;results&gt;\1&lt;/results&gt;" dest="8">
            <RegExp input="$$1" output="&lt;entity&gt;&lt;title&gt;\2&lt;/title&gt;&lt;year&gt;\3&lt;/year&gt;&lt;url cache=&quot;ofdb-\1.html&quot;&gt;http://www.ofdb.de/film/\1,&lt;/url&gt;&lt;/entity&gt;" dest="5+">
                <expression repeat="yes" trim="1">&lt;a href=&quot;film/([0-9]*),[^&lt;]*&lt;b&gt;(.[^\(;]*) \(([0-9]*)\)</expression>
            </RegExp>
            <expression noclean="1" />
        </RegExp>
    </GetSearchResults>

I hope that will help.

Thank you
olum
Reply
#4
Search URL must be wrapped in <url> tags. Make it output="&lt;url&gt;http://www.ofdb.de/view.php?..&lt;/url&gt;" and it should work.

The reason for the bing search was the imo utterly broken search function on ofdb.de when it comes to umlauts and other special chars. I'll check the site again in order to verify.. In case it's fixed now we can revert back to the built-in search function.
Always read the online manual (wiki), FAQ (wiki) and search the forum before posting.
Do not PM or e-mail Team-Kodi members directly asking for support. Read/follow the forum rules (wiki).
Please read the pages on troubleshooting (wiki) and bug reporting (wiki) before reporting issues.
Reply
#5
Thank you.

This was the important hint.
Now it works good.

Maybe we can change it.

Thank you
olum
Reply
#6
Question 
olum Wrote:Now it works good.

Maybe we can change it.

Thank you
olum


hi there!

i kicked my me with IMDB scraping with 9.11
and since 10.0 beta i've been scraping all my (german) movies with OFDB scraper. something like 3000+ german dubbed movies.

found 2 real problems...

1st:

the OFDB scraper only finds 85% of the movies - rest must have an "bla.nfo" with the exact http://www.ofdb.de/823820842 ...

COME ON! 15% NOT SCRAPABLE? THIS NUBER IS TOO HIGH!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


2nd:

the OFDB info is missing in ~ 20% a PLOT SUMMARY - any chance there could be a fallback to eg. CINEFACTS? or IMDB?



someone allready fixed the scraper somehow?

olum! is your version working "way" better?

rest is like this - like that "running slick"!



bb!dc.
Reply
#7
note how the forum description says absolutely nothing to indicate that whiny, demanding posts are ok.
Reply
#8
yes.

can someone pls post the code-snippet to replace the bing search with ofdb search? reason is, that i'm very unhappy with bing search. bing results are most of the time right - but a high amount IS false.

so: please post the "ofdb-search" code-snippet.

thanks!

dc.
Reply

Logout Mark Read Team Forum Stats Members Help
Scraper Debug0