German IMDB scraper, please test it and give feedback

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
mkortstiege Offline
Team-XBMC Developer
Posts: 2,907
Joined: Jan 2008
Reputation: 8
Location: Germany
Post: #31
Nope, sorry. No docs yet (at least i think so). Please have a look at http://xbmc.git.sourceforge.net/git/gitw...dharma-pre for a real-life example.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules
For troubleshooting and bug reporting please make sure you read this first.
find quote
olympia Offline
Team-Kodi Member
Posts: 2,499
Joined: May 2008
Reputation: 32
Post: #32
Eisbahn Wrote:Hi vdrfan,

just found <http://xbmc.git.sourceforge.net/git/gitweb.cgi?p=xbmc/scrapers;a=commit;h=5b59dec81b4e5046a3a515bc0cc6fd68ba408201>. Hope this are actual and proper xml examples, will try it this evening at home.
Are any docs out right now? I know this situation from real life: no docs ready, but client wants an implementation of feature X. No problem, but if the client does not say what he realy wants, it wont be a cheap solution and both sides are frustrated at the end... => Normally I do not accept any contracts without clear rules, or I adapt the price a bit ;=)

Eisbahn

vdrfan Wrote:Nope, sorry. No docs yet (at least i think so). Please have a look at http://xbmc.git.sourceforge.net/git/gitw...dharma-pre for a real-life example.

Or you can download dharma-ready scrapers from any of the mirrors, eg:
http://mirrors.xbmc.org/addons/dharma-pre/

I bet you will now complaining that you again have to "reverse engineer" those scrapers instead of just studying docs, but hey, it took me about 5 minutes to understand the changes when the new structure has been introduced. Wink
find quote
Eisbahn Offline
Junior Member
Posts: 43
Joined: Jun 2010
Reputation: 2
Post: #33
Hello,

just looked at them: seems to be really a five min work ;=) (ok, with some docs, changelog and so on and for my skills half an hour+ ...). Think this evening a version for Dharma can go online.
Just a few question:
- how to get the scraper onto the XBMC server?
- what should included into a zip if addon.xml is as follows and needs the zip a special name or "root path"?
Code:
<addon id="scraper.movies.imdb.de" name="IMDb DE Scraper" version="3.0.0">
...
<extension point="xbmc.metadata.scraper.movies" language="de" library="imdb_de.xml"/>
think following data should be included:
Code:
\resources
    \language
       \English
           strings.xml
       \German
           strings.xml
   settings.xml
addon.xml
changelog.txt
icon.png
imdb_de.xml
- if I open some of the example/given zips, I can see a checksum/hash. How-To and what algorithm (MD5, SH-1, CRC...)?

Eisbahn
find quote
Eisbahn Offline
Junior Member
Posts: 43
Joined: Jun 2010
Reputation: 2
Post: #34
v3.0.0 for XBMC v10x and v2.0.1 for XBMC v9x out now
find quote
theuni Offline
Team-XBMC Communication Manager
Posts: 1,105
Joined: Oct 2007
Reputation: 2
Location: Atlanta, Ga, USA
Post: #35
Eisbahn Wrote:Hello,

just looked at them: seems to be really a five min work ;=) (ok, with some docs, changelog and so on and for my skills half an hour+ ...). Think this evening a version for Dharma can go online.
Just a few question:
- how to get the scraper onto the XBMC server?
- what should included into a zip if addon.xml is as follows and needs the zip a special name or "root path"?
Code:
<addon id="scraper.movies.imdb.de" name="IMDb DE Scraper" version="3.0.0">
...
<extension point="xbmc.metadata.scraper.movies" language="de" library="imdb_de.xml"/>
think following data should be included:
Code:
\resources
    \language
       \English
           strings.xml
       \German
           strings.xml
   settings.xml
addon.xml
changelog.txt
icon.png
imdb_de.xml
- if I open some of the example/given zips, I can see a checksum/hash. How-To and what algorithm (MD5, SH-1, CRC...)?

Eisbahn

Those hashes are inserted by 'git archive', as that's how they are created. I assume they represent the sha1 of the revision where the archive is created, but i'd have to consult the manpages or code (it IS git, afterall) to be sure. As of now, they serve no purpose.

Edit: As for addon.xml docs, see here

TheUni
find quote
Eisbahn Offline
Junior Member
Posts: 43
Joined: Jun 2010
Reputation: 2
Post: #36
theuni Wrote:Edit: As for addon.xml docs, see here

TheUni

Hi,

found that link before (see my postings) but see no usefull infos inside for movie scrapers. Maybe I'm missing something.
If I use my zip file and install in manually in Dharma (point to it on my HDD), XBMC shoms me a short poup: IMDB DE Scraper activated, but I could not find it under movie scrapers. Only if I unpack the files manually, copy them to the correspondig dir in addons (needs my admin pw) and restart XBMC my scraper is recognized.
So I think some things in my zip are broken/not correct.
Next thing i: what tags are supported from Dharma? I don't want to spent again time of my life for tags like certification which are deprecated and not used any more...

Eisbahn
find quote
olympia Offline
Team-Kodi Member
Posts: 2,499
Joined: May 2008
Reputation: 32
Post: #37
1. the scraper in the zip has to be packed in its own dir, eg. metadata.imdb.de
2. I am not sure which scraper were you used as a base for your addon.xml, but please use the layout as all (most) the other ones
3. for addon ID, use "addon id="metadata.imdb.de"

Then it will work.

To be approved in the official repo, there is still quite some work to do.
The common scraper libraries are there to be used for avoiding code duplication. It's even much more easier for you, becuase you only have to call the scraper library without the need of having the code within your scraper.
That's true for scraping info from imdb, tmdb, movieposterdb, etc.
find quote
Eisbahn Offline
Junior Member
Posts: 43
Joined: Jun 2010
Reputation: 2
Post: #38
Hi olympia,

sorry, but I think I could not do all your suggestions or have a different understanding...
1. thanks, now it works fine
2. it has exactly the layout from the wiki. Do you mean the missing CR?
3. wiki said: <type>.<uniquename>, meaning for me = scraper for movies on imdb.de. So I use a corresponding name and see no need to change this. If some rules are preferred, please give some docs ;=)

Sadly the common libraries are not working correctly in 9.11 for german imdb and maybe not in v10 as well. So I decided to do things on my own in the past, because I have no chance to alter the common code.
How can I know if there are changes in the common code? Who documents supported functions with input output parameters? Because I've done the work now: only because of code duplicates (under 10kB code) I shall look at the common functions, analysing what they are doing now (and in future?) and adapt my scraper to use some of them? Sorry, I see no need in this...
Think for a simple scraper it's better to get one working "product" with only a few dependencies. For me testing, porting and updating is extremley easy this way. So I would like not to use the common code (now) :=(

Eisbahn
find quote
Gambler Offline
Junior Member
Posts: 1
Joined: May 2010
Reputation: 0
Post: #39
It would be cool, if your scraper could use cindefacts.de for downloading the movie posters, because the page has all the posters in a very good quality.
find quote
Eisbahn Offline
Junior Member
Posts: 43
Joined: Jun 2010
Reputation: 2
Post: #40
Gambler Wrote:It would be cool, if your scraper could use cindefacts.de for downloading the movie posters, because the page has all the posters in a very good quality.

Sorry, cinefacts (think the "d" in your post is just a typo) does not support searching for unique IDs (like the IMDB number) and gives multiple entries if searched by name. Year could not be used. And all the pics have an add layer "cinefacts" integrated. As far as I could see, cinefacts has not more (or less) pics as the scraper gathers at the moment. Could you give me please an example of a movie title?

Eisbahn
find quote
Eisbahn Offline
Junior Member
Posts: 43
Joined: Jun 2010
Reputation: 2
Post: #41
new version 3.0.3 online
find quote
Eisbahn Offline
Junior Member
Posts: 43
Joined: Jun 2010
Reputation: 2
Post: #42
new versions on <http://github.com/Eisbahn/IMDb_de-Scraper>

3.0.4 (2010-08-07)
- option to use the title from OFDB added (e.g. Evangelion: 2.0 You Can (Not)
Advance results otherwise in Evangerion shin gekij├┤ban: Ha)
- new icon with better readable URL at the edge


2.0.2 (2010-08-07)
- Links to ohost removed and to github included in all files
- project hosted on github because of problems with ohost and better integration
in XBMC
- corrected cleaning of buffers (resulting in recursive calls, e.g. "White Wall
(2010)")
- changed default setting to import only the first 15 actors from IMDb
- option to use the title from OFDB added (e.g. Evangelion: 2.0 You Can (Not)
Advance results otherwise in Evangerion shin gekij├┤ban: Ha)
- new icon with better readable URL at the edge
find quote
llwmuerte Offline
Junior Member
Posts: 4
Joined: Jul 2010
Reputation: 0
Post: #43
the new version is great work again my friend.

sauber Smile
find quote
sportsman Offline
Junior Member
Posts: 19
Joined: Aug 2010
Reputation: 0
Post: #44
how can i install that scraper?
find quote
BurningSky Offline
Senior Member
Posts: 197
Joined: Aug 2009
Reputation: 10
Location: Germany
Post: #45
sportsman Wrote:how can i install that scraper?

Download the .zip file from github.com (you can find the link in Eisbahn's signature) and install it like a normal addon (System --> Add-ons --> install from zip file/Aus ZIP Datei installieren).
(This post was last modified: 2010-08-24 15:56 by BurningSky.)
find quote
Post Reply