[RELEASE] MyMovies.DK Video Scraper

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
ultrabrutal Offline
Posting Freak
Posts: 952
Joined: Feb 2005
Reputation: 0
Location: South of Heaven
Thumbs Up  [RELEASE] MyMovies.DK Video Scraper
Post: #1
Introducing the MyMovies.DK Video Scraper for XBMC. It is now possible to use the MyMovies.DK meta database via an API - without the need for a PC or Server.

MyMovies.DK wiki with information about this scraper and the use of it.

It has taken along time to create this scraper but it has been a lot of fun and I've learned stuff along the way about both XBMC and regular expressions. I've even had to create a few patches to get XBMC to "behave" to my needs (fixed a bug in NfoUrl and made it possible to overrule XBMC's sorting). The support for error dialogs was added by Spiff so that login problems can be reported to the user.
Now it's out there and I hope at least a few people will like it and use it.

Finally I would like to thank Spiff and Nicezia for their help.

Thanks
find quote
ultrabrutal Offline
Posting Freak
Posts: 952
Joined: Feb 2005
Reputation: 0
Location: South of Heaven
Post: #2
Version 1.1 has been released...

Added:
- Support for trailers from filmtrailer.com
- Added support for SortTitle

Changed:
- MPAA rating now just shows PG-13 instead of longer localized text. This means a rating should always be visible for all languages
find quote
pecinko Offline
Donor
Posts: 3,819
Joined: Dec 2008
Reputation: 51
Location: Prague / Belgrade
Post: #3
Hi, ultrabrutal. I did not notice post earlier. Splendid job you did with Brian. Thanks guys!

BTW, don't know why it is in trac and forum that this scraper is commercial only. It can be also freely used by anybody that contributed some amount of DVD titles to mymovies.dk

My skins:

Amber
Quartz

find quote
gusf1 Offline
Junior Member
Posts: 7
Joined: Oct 2009
Reputation: 0
Post: #4
Hi

I recently moved to XBMC from other options, delighted apart from the scraping which I find really hit & miss.

I used metaBrowser before and would like to continue doing so; easy GUI for managing fanart, poster & movie/TV data. http://themetabrowser.com/

The result is a MyMovies.xml file which stores the metadata.

Code:
<Title>
  <LocalTitle>Public Enemies</LocalTitle>
  <OriginalTitle>Public Enemies</OriginalTitle>
  <SortTitle>Public Enemies</SortTitle>
  <Added>20/10/2009 19:14:06</Added>
  <ProductionYear>2009</ProductionYear>
  <RunningTime>140</RunningTime>
  <IMDBrating>7.4</IMDBrating>
  <MPAARating>R</MPAARating>
  <Description>Public Enemies is a 2009 crime film directed by Michael Mann. Set during the Great Depression, it focuses on the true story of FBI agent Melvin Purvis's attempt to stop criminals John Dillinger, Baby Face Nelson, and Pretty Boy Floyd.</Description>
  <Type />
  <AspectRatio>2.35:1</AspectRatio>
  <LockData>False</LockData>
  <IMDB>tt1152836</IMDB>
  <TMDbId>11322</TMDbId>
  <Genres>
    <Genre>Crime</Genre>
    <Genre>Thriller</Genre>
  </Genres>
  <Persons>
    <Person>
      <Name>Michael Mann</Name>
      <Type>Director</Type>
      <Role />
    </Person>
    <Person>
      <Name>Christian Bale</Name>
      <Type>Actor</Type>
      <Role>Melvin Purvis</Role>
    </Person>
    <Person>
      <Name>Johnny Depp</Name>
      <Type>Actor</Type>
      <Role>John Dillinger</Role>
    </Person>
    <Person>
      <Name>Giovanni Ribisi</Name>
      <Type>Actor</Type>
      <Role>Alvin Karpis</Role>
    </Person>
    <Person>
      <Name>Billy Crudup</Name>
      <Type>Actor</Type>
      <Role>J. Edgar Hoover</Role>
    </Person>
    <Person>
      <Name>Emilie de Ravin</Name>
      <Type>Actor</Type>
      <Role>Anna Patzke</Role>
    </Person>
    <Person>
      <Name>Channing Tatum</Name>
      <Type>Actor</Type>
      <Role>Pretty Boy Floyd</Role>
    </Person>
    <Person>
      <Name>Marion Cotillard</Name>
      <Type>Actor</Type>
      <Role>Billie Frechette</Role>
    </Person>
    <Person>
      <Name>Leelee Sobieski</Name>
      <Type>Actor</Type>
      <Role>Polly Hamilton</Role>
    </Person>
  </Persons>
  <Studios>
    <Studio>Forward Pass</Studio>
    <Studio>Universal Pictures</Studio>
  </Studios>
</Title>

My question is - would it be possible to develop a plugin which reads in this file rather for metadata than the MyVideos SQLite DB? There is also the default filename for fanart & posters but think I have a fix for that (although I suppose if it is possible to replace the text then the image name should be possible also?)

If anybody with any experience in this area thinks it would be possible I will give it my best shot; competent developer on other platforms but this is my first go on Linux/XBMC.

Perhaps the best bet would be to modify the metaBrowser tool to write to the SQLite DB - unsure if that would really be possible though.. maybe a script which reads all the XML and stores it in SQLite (including the alternate fanart/poster image file names)

Thanks for any advice/tips etc. Smile

Gus
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 26,221
Joined: Oct 2003
Reputation: 178
Post: #5
XBMC reads MyMovies.xml during a scan.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
sdellutri Offline
Junior Member
Posts: 13
Joined: Dec 2009
Reputation: 0
Post: #6
Hi
I am just recently doing the same thing as you, and ran into similar need. I found that letting XBMC scan my directory with the mymovies.xml files, was not properly finding some titles, and some images.

I wrote a small vbs script that reads the mymovies.xml files, and then outputs a .NFO file that XBMC reads. inside the NFO file, I have put the info needed for the URL to the IMDB library with the movie ID that was already stored into the mymovies.xml file, and the Movie name and Set name tags so I can properly group movie sets.

See my script here... hope its helpful for you..
regards,
Steve

BTW... just wanted to add, that I am also using the Metabrowser software, and found it to be really good. I only wish I could get it to run under Linux/Ubuntu. I am currently dual booting my system... I use Windows 7 with the metabrowser, and my script shown above to manage the file library, but then I boot into Ubuntu to actual use XBMC.
(This post was last modified: 2009-12-18 03:41 by sdellutri.)
find quote
mkortstiege Offline
Team-XBMC Developer
Posts: 2,920
Joined: Jan 2008
Reputation: 8
Location: Germany
Post: #7
Please note that the mymovies.xml parsing is unmaintained and kind of limited compared to the builtin .nfo parser. You should consider using the mymovies scraper or stick with the .nfo files XBMC handles. There are tools out there to generate those as well.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules
For troubleshooting and bug reporting please make sure you read this first.
find quote
gusf1 Offline
Junior Member
Posts: 7
Joined: Oct 2009
Reputation: 0
Post: #8
Thanks for the responses.

@sdellutri - I tried your script and that worked ok, thanks. To get around the dual boot problem my files are on a Samba share which Metabrowser (on a different PC) and XBMC (on ASRock Ion) can access.

@vdrfan - thanks; I guess I am used to a really good scraper so reluctant to change. I will look for manual tools which can generate the .nfo files, perhaps they aren't so bad!
find quote
bleze Offline
Senior Member
Posts: 216
Joined: Nov 2009
Reputation: 0
Post: #9
Version 1.2 has been released...

Added:

- Disc data. Support for loading from Disc database rather than Movie database.
- Support for uniquely matching titles via barcode. NFO file must contain a barcode in the following format; barcode://027616704429.
- Loading movie data via barcode. Alternative to using IMDb ID.

Changed:

- Support for URL encoding of username, password and country. No more needs for doing this manually. NB! Requires a new SVN build later than r26777 of XBMC.
- IMDb rating is now shown with decimals.

- - - - -

This is AFAIK the only scraper which can actually fetch Disc data rather than just Movie data which most other sites contain. This means you can finally find info on DVD/Blu-ray releases which are not listed in IMDb.

Scraper is downloadable here. Wiki also shows how to use Disc info.

Update has been submitted here:trac and if/when approved, it can be accessed via next nightly build of XBMC
find quote
pecinko Offline
Donor
Posts: 3,819
Joined: Dec 2008
Reputation: 51
Location: Prague / Belgrade
Post: #10
bleze Wrote:Version 1.2 has been released...

Update has been submitted here:trac and if/when approved, it can be accessed via next nightly build of XBMC

Some very useful changes/additions. Thanks!

My skins:

Amber
Quartz

find quote
dnadna Offline
Junior Member
Posts: 3
Joined: Jan 2010
Reputation: 0
Post: #11
I have the same issue. I thought mymovies.xml (is there a cASe problem?) would be read by XBMC 9.11.

I use metabrowser on my windows laptop across samba, to get the metadata for the movies.

Is there an update on what is the best way to tell XBMC to recognise the mymovies.xml files?
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #12
this uses features not available in 9.11.
find quote
bleze Offline
Senior Member
Posts: 216
Joined: Nov 2009
Reputation: 0
Post: #13
dnadna, reading of mymovies.xml files has _nothing_ to do with this scraper. mymovies.xml reading works the same way as NFO reading in XBMC. Infact mymovies.xml files are picked up before the NFO file if available
find quote
dnadna Offline
Junior Member
Posts: 3
Joined: Jan 2010
Reputation: 0
Post: #14
Thanks bleze! I think I've been getting it wrong. In fact all the metadata has been picked up quite nicely now Smile.
find quote
pecinko Offline
Donor
Posts: 3,819
Joined: Dec 2008
Reputation: 51
Location: Prague / Belgrade
Post: #15
bleze Wrote:dnadna, reading of mymovies.xml files has _nothing_ to do with this scraper. mymovies.xml reading works the same way as NFO reading in XBMC. Infact mymovies.xml files are picked up before the NFO file if available

I think that was the case. If I'm not mistaken, it was changed recently so NFOs are being processed first.

Could someone from devs confirm?

My skins:

Amber
Quartz

find quote
Post Reply