• 1
  • 16
  • 17
  • 18(current)
  • 19
  • 20
  • 22
Release TheMovieDB movie scraper - PYTHON version
Hello, I am using the TMDB Python Scrapper, and there is something I don't understand. When I go to modify a "cover" I see the Cover of my favorite language (Fr), followed by the English versions hosted by TMDB. This is exactly what is expected. But regarding the "Landscape", I only see the "Fr" versions. Unable to see "En" versions of TMDB, regardless of settings. On the other hand, I manage to see the landscaps of the site fanart.tv.

Is this a bug or a bad setting on my part ?

For example, it is impossible for me to display the English landscapes here :
https://www.themoviedb.org/movie/13576-t...anguage=en
I therefore had to create one in my language so that it would appear the next day.
Reply
Do you have any idea why or how ?
Reply
@Bindou

I have merged your posts into the correct thread.

I can confirm that only French landscape images are listed, but all images are listed for posters.

I don't know which is correct behaviour, so I'll let the developer @rmrector comment

Probably the easiest solution is to download the image you want and save it locally for Kodi to use.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
(2023-01-26, 14:37)robertlaing Wrote: Please could we have an option not to bring back posters?
I don't really know what you mean by this. Can you explain further?
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
I certainly can.

In the configuration settings there is a toggle to whether you bring back fan art for the movie or not.
So if that is toggled off, when you scrape the movie, no fan art is brought back.
Fine with me so far.

However there is no toggle to whether your bring back poster for the movie or not.
It would be useful if such a toggle was implemented.

Then it would be implemented in the same manner when scraping, i.e.
If fanart toggle is true then bring back fan art else no fan art brought back.
If poster toggle is true then bring back poster art else no poster brought back.

I have my own fan art and poster stored locally for each film and when I re-refresh a movie, the poster used is always from the movie db, so I have to manally select my own poster.
For the fan art, as no fan art is brought back as the toggle is off, my fan art is used.

It would be a simple change to make, introduce a poster toggle, like you have with the fan art and put a conditional around the code which brings back the poster for the film.
It should be fairly easy to do and I would really appreciate it.

Many thanks.
Reply
Could someone very kindly implement the above?
Reply
Hi!
Is there a way to scrap the TMDB ID in filenames? The suggested way in kodi "tmbd/12345" isnt working for filenames bacause of the "/".

tmdb12345 would be nice. So i can name my file: my_film_(2023)_tmdb12345.mkv
As for imdb this is working to some degree: tt12345: my_film_(2023)_tt12345.mkv

I know we shouldnt reference to imdb. So pls dont be triggered, but to get the scrapper to use TMDB ID in filenames if present, would be nice.
If Im missing something, please feel free to advice me.

Thanks in advance.
Reply
(2023-02-11, 16:56)Spoon3er Wrote: Is there a way to scrap the TMDB ID in filenames?
No there isn't. The scraper has not been code for that, nor for using IMDB ID's.

I'm guessing you already know about the Search during scraping and using ttxxxxx, tmdb/xxxx and tvdb/xxxxx ID's
You can also use a Parsing NFO file... https://kodi.wiki/view/NFO_files/Parsing
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
a simple way to scrap imdb and tmdb in filename :
in your kodi directory modify the file (save it before just in case)

e:\kodi\portable_data\addons\metadata.themoviedb.org.python\python\lib\tmdbscraper\tmdb.py

in fist line add
python:
import re

then find the line
python:
def _parse_media_id(title):

and add just after
python:

    m=re.search(r'(tt\d+)',title)
    if m: return {'type': 'imdb', 'id':m.group(1)}
    m=re.search(r'tmdb[\/#](\d+)',title)
    if m: return {'type': 'tmdb', 'id':m.group(1)}

as you can see (lines after) the parse function only detect imdb and tmdb if it's in the beginning of the name. I just accept imdb and tmdb everywhere
python:

if title.startswith('tt') and title[2:].isdigit():
        return {'type': 'imdb', 'id':title} # IMDB ID works alone because it is clear
    title = title.lower()
    if title.startswith('tmdb/') and title[5:].isdigit(): # TMDB ID
        return {'type': 'tmdb', 'id':title[5:]}
    elif title.startswith('imdb/tt') and title[7:].isdigit(): # IMDB ID with prefix to match
        return {'type': 'imdb', 'id':title[5:]}
    return None
(don't forget to modify this file every time this addon is update. i made a script to do this automatically)
Reply
(2023-02-12, 10:27)michelb2 Wrote: a simple way to scrap imdb and tmdb in filename :
in your kodi directory modify the file (save it before just in case)

e:\kodi\portable_data\addons\metadata.themoviedb.org.python\python\lib\tmdbscraper\tmdb.py

in fist line add
python:
import re

then find the line
python:
def _parse_media_id(title):

and add just after
python:

    m=re.search(r'(tt\d+)',title)
    if m: return {'type': 'imdb', 'id':m.group(1)}
    m=re.search(r'tmdb[\/#](\d+)',title)
    if m: return {'type': 'tmdb', 'id':m.group(1)}

as you can see (lines after) the parse function only detect imdb and tmdb if it's in the beginning of the name. I just accept imdb and tmdb everywhere
python:

if title.startswith('tt') and title[2:].isdigit():
        return {'type': 'imdb', 'id':title} # IMDB ID works alone because it is clear
    title = title.lower()
    if title.startswith('tmdb/') and title[5:].isdigit(): # TMDB ID
        return {'type': 'tmdb', 'id':title[5:]}
    elif title.startswith('imdb/tt') and title[7:].isdigit(): # IMDB ID with prefix to match
        return {'type': 'imdb', 'id':title[5:]}
    return None
(don't forget to modify this file every time this addon is update. i made a script to do this automatically)

Hi,
meanwhile I took  a quick look at the code and made similar changes in tmdb.py.
I´ve done a pullreqhest at https://github.com/xbmc/metadata.themovi.../182/files to.
To be aware of: This only works, when your tmdb-tag is before the year-tag. Because of how the pattern will be passed to the scrapper.py {"name":"", "year":""}
eg: my_movie_tmdb-12345_(2023).foo will work,
but my_movie_(2023)_tmdb-12345.foo won´t.

But anyways, thanks for the code.
Happy to be on the same page with some other users. Maybee we can get this to be merged, so we don´t need to modify after every update.
Reply
Could anyone implement my idea of toggling the poster art on or off, as per the toggle for the fan art?

Also I have come across a serious oversight when refreshing.

The movie db in it's wisdom does not allow movies to be in sets unless they follow certain criteria.
However Kodi allows you to create your own sets.
I patiently moved all my laurel and hardy movies, shorts and silents into their own sets as the movie db admin staff did not agree with me that they should belong in a set on their site!

So in the Kodi video db I assume that a flag would be set to indicate what set the movie belongs to.

I created my own movie set directory, created a folder for each user movie set and popped in the art work.
When I create my own movie set, it would be useful if Kodi could use this list to give me an option which set to use.
It only brings back the movie db allowed sets, so I have to create a new movie set name , it would be useful if Kodi could scan the movie set directry to find these user created sets and display them in the option of movie sets. Any chance of amending the code to allow that?

However there is a big oversight by Kodi once you have moved the movies into a user created movie set!
If I refesh a movie that is in a user created set, it loses the fact that it in the set!!!!
I assume from this behaviour that the whole row in the db for that movie is wiped and then refeshed when you scrape the movie again, so the flag for the set is lost!
So because the movie db says the movie is not in a set, Kodi ignores the fact that the movie was in a user defined set.
So all my work was lost when I refeshed my movies!!!

Is there anyway that once a movie is moved into a user created movie site, Kodi remembers when refreshing that it is indeed in a user created movie set?

As you can imagine a lot of work is involved moving many movies into a set, Laurel and Hardy has many shorts and silent films!
All my work has been lost and I know I have to set the sets up again, however I would hope that if I ever had to refresh these movies they would not lose the fact that they are in a user defined movie set.

Thanks for reading.
Reply
(2023-02-14, 13:46)robertlaing Wrote: Is there anyway that once a movie is moved into a user created movie site, Kodi remembers when refreshing that it is indeed in a user created movie set?
Once you have edited your library to suit your needs, export your library to Separate Files. This will create nfo files for your entire library. This will contain all your edits.

(2023-02-14, 13:46)robertlaing Wrote: When I create my own movie set, it would be useful if Kodi could use this list to give me an option which set to use.
It does... https://kodi.wiki/view/Movie_sets

(2023-02-14, 13:46)robertlaing Wrote: it would be useful if Kodi could scan the movie set directry to find these user created sets and display them in the option of movie sets.
No. That will cause too many false movie sets. Not every user is quite so careful with organising their filing system.

(2023-02-14, 13:46)robertlaing Wrote: If I refesh a movie that is in a user created set, it loses the fact that it in the set!!!!
Of course it does. You have just asked Kodi to delete the entry and rescrape it. I guess the question is why are you rescraping??

(2023-02-14, 13:46)robertlaing Wrote: Is there anyway that once a movie is moved into a user created movie site, Kodi remembers when refreshing that it is indeed in a user created movie set?
No. Except if you are using nfo files.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
Thanks for the reply, very much appreciated.
I was re-scraping as I wanted the latest IMDB ratings.
I came to the conclusion that when you refresh, it was deleting the entry rather than updating it, hence I lost the info about the movie being in a user created movie set.
Thanks for confirming this.
I think I will have to contact the admin at the moviedb site and fight my corner for my sets to be included.
Have you a contact on this forum?

Back to my previous question, which as you know is long outstanding. LOL!
In the movie scraper, you have an option to toggle the fan art from fan art tv.
I have this toggled off.
I would like to be able to toggle the poster art work.
I know this should be possible as the fan art toggle is implemented.
I do use local art, however when I select artwork and select poster, I am sometimes met with a choice of many many posters, with my local art at the bottom of the list.
When I select fan art, only my local art is shown. I just wanted my Kodi install to be clean and efficient, so I did not want a load of poster icons stored on my system.
My Kodi install on my shield does run very slow on certain skins and keeping my install dedicated to my local artwork has helped a great deal.
I guess it would be easy to add a poster toggle into the configuration settings on the movie scraper? Please kindly could you consider such a feature for me?

Many thanks.
Reply
(2023-02-15, 13:44)robertlaing Wrote: I was re-scraping as I wanted the latest IMDB ratings.
Use this... https://forum.kodi.tv/showthread.php?tid=316342

(2023-02-15, 13:44)robertlaing Wrote: I think I will have to contact the admin at the moviedb site and fight my corner for my sets to be included.
Have you a contact on this forum?
No, don't bother. They will be deleted as soon as they are found, and they won't allow user sets to be added.

Your other question has been answered. If the developer has any interest, he will post here.

I think you need to change your workflow. In fact, your workflow should be the same as mine.
First you need a portable install of Kodi. One that you can mess around in, and not worry about ruining anything... https://kodi.wiki/view/Portable_mode
You use this portable install to scrape your movies and tv shows. And I expect you only add one or two items at a time, not dozens or hundreds.
Then you export to Separate files.
Open the nfo file, delete all the artwork links. Add your movie sets, and whatever other changes you make.
Use Artwork Dump to download the artwork while scraping... https://forum.kodi.tv/showthread.php?tid=360078
Then scan that in to your main Kodi setup.

or

You could use a media manager to organise your library.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
(2023-02-15, 18:51)Karellen Wrote: or

You could use a media manager to organise your library.

UMS is missing from the list of alternatives, however it is possible to turn off all artwork there...
Reply
  • 1
  • 16
  • 17
  • 18(current)
  • 19
  • 20
  • 22

Logout Mark Read Team Forum Stats Members Help
TheMovieDB movie scraper - PYTHON version0