+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Movie Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=302)
+----- Thread: Release Universal Movie Scraper (/showthread.php?tid=129821)
Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
RE: Universal Movie Scraper - Igorth - 2018-09-01
I'm having lots of errors like this with universal move scraper and movie db, days ago they scrapped everything fine, but now I have lots of files missing:
16:02:56.342 T:11124 WARNING: No information found for item 'G:\Ainara\The.Angry.Birds.Movie.2016.2160p.UHD.BluRay.REMUX.HDR.HEVC.Atmos-EPSiLON-esp.mkv', it won't be added to the library.
kodi 18 nightly.
full log: https://pastebin.com/2EYQ5iKS
RE: Universal Movie Scraper - ExTechOp - 2018-09-01
(2018-08-27, 14:02)olympia Wrote: I hope I have got this fixed here:Excellent, that seems to correct it! Thanks!
https://github.com/xbmc/repo-scrapers/commit/3803907ea7e37d3f849c6aec7165d84b5b505d69
RE: Universal Movie Scraper - Karellen - 2018-09-01
@Igorth
It is hard to see what the problem is as you enabled Debug mode after you completed the Update Library. I can see the errors, but cannot see why.
My initial thought is that it is due to poor naming of the files. When you individually refreshed and selected the correct title, the scrape works fine (10 cloverfield lane, Spiderman)
Rename a couple of your movies so they are cleanly and correctly named then test. See the HowTo Create Video Library link in My Signature below. If it still does not work then submit a log following these steps exactly...
- Enable Debugging mode. Go to Settings>System Settings>Logging and it should be the first option
- Restart Kodi
- Replicate the problem.
- Upload the log to https://paste.ubuntu.com/ manually or use the Kodi Logfile Uploader. With either method post the link back here.
RE: Universal Movie Scraper - HeresJohnny - 2018-09-09
The probelm reported by Igorth is exactly the same one I am seeing, again now with "Jurassic World: Fallen Kingdom'".
As far as I understand, the problem lies not with the file naming but with the scraper because it uses file names for lookup instead of folder names. The scraper is set to (at least in my case and that's how it should always operate) "Movies are in separate folders that match the movie title". You can clearly see above that scraping fails because the file name is too complex.
However, in my case the folder name is "Jurassic World Fallen Kingdom (2018)" which should instantly yield the correct result. It doesn't because the scraper submits the whole filename as a query to IMDB like so
Code:
10:54:08.479 T:6424 DEBUG: CNFSFile::Open - opened TV-Movie/_Movie/Sf-Fantasy/Jurassic World Fallen Kingdom (2018)/Jurassic.World.Fallen.Kingdom.2018.720p.BluRay.x264.DTS-HDChina[EtHD].mkv
10:54:08.508 T:6456 DEBUG: Thread VideoInfoDownloader start, auto delete: false
10:54:08.508 T:6456 DEBUG: ADDON::CScraper::FindMovie: Searching for 'Jurassic World Fallen Kingdom 2018 720p BluRay x264 DTS-HDChina' using Universal Movie Scraper scraper (path: 'C:\Users\JoScha\AppData\Roaming\Kodi\addons\metadata.universal', content: 'movies', version: '5.2.6')
10:54:08.541 T:6456 DEBUG: scraper: CreateSearchUrl returned <url>http://akas.imdb.com/find?q=Jurassic%20World%20Fallen%20Kingdom%202018%20720p%20BluRay%20x264%20DTS-HDChina&s=tt|accept-language=en-us</url>
10:54:08.542 T:6456 DEBUG: CurlFile:
arseAndCorrectUrl() adding custom header option 'accept-language: en-us'
10:54:10.383 T:6456 DEBUG: scraper: GetSearchResults returned <?xml version="1.0" encoding="iso-8859-1" standalone="yes"?><results></results>
10:54:10.383 T:6456 DEBUG: Thread VideoInfoDownloader 6456 terminating
10:54:10.403 T:6424 DEBUG: CNFSFile::Close closing file TV-Movie/_Movie/Sf-Fantasy/Jurassic World Fallen Kingdom (2018)/Jurassic.World.Fallen.Kingdom.2018.720p.BluRay.x264.DTS-HDChina[EtHD].mkv
10:54:10.403 T:6424 WARNING: No information found for item 'nfs://192.168.1.185/TV-Movie/_Movie/Sf-Fantasy/Jurassic World Fallen Kingdom (2018)/Jurassic.World.Fallen.Kingdom.2018.720p.BluRay.x264.DTS-HDChina[EtHD].mkv', it won't be added to the library.
It doesn't matter in this context if scraping is initiated at the single movie folder level or one level up at all movies, the query remains the same.
Please correct.
RE: Universal Movie Scraper - K0D1User1138 - 2018-09-10
I'm having issues scraping ratings from various sites. Though I can't speak for the exact version of UMS over the past couple of months I've not been able to scrape IMDB ratings, Rotten Tomatoes and now Metacritic. I have the RT API entered into UMS and it does seem to be scraping both IMDb and RT scores right now, but no longer Metacritic.
I've been tinkering with my skin some more trying to add all those ratings to it but going back through the past 4 months of newly added movies I've found missing ratings for all 3 types at various times.
Is there currently an issue with Metacritic scraping? Were there previous issues with IMDB and RT, after adding your free API?
I'm also looking for a method of re-scraping all 3 rating systems into my library of over 3500 movies but not certain there is an easy solution for that. The old IMDb updater Add-on was great for it but it's now defunct and only works to update the Top 250 ratings and then disables the next time you start Kodi.
RE: Universal Movie Scraper - K0D1User1138 - 2018-09-11
(2018-09-09, 11:06)HeresJohnny Wrote: The probelm reported by Igorth is exactly the same one I am seeing, again now with "Jurassic World: Fallen Kingdom'".Why do you have the movie file named differently than the folder that file is in? That is my understanding of how the "Movies are in separate folders that match movie titles" feature works. It tells Kodi that the movie file and folder that file are in are named the same. I have no issues scanning any movie in the IMDb library when using a same name file and folder method. Plus I could never look at my library and see file names like you've got. Spooopy!
As far as I understand, the problem lies not with the file naming but with the scraper because it uses file names for lookup instead of folder names. The scraper is set to (at least in my case and that's how it should always operate) "Movies are in separate folders that match the movie title". You can clearly see above that scraping fails because the file name is too complex.
However, in my case the folder name is "Jurassic World Fallen Kingdom (2018)" which should instantly yield the correct result. It doesn't because the scraper submits the whole filename as a query to IMDB like so
Code:10:54:08.479 T:6424 DEBUG: CNFSFile::Open - opened TV-Movie/_Movie/Sf-Fantasy/Jurassic World Fallen Kingdom (2018)/Jurassic.World.Fallen.Kingdom.2018.720p.BluRay.x264.DTS-HDChina[EtHD].mkv
10:54:08.508 T:6456 DEBUG: Thread VideoInfoDownloader start, auto delete: false
10:54:08.508 T:6456 DEBUG: ADDON::CScraper::FindMovie: Searching for 'Jurassic World Fallen Kingdom 2018 720p BluRay x264 DTS-HDChina' using Universal Movie Scraper scraper (path: 'C:\Users\JoScha\AppData\Roaming\Kodi\addons\metadata.universal', content: 'movies', version: '5.2.6')
10:54:08.541 T:6456 DEBUG: scraper: CreateSearchUrl returned <url>http://akas.imdb.com/find?q=Jurassic%20World%20Fallen%20Kingdom%202018%20720p%20BluRay%20x264%20DTS-HDChina&s=tt|accept-language=en-us</url>
10:54:08.542 T:6456 DEBUG: CurlFile:
10:54:10.383 T:6456 DEBUG: scraper: GetSearchResults returned <?xml version="1.0" encoding="iso-8859-1" standalone="yes"?><results></results>
10:54:10.383 T:6456 DEBUG: Thread VideoInfoDownloader 6456 terminating
10:54:10.403 T:6424 DEBUG: CNFSFile::Close closing file TV-Movie/_Movie/Sf-Fantasy/Jurassic World Fallen Kingdom (2018)/Jurassic.World.Fallen.Kingdom.2018.720p.BluRay.x264.DTS-HDChina[EtHD].mkv
10:54:10.403 T:6424 WARNING: No information found for item 'nfs://192.168.1.185/TV-Movie/_Movie/Sf-Fantasy/Jurassic World Fallen Kingdom (2018)/Jurassic.World.Fallen.Kingdom.2018.720p.BluRay.x264.DTS-HDChina[EtHD].mkv', it won't be added to the library.
It doesn't matter in this context if scraping is initiated at the single movie folder level or one level up at all movies, the query remains the same.
Please correct.
RE: Universal Movie Scraper - HeresJohnny - 2018-09-11
Your understanding is wrong. It say "match movie titles" not "not match file name". The original file name is needed to find matching subtitles.
RE: Universal Movie Scraper - host505 - 2018-09-11
@HeresJohnny do you have "scan recursively" turned on on your source's settings?
Not sure if this is the cause of your issue but afaik you need to have this on if your folder structure has more than one levels - which it does in your case as I can see from the log.
If scan recursively is turned on, ums should always lookup the folder name that contains the video file, no matter how far inside the folder structure that is.
If it's turned off, it should lookup only the folders under the main source folder.
RE: Universal Movie Scraper - Angelinas - 2018-09-11
(2018-09-10, 06:57)K0D1User1138 Wrote: Is there currently an issue with Metacritic scraping? Were there previous issues with IMDB and RT, after adding your free API?
I confirm that this rating is broken "Metacrtic"....@olympia can you check "metadata.common.imdb.com" some code are changed on IMDB.You can also add as backup rating "Metacrtic" from OMDB...that will work as second source for same rating.
RE: Universal Movie Scraper - olympia - 2018-09-11
OK, fixed in metadata.common.imdb.com v3.1.1.
Do other ratings scrape OK?
RE: Universal Movie Scraper - HeresJohnny - 2018-09-11
(2018-09-11, 13:52)host505 Wrote: @HeresJohnny do you have "scan recursively" turned on on your source's settings?
Not sure if this is the cause of your issue but afaik you need to have this on if your folder structure has more than one levels - which it does in your case as I can see from the log.
If scan recursively is turned on, ums should always lookup the folder name that contains the video file, no matter how far inside the folder structure that is.
If it's turned off, it should lookup only the folders under the main source folder.
It does not matter, I already stated this above. The folder name is never passed as query.
RE: Universal Movie Scraper - K0D1User1138 - 2018-09-12
(2018-09-11, 23:49)HeresJohnny Wrote:I'll again ask why you don't use a proper naming convention for the actual file? What subtitle requirement do you have that you aren't getting in the files you are downloading? Can you not download the subs (if they aren't included on the downloads you are already getting) and mux the subs into the MKV file, then give it a name with a more suitable convention?(2018-09-11, 13:52)host505 Wrote: @HeresJohnny do you have "scan recursively" turned on on your source's settings?
Not sure if this is the cause of your issue but afaik you need to have this on if your folder structure has more than one levels - which it does in your case as I can see from the log.
If scan recursively is turned on, ums should always lookup the folder name that contains the video file, no matter how far inside the folder structure that is.
If it's turned off, it should lookup only the folders under the main source folder.
It does not matter, I already stated this above. The folder name is never passed as query.
For fun I found a torrent of JP:FK in 720 that wasn't exactly the one you have listed but it did come with subs. It's pretty easy to either rename the .srt file and leave it in the same folder as the movie file or use MKVToolNIX to mux the subs into the MKV.
RE: Universal Movie Scraper - K0D1User1138 - 2018-09-12
(2018-09-11, 23:13)olympia Wrote: OK, fixed in metadata.common.imdb.com v3.1.1.I just updated UMS and re-scraped a couple movies. Seems IMBd, RT, Metacritic and TMDB scores are all being scraped now (if they have scores)
Do other ratings scrape OK?
Now to figure out how to display the RT scores as a percent in my skin, not as a decimal and then how to re-scrape my library for just these ratings and premiered date!
Thanks for the fix!
RE: Universal Movie Scraper - K0D1User1138 - 2018-09-16
I am curious how UMS scrapes the ratings for Rotten Tomatoes, Metacritic and TMDb? What I find in the nfo is the following
Jurassic World: Fallen Kingdom
Code:
<ratings>
<rating name="imdb" max="10" default="true">
<value>6.400000</value>
<votes>133415</votes>
</rating>
<rating name="metacritic" max="10">
<value>5.100000</value>
<votes>59</votes>
</rating>
<rating name="themoviedb" max="10">
<value>6.500000</value>
<votes>3214</votes>
</rating>
<rating name="tomatometerallaudience" max="10">
<value>5.300000</value>
<votes>26862</votes>
</rating>
<rating name="tomatometerallcritics" max="10">
<value>4.900000</value>
<votes>335</votes>
</rating>
<rating name="tomatometeravgaudience" max="10">
<value>6.599999</value>
<votes>26862</votes>
</rating>
<rating name="tomatometeravgcritics" max="10">
<value>5.600000</value>
<votes>335</votes>
</rating>
</ratings>Now in pulling this info to display in the skin the RT score will show up as a single digit number before a decimal, "5.3", when RT uses percentage scores so it should read "53" and then in the skin you can add the percent sign to get "53%".
For Metacritic scores it's similar, a single digit before a decimal, but Metacritic scores are out of 100 so "5.1" should be "51".
Does UMS see thee numbers as recorded in the nfo? If so, is it possible to convert them to 2-digit numbers before the decimal? Then using these numbers in the skin becomes much easier. Is there something in the skin code to bump the decimal point back a spot? Angelinas suggested using Skin Helper Service but I'm unsure how to implement that (or if it's the answer I'm looking for) and think if UMS is able to scrape ratings the way they appear to users on those sites that helps everyone. I hope.
RE: Universal Movie Scraper - olympia - 2018-09-16
Core should support % for ratings, but is doesn't. So nothing UMS can do about...