+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Movie Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=302)
+----- Thread: Release Universal Movie Scraper (/showthread.php?tid=129821)
Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
RE: Universal Movie Scraper - sniferx1 - 2019-07-26
(2019-07-14, 08:10)Karellen Wrote: @sniferx1
Please read... https://forum.kodi.tv/showthread.php?tid=336969
@Karellen
I have posted an issue above but no luck..... hoping you can help with this Mystery.... The Avengers 2012 not scraping with Universal Movie Scraper but scraping with MoviesDB scraper... I have attached De-bug log for your reference.
Any help to resolve this mystery will be much appreciated.
Debug Log from starting from Manuel Movie scrap of the The Avengers 2012 via Universal Movie Scraper: [NOT Scraping]
https://paste.osmc.tv/xanimufope.xml
Debug Log from starting from Manuel Movie scrap of the The Avengers 2012 via MoviesDB : [Working]
https://paste.osmc.tv/yebidibija.xml
DB Used: MariaDB [10.3.11]
Scraper Used: Universal Movie Scraper [Latest]
Link in MoviesDB: https://www.themoviedb.org/movie/24428-the-avengers?language=en-US
Link in IMDB: https://www.imdb.com/title/tt0848228/
Note:
- Video TAGS are off in KODI Settings
- Same issue present for other 3 starwars movies.
- All other 4000+ movies scraping fine with Universal Movie Scraper
RE: Universal Movie Scraper - Karellen - 2019-07-26
@sniferx1
Sure, no problems.
A known issue that is related to the shared MariaDB/MySQL setup and can happen with these hugely popular movies. It is caused by too many artwork links at the scraper site which cannot be accommodated in the shared database. It is not an issue with the Kodi's default SQL database.xml:2019-07-26 11:40:57.345 T:2449461984 ERROR: SQL: [MyVideos116] Undefined MySQL error: Code (1406)
2019-07-26 11:40:57.441 T:2449461984 ERROR: SetDetailsForMovie (nfs://192.168.1.117/volume1/4K_Content/4K Movies/The.Avengers.2012.2160p/The.Avengers.2012.2160p.mkv) failed
2019-07-26 11:40:57.888 T:2449461984 DEBUG: Mysql rollback transaction
Not sure how technically minded you are but there are various threads of other members reporting the same issue....
It is explained here... https://forum.kodi.tv/showthread.php?tid=340933
Also here with one solution... https://github.com/xbmc/xbmc/issues/15768
Too complicated?
1. Use TheMovieDB scraper for this particular movie if it is working. It is related to the number of Artworks listed, not a scraper so it can also be a problem with TheMovieDB (and TVDB), but in this case it is not
2. If you have a portable Kodi setup or another profile that does not use the shared database, then scrape to the default sql database, export to NFO Files. Delete about half the artwork links in the file and then scan into MariaDB.
RE: Universal Movie Scraper - sniferx1 - 2019-07-26
(2019-07-26, 04:59)Karellen Wrote: @sniferx1@Karellen Thanks again resolving this mystery. As per your comment workaround fix was to "Remove extra FANART Options from the Scraper settings as I got AB doing that JOB" Issue is now fixed.
Sure, no problems.
A known issue that is related to the shared MariaDB/MySQL setup and can happen with these hugely popular movies. It is caused by too many artwork links at the scraper site which cannot be accommodated in the shared database. It is not an issue with the Kodi's default SQL database.xml:2019-07-26 11:40:57.345 T:2449461984 ERROR: SQL: [MyVideos116] Undefined MySQL error: Code (1406)
2019-07-26 11:40:57.441 T:2449461984 ERROR: SetDetailsForMovie (nfs://192.168.1.117/volume1/4K_Content/4K Movies/The.Avengers.2012.2160p/The.Avengers.2012.2160p.mkv) failed
2019-07-26 11:40:57.888 T:2449461984 DEBUG: Mysql rollback transaction
Not sure how technically minded you are but there are various threads of other members reporting the same issue....
It is explained here... https://forum.kodi.tv/showthread.php?tid=340933
Also here with one solution... https://github.com/xbmc/xbmc/issues/15768
Too complicated?
1. Use TheMovieDB scraper for this particular movie if it is working. It is related to the number of Artworks listed, not a scraper so it can also be a problem with TheMovieDB (and TVDB), but in this case it is not
2. If you have a portable Kodi setup or another profile that does not use the shared database, then scrape to the default sql database, export to NFO Files. Delete about half the artwork links in the file and then scan into MariaDB.
Note that Movies GENRES put differently in MOVIESDB scraper and Universal scraper and this was bit a problem for me... all good now and hopefully KODI team will implement better error handling for this in next releases and show proper error message to the users.
Thanks again for resolving my issue...
RE: Universal Movie Scraper - Dumyat - 2019-07-27
Hi,
I've exported a few items from the Netflix addon into my Kodi library and configured the UMS to include extra artwork such as Clearlogo, Discart, Clearart.
I've not used the UMS for a while, but I had a feeling that these artwork types should get downloaded into the Movie folder automatically, as I'm not seeing this happening at the moment?
Movie titles are getting scrapped and added into the library with no problem, just no sign of the extra artwork.
Is there anything else I need to do to trigger the artwork download? Exported these items using Kodi 18.3 on the Nvidia Shield.
Any feedback would be much appreciated.
Cheers
RE: Universal Movie Scraper - Karellen - 2019-07-27
(2019-07-27, 18:37)Dumyat Wrote: but I had a feeling that these artwork types should get downloaded into the Movie folder automaticallyNo, UMS does not save local artwork. Same with regular Posters and Fanart, they are never saved locally by UMS. It only loads them into Kodi cache.
RE: Universal Movie Scraper - Dumyat - 2019-07-27
(2019-07-27, 19:28)Karellen Wrote:Ahh, ok. Thanks for the info and clarification Karellen.(2019-07-27, 18:37)Dumyat Wrote: but I had a feeling that these artwork types should get downloaded into the Movie folder automaticallyNo, UMS does not save local artwork. Same with regular Posters and Fanart, they are never saved locally by UMS. It only loads them into Kodi cache.
So, if the images are loaded to the Kodi cache, should I see things like the clearlogo on the screen in my library view, if I have that artwork option enabled in the settings? I'm currently not seeing anything at the moment and I'm trying to understand what might be the cause.
RE: Universal Movie Scraper - Karellen - 2019-07-27
(2019-07-27, 20:51)Dumyat Wrote: So, if the images are loaded to the Kodi cache, should I see things like the clearlogo on the screen in my library view, if I have that artwork option enabled in the settings?Yes, you should see the artwork in library view if you are using a compatible skin.
I think you may have only completed half the required steps. Have a read here as I think you have only completed Step 2a... https://kodi.wiki/view/Movie_artwork
RE: Universal Movie Scraper - Dumyat - 2019-07-27
(2019-07-27, 21:00)Karellen Wrote:Aha, success!(2019-07-27, 20:51)Dumyat Wrote: So, if the images are loaded to the Kodi cache, should I see things like the clearlogo on the screen in my library view, if I have that artwork option enabled in the settings?Yes, you should see the artwork in library view if you are using a compatible skin.
I think you may have only completed half the required steps. Have a read here as I think you have only completed Step 2a... https://kodi.wiki/view/Movie_artwork
I had completely forgotten about adding tags to my advanced settings. As soon as I read the Wiki I remembered immediately as I'd done the same thing for my Music library months ago.
Just exported a couple of new items from Netflix to test and everything is working, but in order to get the clearlogo I did have to do a manual refresh on each item to get them to appear. No real hardship, as I'm only adding items a couple at a time.
Thanks for the help. Much appreciated.
RE: Universal Movie Scraper - sniferx1 - 2019-07-29
(2019-07-26, 04:59)Karellen Wrote: @sniferx1
Sure, no problems.
A known issue that is related to the shared MariaDB/MySQL setup and can happen with these hugely popular movies. It is caused by too many artwork links at the scraper site which cannot be accommodated in the shared database. It is not an issue with the Kodi's default SQL database.xml:2019-07-26 11:40:57.345 T:2449461984 ERROR: SQL: [MyVideos116] Undefined MySQL error: Code (1406)
2019-07-26 11:40:57.441 T:2449461984 ERROR: SetDetailsForMovie (nfs://192.168.1.117/volume1/4K_Content/4K Movies/The.Avengers.2012.2160p/The.Avengers.2012.2160p.mkv) failed
2019-07-26 11:40:57.888 T:2449461984 DEBUG: Mysql rollback transaction
Not sure how technically minded you are but there are various threads of other members reporting the same issue....
It is explained here... https://forum.kodi.tv/showthread.php?tid=340933
Also here with one solution... https://github.com/xbmc/xbmc/issues/15768
Too complicated?
1. Use TheMovieDB scraper for this particular movie if it is working. It is related to the number of Artworks listed, not a scraper so it can also be a problem with TheMovieDB (and TVDB), but in this case it is not
2. If you have a portable Kodi setup or another profile that does not use the shared database, then scrape to the default sql database, export to NFO Files. Delete about half the artwork links in the file and then scan into MariaDB.
I have also raised a bug with mariadb team. Hopefully it will get fixed soon.....
https://jira.mariadb.org/browse/MDEV-20197
RE: Universal Movie Scraper - sniferx1 - 2019-08-09
(2019-07-29, 01:15)sniferx1 Wrote:(2019-07-26, 04:59)Karellen Wrote: @sniferx1
Sure, no problems.
A known issue that is related to the shared MariaDB/MySQL setup and can happen with these hugely popular movies. It is caused by too many artwork links at the scraper site which cannot be accommodated in the shared database. It is not an issue with the Kodi's default SQL database.xml:2019-07-26 11:40:57.345 T:2449461984 ERROR: SQL: [MyVideos116] Undefined MySQL error: Code (1406)
2019-07-26 11:40:57.441 T:2449461984 ERROR: SetDetailsForMovie (nfs://192.168.1.117/volume1/4K_Content/4K Movies/The.Avengers.2012.2160p/The.Avengers.2012.2160p.mkv) failed
2019-07-26 11:40:57.888 T:2449461984 DEBUG: Mysql rollback transaction
Not sure how technically minded you are but there are various threads of other members reporting the same issue....
It is explained here... https://forum.kodi.tv/showthread.php?tid=340933
Also here with one solution... https://github.com/xbmc/xbmc/issues/15768
Too complicated?
1. Use TheMovieDB scraper for this particular movie if it is working. It is related to the number of Artworks listed, not a scraper so it can also be a problem with TheMovieDB (and TVDB), but in this case it is not
2. If you have a portable Kodi setup or another profile that does not use the shared database, then scrape to the default sql database, export to NFO Files. Delete about half the artwork links in the file and then scan into MariaDB.
I have also raised a bug with mariadb team. Hopefully it will get fixed soon.....
https://jira.mariadb.org/browse/MDEV-20197
@Karellen Please find the feedback from the Mariadb team. As per the provided recommendation can you please get KODI team to implement this fix part of next major version (KODI 19).
text:[ https://jira.mariadb.org/browse/MDEV-20197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sergei Golubchik closed MDEV-20197.
-----------------------------------
Fix Version/s: N/A
Assignee: Sergei Golubchik
Resolution: Not a Bug
This is not a MariaDB bug.
See the explanation in [this kodi thread|https://forum.kodi.tv/showthread.php?tid=340933].
MariaDB has a rich set of column types to suit various use cases. It appears that KODI uses type TEXT for the column c06 in the table tvshow. Columns of this type are limited to 64K bytes — https://mariadb.com/kb/en/library/text/
The workaround, suggested in the forum, is to change the column type to at least MEDUIMTEXT — https://mariadb.com/kb/en/library/mediumtext/
The proper fix, of course, is for KODI to create the table with that data type for large columns.
> Can't insert SQL when size is large
> -----------------------------------
>
> Key: MDEV-20197
> URL: https://jira.mariadb.org/browse/MDEV-20197
> Project: MariaDB Server
> Issue Type: Bug
> Components: Server
> Affects Versions: 10.3.11
> Environment: Production
> Reporter: Dann Mendis
> Assignee: Sergei Golubchik
> Priority: Minor
> Fix For: N/A
RE: Universal Movie Scraper - Karellen - 2019-08-09
@sniferx1
Thanks for the reply. Do you have a github account? Can you post the MariaDB reply to this Issue (so you take credit for the work)... https://github.com/xbmc/xbmc/issues/15768
If not, let me know and I will post it.
RE: Universal Movie Scraper - 2g4y1 - 2019-08-12
Hi guys,
i get a strange error when i try to scrap some movies.
i found in the log the scraper tries to call the following url. the first one generates a 404 error.
the second one works, since i replaced the | with an &
http://akas.imdb.com/find?q=Hellboy%20(2004)%20-%20ODM&s=tt|accept-language=en-us
http://akas.imdb.com/find?q=Hellboy%20(2004)%20-%20ODM&s=tt&accept-language=en-us
here is a full log: https://paste.kodi.tv/ezubupucoq.kodi
and a second strange error appears in the log: "ERROR: PCRE: Match limit reached"
thanks and best regards
RE: Universal Movie Scraper - Karellen - 2019-08-12
@2g4y1
It think your problem is item 1d here... https://forum.kodi.tv/showthread.php?tid=336969
RE: Universal Movie Scraper - sniferx1 - 2019-08-13
(2019-08-09, 09:44)Karellen Wrote: @sniferx1
Thanks for the reply. Do you have a github account? Can you post the MariaDB reply to this Issue (so you take credit for the work)... https://github.com/xbmc/xbmc/issues/15768
If not, let me know and I will post it.
Thanks @Karellen much appreciated. I would really appreciate if you can post this to correct KODI GIT as i don;t have a GIT account. Thanks again.
RE: Universal Movie Scraper - DaVu - 2019-08-13
@sniferx1 thanks for your effort. That fix was already suggested by the inital bug report See: https://github.com/xbmc/xbmc/issues/15768 . I have to admit, that the person who reports this bug was suggesting to use
LONGTEXT, but interesting that MariaDB says something similar.Additionally, SQiteL (what Kodi uses by default) obviously doesn't know types like
MEDIUMTEXT or LONGTEXT and the limit for TEXT differs much from SQLite (10^9) to MySQL (2^16). So fixing it for MySQL databases will break it for SQLite databases