+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Information Providers (scrapers) (https://forum.kodi.tv/forumdisplay.php?fid=147)
+----- Forum: Movie Scrapers (https://forum.kodi.tv/forumdisplay.php?fid=302)
+----- Thread: Release Universal Movie Scraper (/showthread.php?tid=129821)
Pages:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
RE: [Release] Universal Scraper - olympia - 2013-01-13
(2013-01-13, 08:41)kwoodkicker Wrote: I have been running into an issue today and not sure if it is scraper related or not.No, it's not.
RE: [Release] Universal Scraper - kwoodkicker - 2013-01-13
Any idea what it is, olympia? Below is section of my XBMC log if that helps.
Code:
10:21:14 T:3724 WARNING: XFILE::CFileCurl::CReadState::FillBuffer: curl failed with code 22
10:21:14 T:3724 ERROR: CFileCurl::CReadState::Open, didn't get any data from stream.
10:21:14 T:3724 ERROR: ADDON::CScraper::Run: Unable to parse web site
10:22:11 T:2288 ERROR: Control 1 in window 10099 has been asked to focus, but it can'tAny help would be great, as I have used your scraper many times without issue.
RE: [Release] Universal Scraper - olympia - 2013-01-13
One of the sites you are scraping from is lagging. I suspect it is trak.tv. I had the same experience yesterday and today, but I didn't feel this as painful as you do...
You can try to increase curl timeout and/or ignore these errors in advancedsettings.xml, so you can re-scrape the missed ones afterwards.
RE: [Release] Universal Scraper - kwoodkicker - 2013-01-13
Alright, I will give that a go. I have confirmed TMDB scraper does not have the same issue. I will report back if problems persist, but I think you are correct in that it is a site which is lagging, but I believe it is HD-Trailers.net.
RE: [Release] Universal Scraper - dekkoparsnip - 2013-01-16
I love the Universal Scraper, and I'm in awe of your efforts of putting it together. One thing I did have a question about that I didn't see in this thread is that I have a problem finding films that have a category of "Short" -- is this a setting that I'm just missing? I have a number of short films and it doesn't seem to be able to locate them. I apologize if this has been asked before, but while I found lots on TV series, I didn't see any particular question on problems with scraping info for short films.
Here's one, as an example: http://www.imdb.com/title/tt0135785/
RE: [Release] Universal Scraper - Syncopation - 2013-01-18
Is there a Universal Scraper for TV Shows as well? I noticed this is only for movies. Is there any info about scrapers on the wiki? I was looking for an overview (sorry I can't read 43 pages of this thread).
Having issues with the TVDB.org giving me a wrong result for a TV show. Can I manually correct that?
Also I'm seeing a lot of this: http://cl.ly/image/3G2l3L051Q3s Not sure if that can be improved.
RE: [Release] Universal Scraper - takoi - 2013-01-19
(2013-01-04, 22:07)olympia Wrote:(2013-01-04, 21:50)takoi Wrote: True, agree on the first one. Maybe I remember incorrectly and typed it manually the previous time. But the second though clearly says international english title. Would be nice to have that oneParsing AKA titles from IMDb is a nightmare due to the too many variants. I spent a week back then to stabilize it into the stage where it is today. I am not touching that part of the code unless there are at least 5 other titles which fail to scrape due to the same issue
Caché (Hidden), should be Caché
http://www.imdb.com/title/tt0387898/
Cube²: Hypercube, should be Cube 2: Hypercube
http://www.imdb.com/title/tt0285492/
El laberinto del fauno, should be Pan's Labyrinth
http://www.imdb.com/title/tt0457430/
Triad Election, should be Election 2
http://www.imdb.com/title/tt0491244/
Mushukunin mikogami no jôkichi: Kawakaze ni kako wa nagareta, should be Fearless Avenger
http://www.imdb.com/title/tt0887736/
Full Metal gokudô, should be Full Metal Yakuza
http://www.imdb.com/title/tt0299910/
Goyôkiba, should be Hanzo the Razor: Sword of Justice
http://www.imdb.com/title/tt0068650/
King of Kung Fu, should be The Forbidden Kingdom
http://www.imdb.com/title/tt0865556/
Long zhi ren zhe, should be Ninja in the Dragon's Den
http://www.imdb.com/title/tt0084267/
Chaiya, should be Muay Thai Fighter
http://www.imdb.com/title/tt1090782/
I realize there are many different cases here and the naming on imdb is a mess, but for a few of these it isn't obvious to me at least why it doesn't work. For instance, 1 sometimes it try to get aka title when it clearly shouldnt like with The Forbidden Kingdom. Country is usa and language english 2, it seems to have trouble with the "International (imdb display title) (English title)" titles, picking "usa" over that..
Also, here's a few files it fails to find any results from:
A Fei.zheng.chuan.1990
All.About.Lily.Chou-Chou.2001
Baburu.e.go!!.Taimu.mashin.wa.doramu-shiki
Behind.the.Yellow.Line.1984
Bottle.Rocket.1994
Daai.si.gin.2004
Fist.Of.Fury.1972
Hong.Kong.1941.1984
Now.And.Forever.2006
Ping.Pong.2002
Pirates.Of.The.Caribbean-Dead.Mans.Chest.2006
Ramblers.2003
I think there's some issue related to symbols here, but event after manually typing the query many still fails. What search engine is this scraper using? This is only A-R btw
RE: [Release] Universal Scraper - olympia - 2013-01-19
You will add nfo/mixed nfo files for these.
RE: [Release] Universal Scraper - peppe_sr - 2013-01-21
hi, is there a way to download only the 16:9 images during scrape.
i poste this in the subforum of the skin i'm using,
http://forum.xbmc.org/showthread.php?tid=150880&pid=1305233#pid1305233
but could be useful the ability to set the aspect or the dimension of the fanarts to scrape.
peppe
RE: [Release] Universal Scraper - olympia - 2013-01-21
(2013-01-21, 12:20)peppe_sr Wrote: hi, is there a way to download only the 16:9 images during scrape.
i poste this in the subforum of the skin i'm using,
http://forum.xbmc.org/showthread.php?tid=150880&pid=1305233#pid1305233
but could be useful the ability to set the aspect or the dimension of the fanarts to scrape.
peppe
FYI This is the Universal Movie Scraper, not the music scraper.
Other than that, the quick solution for your issue is to use good quality site for image sources only.
Currently this is only fanart.tv
RE: [Release] Universal Scraper - peppe_sr - 2013-01-21
i'm sorry.
thankyou for your replay.
peppe
RE: [Release] Universal Scraper - jardi - 2013-01-22
Hello everyone,
I have a problem with this scraper (and the non ASCII characters, like accents in french language).
Basically, when there is an accent, it is replaced by the URL-encoded UTF-8 code of this character. For instance, é becomes é.
Not every movies fields are affected by this bug, and it depends on the site used to fetch the data.
For example :
- With the scraper configured for IMDb, the main title is affected.
- With the scraper configured for themoviedb.org, the main title is not affected, but the original title and the description are.
But an example is woth a thousand words, so here is one :
The movie "La cité de la peur" becomes "La cité de la peur".
You can found a complete debug log captured while fetching this movies data (with de script configured to use themoviedb.org) here : http://pastebin.com/hFXQ3LSj
Any help would be really apreciated.
PS: My whole database is full of these crappy-encoded characters, I was thinking about making a script to clean it up (like read, replace, write on the sqlite db), do you know if someone made something remotely like that so that I can build on it, or is it a totally bad idea ?
PS2: I noticed the same behaviour from the now deprecated IMDb scraper (http://wiki.xbmc.org/index.php?title=Add-on:IMDb), so it might not be specific to this scraper and could require a global correction.
PS3: In case you asked : yes it used to work perfectly and no I can't remember when it stop working.
RE: [Release] Universal Scraper - scudlee - 2013-01-22
I can't reproduce this on my end (on Windows). The log results look identical, but everything displays correctly.
RE: [Release] Universal Scraper - jardi - 2013-01-23
Thanks scudlee, I forgot to mention I was using linux (Ubuntu 12.04 to be precise).
So I guess this issue is OS-specific. Any hint on where I should report it ?
RE: [Release] Universal Scraper - olympia - 2013-01-23
Your guess is wrong, works fine here with Ubuntu 12.10.
Don't see what can be the problem either.