TV Show Scrapers

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
LukeS7 Offline
Junior Member
Posts: 30
Joined: Mar 2012
Reputation: 3
Post: #1
I don't know if Scrapers are taken care of on the website side or the XBMC side of things, but if possible I think there should be more XBMC scrapers for TV Shows. While TheTVDB is usually great I have ran into an issue recently. They are currently refusing to recognize Legend of Korra as a separate series from Avatar: The Last Airbender and instead have lumped it in as "Season 4." This decision is apparently due to the fact that 2 of their more than 10 moderators agreed it should be this way. I posted on their forums about this, it was my first post there, and basically got the response "Shut up, you shouldn't have even posted this" and had the thread locked so I couldn't reply. The moderators on their forums are really rude and seem to have a "God Complex" that they can never be wrong and their word is gospel even when presented with evidence that completely contradicts it. It is clearly an unpopular decision as at least 6 threads have been started by different users about it. Any help with getting new scrapers or your opinions on this decision (or posting on TheTVDB forums) would be appreciated.
find quote
Martijn Offline
Team-XBMC
Posts: 10,304
Joined: Jul 2011
Reputation: 154
Location: Dawn of time
Post: #2
You know any other source that has a complete tv show listing?

Scrapers are not the problem, the lack of sites is

Always read the XBMC online-manual, FAQ and search the forums before posting.
Do NOT e-mail Team-XBMC members asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting, make sure you read this first

For your mediacenter artwork go to
[Image: fanarttv.png]
find quote
LukeS7 Offline
Junior Member
Posts: 30
Joined: Mar 2012
Reputation: 3
Post: #3
(2012-03-27 00:17)Martijn Wrote:  You know any other source that has a complete tv show listing?

Scrapers are not the problem, the lack of sites is

TVRage
find quote
Martijn Offline
Team-XBMC
Posts: 10,304
Joined: Jul 2011
Reputation: 154
Location: Dawn of time
Post: #4
(2012-03-27 00:41)LukeS7 Wrote:  
(2012-03-27 00:17)Martijn Wrote:  You know any other source that has a complete tv show listing?

Scrapers are not the problem, the lack of sites is

TVRage

Their site isn't even capable of properly handling the request from TV show next aired add-on. Let alone handle the vast amount if data it would get when lots of users switch to their site for scraping.

If you see the log there a vast amounts of connection time outs.

Ofcourse that doesn't mean a scraper would not possible. Some one just need to create one

Always read the XBMC online-manual, FAQ and search the forums before posting.
Do NOT e-mail Team-XBMC members asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting, make sure you read this first

For your mediacenter artwork go to
[Image: fanarttv.png]
(This post was last modified: 2012-03-27 01:02 by Martijn.)
find quote
LukeS7 Offline
Junior Member
Posts: 30
Joined: Mar 2012
Reputation: 3
Post: #5
(2012-03-27 01:01)Martijn Wrote:  
(2012-03-27 00:41)LukeS7 Wrote:  
(2012-03-27 00:17)Martijn Wrote:  You know any other source that has a complete tv show listing?

Scrapers are not the problem, the lack of sites is

TVRage

Their site isn't even capable of properly handling the request from TV show next aired add-on. Let alone handle the vast amount if data it would get when lots of users switch to their site for scraping.

If you see the log there a vast amounts of connection time outs.

Ofcourse that doesn't mean a scraper would not possible. Some one just need to create one

Ah, okay, so essentially stuck with TheTVDB and their "staff," and I use that term in the smallest sense possible, for the time being
find quote
fr0stbyt3 Offline
Junior Member
Posts: 2
Joined: Mar 2012
Reputation: 0
Post: #6
I know it doesn't really help, but you can use external scraper apps for the time being. I use metabrowser 2.0. I am sure there are free apps like this for xbmc. Basically this show goes into a folder to be processed, before it get to the folder. So all xbmc has to do is load the info that is there.
find quote
Martijn Offline
Team-XBMC
Posts: 10,304
Joined: Jul 2011
Reputation: 154
Location: Dawn of time
Post: #7
(2012-03-27 07:40)fr0stbyt3 Wrote:  I know it doesn't really help, but you can use external scraper apps for the time being. I use metabrowser 2.0. I am sure there are free apps like this for xbmc. Basically this show goes into a folder to be processed, before it get to the folder. So all xbmc has to do is load the info that is there.

And they still get their info from TVDB.

Altough I don't approve the way they respond to question (haven't experienced this myself) I do understand that they lock episode and keep a strict guideline on how things are done.

If you have 1000 users all of them will disagree on certain things. So that why they do things the way they do at the moment

Always read the XBMC online-manual, FAQ and search the forums before posting.
Do NOT e-mail Team-XBMC members asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting, make sure you read this first

For your mediacenter artwork go to
[Image: fanarttv.png]
find quote
LukeS7 Offline
Junior Member
Posts: 30
Joined: Mar 2012
Reputation: 3
Post: #8
(2012-03-27 11:59)Martijn Wrote:  
(2012-03-27 07:40)fr0stbyt3 Wrote:  I know it doesn't really help, but you can use external scraper apps for the time being. I use metabrowser 2.0. I am sure there are free apps like this for xbmc. Basically this show goes into a folder to be processed, before it get to the folder. So all xbmc has to do is load the info that is there.

And they still get their info from TVDB.

Altough I don't approve the way they respond to question (haven't experienced this myself) I do understand that they lock episode and keep a strict guideline on how things are done.

If you have 1000 users all of them will disagree on certain things. So that why they do things the way they do at the moment

Consider yourself lucky that you haven't, here's the typical thread regarding this show (this is quoted exactly from an actual thread):

User: Searching for "Korra" gives 3 AKAs for Avatar: The Last Airbender (the previous series, 74852), a buggy incorrectly-named result (251085), and a valid result (257570). All 3 of those AKAs should be removed, and the development titles "Avatar: Legend of Korra" and "The Last Airbender: Legend of Korra" should be AKA'd to 257570 and the incorrect, buggy 251085 should be removed. The current state causes some API clients to match The Legend of Korra to The Last Airbender, which is certainly incorrect.

Moderator: Don't worry, I deleted the duplicate entry you just added. Unfortunately and as I've said before 251085 can only be removed by an admin, but aliases aren't API searchable anyway so it's not really a problem. Do not add this series again. There are extensive threads on the topic if you search the forum and we will not be changing it unless the conditions I specified are met.

Nothing wrong with that bit, but when the user responds with the information to fulfill the conditions, well I'll let you read:

User: I've read through all of the threads related to The Legend of Korra and seen debates on whether it's a spinoff or a sequel. Honestly, to me, that's pure semantics and doesn't really make a difference; the show has been referred to as a "spin-off", "sequel", "something new", "new from the creators of", etc. I don't see why it even matters what we call it; what's clear to me is that Avatar: The Last Airbender was a show with its own creative and voice cast, set in its own time period. The Legend of Korra takes place at least 70 years later, in a world which has evolved to the point where none of the original settings of The Last Airbender have been shown, or even mentioned. There is an entirely new voice cast, and unlike The Last Airbender, which had a larger creative team who wrote the episodes, each episode of The Legend of Korra will be written entirely by the two co-creators. The only thing that the two series share is that they take place in the same universe, and if that's the only point necessary to have two series merged, then I should ask why there's a Doctor Who and a Doctor Who (2005) in the database when those two share much more than a universe and two writers.

I also should ask, what's the actual reasoning behind moving Korra to Season 4 of TLA? Korra will have two seasons, which will (likely) have their own DVD releases. Will the actual (canonical, defined by Nick/Viacom and the creators) Book 2 of Korra be labeled Season 5 of TLA? This just seems illogical to me, as it makes using this series with HTPC software difficult to impossible for most fans (who, like Nick/Viacom, consider this a new and separate entity). What are the advantages for the users of TheTVDB of having this show grouped in, when its episodes are universally labeled along the lines of "The Legend of Korra S01E01"?

Oh, also, ID 251085 is still in the database with an incorrect title, and it's impossible to edit.

Also also, regarding my original duplicate entry: when I was trying to enter Korra into TheTVDB (because Plex on my NAS wouldn't match it properly [it still won't]), I found 3 AKAs sending it to TLA, which I assumed were leftovers from when the show was first announced and it appeared to be more of a continuation than anything new. I found 251085 and tried to edit it, found it to be too bugged out to work, and set out to make a new entry, hoping that the old, incorrect one would be fixed and that I could find a mod to change the AKAs later (thus my original forum thread). Given the only non-AKA Korra entry being buggy to the point of uselessness, I think adding a duplicate entry was a reasonable course of action in this case.

If you still think that keeping Korra as seasons 4 and 5 (possibly more, note, as Nick may order another extension) of TLA, then I certainly can't stop you. I don't have to use this database, as you've mentioned, and I can certainly set my metadata manually. I just hoped that a DB like this one, which usually works wonderfully to match whatever I throw at it, might bend towards a format that can allow more actual viewers to use it conveniently.

Moderator: Guess how much of a shit I give.

And then they locked the thread so the user couldn't reply.

The Moderators (least the ones I've dealt with) demand respect from the users in that they follow the rules, post in the correct section and include links to everything; but the moderators have 0 respect for the users.

P.S. sorry for kinda steering this off of scrapers, just venting frustration a bit, and if I even tried posting something like this on the forums there they'd ban me most likely.
(This post was last modified: 2012-03-27 22:19 by LukeS7.)
find quote
jmarshall Offline
Team-XBMC Developer
Posts: 25,689
Joined: Oct 2003
Reputation: 169
Post: #9
OffTopic: I really, really, REALLY, don't want to get into this, but your previous post is essentially meaningless unless you can back it up with facts. Include the links so that others can see for themselves what went on and you will have a much stronger case.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
LukeS7 Offline
Junior Member
Posts: 30
Joined: Mar 2012
Reputation: 3
Post: #10
(2012-03-28 02:44)jmarshall Wrote:  OffTopic: I really, really, REALLY, don't want to get into this, but your previous post is essentially meaningless unless you can back it up with facts. Include the links so that others can see for themselves what went on and you will have a much stronger case.

Gladly, I'll post the links to the threads regarding Legend of Korra:

The Thread That I Quoted Above ^^

My thread (after my first post the thread was locked, the second post is from the second thread I started, the moderator just merged them together):

Threads by others: (Thread 1 has the most info about the decision and the most examples of them ignoring evidence of why it should be its own entry)
Thread 1
Thread 2
Thread 3
Thread 4

They also locked each thread so the user couldn't reply. It's pissing me off even more now because they have now switched the series name to just "Avatar" so now I can't use the artwork downloader or the TV Show - Next Aired Plugin with either Legend of Korra or Avatar: The Last Airbender because there isn't a show called "Avatar" which means neither plugin picks it up.

EDIT: response I got when saying that it broke the plugins: Thread and it was of course locked so no reply could be made.
(This post was last modified: 2012-03-28 03:13 by LukeS7.)
find quote
zag Offline
Team-XBMC Member
Posts: 1,551
Joined: Oct 2007
Reputation: 18
Location: UK
Post: #11
Thetvdb source is out there so it would be possible to build an alternative.

HTPC - XBMC Gotham, OpenELEC, Harmony Smart Remote, Intel Haswell NUC, 40gb intel SSD, Core i3, 4gb RAM
Storage - 2 x qnap 8tb 419p+ NAS
Display LG 46" LCD + Casio Bulbless projector [PICS]
[Image: widget]
find quote
KRKA01 Offline
Member
Posts: 97
Joined: Sep 2011
Reputation: 0
Location: Stockholm, Sweden
Post: #12
(2012-03-27 11:59)Martijn Wrote:  
(2012-03-27 07:40)fr0stbyt3 Wrote:  I know it doesn't really help, but you can use external scraper apps for the time being. I use metabrowser 2.0. I am sure there are free apps like this for xbmc. Basically this show goes into a folder to be processed, before it get to the folder. So all xbmc has to do is load the info that is there.

And they still get their info from TVDB.

Altough I don't approve the way they respond to question (haven't experienced this myself) I do understand that they lock episode and keep a strict guideline on how things are done.

If you have 1000 users all of them will disagree on certain things. So that why they do things the way they do at the moment

I would go for fr0stbyt3 suggestion, use an external scrapper. As Martijn states, the external scrapers get their info from TVDB aswel, but they will also generate the .nfo files which you can edit and then let XBMC use this info when scanning these TV shows/episodes.

For Windows there is both Media Companion and Ember Media Manager

You can of cource create the .nfo files from scratch if you like, see the wiki for instructions: tvshow.nfo and episodename.nfo
find quote
Amra Offline
Cimmerian
Posts: 466
Joined: Jun 2009
Reputation: 11
Location: Metropolis
Post: #13
(2012-03-28 14:56)KRKA01 Wrote:  You can of cource create the .nfo files from scratch if you like
@LukeS7
This is what I did for Korra and it works very well. If you'd like I will gladly share what I have, it includes all the necessary images (fanart, poster, banner, season poster, logo, etc) as well as the nfo files with the correct actor info for the new series. Let me know.

PS: For what it's worth, I know what you are talking about with a particular mod at tvdb. I personally, just don't post on their forum anymore & try my best to avoid it.
(This post was last modified: 2012-03-28 15:38 by Amra.)
find quote
LukeS7 Offline
Junior Member
Posts: 30
Joined: Mar 2012
Reputation: 3
Post: #14
(2012-03-28 15:31)Amra Wrote:  
(2012-03-28 14:56)KRKA01 Wrote:  You can of cource create the .nfo files from scratch if you like
@LukeS7
This is what I did for Korra and it works very well. If you'd like I will gladly share what I have, it includes all the necessary images (fanart, poster, banner, season poster, logo, etc) as well as the nfo files with the correct actor info for the new series. Let me know.

PS: For what it's worth, I know what you are talking about with a particular mod at tvdb. I personally, just don't post on their forum anymore & try my best to avoid it.

That would be extremely helpful/awesome, PM'ing my email to ya.
find quote
Amra Offline
Cimmerian
Posts: 466
Joined: Jun 2009
Reputation: 11
Location: Metropolis
Post: #15
Check your PM for a download link. I didn't want to clog your email with attachments or risk your AV program deleting it. Hope it helps. Big Grin
(This post was last modified: 2012-03-28 19:14 by Amra.)
find quote
Post Reply