[RELEASE] Scraper Editor (Based on ScraperXML open source C# Library) - Help wanted!

UsagiYojimbo · 2010-07-16 20:21

I have a question:

Is it possible with the ScraperXML Editor to create a multi-format scraper?
I mean a scraper that can scrape EG movies as well as tv-shows or even music videos?

Nicezia · 2010-07-17 00:41

UsagiYojimbo Wrote:I have a question:

Is it possible with the ScraperXML Editor to create a multi-format scraper?
I mean a scraper that can scrape EG movies as well as tv-shows or even music videos?

that's a request better made in XBMC development, Scraperxml and all projects i work on as derivitives of it are based on XBMC scraper process.

in a way it is possible once the add-on structure is supported in ScraperXML Editor, but i can see that there would be all kinds of problems with a scraper of that sort...
for example... how would you know which scraper type to use when its running?
say you had this scraper with multi-format....

since TV Show scrapers, Movie Scrapers, and Music Video Scrapers all use the same function (CreateSearchUrl) to create a url... how would you tell it to create a different url for whatever type you are using?
i.e

Code:

[b][url]http://foo.com/tv/search.php?q=Family%20Guy[/url][/b]

- or -

[b][url]http://foo.com/musicvideos/search.php?artist=Eminem&video=The%20Way%20I%20Am[/url][/b]

if the site being used makes no distinction at all between types during a search, then i suppose it could be safe to do something like you suggest... but until XBMC officially supports something like this i doubt i'll be incorporating it into ScraperXML Editor

UsagiYojimbo · 2010-07-18 20:26

Nicezia Wrote:say you had this scraper with multi-format....

What I was interested in, is multiple scrapers (for the same site) in a single add-on...
Could this be created with new version of the editor?

Nicezia · 2010-10-05 00:23

Looking for someone who knows how to create GTK+(& possibly QT) GUIs to give me a crash tutorial so i can make the ScraperXml Editor cross-platform through mono (and eventually in pure C++).

flobbes · 2010-10-05 21:38

Nicezia Wrote:Looking for someone who knows how to create GTK+(& possibly QT) GUIs to give me a crash tutorial so i can make the ScraperXml Editor cross-platform through mono (and eventually in pure C++).

I send you a pm.

Nicezia · 2010-11-28 00:41

UsagiYojimbo Wrote:I have a question:

Is it possible with the ScraperXML Editor to create a multi-format scraper?
I mean a scraper that can scrape EG movies as well as tv-shows or even music videos?

The new answer to this question is : the newest version (not yet released) it is possible to make a scraper with 1 of all scraper types all in one addon. Addon's are treated as projects. So its concievable that you can make a addon containing a movie scraper, a tv scraper, a music video scraper, a album scraper, a artist scraper and even a include/import, all package in one. and still build/edit each scraper in it as individual scrapers (as long as each scraper points to its own library file)

UsagiYojimbo · 2011-03-07 17:40

Nicezia Wrote:the newest version (not yet released)

May I ask, when will it be released? Or even a beta version?

mortstar · 2011-03-31 02:53

@Nicezia

Just a quick thank you from a newbie scraper dev. I would have given in very quickly without your tool.

I'm also glad for the person that mentioned they had missed the latest version as your link to the SF project has version 2.15 on the homepage, which I had naively downloaded without properly looking. Glad I found v3.6.9.5 tucked away in the Files section. Maybe you could make your link a little more direct to the latest version?

Anyway, big thanks again!

akuiraz · 2011-04-06 08:20

I would love to see this project adapted to work with XBMC 10.0/10.1's addon style scrapers... I see that you last posted about the new unreleased version back in november... has there been any more recent development?

UsagiYojimbo · 2012-10-09 07:04

No new development ever since?

I put together a similar editor in Java. If anybody interested in it, I could upload somewhere... Cool

UsagiYojimbo Member Posts: 83 Joined: Feb 2010 Reputation: 1 Location: Debrecen, Hungary	2010-07-16 20:21 Post: #131 I have a question: Is it possible with the ScraperXML Editor to create a multi-format scraper? I mean a scraper that can scrape EG movies as well as tv-shows or even music videos? ScraperEdit for XBMC (Java)
	find quote

UsagiYojimbo Member Posts: 83 Joined: Feb 2010 Reputation: 1 Location: Debrecen, Hungary	2010-07-18 20:26 Post: #133 Nicezia Wrote:say you had this scraper with multi-format.... What I was interested in, is multiple scrapers (for the same site) in a single add-on... Could this be created with new version of the editor? ScraperEdit for XBMC (Java)
	find quote

flobbes Senior Member Posts: 141 Joined: Mar 2009 Reputation: 0	2010-10-05 21:38 Post: #135 Nicezia Wrote:Looking for someone who knows how to create GTK+(& possibly QT) GUIs to give me a crash tutorial so i can make the ScraperXml Editor cross-platform through mono (and eventually in pure C++). I send you a pm.
	find quote

UsagiYojimbo Member Posts: 83 Joined: Feb 2010 Reputation: 1 Location: Debrecen, Hungary	2011-03-07 17:40 Post: #137 Nicezia Wrote:the newest version (not yet released) May I ask, when will it be released? Or even a beta version? ScraperEdit for XBMC (Java)
	find quote

akuiraz Member Posts: 75 Joined: Apr 2010 Reputation: 0	2011-04-06 08:20 Post: #139 I would love to see this project adapted to work with XBMC 10.0/10.1's addon style scrapers... I see that you last posted about the new unreleased version back in november... has there been any more recent development?
	find quote

UsagiYojimbo Member Posts: 83 Joined: Feb 2010 Reputation: 1 Location: Debrecen, Hungary	2012-10-09 07:04 Post: #140 No new development ever since? I put together a similar editor in Java. If anybody interested in it, I could upload somewhere... ScraperEdit for XBMC (Java)
	find quote