[RELEASE] Scraper Editor (Based on ScraperXML open source C# Library) - Help wanted!

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
Nicezia Offline
Fan
Posts: 369
Joined: Nov 2006
Reputation: 0
Location: Montgomery, Alabama
Thumbs Up  [RELEASE] Scraper Editor (Based on ScraperXML open source C# Library) - Help wanted! Post: #1
Scraper editor partially finished (finished enough to make scrapers)

Haven't added the tester yet, as i'm having problems with scraperXML's http protocol in c#

Runnning on Linux via Mono (Notice there were a few bugs i had to work out on it namely the bindings, but now it runs error free on linux)[Image: ScraperEditorLinuxviaMono.png]


Running on windows (Notice the highlighting of Matches and turning text of match groups to Red)
[Image: ScraperEditorOnWindowsExpressionEdi.png]

Download


Just a slight note, use ctrl+v to paste text into box below as i haven't included a context menu for that yet, in the expression box i haven't yet finished the context menu, (which will include common regular expression which will be configurable via an XML file)

This will load edit and save any scraper made from scratch or that comes with XBMC. In it is an enhanced version of the Excalibur Scraper (which gets linked actors AND non-linked actors - and instead of guessing at image names actually goes and downloads images from the actor page)

oh and as a reminder !!!!DO NOT!!!!! use XML entities in the editor, put the pure xml in and entities will be managed on compiling the XML file!

ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

[Image: teamumx_sigline.png]
(This post was last modified: 2009-07-05 00:30 by Nicezia.)
find quote
spiff Online
Grumpy Bastard Developer
Posts: 12,180
Joined: Nov 2003
Reputation: 82
Post: #2
nice! just not having to write those xml tags will save tons of time Smile

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
DonJ Offline
Team-XBMC Member
Posts: 490
Joined: May 2005
Reputation: 5
Post: #3
Looks great, thanks for the effort!

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
zag Online
Team-XBMC Member
Posts: 796
Joined: Oct 2007
Reputation: 7
Location: UK
Post: #4
Excellent work, very useful
find quote
Nicezia Offline
Fan
Posts: 369
Joined: Nov 2006
Reputation: 0
Location: Montgomery, Alabama
Post: #5
Currently adding xml editing for those "purists" and integrating ScraperXML for testing scrapers. All previously broken features are fixed, and hope to release the full version of it by end of next week. ScraperXML will be updated as soon as finish all its integration featues (using this for an integration demonstration.

ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

[Image: teamumx_sigline.png]
find quote
C-Quel Offline
Retired Team-XBMC Member
Posts: 1,378
Joined: Aug 2004
Reputation: 0
Post: #6
Big Grin

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]

If scraper related please always grab the latest XML relevant to the content you are trying to grab info for from this link https://xbmc.svn.sourceforge.net/svnroot...m/scrapers

System Specs:

A Computer with loads of shiny things that make a noise and bring life to my tv, and xbmc ofc :)

iNerd Store

iNerd Forum
find quote
Gamester17 Offline
Team-XBMC Forum Moderator
Posts: 10,595
Joined: Sep 2003
Reputation: 9
Location: Sweden
Thumbs Up  Nice Nicezia! Post: #7
Great stuff, thanks! Cool

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
DonJ Offline
Team-XBMC Member
Posts: 490
Joined: May 2005
Reputation: 5
Post: #8
A type of 'debug mode' would be brilliant for the app. So you can throw a query search term or nfo at the app and step through the regexp expressions and functions while seeing the values of the buffers.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
Nicezia Offline
Fan
Posts: 369
Joined: Nov 2006
Reputation: 0
Location: Montgomery, Alabama
Post: #9
DonJ Wrote:A type of 'debug mode' would be brilliant for the app. So you can throw a query search term or nfo at the app and step through the regexp expressions and functions while seeing the values of the buffers.

Well i'm making it so that one can test each functon individually you can give a url (or local file) to test the function with, as far as stepping through i'd have to make a few modifications to the scraperxml library just for the tester, but i'm already doing that with other options (like the individual function testing took a few mods to the source of ScraperXML)


As for the buffers, i curretly have it so you can watch up to three buffers at once (each watch can be selected via combobox) anything more than three at a time can get too crowded

DonJ if you give me a list of your ideas for this "debug mode" i can work on integrating them into the program

ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

[Image: teamumx_sigline.png]
(This post was last modified: 2009-06-18 19:09 by Nicezia.)
find quote
Nicezia Offline
Fan
Posts: 369
Joined: Nov 2006
Reputation: 0
Location: Montgomery, Alabama
Post: #10
New Release coming monday
(I figure i'll give myself the weekend to work out the bugs in linux)


Function Editor
[Image: FunctionEditor.jpg]

RegExp Editor
[Image: RegExpEditor.jpg]

Scraper Info Editor
[Image: ScraperInfoEditor.jpg]

Tester
[Image: Tester.jpg]
(Tester Fully Functional in windows, Linux has a problem with the walk mode)

One can throw anything at the Tester, Simulate XBMC's passed values, open NFO's, XML's saved html.. or set the buffers to whatever you want. As shown below

[Image: Tester_QueryMode.jpg]


Is this what you had in mind DonJ Confused

ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers


I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!

[Image: teamumx_sigline.png]
(This post was last modified: 2009-06-20 21:33 by Nicezia.)
find quote
Post Reply