XBMC Community Forum
[RELEASE] Scraper Editor (Based on ScraperXML open source C# Library) - Help wanted! - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Scraper Development (/forumdisplay.php?fid=60)
+--- Thread: [RELEASE] Scraper Editor (Based on ScraperXML open source C# Library) - Help wanted! (/showthread.php?tid=52929)

Pages: 1 2 3 4 5 6 7 8 9 10 11


- redtapemedia - 2009-08-04 09:58

spiff Wrote:the includes wont make it one bit easier for you to add anything. only difference is the function's code would be in a different .xml file

Not directly no, but I've been working with a text editor thus far. Also modifying the existing imdb scraper for ratings is not really a good idea as there would be two scrapers with nearly the same content. It's probably moreso the stepping functions that will help me as I haven't been able to glean much from the logs.


- Nicezia - 2009-08-04 10:20

well my whole thing (the reason why i started making the editor) is because i could never keep track mentally of what was where, and visualizing what should be (and knowing whether what i am trying to put) in the buffers is actually there, was a priority for me.
That and stupid mistakes like not putting a semicolon on the end of a entity.

somehow all that mushroomed into me creating a complete external scraper library before i actually understood enough about the internal workings of the scraper code to attempt to be able to write the editor.

I'm not completely satisfied with the edtor yet, and i'm doing alot of work on it at the moment (since until they add the new features to scraper code i'm actually and god knows i never thought i'd see the day i'd say this - I'm caught up to the current XBMC code in svn.

As you can see i've added quite a few features in just the last couple of days.

I'm now trying to whittle the scraper editor down to a literally fool-proof construction tool. weeding out the possibility of malformed xml, conditional mispells, stray open tags, entity handling...


- spiff - 2009-08-04 10:33

uhm, why don't you make a 'use rottentomato rating' setting and use that. having a completely new scraper is a waste


- Nicezia - 2009-08-04 12:45

Spiff i have some time on my hands, going to go through the working scrapers that haven't been updated (for the thumbnail changes) and fix those... plus work on some include files. tmdb-fanart i've already made an include file for. (because i kinda got tired of waiting to test a official-made include file Wink)


- spiff - 2009-08-04 12:49

cool. much appreciated Smile


- redtapemedia - 2009-08-04 22:58

spiff Wrote:uhm, why don't you make a 'use rottentomato rating' setting and use that. having a completely new scraper is a waste

I was going that route but I couldn't get my custom function to work properly.
I've managed to get most things working in a stand alone scraper, it's helped me get a better handle on how things work, it's just the custom function that's been causing grief.
I'm actually grabbing more than just the rating now, getting the synopsis, actors, etc. Is it worthwhile if I go that route or should I just do it as a setting like you've suggested?

Sorry for hijacking the thread a little, I don't wanna create a seperate thread until there's something substantial to show.


- spiff - 2009-08-04 23:03

if its more than the rating, it makes sense to have it as a separate one


- Daniel Malmgren - 2009-08-12 18:56

When I open my filmdelta scraper (the version that's in svn) with the editor (version 2.1.153.16) it crashes with the following:

Code:
Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object
  at ScraperXMLEditor.ScraperLib.ScraperFunction.Deserialize (System.Xml.Linq.XElement xFunctionElement) [0x00000]
  at ScraperXMLEditor.ScraperLib.ScraperFunction..ctor (System.Xml.Linq.XElement xFunctionElement) [0x00000]
  at ScraperXMLEditor.ScraperLib.Scraper.Deserialize () [0x00000]
  at ScraperXMLEditor.ScraperLib.Scraper..ctor (System.String xmlFilePath) [0x00000]
  at ScraperXMLEditor.Form1.OpenScraper () [0x00000]
  at ScraperXMLEditor.Form1.openToolStripButton_Click (System.Object sender, System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.OnClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripButton.OnClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.HandleClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.FireEvent (System.EventArgs e, ToolStripItemEventType met) [0x00000]
  at (wrapper remoting-invoke-with-check) System.Windows.Forms.ToolStripItem:FireEvent (System.EventArgs,System.Windows.Forms.ToolStripItemEventType)
  at System.Windows.Forms.ToolStrip.OnMouseUp (System.Windows.Forms.MouseEventArgs mea) [0x00000]
  at System.Windows.Forms.Control.WmLButtonUp (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.ScrollableControl.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.ToolStrip.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control+ControlWindowTarget.OnMessage (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control+ControlNativeWindow.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.NativeWindow.WndProc (IntPtr hWnd, Msg msg, IntPtr wParam, IntPtr lParam) [0x00000]
  at System.Windows.Forms.XplatUIX11.DispatchMessage (System.Windows.Forms.MSG& msg) [0x00000]
  at System.Windows.Forms.XplatUI.DispatchMessage (System.Windows.Forms.MSG& msg) [0x00000]
  at System.Windows.Forms.Application.RunLoop (Boolean Modal, System.Windows.Forms.ApplicationContext context) [0x00000]
  at System.Windows.Forms.Application.Run (System.Windows.Forms.ApplicationContext context) [0x00000]
  at System.Windows.Forms.Application.Run (System.Windows.Forms.Form mainForm) [0x00000]
  at ScraperXMLEditor.Program.Main () [0x00000]

I can't find any errors within the xml. Is it my fault or the editor being buggy?

/Daniel


- Nicezia - 2009-08-12 21:20

Daniel Malmgren Wrote:When I open my filmdelta scraper (the version that's in svn) with the editor (version 2.1.153.16) it crashes with the following:

Code:
Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object
  at ScraperXMLEditor.ScraperLib.ScraperFunction.Deserialize (System.Xml.Linq.XElement xFunctionElement) [0x00000]
  at ScraperXMLEditor.ScraperLib.ScraperFunction..ctor (System.Xml.Linq.XElement xFunctionElement) [0x00000]
  at ScraperXMLEditor.ScraperLib.Scraper.Deserialize () [0x00000]
  at ScraperXMLEditor.ScraperLib.Scraper..ctor (System.String xmlFilePath) [0x00000]
  at ScraperXMLEditor.Form1.OpenScraper () [0x00000]
  at ScraperXMLEditor.Form1.openToolStripButton_Click (System.Object sender, System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.OnClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripButton.OnClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.HandleClick (System.EventArgs e) [0x00000]
  at System.Windows.Forms.ToolStripItem.FireEvent (System.EventArgs e, ToolStripItemEventType met) [0x00000]
  at (wrapper remoting-invoke-with-check) System.Windows.Forms.ToolStripItem:FireEvent (System.EventArgs,System.Windows.Forms.ToolStripItemEventType)
  at System.Windows.Forms.ToolStrip.OnMouseUp (System.Windows.Forms.MouseEventArgs mea) [0x00000]
  at System.Windows.Forms.Control.WmLButtonUp (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.ScrollableControl.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.ToolStrip.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control+ControlWindowTarget.OnMessage (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.Control+ControlNativeWindow.WndProc (System.Windows.Forms.Message& m) [0x00000]
  at System.Windows.Forms.NativeWindow.WndProc (IntPtr hWnd, Msg msg, IntPtr wParam, IntPtr lParam) [0x00000]
  at System.Windows.Forms.XplatUIX11.DispatchMessage (System.Windows.Forms.MSG& msg) [0x00000]
  at System.Windows.Forms.XplatUI.DispatchMessage (System.Windows.Forms.MSG& msg) [0x00000]
  at System.Windows.Forms.Application.RunLoop (Boolean Modal, System.Windows.Forms.ApplicationContext context) [0x00000]
  at System.Windows.Forms.Application.Run (System.Windows.Forms.ApplicationContext context) [0x00000]
  at System.Windows.Forms.Application.Run (System.Windows.Forms.Form mainForm) [0x00000]
  at ScraperXMLEditor.Program.Main () [0x00000]

I can't find any errors within the xml. Is it my fault or the editor being buggy?

/Daniel


would have to see the current state scraper to figure that out but probably something i haven't accounted for


- Daniel Malmgren - 2009-08-12 21:41

Nicezia Wrote:would have to see the current state scraper to figure that out but probably something i haven't accounted for

As I said, it's the version in svn. Haven't changed a byte Wink

/Daniel


- Nicezia - 2009-08-12 23:17

hmmm, app error, will have it fixed in next version (there's not much error checking in the current ScraperXML Editor release, the version i'm working on corrects that...


- Daniel Malmgren - 2009-08-13 07:46

Nicezia Wrote:hmmm, app error, will have it fixed in next version (there's not much error checking in the current ScraperXML Editor release, the version i'm working on corrects that...

Thanks mate. Looking forward to it Nod

/Daniel


- flobbes - 2009-08-28 09:46

Which Wine and which mono version are you using, cant get in running under linux and have to use a virtual machine.

Code:
wine: Call from 0x7b8455b0 to unimplemented function gdiplus.dll.GdipCreateFontFamilyFromName, aborting
err:seh:raise_exception Unhandled exception code c00000fd flags 0 addr 0x68f5d3a4



- Daniel Malmgren - 2009-08-28 19:09

flobbes Wrote:Which Wine and which mono version are you using, cant get in running under linux and have to use a virtual machine.

Code:
wine: Call from 0x7b8455b0 to unimplemented function gdiplus.dll.GdipCreateFontFamilyFromName, aborting
err:seh:raise_exception Unhandled exception code c00000fd flags 0 addr 0x68f5d3a4

I don't use wine at all. Just "mono scraperxmled.exe" does the trick.

/Daniel


- flobbes - 2009-08-28 20:16

Great its working thanks, so I dont have to run my virtual machine with windows everytime anymore Smile