ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work... - Printable Version
+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Scraper Development (/forumdisplay.php?fid=60)
+--- Thread: ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work... (/showthread.php?tid=50055)
SVN Updated - Nicezia - 2009-07-06 00:30
Everything connected and working except TV Shows... I'm tired , and need to get some sleep, probably update again in the morning before heading off to work
stilll pretty proud of it even though its not completely finished, just nodding off at my computer (the TV scraper code is easy to understand anyway... and could be easily implemented (its only creating the TvScraper Object(by passing a scraperInfo object the cache folder path and a path to a log file - both the cachepath and the logfile can be nulled or string.Empty if you don't want to use cache or logging) then calling the 4 public methods as needed.
probably only about an hour's more worth of work for me to implement into the test programs, but i don't have an hour more steam in me.
- xyber - 2009-07-06 19:19
Had a quick look at your SVN update. Like what I see from the C# Test app
Gonna start integrating it into my project now. Will let you know how it goes.
- smeehrrr - 2009-07-06 19:39
What's the ultimate achievement, and how do you know that this one is the penultimate?
Joking aside, your code has saved me a ton of time on the project I'm currently working on, and thanks for it. I'm going to check out the C# update soon, and I should have some feedback for you after that.
- ultrabrutal - 2009-07-06 19:58
You need some more exception handling in there This error is raised when I open the scraper I'm working on. The old version can open it just fine...
- xyber - 2009-07-06 21:56
I noticed that my app wants to connect to thetvdb.com as soon as it starts. I think it happens when my code call your lib to load the scrapers.
Basically this section from your example...
Is there a reason for that?
- spiff - 2009-07-06 22:19
it's probably loading the scraper settings, which query tvdb for available languages
- xyber - 2009-07-06 23:01
Thanks. I'll just popup something incase it dalays too long. Living in SA gives me the "benefit" of seeing what users will experience under the worse internet connection conditions.
Guess that is why tvdb.xml was renamed to tvdb.xml.xox
- smeehrrr - 2009-07-06 23:32
Quick bug report here. On my build, boolean settings are not working properly. The problematic code is in the ScraperSetting constructor:
The problem is that the bool.Parse.ToString returns an uppercased "True" and the scrapers apparently expect it to be lowercase. I added a .ToLower() at the end of that line and everything started working again.
I'll have lots more comments coming. Got my code ported over from the older version to the new one pretty easily modulo this settings problem.
- Nicezia - 2009-07-07 20:08
just add ToLower(); at the end of that bool.Parse(foo);
- Nicezia - 2009-07-07 20:13
ultrabrutal Wrote:You need some more exception handling in there This error is raised when I open the scraper I'm working on. The old version can open it just fine...
What part of your Code is crashing it? what exactley is your scraper doing when it has this problem?