ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work... - Printable Version
+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Scraper Development (/forumdisplay.php?fid=60)
+--- Thread: ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work... (/showthread.php?tid=50055)
- Nicezia - 2009-07-10 22:49
a question for users of this, would you rather the urlencoded trailer link or should i change it to a simple link if its urlencoded?
- smeehrrr - 2009-07-10 23:33
Nicezia Wrote:a question for users of this, would you rather the urlencoded trailer link or should i change it to a simple link if its urlencoded?I don't understand this question.
- Nicezia - 2009-07-11 04:49
well some trailers come urlencoded (i.e. http%3A%2F%2Fwww.foo.com%2Furl.flv)
would you rather me leave it like that before sending to final details or would you like to decode the urlencoding? (seeing as how i'm creating a charset convertor)
- smeehrrr - 2009-07-11 07:06
It would make more sense for your API to return a decoded URL in the MovieTag object, I think. At any rate it should be consistent, so if some scrapers return an encoded url and others return a decoded one, you should pick one and normalize to that form always.
- xyber - 2009-07-11 13:23
Nicezia Wrote:Will edit it to log scraper return values
It was http://www.themoviedb.org/ that was returning a server 500 error as someone else mentioned. Not error in your lib. What I was thinking though is that it would be nice if you could create a class one could access to query errors that occured.
It was one of the XElement SecondPass = CustomFunctionParse(item2); calls that caused the exception. Can't remember if it was the first or second pass one.
Anycase, now that I think bout it, it might be hard to actually tell what went wrong as it could be a query to a server for trailers, or one for fanart or another server for posters. So it won't be easy knowing what to tell the user to turn off in the scraper settings.
- xyber - 2009-07-11 13:25
Nicezia Wrote:well some trailers come urlencoded (i.e. http%3A%2F%2Fwww.foo.com%2Furl.flv)
Don't really care. Long as I can use it to download the trailer via my app when I finally get around coding that part. Guess it would be better to show an unencoded version to the user if an app where to show the user the url.
- xyber - 2009-07-11 16:28
Does the TV Show section of the lib work? I get very strange results or none at all using TvScraper.CreateSearch and TvScraper.GetDetails
this is TvScraper.GetDetails for Heroes on IMDB
and rest of it is only episodeguide, thumbs, backdrops and actor tags.
and I just spot this in scraper.log
- ultrabrutal - 2009-07-11 19:12
when will we see some of these fixes hitting svn and a new build? (don't have C# compiler installed). I still get crashes in GetDetails and they are not solely because of fanart
- Nicezia - 2009-07-12 02:56
Well i'm in the process of testing a new release now, i suppose somewhere between midweek and the weekend I'll be submitting it to svn,
no TV shows isn't working just yet in the version that's in svn, however i have it working fully in the version i'm testing. and hope to upload really soon.
- Nicezia - 2009-07-12 02:59
If anyone knows anything about ISO charsets & windows codepages (for conversion purposes) i need someone to help with a charset conversion utility from non-latin character sets to unicode