+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Scraper Development (/forumdisplay.php?fid=60)
+--- Thread: ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work... (/showthread.php?tid=50055)
- Nicezia - 2009-05-22 16:11
that will definately help keep things clarified
- Nicezia - 2009-05-23 13:47
ok simple question
if you're not using dvd ordering, and you're not using absolute number on a tvshow
what ordering is there left?
nevermind, i have a different question to ask as this phrase seems to be completely locking up my library
from the TVDB scraper:
Code:
<RegExp conditional="!absolutenumber" input="$$1" output="<episode><title>\2</title><url cache="$$10.xml">$$2</url><epnum>\3</epnum><season>\4</season><id>\1</id></episode>" dest="4">
<expression repeat="yes"><Episode>.*?<id>([0-9]+).*?<EpisodeName>([^<]*).*?<EpisodeNumber>([0-9]+)[^<]*.*?<SeasonNumber>([0-9]+)[^<]*.*?</Episode></expression>
</RegExp>i mean it completely freezes, so if i stop the program before this expression runs, the moment it locks up is when matching the expression, it finds an uncountable number of matches
anyone have any suspiscions as to why? all i've changed in the code is added the unzip and concat ability to it, and everything else still works fine, if i disable this phrase and enable dvdorder it works ok as well. which doesn't make sense to me since the expression is almost identical to this one.
Even when i check this expression against the combined contents of the zip in regexBuddy, it freezes up
- Gamester17 - 2009-05-23 14:42
@Nicezia, not sure if it is something you want in your ScraperXML library or not but FYI; changeset r20559 commits "Ability to scrape and scan TV Shows into the video library by air-date via TheTVDB.com" to XBMC scraper code and TheTVDB XML scraper, see patch #5143 on trac for more information.
- Nicezia - 2009-05-23 14:52
Gamester17 Wrote:@Nicezia, not sure if it is something you want in your ScraperXML library or not but FYI; changeset r20559 commits "Ability to scrape and scan TV Shows into the video library by air-date via TheTVDB.com" to XBMC scraper code and TheTVDB XML scraper, see patch #5143 on trac for more information.
I'll look into that, i've got my hands ful with the tvdb as it is, everything else i have solid, i've even finished up the music scrapers.
@spiff does "." (period) match newline or not? in the wiki it says its supposed to, which i allowed for in my library. should that be turned off?
- Nicezia - 2009-05-24 04:07
ah ha! found my problem, the conditional evaluation was just slightly out of place, so it was running statements that it shouldn't have.
ScraperXML now works with all XBMC scrapers. Soon as i do a little housekeeping onthe code, i'm going to up Version 4.0 and start working on Game, Books, And Comics!!!!!
Does anyone know of an GPL .NEt zip lbrary, that's easy to use, i tried sharpziplib, and i couldn't make heads or tails of it, before i release this i need to replace the ziplib i'm using (IonicZip) with a GPL compatible zip lib as from what i read i can't tell if the license its under (MS Public Liscence) is compatible with GPL.
Considering its a Microsoft Public License my first guess would be no.
My first choice would be to use SharpZipLib so if there's anyone that can help me to get it to do the things i need it to do that would be appreciated (and would speed up the release of 4.0, as i can't release it under the GPL as is, i don't think)
Edit: Nevermind again, my friend just dropped in and wrote a beautiful little simple routine for me for SharpZipLib, and made me feel stupid for not figuring this out on my own. damn those career programmers. In anycase, he's taken a copy of the program and is going to convert it into c#, so i should have a mono lib ready by the end of the weekend.
- Nicezia - 2009-05-24 08:52
I guess before i post i'm going to find someway to implement nfoUrl and getEpisodeGuide (seeing as how each of those are dependant upon finding the right scraper to use to retrieve the info - Currently the scraperParser runs through all processes and returns ALL the info as a complete xml - but i want to be able to do it on demand like XBMC does)
I guess i'm going to have to make an object to keep track of all the available scrapers, and be able to run the url against each.
XBMC's Behaviour? - Nicezia - 2009-05-24 09:41
How long does XBMC hold the scraper info in cache?
- Nicezia - 2009-05-24 14:06
Gamester17 Wrote:@Nicezia, not sure if it is something you want in your ScraperXML library or not but FYI; changeset r20559 commits "Ability to scrape and scan TV Shows into the video library by air-date via TheTVDB.com" to XBMC scraper code and TheTVDB XML scraper, see patch #5143 on trac for more information.
Supported now!
- spiff - 2009-05-24 19:55
we clear the cache on the start of each scrape, i.e. before calling CreateSearchUrl
- Nicezia - 2009-05-25 14:03
Just about got this thing wrapped up, (it being a holiday weeked kindatook alittle of my time. But all official xbmc scrapers support is wrapped up, i just need to write some kind of interface to NfoUrl , and EpisodeGuideUrl, and its in the bag