ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work...

Nicezia · 2010-01-08 22:54

thanks

Vincent81 · 2010-02-08 14:46

Hello,
Would it be possible to use ScraperXML with command line or DLL in my program (XBNE : is not developed in C) ?
And How to..

Thanks

Nicezia · 2010-02-09 00:11

i could throw together a comandline inteface for it easily, that returns an xml formatted string.

however the end user would still need .Net 3.5 or (mono 2.0 or greater for linux) to use it.

I've been using your project for quite a while to update my library myself, and been wondering about whether you'd be interested in using ScraperXML.

I'd be happy to work along side you to get it working together with your program.
if you PM me we can discuss a way to make it work smoothly through DLL...

tehashix · 2010-04-23 23:31

Hello Nicezia,

First I would like to thank you a lot for the great work you did with ScraperXML. It's an amazing library Big Grin

I'm trying to use your library in my little software "Dune Explorer" (HDI Dune player customization tool). I started with the sources on Sourceforge to correct a little issue depending on localization :
- When you convert values with the ToDouble function, it depends on windows regional settings and sometimes the dot (".") is not the decimal separator, so you got an error. Here is an example of the correction I've applied on MediaTag.cs (I set a NumberFormatInfo to force the dot (".") as decimal separator)

Code:

internal double ProcessRating(XElement ratingElement)

        {

            if (ratingElement != null)

            {

                NumberFormatInfo provider = new NumberFormatInfo();

                provider.NumberDecimalSeparator = ".";

                provider.NumberGroupSeparator = ",";

                provider.NumberGroupSizes = new int[] { 3 };

                if (ratingElement.Attribute("max") != null)

                {

                    double scale = 10.0 / Convert.ToDouble(ratingElement.Attribute("max").Value,provider);

                    return scale * Convert.ToDouble(ratingElement.Value,provider);

                }

                else

                {

                    return Convert.ToDouble(ratingElement.Value,provider);

                }

            }

            return 0.0;

        }

It works great with Movies but I've got some questions about how the TV Shows work. For TV Shows I use the same functions as for Movies :

ScraperManager.GetResults to search for the tv show (with the tvshow MediaType and tvdb scraper)
ScraperManager.GetDetails to get Details from one ScrapeResultEntity

The GetDetails function returns a TVShowTag which is almost empty, there is only an EpisodeGuide available in the object, nothing else, no actors, no directors, no Episodes, no Fanart, no Thumbnail, etc...

So, my first question, how can I get theses informations in a TVShowTag object ? (I'm a little confused with TV Shows :sad Smile

My second question is about ScraperSettings, I would like to allow the user to configure scraper settings once (and not every time the application is launched). So, is there a way to save the ScrapperSettings and load them on an application startup ?

Again, thank you a lot for your work.

Regards,

TeHashiX

patmtp35 · 2010-05-10 21:07

hi!

just discover your scraper commandline, it works great on imdb.com, but as i m french i would have to use it with ciné-passion, but i can't have it to works please could you help me ?

regards

tehashix · 2010-05-11 13:09

patmtp35 Wrote:hi!

just discover your scraper commandline, it works great on imdb.com, but as i m french i would have to use it with ciné-passion, but i can't have it to works please could you help me ?

regards

Hello,

Ciné-passion uses an API key now for applications. the XBMC scraper doesn't use this API key for the moment. So, I re-wrote the ciné-passion XBMC scraper (for ScraperXML), you can find it here : http://passion-xbmc.org/scraper-cine-pas...c-cle-api/

You just need to request an API key, replace %API_KEY% (in the scraper xml code) by your API key, and the scraper should work with ciné-passion database ;-)

**spiff** · 2010-06-23 16:41

new feature added. returned xml can now hold shit in the format

Code:

<chain function="SomeFunc">texttorunthefunctionon</chain>

very useful for scraper libraries (no need for users to know which url to do a search using etc). used extensively in the scrapers git so you can find tons of stuff to test with there.

**olympia** · 2010-06-23 16:50

Fixchars is also worth to mention Smile

http://trac.xbmc.org/changeset/31124

WannaTheater · 2010-08-12 02:24

Hi, I currently have 1000+ movies that I would like to generate NFOs for.

These are all in a database, end each movie contains its IMDB URL. I would like to use ScraperXML to accomplish this. I have experimented with ScraperXML_commandline, which works great for single movies. But with ScraperXML_commandline I need to perform the following:
1) First do a search to build the "Results" file
2) Then do a "details" to get the details. This details file I can use as NFO.

Here are my issues:
1) I don't need to do a search, as I know the movie URL (but I will if I have to)
2) I would like to call these from WITHIN some code (vs. command line) so I can loop through all movies, move generated nfos to where they belong, etc.

I have downloaded the ScraperXML .dll (TechNuts.dll?), but cannot find ANY documentation on how to use this library.

Can anyone provide assistance, or links to documentation?

Thanks!

WannaTheater · 2010-08-12 12:05

In doing some more experimenting, what I really need is the ScraperXML_Commandline source to use as an example.... is this available?

Thanks!

Nicezia Fan Posts: 369 Joined: Nov 2006 Reputation: 0 Location: Montgomery, Alabama	2010-01-08 22:54 Post: #311 thanks ScraperXML Open Source Web Scraper Library compatible with XBMC XML Scrapers I Suck, and if you act now by sending only $19.95 and a self addressed stamped envelop, so can you!
	find quote

Vincent81 Fan Posts: 355 Joined: Aug 2009 Reputation: 3 Location: France	2010-02-08 14:46 Post: #312 Hello, Would it be possible to use ScraperXML with command line or DLL in my program (XBNE : is not developed in C) ? And How to.. Thanks XBNE : XBMC Video DataBase / Nfo Editor Download - Forum - Donate
	find quote

patmtp35 Junior Member Posts: 1 Joined: May 2010 Reputation: 0	2010-05-10 21:07 Post: #315 hi! just discover your scraper commandline, it works great on imdb.com, but as i m french i would have to use it with ciné-passion, but i can't have it to works please could you help me ? regards
	find quote

olympia Team-XBMC Member Posts: 2,381 Joined: May 2008 Reputation: 30	2010-06-23 16:50 Post: #318 Fixchars is also worth to mention http://trac.xbmc.org/changeset/31124 ASUS P5N7A-VM - Intel C2D E7300 - 2GB RAM - Silverstone GD02 Black - MS MCE Remote - Patriot Warp 32GB SSD Drive
	find quote

WannaTheater Senior Member Posts: 121 Joined: Jun 2010 Reputation: 0	2010-08-12 12:05 Post: #320 In doing some more experimenting, what I really need is the ScraperXML_Commandline source to use as an example.... is this available? Thanks!
	find quote