ScraperXML (Open Source XML Web Scraper C# Library) please help verify my work...

got that, you explaining why i didn't split it as much as one could do

heck honestly, my imdb include file was basically the whole get details (with added conditionals) and custom functions... and my actual imdb was a skeleton with the imdb include call. =)

well i do have a question, it seems that TMDB works in XBMC... but everytime i update it seems to have this one error (in GetSearchResults - output="<results>\1</result>")

how does XBMC seem to process this even though its malformed?

Just finished testing the include files and my code works flawlessly with them. I'm a little dissapointed that i got it right before it ever hit svn! I thought i'd have to do some error tracking but no such luck!

Nicezia Wrote:2) the reason this happens in your code is because the ScrapeResultsEntity gets passed as a refrence, therefore any modifications made to the ScrapeResultsEntity, change the item passed to it, - its probably a good idea to do a clone when sending a cached result that way your cached results stays in tact, i do it this way to keep the amount of memory used by scraperxml down to the absolute minimum, it uses the scraperesults entity info directly - modifying it as the process continues (for each custom function).

I understand the mechanics, I was asking about the API design. I never would have expected that call to modify the ScrapeResultsEntity that I passed in, that's really counter-intuitive and I can't think of anything in the .Net framework that has a similar side-effect without explicitly passing the parameter by reference. If you keep the existing behavior, I'd strongly recommend adding the 'ref' keyword to the ScrapeResultsEntity parameter on those Get*Details methods.

Ideally, though, you should just copy the ScrapeResultsEntity inside your library and operate on the copy. The memory savings you're getting by not doing that is inconsequential.

first of all the item used inside the function IS a copy of the item passed to it, (unless specified by the ref keyword - there's no way to modify the actual item passed as i'm not using any pointers at all and all code is purely .NET if you don't have to specify ref when passing the item then its not going to modify the origional item, - unless the scraperxml code has been modified in someway on your part) and shouldnt be modifying your cached copy. secondly there's absolutely nothing inside the code that should make it copy anything to your cached copy of the item, the ref'd item is only internal as it has to be passed to another internal function and from there another function - i don't really see how this could be causing problems in your code, and i can't tell anything having not seen the code you're using to pass it - granted i'm only an amateur program (very amateur) but i unless you've made some modifications to the scraperxml code that i don't know about it shouldn't really be doing what you say its doing

-message me on yahoo(niceziavincent) or gmail(niceziavincent) or icq (109693377)

Nicezia Wrote:first of all the item used inside the function IS a copy of the item passed to it, (unless specified by the ref keyword - there's no way to modify the actual item passed as i'm not using any pointers at all and all code is purely .NET if you don't have to specify ref when passing the item then its not going to modify the origional item, - unless the scraperxml code has been modified in someway on your part) and shouldnt be modifying your cached copy. secondly there's absolutely nothing inside the code that should make it copy anything to your cached copy of the item, the ref'd item is only internal as it has to be passed to another internal function - i don't really see how this could be causing problems in your code, and i can't tell anything having not seen the code you're using to pass it - granted i'm only an amateur program (very amateur) but i unless you've made some modifications to the scraperxml code that i don't know about it shouldn't really be doing what you say its doing

ScraperResultsItem is a reference type, not a value type. So if you pass it to a function and then modify one of its fields, the field remains modified when you return from the function. Check http://msdn.microsoft.com/en-us/library/...s_example4 for a good example of this.

You're right, though, in that adding the ref keyword won't really address this problem. The only way to fix it is to explicitly copy the object instead of modifying the one that's passed in.

You should be able to reproduce this fairly easily. Get a ScrapeResultsItem using IMDB, make a call into GetMovieDetails, then examine the contents of the item you passed in. It will be different.

smeehrrr Wrote:ScraperResultsItem is a reference type, not a value type. So if you pass it to a function and then modify one of its fields, the field remains modified when you return from the function. Check http://msdn.microsoft.com/en-us/library/...s_example4 for a good example of this.

You're right, though, in that adding the ref keyword won't really address this problem. The only way to fix it is to explicitly copy the object instead of modifying the one that's passed in.

You should be able to reproduce this fairly easily. Get a ScrapeResultsItem using IMDB, make a call into GetMovieDetails, then examine the contents of the item you passed in. It will be different.

what i'm saying is its literally impossible for that to happen in my code, the only thing that comes out of the Get*Details function is the *Tag object, and the item passed in is copied as per general coding rules, the only thing that should have made any kind of change is the copy of the item inside the function, it doesn't make any changes to the object that are external to the function. (General .NET protocol on sending an item to a function - and for that matter every programming language i know, states that an item sent to a function is a copy of that item inside that function unless otherwise specified), how in gods name its managing to change something outside of the function is a mystery, and i can't reproduce this behaviour on my end (yes the item inside the function is modified, but there's no reason at al for that modified item to go outside the function... as there is no multiple returns ONLY the tag item.) i've tried everything to make this happen, and the only way it could posibly happen is if the ref keyword is used on the public function initially used by the program which its not... so you got me Phil...

maybe you should try the new code in svn.... (in its origional form without modification - since through all testing it seems to work as its supposed to)


ps... also i can be reached as nicezia on freenode (IRC)

[quote=Nicezia]what i'm saying is its literally impossible for that to happen in my code, the only thing that comes out of the Get*Details function is the *Tag object, and the item passed in is copied as per general coding rules, the only thing that should have made any kind of change is the copy of the item inside the function, it doesn't make any changes to the object that are external to the function. (General .NET protocol on sending an item to a function - and for that matter every programming language i know, states that an item sent to a function is a copy of that item inside that function unless otherwise specified),[quote]
Take a look at the link I posted. What you're saying is true for value types in C#, but not true for reference types. For reference types (which ScraperResultsItem is, or which an array would be), modifying the information inside the object or array is reflected in the original object.

If you look at examples 4 and 5 at that link you'll see the effect that the 'ref' keyword has when passing reference types. It's not at all intuitive.

Have you tried running the steps I suggested in a debugger? I suspect your test code never tries to reuse a ScraperResultsItem, so you obviously won't see this in your testing unless you look for it.

yes i did and as i said, there's no reason for that changed item to go outside the function (i use the scrape results item urls for downloading the custom functions so when it finds custom function calls (which there are several in IMDB) those custom function callse get set as the URLs and the parser iterates through them as neccessary, yes again the item gets changed internally, but that item is/should be a copy of the origional item not the actual item itself and what's more that item should not be passed outside the function, and in my tests (doing even as you suggested) the item gets changed inside the function but the item passed to it stays intact.

For those playing the home game, Nicezia and I hashed this out last night via IM, and the next version shouldn't have this behavior.