How do you make a site scrapeable?

Excellent! I just downloaded the 0.3, going to have a look if this fits my purpose good enough for it to be something to build on/learn from!

Checked the tvdb framework, after my initial reaction of "what the hell" had settled, I started snooping around. But unfortunately as of now, I am nowhere near grasping the structural layout and inner workings that is thetvdb 0.3.

But I have been working on a prototype that might just do what's necessary anyway. I do however have a question (as I know next to nothing about scraping xml)

If say, an xml data container, contains the following php


does the scraper only read the data (as is) or is it possible to get the actual result of the echo instead, in this case the last modified date?

php is server side..

uuuh.. yes. I know that much. Doesn't really answer my question though, or I'm too dumb to realize that it does. But I am going to assume that the answer is no, and try to find an alternative solution. Thank you for the reply though.

asphinx Wrote:Checked the tvdb framework, after my initial reaction of "what the hell" had settled, I started snooping around. But unfortunately as of now, I am nowhere near grasping the structural layout and inner workings that is thetvdb 0.3.

But I have been working on a prototype that might just do what's necessary anyway. I do however have a question (as I know next to nothing about scraping xml)

If say, an xml data container, contains the following php

Code:

<?php echo date (”Y-m-j h:i:s A (T)”, getlastmod()); ?>

does the scraper only read the data (as is) or is it possible to get the actual result of the echo instead, in this case the last modified date?

0.3 is REALLY old. Sorry about that. We're in the middle of completely overhauling our database and moving it to Postgres (most likely) instead of MySQL, which will make most of the SQL exist only in stored procedures on the database. It should really simplify things.

You missed the point that spiff was making. That PHP code you see will get turned into a date on the server and returned. Client side users and applications won't see any of the PHP code itself.

I'd be happy to give you a bit of guidance for your project... just send me a PM. The main thing is that you'll really need a grasp of PHP and SQL (mysql or Postgres). Also keep in mind that once XBMC and Media Portal hit your site, you're looking at a TON of bandwidth and CPU usage so you'll really need a dedicated host when that time comes (instead of a shared host, which is what many website hosting companies provide).

Ooh, now I get it. Server side being that no, the user only sees the date, not the php code that executes it. I discussed this with a friend as well (who knows php & sql, unlike me). He was also keen to emphasize the need for a dedicated host if I ever got this up n' running.

As of right now I am however finding myself lacking in time/resources to fully commit to this, which is a shame because I (personally) still think there would be a good need for a game database that xbmc could scrape (which is why I asked how to make something scrape able with ease)

I might still take you up that offer sometime in the future though.