MySQL insert script (CLI scraper)
#1
Hi there,

Before i went off to write my own solution I thought i would see if something like this existed already. Now that I have a central XBMC MySQL database on my server I want to have a CLI script that I can pass a filename argument (like a tv episode) have it download all the relevant data and insert in the DB (same as what the XBMC client does basically). This way no xbmc client would have to scan and retrieve a new eps data, they would just exist in the DB already.

Essentially I guess it would be a CLI scraper. Anyone aware of anything like this? Sorry if this is the wrong place to post this question.

EDIT: Thought I might do it as CPAN perl module and utilize MySQL and TVDB modules to access the db and reference thetvdb respectively.

Thanks
Reply
#2
Not that I know of, but let us know if you give this a shot, because a lot of people would be interested in it.
Reply
#3
Thanks for your reply mate.

Yeah definately going to do it if theres nothing else like this. I already have automatic downloading of my shows via an rss feed, then have written scripts to rename and sort new content into there appropriate directories. These scripts already pull all the show data from the TVDB via its API. Would use this existing logic to get all the ep metadata and create the relevant entries in the XBMC MySQL DB. Everything done on the server completely independent of any XBMC client. Nice and neat. Just fire up one of my XBMC machines on my network and new show content already there and ready to watch.
Reply
#4
(2012-06-13, 00:11)aTTilaz Wrote: Thanks for your reply mate.

Yeah definately going to do it if theres nothing else like this. I already have automatic downloading of my shows via an rss feed, then have written scripts to rename and sort new content into there appropriate directories. These scripts already pull all the show data from the TVDB via its API. Would use this existing logic to get all the ep metadata and create the relevant entries in the XBMC MySQL DB. Everything done on the server completely independent of any XBMC client. Nice and neat. Just fire up one of my XBMC machines on my network and new show content already there and ready to watch.

Care to share the script which does your downloading? I've been using Sickbead + SABnzbd but wouldn't mind trying something a little more lightweight.
Reply
#5
(2012-06-13, 00:15)notyjoey Wrote: Care to share the script which does your downloading? I've been using Sickbead + SABnzbd but wouldn't mind trying something a little more lightweight.

It is a torrent based solution so probably no good for you. If you are interested though, its a web frontend for Transmission i helped develop with a friend. Have a look at the following if you like:

http://sourceforge.net/projects/auto-for-trans/

Its monitors a user specified RSS feed for matches to a specified list (inclusions and exclusions), automatically downloads and sorts. Can automatically remove torrent/data at pre-determined days of seeding and days of inactivity. Sends user activty notifications via email.
Reply
#6
(2012-06-13, 00:46)aTTilaz Wrote:
(2012-06-13, 00:15)notyjoey Wrote: Care to share the script which does your downloading? I've been using Sickbead + SABnzbd but wouldn't mind trying something a little more lightweight.

It is a torrent based solution so probably no good for you. If you are interested though, its a web frontend for Transmission i helped develop with a friend. Have a look at the following if you like:

http://sourceforge.net/projects/auto-for-trans/

Its monitors a user specified RSS feed for matches to a specified list (inclusions and exclusions), automatically downloads and sorts. Can automatically remove torrent/data at pre-determined days of seeding and days of inactivity. Sends user activty notifications via email.

Hey now, it would save me using news groups AND I have transmission installed on my box too! I'll definitely check it out Wink
Reply
#7
With the latest code from head you now create a libxbmc.so shared library that can be used to create your own CLI (or server-based) XBMC applications... Smile

Here is a sample Video Library scanner application that uses libxbmc.so to update XBMCs Video Library...

http://pastebin.com/WpEvRTws

Enjoy!
Reply
#8
Is this possible in Windows? I can't find any XBMC.dll in the build directories.
Reply
#9
(2012-06-13, 11:13)jhsrennie Wrote: Is this possible in Windows? I can't find any XBMC.dll in the build directories.

I am not sure as I use Linux but the libxbmc.so is not built by default... You need to run ./configure --enable-shared-lib

Maybe give it a try on Windows and see what gets built?
Reply
#10
(2012-06-13, 10:43)vicbitter Wrote: With the latest code from head you now create a libxbmc.so shared library that can be used to create your own CLI (or server-based) XBMC applications... Smile

Here is a sample Video Library scanner application that uses libxbmc.so to update XBMCs Video Library...

http://pastebin.com/WpEvRTws

Enjoy!

Thanks vicbitter, is there any documentation that explains this process. Not to familar with C++
Reply
#11
(2012-06-14, 00:31)aTTilaz Wrote:
(2012-06-13, 10:43)vicbitter Wrote: With the latest code from head you now create a libxbmc.so shared library that can be used to create your own CLI (or server-based) XBMC applications... Smile

Here is a sample Video Library scanner application that uses libxbmc.so to update XBMCs Video Library...

http://pastebin.com/WpEvRTws

Enjoy!

Thanks vicbitter, is there any documentation that explains this process. Not to familar with C++

Unfortunately, it doesn't seem so at the moment, I had to trawl through the code and built the sample application based on my discoveries... I think the main code is generic enough to use and the only part(s) that would need changing is dependent on the internal APIs you want to use...

I hope that makes sense?

Reply
#12
I'd love to see a script that could also run on a NAS (QNAP, etc) or the "server" that is running the MySQL database.
This way the NAS (or server) could do the scraping for XBMC ... Smile
I'm by no means experienced enough to build my own scraper though Sad
XBMC (OpenElec) on AppleTV (1st gen), Intel Atom, AMD Fusion (with Boblight), Raspberry Pi, K1Plus Android x905 (LibreElec), nVidia Shield 2017.

www.tweaking4all.com
www.tweaking4all.nl
www.weethet.nl
Reply

Logout Mark Read Team Forum Stats Members Help
MySQL insert script (CLI scraper)0