2013-05-17, 14:10
Almost finished rework of media scanner and episode details 
little preview:
little preview:
2013-05-17, 21:42
Thanks so much. This is really awesome stuff here man. I don't know if this has been requested already but can this include a Search feature?
2013-05-18, 05:37
(2013-05-17, 21:42)ptran0606 Wrote: Thanks so much. This is really awesome stuff here man. I don't know if this has been requested already but can this include a Search feature?
Has already been requested, and is on my to do list.
2013-05-18, 11:31
@Sanger,
No problem... I read over your posts again and realized my scripts actually wouldn't really help in your situation anyway...
--
Once I finish moving a bunch of media off from a partially dead drive I'll start adding the new episode stuff into the two themes I did for the app... but one thing I'm curious about (and I know Dead is as well) is getting an idea of what web browsers the users have... In fact, I'm sure there will be a follow up post from Dead in reference to it...
No problem... I read over your posts again and realized my scripts actually wouldn't really help in your situation anyway...
--
Once I finish moving a bunch of media off from a partially dead drive I'll start adding the new episode stuff into the two themes I did for the app... but one thing I'm curious about (and I know Dead is as well) is getting an idea of what web browsers the users have... In fact, I'm sure there will be a follow up post from Dead in reference to it...
2013-05-18, 12:03
Well like he said, for better focusing on the project, it would be nice to know what browsers you people use.
So it would be helpful if you answer this little poll:
http://micropoll.com/t/KFAVoZQoYh
So it would be helpful if you answer this little poll:
http://micropoll.com/t/KFAVoZQoYh
2013-05-18, 14:32
Edit1: I officially gave up on adding any support to actually Stream the videos into the browser, as I can't find any solution for that.. it feels like its impossible.. VLC does not even support AAC audio and MP3 transcoding is buggy as hell.. not to mention JW Player refuses to accept any other format than crappy FLV1 with Mp3 (which is, like i said buggy)
But! I added a way to open the videos in XBMC by just clicking a link
6.3 Beta Release (Not final) (Final version will be called 7.0, due to long time between 6.2 and now)
Please report every single bug you find, this is only a beta release.
Also, please answer the survey about what browser you use: http://micropoll.com/t/KFAVoZQoYh
Download: XBMC_HTML_Generator_installer_63Beta.exe
A few IMPORTANT notes:
MySQL Support is like I said in change log very buggy... so report every bug you encounter .. please..(How to set pass/user is said in change log)
New database format requires deleting of old database (automatically done), which means you have to rescan trailers.
New Media scanner will take way longer (Read Change log to see why)
Change log:
+ Added Episode List view (Episode details)
* Features thumbnails and quality info for each episode (Includes Specials)
* Currently only supports remote thumbnails,i don't even know if there is a way to use local thumbnails for each episode.
+ Added MySQL Support (Very Beta..)
* To set your MySQL Settings (pass, etc) go to Extras->Mysql Settings, will also popup if you click scrape mysql (only when first using)
+ Added option to choose Output folder. (Program must have access to folder in order to write there.. if unsure, then run it as Admin)
+ Added time-neeeded display when scraping library and getting media data.
+ Added more debug options (See Extras->Debug->...)
? Changed Icons, new ones look more identical and fit the theme better in my opinion.
? Changed "Show file Path" now shows actual file:// links named "Open Folder" in Movie/TVShow and Episode view.
? Changed Media Scanner, now scans every video.. (every episode)
* will now take even longer, reported time by scanning 15000videos was arround 20minutes, depends on HDD Speed, and only has to be done once.
* New scanner methode is needed for episode details. Theres currently no way to get the old behavior back. (Its a beta afterall)
! Fixed AC3 Quality Icon not beeing scanned properly.
! Fixed Image quality not beeing saved.
! Fixed a couple crashes and other bugs I forgot about already..
But! I added a way to open the videos in XBMC by just clicking a link
6.3 Beta Release (Not final) (Final version will be called 7.0, due to long time between 6.2 and now)
Please report every single bug you find, this is only a beta release.
Also, please answer the survey about what browser you use: http://micropoll.com/t/KFAVoZQoYh
Download: XBMC_HTML_Generator_installer_63Beta.exe
A few IMPORTANT notes:
MySQL Support is like I said in change log very buggy... so report every bug you encounter .. please..(How to set pass/user is said in change log)
New database format requires deleting of old database (automatically done), which means you have to rescan trailers.
New Media scanner will take way longer (Read Change log to see why)
Change log:
+ Added Episode List view (Episode details)
* Features thumbnails and quality info for each episode (Includes Specials)
* Currently only supports remote thumbnails,i don't even know if there is a way to use local thumbnails for each episode.
+ Added MySQL Support (Very Beta..)
* To set your MySQL Settings (pass, etc) go to Extras->Mysql Settings, will also popup if you click scrape mysql (only when first using)
+ Added option to choose Output folder. (Program must have access to folder in order to write there.. if unsure, then run it as Admin)
+ Added time-neeeded display when scraping library and getting media data.
+ Added more debug options (See Extras->Debug->...)
? Changed Icons, new ones look more identical and fit the theme better in my opinion.
? Changed "Show file Path" now shows actual file:// links named "Open Folder" in Movie/TVShow and Episode view.
? Changed Media Scanner, now scans every video.. (every episode)
* will now take even longer, reported time by scanning 15000videos was arround 20minutes, depends on HDD Speed, and only has to be done once.
* New scanner methode is needed for episode details. Theres currently no way to get the old behavior back. (Its a beta afterall)
! Fixed AC3 Quality Icon not beeing scanned properly.
! Fixed Image quality not beeing saved.
! Fixed a couple crashes and other bugs I forgot about already
..
2013-05-20, 10:17
I believe the episode thumbnails are stored on the local filesystem in the XBMC userdata folder - it makes use of Textures13.db to keep track of which files have been downloaded to the cache, but the URLs are stored in the video database. Assuming a default location of %appdata%\xbmc\ for these files, it could pull these in. It's just if someone is using path substitution for their thumbnails where you might need to look in the advancedsettings.xml file and make the mappings as necessary.
2013-05-20, 14:27
Update on future .. Updates:
Results of the browser poll: Chrome and Firefox won the race (big suprise.... :p)
Anyway, that means I will not support Opera/Safari and all the other stuff anymore, means.. if a specific javascript function or css or something does not work on there, then you have to life with it or fix it yourself.
To the NowPlaying/Stop/Play stuff, in the next update I will only make it work for people with PHP.
Beside that I noticed that no one really complained about the MySQL support.. so I guess it works fine?
Oh and this is the now playing/start/stop thing I mean:
(Has IP whitelist, to prevent outsiders to control your XBMC.. can also be limited to LAN this way.)
Thats not what I meant,I also decided to not use textures DB, for more combability.
What I meant was, that I don't know if its possible to manually define thumbnails, like you can do with posters (e.g fileName-poster.jpg, while I never tested: episodeFile-thumb.jpg)
Also, creating a PHP Addon at the moment, for people that have their html-library on a php powered webserver. This script will allow you to start/pause/stop and see now playing of shows/movies.
I am going to also create a alternative javascript only way to pause/stop/play videos without a PHP server BUT that wont support "now playing" info. (Due to security limitations of Browsers and cross domain ajax, as long as XBMC doesnt add JSONP (note the P) it will not be possible to get now playing without PHP from my html files)
Results of the browser poll: Chrome and Firefox won the race (big suprise.... :p)
Anyway, that means I will not support Opera/Safari and all the other stuff anymore, means.. if a specific javascript function or css or something does not work on there, then you have to life with it or fix it yourself.
To the NowPlaying/Stop/Play stuff, in the next update I will only make it work for people with PHP.
Beside that I noticed that no one really complained about the MySQL support.. so I guess it works fine?
Oh and this is the now playing/start/stop thing I mean:
(Has IP whitelist, to prevent outsiders to control your XBMC.. can also be limited to LAN this way.)
(2013-05-20, 10:17)ConfusedTA Wrote: I believe the episode thumbnails are stored on the local filesystem in the XBMC userdata folder - it makes use of Textures13.db to keep track of which files have been downloaded to the cache, but the URLs are stored in the video database. Assuming a default location of %appdata%\xbmc\ for these files, it could pull these in. It's just if someone is using path substitution for their thumbnails where you might need to look in the advancedsettings.xml file and make the mappings as necessary.
Thats not what I meant,I also decided to not use textures DB, for more combability.
What I meant was, that I don't know if its possible to manually define thumbnails, like you can do with posters (e.g fileName-poster.jpg, while I never tested: episodeFile-thumb.jpg)
Also, creating a PHP Addon at the moment, for people that have their html-library on a php powered webserver. This script will allow you to start/pause/stop and see now playing of shows/movies.
I am going to also create a alternative javascript only way to pause/stop/play videos without a PHP server BUT that wont support "now playing" info. (Due to security limitations of Browsers and cross domain ajax, as long as XBMC doesnt add JSONP (note the P) it will not be possible to get now playing without PHP from my html files)
2013-05-24, 07:00
I just registered to the forums to say thank you DeadSix27 for creating and sharing this program. I had been looking around for the past few months for something that would do exactly what your program does, and found this post by accident, and just as you added MYSQL support.
MYSQL scraping has worked fine for me, the only issue I've come across whilst scraping my library was the majority of movie covers didn't scrape in English, they mostly scraped in either French, German or Russian, whilst the TV covers scraped fine.
That said, I didn't use the latest build, I used one you posted a couple of pages back when you first implemented MYSQL. I will grab the new beta when I get home from work and test it out.
Thank you again for all the hard work you've put in.
MYSQL scraping has worked fine for me, the only issue I've come across whilst scraping my library was the majority of movie covers didn't scrape in English, they mostly scraped in either French, German or Russian, whilst the TV covers scraped fine.
That said, I didn't use the latest build, I used one you posted a couple of pages back when you first implemented MYSQL. I will grab the new beta when I get home from work and test it out.
Thank you again for all the hard work you've put in.
2013-05-24, 16:09
bbbd,
If you don't use local images, the language 'issue' with remote images technically isn't a problem with this app, it's the sites that it grabs the images from. Since all of these {image} sites are open source (TheMovieDB, TheTVDB, etc.) if a user uploads a bunch of pictures and either forgets or doesn't care about marking a language for the image, it'll be pulled in the normal scrape as if it's English. I had this problem on a group of my stuff (back before local images worked properly) but since I've pretty much always used locals it's not really an issue.
If you don't use them and are interested in doing it that way, feel free to PM me and I can help you out with the naming you need for things if you're unsure... warning, if you haven't been using local images and you have a large library, even with the help of Ember, MC, etc. this can take a lot of time... especially if you care about the actual images used.
Anyway, side note.... I had 3TB's (1x2TB, 1x1TB) worth of hard drives die on me last weekend so I've been dealing with recovery all week... plus I had more doctor crap. Sorry for any missed posts/pm's.
If you don't use local images, the language 'issue' with remote images technically isn't a problem with this app, it's the sites that it grabs the images from. Since all of these {image} sites are open source (TheMovieDB, TheTVDB, etc.) if a user uploads a bunch of pictures and either forgets or doesn't care about marking a language for the image, it'll be pulled in the normal scrape as if it's English. I had this problem on a group of my stuff (back before local images worked properly) but since I've pretty much always used locals it's not really an issue.
If you don't use them and are interested in doing it that way, feel free to PM me and I can help you out with the naming you need for things if you're unsure... warning, if you haven't been using local images and you have a large library, even with the help of Ember, MC, etc. this can take a lot of time... especially if you care about the actual images used.
Anyway, side note.... I had 3TB's (1x2TB, 1x1TB) worth of hard drives die on me last weekend so I've been dealing with recovery all week... plus I had more doctor crap. Sorry for any missed posts/pm's.
2013-05-24, 16:41
(2013-05-24, 07:00)bbbd Wrote: I just registered to the forums to say thank you DeadSix27 for creating and sharing this program. I had been looking around for the past few months for something that would do exactly what your program does, and found this post by accident, and just as you added MYSQL support.I have not yet noticed any language tag within the Remote image data, ill look more into it.. maybe there is one. (Usually im just pulling the first image in the list)
MYSQL scraping has worked fine for me, the only issue I've come across whilst scraping my library was the majority of movie covers didn't scrape in English, they mostly scraped in either French, German or Russian, whilst the TV covers scraped fine.
That said, I didn't use the latest build, I used one you posted a couple of pages back when you first implemented MYSQL. I will grab the new beta when I get home from work and test it out.
Thank you again for all the hard work you've put in.
Also be sure to set the language right in movie scraper:
Here is a example of remote poster data for the show "Doctor Who (2005)",
as you can see, there is no language tag in it.. so that's why I just take the first.
Code:
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-23.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-34.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-13.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-29.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-27.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-18.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-20.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-30.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-22.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-16.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-17.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-26.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-6.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-35.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-8.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-10.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-24.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-39.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-1.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-2.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-3.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-11.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-9.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-21.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-37.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-36.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-33.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-25.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-14.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-4.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-12.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-15.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-7.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-28.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-19.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-32.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-31.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-38.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-40.jpg</thumb>
<thumb aspect="poster">http://thetvdb.com/banners/posters/78804-5.jpg</thumb>
2013-05-25, 06:48
Thanks for the heads up re: the images, it wasn't a big deal for me, I just went through and changed the ones that didn't scrape in English, long task but I thought I'd report it in case there was something that could be looked at. My language settings in the XBMC scraper settings were already set to EN so I guess it was just the sites like you pointed out.
I had been using local shared images when I was running Eden, but stopped with Frodo, looks like I might go re-enable that.
Now the next problem, I downloaded the latest 6.3 Beta you have linked, and tried a fresh install, and have run into a problem with generating the HTML pages. I scrape the MYSQL fine, no errors in the logs, I click on "Generate HTML", I get a pop up saying "HTML Generator successfully created your page. You can now Download Poster and or Fanart's. [Movies Processed: 0 TV Shows Processed: 297], [Movies Saved in HTML: 0 TV Shows Saved in HTML: 297]"
There are no HTML files generated apart from a single 0kb .htm file in the "generated/episodedetails" folder.
If I revert back to build 6.2 rev 32404, the HTML files generate fine, and I get [Movies Processed: 1215 TV Shows Processed: 297].
I had been using local shared images when I was running Eden, but stopped with Frodo, looks like I might go re-enable that.
Now the next problem, I downloaded the latest 6.3 Beta you have linked, and tried a fresh install, and have run into a problem with generating the HTML pages. I scrape the MYSQL fine, no errors in the logs, I click on "Generate HTML", I get a pop up saying "HTML Generator successfully created your page. You can now Download Poster and or Fanart's. [Movies Processed: 0 TV Shows Processed: 297], [Movies Saved in HTML: 0 TV Shows Saved in HTML: 297]"
There are no HTML files generated apart from a single 0kb .htm file in the "generated/episodedetails" folder.
If I revert back to build 6.2 rev 32404, the HTML files generate fine, and I get [Movies Processed: 1215 TV Shows Processed: 297].
2013-05-25, 15:52
I'm going to set up Nightly builds soon. This allows you guys to stay up to date with the builds without me having to release a beta every time..
2013-05-25, 19:28
Thought about trying this out. Looks like a great idea. I dont know how to use it with my setup tho.
I have an Pivos XIOS with XBMC running on it. This HTML Generator app is installed on my PC. I thought I could just browse to my movie directory which is on my NAS and then the program would go from there. However it wants the XBMC database file. This file is not located on my PC, its on the XIOS.
Is there any other way this can be set up? Maybe I dont see the right option..
I have an Pivos XIOS with XBMC running on it. This HTML Generator app is installed on my PC. I thought I could just browse to my movie directory which is on my NAS and then the program would go from there. However it wants the XBMC database file. This file is not located on my PC, its on the XIOS.
Is there any other way this can be set up? Maybe I dont see the right option..
