+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) (/showthread.php?tid=98210)
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-15
(2012-08-15, 19:05)spyder Wrote: I had another thought for this, would there be a way to scrape favorites, and/or Naxi-X favorites? I took a look at the Navi-X favorites file which is a plx file, would this file type work?
I have never used navi-x so that isn't something I will be pursuing.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - spanktastic2120 - 2012-08-15
(2012-08-15, 19:30)angrycamel Wrote:(2012-08-15, 19:05)spyder Wrote: I had another thought for this, would there be a way to scrape favorites, and/or Naxi-X favorites? I took a look at the Navi-X favorites file which is a plx file, would this file type work?
I have never used navi-x so that isn't something I will be pursuing.
you should definitely check out navi-x, it has a lot of really cool stuff including live tv channels and hd tv shows and movies available for streaming.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - scoombe - 2012-08-15
(2012-08-15, 18:39)angrycamel Wrote: Nice work, thanks for applying the patches and for the bug fixes.
IDEA
I want a way to get non-tv content to be supported so that I can really just create the directory full of stream files. I am using this to assist in merging streamed content with local content in PseudoTV. With that goal I also want to pull in TED streams but since they are not true tv shows, it skips them.
I'm considering adding a new type of misc that would skip the tvdb lookup and simply create the stream files, name them according to what the plugin showed on the response, then save them in a configurable directory. By doing this I should be able to setup my TED channel in PTV as a directory channel to point at that configurable directory the stream files were written to.
Would anyone else get any benefit from this kind of a feature?
I very well could be wrong about this, but wouldn't the manual archiving feature in the config file be able to do this? Particularly now that your able to filter by run time, that feature will make .nfo files for anything that didn't register a hit on tvdb. It would be cumbersome to edit the .nfo files, but its a start. I haven't ever used that particular feature, but now that anything less than 20 mins gets skipped, it could become much more useful.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-15
I wasn't aware of that feature. Despite contributing to it, I'm not 100% familiar with all of the features of the app yet. That's actually part of why I asked. I would hate to embark on implementing something that is at the least partially available already. Thanks for the info and I will take a look at the config file for more info. I'll give it a try and see of it will work for my TED channel use case.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - bradvido88 - 2012-08-15
The manual archiving will create .nfo files based on the best guess of the series name and episode season number and title.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-15
It looks like it is based on a time delay. In the case of TED videos I know ahead of time that it will not find anything about an episode or season or show. Is there a way to configure a search filter to perform that manual archiving without waiting for the time delay? I think that is what I'm looking for.
EDIT:
If it isn't possible yet, I'm quite happy to add that functionality to meet my needs but as I said, just don't want to duplicate efforts.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - scoombe - 2012-08-16
I think something is broken in the latest update. I dont get any returns now. The log says unable to find source, and skips through every entry. Something must have gotten borked in the JSON interactions. I will post my log if you want, but it doesnt say much. It goes through the initial procedures, succesfully shuts down and restarts xbmc, then skips every entry in the config file after it says cannot find source, and finishes within 15 seconds of starting. I've tried using a config file I knew worked yesterday, no luck. I tried using the binary only with the file structure from 1.3.1 to see if it was something in the /lib directory, no luck. I did use the 1.3.1 and it worked, although with the known Netflix problems.
Anyone else tested and see this behavior?
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - scoombe - 2012-08-16
Ok, ran a session with a full debug log. Here it is: http://pastebin.com/ckfV6F3y
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-16
In updating my local copy to the head revision I can see that the latest does not have all of my changes in it, so that's likely what you're running into. I am merging and correcting the latest and I will give Brad a patch that applies to his current head revision. I think some things were just lost in the complexities of two sets of fixes from him as well as two patches from me.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-16
I did some more work on the tool tonight. I fixed the problems with the recently applied patches and I also added a couple of new features for TV Show types.
New Feature #1
The first feature addresses problem with the series and title parsing using the default parser. This was reported by ThermoDust originally, and brought up again recently here. The way that I thought would be best to address it is by giving the user the ability to specify a regex pattern in the subfolder configuration that would be used to parse the series and title. It expects the pattern to specify 2 groups (denoted with rounded brackets: read more here). The first group is expected to be the series name and the second group is expected to be the title of the episode. A couple of resources for building regular expressions are: here and here.
It is specified in the configuration from within the subfolder element like so:
Code:
<subfolder name="My Show" type="episodes" >
<!--Parse the series name then the title of the episode -->
<parser>
<regexp>([\w\s*'-]*):([\w\s*'-]*)</regexp> <!-- ex: "Show Name: Title of the Episode" -->
</parser>
</subfolder>Example debug output:
Code:
08/16/2012 12:47:19 AM DEBUG Archiving Attempting default parsing of TV Show...
08/16/2012 12:47:19 AM DEBUG Archiving Found custom series and title parser.
08/16/2012 12:47:19 AM DEBUG Archiving Custom series and title parser regex: ([\w\s*'-]*):([\w\s*'-]*)
08/16/2012 12:47:19 AM DEBUG Archiving Found series "Stephen Ritz: A teacher growing green in the South Bronx", and title "Stephen Ritz", from file label "Stephen Ritz: A teacher growing green in the South Bronx" using a custom parser. Will use this info to look up on the TVDB.comNew Feature #2
The second feature allows a user to completely override any parsed series name, even if it is one that is parsed with the custom parser above. It was injected at the start of the TVDB lookup method, so if that is not called, it will not be applied however. Items that have a proper sXXeXX format found in them will be parsed without the override, even if it is found in the config.
I developed this feature specifically for my TED need described in previous posts in this thread. The suggestion to use the ManualArchiving worked "OK" when configured to a HoursThreshold of "0.001" but it ended up putting everything into separate folders. I didn't want it trying to figure out a series from the titles of the TED talks and preferred instead that they all just write into a single directory called "TED". So this is how I accomplished that.
It is specified in the configuration from within the subfolder element like so:
Code:
<!-- Newest TED Talks -->
<subfolder name="Newest Talks" type="episodes" suffix=" (TED)" >
<!--Override any parsed series name with the following constant -->
<ForceSeries>TED</ForceSeries>
</subfolder>You can download the binary from here
The patch is available here: http://code.google.com/p/xbmc-mylibrary/issues/detail?id=3
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - spanktastic2120 - 2012-08-16
(2012-08-16, 07:17)angrycamel Wrote: I did some more work on the tool tonight. I fixed the problems with the recently applied patches and I also added a couple of new features for TV Show types.
New Feature #1
The first feature addresses problem with the series and title parsing using the default parser. This was reported by ThermoDust originally, and brought up again recently here. The way that I thought would be best to address it is by giving the user the ability to specify a regex pattern in the subfolder configuration that would be used to parse the series and title. It expects the pattern to specify 2 groups (denoted with rounded brackets: read more here). The first group is expected to be the series name and the second group is expected to be the title of the episode. A couple of resources for building regular expressions are: here and here.
It is specified in the configuration from within the subfolder element like so:
Code:<subfolder name="My Show" type="episodes" >
<!--Parse the series name then the title of the episode -->
<parser>
<regexp>([\w\s*'-]*):([\w\s*'-]*)</regexp> <!-- ex: "Show Name: Title of the Episode" -->
</parser>
</subfolder>
Example debug output:
Code:08/16/2012 12:47:19 AM DEBUG Archiving Attempting default parsing of TV Show...
08/16/2012 12:47:19 AM DEBUG Archiving Found custom series and title parser.
08/16/2012 12:47:19 AM DEBUG Archiving Custom series and title parser regex: ([\w\s*'-]*):([\w\s*'-]*)
08/16/2012 12:47:19 AM DEBUG Archiving Found series "Stephen Ritz: A teacher growing green in the South Bronx", and title "Stephen Ritz", from file label "Stephen Ritz: A teacher growing green in the South Bronx" using a custom parser. Will use this info to look up on the TVDB.com
New Feature #2
The second feature allows a user to completely override any parsed series name, even if it is one that is parsed with the custom parser above. It was injected at the start of the TVDB lookup method, so if that is not called, it will not be applied however. Items that have a proper sXXeXX format found in them will be parsed without the override, even if it is found in the config.
I developed this feature specifically for my TED need described in previous posts in this thread. The suggestion to use the ManualArchiving worked "OK" when configured to a HoursThreshold of "0.001" but it ended up putting everything into separate folders. I didn't want it trying to figure out a series from the titles of the TED talks and preferred instead that they all just write into a single directory called "TED". So this is how I accomplished that.
It is specified in the configuration from within the subfolder element like so:
Code:<!-- Newest TED Talks -->
<subfolder name="Newest Talks" type="episodes" suffix=" (TED)" >
<!--Override any parsed series name with the following constant -->
<ForceSeries>TED</ForceSeries>
</subfolder>
You can download the binary from here
The patch is available here: http://code.google.com/p/xbmc-mylibrary/issues/detail?id=3
Youre doing some really cool work on this project and adding a lot of great features but i dont understand why you made this last one ignore the user specified series if the episodes are in the proper format. If im reading this correctly then isnt it entirely possible to encounter a show that gets assigned the wrong name because of something like a colon in the title, but because all the episodes for it have proper season/episode numbering the user cannot correct the series name? It just seems strange to me to add a user override than can be overridden.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - EZ1976 - 2012-08-16
hi
well i tried to edit the config file and have it use one of israeli's vod websites.
i have this in my config file:
<hotVOD path="plugin://plugin.video.hotVOD.video" recursive="true" >
<subfolder name="8 ץורע" type="movies" >
and i get this :
C:\1\XBMC.MyLibrary-1.3.1\res\scraper.db
08/16/2012 12:42:02 PM ERROR Init... Could not find valid xml document at: C:\1\XBMC.MyLibrary-1.3.1\Config.xml. Cannot continue... Please check config.xml with an XML validator.
08/16/2012 12:42:02 PM ERROR Init... Failed while loading Configuration and testing connections... cannot continue. Please check your settings in Config.xml
08/16/2012 12:42:02 PM NOTICE Summary:---Overa ----------------------Archiving
Summary for ---Overall-------------------------
08/16/2012 12:42:02 PM NOTICE Summary:---Overa TV Success: 0, TV Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Movie Success: 0, Movie Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Music Video Success: 0, Music Video Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa New videos archived: 0, existing videos updated: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Overall: Success: 0, Skip: 0, Fail: 0
08/16/2012 12:42:02 PM NOTICE Ending Done... Total processing time: 0 minute(s), 0 second(s)
any idea?
can it support hebrew titles?
our VOD 's episode titles are in hebrew and i suspect that is why it is giving me errors.
what should i do to support hebrew languages?
thanks
config.xml file - http://pastebin.com/svYKx71e
no log available (didnt create one)
so the main idea is does this support non-English characters in the stream titles
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-16
(2012-08-16, 09:43)spanktastic2120 Wrote: Youre doing some really cool work on this project and adding a lot of great features but i dont understand why you made this last one ignore the user specified series if the episodes are in the proper format. If im reading this correctly then isnt it entirely possible to encounter a show that gets assigned the wrong name because of something like a colon in the title, but because all the episodes for it have proper season/episode numbering the user cannot correct the series name? It just seems strange to me to add a user override than can be overridden.
I think you're right. I'll take another look when I get a chance. (The next couple days are busy for me). The changes needed are in public static boolean addTVMetaDataFromSxxExx(XBMCFile video, String seasonEpisodeNaming) on line 868 of Archiver.java
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - spanktastic2120 - 2012-08-16
(2012-08-16, 11:46)EZ1976 Wrote: hi
well i tried to edit the config file and have it use one of israeli's vod websites.
i have this in my config file:
<hotVOD path="plugin://plugin.video.hotVOD.video" recursive="true" >
<subfolder name="8 ץורע" type="movies" >
and i get this :
C:\1\XBMC.MyLibrary-1.3.1\res\scraper.db
08/16/2012 12:42:02 PM ERROR Init... Could not find valid xml document at: C:\1\XBMC.MyLibrary-1.3.1\Config.xml. Cannot continue... Please check config.xml with an XML validator.
08/16/2012 12:42:02 PM ERROR Init... Failed while loading Configuration and testing connections... cannot continue. Please check your settings in Config.xml
08/16/2012 12:42:02 PM NOTICE Summary:---Overa ----------------------Archiving
Summary for ---Overall-------------------------
08/16/2012 12:42:02 PM NOTICE Summary:---Overa TV Success: 0, TV Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Movie Success: 0, Movie Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Music Video Success: 0, Music Video Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa New videos archived: 0, existing videos updated: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Overall: Success: 0, Skip: 0, Fail: 0
08/16/2012 12:42:02 PM NOTICE Ending Done... Total processing time: 0 minute(s), 0 second(s)
any idea?
can it support hebrew titles?
our VOD 's episode titles are in hebrew and i suspect that is why it is giving me errors.
what should i do to support hebrew languages?
thanks
config.xml file - http://pastebin.com/svYKx71e
no log available (didnt create one)
so the main idea is does this support non-English characters in the stream titles
i dont know if it can support non-english characters or not but youre getting that error because your xml file has something wrong in it. most likely you opened a tag without closing it or closed a tag you hadnt opened. I dont see any non-english characters in your xml file, so if it was going to error because of that it would be after loading the config. Youll need to meticulously skim through your config file and make sure everything is in the proper syntax or try some xml verifiers online, though ive had a lot of trouble with those giving bogus errors. Good luck.
RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - spanktastic2120 - 2012-08-17
Ive noticed a large number of shows on Free Cable have sXXeXX numbering but on their host websites the episode numbers are absolute. For example the show Adventure Time has all the episodes named with the correct season number but then the absolute episode number (s04e44). Boston Legal is done in a similar way but using s00eXX where XX is the absolute number for the episode. They are all named this way on the websites so they are getting parsed correctly by Free Cable and MyLibrary they just dont fit with the look-up conventions. The only solution to this i know is to manually change the episode titles to fit the format but they would get replaced every time MyLibrary is run (or the streams would get dropped from the host site and never updated), so does anyone have any ideas or suggestions to correct this?
There are probably about 50 or 60 shows across the entire Free Cable add-on that ive found this with (i parse everything i can from it to populate pseudotv).