• 1
  • 30
  • 31
  • 32(current)
  • 33
  • 34
  • 100
XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins)
Thanks for working on this. I'm trying the last patch that included the ForceSeries and parser tags (post #460) and it is not working. It gets the list of shows, but each one is skipped because it doesn't match any filters. The only filter is runtime. If I use the previous patch that added the runtime tag, it works (but has the colon problem). Here's my search filter and log:

Code:
            <subfolder name="Biography/Full Biographies" type="episodes" >
                <!--Parse the series name then the title of the episode -->
                <filter>
                    <runtime>GT|1200</runtime>
                </filter>
            </subfolder>

Debug log

I get the same result if I add a ForceSeries or parser tag, too.

Thanks
Reply
(2012-08-17, 04:23)jcato Wrote: Thanks for working on this. I'm trying the last patch that included the ForceSeries and parser tags (post #460) and it is not working. It gets the list of shows, but each one is skipped because it doesn't match any filters. The only filter is runtime. If I use the previous patch that added the runtime tag, it works (but has the colon problem). Here's my search filter and log:

Code:
            <subfolder name="Biography/Full Biographies" type="episodes" >
                <!--Parse the series name then the title of the episode -->
                <filter>
                    <runtime>GT|1200</runtime>
                </filter>
            </subfolder>

Debug log

I get the same result if I add a ForceSeries or parser tag, too.

Thanks

most likely the version youre using hasnt had every patch added to it, but as a quick fix you can just get rid of that filter, that directory only contains full bios anyway, no clips or anything.
Reply
I think there are still problems from the multiple patches applied at once by Brad. I'll take a look tomorrow evening.
Integrating with XBMC through the JSON-RPC interface?

Try the JSON-RPC Browser

Image
Reply
(2012-08-16, 23:38)spanktastic2120 Wrote:
(2012-08-16, 11:46)EZ1976 Wrote: hi
well i tried to edit the config file and have it use one of israeli's vod websites.

i have this in my config file:

<hotVOD path="plugin://plugin.video.hotVOD.video" recursive="true" >
<subfolder name="8 ץורע" type="movies" >


and i get this :

C:\1\XBMC.MyLibrary-1.3.1\res\scraper.db
08/16/2012 12:42:02 PM ERROR Init... Could not find valid xml document at: C:\1\XBMC.MyLibrary-1.3.1\Config.xml. Cannot continue... Please check config.xml with an XML validator.
08/16/2012 12:42:02 PM ERROR Init... Failed while loading Configuration and testing connections... cannot continue. Please check your settings in Config.xml
08/16/2012 12:42:02 PM NOTICE Summary:---Overa ----------------------Archiving
Summary for ---Overall-------------------------
08/16/2012 12:42:02 PM NOTICE Summary:---Overa TV Success: 0, TV Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Movie Success: 0, Movie Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Music Video Success: 0, Music Video Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa New videos archived: 0, existing videos updated: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Overall: Success: 0, Skip: 0, Fail: 0
08/16/2012 12:42:02 PM NOTICE Ending Done... Total processing time: 0 minute(s), 0 second(s)

any idea?
can it support hebrew titles?
our VOD 's episode titles are in hebrew and i suspect that is why it is giving me errors.
what should i do to support hebrew languages?
thanks

config.xml file - http://pastebin.com/svYKx71e
no log available (didnt create one)

so the main idea is does this support non-English characters in the stream titles

i dont know if it can support non-english characters or not but youre getting that error because your xml file has something wrong in it. most likely you opened a tag without closing it or closed a tag you hadnt opened. I dont see any non-english characters in your xml file, so if it was going to error because of that it would be after loading the config. Youll need to meticulously skim through your config file and make sure everything is in the proper syntax or try some xml verifiers online, though ive had a lot of trouble with those giving bogus errors. Good luck.

thanks for your reply.
when i added a plugin with an english subdirectory it managed to find it but couldnt retrieve any of the hebrew titled shows.
how can i get it to support non-english characters. is it in the java or the xml file?
if i can get it to work it will help literary thousands of Hebrew speaking xbmc users


Reply
(2012-08-17, 10:49)EZ1976 Wrote:
(2012-08-16, 23:38)spanktastic2120 Wrote:
(2012-08-16, 11:46)EZ1976 Wrote: hi
well i tried to edit the config file and have it use one of israeli's vod websites.

i have this in my config file:

<hotVOD path="plugin://plugin.video.hotVOD.video" recursive="true" >
<subfolder name="8 ץורע" type="movies" >


and i get this :

C:\1\XBMC.MyLibrary-1.3.1\res\scraper.db
08/16/2012 12:42:02 PM ERROR Init... Could not find valid xml document at: C:\1\XBMC.MyLibrary-1.3.1\Config.xml. Cannot continue... Please check config.xml with an XML validator.
08/16/2012 12:42:02 PM ERROR Init... Failed while loading Configuration and testing connections... cannot continue. Please check your settings in Config.xml
08/16/2012 12:42:02 PM NOTICE Summary:---Overa ----------------------Archiving
Summary for ---Overall-------------------------
08/16/2012 12:42:02 PM NOTICE Summary:---Overa TV Success: 0, TV Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Movie Success: 0, Movie Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Music Video Success: 0, Music Video Fail: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa New videos archived: 0, existing videos updated: 0
08/16/2012 12:42:02 PM NOTICE Summary:---Overa Overall: Success: 0, Skip: 0, Fail: 0
08/16/2012 12:42:02 PM NOTICE Ending Done... Total processing time: 0 minute(s), 0 second(s)

any idea?
can it support hebrew titles?
our VOD 's episode titles are in hebrew and i suspect that is why it is giving me errors.
what should i do to support hebrew languages?
thanks

config.xml file - http://pastebin.com/svYKx71e
no log available (didnt create one)

so the main idea is does this support non-English characters in the stream titles

i dont know if it can support non-english characters or not but youre getting that error because your xml file has something wrong in it. most likely you opened a tag without closing it or closed a tag you hadnt opened. I dont see any non-english characters in your xml file, so if it was going to error because of that it would be after loading the config. Youll need to meticulously skim through your config file and make sure everything is in the proper syntax or try some xml verifiers online, though ive had a lot of trouble with those giving bogus errors. Good luck.

thanks for your reply.
when i added a plugin with an english subdirectory it managed to find it but couldnt retrieve any of the hebrew titled shows.
how can i get it to support non-english characters. is it in the java or the xml file?
if i can get it to work it will help literary thousands of Hebrew speaking xbmc users

if it skips those directories entirely then someone who has contributed to this project is going to need to help you solve that. barring that, there are pretty much just 2 reasons why it would not add them to your library. 1) the hebrew show names are not on thetvdb so the lookup fails or 2) the xbmc database doesnt support hebrew so they cannot be added. for 1 you should be able to fix it with manual archiving, but you wont have any information about that show in your library. for 2 you may be able to specify a different encoding somehow that supports hebrew. i think its most likely number 1. i hope this gives you some more troubleshooting options, good luck.
Reply
hi
first of all thanks for your reply.
i didnt know that the thetvdb is a critical part of the process.
i really dont mind not having the information (although they are available in the info on the xbmc plugin).

if i find an israeli plugin with an english subdirectory it manage to process it but cant find anything (since all of the shows names are in hebrew) but if i request to search for a subdirectory which has a hebrew title it tells me it cant find it (although in the log it is written correct).


my guess is that it doesnt support hebrew (if you have any idea how i can add it. my guess its in the jar file of the software).
can you guide me how to do a manual archiving to fix guess number 1?

thanks


this is the log

08/17/2012 10:18:28 PM INFO Init... Initializing SQLite database at: C:\1\XBMC.MyLibrary-1.3.1\res\ArchivedFiles.db
08/17/2012 10:18:28 PM INFO Init... Initializing SQLite database at: C:\1\XBMC.MyLibrary-1.3.1\res\QueuedMetaDataChanges.db
08/17/2012 10:18:28 PM INFO Init... Initializing SQLite database at: C:\1\XBMC.MyLibrary-1.3.1\res\scraper.db
08/17/2012 10:18:31 PM ERROR Find:Subfolder No matching subfolder named "makoVOD/הפרלמנט" was found. Skipping

and this is the config file

<makoVOD path="plugin://plugin.video.makoVOD.video" recursive="true" custom_parser="PlayOn" >

<!-- All episodes over 20 minutes long in any subdirectory of the following shows -->
<!-- Had to list each show separately because the tool doesn't allow you to specify the plugin root -->
<subfolder name="הפרלמנט" type="episodes" >
<filter>
<runtime>GT|1200</runtime>
</filter>
</subfolder>


the hebrew subfolder is correct. it seems that the java file cant recognize hebrew characters. i am guessing its the non-english characters and i am guessing many xbmc users around the world would like to use this.
so please please please how can i have this recognize non-english folders?
thanks
Reply
The latest round of bug fixes and enhancements are now complete.

Change Log
  1. Fixed a problem with the runtime filter not working correctly in the previous patch
  2. Added a new type called "generic" to contrast the 3 very specific types previously supported to provide support for content that does not have metadata readily available from sources like TVDB.
  3. Changed where the forced_series resides in the config XML. You will now find it as an attribute of the sub-folder instead of a child element. (Inheritance now works).
  4. Changed how the force_series feature is applied. It will now apply to ALL scenarios for that sub-folder.

The New Generic Type
This type will scrapes series and title only to build a set of strm files without producing .nfo files (since there is no metadata to write). The streams do not belong in the movies, tv shows, or music videos output folders so they instead write into a folder called "generic" that sits along side the TV Shows folder at the same level. Under that level is a sub-folder named using the parsed series name, then the series and title are both used in naming the written strm file. This works well for content like TED talks, and works even better in conjunction with the parser and force_series features. The reason I developed this was to produce a directory that is populated with strm files that are not scanned into the XBMC library (since they are not movies, music videos, or tv shows). Each resulting series level directory is intended to be made available to PseudoTV as a directory channel for documentary and learning type channels.

Code:
<!-- TED Videos -->
<TED path="plugin://plugin.video.ted.talks" recursive="true" >
    <!-- Newest TED Talks -->
    <subfolder name="Newest Talks" type="generic" suffix=" (TED)" force_series="TED Talks" >
    
        <!--Parse the series name then the title of the episode -->
        <parser>
            <regexp>()([\w\s*'-:]*)</regexp> <!-- ex: Include the entire name "Show Name: Title of the Episode" -->
        </parser>
        
    </subfolder>        
</TED>

Output of the Above Configuration
Image

Force Series Feature
The force series feature simply overrides the series parsed by the tool. No matter how it came up with the series that it did, this configuration will override the parsed value with the value specified instead. The place where it is specified in the configuration file has just moved from it's own child element over into an attribute of the sub-folder element. This means it will follow the same inheritance model as all other sub-folder attributes (child sub-folder configurations will assume the same feature unless overridden with their own).

Code:
<subfolder name="Newest Talks" type="generic" suffix=" (TED)" force_series="TED Talks" >



The Patch: http://pastebin.com/ZMJDWTX7
The Binaries: AngryCamel - 20120817 1620 - bin.zip
The Commit Suggestion: http://code.google.com/p/xbmc-mylibrary/...etail?id=4
Integrating with XBMC through the JSON-RPC interface?

Try the JSON-RPC Browser

Image
Reply
[quote='angrycamel' pid='1171802' dateline='1345262558']
The latest round of bug fixes and enhancements are now complete.

Change Log
  1. Fixed a problem with the runtime filter not working correctly in the previous patch
  2. Added a new type called "generic" to contrast the 3 very specific types previously supported to provide support for content that does not have metadata readily available from sources like TVDB.
  3. Changed where the forced_series resides in the config XML. You will now find it as an attribute of the sub-folder instead of a child element. (Inheritance now works).
  4. Changed how the force_series feature is applied. It will now apply to ALL scenarios for that sub-folder.

The New Generic Type
This type will scrapes series and title only to build a set of strm files without producing .nfo files (since there is no metadata to write). The streams do not belong in the movies, tv shows, or music videos output folders so they instead write into a folder called "generic" that sits along side the TV Shows folder at the same level. Under that level is a sub-folder named using the parsed series name, then the series and title are both used in naming the written strm file. This works well for content like TED talks, and works even better in conjunction with the parser and force_series features. The reason I developed this was to produce a directory that is populated with strm files that are not scanned into the XBMC library (since they are not movies, music videos, or tv shows). Each resulting series level directory is intended to be made available to PseudoTV as a directory channel for documentary and learning type channels.

Code:
<!-- TED Videos -->
<TED path="plugin://plugin.video.ted.talks" recursive="true" >
    <!-- Newest TED Talks -->
    <subfolder name="Newest Talks" type="generic" suffix=" (TED)" force_series="TED Talks" >
    
        <!--Parse the series name then the title of the episode -->
        <parser>
            <regexp>()([\w\s*'-:]*)</regexp> <!-- ex: Include the entire name "Show Name: Title of the Episode" -->
        </parser>
        
    </subfolder>        
</TED>


hi.
thank you for your software. it does complete the missing features in my media center.
any chance this fixes my issue with non-english characters?
Reply
(2012-08-18, 08:12)EZ1976 Wrote: hi.
thank you for your software. it does complete the missing features in my media center.
any chance this fixes my issue with non-english characters?

I'm sorry but I do not know how to deal with non-english characters so I will not be able to help you.
Integrating with XBMC through the JSON-RPC interface?

Try the JSON-RPC Browser

Image
Reply
Another Feature
I got a little carried away tonight and knocked out another feature that I felt was missing from the tool, support for an OR logical operator in the filters section. This feature allows for a much cleaner configuration when filtering the content that you want included. Simply put, you can put a list of matches and it if matches any of them, then that contains filter will be considered successful. Between multiple contains elements it is still an AND operation, but within it can now be a list of items.

Quote:Only videos that match ALL filters will be included (unless it matched an exclude)
That is what the documentation stated for the contains element of a filter. Unfortunately, that does not provide any way of specifying a logical operator OR to the filter. Because it must match ALL, it is always performing an AND comparison for each "contains" item listed. This features keeps that AND behavior between each contains element but allows the text of the contains element body to be delimited by an OR operator "||". This double pipe OR operator will be used as a delimiter to split the contains criteria and if any of them match, then the filter will pass.

It's easier to see it...
Code:
<!-- All full episodes over 20 minutes long for a subset of the History network shows -->
<subfolder name="History Channel" type="episodes" >
    <filter>
        <contains>Full Episodes</contains>
        <runtime>GT|1200</runtime>
        <contains>Modern Marvels || Top Gear || The Universe || Ice Road Truckers || Ax Men</contains>
    </filter>
</subfolder>

Here it is with some context... (scoombe, the last filter in this example may help you on your way learning Java)
Code:
<!--
<SearchFilters>                
    This is where you define what sources in XBMC should be added to the library.
    <source> - The source to search. The element name can be whatever you want. If the name matches a source in XBMC, you do no need to specify the path attribute.
        path - This is the actual path that XBMC uses for this source. If you are unsure what the path is, check your sources.xml is your userdata folder. Not needed if source name matches the XBMC source name.            
        <subfolder> - Each source can have multiple subfolder elements. These are directories that should be scanned and added to the library.
            *Note: all of the attributes below can be used at the subfolder level or the source level. If specified at both levels, the subfolder takes precedence.
                    If specified at the source level, the attribute will be inherited by all subfolders.
                    Also, subfolders can be nested inside of other subfolders, and all attributes will be inherited. The name attribute will be appended as you would expect.
            name - required attribute; The name of the subfolder. (do not include the source name, just the subfolder). Use '/' to seperate folders in the name.
            type - optional attribute; default=auto determine; This is the type of content in the directory. Only set this if all content in the directory is the same type.
                    If the content type is all known to be TV Shows, use 'episodes';
                    If the content type is all known to be movies Movies/Films, use 'movies'
                    If the content type is all known to be movies Music Videos, use 'music_videos'
                    If the content type is all known to be none of the above, use 'generic' - This will simply write a directory full of the strm files
                    NOTE: this does not filter the content in the directory; it tells this program what kind of content to expect!
            recursive - optional attribute; default=false; If true, the directory and all subfolders will be scanned for content. Otherwise only the top directory will be scanned.
            regex_name - optional, If true, the name attribute wil be evaluated as a regular expression. The regex must match the full actual name of the subfolder. Partial regex matches will be discarded!                                
            force_tvdb - optional, default=false; For episodes only. If true, the season/episode numbers retrieved from the source will be ignored and the show will be looked up on TheTVDB.com. (useful if the source incorrectly/absolutely numbers episodes for some series)
            max_series - optional attribute; default=unlimited; If TV Shows are in this subfolder, this is the maxinum number of TV series to archive. Useful for Hulu/Popular Shows filtering the top X series you want in your library.
            max_videos - optional attribute; default=unlimited; This is the maxinum number of episodes/movies/music videos to retrieve from the subfolder.
            movie_set - optional attribute; default=none; Specifies the name of the movie set to add the movie to; (Ignored for TV Episodes/Music Videos). More info: http://wiki.xbmc.org/index.php?title=Movie_Sets
            prefix - optional attribute; default=none; This will be prepended to the title of TV Episodes/Music Vidoes (Ignored for movies) in the directory. Can be useful if you want to identify which episodes are from which source.
            suffix - optional attribute; default=none; This will be appended  to the title of TV Episodes/Music Vidoes (Ignored for movies) in the directory. Can be useful if you want to identify which episodes are from which source.
            multi_part - optional arrtibute; default=false; Set this to true if the subfolder might contain any multi-part vidoes                
            force_series - optional attribute; default=none; This will override any series name that gets parsed.
            <exclude> - optional; If a video's path matches ANY of these, it will be skipped.
            <filter>  - optional; Only videos that match ALL filters will be included (unless it matched an exclude)
                <contains> - Matches if the video path contains at least one of the literal texts delimited by double pipes "||" (case-insensitive)
                <regexp> - Matches if any part of the video path matches this Regular Expression (case-insensitive) - see here for info on regex: http://java.sun.com/developer/technicalArticles/releases/1.4regex/
                <runtime> - Matches if the runtime of the file fits the criteria specified in seconds along with the relational operator value. The format is "<relational_operator>|<runtime_seconds>". Posible relational operators are: EQ:Equal to, GT:Greater than, LT:Less than, NE:Not equal to, GE:Greater than or equal to, LE:Less than or equal to. Matches only on files and not directories.
            <parser>  - optional; Overrides the default series and title parser with a regular expression that must contain two defined groups.
                <regexp> - Pattern must contain two groups like in the following: ([\w\s*'-]*):([\w\s*'-]*) pattern. The first will be the series, and the second the title. - see here for info on regex: http://java.sun.com/developer/technicalArticles/releases/1.4regex/
            
        Note: The exclude/filters filter on the video path. A typical video path looks something like "Hulu/Popular/Popular Episodes/s01e01 - MyTVShow - Pilot"
        Note: Be careful what you recursively search for. For example, a recursive search of PlayOn's Netflix/Browse Genres returns about 75,000 videos!
        Note: The '/' character is used exclusively to seperate folders. It cannot otherwise be used in the name attribute (there is currently no way to escape it)            
        
-->

<SearchFilters>
    <!--
    See here for sample search filter configurations: https://docs.google.com/document/d/1CDbdU1GOJlIblwGH8vaJT5c_99lK5H-gRMe3CFpZpUs/edit?hl=en#bookmark=id.g4i0iqa50b0b
    -->
    
    <!-- FreeCable Streaming TV Network Content -->
    <FreeCable path="plugin://plugin.video.free.cable" recursive="true" prefix="(F) " >
        <!-- All full episodes over 20 minutes long for a subset of the USA network shows -->
        <subfolder name="USA" type="episodes" >
            <filter>
                <contains>Full Episodes</contains>
                <runtime>GT|1200</runtime>
                <contains>Fairly Legal || In Plain Sight || Burn Notice</contains>
            </filter>
        </subfolder>
        <!-- All full episodes over 20 minutes long for a subset of the History network shows -->
        <subfolder name="History Channel" type="episodes" >
            <filter>
                <contains>Full Episodes</contains>
                <runtime>GT|1200</runtime>
                <contains>Modern Marvels || Top Gear || The Universe || Ice Road Truckers || Ax Men</contains>
            </filter>
        </subfolder>
    </FreeCable>
    
    <!-- TED Videos -->
    <TED path="plugin://plugin.video.ted.talks" recursive="true" >
        <!-- Newest TED Talks -->
        <subfolder name="Newest Talks" type="generic" suffix=" (TED)" force_series="TED Talks" >
            <!--Parse the series name then the title of the episode -->
            <parser>
                <regexp>()([\w\s*'-:]*)</regexp> <!-- ex: Include the entire name "Show Name: Title of the Episode" -->
            </parser>
        </subfolder>        
    </TED>
    
    <!-- Lectures -->
    <VideoLectures path="plugin.video.videolectures.net" recursive="true" regex_name="true" >
        <!--Java training courses -->
        <subfolder name="Categories/Computers \([0-9]+\)/Programming \([0-9]+\)/Java \([0-9]+\)" type="generic" force_series="Java Training" >  <!-- matches on directories like: /Categories/Computers (369)/Programming (138)/Java (4)/ -->
            <parser>
                <regexp>()([\w\s*'-:]*)</regexp>
            </parser>
            
            <filter>
                <runtime>GE|900</runtime>
                <regexp>Lecture [0-9]+</regexp>                
            </filter>
        </subfolder>
    </VideoLectures>

</SearchFilters>

The Patch: http://pastebin.com/mX2zM5sm
The Binaries: AngryCamel - 20120818 0201 - bin.zip
The Commit Suggestion: http://code.google.com/p/xbmc-mylibrary/...etail?id=6
Integrating with XBMC through the JSON-RPC interface?

Try the JSON-RPC Browser

Image
Reply
(2012-08-18, 09:11)angrycamel Wrote:
(2012-08-18, 08:12)EZ1976 Wrote: hi.
thank you for your software. it does complete the missing features in my media center.
any chance this fixes my issue with non-english characters?

I'm sorry but I do not know how to deal with non-english characters so I will not be able to help you.

too bad.
there are a few thousand xbmc users in israel that will love this feature.
it does recognize the folders but cant match it for some reason.
any idea how i can start to investigate? it is really important
thanks
Reply
(2012-08-18, 12:44)EZ1976 Wrote:
(2012-08-18, 09:11)angrycamel Wrote:
(2012-08-18, 08:12)EZ1976 Wrote: hi.
thank you for your software. it does complete the missing features in my media center.
any chance this fixes my issue with non-english characters?

I'm sorry but I do not know how to deal with non-english characters so I will not be able to help you.

too bad.
there are a few thousand xbmc users in israel that will love this feature.
it does recognize the folders but cant match it for some reason.
any idea how i can start to investigate? it is really important
thanks

well since you have to specify the folder because it wont take root folders try to find a plugin or folder that has a structure similar to "plugin://[english folder name]/[hebrew folder name]" then you can just specify the english name and recursion and see if it matches anything then. with the error youre getting manual archiving will not fix it.
Reply
Is it possible to have 2 different plugin scrapers in the same xml file, if so how? I tried making a Crackle2 filter and added it to the top of the config file and get an error(tried copying and pasting hulu filter with changes). Has anyone made any other filters for other plugins, I'm sure these would be universal so that anyone could use them.
Reply
sorry for a dumb question - i downloaded mylibrary and tried to install it on my raspberry pi - via install add on zip file. it gave me an error - as not right folder structure.
Reply
(2012-08-19, 02:12)bittihuduga Wrote: sorry for a dumb question - i downloaded mylibrary and tried to install it on my raspberry pi - via install add on zip file. it gave me an error - as not right folder structure.

It is not an addon
Integrating with XBMC through the JSON-RPC interface?

Try the JSON-RPC Browser

Image
Reply
  • 1
  • 30
  • 31
  • 32(current)
  • 33
  • 34
  • 100

Logout Mark Read Team Forum Stats Members Help
XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins)10