XBMC Community Forum
XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (/forumdisplay.php?fid=116)
+--- Thread: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) (/showthread.php?tid=98210)



RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - spanktastic2120 - 2012-08-05 21:00

(2012-07-26 01:30)scoombe Wrote:  And one more question. How does the script handle "&" in the parsing? I'm trying to get a video from the following location from Netflix through PlayOn:

<subfolder name="Browse Genres/Music/Rock & Pop Concerts/Classic Rock/L-P/Little Feat: High Wire Act Live 2003" type="music_videos" />

Any suggestions?

(2012-08-01 16:00)slopokdave Wrote:  Can someone tell me how to include a show with a forward slash in it's name in Hulu? I.e. "ABC 20/20". I don't want it to think the / is a break in the folder. Edit: I tried something, but it didn't work... now I've removed ABC 20/20 and yet it's skipping all my shows saying:

This label contains a '/', will remove it because it is reserved: "ABC 20/20"

In the meantime, here is code that is working for me to pull selective episodes from Hulu:
Code:
<!--Hulu videos from BlueCop's hulu plugin -->
<Hulu path="plugin://plugin.video.hulu" recursive="true" regex_name="true" >                                                                                                                    

<!--Path to full selected show from Subscriptions -->
        <subfolder name="Subscriptions/Bachelor Pad/Episodes \([0-9]+\)"
                max_series="10" type="episodes" suffix=" (Hulu)" >
                <filter>
                        <regexp>/Episodes \([0-9]+\)/</regexp>                        
                </filter>
                <exclude>                                      
                        <regexp>/season[s]? \([0-9]+\)</regexp>              o          
                </exclude>
        </subfolder>                                    

        <exclude>
                <contains>/Recommended</contains>
    <contains>/Clips</contains>
                <contains>previous page</contains>                                    
                <contains>next page</contains>                                        
        </exclude>
</Hulu>
Also, here is one I found burried in the thread on XBMC Flix (netflix):

Code:
<XBMCFlix path="plugin://plugin.video.xbmcflicks" recursive="true" prefix="[NF] " movie_set="XBMCFlix">
        <subfolder name="Instant Movies and Shows/Instant Queue: All"  type="movies" recursive="false">
            <exclude>
                <regexp>s[0-9]+e[0-9]+</regexp>
            </exclude>
        </subfolder>
        
        <subfolder name="Instant Movies and Shows/Instant Queue: All"  type="episodes">
            <filter>
                <regexp>s[0-9]+e[0-9]+</regexp>
            </filter>
        </subfolder>
    
    </XBMCFlix>

use regular expression and substitute "." for all "/" characters. thats how i added 20/20 from free cable.


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-06 08:52

I just finished a modification that allows for a filter to be created based on the duration or runtime of an episode. This can be very helpful when finding a common pattern for full episodes across different networks and shows is difficult.

Here is the text config I am using. I am running it now to build out all shows from the history channel then it will populate PseudoTV. Still waiting for it to finish running, so I can't say yet on how well it works for that end to end scenario, but I figured it will be better than manually setting up filters for each show.

Code:
    <SearchFilters>
        
        <FreeCable path="plugin://plugin.video.free.cable" recursive="true" prefix="(F) " movie_set=" FreeCable" >
    
            <!-- All History Channel Shows with Episodes over 20 minutes long -->
            <subfolder name="History Channel" type="episodes" >
                <filter>
                    <runtime>GT|1200</runtime>                
                </filter>
            </subfolder>                        
                        
            <exclude>                
                <contains>previous page</contains>                    
                <contains>next page</contains>            
                <contains>/Recommended</contains>                    
                <contains>/Clips</contains>                        
            </exclude>    
                
        </FreeCable>

    </SearchFilters>

It would be nice to have a default config with this tool that would build out a library from scratch (even if there is no content from a local source) with a filter based on the root of FreeCable, runtime of >20 minutes or so, then a few other filters to get rid of the making of and stuff like that. Then the built out PseudoTV channels from the resulting library would be pretty good, with lots of diverse content, for only investing in running two automated scripts.

I added to the default XML to document it's intended usage:

Code:
<runtime> - Matches if the runtime of the file fits the criteria specified in seconds along with the relational operator value. The format is "<relational_operator>|<runtime_seconds>". Posible relational operators are: EQ:Equal to, GT:Greater than, LT:Less than, NE:Not equal to, GE:Greater than or equal to, LE:Less than or equal to. Matches only on files and not directories.

Here is an example filter that with recursive set to true will pick out every show and episode for it that has a duration longer than 20 minutes. It will ignore anything else.


The patch is below:
http://pastebin.com/M8UBVV1v


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-06 09:13

The script finished and it scraped 18 different shows from the history channel in FreeCable, each with full episodes only, organized by season and episode numbers using the new runtime filter option only (see the previous post).

See an image here:
http://postimage.org/image/lk522c6jj/

For those that are interested, I rebuilt the default channels in PseudoTV and all the new shows played great there. It was seeking to the right point in time too. (Note: had to turn on IceLibrary support).


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-06 15:15

Just found a bug in my patch. The content of the runtime property is not consistently formatted. Some appear to load HH:MM format into it instead of converting first to a count of minutes. I will have to detect this and convert on the fly. I'll get that knocked out tonight.

Code:
{
   "id":"1",
   "jsonrpc":"2.0",
   "result":{
      "files":[
         {
            "fanart":"http://thetvdb.com/banners/fanart/original/72289-1.jpg",
            "file":"plugin://plugin.video.free.cable/?url=\"http%3A%2F%2Fabc.go.com%2Fwatch%2F2020%2FSH559026%2FVD55224469%2F2020-83-i-married-twins-and-donor-sexuals\"&mode=\"abc\"&sitemode=\"play\"",
            "filetype":"file",
            "label":"20/20 8/3: I Married Twins and Donor Sexuals",
            "runtime":"38:40",
            "thumbnail":"http://cdn.video.abc.com/abcvideo/video_fep/thumbnails/220x124/2020_20120803_IMarriedTwinsDonerSe_episode_1a4881e0-dc64-46ec-916c-2c674133c849_4752487_220x124.jpg",
            "title":"20/20 8/3: I Married Twins and Donor Sexuals",
            "type":"movie"
         }
      ]
   }
}
08/06/2012 09:10:29 AM DEBUG   Search:FreeCable Found 110 files and 0 directories in FreeCable/ABC/2020 /Season 27
08/06/2012 09:10:29 AM DEBUG   Search:FreeCable Regular recursive match ? true: freecable/abc/2020 /season 27 [starts with] freecable/abc
08/06/2012 09:10:29 AM ERROR   Search:FreeCable Error while parsing json: org.json.JSONException: JSONObject["runtime"] is not an int.



RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - spanktastic2120 - 2012-08-06 17:18

wow thats extremely useful, i hope it gets added. nice work.


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-07 05:34

(2012-08-06 15:15)angrycamel Wrote:  Just found a bug in my patch. The content of the runtime property is not consistently formatted. Some appear to load HH:MM format into it instead of converting first to a count of minutes. I will have to detect this and convert on the fly. I'll get that knocked out tonight.

Fixed this with the following updated patch. http://pastebin.com/keH5BcKE
It will now auto detect the format, parse it out, and convert it to a number of seconds.
If debug logging is turned on you will see something like this:

Code:
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable Parsed 40:34 to 2434 mins
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable Checking runtime filter: GT|1200
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable    Runtime Filter - operator: GT
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable    Runtime Filter - runtime: 1200
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable    Actual Runtime: 2434

(2012-08-06 17:18)spanktastic2120 Wrote:  wow thats extremely useful, i hope it gets added. nice work.

Thanks!


RE: XBMC.MyLibrary - having problems with JSON - nitsuj - 2012-08-08 03:09

I tried to get everything running properly but I had some trouble with JSON.

Here is my myLibrary log file: http://pastebin.com/MeniEVkk
Here is my config file: http://pastebin.com/VBJRzvvH

Any assistance would be greatly appreciated. Thank you in advance!


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - nitsuj - 2012-08-09 19:16

Never mind. I got it all figured out. I ran the given hulu top ten example in the config file and the results were populated in xbmc. They all stream flawlessly from the library.

However, I am having some trouble running the hulu shows through pseudotv. No matter where the psuedotv time is, it just plays the opening ad a few times and then skips to the next thing on that channel until it hits a locally stored video. Are there setting to get this to work correctly or am I just out of luck?


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - spyder - 2012-08-09 19:57

(2012-08-09 19:16)nitsuj Wrote:  Never mind. I got it all figured out. I ran the given hulu top ten example in the config file and the results were populated in xbmc. They all stream flawlessly from the library.

However, I am having some trouble running the hulu shows through pseudotv. No matter where the psuedotv time is, it just plays the opening ad a few times and then skips to the next thing on that channel until it hits a locally stored video. Are there setting to get this to work correctly or am I just out of luck?

I think the only way around this is to disable all commercials in the Hulu plugin.


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - scoombe - 2012-08-09 21:04

(2012-08-07 05:34)angrycamel Wrote:  
(2012-08-06 15:15)angrycamel Wrote:  Just found a bug in my patch. The content of the runtime property is not consistently formatted. Some appear to load HH:MM format into it instead of converting first to a count of minutes. I will have to detect this and convert on the fly. I'll get that knocked out tonight.

Fixed this with the following updated patch. http://pastebin.com/keH5BcKE
It will now auto detect the format, parse it out, and convert it to a number of seconds.
If debug logging is turned on you will see something like this:

Code:
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable Parsed 40:34 to 2434 mins
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable Checking runtime filter: GT|1200
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable    Runtime Filter - operator: GT
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable    Runtime Filter - runtime: 1200
08/06/2012 11:24:12 PM DEBUG   Search:FreeCable    Actual Runtime: 2434

(2012-08-06 17:18)spanktastic2120 Wrote:  wow thats extremely useful, i hope it gets added. nice work.

Thanks!

I would really like to get this working but do not know a thing about java programming/compiling, although I am really eager to learn Big Grin Any chance you could point to me to a good source for compiling the source with your patch or give me some guidance? Thanks for your work upgrading this great tool!


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-09 23:29

I'll upload a binary somewhere until the original tool's author can decide on whether or not to commit the patch to the trunk. Give me a few minutes and I'll edit this reply with it. (heading home from work)

EDIT
I threw it onto Dropbox here https://dl.dropbox.com/u/60216488/AngryCamel%20-%2020120806%202327%20-%20Bin.zip

It contains the sample config that I used to test a very large batch run. Be warned it will take a long time to run this. On my PC, I think it took a couple hours, if I remember correctly. After that though, I had a folder with full episode stream files, properly numbered and named for 200 shows! All in the library perfectly and supported by PseudoTV to boot.

Let us know how it works for you using the duration filter.

EDIT 2
FYI: This file is not a full release. It only contains the patched jar file and the alternative config from my testing as described above. You will need to download the latest from the original author and replace those files with mine, then run it per his normal instructions.


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - scoombe - 2012-08-10 20:04

Thank you! I will try it out tonight and let you know how it works.


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - bradvido88 - 2012-08-10 21:10

Angrycamel, i like the new filter. Sorry i've been out of this thread for a while. I'm waiting for Frodo churn to slow down before making a compatible version.

Did you submit an official patch anywhere? I'm not sure I saw it.

If you are interested, i could add you as a contributer to the google code project.

P.S. does anyone know how to make it so e-mail subscriptions to a thread don't get disabled after you visit the thread?


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - angrycamel - 2012-08-11 03:21

I uploaded the patches to Pastebin and linked them in my posts. I can commit if you want to add me to the project or I can add an issue to the project page with the patch attached. However you want to do it, I don't really have a preference.

EDIT
I went ahead and added it onto the project page just in case you don't get back to this right away. http://code.google.com/p/xbmc-mylibrary/issues/detail?id=2&thanks=2&ts=1344648265


RE: XBMC.MyLibrary - Scan any source to the library (Hulu/Netflix/Plugins) - anime513 - 2012-08-12 03:19

(2012-08-09 19:16)nitsuj Wrote:  Never mind. I got it all figured out. I ran the given hulu top ten example in the config file and the results were populated in xbmc. They all stream flawlessly from the library.

However, I am having some trouble running the hulu shows through pseudotv. No matter where the psuedotv time is, it just plays the opening ad a few times and then skips to the next thing on that channel until it hits a locally stored video. Are there setting to get this to work correctly or am I just out of luck?

Having the same issue, ads play non-stop on PsuedoTV. Files play fine though the regular XBMC interface though. I tried disabling ads in the plugin but they still show up. Do I need Hulu+ to work around this?