XBMC Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for XBMC (/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



- GregoryK - 2012-01-27 19:06

Adam: OK, no worries. Thanks for the reply and for considering it. /Greg


- jerkychew - 2012-01-29 05:23

First off, so far this. is. awesome! I've pointed my Boxee Box to the Episodes directory on my MythTV box and I'm so psyched to see my recordings listed in a sane manner. I re-designed my MythTV setup to get this and MythExport working.

I'm sure I'm missing this, but is there a cfg file or something that lets me change the settings? The script is deleting shows from MythTV box and moving them to the Episodes folder. I see from the wiki that this is the default behavior and I want to change that. I want to set it so that the root Myth recordings aren't modified and symlinks are created in the Episodes folder.

When I initially ran the install script I don't remember seeing any options to change the file manipulation behavior, and I must be dumb because I can't find where to change this in the wiki. I've poked around the .mythicallibrarian folder and can't find any obvious config files.

Thanks in advance, this is an awesome, awesome app.


- GregoryK - 2012-01-29 05:50

jerkychew:

There is a dialog during the mythicalSetup process that lets you choose whether to create links or not.

Check out this section of the Wiki for details. Here's the relevant part:

Quote:Extra settings documentation. The SYMLINK variable specifies to mythicalLibrarian how to treat the files. There are three modes to SYMLINK. MOVE is the preferred mode.

SYMLINK=MOVE - Allows XBMC to see the file and know when it has been removed from the library. When SYMLINK=MOVE and you have set up mythtv-setup to follow symlinks when deleting, MythTV's Library will function normally and XBMC will have access to all content.

SYMLINK=LINK - Keeps the original MythTV file in it's place and creates a link for XBMC to read. This link is not maintained and the symlink will remain after the original file has been deleted

SYMLINK=Disabled - Moves the file to the new location and does not provide a symlink to MythTV. On top of this, it will purge MythTV's database of all information and pictures relating to the video content. MythTV will have no memory of the recording. This mode can be used if you wish to delete files from XBMC.


Personally, I leave it at the default and use XBMC to watch most of my shows recorded by MythTV. Works fine, including deletion, etc.

Hope that helps.

/Greg


- outleradam - 2012-01-29 06:07

Set mythtv to "follow symlinks" when deleting. The default settings allow mythtv to control the original recording, serve it and delete it. Symlinks allow the files to be grouped properly and retain backwards compatibility with mythtv.


- jerkychew - 2012-01-29 07:43

Hmm, are symlinks not supported on a CIFS mount? I just tried again with the same behavior and I think I found something...

My recordings directory is hosted on a Windows Storage Server VM (I'm a Windows admin, sorry in advance Big Grin ) and the output.log says ERROR CREATING SYMLINK: FILESYSTEM MAY NOT SUPPORT SYMLINKS.


- outleradam - 2012-01-29 08:26

I need an output.log. Here's some commands in case you're not familiar with linux.
Code:
sudo su *username*
cat ~/.mythicalLibrarian/output.log
where *username*=the username you run mythicalLibrarian under. Generally it's mythtv.

You should always record your shows locally. If you wish, mythicalSetup has some advanced options for placing your folders in separate mounts. Then have MythicalLibrarian move them there and symlink back. If you're having mythtv record to a remote computer, that's a bad idea. Mythtv should record to the same computer, then move the recording as a job. MythicalLibrarian can handle this by using /Episodes /Showings and /Movies mounts on your recordings folder.

mythicalLibrarian cannot handle networked recordings.

So yeah.. use the mythtv defaults and mount your network folders to /Episodes, /Showings and /Movies. This way MythTV records locally on a native Linux filesystem, then moves them off to your NAS as a user-job with a symlink back to the original which will allow MythTV to maintain control over the files.


- jerkychew - 2012-01-29 18:14

Log dumped to http://pastebin.com/tvfqUL6f

I'll have to think about re-arranging my setup, I wonder if I switch from CIFS to NFS if the symlinks will then work.

Without going into too much detail, my master BE is a vm on an ESXi 5 server, connected to a pair of fiber-channel disk arrays. I front-ended the storage with a Server 2008 VM because A) It's what I know the best and B) I can grow the storage behind the scenes as needed. That Server 2008 box is a hub for all of my other VMs and machines at my house. I have a Plex Media Server, Boxee Media Manager, PlayOn server and Orb caster all pointing back at that server, because, well, I'm insane. Big Grin


- aormond - 2012-02-03 17:00

I'm investigating using mythicalLibrarian with my Mythbuntu box. It looks like SYMLINK=MOVE would be my preferred method, however, does mythicalLibrarian have any awareness of MythTV storage groups that are spread across multiple drives? Specifically, is it able to move files to multiple directories on different drives? I let MythTV keep all three of my hard drives quite full, so it would be impossible for me to let mythicalLibrarian try to move every recording to a unified location.

Alternatively, I could use SYMLINK=LINK. But then I would need to figure out a way to clean up orphaned symlinks. Or is there another way to to deal with this that I'm not thinking of?

Thanks! I'm just trying to plan ahead before I install anything.


- outleradam - 2012-02-03 21:25

jerkychew Wrote:Log dumped to http://pastebin.com/tvfqUL6f

I'll have to think about re-arranging my setup, I wonder if I switch from CIFS to NFS if the symlinks will then work.

Without going into too much detail, my master BE is a vm on an ESXi 5 server, connected to a pair of fiber-channel disk arrays. I front-ended the storage with a Server 2008 VM because A) It's what I know the best and B) I can grow the storage behind the scenes as needed. That Server 2008 box is a hub for all of my other VMs and machines at my house. I have a Plex Media Server, Boxee Media Manager, PlayOn server and Orb caster all pointing back at that server, because, well, I'm insane. Big Grin
All I see are the test logs from setting up mythicalLibrarian.. Do you have any actual logs run by mythtv?


aormond Wrote:I'm investigating using mythicalLibrarian with my Mythbuntu box. It looks like SYMLINK=MOVE would be my preferred method, however, does mythicalLibrarian have any awareness of MythTV storage groups that are spread across multiple drives? Specifically, is it able to move files to multiple directories on different drives? I let MythTV keep all three of my hard drives quite full, so it would be impossible for me to let mythicalLibrarian try to move every recording to a unified location.

Alternatively, I could use SYMLINK=LINK. But then I would need to figure out a way to clean up orphaned symlinks. Or is there another way to to deal with this that I'm not thinking of?

Thanks! I'm just trying to plan ahead before I install anything.
yes. You'll also want:
Code:
TargetPathIsInputPath=Enabled

This will keep the files in the folder MythTV selects, but it will move them into the /Episodes /Showings or /Movies folder IN that folder.

Your other option is to disable that setting and setup a primary or alternate folder. I'd recommend using all mythicalSetup defaults for your setup.

The defaults will work for 99% of all applications.


- aormond - 2012-02-03 22:16

Thanks for the quick response! I'll give it a shot.


- jerkychew - 2012-02-05 07:26

I'm now running ML in LINK mode, still trying to work out the kinks. I ran ML with the --scan mpg /path/to/recordings arguments. After a few days I realized that i need to set up the userjob for each recording rule individually, as any recordings that have run since that first run have not been exported. So, I just ran the command again and found that it created duplicates of links that were already there, appending a -1 to the new file. Is there a way to run ML and have it not create a link if one already exists?


- dougmeredith - 2012-02-17 00:00

I love what this script does, but I'm very nervous about messing up my MythTV setup. Is there any documentation on this script other than the page at xbmc.org? I just tried installing it on a test system, and it did some very unexpected things, such as creating a user.

How does it insert things into the XBMC library? Does it talk to the database directly, or does it need a running XBMC instance?


- sdsnyr94 - 2012-02-26 02:31

I've had mythicalLibrarian running successfully for a while, with a user job running handbrake to encode the files to an h.264 mkv. Recently, I started using the "Appletv2" setting to encode my shows into a format that will be playable in XBMC on the atv2, but I am having trouble with stations that broadcast at 60fps... the conversion to 30fps causes video to be slightly jittery.

Can I have multiple mythtv user jobs, each referencing a different mythicalLibrarian.sh, so I can use different handbrake settings in each? I thought I read a while ago that it was supported, but this thread has gotten to large to look through, and I saw nothing in the wiki.

Is it as simple as creating a mythicalLibrarian2.sh and pointing a new mythtv user job to it?

Thanks!


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - mauibay - 2012-03-26 00:33

My mythbackend has 6 tuners and often has multiple recordings ending at the same time, does mythicalLibrarian handle this ok, and are the any special issues to be aware? Does XBMC have any issues receiving multiple library update commands within the same several seconds?


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - mauibay - 2012-03-26 20:07

Now that I have mythicalLibrarian working, I'd like to do some fine tuning to eliminate incorrectly recognized episodes. I see the instructions for showtranslations, which I expect will work well for shows that are in the theTVdb database. What can I do about shows that are not in the database and don't have a showID at all? Is there an existing mechanism for naming shows and episodes that can't be matched at all? Ideally I'd like to provide the show name myself and have each each episode titled with the date. I record at least 3 episodes each day for shows that theTVdb doesn't have, and they are all getting misrecognized as episodes of the show "Episodes" with Matt LeBlanc, lol!

On that note, since XBMC pulls all the show info itself when the episodes are added to the library, is there a way to stop it from also misrecognizing shows that theTVdb doesn't have? When a show doesn't have a match, I'd like it to just use the show and episode names from the directory tree and leave the other meta info empty. It's a waste to have to poll theTVdb twice as it is, but it's worse than useless if the second pass by XBMC has to screw up the showtranslation done in the first pass. (Yes, I know the purpose of showtranslation is to get an exact match for theTVdb, but what about shows that don't have _any_ match in theTVdb?)

Perhaps the best solution is to try to add the missing shows to theTVdb myself? I haven't ever done that but maybe some of the shows can be added, such as daily local news.
------------------------------------------------------------
Update... I checked some of the shows that are being misrecognized, and discovered that it's not mythicalLibrarian that's getting it wrong! For example, mythicalLibrarian correctly created this:

Episodes/Awake/Season 1/Awake S01E01 (Pilot).mpg

But when XBMC updated the library it added is as Episodes/ 1x01. Episode One. Apparently there's a TV show named "Episodes" and several of my recordings are being named correctly by mythicalLibrarian but getting mangled to this other name by XBMC. I have no idea why, but suspect it has something to do with the mythicalLibrarian directory being named "Episodes". Unless there's a known fix for this, I plan to try changing that directory name to see if the problem stops.

At this point I have mythicalLibrarian appearing to correctly name ALL my recordings, so the dozen or so shows that are misrecognized in the library must be XBMC's fault. XBMC is not adding the correct show and episode names to the library even though they are correct in the mythicalLibrarian directories. I'm baffled, don't they both use the same theTVdb info? What can I do to fix this?

----------------------------------------------------------------------
Update.. Interesting. I renamed the Episodes directory to Recordings and edited mythicalLibrarian to match, then updated the source path in XBMC and cleaned the library and rescanned, and everything got recognized correctly. Note I didn't actually change any of the symlinks in the Episodes directory, I only renamed it to Recordings and that was all XBMC needed to recognize everything correctly.