mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - outleradam - 2009-12-31 20:33
- Episodes renamed to Show name.SxxExx (episode title).ext
- Movies renamed to Movie (year).ext
- Special provisions for naming sports and news which is compatible with XBMC.
- Symlinks allow MythTV to maintain your library.
- TvDb recognition of Show Name and Episode Name
- Zap2it ID and fuzzy logic Title matching allows for perfect naming of Titles
- Airdate and fuzzy logic episode matching allows for perfectly named episodes
- User defined show name translations for improper guide data
- User is notified at the end of each operation, XBMC is updated
- Dynamic mount support with alternate move dir
- mythicalSetup allows for easy installation and selection of options, while also making decisions based on your setup.
- Can be run as MythTV job.
- Scan mode will set the naming convention of your existing library
- User jobs can be configured to run using the information mythicalLibrarian has collected
How to use:
mythicalLibrarian myfile.ext "title to match" "subtitle to match"
--or as a user job in mythtv--
mythicalLibrarian will generate usable content for XBMC out of your MythTV recorded libraries by utilizing the standard Showname.SxxExx (Episode Title).ext format.
run the mythicalSetup.sh program
run mythicalLibrarian --help
This will explain how to use everything.
It can be configured in the following ways
- Move your files and create symlinks in place of the original so that MythTV can manage your library
- Leave the original in place and create symlinks to your folder
- Move files to a NAS and if the NAS is not present, move it to a local folder
- After extinguishing all other resources, create a symlink to the original file in a local folder
I would like any and all problem reports for mythicalLibrarian to be filed here.
A problem report consists of a brief discription of the problem and a copy of a single debug log from output.log Please report anything you consider to be a problem as it will help out others and improve the development of the program. Before posting, please look at the log to see if it contains any valuable information.
The entire debug log is located in ~/.mythicalLibrarian/output.log and a single log starts with "NEW SEARCH INITIATED AT" and contains the words "DEBUG MODE ENABLED". Logs should be wrapped in [ code] [ /code] tags for clenliness.
Here is an example debug log:
Please report all errors you find.
@@@@@@@@@@@NEW SEARCH INITIATED AT Mon Sep 20 18:30:42 CDT 2010@@@@@@@@@@@@@@
Revision 600 Mon Sep 20 17:56:31 CDT 2010 on Linux
PERFORMING MAINTENANCE ROUTINE
REMOVING ORPHAN /home/mythtv/Videos/Episodes/SpongeBob SquarePants/Season 6/SpongeBob SquarePants S06E16 (Boating Buddies).txt
REMOVING ORPHAN /home/mythtv/Videos/Episodes/SpongeBob SquarePants/Season 6/SpongeBob SquarePants S06E10 (A Life in a Day).txt
REMOVING ORPHAN FOLDER:/home/mythtv/.mythicalLibrarian/mythicalSetup/Episodes/Mister Rogers Neighborhood/Season 7
REMOVING ORPHAN FOLDER:/home/mythtv/.mythicalLibrarian/mythicalSetup/Episodes/Mister Rogers Neighborhood
SEARCHING: www.TheTvDb.com SHOW NAME: Tosh.0 EPISODE:
DATE: 2009-08-06 FILE NAME: /home/mythtv/Videos/1056_20100920180000.mpg
SEARCH FOUND:Tosh.0 ID#: 97731
MATCH!!!! Zap2itID:Tosh.0 ID.97731
###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/home/mythtv/Videos/Episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/home/mythtv/Videos/Episodes-
USE ORIGINAL DIR:Enabled NOTIFICATIONS:Enabled
INPUT SHOW NAME:Tosh.0- LOCAL SHOW NAME TRANSLATION:- ShowName:Tosh.0
DATABASE UPDATED:- TVDB LAST UPDATED:1285090977- CURRENT:1285026385-
RESOLVED SERIES ID:97731- RESOVED SHOW NAME:Tosh.0-
INPUT EPISODE NAME:- ABSOLOUTE EPISODE NUMBER:9- RESOLVED EPISODE NAME:Web Redemption Drunk Dunk-
SEASON:S01- EPISODE:E09- SYMLINK MODE:MOVE- FILESIZE: 0kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:4355428kB- WORKINGDIR:1 FREE:4355364kB-
MOVEDIRWRITABLE:1- FREE:4355428kB- ALTERNATEMOVEDIR:1- FREE:4355428kB-
PRIMARYSHOWDIRWRITABLE:1- FREE:4355428kB-ALTERNATESHOWDIRWRITABLE:1- FREE:4355428kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:4355428kB- ALTERNATEMOVIEDIR:1- FREE:4355428kB-
RECSTART:2010-09-20 18:00:00- MOVIE YEAR:- ORIGINAL SERIES DATE:2009-08-06-
PROGRAMID:EP011496300009- CHANNEL ID:1056- CATEGORY:Comedy- GOFORDOOVER:1-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:1149630- MATCHED TVDB SERIES ID:97731-
PLOT: Daniel's Popcorn Challenge; Dizzy Dunk's web redemption; Tommy Chong.
####################END OF DEBUG LOG#####################
MOVING FILE: '/home/mythtv/Videos/1056_20100920180000.mpg' to '/home/mythtv/Videos/Episodes/Tosh.0/Season 1/Tosh.0 S01E09 (Web Redemption Drunk Dunk).mpg'
@@@@@@@@@@@@@OPERATION COMPLETE Mon Sep 20 18:30:47 CDT 2010 @@@@@@@@@@@@@@@@
I am subscribed to this thread and will be notified within about 15 minutes on my Android.
Last "stable" is here: http://mythicallibrarian.googlecode.com/files/mythicalSetup.sh
Newest version will be here: http://mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh
I am accepting donations at my sourceforge page as a way to justify the time I spend on this project to my girlfriend: https://sourceforge.net/projects/mythicallibrari/
- outleradam - 2010-01-03 20:58
Corrected problem: when a folder becomes disconnected the comskip tracking are all removed.
- outleradam - 2010-01-05 07:34
Patch has been applied to allow users to use their local database even without new data from thetvdb. It's working now. replace mythicalLibrarian.sh and it will use your current database until TheTvDb comes back up online and mythicalLibrarian's database can be updated again.
This is a perfect time to explain the doover.sh as well. doover.sh is just what it is named, it's a do-over. It records the jobs that have been run and allows you to re-run them if they've failed.
mythicalLibrarian will then delete the list, cycle through each and every failed job it has performed, and rebuild the list of failed jobs. This way if TheTvDb is down, it will be easy to redo the jobs. Also, if the information is not in TheTvDb, you can add it, then go back and re-run doover.sh.
sudo su mythtv
sudo chmod +x /home/mythtv/mythicalLibrarian/doover.sh
Between the two of these, downtime of TheTvDb should mean minimal impact
- althekiller - 2010-01-05 08:12
How many threads do you need for this thing?
- outleradam - 2010-01-05 14:45
Two. Are you saying I should close down the MythSExx thread?
- Bueller - 2010-01-12 12:02
Thanks for the great script, I've been using it for about 1 week. I've run across an issue where the search logic doesn't find the correct series. It's for the show "Castle".
MythTV runs the following after the show has been recorded:
mythicalLibrarian.sh "Castle" "A Rose for Everafter" "/Recordings/1151_20100111210000.mpg", if fails with the following output:
RECSTART:2010-01-11 21:00:00 MOVIEYEAR: SERIESDATE:2010-01-11
PROGRAMID:EP010855880022 ShowCategory:Crime STARS:0 RATING:0
PLOT: While investigating the death of a bridesmaid at a wedding, Castle runs into an old flame (Alyssa Milano).
COMMERCIAL DATA START:25020 STOP:38123
COMMERCIAL DATA START:67537 STOP:81274
COMMERCIAL DATA START:109253 STOP:125929
COMMERCIAL DATA START:149902 STOP:163637
COMMERCIAL DATA START:179625 STOP:188904
FUZZY Exx NUMBER:
EPISODE:A Rose for Everafter ABSOLUTE NUMBER:
MOVEDIR FREE SPACE: 338262196kB- WRITABLE:1
ALTMOVEDIR FREE SPACE: 54666004kB- WRITABLE:1
MOVIEDIR FREE SPACE: 338262196kB- WRITABLE:1
ALTMOVIEDIR FREE SPACE: 54666004kB- WRITABLE:1
ERROR: INFORMATION COULD NOT BE OBTAINED
Manually looking up the series on thetvdb.com show the show name as "Castle (2009)".
Changing the line in doover.sh to: mythicalLibrarian.sh "Castle (2009)" "A Rose for Everafter" "/Recordings/1151_20100111210000.mpg" causes the script to find the show and complete successfully.
Any chance you can add some logic to the script on failure to look for series names with years appended to them.
- outleradam - 2010-01-12 15:39
Doh! I left that woot in there...
I will need the single debug log entry from /mythicalLibrarian/output.log to do any troubleshooting. Also I will release an un"woot"ed version tonight.
- outleradam - 2010-01-12 16:01
Actually, create a file in mythicalLibrarian/ called showtranslations. Then put the following line in it:
Castle = Castle (2009)
This is the built in option.
During the day at work I check this thread on my iPhone. I just saw a terminal output and asked for a debug log. A debug log is always the best.
As for looking up a non specified name, that would be bad. The fuzzy logic would eventually make a match of a show which is completely unrelated. If searching battlestar gallactica, it would search 4 different shows to make a match. If mythicalLibrarian were to pull the original airdate year and apply it to the show name it would only be valid for season 1.
Shows which do not conform to thetvdb can be taken up with your guide data provider. mythicalLibrarian will work with what it is supplied by showtranslations, the guide data, thetvdb and MythTV. The problem in this case is the guide data, however mythicalLibrarian supplies a workaround.
- outleradam - 2010-01-12 19:58
I was just reviewing thetvdb.com and found that the XML contains a zap2it Id. I will look into using this tonight.
The basic flow I am envisioning would be alot of work and would severely slow down the execution of mythicalLibrarian on certain shows, but if possible it would be worth it.
Basically this check would involve getting info from mythtv, performing the seriesid search, then downlading and parsing the first result, verifying the zaptoit ID, if it does not pass, then downloading the next XML and parsing it.. This will result in creation of many unneeded databases. I will see what I can come up with
- outleradam - 2010-01-13 06:13
I came up with a fix. Here's the proof:
You can find the "unstable" version at sourceforge https://sourceforge.net/projects/mythicallibrari/files/
/home/mythtv/mythicalLibrarian/mythicalLibrarian.sh "castle" "A Rose For Ever After" "/home/adam/Desktop/new file .ext"
Accessing MythTV DataBase:
RECSTART: MOVIEYEAR: SERIESDATE:
PROGRAMID: ShowCategory: STARS: RATING:0
Searching TheTvDb FOR Zap2it SERIES ID MATCH SH1085588
Please add Castle ID 82607 Zap2it entries to theTvDb.com
Please add Lavender Castle ID 72403 Zap2it entries to theTvDb.com
Please add Queenie's Castle ID 74533 Zap2it entries to theTvDb.com
Please add Union Castle ID 75368 Zap2it entries to theTvDb.com
Please add Eureeka's Castle ID 78403 Zap2it entries to theTvDb.com
Please add The Toy Castle ID 74602 Zap2it entries to theTvDb.com
Please add Blanding's Castle ID 71008 Zap2it entries to theTvDb.com
Please add Takeshi's Castle ID 74787 Zap2it entries to theTvDb.com
FOUND MATCH: Castle (2009) Zap2it: SH1085588 TheTvDb: 83462
Please add Up the Elephant and Round the Castle ID 96601 Zap2it entries to theTvDb.com
Please add Castle Haven ID 70971 Zap2it entries to theTvDb.com
IDENTIFIED SERIES AS ID:83462 - Castle (2009)
FUZZY Exx NUMBER:22
EPISODE:A Rose For Ever After ABSOLUTE NUMBER:22 S02E12
MOVEDIR FREE SPACE: 139744kB- WRITABLE:1
ALTMOVEDIR FREE SPACE: 3060236kB- WRITABLE:1
MOVIEDIR FREE SPACE: 139744kB- WRITABLE:1
ALTMOVIEDIR FREE SPACE: 3060236kB- WRITABLE:1
SET TARGET DIR AS: /home/mythtv/NAS/Video/shows
FILE NAME EXISTS. FILE WILL BE KNOWN AS: Castle 2009.S02E12 (A Rose For Ever After)-2
CREATING LINK IN LINK MODE