Kodi Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



- outleradam - 2010-06-02

I am using the same build on a OS X 10.5 machine.


- kvandesteeg - 2010-06-02

Code:
mini:~ htpc$ sudo su htpc
Password:
bash-3.2$ cat ~/.profile
PATH=/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/bin:/usr/local/bin:/usr/local/mysql:/usr/local/mysql/bin:/usr/local/mysql



bash-3.2$ mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 330
Server version: 5.0.89 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> exit
Bye
bash-3.2$ /bin/sh
sh-3.2$ mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 331
Server version: 5.0.89 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> exit
Bye
sh-3.2$ /bin/bash
bash-3.2$ mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 332
Server version: 5.0.89 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> exit
Bye



- outleradam - 2010-06-03

I'm still trying to understand the difference here. It makes no sense. If the username is htpc, then htpc should have access to the database.

This is the area which you are having problems getting past. It's very simple. This is how mythicalLibrarian obtains a Title and Subtitle for the episode. mythtv is supplying $InputFileBasename via user job.

You are calling /usr/local/bin/mythicalLibrarian /path/to/recording/29212_221.mpg

$InputFileBasename is the very last portion of that path, it's all after the last "/" in the input path.

You put in your MySQLuser and your MySQLpass when you set up mythTV. the $MySQLMythDb is always mythconverg with mythtv.

This is the code which is failing, it's one of the very first things mythicalLibrarian does.
Code:
#####INITIAL DATASE GRAB#####
##This function grabs title and subtitle from the database for use in the rest of the script
GrabTitleAndSubtitle () {
    if [ "$InputTitle" = "" ]; then

        InputTitle=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select title from recorded where basename like '$InputFileBasename'; "|sed s/"airdate"/""/g|sed -n "2p"`

#get year for movies
        InputSubtitle=`mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select subtitle from recorded where basename like '$InputFileBasename' ; "|sed s/"Subtitle"/""/g|sed -n "2p"`


    fi
    return 0
}
So basically, you call mythicalLibrarian /path/to/file.mpg, it looks up in the mythtvdb what file.mpg is, and then it sets InputTitle and Input Subtitle variables from there for use through the rest of the program. Because it cannot get those variables, it is closing like it does not recognize the file as a mythtv library file and generating an error.

The error suggests that for one reason or another user htpc is not able to run the command mysql.

try this

1. set this up as a user job in mythtv and run it on any file or all files. It will not interfere with any operations in mythtv.
Code:
whoami && mysql -u*USERNAME* -p*PASSWORD* & sleep 2 && killall mysql
2. run it from the terminal as user htpc to verify the results
This should provide all the information required to set things right.

^^^PLEASE DO THIS FIRST SO I CAN GET DIAGNOSTIC INFORMATION^^^^

3. type this to create a link to the command mysql in an easier location.
Code:
sudo ln -s /usr/local/mysql/bin/mysql /usr/bin/mysql

And that should fix your problem.


- kvandesteeg - 2010-06-04

that could be it.

Code:
mini:~ htpc$  whoami && mysql -u*mythtv* -p*[PASSWORD]* & sleep 2 && killall mysql
[1] 68024
htpc
ERROR 1045 (28000): Access denied for user '*mythtv*'@'localhost' (using password: YES)

I tried granting privileges; reseting the password, and still getting access denied. I'll google it some more tomorrow


- outleradam - 2010-06-05

Cool Let me know how you end up fixing it. I'll try to implement something in the Darwin part of mythicalSetup


- outleradam - 2010-06-08

Well, it seems that there are no more bug reports. I am putting this project on the back burner and working on a JAVA Linux solution for OBD-II (car diagnostics). This program will be called OBD-2ner (OBD Tuner).

There is only a slight bit more to be done to the Mac platform to remove some out-of-date messages and release a new stable version. This will occur within the next 2 weeks.

I will continue to support mythicalLibrarian indefinitely. Please submit bug reports or problems with Linux or Mac platforms


- outleradam - 2010-06-13

Let me know if there are any problems currently.


- User 57826 - 2010-06-23

Just a quick question. If I have mythicalibrarian set to link instead of move the files (so i can watch in either MythTV or XBMC; will this prevent me from deleting files from within XMBC?


- outleradam - 2010-06-23

Yes. You should use move mode or disabled mode.

Move mode will allow you to delete files from within XBMC, but mythtv will not be aware of it.

Link mode will allow you to delete the link, but nothing will happen to the actual file, and XBMC will not be aware if the actual file is deleted

Disabled mode will disassociate the file completely with mythtv after processing. Disabled will remove all database entries and pictures which mythtv created, leaving the file orphaned and ready to be viewed and deleted from XBMC.


- outleradam - 2010-06-23

Image

Move=MythTV controls files
Link=MythTV controls original, but nothing controls the symlink, it is orphaned.
Disabled=MythTV does not control the file

it's all about the relationship you want to maintain. I recommend Move with mythtv controlling free space so that it will automatically delete the oldest file when it creates a new file. This way MythTV can work like it normally does and XBMC serves as a front end.


- kvandesteeg - 2010-06-24

Finally got a chance to resolve the issues.
I had to Grant on both % and localhost. I would have expected % to cover localhost.

i also needed to create the symlink

Thanks for the help


- Dobyken - 2010-06-25

outleradam Wrote:Let me know if there are any problems currently.

Adam, can you tell from my log extract why Glee didn't get properly processed? It's not really a problem but it might help you make ML better. I checked tvdb and the episode is properly listed in series 83610. It appears for some reason that the match was not made between the Zap2it data and the TVDB data. I've had a few others in the last few weeks but this is the first one I'm whining about...actually it's the wife. I hate Glee but she wants to watch it. Usually I just manually rename and move the files like this.

###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:90- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/media/WDDrive/TVSeries- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/home/mythtv/Episodes-
USE ORIGINAL DIRBig Grinisabled NOTIFICATIONSBig Grinisabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Glee- LOCAL SHOW NAME TRANSLATION:- ShowName:Glee
DATABASE UPDATED:- TVDB LAST UPDATED:10069999- CURRENT:9999999-
RESOLVED SERIES ID:83610- RESOVED SHOW NAME:Glee-
INPUT EPISODE NAMETonguereggers- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAMETonguereggers-
SEASON:- EPISODE:- SYMLINK MODEBig Grinisabled- FILESIZE: 6977499kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:266299712kB- WORKINGDIR:1 FREE:266299712kB-
MOVEDIRWRITABLE:1- FREE:258002436kB- ALTERNATEMOVEDIR:1- FREE:266299712kB-
PRIMARYSHOWDIRWRITABLE:1- FREE:266299712kB-ALTERNATESHOWDIRWRITABLE:1- FREE:266299712kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:258002436kB- ALTERNATEMOVIEDIR:1- FREE:266299712kB-
DATABASE TYPEConfusedchedulesdirect1-
RECSTART:2010-06-24 19:59:00- MOVIE YEAR:- ORIGINAL SERIES DATE:2009-09-23-
PROGRAMID:EP011413890004- CHANNEL ID:1131- CATEGORY:Musical comedy- GOFORDOOVER:1-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:1141389- MATCHED TVDB SERIES ID:-
PLOT: Kurt tries out for the football team; two members of the glee club receive life-altering news.
####################END OF DEBUG LOG#####################
WARNING:SHOW WAS CONSIDERED AN EPISODE BUT INFORMAITON WAS NOT OBTAINABLE
%%%%%%%%%%%%Please consider helping out and adding to thetvdb%%%%%%%%%%%%
HANDLING EPISODE AS GENERAL SHOWING AND DUMPING INTO --doover USER JOB
CREATING FOLDER: /home/mythtv/Showings/Glee
GENERIC GUIDE DATA WAS SUPPLIED TYPE: Series With Episode Data- Glee, Preggers
WARNING:%%%%%%%%%%PROGRAM GUIDE DATA IS NOT COMPLETE%%%%%%%%%%%%%%%%%%%%
MOVING FILE: /var/lib/mythtv/recordings/1131_20100624195900.mpg to /home/mythtv/Showings/Glee/Glee S0E0 (Preggers Recorded [email protected] on 1131).mpg
@@@@@@@@@@@@@OPERATION COMPLETE Fri Jun 25 07:19:02 EDT 2010 @@@@@@@@@@@@@@@@
REMOVING - 1131_20100624195900.mpg - THUMBNAILS - DATABASE ENTRIES
@@@@@@@@@@@NEW SEARCH INITIATED AT Fri Jun 25 07:27:53 EDT 2010@@@@@@@@@@@@@@
Revision 436 Mon May 10 09:39:37 EDT 2010


- outleradam - 2010-06-26

It's very difficult to read that without code tags.

Code:
###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:90- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian  WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/media/WDDrive/TVSeries- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE  MOVE DIR:/home/mythtv/Episodes-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Glee- LOCAL SHOW NAME TRANSLATION:- ShowName:Glee
DATABASE UPDATED:- TVDB LAST UPDATED:10069999- CURRENT:9999999-
RESOLVED SERIES ID:83610- RESOVED SHOW NAME:Glee-
INPUT EPISODE NAME:Preggers- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE  NAME:Preggers-
SEASON:- EPISODE:- SYMLINK MODE:Disabled- FILESIZE: 6977499kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:266299712kB- WORKINGDIR:1  FREE:266299712kB-
MOVEDIRWRITABLE:1- FREE:258002436kB- ALTERNATEMOVEDIR:1-  FREE:266299712kB-
PRIMARYSHOWDIRWRITABLE:1-  FREE:266299712kB-ALTERNATESHOWDIRWRITABLE:1-  FREE:266299712kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:258002436kB- ALTERNATEMOVIEDIR:1-  FREE:266299712kB-
DATABASE TYPE:schedulesdirect1-
RECSTART:2010-06-24 19:59:00- MOVIE YEAR:- ORIGINAL SERIES  DATE:2009-09-23-
PROGRAMID:EP011413890004- CHANNEL ID:1131- CATEGORY:Musical comedy-  GOFORDOOVER:1-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode  Data- STARS:0 RATING:0
ZAP2IT SERIES ID:1141389- MATCHED TVDB SERIES ID:-
PLOT: Kurt tries out for the football team; two members of the glee club  receive life-altering news.
####################END OF DEBUG LOG#####################
WARNING:SHOW WAS CONSIDERED AN EPISODE BUT INFORMAITON WAS NOT  OBTAINABLE
%%%%%%%%%%%%Please consider helping out and adding to  thetvdb%%%%%%%%%%%%
HANDLING EPISODE AS GENERAL SHOWING AND DUMPING INTO --doover USER JOB
CREATING FOLDER: /home/mythtv/Showings/Glee
GENERIC GUIDE DATA WAS SUPPLIED TYPE: Series With Episode Data- Glee,  Preggers
WARNING:%%%%%%%%%%PROGRAM GUIDE DATA IS NOT COMPLETE%%%%%%%%%%%%%%%%%%%%
MOVING FILE: /var/lib/mythtv/recordings/1131_20100624195900.mpg to  /home/mythtv/Showings/Glee/Glee S0E0 (Preggers Recorded  [email protected] on 1131).mpg
@@@@@@@@@@@@@OPERATION COMPLETE Fri Jun 25 07:19:02 EDT 2010  @@@@@@@@@@@@@@@@
REMOVING - 1131_20100624195900.mpg - THUMBNAILS - DATABASE ENTRIES
@@@@@@@@@@@NEW SEARCH INITIATED AT Fri Jun 25 07:27:53 EDT  2010@@@@@@@@@@@@@@
Revision 436 Mon May 10 09:39:37 EDT 2010

working on it now.


- adammw - 2010-07-08

Tried to install mythicalLibrarian today on our mythtv installation, however testing it with an older recording worked in the wrong way. It found the show name OK, but listed it as an entirely different episode and season.

It seems to be getting episode data "fuzzy logic" out of a random number generator. In MythWeb, it clearly has Episode Number: E4S3 from the syndicatedepisodenumber field of the recordedprogram table. The only other number that makes sense to produce 1x07 could possibly be the Program ID: EP7470632243.

Config file:
Code:
#######################USER SETTINGS##########################
###Stand-alone mode values###
#MoveDir is the folder which mythicalLibrarian will move the file.  No trailing / is accepted eg. ~/videos
MoveDir=/home/shared/mythicalLibrarian/Episodes
#AlternateMoveDir will act as a seccondary MoveDir if the primary MoveDir fails.  No trailing / is accepted eg. ~/videos
AlternateMoveDir=~/.mythicalLibrarian/Episodes
#If UseOriginalDir is Enabled, original dir will override MoveDir.  Useful for multiple recording dirs.
#UseOriginalDir will separate episodes from movies and shows. Enabled|Disabled
UseOriginalDir=Disabled
#When Enabled, mythicalLibrarian will move the file to a folder of the same name as the show. This is not affected by UseOriginalDir. Enabled|Disabled
UseShowNameAsDir=Enabled
#Internet access Timeout in seconds: Default Timeout=50 (seconds)
Timeout=50
#Update database time in secconds, Longer duration means faster processing time and less strain on TheTvDb. Default='70000' (almost a day)
UpdateDatabase=70000
#mythicalLibrarian working file dir: Default=~/.mythicalLibrarian (home/username/mythicalLibraian)
mythicalLibrarian=~/.mythicalLibrarian
#FailSafe mode will enable symlinks to be formed in FailSafeDir if the move or symlink operation fails. Enabled|Disabled
FailSafeMode=Enabled
#FailSafeDir is used when the file cannot be moved to the MoveDir. FailSafe will not create folders. eg. /home/username
FailSafeDir='/home/mythtv/FailSafe'
#DirTracking will check for and remove the folders created by mythicalLibrarian
DirTracking=Enabled
#the following line contains the API key from www.TheTvDb.Com. Default project code: 6DF511BB2A64E0E9
APIkey=6DF511BB2A64E0E9
#Language setting
Language=en
#SYMLINK has 3 modes.  MOVE|LINK|Disabled: Default=MOVE
SYMLINK=LINK

###Database Settings###
#Guide data type
#Database access Enabled|Disabled
Database=Enabled
#Database Type Default=MythTV
DatabaseType=MythTV
#Guide data type
GuideDataType=SchedulesDirect
#MySQL User name: Default=mythtv
MySQLuser=mythtv
#MySQL Password: Default=mythtv
MySQLpass=*******
#MySQL Myth Database: Default=mythconverg
MySQLMythDb=mythconverg
#Primary Movie Dir. mythicalLibrarian will attempt to move to this dir first. No trailing / is accepted eg. '~/videos'
PrimaryMovieDir=/home/shared/mythicalLibrarian/Movies
#AlternateMoveDir will act as a Seccondary move dir if the primary moive dir fails
AlternateMovieDir=~/.mythicalLibrarian/Movies
#ShowStopper = Enabled prevents generic shows and unrecognized episodes from being processed
ShowStopper=Disabled
PrimaryShowDir=/home/shared/mythicalLibrarian/Showings
#AlternateShowDir will act as a Seccondary move dir if the primary Show dir fails
AlternateShowDir=~/.mythicalLibrarian/Showings
#CommercialMarkup will generate comskip files for recordings when they are moved. Enabled|Disabled
CommercialMarkup=Enabled
#CommercialMarkupCleanup will execute a maintenance routine which will remove comskip files if they are not needed
CommercialMarkupCleanup=Enabled
###Reporting/Communications###
#If notifications are enabled, NotifyUserName should be the same as the user logged into the GNOME Session. (your username)
NotifyUserName=
#Notify tells mythicalLibrarian to send a notification to GNOME Desktop upon completion. Enabled|Disabled
#Notify tells mythicalLibrarian to send a notification to GNOME Desktop upon completion. Enabled|Disabled
Notify=Disabled
#If notifications are enabled, NotifyUserName should be the same as the user logged into the GNOME Session. (your username)
NotifyUserName=''
#Ip Address and port for XBMC Notifications Eg.XBMCIPs=( 192.168.1.110:8080 192.168.1.111:8080 XBOX:8080 )
XBMCIPs=''
#Send a notification to XBMC to Update library upon successful move job Enabled|Disabled
XBMCUpdate=Disabled
#Send Notifications to XBMC UI when library is updated Enabled|Disabled
XBMCNotify=Disabled
#Send a notification to XBMC to cleanup the library upon successful move job Enabled|Disabled
XBMCClean=Disabled
#DailyReport provides a local log of shows added to your library per day. Enabled|Disabled
DailyReport=Enabled
#Enables debug mode.  This is a verbose mode of logging which should be used for troubleshooting.  Enabled|Disabled
DEBUGMODE=Enabled
#maxItems controls the number of items in the RSS. RSS Can be activated by creating a folder in /var/www/mythical-rss.
maxItems=8
#########################USER SETTINGS##########################

Debug Log:
Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Fri Jul  9 05:25:19 EST 2010@@@@@@@@@@@@@@
Revision 478 Fri Jul 9 04:58:55 EST 2010
PERFORMING MAINTENANCE ROUTINE
SEARCHING: www.TheTvDb.com SHOW NAME: The Gruen Transfer EPISODE: S3, Ep4 (2010-07-07)
FILE NAME: /var/lib/mythtv/recordings/1022_20100708203000.mpg
SEARCH FOUND:The Gruen Transfer ID#: 82135
USING FUZZY LOGIC FOR EPISODE RECOGNITION Please update TheTvDb.com
SEARCH FOUND:The Gruen Transfer ID#: 82135
DEFINED ABSOLOUTE EPISODE NUMBER: 7
###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/home/shared/mythicalLibrarian/Episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/home/mythtv/.mythicalLibrarian/Episodes-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled DEBUG MODE:Enabled-
INPUT SHOW NAME:The Gruen Transfer- LOCAL SHOW NAME TRANSLATION:- ShowName:The Gruen Transfer
DATABASE UPDATED:- TVDB LAST UPDATED:1278687244- CURRENT:1278617918-
RESOLVED SERIES ID:82135- RESOVED SHOW NAME:The Gruen Transfer-
INPUT EPISODE NAME:S3, Ep4 (2010-07-07)- ABSOLOUTE EPISODE NUMBER:7- RESOLVED EPISODE NAME:Road Safety-
SEASON:S01- EPISODE:E07- SYMLINK MODE:LINK- FILESIZE: 0kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:5063340kB- WORKINGDIR:1 FREE:5063340kB-
MOVEDIRWRITABLE:1- FREE:2999668kB- ALTERNATEMOVEDIR:1- FREE:5063340kB-
PRIMARYSHOWDIRWRITABLE:1-  FREE:2999668kB-ALTERNATESHOWDIRWRITABLE:1- FREE:5063340kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:2999668kB- ALTERNATEMOVIEDIR:1- FREE:5063340kB-
DATABASE TYPE:tv_grab_au-
RECSTART:2010-07-08 20:30:00- MOVIE YEAR:2010- ORIGINAL SERIES DATE:-
PROGRAMID:EP7470632243- CHANNEL ID:1022- CATEGORY:talk show- GOFORDOOVER:1-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:747063- MATCHED TVDB SERIES ID:-
PLOT: Hosted by Wil Anderson, with Todd Sampson and Russel Howcroft, The Gruen Transfer lifts the lid on the business of persuasion. It is fast, furious and funny.
####################END OF DEBUG LOG#####################
Symlink created /home/shared/mythicalLibrarian/Episodes/The Gruen Transfer/Season 1/The Gruen Transfer S01E07 (Road Safety)-1.mpg
@@@@@@@@@@@@@OPERATION COMPLETE Fri Jul 9 05:25:22 EST 2010 @@@@@@@@@@@@@@@@

Also, what is the best way to temporarily disable mythicalLibrarian until I can get it to work?


- GregoryK - 2010-07-09

Here's a tiny nitpicking bug report that I never noticed until yesterday despite having the RSS feed running for months now:

The title of the RSS feed is misspelled with an extra "t": "mythticalLibrarian"

Critical, I know. Laugh

/gkk