Kodi Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



- outleradam - 2010-11-01

Thank you for your input Ubuntite. I will make the necessary corrections.

--doMaintenance will delete all mythicalLibrarian database folders which it has created. --undo will undo all actions mythicalLibrarian took on your filesystem. Then you --scan your video folder again, and mythicalLibrarian will rescan your library.

Keep in mind that if the information was not in theTvDb at the time, or you were offline during an operation, mythicalLibrarian may put your file in the --doover option. It knows when there should be information on theTvDb, but it was not able to find it.


- ubuntite - 2010-11-01

outleradam Wrote:Thank you for your input Ubuntite. I will make the necessary corrections.

--doMaintenance will delete all mythicalLibrarian database folders which it has created. --undo will undo all actions mythicalLibrarian took on your filesystem. Then you --scan your video folder again, and mythicalLibrarian will rescan your library.

Keep in mind that if the information was not in theTvDb at the time, or you were offline during an operation, mythicalLibrarian may put your file in the --doover option. It knows when there should be information on theTvDb, but it was not able to find it.

--doMaintenance doesn't seem to work as advertised. I assume the database folders you are referring to are the show-named folders. --doMaintenance does not delete them on my system.

I am sorry, but I am still not sure what --scan does. Is --scan equivalent to the MythTV user job 'mythicalLibrarian'? If so, if SYMLINK=Disabled, assume there is no way (and no point) in re-'scan'ing file.

Is the 'video folder' you are referring to the folder which MythTV uses for recordings?


- outleradam - 2010-11-02

you must sudo su mythtv first, if the database was created by mythtv.
Code:
adam@adam-desktop:~/.mythicalLibrarian$ mythicalLibrarian --doMaintenance
--You have entered Maintenance mode.
mythicalLibrarian will perform maintenance on it's records.
Please ensure that All network attached storage and removable
storage drives are connected and mounted properly before continuing
Would you like mythicalLibrarian to conduct a scan? y/(n)>y
REMOVING ORPHAN FOLDER:/home/adam/newfi/Mister Rogers Neighborhood
REMOVING ORPHAN FOLDER:/home/adam/.mythicalLibrarian/mythicalSetup/Episodes/Mister Rogers Neighborhood/Season 7
Removing mythicalLibrarian database from:/home/adam/.mythicalLibrarian/Battlestar Galactica
Removing mythicalLibrarian database from:/home/adam/.mythicalLibrarian/Battlestar Galactica (2003)
Removing mythicalLibrarian database from:/home/adam/.mythicalLibrarian/DailyReport
Removing mythicalLibrarian database from:/home/adam/.mythicalLibrarian/Mister Rogers' Neighborhood
Removing mythicalLibrarian database from:/home/adam/.mythicalLibrarian/South Park
done.

Yes, symlink=disabled would not work for rescanning.


Yes, that is the video folder. Let me make sure that you can use --doover with SYMLINK=Disabled. Without high confidence, mythicalLibrarian should never modify the database.


- outleradam - 2010-11-02

SYMLINK=Disabled was logging things improperly. Please delete any doover.sh doover.sh.old, undo.sh or undo.sh.old files which are in your ~/mythicalLibrarian folder. Then update to the SVN.

I corrected all problems which were mentioned in your first post in SVN revisions 742 - 749. 750 addressed symlink=disabled issues. you will no longer generate an undo and you will generate doovers only for those files which mythicalLibrarian says should be --doover'd later.

Please test again and let me know about the python bindings issue. 0.24 has switched to python bindings because very soon, it will be impossible to perform a direct database search of MythTV. The MythTV devs will be changing their database significantly.

The python bindings abstract the database. From now on, MythTV Developers will support the connection to extract information rather then me worrying about having to rewrite my database extractor every time MythTV's version changes. I just have to support our MythDataGrabber python script, and that's the end of the worrying.

I would like to see mythicalDiagnostics final output regarding python bindings.


- ubuntite - 2010-11-02

outleradam Wrote:you must sudo su mythtv first, if the database was created by mythtv.

Is that true for ANY mythicalLibrarian command I run, i.e.:

--scan
--doMaintenance
--update
--doover
--undo

I always run update as 'sudo mythicalLibrarian --update'. Should I 'sudo su mythtv' before updating, or does that only apply to modes that touch the mythtv database? If so, I assume those would only be --scan & --doover. Am I correct? It seems a bit confusing. Maybe some additional help on the wiki or in the --help file would be a good idea.


- outleradam - 2010-11-02

Generally, it's always best to run mythicalLibrarian as a single user at all times. In your case it would not matter much, as there does not need to be consistancy between MythTV and the generated files.

For --scan, it does not matter except that the showing will be created by one user and the accessory files and folders(folders, .NFOs and comskips) are created by another. That means the accessory files and folders will not be removed automatically until the user who created the files runs mythicalLibrarian again. It just throws off/delays file and folder tracking a bit.

for --doMaintenance, it must be run per user
I believe I've removed all issues with --update on multiple users. That is not an issue.
For --doover, it must be run per user
For --undo, it must be run per user

I can add some stars in the help files to denote the user dependent commands.

I've played around with the idea of a unified database and tracking. However, it would not work right for several reasons.

1. The log must be written every time by every user, it could be chmod'd 777
2. The folder.tracking and created.tracking would need to be chmod'd 777
3. Each created folder and each created file per database would need to be chmod'd 777
4. every comskip and MythTV recording would need to be chmod'd 777
5. If the user was to terminate early, or an error was experienced before the files were chmod'd mythicalLibrarian would cease to function for the next user, which would probly be mythtv.

There's more too.

I will update the help tonight. The wiki will require a rewrite on several sections within the next week.

Can you show me what the problem is with the python bindings? I would really like to make sure things are working as they should.


- ubuntite - 2010-11-02

outleradam Wrote:SYMLINK=Disabled was logging things improperly. Please delete any doover.sh doover.sh.old, undo.sh or undo.sh.old files which are in your ~/mythicalLibrarian folder. Then update to the SVN.

I corrected all problems which were mentioned in your first post in SVN revisions 742 - 749. 750 addressed symlink=disabled issues. you will no longer generate an undo and you will generate doovers only for those files which mythicalLibrarian says should be --doover'd later.

Please test again and let me know about the python bindings issue. 0.24 has switched to python bindings because very soon, it will be impossible to perform a direct database search of MythTV. The MythTV devs will be changing their database significantly.

The python bindings abstract the database. From now on, MythTV Developers will support the connection to extract information rather then me worrying about having to rewrite my database extractor every time MythTV's version changes. I just have to support our MythDataGrabber python script, and that's the end of the worrying.

I would like to see mythicalDiagnostics final output regarding python bindings.

I just updated to SVN 750...

1. Trailing / still present in MoveDir
2. Extra equal sign at end of default DBHostName (i.e. 'DBHostName=localhost=')
3. AlternateShowDir defaults to PrimaryShowDir as opposed to ./Showings like Movies and Episodes
4. Trailing /s present in PrimaryShowDir and AlternateShowDir


- ubuntite - 2010-11-02

Another thing I am noticing is that before, when an Episode couldn't be identified by thetvdb.com, the file would be renamed to include the date it was recorded. Recently, that seems to be broken. Now Showings are named:

ShowName S0E0 ( Recorded on ).mpg

Also, the commercial skip file is not being created anymore...


- outleradam - 2010-11-02

I will check out the showings naming convention.

I am not reproduce any of the other issues. Did you try to remove it in mythicalSetup? Are those trailing /'s showing up in mythicalSetup? If they are there, then you should remove them. It sounds like you may not have database connectivity for the other issues.

As for the trailing /'s, mythicalSetup stores the last value used in the ~/.mythicalLibrarian/mythicalSetup folder. Whatever is put in the dialog box in mythicalSetup is what will be put into the built program.

Check your ~/.mythtv/mysql.txt file. The database defaults eg.. "Localhost=" is pulled from your ~/.mythtv/mysql.txt file, and dropped directly into the variable.

Run mythicalLibrarian --update twice, once to set the values and a second time to verify the proper things were put in the boxes.

I will add something which truncates any trailing /'s from mythicalSetup.


- ubuntite - 2010-11-03

Also, it appears that recordings aren't being deleted and removed from the MythTV database after being moved according to SYMLINK=Disabled... This just recently started happening after upgrading to 750.


- outleradam - 2010-11-03

ubuntite Wrote:Also, it appears that recordings aren't being deleted and removed from the MythTV database after being moved according to SYMLINK=Disabled... This just recently started happening after upgrading to 750.

I'm at work, so I'm just making notes.
there is an extra test statement at line 2460
Code:
test test "$mythicalEpiShow" != "1" && test "$SYMLINK" = "Disabled" && chmod 775 "$MoveDir/$ShowFileName.$OriginalExt" && DeleteRelevantDatabaseFilesAndPictures

Can you get me a instance of a improperly named file in ~/.mythicalLibrarian/output.log? I will not be able to make any corrections to how mythicalLibrarian obtains data without knowing what it is obtaining. When you see a file that is named improperly, gedit ~/.mythicalLibrarian/output.log and do a search for the improperly named file, or the title. Then post that new search initiated here. This way I can see what decisions mythicalLibrarian made along the way.

Also, I would really like to see the terminal output from MythDataGrabber in mythicalLibrarian to figure out if there is a problem with it.


- ubuntite - 2010-11-03

outleradam Wrote:I'm at work, so I'm just making notes.
there is an extra test statement at line 2460
Code:
test test "$mythicalEpiShow" != "1" && test "$SYMLINK" = "Disabled" && chmod 775 "$MoveDir/$ShowFileName.$OriginalExt" && DeleteRelevantDatabaseFilesAndPictures

Can you get me a instance of a improperly named file in ~/.mythicalLibrarian/output.log? I will not be able to make any corrections to how mythicalLibrarian obtains data without knowing what it is obtaining. When you see a file that is named improperly, gedit ~/.mythicalLibrarian/output.log and do a search for the improperly named file, or the title. Then post that new search initiated here. This way I can see what decisions mythicalLibrarian made along the way.

Also, I would really like to see the terminal output from MythDataGrabber in mythicalLibrarian to figure out if there is a problem with it.

Code:
###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/media/recordings/Episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/home/mythtv/Episodes-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled
INPUT SHOW NAME:The Colbert Report- LOCAL SHOW NAME TRANSLATION:- ShowName:The Colbert Report
DATABASE UPDATED:- TVDB LAST UPDATED:- CURRENT:-
RESOLVED SERIES ID:- RESOVED SHOW NAME:The Colbert Report-
INPUT EPISODE NAME:- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:-
SEASON:- EPISODE:- SYMLINK MODE:Disabled- FILESIZE: 0kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:39666208kB- WORKINGDIR:1 FREE:39666208kB-
MOVEDIRWRITABLE:1- FREE:13119746108kB- ALTERNATEMOVEDIR:1- FREE:39666208kB-
PRIMARYSHOWDIRWRITABLE:1-  FREE:13119746108kB-ALTERNATESHOWDIRWRITABLE:1- FREE:39666208kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:13119746108kB- ALTERNATEMOVIEDIR:1- FREE:39666208kB-
DATABASE TYPE:schedulesdirect1-
RECSTART:- MOVIE YEAR:- ORIGINAL SERIES DATE:-
PROGRAMID:- CHANNEL ID:- CATEGORY:- GOFORDOOVER:0-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:- STARS: RATING:0
ZAP2IT SERIES ID:- MATCHED TVDB SERIES ID:-
PLOT:
####################END OF DEBUG LOG#####################
WARNING:SHOW WAS CONSIDERED AN EPISODE BUT INFORMAITON WAS NOT OBTAINABLE
%%%%%%%%%%%%Please consider helping out and adding to thetvdb%%%%%%%%%%%%
HANDLING EPISODE AS GENERAL SHOWING AND DUMPING INTO --doover USER JOB
GENERIC GUIDE DATA WAS SUPPLIED TYPE: - The Colbert Report,
WARNING:%%%%%%%%%%PROGRAM GUIDE DATA IS NOT COMPLETE%%%%%%%%%%%%%%%%%%%%
MOVING FILE: '/var/lib/mythtv/recordings/1042_20101103043000.mpg' to '/media/recordings/Showings/The Colbert Report/The Colbert Report S0E0 ( Recorded  on )-1.mpg'
@@@@@@@@@@@@@OPERATION COMPLETE Wed Nov 3 05:02:17 PDT 2010 @@@@@@@@@@@@@@@@

Hope this helps.


- outleradam - 2010-11-03

Yes, it does help. You are not pulling any information from the database for some reason.

If you are using 0.24, please run this:
Code:
/usr/local/bin/MythDataGrabber --Diagnostic=True
and if that passes, then run
Code:
/usr/local/bin/MythDataGrabber  NAME_OF_MYTHTV_FILE.mpg
I'd like to see the output from those.

If you are using mythtv 0.23, then your username and password are incorrect.

MythTV0.24 requires the use of our MythDataGrabber python script in order to obtain information from the database.

mythicalLibrarian --mythicalDiagnostics should analyze and give tips for any problems with the python bindings.


- outleradam - 2010-11-04

I fixed 2 problems based on your log.

1. It was finding "" which would get run though showtranslations and pick the first item.
2. additional test statement.

Lets see why you are not getting database information. It should help me to set things up for others.


- ubuntite - 2010-11-04

outleradam Wrote:I fixed 2 problems based on your log.

1. It was finding "" which would get run though showtranslations and pick the first item.
2. additional test statement.

Lets see why you are not getting database information. It should help me to set things up for others.

BTW, I am using MythTV 0.23.

752 Comments:

Trailing /s properly eliminated.
Properly detecting and reusing previous folder and alternate folder names.
Still putting '=' behind DBHostName (localhost in my case).
Recordings still not being removed from database after move using SYMLINK=Disabled.
Commercial skip files still not being created.