XBMC Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for XBMC (/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



- outleradam - 2010-11-03 15:13

ubuntite Wrote:Also, it appears that recordings aren't being deleted and removed from the MythTV database after being moved according to SYMLINK=Disabled... This just recently started happening after upgrading to 750.

I'm at work, so I'm just making notes.
there is an extra test statement at line 2460
Code:
test test "$mythicalEpiShow" != "1" && test "$SYMLINK" = "Disabled" && chmod 775 "$MoveDir/$ShowFileName.$OriginalExt" && DeleteRelevantDatabaseFilesAndPictures

Can you get me a instance of a improperly named file in ~/.mythicalLibrarian/output.log? I will not be able to make any corrections to how mythicalLibrarian obtains data without knowing what it is obtaining. When you see a file that is named improperly, gedit ~/.mythicalLibrarian/output.log and do a search for the improperly named file, or the title. Then post that new search initiated here. This way I can see what decisions mythicalLibrarian made along the way.

Also, I would really like to see the terminal output from MythDataGrabber in mythicalLibrarian to figure out if there is a problem with it.


- ubuntite - 2010-11-03 18:58

outleradam Wrote:I'm at work, so I'm just making notes.
there is an extra test statement at line 2460
Code:
test test "$mythicalEpiShow" != "1" && test "$SYMLINK" = "Disabled" && chmod 775 "$MoveDir/$ShowFileName.$OriginalExt" && DeleteRelevantDatabaseFilesAndPictures

Can you get me a instance of a improperly named file in ~/.mythicalLibrarian/output.log? I will not be able to make any corrections to how mythicalLibrarian obtains data without knowing what it is obtaining. When you see a file that is named improperly, gedit ~/.mythicalLibrarian/output.log and do a search for the improperly named file, or the title. Then post that new search initiated here. This way I can see what decisions mythicalLibrarian made along the way.

Also, I would really like to see the terminal output from MythDataGrabber in mythicalLibrarian to figure out if there is a problem with it.

Code:
###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/media/recordings/Episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/home/mythtv/Episodes-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled
INPUT SHOW NAME:The Colbert Report- LOCAL SHOW NAME TRANSLATION:- ShowName:The Colbert Report
DATABASE UPDATED:- TVDB LAST UPDATED:- CURRENT:-
RESOLVED SERIES ID:- RESOVED SHOW NAME:The Colbert Report-
INPUT EPISODE NAME:- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:-
SEASON:- EPISODE:- SYMLINK MODE:Disabled- FILESIZE: 0kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:39666208kB- WORKINGDIR:1 FREE:39666208kB-
MOVEDIRWRITABLE:1- FREE:13119746108kB- ALTERNATEMOVEDIR:1- FREE:39666208kB-
PRIMARYSHOWDIRWRITABLE:1-  FREE:13119746108kB-ALTERNATESHOWDIRWRITABLE:1- FREE:39666208kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:13119746108kB- ALTERNATEMOVIEDIR:1- FREE:39666208kB-
DATABASE TYPE:schedulesdirect1-
RECSTART:- MOVIE YEAR:- ORIGINAL SERIES DATE:-
PROGRAMID:- CHANNEL ID:- CATEGORY:- GOFORDOOVER:0-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:- STARS: RATING:0
ZAP2IT SERIES ID:- MATCHED TVDB SERIES ID:-
PLOT:
####################END OF DEBUG LOG#####################
WARNING:SHOW WAS CONSIDERED AN EPISODE BUT INFORMAITON WAS NOT OBTAINABLE
%%%%%%%%%%%%Please consider helping out and adding to thetvdb%%%%%%%%%%%%
HANDLING EPISODE AS GENERAL SHOWING AND DUMPING INTO --doover USER JOB
GENERIC GUIDE DATA WAS SUPPLIED TYPE: - The Colbert Report,
WARNING:%%%%%%%%%%PROGRAM GUIDE DATA IS NOT COMPLETE%%%%%%%%%%%%%%%%%%%%
MOVING FILE: '/var/lib/mythtv/recordings/1042_20101103043000.mpg' to '/media/recordings/Showings/The Colbert Report/The Colbert Report S0E0 ( Recorded  on )-1.mpg'
@@@@@@@@@@@@@OPERATION COMPLETE Wed Nov 3 05:02:17 PDT 2010 @@@@@@@@@@@@@@@@

Hope this helps.


- outleradam - 2010-11-03 20:16

Yes, it does help. You are not pulling any information from the database for some reason.

If you are using 0.24, please run this:
Code:
/usr/local/bin/MythDataGrabber --Diagnostic=True
and if that passes, then run
Code:
/usr/local/bin/MythDataGrabber  NAME_OF_MYTHTV_FILE.mpg
I'd like to see the output from those.

If you are using mythtv 0.23, then your username and password are incorrect.

MythTV0.24 requires the use of our MythDataGrabber python script in order to obtain information from the database.

mythicalLibrarian --mythicalDiagnostics should analyze and give tips for any problems with the python bindings.


- outleradam - 2010-11-04 01:50

I fixed 2 problems based on your log.

1. It was finding "" which would get run though showtranslations and pick the first item.
2. additional test statement.

Lets see why you are not getting database information. It should help me to set things up for others.


- ubuntite - 2010-11-04 04:11

outleradam Wrote:I fixed 2 problems based on your log.

1. It was finding "" which would get run though showtranslations and pick the first item.
2. additional test statement.

Lets see why you are not getting database information. It should help me to set things up for others.

BTW, I am using MythTV 0.23.

752 Comments:

Trailing /s properly eliminated.
Properly detecting and reusing previous folder and alternate folder names.
Still putting '=' behind DBHostName (localhost in my case).
Recordings still not being removed from database after move using SYMLINK=Disabled.
Commercial skip files still not being created.


- outleradam - 2010-11-04 04:30

Thank you. I fixed the trailing = sign

Recordings which are not matched properly will not be removed unless they have proper episode matching data. Were the shows in question properly identified, or are they in the --doover que?

For the commercial markups, please give me the output of the following commands
Code:
cat ~/.mythicalLibrarian/markupstart.txt
cat ~/.mythicalLibrarian/markupstop.txt
cat ~/.mythicalLibrarian/markupframes.txt
grep CommercialMarkup /usr/local/bin/mythicalLibrarian
I cannot reproduce that here. Comskip.txt files will only be generated if the MythTV commflag job is enabled. There is no .txt file present in the recordings folders?


- ubuntite - 2010-11-04 05:21

outleradam Wrote:Thank you. I fixed the trailing = sign

Recordings which are not matched properly will not be removed unless they have proper episode matching data. Were the shows in question properly identified, or are they in the --doover que?

For the commercial markups, please give me the output of the following commands
Code:
cat ~/.mythicalLibrarian/markupstart.txt
cat ~/.mythicalLibrarian/markupstop.txt
cat ~/.mythicalLibrarian/markupframes.txt
grep CommercialMarkup /usr/local/bin/mythicalLibrarian
I cannot reproduce that here. Comskip.txt files will only be generated if the MythTV commflag job is enabled. There is no .txt file present in the recordings folders?

--doover que is empty. The files have been deleted from the recordings directory, but they are not being removed from the MythTV database. This used to work properly.

markupstart.txt is empty.
markupstop.txt is empty.
markupframes.txt is empty (except for a header row).

grep CommercialMarkup /usr/local/bin/mythicalLibrarian:
Code:
#CommercialMarkup will generate comskip files for recordings when they are moved. Enabled|Disabled
CommercialMarkup=Enabled
#CommercialMarkupCleanup will execute a maintenance routine which will remove comskip files if they are not needed
CommercialMarkupCleanup=Enabled
            CommercialMarkup="Created"
        if [ "$CommercialMarkup" = "Enabled" ]; then    
    if [ "$CommercialMarkup" = "Enabled" ]; then
    if [ "$CommercialMarkupCleanup" = "Enabled" -a -f "$mythicalLibrarian/created.tracking" ]; then
            if [ "$CommercialMarkup" = "Created" ]; then
         if [ "$CommercialMarkup" = "Created" ]; then



- outleradam - 2010-11-04 05:54

Your markups and headered file says that the last time you ran it, there was no comskip data to work with, or mythtv could not access the file. Make sure the permissions are set properly
Code:
sudo chown -R mythtv:mythtv /home/mythtv/.mythicalLibrarian
sudo chmod -R 666 /home/mythtv/.mythicalLibrarian
sudo chmod -R 600 /home/mythtv/.mythicalLibrarian/mythicalSetup


ok, so lets verify you have data to work with. We are going into the raw mysql commands which grab data from mythtv 0.23 and below.

replace -umythtv -pmythtv with your -uUSERNAME and your -pPASSWORD
Code:
mysql -umythtv -pmythtv -e 'use mythconverg; select chanid, starttime from recordedmarkup where type = "4";'
will return something like this
Code:
chanid             starttime
------------------------------------
|   1956 | 2010-11-02 23:30:00 |
|   1956 | 2010-11-02 23:30:00 |
|   1956 | 2010-11-02 23:30:00 |
+--------+---------------------+
Then, pick one set of chanid/starttime

ok, so we have our startime and chanid. We need to find out what file they go to.

replace 1956 with one of your chanids and 2010-11-02 23:30:00 with one of your starttimes
Code:
mysql -umythtv -pmythtv -e 'use mythconverg; select basename from recorded where chanid like "1956" and starttime like "2010-11-02 23:30:00"'
+-------------------------+
| basename                |
+-------------------------+
| 1956_20101102233000.mpg |
+-------------------------+

This will identify a show in your database which for sure has comskip data to work with.

Code:
mythicalLibrarian /path_to_my_videos/1956_20101102233000.mpg

That file should have no problem generating a comskip.

If there is a problem, I need to see the markupstart, markupstop and the markupframes file.


- ubuntite - 2010-11-04 07:12

OK, here you go:

Code:
mythicalLibrarian /var/lib/mythtv/recordings/1018_20101103210000.mpg
Revision 752 Wed Nov 3 18:55:38 PDT 2010
Accessing MythTV DataBase:
RECSTART: DATE:
PROGRAMID: ShowCategory:
PLOT:
JOB: mythicalLibrarian '/var/lib/mythtv/recordings/1018_20101103210000.mpg' 'Larry King Live' ''
WARNING: EPISODE INFORMATION COULD NOT BE OBTAINED
HANDLING AS A GENERAL SHOWING
Please consider helping out and adding to thetvdb.
TARGET SET:/media/recordings/Showings/Larry King Live/Larry King Live S0E0 ( Recorded  on )
FILE NAME EXISTS.  FILE WILL BE KNOWN AS: Larry King Live S0E0 ( Recorded  on )-10
/usr/local/bin/mythicalLibrarian: line 921: [: too many arguments
mv: preserving times for `/media/recordings/Showings/Larry King Live/Larry King Live S0E0 ( Recorded  on )-10.mpg': Operation not permitted
mv: preserving permissions for `/media/recordings/Showings/Larry King Live/Larry King Live S0E0 ( Recorded  on )-10.mpg': Operation not permitted
MOVED: /var/lib/mythtv/recordings/1018_20101103210000.mpg
TO: /media/recordings/Showings/Larry King Live/Larry King Live S0E0 ( Recorded  on )-10.mpg
chmod: changing permissions of `/media/recordings/Showings/Larry King Live/Larry King Live S0E0 ( Recorded  on )-10.mpg': Operation not permitted
SENDING REQUESTED COMMANDS TO:192.168.0.100:8080
/usr/local/bin/mythicalLibrarian: line 542: test: too many arguments
/usr/local/bin/mythicalLibrarian: line 543: test: too many arguments
RSS ENTRY SUCCESSFULLY CREATED: /var/www/mythical-rss/rss.xml
SUCCESSFUL COMPLETEION TYPE: MoveModeSuccessful

Some additional information... my recordings are being moved to a samba share. I don't know if that will be relevant or not.


- outleradam - 2010-11-05 02:21

There's some wierd stuff going on there. Are you getting any errors at all during mythicalSetup? When you run mythicalLibrarian --update, can you post the entire diagnostic?

This may be unrelated, but I'm removing all test statements with || in it starting now.


Code:
adam@adam-desktop:~$ $a=1
0=1: command not found
adam@adam-desktop:~$ echo $?
127
adam@adam-desktop:~$ a=0
adam@adam-desktop:~$ b=1
adam@adam-desktop:~$ c=1
adam@adam-desktop:~$ d=1
adam@adam-desktop:~$ test $a = 1 || test $b = 0 &&  test $c = 0  && test $d = 0 && echo "why is this true?"
adam@adam-desktop:~$ test $a=1 || test $b=0 &&  test $c=0 && test $d=0 && echo "why is this true?"
why is this true?
adam@adam-desktop:~$
I cannot explain this... But I know that there may be a problem with the test command in Maverick.