Kodi Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



- watson540 - 2010-01-31

also, i thought deleting files from the xbox was part of the built in myth:// protocol..

Code:
2010-01-30 22:20:48.288 ERROR when trying to delete file: /GetPlaybackURL/UNABLE/TO/FIND/LOCAL/FILE/ON/myth/1032_20100128000000.mpg. File doesn't exist.  Database metadata will not be removed.              
2010-01-30 22:20:48.341 ProgramInfo, Error: GetPlaybackURL: '1048_20100128013100.mpg' should be local, but it can not be found.                                                                                
2010-01-30 22:20:48.400 ERROR when trying to delete file: /GetPlaybackURL/UNABLE/TO/FIND/LOCAL/FILE/ON/myth/1048_20100128013100.mpg. File doesn't exist.  Database metadata will not be removed.              
2010-01-30 22:20:48.473 ProgramInfo, Error: GetPlaybackURL: '1048_20100128020100.mpg' should be local, but it can not be found.                                                                                
2010-01-30 22:20:48.476 ERROR when trying to delete file: /GetPlaybackURL/UNABLE/TO/FIND/LOCAL/FILE/ON/myth/1048_20100128020100.mpg. File doesn't exist.  Database metadata will not be removed.              
2010-01-30 22:20:48.488 ProgramInfo, Error: GetPlaybackURL: '1048_20100128023100.mpg' should be local, but it can not be found.                                                                                
2010-01-30 22:20:48.495 ERROR when trying to delete file: /GetPlaybackURL/UNABLE/TO/FIND/LOCAL/FILE/ON/myth/1048_20100128023100.mpg. File doesn't exist.  Database metadata will not be removed.              
2010-01-30 22:20:48.544 ProgramInfo, Error: GetPlaybackURL: '1048_20100128030100.mpg' should be local, but it can not be found.                                                                                
2010-01-30 22:20:48.546 ERROR when trying to delete file: /GetPlaybackURL/UNABLE/TO/FIND/LOCAL/FILE/ON/myth/1048_20100128030100.mpg. File doesn't exist.  Database metadata will not be removed.



- outleradam - 2010-01-31

You've got to enable follow symlinks when deleting. see the installation instructions for mythtv, or the installation instructions for mythicalLibrarian. This is not a mythicalLibrarian issue. MythicalLibrarian bridges a gap, and does not create additional handling for other programs. If the symlink is there and valid, then MythTV owns the file.

I am not a Team XBMC developer. Things get changed alot in XBMC. Arnova is the only active dev for xbox. Maybe you could talk to him, or pick up a cheap computer and run LIVE. Comskip data is not an issue with mythicalLibrarian. I have had some problems with comskip removal though. It sounds like an XBOX speciffic issue.


- tim- - 2010-01-31

I am trying to gather the info for MC2XML... having some weird issues

http://pastebin.com/mdf1de5d

it never even lists the filename


- barney_1 - 2010-01-31

Any idea how I can have mythicalLibrarian delete the recording from the MythTV database after it moves the file?

I'd like to go completely away from using the Myth frontend and manually delete files from XBMC after I watch them.


- outleradam - 2010-01-31

tim- Wrote:I am trying to gather the info for MC2XML... having some weird issues

http://pastebin.com/mdf1de5d

it never even lists the filename
It would appear that it won't give you the file name without database processing. Reenable that to get the file name. Change GuideDataType=SchedulesDirect, then rerun the mythicalLibrarian job Sorry, I didn't realize it would do that without proper GuideDataType, I've never run with anything except SchedulesDirect data.

barney_1 Wrote:Any idea how I can have mythicalLibrarian delete the recording from the MythTV database after it moves the file?

I'd like to go completely away from using the Myth frontend and manually delete files from XBMC after I watch them.
That would be outside the scope of mythicalLibrarian's field, which is to preserve the file handling. You may want to check on the mythtv forums for that. It sounds like you need a separate user job. You can disable SYMLINK mode and that will remove the ability to control the file from mythtv.

Actually, on seccond thought you could do something like this with a one-liner as a user job to be run after mythicalLibrarian.

Code:
test -L "%DIR%/%FILE%" && mysql -uUSERNAME -pPASSWORD "use 'mythconverg' ; DELETE * from recorded where basename like %FILE%; "; *DO THE SAME FOR EACH OF THE DATABASES*; test -L "%DIR%/%FILE%" && rm "%DIR%/%FILE%"



- barney_1 - 2010-01-31

Thanks, that line of code is exactly the type of thing I was looking for. But, I've also thought of another solution and will share it if anyone else is looking to do the same thing.

So, I've got mythicalLibrarian in MOVE mode so that it moves the files then symlinks to them from their original location. Then I DO NOT allow mythtv to follow the Symlinks. That way when I delete a file from MythTV it just removes the link and the database info but leaves the file. This way I can delete the files from XBMC and it has no effect on MythTV.

To get MythTV to automatically remove recordings I could use the code Outleradam gave me, but instead, I'm setting each recording rule up to record 3 episodes, expiring the old and recording the new. This way there will always be three of each episode recored. But I have the mythicalLibrarian user job run for each episode as soon as it's done recording. This means I have the MythTV data for the last three episodes recorded in case mythicalLibrarian fails but I don't ever need to manually delete recordings.

I hope that makes sense. Once again, great script, thanks!


- watson540 - 2010-02-01

outleradam Wrote:You've got to enable follow symlinks when deleting. see the installation instructions for mythtv, or the installation instructions for mythicalLibrarian. This is not a mythicalLibrarian issue. MythicalLibrarian bridges a gap, and does not create additional handling for other programs. If the symlink is there and valid, then MythTV owns the file.

I am not a Team XBMC developer. Things get changed alot in XBMC. Arnova is the only active dev for xbox. Maybe you could talk to him, or pick up a cheap computer and run LIVE. Comskip data is not an issue with mythicalLibrarian. I have had some problems with comskip removal though. It sounds like an XBOX speciffic issue.

it is an xbox specific issue and i apologize. i guess it just seems like there are only certain places active on here..and the threads i start arent one of them. I do have an older laptop im considering donating to the cause..I just like the convenience and low power of the xbox

about the not deleting file..i had mythtv set to follow symlinks, but with the script in move mode, which is how i had it it seems something is awry but i cant quite explain it..im trying it in link mode now, hoping it will work ..cause in link mode mythtv will actually have a symlink to follow..not so in move mode..correct??

I just wanted to reply to report an error in the logic maybe??

I keep getting madhouse the tv series show up as a movie even though the episode info is there..
Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Sun Jan 31 23:01:42 EST 2010@@@@@@@@@@@@@                                                                                                                                  
PERFORMING MAINTENANCE ROUTINE                                                                                                                                                                                
SEARCHING: www.TheTvDb.com SHOW NAME: Madhouse EPISODE: Brother Against Brother                                                                                                                                
FILE NAME: /mnt/ntfs/myth/1040_20100131220000.mpg                                                                                                                                                              
SEARCH FOUND:Madhouse ID#: 138011                                                                                                                                                                              
SEARCH FOUND:Madhouse ID#: 138011
DEFINED ABSOLOUTE EPISODE NUMBER:
###################DEBUG MODE ENABLED####################
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/mythicalLibrarian-
MOVE DIR:/mnt/ntfs/tv-episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Disabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/mnt/extra/tv-episodes-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Enabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Madhouse- LOCAL SHOW NAME TRANSLATION:- ShowName:Madhouse
SENT TVDB SHOW NAME:Madhouse-
RESOLVED SERIES ID:138011- RESOVED SHOW NAME:Madhouse-
INPUT EPISODE NAME:Brother Against Brother- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:Brother Against Brother-
SEASON:- EPISODE:- SYMLINK MODE:MOVE- FILESIZE: 1180674'kB'
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:86409668kB- WORKINGDIR:1 Free:1815544kB-
MOVEDIR:1- FREE:86409668kB- ALTERNATEMOVEDIR:1- FREE:23746604kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:23746604kB- ALTERNATEMOVIEDIR:1- FREE:86409668kB-
DATABASE INFORMATION
RECSTART:2010-01-31 22:00:00- MOVIE YEAR:2010- ORIGINAL SERIES DATE:-
PROGRAMID:EP012123690004- CHANNEL ID:1040- CATEGORY:Episodic-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:128936357- MATCHED TVDB SERIES ID:-
####################END OF DEBUG LOG#####################



- outleradam - 2010-02-01

Upgrade to the latest version of mythicalLibrarian, it will stop the movie recognitions of shows on certain guide data types. Right now I only support SchedulesDirect/Zap2it/tivo database. I'm expecting some info from tim- for MC2XML data. In the mean time, all you can do is upgrade and fail out those recordings.

I see that the IDs are almost the same as Zap2it ids, so it may be possible to work with it by just removing the EP and SH identifiers from the shows and just work with the numbers of the episodes to do the matching. We'll see. I think it's promissing though.


- watson540 - 2010-02-01

thanks..also..im unclear on one other thing

when I add the folders to the xbmc library by setting the content..do I add the folders "tv episodes"" and "movies" under the mythtv source, ot do I add the real folders (over xbmp for me) such a s/mnt/ntfs/tv-episodes?? seems to only work right with the latter


- watson540 - 2010-02-01

issues ..nothings changed..but failsafe mode keeeps activating with new version..presumable cause i keep them on an ntfs drive which i cannot change p[ermissions for

Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Mon Feb  1 04:41:06 EST 2010@@@@@@@@@@@@@
%%%NO DATABASE INFORMATION. CHECK LOGIN/PASS OR FILE %%%%%
SEARCHING: www.TheTvDb.com SHOW NAME: Madhouse EPISODE: Full Moon Madness
FILE NAME: /mnt/ntfs/myth/1040_20100131190000.mpg
SEARCH FOUND:Madhouse ID#: 138011
USING FUZZY LOGIC FOR EPISODE RECOGNITION
No direct Zap2it match found. Please update TheTvDb.com
SEARCH FOUND:Madhouse ID#: 138011
DEFINED ABSOLOUTE EPISODE NUMBER:
FILE IS A LINK ACTIVATING SYMLINK LINK MODE
###################DEBUG MODE ENABLED####################
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/mythicalLibrarian-
MOVE DIR:/mnt/ntfs/tv-episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Disabled- FAILSAFE DIR:/home/mythtv- ALTERNATE MOVE DIR:/mnt/extra/tv-episodes-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Madhouse- LOCAL SHOW NAME TRANSLATION:- ShowName:Madhouse
SENT TVDB SHOW NAME:Madhouse-
RESOLVED SERIES ID:138011- RESOVED SHOW NAME:Madhouse-
INPUT EPISODE NAME:Full Moon Madness- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:Full Moon Madness-
SEASON:- EPISODE:- SYMLINK MODE:LINK- FILESIZE: 0'kB'
CREATE AND DELETE FLAGS: ORIGINALDIR:0- FREE:85794292kB- WORKINGDIR:0 Free:1821868kB-
MOVEDIR:0- FREE:85794292kB- ALTERNATEMOVEDIR:0- FREE:22564784kB-
PRIMARYMOVIEDIRWRITABLE:0- FREE:85794292kB- ALTERNATEMOVIEDIR:0- FREE:22564784kB-
DATABASE INFORMATION
RECSTART:2010-01-31 19:00:00- MOVIE YEAR:2010- ORIGINAL SERIES DATE:2010-01-24-
PROGRAMID:EP012123690003- CHANNEL ID:1040- CATEGORY:Episodic-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:128936357- MATCHED TVDB SERIES ID:-
####################END OF DEBUG LOG#####################
FAILSAFE FLAG WAS SET CHECK PERMISSIONS AND FOLDERS



- watson540 - 2010-02-01

nm i give up for real..time to find something that works but thanks for your hard work..mebbe in a year ill come back to deleting all of my recordings a nightly basis just to see pretty album art

i mean..what is this?!
Code:
mythtv@myth ~/mythicalLibrarian $ ./mythicalLibrarian.sh "SpongeBob SquarePants" "Sand Castles in the Sand" /mnt/ntfs/myth/1054_20100131200000.mpg
Accessing MythTV DataBase:
COMMERCIAL DATA START:0 STOP:1940
COMMERCIAL DATA START:21586 STOP:28662
COMMERCIAL DATA START:48297 STOP:53757
0
mythtv@myth ~/mythicalLibrarian $ cat output.log
@@@@@@@@@@@NEW SEARCH INITIATED AT Mon Feb  1 06:48:32 EST 2010@@@@@@@@@@@@@
%%%NO DATABASE INFORMATION. CHECK LOGIN/PASS OR FILE %%%%%
GENERIC GUIDE DATA WAS SUPPLIED TYPE: Generic Episode With No Data- ,
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%PROGRAM GUIDE DATA IS NOT COMPLETE%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%OPERATION FAILED Mon Feb 1 06:48:32 EST 2010 %%%%%%%%%%%%%%%%%
not knocking your hard work man but wtf..it obviously accessed the DB if it got the COMMERCIAL INFO..does that now no matter what version of the script i use..the new one i got today or the old one that DID work and now doesnt


- outleradam - 2010-02-01

mythicalLibrarian does not process mc2xml? It's looking for channelID for the information. It's not getting proper database information. I'm waiting for information to process mc2xml. currently mythicalLibrarian handles SchedulesDirect. If you would like, you can go back a few pages and get the information for me so that I can get the ball rolling on it.

You can change permissions by editing the /etc/fstab file for mounting and/or adding the mythtv user to your user group. I should add this to the FAQ. BTW.. check the installation page and look down at the bottom to fix most common issues.

Do a bit of searching on how to fix your problems. As far as I have seen, mythicalLibrarian, configured correctly, with mythtv, using SchedulesDirect data handles 100% of movies and 99.7% of episodes, adds comskips, and manages it's files well.

All I'm doing until I get the required data to support additional guide data types is maintaining. Until a new datatype comes up and I can begin a new development cycle, this program is in maintenance phase and there's not much to do with mythtv under SchedulesDirect data.

Here's a visualization:
Image
SchedulesDirect is in maintenance phase
ALL OTHERS are in Requirements phase

This is what I need for new data types http://forum.xbmc.org/showpost.php?p=496721&postcount=90


2 things I need to add to the log is data provider and user running mythicalLibrarian


- outleradam - 2010-02-01

tim- could you rerun the jobs please? I would like to get that info from you.


- watson540 - 2010-02-02

yeah my permissions havent changed..i know how to setup fstab and its not been changed for over a year. i didnt start getting permissions errors till ii upgraded your sscript

i would give you the info but you said you had it taken care of

oh yeah..if you know how to set permissions on an ntfs drive im all ears (hint: its not possible)


- watson540 - 2010-02-02

and oh h gee sorry for trting to report a bug it must be my dumb ass thats been runnning gentoo for 8 years now..thats it.,..i just dont know how to use linux
Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Mon Feb  1 19:00:13 EST 2010@@@@@@@@@@@@@
%%%NO DATABASE INFORMATION. CHECK LOGIN/PASS OR FILE %%%%%
SEARCHING: www.TheTvDb.com SHOW NAME: Chowder EPISODE: The Thousand Pound Cake
FILE NAME: /mnt/ntfs/myth/1053_20100201183000.mpg
SEARCH FOUND:Chowder ID#: 81500
USING FUZZY LOGIC FOR EPISODE RECOGNITION
No direct Zap2it match found. Please update TheTvDb.com
SEARCH FOUND:Chowder ID#: 81500
DEFINED ABSOLOUTE EPISODE NUMBER: 15
###################DEBUG MODE ENABLED####################
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/mythicalLibrarian-
MOVE DIR:/mnt/ntfs/tv- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv- ALTERNATE MOVE DIR:/mnt/extra/tv-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Chowder- LOCAL SHOW NAME TRANSLATION:- ShowName:Chowder
SENT TVDB SHOW NAME:Chowder-
RESOLVED SERIES ID:81500- RESOVED SHOW NAME:Chowder-
INPUT EPISODE NAME:The Thousand Pound Cake; The Rat Sandwich- ABSOLOUTE EPISODE NUMBER:15- RESOLVED EPISODE NAME:The Thousand Pound Cake  The Rat Sandwich-
SEASON:S01- EPISODE:E15- SYMLINK MODE:LINK- FILESIZE: 584386'kB'
CREATE AND DELETE FLAGS: ORIGINALDIR:0- FREE:95568904kB- WORKINGDIR:0 Free:1819352kB-
MOVEDIR:0- FREE:95568904kB- ALTERNATEMOVEDIR:0- FREE:28225008kB-
PRIMARYMOVIEDIRWRITABLE:0- FREE:95568904kB- ALTERNATEMOVIEDIR:0- FREE:28225008kB-
DATABASE INFORMATION
RECSTART:2010-02-01 18:30:00- MOVIE YEAR:2008- ORIGINAL SERIES DATE:2008-06-19-
PROGRAMID:EP000152650023- CHANNEL ID:1053- CATEGORY:Animated-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:167172738- MATCHED TVDB SERIES ID:-
####################END OF DEBUG LOG#####################
FAILSAFE FLAG WAS SET CHECK PERMISSIONS AND FOLDERS
PERMISSION ERROR OR DRIVE FULL
ATTEMPTING SYMLINK TO FAILSAFE DIR: /home/mythtv
FAILSAFE MODE COMPLETE: SYMLINK CREATED
FAILSAFE MODE FAILURE CHECK PERMISSIONS AND FREE SPACE IN /home/mythtv
hell..im so dumb..evcen my user mythtv cant even write to its own home directory..pshawww
Code:
mythtv@myth ~/mythicalLibrarian $ touch /home/mythtv/i-canfucking-write-here
mythtv@myth ~/mythicalLibrarian $ touch /mnt/ntfs/tv/look-ma-i-can-write-here-too
mythtv@myth ~/mythicalLibrarian $ touch /mnt/extra/tv/oh-gee-no-permission-errors-here
mythtv@myth ~/mythicalLibrarian $ touch /mnt/ntfs/movies/must-be
mythtv@myth ~/mythicalLibrarian $ touch /mnt/extra/movies/the-script
mythtv@myth ~/mythicalLibrarian $
and for good meausre
Code:
myth Downloads # fdisk -l                                                                                                                                                                                    

Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x039a039a                    

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1           9       72261   83  Linux
/dev/sda2              10          75      530145   83  Linux
/dev/sda3              76        9729    77545755   83  Linux

Disk /dev/sdb: 300.1 GB, 300090728448 bytes
255 heads, 63 sectors/track, 36483 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xf9acd788                    

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1       32629   262092411    7  HPFS/NTFS
/dev/sdb2   *       32630       36483    30957255   83  Linux    
myth Downloads # df -h                                          
Filesystem            Size  Used Avail Use% Mounted on          
rootfs                 73G   68G  1.8G  98% /                    
/dev/root              73G   68G  1.8G  98% /                    
rc-svcdir             1.0M  148K  876K  15% /lib64/rc/init.d    
udev                   10M  284K  9.8M   3% /dev                
shm                   994M   48K  994M   1% /dev/shm            
/dev/sda1              69M   47M   19M  72% /boot                
/dev/sdb1             250G  161G   90G  65% /mnt/ntfs            
/dev/sdb2              28G  173M   27G   1% /mnt/extra
sorry for beinmg a moron