[jncl]

myth2kodi
for viewing MythTV recordings in XBMC (Kodi)

Features

• List "Specials" and regular episodes under the same series in the "TV Shows" section of XBMC
  
• Pull "watched" flag from MythTV so that episodes are also checked as watched in XBMC
  
• Scrape TTVDB or TMDB info (whichever is stored in MythTV) to nfo files for each series and episode, and download poster, fanart, and banner images for each series
  
• Use Comskip for commercial detection
  
• Run as a MythTV user job when a recording is finished, or to clean myth2kodi files when a recording is deleted (WIP)
  

Download and Setup
myth2kodi is maintained on GitHub here:

• https://github.com/joncl/myth2kodi
  

For setup details, please see the myth2kodi wiki here:

• https://github.com/joncl/myth2kodi/wiki
  

Change Log

• 2014-12-11 (v0.1.36)
  1. Fixed --comskip argument not able to run
  2. Fixed hard coded log file location - log file now written adjacent to myth2kodi.py
  3. Removed airdate from file name of specials which required new --clean argument below
  4. Changed attribute value for "aired" in each nfo file from the Mythtv value of "StartTime" to "Airdate"
  5. Added --clean argument: --clean traverses entire symlink directory updating file names while keeping all comskip files
  6. Added auto-clean mechanism
  7. Added --refresh-nfos: updates all existing nfos files with the correct "aired" value (see 4 above)
  8. Added mechanism to continue searching for series info from either TTVDB or TMDB in case the first episode in the series happens to be a "special"
  9. Updated log format
  
• 2014-11-13
  - Implement Python logging
  - Add --log-debug argument
  - Fix comskip (fixed command to call comskip with wine)
  - Add --comskip-status to show number of recordings not yet comskipped
  - Fix logic when certain arguments are given
  
• 2014-11-3
  - Write a separate log file with name corresponding to the recording only if an error was encountered and in conjunction with --add (which is used for a single recording after it completes - previously the log file was overwritten after every execution of myth2kodi.py which erased any needed error information)
  
• 2014-11-2
  - Added argument --import-recording-list to test recording lists from any MythTV backend
  - Added argument --add-match-programid to match only on program id
  - Set default encoding to 'utf-8' to handle non-ascii characters
  - Set MySQL connection to use utf-8 and unicode
  - Set additional variables as unicode
  
• 2014-10-18
  - Always write a log file (previously only wrote a log file if an error was encountered).
  - Changed argument "--scan-all" to "--add-all"
  - Added argument "--show-all" - use this to test the script without adding any new files
  - Added argument "--comskip" - used to comskip just a single recording
  
• 2014-10-04
  - Switched to using Comskip for commercial detection (Comskip ROCKS!) Run the script with --comskip-all on your current symlink directory (warning it may take many hours, it took 36 hours to scan my 670 recordings). Or use --add <full path to MythTV mpg> to Comskip just one recording for testing (this is the same command used for the MythTV user job BTW).
  - Updated logging and error trapping... if any errors are encountered, you should get a myth2kodi.log file in your myth2kodi directory.
  - Added argument --scan-all, and other arguments used mostly for testing
  - Updated help output, use -h or --help for details
  - More testing, working nicely here now.
  
• 2014-10-01
  - Added MythTV database look-up of commercial flags that get written to adjacent comskip txt files
  - Added logging mechanism, enable with argument --log
  - Added error handling (upon error, full log file is written)
  
• 2014-09-30
  - Added MythTV database look-up of watched flag, to be stored under 'playcount' in epidsode nfo files
  - Updated wiki with step to add 'advancedsettings.xml' file to XBMC box (to show the playcount flag)
  - Updated wiki with step to install PIL (Python Imagin Library) which is used to create a banner image for TMDB look-ups
  
• 2014-09-28
  - Fixed issue with MythTV giving an error when running the script as a user job (MythTV needs an an exit code of 0)
  - Update wiki with step to copy script to /usr/local/bin/myth2kodi/myth2kodi.py as root, and set execute permission (+x)
  

myth2kodi is written in Python 2.7.3 using PyCharm 3.4, tested in Ubuntu 12.04 running Mythbackend .27, and tested on the frontend with XBMC 13.2.

I hope someone will find this useful!

Please post with any comments/questions.

[ChrisMyhre]

Very cool, can't wait to try this out.

[mikeg321]

I had the same issues you had with the other scripts and gave up on using TV shows in XBMC. We just watch from the LiveTV-->Recordings group but not as nice. So, I'm hoping your script may work for our use case as well. Also, nice job on the Wiki and documenting!

I'm having some issues getting it going so have a couple questions:
1. A lot of my recordings don't have an inetref # associated. Would that render this script not usable in my case, or is other generic data inserted? Even when I try and look up metadata most of my recordings don't seem to find a reference and update so might be a lot of work to manually input for older recordings.

2. When I run myth2kodi.py --comskip-off -l I get a blank log. The script runs in about 1 second so is tough to tell where/why it's failing. Any ideas on troubleshooting? I've done the basics like checking permissions on folder etc and all seems in order.

myth2kodi.py --print-config

hostname: 192.168.2.6
host_port: 6544
myth_recording_dirs: ['/mnt/Media1/Recordings1', '/mnt/Media2/Recordings2', '/mnt/Media3/Recordings3']
symlinks_dir: /mnt/Videos/Myth2Kodi_TVLinks/Links/
ttvdb_key: ************
ttvdb_zips_dir: /mnt/Videos/Myth2Kodi_TVLinks/TVDB
tmdb_key: ****************************
db_user: mythtvuser
db_passwd: ********
db_name: mythconverg

Thanks

[jncl]

Hey there, thanks for trying this out! I just checked in an update to myth2kodi.py so that it should always write a log file now. Also, try with --scan-all --comskip-off. I believe the reason it ran for just 1 second is that without either --scan-all or --add <filename>, it just ran right through and exited without actually doing anything (see the try-except-else block at the very end of the script). Let me know if that helps.

[mikeg321]

OK, maybe a little further now. The log shows the script starting to go through some files and then an error:

Code:

There's about 200 shows processed before this
...
20141017_205841.492 ERROR: MISSING INETREF!: The Event
20141017_205841.492  INFO: PROCESSING PROGRAM:
                               title: Spartacus: Blood and Sand
                               filename: 436_20101004210000.mpg
                               program id: None
20141017_205841.492 ERROR: MISSING INETREF!: Spartacus: Blood and Sand




'ascii' codec can't encode character u'\xe9' in position 2: ordinal not in range(128)

Perhaps some character in my database isn't making the script happy... I'll have to look in the DB at some of the text for this last show. I assume it's choking on some character there. I actually deleted a show before the error and it just failed on another one so must be a few of these in my DB.

Let me know if something seems obviously wrong. Thanks!

[jncl]

Okay, yes, if the script doesn't find an inetref in MythTV, it will skip that recording and continue reading the rest. If you click the "Look up Metadata" button in mythweb, does it give you an Internet Reference?:



This doesn't apply for existing recordings though because I believe "mythmetadatalookup --refresh-all-rules" only updates the inetref for recording schedules. For recordings, I think you have to manually change the value in the MythTV database. At least that's how I did it. The commands for that are in the wiki at the bottom of the page under "Tips / Workarounds".

Regarding the ascii codec error, did some quick checking and it seems this is related to how Python 2.7 implicitly handles codecs and maybe ran into some non-ascii characters. So what I did is add "# -- coding: utf-8 --" at the top of the script. I'm hoping that fixes the error you encountered.

I also made some changes to the arguments:

• Changed "--scan-all" to "--add-all"
  
• Added "--show-all" - use this to test the script without adding any new files
  
• Added "--comskip" - used to comskip just a single recording
  
• Need to add "--comskip-missing" to comskip any recordings not yet comskipped. Right now it just comskips with "--add <filename>" or "--comskip-all"
  

[mikeg321]

Definitely getting close now. I did some reading last night as I figured I'd need those inetref values populated. I ran mythmetadatalookup --refresh-all and that has populated the majority of my database now. When i ran the latest script I actually now have a few fully populated directories for TV shows. I ran into another error that again looks like something with the data in my particular DB. If you have a chance or any ideas to get around it let me know. I'm more of a 101 level programmer... Really appreciate your help on this and sharing your work.

log:

Code:

20141018_111056.588  INFO: PROCESSING PROGRAM:
                               title: Surviving Disaster
                               filename: 418_20090905210000.mpg
                               program id: EP011728020001
20141018_111056.588  INFO: Looking up watched flag from db
20141018_111056.588  INFO: Looking up commercial markers
20141018_111056.589  INFO: Adding new series from TTVDB: Surviving Disaster
20141018_111056.589  INFO: TTVDB zip does not exist, downloading ttvdb zip file to: /mnt/Videos/Myth2Kodi_TVLinks/TVDB/Surviving_Disaster_253053.zip
20141018_111056.696  INFO: ttvdb zip exists, reading xml contents...
20141018_111056.696  INFO: Reading en.xml...
20141018_111056.696  INFO: Writing series nfo file...
20141018_111056.696  INFO: Retrieving poster, fanart, and banner...




cannot concatenate 'str' and 'NoneType' objects

Here's a CSV of the Surviving Disaster info in my DB for the Recorded table too.

Code:

chanid,starttime,endtime,title,subtitle,description,season,episode,category,hostname,bookmark,editing,cutlist,autoexpire,commflagged,recgroup,recordid,seriesid,programid,inetref,lastmodified,filesize,stars,previouslyshown,originalairdate,preserve,findid,deletepending,transcoder,timestretch,recpriority,basename,progstart,progend,playgroup,profile,duplicate,transcoded,watched,storagegroup,bookmarkupdate
418,"2009-09-06 03:00:00","2009-09-06 04:00:00","Surviving Disaster",Hijack,"Navy SEAL Cade Courtley fights terrorists in a simulated aircraft hijacking.",0,0,Reality,MythServer4,1,0,0,0,1,TV,1247,EP01172802,EP011728020001,ttvdb.py_253053,"2014-10-17 22:22:31",1128151340,0,0,2009-09-01,0,0,0,0,1,0,418_20090905210000.mpg,"2009-09-06 03:00:00","2009-09-06 04:00:00",Default,Default,1,0,0,Default,"2009-11-14 20:53:44"
418,"2009-09-17 05:06:00","2009-09-17 06:09:00","Surviving Disaster",Hurricane,"Survival methods and military techniques used to endure a hurricane.",0,0,Reality,MythServer4,0,0,0,0,1,TV,1258,EP01172802,EP011728020008,ttvdb.py_253053,"2014-10-17 22:22:50",1304370508,0,0,2009-09-15,0,0,0,0,1,0,418_20090916230600.mpg,"2009-09-17 05:06:00","2009-09-17 06:09:00",Default,Default,1,0,0,Default,"2009-09-17 00:38:07"

Also seems that the UTF-8 change may have stopped that error, although the script may not have got as far through this time so can't confirm just yet.

[jncl]

Ok, that's good. But maybe setting utf-8 at the top is causing this error now, not sure. I updated the script with some error handling in that section where it looks up ttvdb urls for posters, banners, and fanarts. It should still encounter the error, but at least (hopefully) the log will provide more info on where it failed.

[mikeg321]

OK, I ran again and here's a few errors. I could run the script 3 times and it kind of picked up where it left off. Now it's stuck on the last error and won't go any further. The 3 error instances are separated in the log by >>>>>. Also, Bridget Jone's's is not mine :-) ....the Wife's. I'd be happy to delete to get rid of the error!

Code:

20141018_131248.709  INFO: PROCESSING PROGRAM:
                               title: Bridget Jones's Diary
                               filename: 283_20091230200000.mpg
                               program id: MV001062410000
20141018_131248.709  INFO: Looking up watched flag from db
20141018_131248.710  INFO: Adding new series from TMDB: Bridget Jones's Diary
20141018_131249.138  INFO: Getting path to poster...
20141018_131249.138  INFO: Getting path to fanart...
20141018_131250.215  INFO: Making 758 x 140 banner image from poster...




'ascii' codec can't encode character u'\u2019' in position 13: ordinal not in range(128)

>>>>>>>>>

20141018_131722.429  INFO: PROCESSING PROGRAM:
                               title: The Hours
                               filename: 90_20100326232000.mpg
                               program id: MV001257080000
20141018_131722.429  INFO: Looking up watched flag from db
20141018_131722.430  INFO: Adding new series from TMDB: The Hours
20141018_131722.782  INFO: Getting path to poster...
20141018_131722.782  INFO: Getting path to fanart...
20141018_131723.779  INFO: Making 758 x 140 banner image from poster...




'ascii' codec can't encode character u'\u201c' in position 70: ordinal not in range(128)


>>>>>>>>>>>>>>>
20141018_132655.582  INFO: PROCESSING PROGRAM:
                               title: Spartacus: Blood and Sand
                               filename: 436_20101004210000.mpg
                               program id: None
20141018_132655.582 ERROR: MISSING PROGRAMID! Spartacus: Blood and Sand




'ascii' codec can't encode character u'\xe9' in position 2: ordinal not in range(128)

>>>>>>>>>>>

[jncl]

K, just updated it. Changed str calls to unicode calls. See if that helps.

[mikeg321]

Still isn't quite happy. I deleted all links and started from scratch. Fail each time at the same entry. Still not liking this one item for some reason. I may remove from my database later and see if it's just this one entry. Does the missing Program ID have any bearing on this? I noticed that a fair number of my shows son't have a ProgramID. edit: (I put a program id in for this and no change so must not be a factor)

Code:

20141018_162822.199  INFO: PROCESSING PROGRAM:
                               title: Spartacus: Blood and Sand
                               filename: 436_20101004210000.mpg
                               program id: None
20141018_162822.199 ERROR: MISSING PROGRAMID! Spartacus: Blood and Sand




'ascii' codec can't encode character u'\xe9' in position 2: ordinal not in range(128)

[jncl]

Sorry it's not working for you yet. I thought that the ProgramID would always be there, but after looking at the description of it here, there are situations where it might be empty. I'm only using the ProgramID to find the playcount. So I need to look for a different attribute that's always there and unique to a recording.

In this case though, the missing program id shouldn't throw that error. Only thing that happens without a ProgramID is the recording gets skipped, and the script moves to the next one. I think the ascii codec exception is getting thrown somewhere on the info of the recording just after Spartacus.

I made some changes which you can try:

• Added "--print-match-title <title match>" - enter something like --print-match-title Spartacus. That should print the recording info in xml format so you can see the actual values of all the fields.
  
• Changed the code so that a recording with a missing ProgramID will still get added. Only thing missing in the resulting nfo file will be the playcount, which is what relies on ProgramID.
  
• Wrapped all of the values read from the incoming MythTV recording info with unicode(). I'm hoping that will squash the ascii exception!
  

[mikeg321]

Oh no problem at all, happy to just play around and don't want to be a bother either. I've been doing a little reading as well and stumbled across that link last night too. Trying to see what this is all doing... I was able to get a good deal of my library to scan by using the --ad-match-title command and going through the alphabet. Gave me a chance to do an XBMC source import and see how things would look at least. Didn't get a lot of time to play though. One thing I was thinking might be nice is having the subtitle info appended onto the title. Like Title Name - Subtitle name. That way it'll show in XBMC with episode names. Not sure if that would be useful to folks or not though. I was looking at your code to see how this might be done and although I couldn't do it I think it could be fairly straight forward by what I saw. Never done much programming other than Assembler...and that was quite some time ago at that.


I think the new code to deal with the lack of ProgramID has a glitch. The first recording without ProgID throws this:

Code:

20141019_164553.687     INFO: PROCESSING PROGRAM:
                               title: Major Payne
                               filename: 5301_20070609164500.mpg
                               program id: None
20141019_164553.688     INFO: Looking up watched flag from db


list index out of range

[jncl]

No worries. I'd like to get this working for your database because it gives me a chance to fix issues I didn't encounter with mine. So thanks for taking the time to test it out!

Okay, so I added a new argument "--export-recording-list". What this does is write out the entire MythTV recording list to an xml file. What you can do if you want is send me your output xml file, and then I can debug against it directly and hopefully figure out exactly what's going on. I'll IM you my email address.

[mikeg321]

I'll try and send to you a bit later.