Kodi Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - mortenls - 2012-09-02

Hello and thanks for making mythicalLibrarian

I'm having an problem getting the database information to the mythicalLibrarian script. It seems that the script do not like the danish letters å æ ø. I confirmed this by editing the letters in question, and that makes it run as it should. It is not a big problem as i anyway have to edit the xmltv data to the english titles, as many of them are in danish. If there is a quick fix to this i would really appreciate it.


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - outleradam - 2012-09-23

hey guys, update...

Everything is working properly in 0.22, 0.23, and 0.24 I need someone to test the mythDataGrabber script in 0.25 to verify everything is ready to roll for the upgrade.

I just committed changes to fix the ~/.mythicalLibrarian/showTranslations file.. there was problems in the documentation and the code. some referred to ShowTranslations, some showTranslations and some showtranslations. This needed consistancy and I didn't realize it was a problem until I tried to scan Wilfred and it was a PITA.. Anyway, this will never change again.

The proper way is now:
~/.mythicalLibrarian/showTranslations

hope this does not cause any problems for you guys.


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - TechnoSwiss - 2012-09-26

Quick question. The folks over at thetvdb.com have decided that Masterpiece shows are a violation of their duplicate rule, as most of the shows air earlier in the UK.

The issue for us here being, they air in the US under the Masterpiece series title, and of course with a different original air date if you're using MythTV and getting your data from the EPG.

Their belief is we should be naming our shows off the BBC/ITV series title (IE Downton Abby), instead of the PBS (which would be Masterpiece Classic, with an Episode title of Downton Abby). Would it be possible to setup the script to parse the series title out of the Episode name if it can't find the series information on thetvdb.com?

Or maybe it would be possible to use tvrage.com or imdb.com for information if a lookup to thetvdb.com doesn't yield results? Both of those sources still list Masterpiece as its own series.

Thanks for your great work on this outleradam.


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - outleradam - 2012-09-26

If you can figure out how to convert theTvDb parser over to TvRage, I will add it as an option in setup. Also, it can be a fail over and possibly add 8 additional points of confidence to a recording. I do not have any plans to make my own TV rage grabber though.


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - jerkychew - 2012-09-26

Hi,

I love ML; it has done a great job of closing the gap between MythTV and watching recordings using Plex or Boxee.

Now that 0.25 has metadata information right in the database, I was wondering if there were any plans to start using that as a source for ML rather than querying the Internet? It looks like the mythrename.pl script is almost there, but its output still doesn't contain season or episode number. Plus, I like being able to run ML as a userjob. Smile

Thanks!


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - TechnoSwiss - 2012-09-27

Thanks, I'll take a look and see if I can figure it out.


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - sdsnyr94 - 2012-10-03

OutlerAdam-

I just updated mythicalLibrarian for the first time in at least a year. Now I am running into a small issue. Prior to the update, I would run a user job that runs handbrake to re-encode the video after the file has been renamed and transfered, and then the script MythDataDeleter to remove the info from the DB if the Handbrake job completed. Mythicallibrarian would still be listed as running in the job queue until Handbrake and MythDataDeleter completed. I previously used the 'SYMLINK=Move' setting.

After the update, it appears that I can skip the MythDataDeleter step, as you have the option for 'SYMLINK=Disabled', which can remove the DB info. However, when I run mythicalLibrarian now it moves the file to the proper location, begins Handbrake, and then deletes all references to the file from MythTV..... including the job that is running in the queue. Handbrake is still currently running if I do 'top', so it's not actually killing the job.....but MythTV no longer knows this job is active, and this may prevent me from queuing up a batch of videos that I need to rename/re-encode (I think I may end up with multiple Handbrake sessions trying to run at once).

Is there any way that I can correct this? Maybe delay the removal of Database information until after the User Job is complete?

Thanks.

{Edit} In the meantime, I just went back to using 'SYMLINK=MOVE' and MythDataDeleter {/EDIT}



RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - zipper456 - 2012-10-13

Thank you so much for making mythicallibrarian.

I have an issue: even though the episode is new, mythicallibrarian matches with the first on the list, so my links are being Season 1, Ep 1.

Here is the Log
*******************************************************************
Stable - Downloaded Thu Oct 11 02:00:22 CDT 2012
Attemting to pull database information on /var/lib/mythtv/recordings/2081_20121010200000.mpg
COMMERCIAL DATA START:50559 STOP:55598
/usr/local/bin/mythicalLibrarian: line 718: None=: command not found
RECSTART:2012-10-10 20:00:00 DATE: None
PROGRAMID: ShowCategory:
PLOT: Schooled: Claire and Phil embarrass Haley when they drop her off at college; Manny makes Gloria and Jay take a class; Lily gets into a tussle on her first day of kindergarten.
PERFORMING MAINTENANCE ROUTINE...
SEARCHING: www.TheTvDb.com SHOW NAME: Modern Family EPISODE:
TESTING FOR ZAP2ITID MATCH:EP01158124 Modern Family
FOUND:Modern Family ID#:95011 WITH CONFIDENCE:-1
DATABASE IS OUT OF DATE. NEXT UPDATE 1350007525 Current 1350136708
Parsing Downloaded information: Modern Family.xml
AIRDATE FAILED. TITLE MATCH ON ABSOLUTE EPISODE NUMBER:
TOTAL MATCHES:14 TOTAL VERIFIED FUZZY ERRORS:0
EPISODETongueilot EPISODE NUMBER:2 S01E01 WITH CONFIDENCE: -1
WARNING: FILE IS A LINK FORCING SYMLINK LINK MODE
CONFIDENCE:-2 LOGIC PATH: Attempted Match On Zap2itID: Failed; Attemted match on Standard Logic: Sucessful; Attemped match on OriginalAirDate: Failed; Attemted SubTitle match on Standard logic: Sucessful;
JOB: mythicalLibrarian '/var/lib/mythtv/recordings/2081_20121010200000.mpg' 'Modern Family' ''
TARGET SET:/var/lib/mythtv/recordings/Episodes/Modern Family/Season 1/Modern Family S01E01 (Pilot).mpg
FILE NAME EXISTS. FILE WILL BE KNOWN AS: Modern Family S01E01 (Pilot)-2
CREATING LINK IN LINK MODE WITH CONFIDENCE:-2
SENDING REQUESTED COMMANDS TO:192.168.99.190:8080
RSS generation failed:
File exists but is not writeable: /var/www/mythical-rss/rss.xml
SUCCESSFUL COMPLETEION TYPE: LinkModeSuccessful
******************************************************************************



RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - TechnoSwiss - 2012-10-29

Okay, so I've got a couple of updates to share. Initially I was playing with getting TVRage working, then I decided I really wanted to be able to use some local XML files for my shows, instead of downloading them. So I added an option to skip downloading for some shows and use local files instead.

So the first change, to allow using a file called ~/.mythicalLibrarian/skipDownload the contains a list of shows you want to use a local XML instead downloading from thetvdb.com

For the example I'll use Masterpiece Classic

~/.mythicalLibrarian/skipDownload
Code:
##############################################################
# List of Shows to use local XML files instead of thetvdb    #
# The following files must exist for each show               #
# <show>.working.xml - thetvdb seach listing for the show    #
# <show>.xml - thetvdb episode listing for the show          #
##############################################################
Masterpiece Classic

~/.mythicalLibrarian/Masterpiece Classic.working.xml
Code:
<?xml version="1.0" encoding="UTF-8" ?>
<Data>
<Series>
<seriesid>8256</seriesid>
<language>en</language>
<SeriesName>Masterpiece Classic</SeriesName>
<banner>graphical/78257-g.jpg</banner>
<Overview>Masterpiece Theatre premiered on PBS... </Overview>
<FirstAired>1971-01-01</FirstAired>
<IMDB_ID>tt0066684</IMDB_ID>
<zap2it_id>SH002814</zap2it_id>
<id>78257</id>
</Series>
</Data>

~/.mythicalLibrarian/Masterpiece Classic.xml
Code:
<?xml version="1.0" encoding="UTF-8"?>
<Data><Series>
  <id>78257</id>
  <Actors>|Alistair Cooke|Russel Baker|</Actors>
  <Airs_DayOfWeek>Sunday</Airs_DayOfWeek>
  <Airs_Time>9:00 PM</Airs_Time>
  <ContentRating/>
  <FirstAired>1971-01-01</FirstAired>
  <Genre>|Drama|</Genre>
  <IMDB_ID>tt0066684</IMDB_ID>
  <Language>en</Language>
  <Network>PBS</Network>
  <NetworkID/>
  <Overview>Masterpiece Theatre premiered on PBS...  </Overview>
  <Rating>9.0</Rating>
  <RatingCount>1</RatingCount>
  <Runtime>1</Runtime>
  <SeriesID>8256</SeriesID>
  <SeriesName>Masterpiece Classic</SeriesName>
  <Status>Continuing</Status>
  <added/>
  <addedBy/>
  <banner>graphical/78257-g.jpg</banner>
  <fanart>fanart/original/78257-1.jpg</fanart>
  <lastupdated>1345225860</lastupdated>
  <poster>posters/78257-1.jpg</poster>
  <zap2it_id>SH002814</zap2it_id>
</Series>
<Episode>
  <EpisodeName>Downton Abbey, II (1)</EpisodeName>
  <EpisodeNumber>1</EpisodeNumber>
  <FirstAired>2012-01-08</FirstAired>
  <SeasonNumber>41</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Downton Abbey, II (2)</EpisodeName>
  <EpisodeNumber>2</EpisodeNumber>
  <FirstAired>2012-01-15</FirstAired>
  <SeasonNumber>41</SeasonNumber>
</Episode>
</Data>

and finally, the updates to mythicalLibrarian to support this

Code:
Replace:
        curl -L -s -m"$Timeout" "http://www.thetvdb.com/api/$APIkey/series/$SeriesID/all/$Language.xml"| grep -vE "(</id>|</Combined_episodenumber>|</Combined_season>|</DVD_chapter>|</DVD_discid>|</DVD_episodenumber>|</DVD_season>|</Director>|</EpImgFlag>|</GuestStars>|</IMDB_ID>|</Language>|</Overview>|</ProductionCode>|</RatingCount>|</filename>|</seasonid>|</Writer>|</Rating>|</lastupdated>|</seriesid>|</absolute_number>)">"$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest" && echo "$CurrentTime" > "$mythicalLibrarian/$NewShowName/lastupdated.time"

With:
        skipDownload=`grep "$ShowName" "$mythicalLibrarian/skipDownload"`
        if [ "$skipDownload" != "" -a -f "$mythicalLibrarian/$ShowName.xml" ]; then
            cp "$mythicalLibrarian/$ShowName.xml" "$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest"
        else
            curl -L -s -m"$Timeout" "http://www.thetvdb.com/api/$APIkey/series/$SeriesID/all/$Language.xml"| grep -vE "(</id>|</Combined_episodenumber>|</Combined_season>|</DVD_chapter>|</DVD_discid>|</DVD_episodenumber>|</DVD_season>|</Director>|</EpImgFlag>|</GuestStars>|</IMDB_ID>|</Language>|</Overview>|</ProductionCode>|</RatingCount>|</filename>|</seasonid>|</Writer>|</Rating>|</lastupdated>|</seriesid>|</absolute_number>)">"$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest" && echo "$CurrentTime" > "$mythicalLibrarian/$NewShowName/lastupdated.time"
        fi

Replace:
        curl -L -s -m"$Timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$tvdbshowname>"$mythicalLibrarian/working.xml"

With:
        skipDownload=`grep "$ShowName" "$mythicalLibrarian/skipDownload"`
        if [ "$skipDownload" != "" ] && [ -f "$mythicalLibrarian/$ShowName.working.xml" ]; then
            cp "$mythicalLibrarian/$ShowName.working.xml" "$mythicalLibrarian/working.xml"
        else
            curl -L -s -m"$Timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$tvdbshowname>"$mythicalLibrarian/working.xml"
        fi

Having the <showname>.working.xml might be overdoing it a bit, but it looked easier to include that instead of bypassing the logic that was already in place.

The <showname>.xml file contains all your episode information, you can start with the XML from thetvdb.com and modify it to suit your need. In my case since thetvdb.com removed Masterpiece, I just grabbed an old copy of the XML I had from before they removed it, and added in this seasons shows.


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - TechnoSwiss - 2012-10-29

I was also able to use TVRage as a source, by modifying two functions downloadInfoAboutMatchedTitleAndCreateTxtFileTables () and searchTheTvDbAndReturnTxtFileTables(), TVRage doesn't have Zap2It IDs and doesn't track times for when their database is updated. I was able to leave those functions alone however and by just making the changes to the two functions mentioned, run mythicalLibrarian with TVRage as the datasource.

Sorry outleradam, I was going to try and make the changes to support TVRage as a second/backup source, but my bashfoo is weak. Hopefully this is enough for you to be able to add it in as a second/backup source.

The changes are actually really minor, but I'm just going to include the entirety of the two functions below.

Code:
#####GET EPISODE INFORMATION#####
downloadInfoAboutMatchedTitleAndCreateTxtFileTables () {
#Download information from server
        echo -ne "Downloading Series: $SeriesID
" 1>&2
        curl -L -s -m"$Timeout" "http://services.tvrage.com/feeds/episode_list.php?sid=$SeriesID"| sed -e s/\>\</\>\\\n\</g >"$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest" && echo "$CurrentTime" > "$mythicalLibrarian/$NewShowName/lastupdated.time"

#Validate that the show has a EpisodeName tag and is a valid download before clobering our old download.
        grepTest=""
        grepTest=`grep "title" "$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest"`
        test "$grepTest" != "" && mv "$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest" "$mythicalLibrarian/$NewShowName/$NewShowName.xml"

#create a folder/file "database" Strip XML tags.  Series, Exx and Sxx are separated into different files
        if [ -f "$mythicalLibrarian/$NewShowName/$NewShowName.xml" ]; then
            Ename=""
            actualEname=""
            FAired=""
            SeasonNr=""
            EpisodeNr=""
            recordNumber=0
            
            echo "Parsing Downloaded information: $NewShowName.xml "
             while read line
            do

                if [[ $line == \<\/episode\> ]]; then
                    (( ++recordNumber ))
                    echo -ne "Building Record:$recordNumber ${actualEname:0:20}          
" 1>&2
                    echo "$actualEname" >> "$mythicalLibrarian/$NewShowName/$NewShowName.actualEname.txt"
                    
                    echo "$Ename" >> "$mythicalLibrarian/$NewShowName/$NewShowName.Ename.txt"
                    echo "$FAired" >> "$mythicalLibrarian/$NewShowName/$NewShowName.FAired.txt"
                    echo "$SeasonNr" >> "$mythicalLibrarian/$NewShowName/$NewShowName.S.txt"
                    echo "$EpisodeNr" >> "$mythicalLibrarian/$NewShowName/$NewShowName.E.txt"
                    Ename=""
                    actualEname=""
                    FAired=""
                    SeasonNr=""
                    EpisodeNr=""
                 
#Get actual show name    
                 elif [[ $line == \<title\>* ]]; then
                    line=${line/<\/title>/}
                    line=${line/<title>/}
                    line=${line/&lt;/}
                    line=${line/&gt;/}
                    line=${line/&quot;/}
                    line=${line/&amp;/&}
                    line=${line/|/}
                    line=${line/\?/}
                    line=${line/\*/}
                    line=${line/:/}
                    line=${line/+/}
                    line=${line/\/}
                    line=${line/\//}
                    line=${line/[/}
                    line=${line/]/}
                    line=${line/\'/}
                    line=${line/\"/}
                    actualEname=${line/&ndash;/-}
                    Ename=${actualEname/;*/}
                
#Get OriginalAirDate
                elif [[ $line == \<airdate\>* ]]; then
                    line=${line/<\/airdate>/}
                    line=${line/<airdate>/}
                    FAired=$line
                
#Get Season number
                 elif [[ $line == \<seasonnum\>* ]]; then
                    line=${line/<\/seasonnum>/}
                    line=${line/<seasonnum>/}
                    SeasonNr=$line

#Get Episode number
                elif [[ $line == \<epnum\>* ]]; then
                    line=${line/<\/epnum>/}
                    line=${line/<epnum>/}
                     EpisodeNr=$line
                fi
            done < "$mythicalLibrarian/$NewShowName/$NewShowName.xml"


            chmod 666 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".actualEname.txt
            chmod 666 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".Ename.txt
            chmod 666 "$mythicalLibrarian/$NewShowName/$NewShowName".FAired.txt
            chmod 666 "$mythicalLibrarian"/"$NewShowName"/"$NewShowName".S.txt
            chmod 666 "$mythicalLibrarian/$NewShowName/$NewShowName".E.txt
             GotNewInformation=1


        elif [ ! -f "$mythicalLibrarian/$NewShowName/$NewShowName.xml" ]; then
            echo "COULD NOT DOWNLOAD:services.tvrage.com/feeds/episode_list.php?sid=$SeriesID">>"$mythicalLibrarian"/output.log
        fi
#check if files were created and generate message
        if [ "$GotNewInformation" = "1" ]; then
return 0
             echo $CurrentTime>"$mythicalLibrarian/$NewShowName/lastupdated.time" && chmod 666 $CurrentTime>"$mythicalLibrarian/$NewShowName/lastupdated.time"
            echo "LOCAL DATABASE UPDATED:$mythicalLibrarian/$NewShowName">>"$mythicalLibrarian"/output.log
        elif [ ! -f "$mythicalLibrarian/$NewShowName/$NewShowName.Ename.txt" ]; then
            echo "*** PERMISSION ERROR $mythicalLibrarian/$NewShowName/">>"$mythicalLibrarian"/output.log
return 1
        fi

    return 1
}

The Zap2It parsing can probably be pulled out since TVRage doesn't include the Zap2It number in it's public listings, I left it in however, so zap2it.txt will get cleared in case it still exists from a previous run. (although I do believe that mythicalLibrarian cleans all that up after it's done anyway). Also the call to searchTheTvDbAndReturnTxtFileTables() has to be changed to this function if you're just doing a straight change from thetvdb to TVRage.
Code:
#####SEARCH TVRage AND CREATE DATA FILES#####
searchTVRageAndReturnTxtFileTables(){

#TODO Instance searching rather then global.  Need to create an instance varible based on time and chanid.

#Search TheTvDb and return an XML file with results
    curl -L -s -m"$Timeout" services.tvrage.com/feeds/search.php?show=$tvdbshowname>"$mythicalLibrarian/working.xml"

#Separate the XML into textual DBs conatining SeriesID, ShowName and Zap2it IDs
    cat "$mythicalLibrarian/working.xml" | grep "<showid>"|sed -e s/"<showid>"/""/ -e s/"<\/showid>"/""/ -e s/'&quot;'/'"'/ -e s/"&amp;"/"\&"/ -e s/'&ndash;'/'-'/ -e s/"&lt;"/"<"/ -e s/"&gt;"/">"/ >"$mythicalLibrarian/sid.txt"
    cat "$mythicalLibrarian/working.xml" | grep "<name>"| sed -e s/"<name>"// -e s/"<\/name>"// -e s/'&quot;'/'"'/ -e s/"&amp;"/"\&"/ -e s/'&ndash;'/'-'/ -e s/"&lt;"/"<"/ -e s/"&gt;"/">"/ >"$mythicalLibrarian/shn.txt"
    

#parse the XML for shows with Zap2ItIDs
    test -f "$mythicalLibrarian/zap2it.txt" && rm -f "$mythicalLibrarian/zap2it.txt"
    while read line
    do
        test "$line" = "<show>" && NextShow=1
        test "$line" = "</show>" && test "$NextShow" = "1" && echo "" >>"$mythicalLibrarian/zap2it.txt" && NextShow=0

        if [[ $line =~ '<zap2it_id>'* ]] ; then
            echo $line| sed -e s/"<zap2it_id>"/""/ -e s/"<\/zap2it_id>"/""/ -e s/'&quot;'/'"'/ -e s/"&amp;"/"\&"/ -e s/'&ndash;'/'-'/ -e s/"&lt;"/"<"/ -e s/"&gt;"/">"/ >>"$mythicalLibrarian/zap2it.txt"
            NextShow=0
             fi
    done < "$mythicalLibrarian/working.xml"

}



RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - outleradam - 2012-11-09

Whoah, what happpened to my email notifications? K.. anyway..

(2012-09-26, 23:15)jerkychew Wrote: Hi,

I love ML; it has done a great job of closing the gap between MythTV and watching recordings using Plex or Boxee.

Now that 0.25 has metadata information right in the database, I was wondering if there were any plans to start using that as a source for ML rather than querying the Internet? It looks like the mythrename.pl script is almost there, but its output still doesn't contain season or episode number. Plus, I like being able to run ML as a userjob. Smile

Thanks!
I havn't taken much of a look at the 0.25 database. are you saying that it has Sxx and Exx in it? If so, that's great and I can offload.


(2012-10-03, 19:57)sdsnyr94 Wrote: OutlerAdam-

I just updated mythicalLibrarian for the first time in at least a year. Now I am running into a small issue. Prior to the update, I would run a user job that runs handbrake to re-encode the video after the file has been renamed and transfered, and then the script MythDataDeleter to remove the info from the DB if the Handbrake job completed. Mythicallibrarian would still be listed as running in the job queue until Handbrake and MythDataDeleter completed. I previously used the 'SYMLINK=Move' setting.

After the update, it appears that I can skip the MythDataDeleter step, as you have the option for 'SYMLINK=Disabled', which can remove the DB info. However, when I run mythicalLibrarian now it moves the file to the proper location, begins Handbrake, and then deletes all references to the file from MythTV..... including the job that is running in the queue. Handbrake is still currently running if I do 'top', so it's not actually killing the job.....but MythTV no longer knows this job is active, and this may prevent me from queuing up a batch of videos that I need to rename/re-encode (I think I may end up with multiple Handbrake sessions trying to run at once).

Is there any way that I can correct this? Maybe delay the removal of Database information until after the User Job is complete?

Thanks.

{Edit} In the meantime, I just went back to using 'SYMLINK=MOVE' and MythDataDeleter {/EDIT}
I don't know what handbrake does. You can launch handbrake as a mythicalLibraian job if you'd like. User Jobs are documented in the wiki with it's own page.

(2012-10-13, 16:02)zipper456 Wrote: Thank you so much for making mythicallibrarian.

I have an issue: even though the episode is new, mythicallibrarian matches with the first on the list, so my links are being Season 1, Ep 1.

Here is the Log
*******************************************************************
Stable - Downloaded Thu Oct 11 02:00:22 CDT 2012
Attemting to pull database information on /var/lib/mythtv/recordings/2081_20121010200000.mpg
COMMERCIAL DATA START:50559 STOP:55598
/usr/local/bin/mythicalLibrarian: line 718: None=: command not found
RECSTART:2012-10-10 20:00:00 DATE: None
PROGRAMID: ShowCategory:
PLOT: Schooled: Claire and Phil embarrass Haley when they drop her off at college; Manny makes Gloria and Jay take a class; Lily gets into a tussle on her first day of kindergarten.
PERFORMING MAINTENANCE ROUTINE...
SEARCHING: www.TheTvDb.com SHOW NAME: Modern Family EPISODE:
TESTING FOR ZAP2ITID MATCH:EP01158124 Modern Family
FOUND:Modern Family ID#:95011 WITH CONFIDENCE:-1
DATABASE IS OUT OF DATE. NEXT UPDATE 1350007525 Current 1350136708
Parsing Downloaded information: Modern Family.xml
AIRDATE FAILED. TITLE MATCH ON ABSOLUTE EPISODE NUMBER:
TOTAL MATCHES:14 TOTAL VERIFIED FUZZY ERRORS:0
EPISODETongueilot EPISODE NUMBER:2 S01E01 WITH CONFIDENCE: -1
WARNING: FILE IS A LINK FORCING SYMLINK LINK MODE
CONFIDENCE:-2 LOGIC PATH: Attempted Match On Zap2itID: Failed; Attemted match on Standard Logic: Sucessful; Attemped match on OriginalAirDate: Failed; Attemted SubTitle match on Standard logic: Sucessful;
JOB: mythicalLibrarian '/var/lib/mythtv/recordings/2081_20121010200000.mpg' 'Modern Family' ''
TARGET SET:/var/lib/mythtv/recordings/Episodes/Modern Family/Season 1/Modern Family S01E01 (Pilot).mpg
FILE NAME EXISTS. FILE WILL BE KNOWN AS: Modern Family S01E01 (Pilot)-2
CREATING LINK IN LINK MODE WITH CONFIDENCE:-2
SENDING REQUESTED COMMANDS TO:192.168.99.190:8080
RSS generation failed:
File exists but is not writeable: /var/www/mythical-rss/rss.xml
SUCCESSFUL COMPLETEION TYPE: LinkModeSuccessful
******************************************************************************
Use mythicalLibrarian --doover to rerun this and all other low confidence jobs. Anything less than 0 is automatically done over when --doover is commanded.


(2012-09-26, 23:15)jerkychew Wrote: Hi,

I love ML; it has done a great job of closing the gap between MythTV and watching recordings using Plex or Boxee.

Now that 0.25 has metadata information right in the database, I was wondering if there were any plans to start using that as a source for ML rather than querying the Internet? It looks like the mythrename.pl script is almost there, but its output still doesn't contain season or episode number. Plus, I like being able to run ML as a userjob. Smile

Thanks!

(2012-09-27, 20:47)TechnoSwiss Wrote: Thanks, I'll take a look and see if I can figure it out.

(2012-10-03, 19:57)sdsnyr94 Wrote: OutlerAdam-

I just updated mythicalLibrarian for the first time in at least a year. Now I am running into a small issue. Prior to the update, I would run a user job that runs handbrake to re-encode the video after the file has been renamed and transfered, and then the script MythDataDeleter to remove the info from the DB if the Handbrake job completed. Mythicallibrarian would still be listed as running in the job queue until Handbrake and MythDataDeleter completed. I previously used the 'SYMLINK=Move' setting.

After the update, it appears that I can skip the MythDataDeleter step, as you have the option for 'SYMLINK=Disabled', which can remove the DB info. However, when I run mythicalLibrarian now it moves the file to the proper location, begins Handbrake, and then deletes all references to the file from MythTV..... including the job that is running in the queue. Handbrake is still currently running if I do 'top', so it's not actually killing the job.....but MythTV no longer knows this job is active, and this may prevent me from queuing up a batch of videos that I need to rename/re-encode (I think I may end up with multiple Handbrake sessions trying to run at once).

Is there any way that I can correct this? Maybe delay the removal of Database information until after the User Job is complete?

Thanks.

{Edit} In the meantime, I just went back to using 'SYMLINK=MOVE' and MythDataDeleter {/EDIT}

(2012-10-13, 16:02)zipper456 Wrote: Thank you so much for making mythicallibrarian.

I have an issue: even though the episode is new, mythicallibrarian matches with the first on the list, so my links are being Season 1, Ep 1.

Here is the Log
*******************************************************************
Stable - Downloaded Thu Oct 11 02:00:22 CDT 2012
Attemting to pull database information on /var/lib/mythtv/recordings/2081_20121010200000.mpg
COMMERCIAL DATA START:50559 STOP:55598
/usr/local/bin/mythicalLibrarian: line 718: None=: command not found
RECSTART:2012-10-10 20:00:00 DATE: None
PROGRAMID: ShowCategory:
PLOT: Schooled: Claire and Phil embarrass Haley when they drop her off at college; Manny makes Gloria and Jay take a class; Lily gets into a tussle on her first day of kindergarten.
PERFORMING MAINTENANCE ROUTINE...
SEARCHING: www.TheTvDb.com SHOW NAME: Modern Family EPISODE:
TESTING FOR ZAP2ITID MATCH:EP01158124 Modern Family
FOUND:Modern Family ID#:95011 WITH CONFIDENCE:-1
DATABASE IS OUT OF DATE. NEXT UPDATE 1350007525 Current 1350136708
Parsing Downloaded information: Modern Family.xml
AIRDATE FAILED. TITLE MATCH ON ABSOLUTE EPISODE NUMBER:
TOTAL MATCHES:14 TOTAL VERIFIED FUZZY ERRORS:0
EPISODETongueilot EPISODE NUMBER:2 S01E01 WITH CONFIDENCE: -1
WARNING: FILE IS A LINK FORCING SYMLINK LINK MODE
CONFIDENCE:-2 LOGIC PATH: Attempted Match On Zap2itID: Failed; Attemted match on Standard Logic: Sucessful; Attemped match on OriginalAirDate: Failed; Attemted SubTitle match on Standard logic: Sucessful;
JOB: mythicalLibrarian '/var/lib/mythtv/recordings/2081_20121010200000.mpg' 'Modern Family' ''
TARGET SET:/var/lib/mythtv/recordings/Episodes/Modern Family/Season 1/Modern Family S01E01 (Pilot).mpg
FILE NAME EXISTS. FILE WILL BE KNOWN AS: Modern Family S01E01 (Pilot)-2
CREATING LINK IN LINK MODE WITH CONFIDENCE:-2
SENDING REQUESTED COMMANDS TO:192.168.99.190:8080
RSS generation failed:
File exists but is not writeable: /var/www/mythical-rss/rss.xml
SUCCESSFUL COMPLETEION TYPE: LinkModeSuccessful
******************************************************************************

(2012-10-29, 05:00)TechnoSwiss Wrote: Okay, so I've got a couple of updates to share. Initially I was playing with getting TVRage working, then I decided I really wanted to be able to use some local XML files for my shows, instead of downloading them. So I added an option to skip downloading for some shows and use local files instead.

So the first change, to allow using a file called ~/.mythicalLibrarian/skipDownload the contains a list of shows you want to use a local XML instead downloading from thetvdb.com

For the example I'll use Masterpiece Classic

~/.mythicalLibrarian/skipDownload
Code:
##############################################################
# List of Shows to use local XML files instead of thetvdb    #
# The following files must exist for each show               #
# <show>.working.xml - thetvdb seach listing for the show    #
# <show>.xml - thetvdb episode listing for the show          #
##############################################################
Masterpiece Classic

~/.mythicalLibrarian/Masterpiece Classic.working.xml
Code:
<?xml version="1.0" encoding="UTF-8" ?>
<Data>
<Series>
<seriesid>8256</seriesid>
<language>en</language>
<SeriesName>Masterpiece Classic</SeriesName>
<banner>graphical/78257-g.jpg</banner>
<Overview>Masterpiece Theatre premiered on PBS... </Overview>
<FirstAired>1971-01-01</FirstAired>
<IMDB_ID>tt0066684</IMDB_ID>
<zap2it_id>SH002814</zap2it_id>
<id>78257</id>
</Series>
</Data>

~/.mythicalLibrarian/Masterpiece Classic.xml
Code:
<?xml version="1.0" encoding="UTF-8"?>
<Data><Series>
  <id>78257</id>
  <Actors>|Alistair Cooke|Russel Baker|</Actors>
  <Airs_DayOfWeek>Sunday</Airs_DayOfWeek>
  <Airs_Time>9:00 PM</Airs_Time>
  <ContentRating/>
  <FirstAired>1971-01-01</FirstAired>
  <Genre>|Drama|</Genre>
  <IMDB_ID>tt0066684</IMDB_ID>
  <Language>en</Language>
  <Network>PBS</Network>
  <NetworkID/>
  <Overview>Masterpiece Theatre premiered on PBS...  </Overview>
  <Rating>9.0</Rating>
  <RatingCount>1</RatingCount>
  <Runtime>1</Runtime>
  <SeriesID>8256</SeriesID>
  <SeriesName>Masterpiece Classic</SeriesName>
  <Status>Continuing</Status>
  <added/>
  <addedBy/>
  <banner>graphical/78257-g.jpg</banner>
  <fanart>fanart/original/78257-1.jpg</fanart>
  <lastupdated>1345225860</lastupdated>
  <poster>posters/78257-1.jpg</poster>
  <zap2it_id>SH002814</zap2it_id>
</Series>
<Episode>
  <EpisodeName>Downton Abbey, II (1)</EpisodeName>
  <EpisodeNumber>1</EpisodeNumber>
  <FirstAired>2012-01-08</FirstAired>
  <SeasonNumber>41</SeasonNumber>
</Episode>
<Episode>
  <EpisodeName>Downton Abbey, II (2)</EpisodeName>
  <EpisodeNumber>2</EpisodeNumber>
  <FirstAired>2012-01-15</FirstAired>
  <SeasonNumber>41</SeasonNumber>
</Episode>
</Data>

and finally, the updates to mythicalLibrarian to support this

Code:
Replace:
        curl -L -s -m"$Timeout" "http://www.thetvdb.com/api/$APIkey/series/$SeriesID/all/$Language.xml"| grep -vE "(</id>|</Combined_episodenumber>|</Combined_season>|</DVD_chapter>|</DVD_discid>|</DVD_episodenumber>|</DVD_season>|</Director>|</EpImgFlag>|</GuestStars>|</IMDB_ID>|</Language>|</Overview>|</ProductionCode>|</RatingCount>|</filename>|</seasonid>|</Writer>|</Rating>|</lastupdated>|</seriesid>|</absolute_number>)">"$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest" && echo "$CurrentTime" > "$mythicalLibrarian/$NewShowName/lastupdated.time"

With:
        skipDownload=`grep "$ShowName" "$mythicalLibrarian/skipDownload"`
        if [ "$skipDownload" != "" -a -f "$mythicalLibrarian/$ShowName.xml" ]; then
            cp "$mythicalLibrarian/$ShowName.xml" "$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest"
        else
            curl -L -s -m"$Timeout" "http://www.thetvdb.com/api/$APIkey/series/$SeriesID/all/$Language.xml"| grep -vE "(</id>|</Combined_episodenumber>|</Combined_season>|</DVD_chapter>|</DVD_discid>|</DVD_episodenumber>|</DVD_season>|</Director>|</EpImgFlag>|</GuestStars>|</IMDB_ID>|</Language>|</Overview>|</ProductionCode>|</RatingCount>|</filename>|</seasonid>|</Writer>|</Rating>|</lastupdated>|</seriesid>|</absolute_number>)">"$mythicalLibrarian/$NewShowName/$NewShowName.xmlTest" && echo "$CurrentTime" > "$mythicalLibrarian/$NewShowName/lastupdated.time"
        fi

Replace:
        curl -L -s -m"$Timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$tvdbshowname>"$mythicalLibrarian/working.xml"

With:
        skipDownload=`grep "$ShowName" "$mythicalLibrarian/skipDownload"`
        if [ "$skipDownload" != "" ] && [ -f "$mythicalLibrarian/$ShowName.working.xml" ]; then
            cp "$mythicalLibrarian/$ShowName.working.xml" "$mythicalLibrarian/working.xml"
        else
            curl -L -s -m"$Timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$tvdbshowname>"$mythicalLibrarian/working.xml"
        fi

Having the <showname>.working.xml might be overdoing it a bit, but it looked easier to include that instead of bypassing the logic that was already in place.

The <showname>.xml file contains all your episode information, you can start with the XML from thetvdb.com and modify it to suit your need. In my case since thetvdb.com removed Masterpiece, I just grabbed an old copy of the XML I had from before they removed it, and added in this seasons shows.

Cool. I don't know how to integrate this.. What do you recommend, and why would someone want to use it instead of TVDB?


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - sdsnyr94 - 2012-11-11

Quote:I don't know what handbrake does. You can launch handbrake as a mythicalLibraian job if you'd like. User Jobs are documented in the wiki with it's own page.

I am running Handbrake as a JobSuccessfull job..... My issue was that with 'SYMLINK=Disabled', all the Database info was being removed PRIOR to the launch of the User Job... so according to MythBackend there was no longer a MythTV User Job running (because you have now just removed all information about that program), and it would begin the next MythTV User Job.

Is there a way that you can delay the removal of DataBase information until after the JobSuccessfull user job has completed?


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - sdsnyr94 - 2012-11-28

Anyone using this with 0.26? Any issues?


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - theli - 2012-12-26

I'm also interested in using mythicalLibraian with version 0.26. Has anyone tried it?


RE: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - GregoryK - 2012-12-26

I am using it with 0.26 successfully. Shortly after I upgraded from 0.25, mythicalLibrarian threw some errors when running (but, strangely, not immediately after.) I simply ran mythicalLibrarian --update and went through the update process and it has worked fine thereafter.

/Greg