MythTV XBMC renamer.

theophile · 2009-12-03 21:46

outleradam Wrote:the ring buffer gets processed by local mythtv jobs?

interesting.. does the ring buffer get show squeezed? I'm transferring from windows BTV and I don't know the proper term.. show squeeze = when you record a show in mpg format, then compress it to avi or wmp.

This program, when finished, will only output "SxxExx" So, if the rename fails, it will still be moved and renamed showname/epname. This program is not designed to work independently. It will either get the information, or fail quietly. I think that's the proper way of doing it.

Either way, I don't want recordings on the HD. They need to go on my NAS, which will be mounted to the optimized folder after show squeeze so that I can maintain a rotating library.

MythTV does not differentiate between file recorded in live TV mode and scheduled recordings in terms of what it writes onto the HD. If you were to run mythrename.pl on your storage directory, I believe it would attempt to rename every file, including those still present from the ringbuffer. While this works, IMHO, it injects an unnecessary risk of DB corruption. I may have misunderstood what you said you intended to do, but I understood you to be saying that you wanted to use your script to generate new filename which you would then pass to mythrename.pl, running mythrename.pl as the primary operation. Even if it's possible to run mythrename.pl as a user job and specify the output of another bash script as one of the arguments, I think it's cleaner to use a wrapper script that includes the desired behavior of mythrename.pl.

You also mentioned show squeezing, which isn't a term I'm familiar with, but it looks like you're referring to transcoding of recordings. MythTV does not do this by default. MythTV does have transcoding profiles for most tuner types (HD-PVR excluded) and this is the operation that can be used to compress recordings and excise commercials.

In my case, I record, watch, then delete most shows I follow. On occasion, certain shows and movies I want to keep around on a more long-term basis. So for most shows, I don't care what MythTV calls them and I don't need/want to watch them in XBMC. So my script is really designed as a lossless archival script for HD-PVR recordings. Even though we have different applications, I still think you could benefit from the string matching subroutines.

outleradam · 2009-12-03 22:46

By string matching, do you mean removing case and small words like the, and, episode and punctuation? That can be done in a single line of code with tr -d.

Mythrename can be run as a targeted job on a file.

theophile · 2009-12-03 23:04

outleradam Wrote:By string matching, do you mean removing case and small words like the, and, episode and punctuation? That can be done in a single line of code with tr -d.

No, I mean that strings are submitted to TVDB and the script then compares the results to the input string and returns the result closest to the input. This means it will (usually) find the correct result even though the input has a typo, missing or extra punctuation, too many consecutive capitals, etc., etc. Some eventualities you can plan for with string replacement rules in the script, but if that's all you do, as soon as the script gets something that doesn't match exactly and that isn't handled by one of the predefined filters, it will die or return nothing.

outleradam · 2009-12-04 01:58

well, the name of the show will be covered, I will see how to go about episode identification. I think removing small words and making the db lower case should work for 99.9% of the shows. You are right about the typos and my program will never handle typos.

are you saying that you've never had a show which did not match what it was supposed to be?

theophile · 2009-12-04 02:05

outleradam Wrote:are you saying that you've never had a show which did not match what it was supposed to be?

Correct. Then again, I haven't intentionally given it wrong names either. You'll probably be fine for most shows, especially if your input is being grabbed from the MythTV DB which in turn comes form the EPG grabber (SD or other).

outleradam · 2009-12-04 07:12

currently it is broke, however, i will troubleshoot it this weekend. Hopefully I will make some good progress. Just have to finish rewriting the database portion and getting error handling handled and it will be ready to start renaming. I will see about implementing a string error matching script as well.

Code:

#/bin/sh

#BashSExx by Adam Outler

#

#usage is as follows-  BashSExx.sh -v "show name" "episode name"  

#  eg. BashSExx.sh "South Park" "Here Comes the Neighborhood"

#

#

#

#Log file will show information for troubleshooting. You can find the log file in the working folder

#Default: /home/$username/BashSExx

#If the API key does not work you can Register for a new API key on www.TheTvDb.com

#

#Dependencies:file depends on "Curl" install curl with "apt-get install curl"  

#Settings:

#the following line contains the API key from www.TheTvDb.Com: get a new one http://thetvdb.com/?tab=apiregister

apikey="3B81A89C73E4EE95"

#The following line is the internet access timeout in seconds: Default timeout=50 

timeout=50

#The following line is the BashSExx txt file Database folder: BashSExx=BashSExx  "~/BashSExx"

BashSExx=BashSExx

####################################################

###########Do not modify below this line############

####################################################

#make our home dir if it does not exist

if [ ! -d "$BashSExx" ]; then

    mkdir BashSExx

    echo "creating home/BashSExx and log file"> BashSExx/output.log

fi

#Set epn and shn from the input parameters

showname=$1

epn=$2 

#todo reference BashSExx.config file for name changes

#epn = cat grep

#change series name to one which is friendly for URL usage

showname=`echo $showname|replace " " "%20"`

#note the show for the log

echo "checking for show named" $shn "on TVDB.com the episode name is" $2 >> $BashSExx/output.log

#send show name and get series id from TVDB

seriesid=`curl -s -m"$timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$showname|grep -m 1 "<seriesid>"`

#clear out the junk from the last operation if it needs to be cleared.

if [ -f $BashSExx/working.xml ]; then

    rm $BashSExx/working.xml

fi

if [ -f $BashSExx/shn.txt ]; then

    rm $bashSExx/shn.txt

fi

if [ -f $BashSExx/sid.txt ]; then

    rm $BashSExx/sid.txt

fi

#download series info for show

#curl -s -m"$timeout" http://www.thetvdb.com/api/$apikey/series/$seriesid/all/en.xml > $BashSExx/working.xml

#create a folder/file "database" Strip XML tags.  Series, Exx and Sxx are separated into different files

echo $epn $showname $seriesid >>$BashSExx/output.log

#if1: jump to near end of program if no series id obtained 

if [ "$seriesid" != "" ]; then

#debug line

echo "series id is not blank"

    #remove XML tags

    seriesid=`echo $seriesid|tr -d "<seriesid>"|tr -d "</seriesid>"`

    echo "got series ID for $1 now downloading show information">>$BashSExx/output.log

###TODO this section will be skipped if our db is up to date

###need to save first xml to $BashSExx working dir for reference

### while I'm at it, split the db into a list of names and showids, then get correct name based on show id.  

### a show id log generator would be nice.

###if last db update time is newer then our last update, then delete show and get new db

###get current tvdb time and save to $BashSExx/$showname/time

#    echo creating/recreating $BashSExx/$showname/$showname.xml>>$BashSExx/output.log

#    if [ -d $BashSExx/$showname ]; then

#        rm $BashSExx/$showname/*

#    else 

        mkdir $BashSExx/$showname

#    fi 

#download series info for show

    curl -s -m"$timeout" http://www.thetvdb.com/api/$apikey/series/$seriesid/all/en.xml > $BashSExx/$showname/$showname.xml

#create a folder/file "database" Strip XML tags.  Series, Exx and Sxx are separated into different files

    cat $BashSExx/$showname/$showname.xml | grep "<EpisodeName>"|replace "<EpisodeName>" ""|replace "</EpisodeName>" "">$BashSExx/$showname/$showname.Ename.txt

    cat $BashSExx/$showname/$showname.xml | grep "<SeasonNumber>"|replace "<SeasonNumber>" ""|replace "</SeasonNumber>" ""|replace " " "">$BashSExx/$showname/$showname.S.txt

    cat $BashSExx/$showname/$showname.xml | grep "<EpisodeNumber>"|replace "<EpisodeNumber>" ""|replace "</EpisodeNumber>" ""|replace " " "">$BashSExx/$showname/$showname.E.txt

#debug line

    echo "show parsed"

#grep the episode name in the episode name list, output only the line number

    linenumber=`grep -nm 1 "$epn" $BashSExx/$showname/$showname.Ename.txt| sed 's/:.*//'`

echo $linenumber >> OUTPUT.TXT

#if2: if line match is obtained, then continue

    if [ $linenumber =  ""  ]; then

#add a 'p' after the line number for use with sed.

        linenumber=$linenumber"p"

        echo linenumber is not null it is $linenumber >>$BashSExx/output.log

#gather series and episode names from files created earlier.

        exx=`sed -n $linenumber $BashSExx/$showname/$showname.E.txt`

        sxx=`sed -n $linenumber $BashSExx/$showname/$showname.S.txt`

# Single digit episode and show names are not allowed Ex and Sx replaced with Exx Sxx

        if [ "$exx" -lt "10" ]; then 

            exx=`echo E0$exx`

        elif [ "$exx" -gt 9 ]; then 

            exx=`echo E$exx`

        fi

        if [ "$sxx" -lt 10 ]; then 

            sxx=`echo S0$sxx`

        elif [ "$sxx" -gt 9 ]; then 

            sxx=`echo S$sxx`

        fi

#debug line

        echo "finished processing"

#if2 exit point

    fi

#debug line

echo endif2

#if1 exit point

elif [ "$seriesid" == "" ]; then 

echo "series was not found the tvdb may be down" >> $BashSExx/output.log

fi

#debug line

echo endif1

#output format  shn=showname sxx=seasonnumber exx=episodenumber epn=episodename

echo got episode information: $shn - $sxx$exx "($epn)" >> $BashSExx/output.log

echo $sxx$exx

#export  $sxx

#export  $exx

outleradam · 2009-12-04 17:44

I did some research and found agrep does fuzzy matching

outleradam · 2009-12-05 03:52

added fuzzy logic

Code:

#/bin/sh

#BashSExx by Adam Outler

#

#usage is as follows-  BashSExx.sh -v "show name" "episode name"  

#  eg. BashSExx.sh "South Park" "Here Comes the Neighborhood"

#

#

#

#Log file will show information for troubleshooting. You can find the log file in the working folder

#Default: /home/$username/BashSExx

#If the API key does not work you can Register for a new API key on www.TheTvDb.com

#

#Dependencies: depends on "Curl" and "agrep" 

#install curl with "apt-get install curl"  

#install agrep with "apt-get install agrep"

#Settings:

#the following line contains the API key from www.TheTvDb.Com: get a new one http://thetvdb.com/?tab=apiregister

apikey="3B81A89C73E4EE95"

#The following line is the internet access timeout in seconds: Default timeout=50 

timeout=50

#The following line is the BashSExx txt file Database folder: BashSExx=BashSExx  "~/BashSExx"

BashSExx=BashSExx

#No show names or episode names longer then 32 charactors

####################################################

###########Do not modify below this line############

####################################################

#make our home dir if it does not exist

if [ ! -d "$BashSExx" ]; then

    mkdir $BashSExx

    echo "creating home/BashSExx and log file"> BashSExx/output.log

fi

#Set epn and shn from the input parameters

showname=$1

epn=$2 

#todo reference BashSExx.config file for name changes and parameters

#epn = cat grep

#change series name to one which is friendly for URL usage

showname=`echo $showname|replace " " "%20"`

#set up folder for local show db storage

if [ ! -d "$BashSExx/$showname" ]; then

    mkdir $BashSExx/$showname

    echo "creating home/BashSExx and log file"> BashSExx/output.log

fi

#note the show for the log

echo "checking for show named" $shn "on TVDB.com the episode name is" $2 >> $BashSExx/output.log

#send show name and get series id from TVDB

seriesid=`curl -s -m"$timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$showname|grep -m 1 "<seriesid>"`

#working folder maintenance.  clear out if it exists.

if [ -f $BashSExx/working.xml ]; then

    rm $BashSExx/working.xml

fi

if [ -f $BashSExx/shn.txt ]; then

    rm $BashSExx/shn.txt

fi

if [ -f $BashSExx/sid.txt ]; then

    rm $BashSExx/sid.txt

fi

#download series info for show, parse into temporary text db- sid.txt shn.txt

#curl -s -m"$timeout" www.thetvdb.com/api/GetSeries.php?seriesname=$showname > $BashSExx/working.xml

#cat $BashSExx/working.xml | grep "<seriesid>"|replace "<seriesid>" ""|replace "</seriesid>" "">$BashSExx/sid.txt

#cat $BashSExx/working.xml | grep "<SeriesName>"|replace "<SeriesName>" ""|replace "</SeriesName>" ""|replace " " "">shn.txt

#use fuzzy logic to make the best match of the show name

#maybe fuzzy logic is not the best here because movies will be detected.

#serieslinenumber=`agrep -Byn "$1" $BashSExx/shn.txt| sed 's/:.*//'`

#get the seriesid based on the showname

#seriesid=`sed -n $serieslinenumber $BashSExx/sid.txt`

#newshowname=`sed -n $serieslinenumber $BashSExx/shn.txt`

#check for valid entry in series id

#if [ "$serieslinenumber" !=  ""  ]; then

#add a 'p' after the line number for use with sed.

#    serieslinenumber=$serieslinenumber"p"

#fi

#create a folder/file "database" Strip XML tags.  Series, Exx and Sxx are separated into different files

echo $epn $showname $seriesid >>$BashSExx/output.log

#if1 Series id is not blank so continue

if [ "$seriesid" != "" ]; then

#debug line

    echo "series id is not blank" $seriesid $newshowname

#remove XML tags

    seriesid=`echo $seriesid|tr -d "<seriesid>"|tr -d "</seriesid>"`

    echo "got series ID for $1 now downloading show information">>$BashSExx/output.log

###TODO this section will be skipped if our db is up to date

###need to save first xml to $BashSExx working dir for reference

### while I'm at it, split the db into a list of names and showids, then get correct name based on show id.  

### a show id log generator would be nice.

###if last db update time is newer then our last update, then delete show and get new db

###get current tvdb time and save to $BashSExx/$showname/time

#    echo creating/recreating $BashSExx/$showname/$showname.xml>>$BashSExx/output.log

    if [ -f $BashSExx/$showname/$showname.xml ]; then

        rm $BashSExx/$showname/$showname.xml

    fi

    if [ -f $BashSExx/$showname/$showname.Ename.txt ]; then

        rm $BashSExx/$showname/$showname.Ename.txt

    fi

    if [ -f $BashSExx/$showname/$showname.E.txt ]; then

        rm $BashSExx/$showname/$showname.E.txt

    fi

    if [ -f $BashSExx/$showname/$showname.S.txt ]; then

        rm $BashSExx/$showname/$showname.S.txt

    fi

    if [ ! -d $BashSExx/$showname ]; then

        mkdir $BashSExx/$showname

    fi 

#download series info for show

    curl -s -m"$timeout" http://www.thetvdb.com/api/$apikey/series/$seriesid/all/en.xml > $BashSExx/$showname/$showname.xml

#create a folder/file "database" Strip XML tags.  Series, Exx and Sxx are separated into different files

    cat $BashSExx/$showname/$showname.xml | grep "<EpisodeName>"|replace "<EpisodeName>" ""|replace "</EpisodeName>" "">$BashSExx/$showname/$showname.Ename.txt

    cat $BashSExx/$showname/$showname.xml | grep "<SeasonNumber>"|replace "<SeasonNumber>" ""|replace "</SeasonNumber>" ""|replace " " "">$BashSExx/$showname/$showname.S.txt

    cat $BashSExx/$showname/$showname.xml | grep "<EpisodeNumber>"|replace "<EpisodeNumber>" ""|replace "</EpisodeNumber>" ""|replace " " "">$BashSExx/$showname/$showname.E.txt

#debug line

    echo "show parsed"

#grep use fuzzy logic to find the closest show name from the locally created database

    linenumber=`agrep -Byn "$epn" $BashSExx/$showname/$showname.Ename.txt| sed 's/:.*//'`

#debug lines

    echo $linenumber >> OUTPUT.TXT

    echo $linenumber

#if2: if line match is obtained, then continue

    if [ "$linenumber" !=  ""  ]; then

#add a 'p' after the line number for use with sed.

        linenumber=$linenumber"p"

        echo linenumber is not null it is $linenumber >>$BashSExx/output.log

#gather series and episode names from files created earlier.

        exx=`sed -n $linenumber $BashSExx/$showname/$showname.E.txt`

        sxx=`sed -n $linenumber $BashSExx/$showname/$showname.S.txt`

# Single digit episode and show names are not allowed Ex and Sx replaced with Exx Sxx

        if [ "$exx" -lt 10 ]; then 

            exx=`echo E0$exx`

        elif [ "$exx" -gt 9 ]; then 

            exx=`echo E$exx`

        fi

        if [ "$sxx" -lt 10 ]; then 

            sxx=`echo S0$sxx`

        elif [ "$sxx" -gt 9 ]; then 

            sxx=`echo S$sxx`

        fi

#debug line

    echo "finished processing"

#if2 exit point

    fi

#debug line

    echo endif2

#if1 exit point

elif [ "$seriesid" == "" ]; then 

echo "series was not found the tvdb may be down try renaming $1" >> $BashSExx/output.log

fi

#debug line

echo endif1

#output format  shn=showname sxx=seasonnumber exx=episodenumber epn=episodename

echo got episode information: $shn - $sxx$exx "($epn)" >> $BashSExx/output.log

echo $sxx$exx

#export  $sxx

#export  $exx

outleradam · 2009-12-05 20:23

I consider this a milestone:

This is the log file from BashSExx

Quote:@@@@@@@@NEW SEARCH INITIATED AT Sat Dec 5 12:18:31 CST 2009@@@@@@@@@@
SEARCHING: http://www.TheTvDb.com SHOW NAME: south park EPISODE: spirt of christmas
SEARCH FOUND:SouthPark ID#: 75897
LOCAL DATABASE UPDATED:/home/adam/BashSExx/south%20park/
DEFINED ABSOLOUTE EPISODE NUMBER: 2
EPISODE INFORMATION: SouthPark - S00E02 ( The Spirit Of Christmas (Jesus vs. Santa))

Notice the name was "south park" not "SouthPark" and the episode name was mispelled, and also abbreviated to spirt of christmas

My program was able to translate"spirt of christmas" to "The Spirit Of Christmas (Jesus vs. Santa) "

I'm very happy. I'm continuing work now.

theophile · 2009-12-05 21:53

outleradam Wrote:Notice the name was "south park" not "SouthPark" and the episode name was mispelled, and also abbreviated to spirt of christmas

My program was able to translate"spirt of christmas" to "The Spirit Of Christmas (Jesus vs. Santa) "

I'm very happy. I'm continuing work now.

That is very awesome. Nice work!

theophile Fan Posts: 569 Joined: Jan 2009 Reputation: 30	2009-12-04 02:05 Post: #25 outleradam Wrote:are you saying that you've never had a show which did not match what it was supposed to be? Correct. Then again, I haven't intentionally given it wrong names either. You'll probably be fine for most shows, especially if your input is being grabbed from the MythTV DB which in turn comes form the EPG grabber (SD or other).
	find quote

theophile Fan Posts: 569 Joined: Jan 2009 Reputation: 30	2009-12-05 21:53 Post: #30 outleradam Wrote:Notice the name was "south park" not "SouthPark" and the episode name was mispelled, and also abbreviated to spirt of christmas My program was able to translate"spirt of christmas" to "The Spirit Of Christmas (Jesus vs. Santa) " I'm very happy. I'm continuing work now. That is very awesome. Nice work!
	find quote