Kodi Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



- iamhungry - 2010-04-20

outleradam Wrote:^^ It will require a separate database processor...

That sounds a bit more involved than I was thinking. I don't have any idea what the best way to get my recorded Japanese Tv shows into the XBMC db, but I was wondering if recording groups could be used.

Let me try to explain myself a little better. (Please remember at any point, I may have no idea what I am talking about!)

I want to stop flipping between Mythtv and XBMC, and use XBMC to watch everything.

Myth:// and mythbox are cool, but don't sort the Japanese named genres of course. I thought about trying to find the English genre words and replacing them with Japanese, but my EPG data is crap to start off with. The genres are rarely right, and change often.

Show names are a nightmare too. I have the discovery channel on cable (in Japan). For example, Mythbusters will be named "Mythbusters" one week, "You Tube Special MYTHBUSTERS” the next, and "怪しい伝説” another week.

Japanese Tv shows are the same. It may be ”あらびき団” one week, and "あらびき団SP" or some other variant another.

I thought your show translation settings would work, but the name changes would be a headache, and I would need to make a new translation rule for every show I wanted to record and watch in XBMC. (I have about 85 schedule rules right now)

Mythtv does a prettty good job finding shows, but I have to keep an eye on things to make sure nothing gets missed because of a wacky name.

All this chaos is viewable in the mythtv menus because of recording groups. I have the 85 rules divided into 10 recording groups. When I go to the "watch recordings" part of mythtv, the recordings are listed by recording group, not genre or show name.

Another example: My music recording group gets a lot of shows like, ジョディ・ワトリー 特集/ARTIST’S洋楽, コールドプレイ ビデオ, The Life and Rhymes of カニエ・ウェスト, and Unplugged:キーシャ・コール. Some of these shows are available in the US (unplugged) and data could be found, but others are just an hour of music videos from one artist. However, all these shows reside in the "music" recording group.
I use the metallurgy theme in mythtv, so the "music" recording group is on the left side of my recordings screen, and on the right is a long list of the different shows in that group.

I think you can see the trouble I would have trying to get each show into the XBMC database. Sure, a seperate "data input script" would be an answer, but another step seems unnecessary.

Couldn't recording groups be treated like shows and added to the XBMC db? So I fire up XBMC, go to tv shows, and see the daily show (from torrent), mythbusters (actually a recording group), music (recording group full of music video related stuff), The sopranos (from DVD), and news (a recording group that contains 3 shows).

Going into "news", "music" or "mythbusters" would show a list of shows with no meta data other than the name and air date. The sopranos and daily show would have all the data that XBMC can find.

I hope I have made my question a little more clear. I don't know if this is a good idea, or even possible, but thank you for taking the time to humor me.


- outleradam - 2010-04-20

^^ Making a new database processor is not a very complicated procedure. It's just a block of code in mythicalLibrarian which is about 200 lines. I can basically copy and paste what's there. change a few lines and create a new option for no TVDB Processing.

I would already be working on these, but I'm booting off of flash drives right now and cannot even install mythtv.


- jaygardner - 2010-04-22

Hi-
i've been using same version since January without any trouble, but recently, i've found that mythicalLibrarian won't succeed unless the show directory already exists. here's the log from a failure...

Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Wed Apr 21 23:41:00 MDT 2010@@@@@@@@@@@@@
%%%NO DATABASE INFORMATION. CHECK LOGIN/PASS OR FILE %%%%%
SEARCHING: www.TheTvDb.com SHOW NAME: Family Guy EPISODE: Fish Out of Water
FILE NAME: /media/datadrive/tv-recordings/12311_20100421225800.mpg
creating home mythicalLibrarian and log file
SEARCH FOUND:Family Guy ID#: 75978
USING FUZZY LOGIC FOR EPISODE RECOGNITION
No direct Zap2it match found. Please update TheTvDb.com
SEARCH FOUND:Family Guy ID#: 75978
DEFINED ABSOLOUTE EPISODE NUMBER:
###################DEBUG MODE ENABLED####################
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/mythicalLibrarian-
MOVE DIR:/media/datadrive/TVShows- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/jay/Shared/TVShows- ALTERNATE MOVE DIR:/home/jay/Shared/TVShows-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Family Guy- LOCAL SHOW NAME TRANSLATION:Inactive- ShowName:Family Guy
SENT TVDB SHOW NAME:Family%20Guy-
RESOLVED SERIES ID:75978- RESOVED SHOW NAME:Family Guy-
INPUT EPISODE NAME:Fish Out of Water- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:Fish Out of Water-
SEASON:- EPISODE:- SYMLINK MODE:MOVE- FILESIZE: 2985401'kB'
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:505209564kB- WORKINGDIR:1 Free:202979388kB-
MOVEDIR:1- FREE:505209564kB- ALTERNATEMOVEDIR:1- FREE:202979388kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:505209564kB- ALTERNATEMOVIEDIR:1- FREE:202979388kB-
DATABASE INFORMATION
RECSTART:2010-04-21 22:58:00- MOVIE YEAR:- ORIGINAL SERIES DATE:2001-09-12-
PROGRAMID:EP002960010039- CHANNEL ID:12311- CATEGORY:Animated-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:EP00296001- MATCHED TVDB SERIES ID:-
####################END OF DEBUG LOG#####################
%%%%%%%%%%www.TheTvDB.com information is incomplete Family Guy, Fish Out of Water
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%Please consider helping out and adding to thetvdb%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%OPERATION FAILED Wed Apr 21 23:41:04 MDT 2010 %%%%%%%%%%%%%%%%%

Any suggestions on what i should change?

the MOVEDIR, /media/datadrive/TVShows, permissions are:
drwxrwxrwx 13 mythtv jay 4096 2010-04-21 23:59 TVShows

thx, Jay


- outleradam - 2010-04-22

please run sudo mythicalLibrarian --update, choose SVN, hit enter to all options given, and then delete all database folders in ~/.mythicalLibrarian/


Everything looks good except that you're using an older version and it should have grabbed the fuzzy logic version because thetvdb.com has the wrong original air date for that episode. mythicalLibrarian may have a problem updating it's database in the version you are using.


- outleradam - 2010-04-22

Jay, unrecognized episodes are now added to the library and cycled into the --doover que for processing when information becomes available. Also, other new features have been added.


- jaygardner - 2010-04-22

thanks- will try when i get home later...

i didn't realize i had a version recent enough to run the updater... also, my install is still in ./mythicalLibrarian and not in ./.mythicalLibrarian

should i still run the updater or should i do a fresh install of mythicalLibrarian?

thx, jay


- jaygardner - 2010-04-23

i've some more questions -

it seems i have to install new - the january version i was using doesn't respond to the sudo mythicalLibrarian --update command - says it can't find mythicalLibrarian.

in the wiki the instructions are to

Quote:Place mythicalLibrarianin /usr/local/bin
mkdir /home/mythtv/.mythicalLibrarian
Make mythicalLibrarian executable by typing the following:
sudo chmod +x /home/mythtv/mythicalLibrarian

given above, wouldn't i want to put mythicalLibrarian in /home/mythtv/.mythicalLibrarian ?

also i'm confused by two move related directories:
MOVEDIR and PrimaryShowDir

Both of these are commented as the directory to which MythicalLibrarian will first try to move shows...

and you initialize them to different paths - /home/mythtv/Episodes and /home/mythtv/Showings. (I realize i have to change them, but wondering why have two variables and why they wouldn't be initialized to the same thing).

Finally, am guessing i can just delete my /home/mythtv/mythicalLibrarian directory and reinstall new - just need clarification on above (/usr/local/bin vs /home/mythtv/.mythicalLibrarian)

thx, jay


- outleradam - 2010-04-23

jaygardner Wrote:i've some more questions -

it seems i have to install new - the january version i was using doesn't respond to the sudo mythicalLibrarian --update command - says it can't find mythicalLibrarian.

in the wiki the instructions are to



given above, wouldn't i want to put mythicalLibrarian in /home/mythtv/.mythicalLibrarian ?
thanks for pointing that out. it's been changed.

Quote:also i'm confused by two move related directories:
MOVEDIR and PrimaryShowDir

Both of these are commented as the directory to which MythicalLibrarian will first try to move shows...

and you initialize them to different paths - /home/mythtv/Episodes and /home/mythtv/Showings. (I realize i have to change them, but wondering why have two variables and why they wouldn't be initialized to the same thing).
mythicalLibrarian went through some substantial changes since you last upgraded.
1. ability to add unrecognized non-tvdb shows
2. ability to identify season and episode for sports and news
These two implementations require a separate folder for categorization. The new folder does not get scraped against thetvdb and is recognized by XBMC by using S0E0(showing identifer) or a unique SyearEmonthday(categoric identifer). Both of these require a separate folder.
Episodes=recognized tvdb naming
Showings=unrecognized but filed recordings

Quote:Finally, am guessing i can just delete my /home/mythtv/mythicalLibrarian directory and reinstall new - just need clarification on above (/usr/local/bin vs /home/mythtv/.mythicalLibrarian)

thx, jay
mythicalLibrarian uses ./mythicalLibrarian for it's settings now. The script is located in /usr/local/bin to conform to Linux filesystem guidelines. I may be relocating settings to /usr/share/mythicalLibrarian. Either way... here's what you need:

Code:
sudo apt-get install curl
mkdir ~/.mythicalLibrarian && mkdir ~/.mythicalLibrarian/mythicalSetup && cd ~/.mythicalLibrarian/mythicalSetup
curl [url=http://mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh][color=#0000ff]http://mythicallibrarian.googlecode.com/svn/trunk/mythicalSetup.sh[/color][/url]>./mythicalSetup.sh
sudo chmod +x ./mythicalSetup.sh && sudo ./mythicalSetup.sh

You can rm -Rf ~/mythicalLibrarian if you would like. There is no cross-compatability between the old tracking/database system from January and the new. This was the primary reason for the tracking/database folder change

As of last night I was able to get my old Ubuntu running again with pretty much all new system components. No more booting from a flash drive. The last SVN commit was just a test of SVN submission. This weekend I intend to sit down and perform all the requested changes on the previous page of this thread.


- jaygardner - 2010-04-24

ok thanks...

looks like you also moved the settings etc from /home/mythtv/ to ~/ .... does it matter if i put it in my login id's directory structure vs /home/mythtv?<<<--- ignore, i think i figured this out

Also, on the wiki...
wrt the section titled:
Use of mythicalLibrarian on MythTV Applications

is that extra setup or setup for a different use case?

Finally, wrt:
# The user job can be called as follows:
# /usr/local/bin/mythicalLibrarian "%DIR%/%FILE%"
#
#Usage:
# mythicalLibrarian "Target File" "show name" "episode name"
# eg. mythicalLibrarian "/home/myth/recordings/2308320472023429837.mpg" "South Park" "Here Comes the Neighborhood"

We don't need to have myth call the user job with "%TITLE%" & "%SUBTITLE" any more?


- outleradam - 2010-04-24

SVN 310 was just committed.
Code:
/MoveDir/Episode Title/Season number/Episode Title SxxExx (subtitle).ext
Testing now


- jaygardner - 2010-04-24

i ran sudo ./mythicalSetup.sh and it seems to be in an endless loop - it keeps asking me if i want to install the librarian notify send script... i answer the question, and it verifies that some things are installed, and then returns to ask me the same question again...

I'm answering "no" to the send script...

thx,
jay


- outleradam - 2010-04-24

I am on a fresh install of Ubuntu and was able to reproduce the problem. i'm working on it now.


- outleradam - 2010-04-24

Well, for the first time in about 3 months mythicalLibrarian's distribution system is broken. Something in these lines is causing it.

Code:
while read line
     do
        test "$line" = " ########################## USER JOBS############################" && let startwrite=$startwrite+1 && echo WOOT        
     if [ $startwrite = 2 ]; then
             echo -e "$line" >> ./librarian
              echo $startwrite
     fi

I will be looking into this sometime tomorrow night. Keep in mind that there is still the manual setup until then.


- jaygardner - 2010-04-24

right, am trying to do manual install.

a few questions still...

from the wiki in the Use of mythicalLibrarian on MythTV Application Section

Code:
sudo apt-get install curl agrep libnotify-bin
sudo mkdir /home/mythtv /home/mythtv/mythicalLibrarian /home/mythtv/Episodes /home/mythtv/Movies /home/mythtv/Videos /home/mythtv/FailSafe
sudo chown -R mythtv:mythtv /home/mythtv /home/mythtv/mythicalLibrarian /home/mythtv/Episodes /home/mythtv/Movies /home/mythtv/Videos /home/mythtv/FailSafe
sudo mv ~/Desktop/mythicalLibrarian.sh /home/mythtv/mythicalLibrarian/mythicalLibrarian.sh
sudo mv ~/Desktop/librarian-notify-send /usr/local/bin/librarian-notify-send
sudo chmod +x /usr/local/bin/librarian-notify-send
sudo chmod +x /home/mythtv/mythicalLibrarian/mythicalLibrarian.sh

should the last line be:
sudo chmod +x /usr/local/bin/mythicalLibrarian
?
and given the initial setup putting mythicalLibrarian in /usr/local/bin, do we need the 4th line of above?

also, from wiki and code comments:
does this line:
Code:
User Job #1 Command: /usr/local/bin/mythicalLibrarian "%DIR%/%FILE%"
need to include "%TITLE%" and "%SUBTITLE%"


- jaygardner - 2010-04-24

ok, i think i got it set up, i was confused earlier because you're using the notation ~/ - which i understand indicates a directory in my home folder path, but when the job runs as a myth job, ~ gets interpreted as /home/mythtv - which is not my login user account. So if I run any of the setup commands from my login account, the directories aren't getting setup under the mythtv user - but the myth user job is looking for them to be under /home/mythtv...

& if i manually run mythicalLibrarian from my login account, i need a parallel set of directories for that account in /home/login-account structure

So not sure what your intention is &/or if it matters whether the .myhthicalLibrarian dir and the other dirs are part of my /home/mythtv hierarchy or my /home/login-account hierarchy, but it is working...

thanks, Jay