XBMC Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for XBMC (/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)



- Bueller - 2010-01-13 06:40

Awesome, I'll start using the unstable version and let you know if I run into any issues. Thanks.


- jaygardner - 2010-01-15 09:25

Tried mythicalLibrarian tonight after MythSExx failed to find a show. was hoping the airtime match would kick in and find it...

dunno if this is your 1 in 400 problem or another punctuation issue, but this show name has a colon in its name...

here's the log (i ran it manually since i haven't set it up in myth yet)...

Code:
creating home/mythicalLibrarian and log file
SEARCHING: www.TheTvDb.com SHOW NAME: Law & Order: Special Victims Unit EPISODE: Shadow
FILE NAME: /media/datadrive/tv-recordings/3091_20100113195800.mpg
creating home mythicalLibrarian and log file
SEARCH FOUND:Martial Law ID#: 71240
DEFINED ABSOLOUTE EPISODE NUMBER:
###################DEBUG MODE ENABLED####################
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/jay/mythicalLibrarian-
MOVE DIR:/media/datadrive/TVShows- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/jay- ALTERNATE MOVE DIR:/home/jay/Shared-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Law & Order: Special Victims Unit- LOCAL SHOW NAME TRANSLATION:- SENT TVDB SHOW NAME:Law%20&%20Order:%20Special%20Victims%20Un$
RESOLVED SERIES ID:71240- RESOVED SHOW NAME:Martial Law-
INPUT EPISODE NAME:Shadow- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:Shadow-
SEASON:- EPISODE:- SYMLINK MODE:MOVE- FILESIZE: 4236841'kB'
FLAGS: MOVEDIR:1- FREE:603863660'kB'- ALTERNATEMOVEDIR:1- FREE:203847108'kB'-
PRIMARYMOVIEDIRWRITABLE:0- FREE:'kB'- ALTERNATEMOVIEDIR:1- FREE:203847108'kB'-
*DATABASE* RECSTART:2010-01-13 19:58:00- MOVIE YEAR:- ORIGINAL SERIES DATE:2010-01-13-
ProgramID:EP003169780257- CHANNEL ID:3091- CATEGORY:Crime drama-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
PLOT: Detectives Benson and Stabler speak to the daughter of a powerful couple who were found murdered in their bedroom.
##############LISTING FOLDER PERMISSIONS#################
ORIGIONAL FILE>ls -l /media/datadrive/tv-recordings/3091_20100113195800.mpg
-rw-r--r-- 1 jay jay 4338526008 2010-01-14 23:56 /media/datadrive/tv-recordings/3091_20100113195800.mpg
mythicalLibrarian WORKING DIR>lsmod -l /home/jay/mythicalLibrarian/Martial Law/
total 0
-rw-r--r-- 1 jay jay 0 2010-01-15 00:11 current.time
####################END OF DEBUG LOG#####################
%%%%%%%%%%www.TheTvDB.com information is incomplete Law & Order: Special Victims Unit, Shadow
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%Please consider helping out and adding to thetvdb%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%OPERATION FAILED Fri Jan 15 00:11:02 MST 2010 %%%%%%%%%%%%%%%%%



- outleradam - 2010-01-15 17:42

Looks like a punctuation issue. I can correct that easily.

The unstable version mentioned at the bottom of the first page of the support page would probly resolve that issue assuming thetvdb has a zap2it i'd for the show.

I will address that issue tonight.


- outleradam - 2010-01-16 05:09

This is the output from the unstable version with no database information. It seems to be a bit more stable then the older version.
Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Fri Jan 15 21:02:56 CST 2010@@@@@@@@@@@@@
SEARCHING: www.TheTvDb.com SHOW NAME: Law & Order: Special Victims Unit EPISODE: Shadow
FILE NAME: /home/adam/Desktop/new file .ext
SEARCH FOUND:Martial Law ID#: 71240
SEARCH FOUND:Law and Disorder ID#: 70458
SEARCH FOUND:Common Law ID#: 70898
SEARCH FOUND:Law & Order: Special Victims Unit ID#: 75692
SEARCH FOUND:Law & Order: Criminal Intent ID#: 71489
SEARCH FOUND:Murphy's Law ID#: 73425
SEARCH FOUND:Carson's Law ID#: 71995
SEARCH FOUND:Owen Marshall, Counselor At Law ID#: 75270
SEARCH FOUND:Law & Order ID#: 72368
SEARCH FOUND:Law of the Land ID#: 76981
SEARCH FOUND:The Character of Physical Law ID#: 127671
SEARCH FOUND:Marshall Law ID#: 78216
SEARCH FOUND:Law & Order: UK ID#: 85228
SEARCH FOUND:Law of the Plainsman ID#: 78445
SEARCH FOUND:Against the Law ID#: 78661
SEARCH FOUND:New Street Law ID#: 80375
SEARCH FOUND:Burke's Law ID#: 76697
SEARCH FOUND:L.A. Law ID#: 72419
SEARCH FOUND:I Fought the Law ID#: 73308
SEARCH FOUND:Brink Of Law ID#: 84397
SEARCH FOUND:Canterbury's Law ID#: 81502
SEARCH FOUND:Law and Order (1978) ID#: 91651
SEARCH FOUND:Sutherland's Law ID#: 73340
SEARCH FOUND:Family Law ID#: 73686
SEARCH FOUND:Harvey Birdman, Attorney at Law ID#: 77119
SEARCH FOUND:Law & Order: Trial by Jury ID#: 74191
SEARCH FOUND:McClain's Law ID#: 78296
SEARCH FOUND:Garrow's Law - Tales from the Old Bailey ID#: 122281
SEARCH FOUND:Louis Theroux - Law and Disorder ID#: 116971
SEARCH FOUND:The Law of Ueki ID#: 76031
SEARCH FOUND:Burke's Law (1994) ID#: 77692
SEARCH FOUND:The Law & Harry McGraw ID#: 73736
SEARCH FOUND:Survivors's Law II ID#: 81435
SEARCH FOUND:Murphy's Law (1988) ID#: 76257
SEARCH FOUND:The Law Firm ID#: 75183
SEARCH FOUND:The Law and Mr. Jones ID#: 71816
DEFINED ABSOLOUTE EPISODE NUMBER: 240
###################DEBUG MODE ENABLED####################
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/adam/mythicalLibrarian-
MOVE DIR:/home/mythtv/NAS/Video/shows- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/adam- ALTERNATE MOVE DIR:/home/mythtv/Shared-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Enabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Law & Order: Special Victims Unit- LOCAL SHOW NAME TRANSLATION:- SENT TVDB SHOW NAME:Law%20&%20Order:%20Special%20Victims%20Unit-
RESOLVED SERIES ID:75692- RESOVED SHOW NAME:Law & Order: Special Victims Unit-
INPUT EPISODE NAME:Shadow- ABSOLOUTE EPISODE NUMBER:240- RESOLVED EPISODE NAME:Shadow-
SEASON:S11- EPISODE:E12- SYMLINK MODE:MOVE- FILESIZE: 0'kB'
FLAGS WORKINGDIR:1- FREE:1630516'kB'- ORIGINALDIR:1- FREE:1630516'kb'-
MOVEDIR:1- FREE:378304'kB'- ALTERNATEMOVEDIR:1- FREE:1630516'kB'-
PRIMARYMOVIEDIRWRITABLE:1- FREE:378304'kB'- ALTERNATEMOVIEDIR:1- FREE:1630516'kB'-
*DATABASE* RECSTART:- MOVIE YEAR:- ORIGINAL SERIES DATE:-
ProgramID:- CHANNEL ID:- CATEGORY:- Zap2itID:NO DATABASE ENTRY-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:- STARS: RATING:0
PLOT:
CREATING FOLDER: /home/mythtv/NAS/Video/shows/Law amp Order Special Victims Unit
MOVING FILE: /home/adam/Desktop/new file .ext to /home/mythtv/NAS/Video/shows/Law amp Order Special Victims Unit/Law amp Order Special Victims Unit.S11E12 (Shadow).ext
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%WROTE 0 LENGTH FILE%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%OPERATION FAILED Fri Jan 15 21:03:13 CST 2010 %%%%%%%%%%%%%%%%%
The error in that log is because I made a blank file for this job and it contained no information. The amp thing is being changed now...

in fact this is the newest one
Code:
FILE NAME EXISTS.  FILE WILL BE KNOWN AS: Law and Order Special Victims Unit.S11E12 (Shadow)-1
CREATING LINK IN LINK MODE
It still needs a bit more work before I make it the new official version and rewrite the documentation.

I'm uploading to sourceforge now.


- jaygardner - 2010-01-16 09:08

that does work to identify the show, but xbmc never updates the library to include that show.

I'm guessing that since it writes the show title out without the punctuation, xbmc doesn't match it with what tvdb shows...

almost like you need the opposite function of what showtranslations does - or better yet, (probably harder for you though), if the script strips out any punctuation during the match process, have it put it back in before it writes to disk...

---
update- i renamed the moved files, matching the punctuation to match what tvdb shows as the name of the series title, but xbmc still doesn't update the library to include that show...


- outleradam - 2010-01-16 20:21

Ok, then I've done what I can. I have put in a request to turn the word "and" into "%26" on failed scrapes, as a backup. Reference: http://forum.xbmc.org/showthread.php?tid=67569

Not all file systems handle "&" in the file name, but they all handle "and". %26 is the web friendly version of &, and the only way to put the data into thetvdb.com for recognition. This would be a XBMC issue as a library should be human readable, the computer should turn the human readable stuff into computer stuff. "and" "&" "%26" "#38;" and "&" are different file systems handling of "and"


- jaygardner - 2010-01-16 22:38

well, i spoke to soon - i tried one more permutation which worked. renaming the files (& Show directory) to Law & Order Special Victims Unit results in a tvdb match to Law & Order: Special Victims Unit. The colon was throwing it off, but interestingly it shows up in my xbmc library with the colon intact in the show name.

Moreover, i've tried this in both windows and mac XBMC environments with same results.

Guessing this was tvdb developers simple fix for fact that not all file systems accommodate punctuation (at least : ) in file names?

So currently, mythicalLibrarian is writing "&" as "and", and for this show we need to drop the colon from the name. Is this too much of a corner case to fix?

if so, see if this logic on my end makes sense to you...

another thing i found poking around in tvdb... i guess because there's no exact match for the full Law & Order Special Victims Unit title, ML can't simply get the series ID (it returns a page and a half of various unrelated titles although the title of interest is on the page about 1/4 way down)... but i found that if i searched instead just for Special Victims Unit, it returns just the series info in question.

So perhaps what i need to do is create a show translation where:
Law & Order: Special Victims Unit = Special Victims Unit, let ML process that and find the series, let it write out the files, and then i simply create another script to move and change the names of the files, which effectively drops the colon out of the name, if needed, and ensures the name uses "&" instead of "and". I'd only run this script for this show...


- outleradam - 2010-01-16 23:49

Let me wrap my head around this for a bit.

Make sure I'm understanding what is going on here... XBMC does not recognize Law and Order. mythicalLibrarian translates Law & Order into Law and Order. XBMC has a problem with Law & Order: Special Victims Unit.

I will come up with some sort of work-around for this. using & in bash is a control charactor.


- jaygardner - 2010-01-17 00:21

you have it correct... & taking it one more step, xbmc does recognize Law & Order Special Victims Unit (no colon)

thanks, jay


- tim- - 2010-01-17 00:30

outleradam:

I have yet to get this working, but I also havent spent much time with it yet. I am getting errors writing the files over to my nfs mount I believe. I should be able to get that resolved fairly easily.

I had another question. For my xbmc boxes, is there a way to set the username and password? Can I just set that as part of the URL (user:pass@ip)?

tim