mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thread Closed
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #16
Holy cow... alot of responses tonight. I was on my iPhone and I didn't see past that last one.

tim- Wrote:outleradam: I finally had a chance to set this up today. I have a few jobs running over the next hour and we'll see how it goes.

I think your wiki page needs a little cleaning up (still refers to MythSExx on one line) but other than that it seemed easy enough to get going. Also, there is a screenshot with some labels that doesnt have a larger version so its too small to actually read.

I will take care of that MythSExx line. There are problems with the wiki as noted in this thread: http://forum.xbmc.org/showthread.php?tid=65651 , and images cannot be linked properly. click the links below the image until the problem with the wiki software is fixed and I can update the page. I am considering moving the page to the Google code page.

jaygardner Wrote:just a clarification on difference between MythSExx and mythicalLibrarian...

both move shows & optionally do notifications, but ML also does movies, ML also moves commercial files. & ML does airtime matching if the fuzzy logic match fails. anything significant i'm leaving out?
It also works offline. That's about it as far as functionality. Additional improvements are better logging and improved regex recognition and better naming of the files. It also includes a daily report and a improved doover.sh generation.

Creation of a NFO XML for unscrapable files is not very far off after XBMC dev team lets me know what information they can actually use without season and episode.

I tend to think of MythSExx as "subtitle2season".. a very minimalist bash soloution when you don't have airdate for air2season.py or access to mythconverg. mythicalLibrarian does it all.

Quote:a question - if you delete a show from within xbmc, is ML doing any cleanup so that all of the files on the myth box (symlink, pngs), the actual video file (wherever it is as a result of the move), & all of the commercial files get deleted or is there manual clean up needed?

Also, should the mythconverg database detect this deletion?
mythicalLibrarian tracks every comskip file it creates. It will delete it's comskip file as a part of the maintenance routine if it is the only file of that basename in it's folder.
Quote:what would be the recommended way to delete files if not through xbmc... personally, i'm not a fan of letting stuff just age out - i'd rather explicitly delete it when done with it...

thanks, jay
Just delete it explicitly. You can delete the file if you want, but it's best to go through MythTV because it will still be in the database. Also, MythTV will delete renamed recordings via the symlink provided in mythicalLibrarian. Just make sure you have it set up like I specified in the wiki. mythicalLibrarian will perform it's maintenance routine on it's next run Smile

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
(This post was last modified: 2010-01-09 06:24 by outleradam.)
find
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #17
Jaygardner: I think you could get more out of your setup by using autoexpire and the mythweb interface. I have mine set up to keep 20GiB of free space on my computer. I have yet to delete anything but if I did, it would be through the MythTV interface, otherwise MythTV would still maintain it's database. If you go through Mythweb, MythTV will delete it's database entries, the symlink, and the original file for sure. mythicalLibrarian will clean up it's generated comskip files. I just let everything run and it runs smoothly.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #18
After further research I determined that better orphan handling was not needed. mythicalLibrarian is working properly the way it is. mythicalLibrarian will not generate files of the same name with different extensions unless the user generates them, or enables recording of duplicate shows and enables transcoding of only one of them. In this case, there is only an allowance for one comskip file anyway and the new comskip will overwrite the old one.

If the user deletes a file instead of going through MythTV, the png files and the database entries in MythTV will be left. This is the same behavior that MythTV would exibit if the user had deleted the original recording file before mythicalLibrarian had touched it. Therefore, mythicalLibrarian does not operate any differently then the MythTV core would. mythicalLibrarian just moves the file to a XBMC/Boxee recognizable folder and tells MythTV where to find the file for access and deletion.

Just to clarify:
MythTV handles it's png files, the symlink and the actual movie file.
mythicalLibrarian will clean up it's generated comskip files.

If you let MythTV auto-expire videos, when MythTV hits it's maximum space allowance, it will delete all recording files without a trace. Once the recording file is deleted, mythicalLibrarian will perform it's maintenance routine on the next cycle and delete the comskip file.

Currently I am working on folder tracking. I have a few empty folders which were created by mythicalLibrarian. This shows up in the XBMC library as a show with zero episodes. This is the responsibility of mythicalLibraian to clean up. This will be included in the next release.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
(This post was last modified: 2010-01-12 03:21 by outleradam.)
find
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #19
I am currently working on Zap2it/SchedulesDirect recognition of series ID as per request of this post: http://forum.xbmc.org/showpost.php?p=481812&postcount=6

If you search through TheTvDb, you will find that alot of those Zap2it IDs are missing. Zap2it and SchedulesDirect supply the same data.

With that said, please update your favorite shows in the open database theTvDb.com. Just sign up for an account and add the schedules direct ID. This can be found at the zap2it site:
http://tvlistings.zap2it.com/tv/wordgirl/EP00948842
You just take the last bit of the data, ie. EP00948842 and copy/paste it into theTvDb.com like I did here: http://thetvdb.com/?tab=series&id=113781&lid=7

showtranslations will still be used to correct spelling errors for the initial lookup and full guide information for those who do not have guide data through mythtv.

I would like to get a few examples of iTivo guide data as well, I need a few "sh"s "ep"s and "mv" examples.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find
malte Offline
Skilled Python Coder
Posts: 1,346
Joined: Jan 2010
Reputation: 28
Location: Germany
Post: #20
I'm using your script about a week now and it gets data for most of my shows. Thanks a lot! As mentioned in an earlier post by pinthenet you could think about adding some language support. I have german EPG data and mythtv sends german title and subtitle to your script. So it won't find anything using the english interface. Actually I changed some lines to fit my needs: I replaced every "en.xml" and "en.zip" with "de.xml" and "de.zip". Then I changed the thetvdb GetSeries-url to "www.thetvdb.com/api/GetSeries.php?seriesname=$tvdbshowname\&language=de" (EDIT: it must be "language=de" (without whitespace). I don't know why its inserted here....). With these changes your script can handle german EPG. Maybe you want to add this as a configurable option.

Quote:Creation of a NFO XML for unscrapable files is not very far off after XBMC dev team lets me know what information they can actually use without season and episode.

Does it mean you don't need any entries from an online source and just "copy" the information from mythtv database to xbmc? That's exactly what I'm looking for! Please tell me if you will implement this in the next days/weeks. Maybe I will try it by myself but I am not very skilled at bash scrptiing.

One more feature I will try to implement myself (its very special so I don't think anyone else will use it, but who knows...):
I can't rely on all this movie or tv show markers. I don't have "MV" or "SH" entries in my database or something else. But I know it when scheduling my recordings. So I will use different storagegroups for movies and tv shows to have them stored in different directories. In your script I will parse the %DIR% parameter to decide if it is dealing with a movie or tv show. Maybe not the best approach but I am sure it will work.

regards,
malte
(This post was last modified: 2010-01-14 10:47 by malte.)
find
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #21
malte Wrote:Maybe you want to add this as a configurable option.
I was considering that, but then it needs a backup if it gets no information, then, some of my friends in Germany who watch French and Portugeese and Turkish Television would have me create a better system. I guess a Default language will work


Quote:Does it mean you don't need any entries from an online source and just "copy" the information from mythtv database to xbmc? That's exactly what I'm looking for! Please tell me if you will implement this in the next days/weeks. Maybe I will try it by myself but I am not very skilled at bash scrptiing.

One more feature I will try to implement myself (its very special so I don't think anyone else will use it, but who knows...):
I can't rely on all this movie or tv show markers. I don't have "MV" or "SH" entries in my database or something else. But I know it when scheduling my recordings. So I will use different storagegroups for movies and tv shows to have them stored in different directories. In your script I will parse the %DIR% parameter to decide if it is dealing with a movie or tv show. Maybe not the best approach but I am sure it will work.

regards,
malte
please post up some of your guide data descriptors. If you're not using zap2it who are you using?

here is the code for the NFO file generator I've been working on... however, it is not complete and it does not do anything for XBMC yet as XBMC only supports movies and episodes.

Code:
#Create an NFO file based upon extracted information
GenNFO () {
echo "<episodedetails>">$mythicalLibrarian/NFOFILE.NFO
echo "        <title>$ShowName</title>">>"$mythicalLibrarian"/NFOFILE.NFO
test ! -z $seriesid && echo "        <episodeguide>http://www.thetvdb.com/api/$APIkey/series/$seriesid/all/en.zip<episodeguide>">>"$mythicalLibrarian"/NFOFILE.NFO
echo "        <rating>$rating</rating>">>"$mythicalLibrarian"/NFOFILE.NFO
echo "        <plot>$plot</plot>">>"$mythicalLibrarian"/NFOFILE.NFO
echo "        <genre>$ShowCategory</genre>">>"$mythicalLibrarian"/NFOFILE.NFO
test ! -z $OriginalAirDate && echo "        <aired>$OriginalAirDate</aired>">>"$mythicalLibrarian"/NFOFILE.NFO
echo "        <recordstart>$ShowStartTime</recordstart>">>"$mythicalLibrarian"/NFOFILE.NFO

echo "</episodedetails>">>"$mythicalLibrarian"/NFOFILE.NFO

}

It will create an NFO file, but it does not do anything for XBMC because XBMC only supports movies and episodes.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
(This post was last modified: 2010-01-14 18:10 by outleradam.)
find
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #22
Please post debug logs from a movie, a episode and a generic episode without subtitle or news/sports und kennt du dine EPG GmbH oder?

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
(This post was last modified: 2010-01-14 17:40 by outleradam.)
find
malte Offline
Skilled Python Coder
Posts: 1,346
Joined: Jan 2010
Reputation: 28
Location: Germany
Post: #23
Quote:please post up some of your guide data descriptors. If you're not using zap2it who are you using?

First I tried with the original EPG data that comes with my DVB-T. But that was missing complete channels and also subtitles for most of the tv shows. Now I am using a scraper that will read the data from a german tv magazine. What do you mean with guide data descriptors?

Quote:Please post debug logs from a movie, a episode and a generic episode without subtitle or news/sports und kennt du dine EPG GmbH oder?

I will do that when I'm back home. You need it with the original version of your script or with my "de"-modifications?
find
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #24
What I really need is the database entries. The mythconverg database contains massive amounts of indormation which is required to properly identify movies and distinguish episodes with data from episodes without data.

The way this is accomplished now is to check the zaptoit/schedules direct program ID. If the ID starts with mv it is a movie. Ep is episode. Sh is show with no episode information. This can be found in mythtv and iTivo txt files. I need to see the debug logs to build the program with an option to gain info about your recordings. MythicalLibrarian must have mysql access set up properly. A debug log should contain all the information I need to support DE data.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find
malte Offline
Skilled Python Coder
Posts: 1,346
Joined: Jan 2010
Reputation: 28
Location: Germany
Post: #25
Ok, here are some database values. I already checked that earlier. I don't have any of these information in my database (programid, orig airdate, syndicatedepisodenumber, ...). I did not find the txt files you mentioned. I copy and pasted some values from database directly (I hope you can read it):

Table "recorded"
Code:
'title', 'subtitle', 'category', 'recordid', 'seriesid', 'programid', 'originalairdate'
'Tagesschau', '', 'film', 13, '212613461', 'MV212613461', '0000-00-00'
'Tatort: Acht Jahre später', 'Fernsehfilm Deutschland 1974', 'Original Language', 15, '', '', '0000-00-00'
'Scrubs - Die Anfänger', 'Mein scharfes Kindermädchen', 'serie', 11, '60697138', '', '0000-00-00'

Table "recordedprogram"
Code:
'title', 'subtitle', 'category', 'category_type', 'airdate', 'seriesid', 'originalairdate', 'showtype', 'syndicatedepisodenumber', 'programid', 'manualid', 'generic', 'listingsource'
'Tagesschau', '', 'film', 'movie', '2010', '212613461', , '', '', 'MV212613461', 0, 0, 4
'Tatort: Acht Jahre später', 'Fernsehfilm Deutschland 1974', 'Original Language', 'tvshow', '0000', '', , '', '', '', 0, 0, 1
'Scrubs - Die Anfänger', 'Mein scharfes Kindermädchen', 'serie', '', '2006', '60697138', , '', '', '', 0, 0, 4

The interesting one is the first entry "Tagesschau". It's just a 15 minutes news show but it's the only entry in my whole database with a programid and marked as "movie". I record this every day and it's always the same behaviour. So if you manage to detect this correctly you are my man;-)

As already said I am using a scraper that gets my epg data from a tv magazine online service. I searched some german forums before I set this up and it seems to be used quite often. I am satisfied with this data if I could manage to import it into XBMC database (if there is no entry at tvdb or somewhere else).

I did not attach any debug logs because I don't think there is a chance to fill the gaps in this epg data. Please tell me if you want to have a look at the logs or if you need more information from database.

Quote:It will create an NFO file, but it does not do anything for XBMC because XBMC only supports movies and episodes.

What do you mean with this? I thought XBMC would use information from nfo files if provided and that your script is intended to handle movies and episodes.
find
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #26
Again, I need the debug logs. I need to see what is being picked up and how it is being picked up.

The whole idea of using an NFO file is for shows with no episode or movie information. Handling for episodes and movies is down pact. The only problem is handling unscrapable show. The idea of the NFO file is that it can get information into the XBMC database when it is not possible by other means. Right now, unscrapable shows are non-library shows. What I'm waiting for is the next version of XBMC so I can add that information into XBMC for things like news/sports, episodes with no information, documentaries.

Ok, it looks like your guide data is a bit different then mine. I need more information about the log. Do you get comskip data generated? Do you have coloumn "airdate"? Is that all the information in the program guide?

I need specific information here. The DE modification will work well. What is the german TV magazine. Where can I find them online? If you would like to speed this along, tell me what will be the differences between generic episodes, episodes with scrapable information, and movies. Also, get me those debug logs so I can see what needs to be changed and what does not. I will have to rewrite the database/database processing sections to extrapolate needed information from your database. Basically, I have to gather enough information from your EPG to not only identify the show, but to identify if the show can be identifed and the type.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
(This post was last modified: 2010-01-15 05:16 by outleradam.)
find
malte Offline
Skilled Python Coder
Posts: 1,346
Joined: Jan 2010
Reputation: 28
Location: Germany
Post: #27
Of course, I will provide you all data you need to improve your script. But I really think you should not dig too deep into this. My lack of epg data is my problem not yours. Have a look at my logs and an extract of my epg data. but I think there's no chance to get along with it.

Here's the log of a movie that is not recognized as a movie:

Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Fri Jan 15 02:17:54 CET 2010@@@@@@@@@@@@@
PERFORMING MAINTENANCE ROUTINE
SEARCH FOUND: ID#:
series was not found the tvdb or this is a movie may be down try renaming 5 x 2 - Fünf mal zwei
DB ENTIRES- RECSTART:2010-01-15 00:50:00- MOVIE:2004- ORIGAIRDATE:- CHID:1002- CAT:spielfilm-
###################DEBUG MODE ENABLED####################
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/lom/Dokumente/XBMC/Tools/MythicalLibrarian-
MOVE DIR:/home/lom/Videos/Serien- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/lom- ALTERNATE MOVE DIR:/home/lom/Videos/Serien-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled DEBUG MODE:Enabled-
INPUT SHOW NAME:5 x 2 - Fünf mal zwei- LOCAL SHOW NAME TRANSLATION:- SENT TVDB SHOW NAME:-
RESOLVED SERIES ID:- RESOVED SHOW NAME:-
INPUT EPISODE NAME:- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:-
SEASON:- EPISODE:- SYMLINK MODE:MOVE- FILESIZE: 1907188'kB'
FLAGS: MOVEDIR:1- FREE:1226287148'kB'- ALTERNATEMOVEDIR:1- FREE:1226287148'kB'-
PRIMARYMOVIEDIRWRITABLE:1- FREE:1226287148'kB'- ALTERNATEMOVIEDIR:1- FREE:1226287148'kB'-
*DATABASE* RECSTART:2010-01-15 00:50:00- MOVIE YEAR:2004- ORIGINAL SERIES DATE:-
ProgramID:- CHANNEL ID:1002- CATEGORY:spielfilm-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:- STARS:0 RATING:1
PLOT: Kurzkritik: Brillant gespieltThrill ** / Anspruch * / Erotik * / Allgemein *** / tv14-Allgemein **** / tv14-Anspruch * / tv14-Thrill ** / tv14-Kamera ** / tv14-Musik * / tv14-Story * / tv14-Erotik * / tv14-GefühlBitter: Rückblenden zeigen den unaufhaltsamen Verfall einer einst großen Liebe. Von François Ozon ("Swimming Pool").Szenen einer gescheiterten Ehe - das Besondere an der tragischen Lovestory zwischen Valeria Bruni-Tedeschi und Stéphane Freiss : François Ozon ("8 Frauen") erzählt sie rückwärts - geschieden, verheiratet, verlobt, verliebt.Info: Die Geschichte des Paares wird rückwärts erzählt - von der Scheidung zum Verlieben. Ähnlich verfährt der Psycho-Thriller "Memento" (2000).
##############LISTING FOLDER PERMISSIONS#################
ORIGIONAL FILE>ls -l /home/lom/Videos/Aufnahmen/Filme/1002_20100115005000.mpg
-rw-r--r-- 1 mythbackend video 1952961108 Jan 15 02:17 /home/lom/Videos/Aufnahmen/Filme/1002_20100115005000.mpg
mythicalLibrarian WORKING DIR>lsmod -l /home/lom/Dokumente/XBMC/Tools/MythicalLibrarian//
total 308
drwxrwxr-x 2 lom         video   4096 Jan 14 11:36 DailyReport
drwxrwxr-x 2 lom         video   4096 Jan  7 19:30 Die Simpsons
drwxr-xr-x 2 mythbackend video   4096 Jan 12 22:16 Dr. House
drwxrwxr-x 2 lom         video   4096 Jan  6 20:04 King of Queens
drwxrwxr-x 2 lom         video   4096 Jan  6 14:44 Scrubs
drwxrwxr-x 2 lom         video   4096 Jan  9 16:47 Tagesschau
drwxrwxr-x 2 lom         video   4096 Jan  9 16:46 Tatort
drwxr-xr-x 2 lom         users   4096 Jan 11 08:47 Tatort:Klassentreffen
-rw-r--r-- 1 mythbackend video    660 Jan 15 02:17 comskiplog.tracking
-rw-rw-rw- 1 lom         users    945 Jan 14 00:21 doover.sh
-rw-rw-r-- 1 lom         video    749 Jan 11 08:48 doover.sh~
-rw-r--r-- 1 mythbackend video     50 Jan 15 02:17 markupframes.txt
-rw-rw-r-- 1 lom         video      1 Jan 15 02:17 markupstart.txt
-rw-rw-r-- 1 lom         video      1 Jan 15 02:17 markupstop.txt
-rwxrwxr-x 1 lom         video  46294 Jan 14 19:56 mythicalLibrarian.sh
-rw-rw-r-- 1 lom         video  46294 Jan 14 19:55 mythicalLibrarian.sh~
-rw-rw-r-- 1 lom         video 139286 Jan 15 02:17 output.log
-rw-rw-r-- 1 lom         video     37 Jan 14 18:41 shn.txt
-rw-rw-r-- 1 lom         video     12 Jan 14 18:41 sid.txt
-rw-rw-r-- 1 lom         video   3116 Jan 14 18:41 working.xml
####################END OF DEBUG LOG#####################
MOVING FILE: /home/lom/Videos/Aufnahmen/Filme/1002_20100115005000.mpg to /home/lom/Videos/Filme// (2004).mpg
@@@@@@@@@@@@@OPERATION COMPLETE Fri Jan 15 02:17:56 CET 2010 @@@@@@@@@@@@@@@@

In this case you had a chance to recognize it because it used category = 'spielfilm' (that means movie in german). But that's only reliable in one direction: I did not find a tvshow marked as "spielfilm" but a lot of movies that are marked with category = 'Original language' and category_type = 'tvshow'. And there are a lot of tvshows also marked as 'Original language'.


Another log with 'Original Language' as category (again, that`s also used for tvshows, news and anything else)

Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Thu Jan 14 00:21:28 CET 2010@@@@@@@@@@@@@
PERFORMING MAINTENANCE ROUTINE
SEARCHING: www.TheTvDb.com SHOW NAME: Tatort: Acht Jahre später EPISODE: Fernsehfilm Deutschland 1974
FILE NAME: /home/lom/Videos/Aufnahmen/Tatort/1260_20100113224900.mpg
SEARCH FOUND: ID#:
series was not found the tvdb or this is a movie may be down try renaming Tatort: Acht Jahre später
DB ENTIRES- RECSTART:2010-01-13 22:49:00- MOVIE:- ORIGAIRDATE:- CHID:1260- CAT:Original Language-
###################DEBUG MODE ENABLED####################
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/lom/Dokumente/XBMC/Tools/MythicalLibrarian-
MOVE DIR:/home/lom/Videos/Serien- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/lom- ALTERNATE MOVE DIR:/home/lom/Videos/Serien-
USE ORIGINAL DIR:Disabled NOTIFICATIONS:Disabled DEBUG MODE:Enabled-
INPUT SHOW NAME:Tatort: Acht Jahre später- LOCAL SHOW NAME TRANSLATION:- SENT TVDB SHOW NAME:Tatort:%20Acht%20Jahre%20später-
RESOLVED SERIES ID:- RESOVED SHOW NAME:-
INPUT EPISODE NAME:Fernsehfilm Deutschland 1974- ABSOLOUTE EPISODE NUMBER:- RESOLVED EPISODE NAME:Fernsehfilm Deutschland 1974-
SEASON:- EPISODE:- SYMLINK MODE:MOVE- FILESIZE: 2148660'kB'
FLAGS: MOVEDIR:1- FREE:1237654968'kB'- ALTERNATEMOVEDIR:1- FREE:1237654968'kB'-
PRIMARYMOVIEDIRWRITABLE:1- FREE:1237654968'kB'- ALTERNATEMOVIEDIR:1- FREE:1237654968'kB'-
*DATABASE* RECSTART:2010-01-13 22:49:00- MOVIE YEAR:- ORIGINAL SERIES DATE:-
ProgramID:- CHANNEL ID:1260- CATEGORY:Original Language-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:- STARS:0 RATING:1
PLOT: Kommissar Haferkamp ist in Gefahr. Der Einbrecher Brossberg wird nach 8 Jahren Haft entlassen. Haferkamp hatte ihn gestellt. Und bei einem Feuergefecht, das der Verhaftung vorausging, hat Haferkamp Brossbergs Bruder getötet.
##############LISTING FOLDER PERMISSIONS#################
ORIGIONAL FILE>ls -l /home/lom/Videos/Aufnahmen/Tatort/1260_20100113224900.mpg
-rw-r--r-- 1 mythbackend video 2200228108 Jan 14 00:21 /home/lom/Videos/Aufnahmen/Tatort/1260_20100113224900.mpg
mythicalLibrarian WORKING DIR>lsmod -l /home/lom/Dokumente/XBMC/Tools/MythicalLibrarian//
total 284
drwxrwxr-x 2 lom         video   4096 Jan 13 10:56 DailyReport
drwxrwxr-x 2 lom         video   4096 Jan  7 19:30 Die Simpsons
drwxr-xr-x 2 mythbackend video   4096 Jan 12 22:16 Dr. House
drwxrwxr-x 2 lom         video   4096 Jan  6 20:04 King of Queens
drwxrwxr-x 2 lom         video   4096 Jan  6 14:44 Scrubs
drwxrwxr-x 2 lom         video   4096 Jan  9 16:47 Tagesschau
drwxrwxr-x 2 lom         video   4096 Jan  9 16:46 Tatort
drwxr-xr-x 2 lom         users   4096 Jan 11 08:47 Tatort:Klassentreffen
-rw-r--r-- 1 mythbackend video    660 Jan 14 00:21 comskiplog.tracking
-rw-rw-rw- 1 lom         users    755 Jan 13 11:28 doover.sh
-rw-rw-r-- 1 lom         video    749 Jan 11 08:48 doover.sh~
-rw-r--r-- 1 mythbackend video     50 Jan 14 00:21 markupframes.txt
-rw-rw-r-- 1 lom         video      1 Jan 14 00:21 markupstart.txt
-rw-rw-r-- 1 lom         video      1 Jan 14 00:21 markupstop.txt
-rwxrwxr-x 1 lom         video  46287 Jan 12 08:38 mythicalLibrarian.sh
-rw-rw-r-- 1 lom         video  46205 Jan 12 08:37 mythicalLibrarian.sh~
-rw-rw-r-- 1 lom         video 123028 Jan 14 00:21 output.log
-rw-rw-r-- 1 lom         video      0 Jan 14 00:21 shn.txt
-rw-rw-r-- 1 lom         video      0 Jan 14 00:21 sid.txt
-rw-rw-r-- 1 lom         video     54 Jan 14 00:21 working.xml
####################END OF DEBUG LOG#####################
find
malte Offline
Skilled Python Coder
Posts: 1,346
Joined: Jan 2010
Reputation: 28
Location: Germany
Post: #28
The data I posted yesterday was an extract of my mythconverg database. I only pasted some columns because most information like description, starttime, endtime, ... will be useless in this case. But here is a little extract from my original epg data file that is used with the mythfilldatabase to get the data into mythconverg:

Code:
<programme start="20100113225000" stop="20100114002000" channel="wdr">
        <title lang="de">Tatort</title>        
        <sub-title lang="de">Acht Jahre sp&#xE4;ter</sub-title>        
        <category lang="de">Spielfilm</category>
        <genre lang="de">Krimi</genre>
        <date lang="de">1973</date>
        <video>
        <aspect>4:3</aspect>
        </video>
        <credits>
        <director>Wolfgang Becker</director>
        <actor>Max Mairich (Direktor)</actor>
        ...
        </credits>        
        <desc lang="de">Kurzkritik: ...</desc>    
</programme>
<programme start="20100115200000" stop="20100115201500" channel="wdr">
        <title lang="de">Tagesschau</title>        
        <category lang="de">Film</category>
        <genre lang="de">Nachrichten</genre>
        <date lang="de">2010</date>
        <audio>
        <stereo>Stereo</stereo>
        </audio>
        <video>
        <aspect>4:3</aspect>
        </video>
    </programme>

First one is marked as "Spielfilm" (movie) in my epg but will be stored as "Original Language" in mythconverg. Second one is a news show that is marked "Film" (short word for movie) and marked as "movie" in mythconverg.

Here's an extract of mythconverg program table (all columns now. sorry, if it is unreadable):

Code:
'chanid', 'starttime', 'endtime', 'title', 'subtitle', 'description', 'category', 'category_type', 'airdate', 'stars', 'previouslyshown', 'title_pronounce', 'stereo', 'subtitled', 'hdtv', 'closecaptioned', 'partnumber', 'parttotal', 'seriesid', 'originalairdate', 'showtype', 'colorcode', 'syndicatedepisodenumber', 'programid', 'manualid', 'generic', 'listingsource', 'first', 'last', 'audioprop', 'subtitletypes', 'videoprop'
1259, '2010-01-13 22:50:00', '2010-01-14 00:20:00', 'Tatort: Acht Jahre später', '', 'Fernsehfilm Deutschland 1974', 'Original Language', 'tvshow', '0000', 0, 0, '', 0, 0, 0, 0, 0, 0, '', , '', '', '', '', 0, 0, 1, 1, 1, '', '', ''
1515, '2010-01-11 20:00:00', '2010-01-11 20:15:00', 'Tagesschau', '', '', 'film', 'movie', '2010', 0, 0, '', 0, 0, 0, 0, 0, 0, '212613461', , '', '', '', 'MV212613461', 0, 0, 4, 0, 0, '', '', ''

Again, I don't want you to fix my epg problems;-). I think I could manage this with different storage groups in mythtv and then have a little branch in your script that will ask for the directory name and take this to decide if it is a movie, a tvshow or whatever. I don't expect you to put something like this in your script because it seems to be a very special problem. So I will hijack this in my local copy.


Quote:The whole idea of using an NFO file is for shows with no episode or movie information

That's exactly what I was talking about. I want to import information like title, subtitle, description, actors, ... that is already stored in my epg data into XBMC database if the scraper can't find it at his online source. I tried this with some of my unscrapable shows and it seems to work as I expect it. I only have to provide a unique Series and Episode number that I am sure of that it's not existing for another episode. I tested something like S01E10000 and it was imported without any problem. Thank you very much for this usefull information! I did not know about nfo files before...
find
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #29
Can you find the movie year in categories other then movies?

Here is what I am thinking as a soloution for your guide data. Tell me what you think. I am going to restructure the database section to perform processing for different types of guide data suppliers. Also cleaning up the code so databases like iTivo can be added.

If movie year is not 0000 or "" or category is film then it is a movie?

If movie year = 0000 or "" and there is a subtitle Or a original airdate then it is a series?

If none of the above then it is a show with no data

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
(This post was last modified: 2010-01-15 20:26 by outleradam.)
find
malte Offline
Skilled Python Coder
Posts: 1,346
Joined: Jan 2010
Reputation: 28
Location: Germany
Post: #30
Good idea, but that isn't reliable in my case. I have a lot of movies without airdate and a lot of tvshows that have one.

Thank you for digging into this but I think I have to do this storagegroup thing. I will tell you if this works. And thanks again for the hint with the nfo and the code to do this. I will try to get this working if there aren't any entries at tvdb.
find