mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings

  Thread Rating:
  • 3 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #901
(2012-02-17 00:00)dougmeredith Wrote:  I love what this script does, but I'm very nervous about messing up my MythTV setup. Is there any documentation on this script other than the page at xbmc.org? I just tried installing it on a test system, and it did some very unexpected things, such as creating a user.

How does it insert things into the XBMC library? Does it talk to the database directly, or does it need a running XBMC instance?
Mythical librarian talks to the MythTV library. It creates a user for consistancy. I could probly remove that feature, but weighing the problems, adding mythtv as a user takes care of more problems than it causes.

It talks to the mythtv database via python bindings. It then requests XBMC to refresh the library via HTTP protocol.

(2012-02-26 02:31)sdsnyr94 Wrote:  I've had mythicalLibrarian running successfully for a while, with a user job running handbrake to encode the files to an h.264 mkv. Recently, I started using the "Appletv2" setting to encode my shows into a format that will be playable in XBMC on the atv2, but I am having trouble with stations that broadcast at 60fps... the conversion to 30fps causes video to be slightly jittery.

Can I have multiple mythtv user jobs, each referencing a different mythicalLibrarian.sh, so I can use different handbrake settings in each? I thought I read a while ago that it was supported, but this thread has gotten to large to look through, and I saw nothing in the wiki.

Is it as simple as creating a mythicalLibrarian2.sh and pointing a new mythtv user job to it?

Thanks!

Yes. you can use the "global settings" file to override changes to all instances of mythicalLibrarian. otherwise variables are pulled directly from each script.
(2012-03-26 00:33)mauibay Wrote:  My mythbackend has 6 tuners and often has multiple recordings ending at the same time, does mythicalLibrarian handle this ok, and are the any special issues to be aware? Does XBMC have any issues receiving multiple library update commands within the same several seconds?

install mythicalLibrarian on each machine. the local machine should do its own user jobs.
(2012-03-26 20:07)mauibay Wrote:  Now that I have mythicalLibrarian working, I'd like to do some fine tuning to eliminate incorrectly recognized episodes. I see the instructions for showtranslations, which I expect will work well for shows that are in the theTVdb database. What can I do about shows that are not in the database and don't have a showID at all? Is there an existing mechanism for naming shows and episodes that can't be matched at all? Ideally I'd like to provide the show name myself and have each each episode titled with the date. I record at least 3 episodes each day for shows that theTVdb doesn't have, and they are all getting misrecognized as episodes of the show "Episodes" with Matt LeBlanc, lol!

On that note, since XBMC pulls all the show info itself when the episodes are added to the library, is there a way to stop it from also misrecognizing shows that theTVdb doesn't have? When a show doesn't have a match, I'd like it to just use the show and episode names from the directory tree and leave the other meta info empty. It's a waste to have to poll theTVdb twice as it is, but it's worse than useless if the second pass by XBMC has to screw up the showtranslation done in the first pass. (Yes, I know the purpose of showtranslation is to get an exact match for theTVdb, but what about shows that don't have _any_ match in theTVdb?)

Perhaps the best solution is to try to add the missing shows to theTVdb myself? I haven't ever done that but maybe some of the shows can be added, such as daily local news.
------------------------------------------------------------
Update... I checked some of the shows that are being misrecognized, and discovered that it's not mythicalLibrarian that's getting it wrong! For example, mythicalLibrarian correctly created this:

Episodes/Awake/Season 1/Awake S01E01 (Pilot).mpg

But when XBMC updated the library it added is as Episodes/ 1x01. Episode One. Apparently there's a TV show named "Episodes" and several of my recordings are being named correctly by mythicalLibrarian but getting mangled to this other name by XBMC. I have no idea why, but suspect it has something to do with the mythicalLibrarian directory being named "Episodes". Unless there's a known fix for this, I plan to try changing that directory name to see if the problem stops.

At this point I have mythicalLibrarian appearing to correctly name ALL my recordings, so the dozen or so shows that are misrecognized in the library must be XBMC's fault. XBMC is not adding the correct show and episode names to the library even though they are correct in the mythicalLibrarian directories. I'm baffled, don't they both use the same theTVdb info? What can I do to fix this?

----------------------------------------------------------------------
Update.. Interesting. I renamed the Episodes directory to Recordings and edited mythicalLibrarian to match, then updated the source path in XBMC and cleaned the library and rescanned, and everything got recognized correctly. Note I didn't actually change any of the symlinks in the Episodes directory, I only renamed it to Recordings and that was all XBMC needed to recognize everything correctly.

mythicalLibrarian creates 3 folders, Movies, Episodes and Showings. each instance is recognized appropriately by its guide information.
You scan each folder separately with XBMC.. Movies, Episodes and Episodes. This way there are no problems with overrides or naming.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
(This post was last modified: 2012-03-28 23:33 by outleradam.)
find quote
mrplow Offline
Junior Member
Posts: 10
Joined: Sep 2008
Reputation: 0
Post: #902
For those running mythtv 0.25, How about incorperating a damaged recording lookup and mark it as such, perhaps going as far as putting them in a seperate "Damaged" folder

from mythtv 0.25 release notes
Add a recording quality tracker - can see if a digital recording is damaged beyond a certain threshold and attempt to reschedule the recording at a later airing

([ca0419d], [72d4370])
find quote
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #903
(2012-04-01 07:24)mrplow Wrote:  For those running mythtv 0.25, How about incorperating a damaged recording lookup and mark it as such, perhaps going as far as putting them in a seperate "Damaged" folder

from mythtv 0.25 release notes
Add a recording quality tracker - can see if a digital recording is damaged beyond a certain threshold and attempt to reschedule the recording at a later airing

([ca0419d], [72d4370])

This seems to be outside the scope of mythicalLibrarian's operation... there are several pieces of metadata which are accessible that are not used my mythicalLibrarian like stop-time and other irrelevant scheduling information.

Maybe if you could explain the relevance or how that would be useful? In order for that to show up in the library, it would need to be in TheTvDb.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find quote
mrplow Offline
Junior Member
Posts: 10
Joined: Sep 2008
Reputation: 0
Post: #904
(2012-04-01 16:25)outleradam Wrote:  
(2012-04-01 07:24)mrplow Wrote:  For those running mythtv 0.25, How about incorperating a damaged recording lookup and mark it as such, perhaps going as far as putting them in a seperate "Damaged" folder

from mythtv 0.25 release notes
Add a recording quality tracker - can see if a digital recording is damaged beyond a certain threshold and attempt to reschedule the recording at a later airing

([ca0419d], [72d4370])

This seems to be outside the scope of mythicalLibrarian's operation... there are several pieces of metadata which are accessible that are not used my mythicalLibrarian like stop-time and other irrelevant scheduling information.

Maybe if you could explain the relevance or how that would be useful? In order for that to show up in the library, it would need to be in TheTvDb.

Some of my recordings from my ATSC tuner are damaged due to bad reception/wind/etc, It would be nice have an indication of these damaged recordings, such as a different directory. This has everything to do with a badly recorded file and nothing to do with missing episode information.
find quote
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #905
(2012-04-01 19:32)mrplow Wrote:  
(2012-04-01 16:25)outleradam Wrote:  
(2012-04-01 07:24)mrplow Wrote:  For those running mythtv 0.25, How about incorperating a damaged recording lookup and mark it as such, perhaps going as far as putting them in a seperate "Damaged" folder

from mythtv 0.25 release notes
Add a recording quality tracker - can see if a digital recording is damaged beyond a certain threshold and attempt to reschedule the recording at a later airing

([ca0419d], [72d4370])

This seems to be outside the scope of mythicalLibrarian's operation... there are several pieces of metadata which are accessible that are not used my mythicalLibrarian like stop-time and other irrelevant scheduling information.

Maybe if you could explain the relevance or how that would be useful? In order for that to show up in the library, it would need to be in TheTvDb.

Some of my recordings from my ATSC tuner are damaged due to bad reception/wind/etc, It would be nice have an indication of these damaged recordings, such as a different directory. This has everything to do with a badly recorded file and nothing to do with missing episode information.

Yes. But everything displayed on-screen in library mode, when looked up, comes from the library. I don't know how I would insert that functionality. If you would look up or contribute to XBMC's functionality in this department I can help. However, I can't insert titles into the library, only the filename and that is billed by the way XBMC handles its library.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find quote
mrplow Offline
Junior Member
Posts: 10
Joined: Sep 2008
Reputation: 0
Post: #906
having the files in a separate damaged directory can be benificial in the sense that one could choose to have that source not scanned into the db and then one could setup a shortcut to the damaged videos folder. It was just an idea
find quote
sdsnyr94 Offline
Fan
Posts: 476
Joined: Aug 2009
Reputation: 2
Location: Greensboro, NC
Post: #907
outleradam - Mythtv 0.25 stable is right around the corner.... are we safe to update and still have mythicalLibrarian work, or are there still bugs to be worked out?
find quote
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #908
^^As long as the python bindings work, mythicalLibrarian will work. Everything is the same as far as I know. Just don't use the 0.23 method when you update.
(2012-04-01 21:06)mrplow Wrote:  having the files in a separate damaged directory can be benificial in the sense that one could choose to have that source not scanned into the db and then one could setup a shortcut to the damaged videos folder. It was just an idea
my copy of mythtv is down right now. I'm going to fix it when I update to XBMC 11 soon. Setting up my channels really sucks because mythtv sets up a ton of additional bogus channels on my cable for some reason.

Please run MythDataGrabber and post the output here to see if any changes are required. If the information is obtained
http://www.mythtv.org/wiki/MythDataGrabber

If MythDataGrabber obtains the new damaged indicator, a use job can be created to extend this functionality.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find quote
mrplow Offline
Junior Member
Posts: 10
Joined: Sep 2008
Reputation: 0
Post: #909
(2012-04-02 18:59)outleradam Wrote:  Please run MythDataGrabber and post the output here to see if any changes are required. If the information is obtained
http://www.mythtv.org/wiki/MythDataGrabber

If MythDataGrabber obtains the new damaged indicator, a use job can be created to extend this functionality.

mrplow@mythtv-server:/home/mythtv/videos2/recordings$ ~/mythdatagrabber.py --filename=122
3_20120402090000.mpg
Establishing database connection
No comskip information found.
chanid = "1223"
starttime = "2012-04-02 09:00:00"
endtime = "2012-04-02 09:30:00"
title = "Three's Company"
subtitle = "Mighty Mouth"
description = "Jack joins an exercise class to court the teacher."
season = "4"
episode = "16"
category = "Sitcom"
hostname = "mythtv-server"
bookmark = "0"
editing = "0"
cutlist = "0"
autoexpire = "1"
commflagged = "2"
recgroup = "Default"
recordid = "14"
seriesid = "EP00004459"
programid = "EP000044590144"
inetref = "77505"
lastmodified = "2012-04-02 09:07:08"
filesize = "135093040"
stars = "0.0"
previouslyshown = "1"
originalairdate = "1980-01-15"
preserve = "0"
findid = "0"
deletepending = "0"
transcoder = "28"
timestretch = "1.0"
recpriority = "0"
basename = "1223_20120402090000.mpg"
progstart = "2012-04-02 09:00:00"
progend = "2012-04-02 09:30:00"
playgroup = "Default"
profile = "Default"
duplicate = "0"
transcoded = "0"
watched = "0"
storagegroup = "Default"
bookmarkupdate = "None"
xmltvgrabber = "schedulesdirect1"
Operation complete

darn, don't see it there. Not as simple as I thought
find quote
mrplow Offline
Junior Member
Posts: 10
Joined: Sep 2008
Reputation: 0
Post: #910
I case it helps I found the DAMAGED properties, the recordedprogram table contains a videoprop field, a bunch of my results contain "HDTV,720,DAMAGED" or variations.
The damaged recording are due to me living on the fringe of many ATSC stations, sometimes they record great other times they're not worth watching.
(This post was last modified: 2012-04-05 03:02 by mrplow.)
find quote
sdsnyr94 Offline
Fan
Posts: 476
Joined: Aug 2009
Reputation: 2
Location: Greensboro, NC
Post: #911
OK, I upgraded to 0.25 and reran mythicalSetup. I now have this error:

Python Database Connection: Database schema mismatch: we speak 1264 but database speaks 1299
find quote
outleradam Offline
Donor
Posts: 1,722
Joined: Sep 2009
Reputation: 20
Location: Louisiana
Post: #912
(2012-04-13 21:06)sdsnyr94 Wrote:  OK, I upgraded to 0.25 and reran mythicalSetup. I now have this error:

Python Database Connection: Database schema mismatch: we speak 1264 but database speaks 1299

Your python bindings are out-of-date. upgrade libmyth-python or something like that... It's best to stick with official packages because of problems like this. Basically, you havn't fully upgraded MythTV yet.

Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[Image: newsig-1.png]
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
find quote
sdsnyr94 Offline
Fan
Posts: 476
Joined: Aug 2009
Reputation: 2
Location: Greensboro, NC
Post: #913
(2012-04-13 22:00)outleradam Wrote:  
(2012-04-13 21:06)sdsnyr94 Wrote:  OK, I upgraded to 0.25 and reran mythicalSetup. I now have this error:

Python Database Connection: Database schema mismatch: we speak 1264 but database speaks 1299

Your python bindings are out-of-date. upgrade libmyth-python or something like that... It's best to stick with official packages because of problems like this. Basically, you havn't fully upgraded MythTV yet.

Thanks... that did the trick. Probably caused because I did not want to do a dist-upgrade, so I missed the linmyth-python entry.

Thanks for the really quick response!
find quote
TechnoSwiss Offline
Junior Member
Posts: 7
Joined: Mar 2009
Reputation: 0
Post: #914
MythDataGrabber is having a problem doing a DB lookup with a recent Episode of CSI: NY.

http://thetvdb.com/?tab=episode&seriesid...id=4297315

I originally tried 'mythicalLibrarian 2061_20120427210000.mpg CSI\ NY Slainte', when I do 'mythicalLibrarian --doover' to bring the error back up I get...

Moving 2061_20120427210000.mpg to original location
Revision 946 Tue May 8 10:43:59 PDT 2012
Attemting to pull database information on 2061_20120427210000.mpg
Traceback (most recent call last):
File "/usr/local/bin/MythDataGrabber", line 272, in <module>
writeData()
File "/usr/local/bin/MythDataGrabber", line 226, in writeData
f.write('%s = "%s"\n' % x)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 14: ordinal not in range(128)
RECSTART:2012-04-27 21:00:00 DATE:
PROGRAMID: ShowCategory:
PLOT:
PERFORMING MAINTENANCE ROUTINE...
REMOVING ORPHAN FOLDER:/recordings/Episodes/CSI NY/Season 7
SEARCHING: http://www.TheTvDb.com SHOW NAME: CSI NY EPISODE: Slainte
TESTING FOR ZAP2ITID MATCH:EP00679714 CSI: NY
agrep: 1 word matches within 1 error
FOUND:CSI: NY ID#:73696 WITH CONFIDENCE:-2
Parsing Downloaded information: CSI: NY.xml
AIRDATE FAILED. TITLE MATCH ON ABSOLUTE EPISODE NUMBER:
AIRDATE/SANDARD LOGIC FAILED. FUZZY LOGIC ABSOLUTE:
agrep: 2 words match within 2 errors
TOTAL MATCHES:2509 TOTAL VERIFIED FUZZY ERRORS:2908
EPISODE:Vigilante EPISODE NUMBER:156 S07E15 WITH CONFIDENCE: -2
CONFIDENCE:-4 LOGIC PATH: Attempted Match On Zap2itID: Failed; Attemted match on Standard Logic: Failed; Attemted Title match on Fuzzy Logic: Sucessful; Attemped match on OriginalAirDate: Failed; Attemted SubTitle match on Standard logic: Failed; Attemted SubTitle match on Fuzzy logic: Sucessful;
JOB: mythicalLibrarian '2061_20120427210000.mpg' 'CSI NY' 'Slainte'
TARGET SET:/recordings/Episodes/CSI NY/Season 7/CSI NY S07E15 (Vigilante).mpg
CREATING SYMLINK IN MOVE MODE
MOVED: 2061_20120427210000.mpg
TO: /recordings/Episodes/CSI NY/Season 7/CSI NY S07E15 (Vigilante).mpg
RSS ENTRY SUCCESSFULLY CREATED: /var/www/mythical-rss/rss.xml
SUCCESSFUL COMPLETEION TYPE: MoveModeSuccessful
done.
find quote
djroketboy Offline
Senior Member
Posts: 110
Joined: Nov 2009
Reputation: 1
Post: #915
What's the easiest way to fully remove/uninstall mythicalLibrarian?
find quote
Post Reply