XBMC Community Forum
mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for XBMC (/forumdisplay.php?fid=116)
+--- Thread: mythicalLibrarian - a tool to build Movie and TV Show library from MythTV recordings (/showthread.php?tid=65644)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96

- outleradam - 2011-01-08 17:50

REV 909 to Stable

1. There are several speed-ups relating to the way mythicalLibrarian handles it's multiple internal databases of files and shows.
2. Saves TvDb's bandwidth by ONLY downloading information about a show when it is required. --doMaintenance will reset this.
3. This also fixes a few problems in regards to deleting MythTV database entries in "Disabled" mode.

You should run "mythicalLibrarian --doMaintenance" to clear the database after the upgrade or you will have problems if it's been a while since your last update. The database has changed significantly and all of your shows will be 1 number off of the proper episode.

- colbert - 2011-01-09 08:09

Okay back home alas and just tried it, after initial install I accidentally closed that shell so couldn't paste output, I ran --mythicalDiagnostics again so here is the output:

Code:
ââ> mythicalLibrarian --mythicalDiagnostics
--You have entered diagnostic mode
mythicalLibrarian will now conduct a series of tests.
press any key to verify installed packages...

package 'curl' is installed
package 'agrep' or 'tre-agrep' is installed
package 'libnotify-bin' is installed
Installed packages check ***PASSED***
Press any key to perform Desktop Message test...

Performing librarian-notify-send test
This test verifies that the message was sent to the DBUS.
This test does not verify the user has a valid desktop open.
sudo: /usr/local/bin/librarian-notify-send: command not found
Desktop Notifications test ***FAILED***
Press any key to perform MythTV Database test...

Performing MythTV Python Bindings database test
  File "/usr/local/bin/MythDataGrabber", line 19
    """ % os.path.basename(sys.argv[0])
      ^
SyntaxError: invalid syntax
MythTV Python Bindings are installed properly
MythTV Database Test ***PASSED***
Press any key to perform XBMC Notification test...

Performing XBMC Notifications test
SENDING REQUESTED COMMANDS TO:192.168.1.101:8080
COMMUNICATIONS NOT ESTABLISHED: 192.168.1.101:8080
XBMC Communications ***FAILED***
Press any key to test file system permissions...

Testing file system permissions
TargetPathIsInputPath is selected. Testing with current folder
USER SETTING     |FLAG|Folder name
MoveDir          |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes
AlternateMoveDir |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes
PrimaryMovieDir  |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Movies
AlternateMovieDir|1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Movies
PrimaryShowDir   |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Showings
AlternateShowDir |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Showings
mythicalLibrarian|1   |/home/bobby2/.mythicalLibrarian
Critical Filesystem checks ***PASSED***
press any key to test mythicalLibrarian...

Recordings Database is not used for testing.
Performing fuzzy logic lookup on database file.  Please wait.
It may take a moment to build mythical librarian database files
executing mythicalLibrarian testfile.ext 'mister rogers' 'show 1332'
testing for exit status 0
Revision 909 Sun Jan 9 00:56:01 EST 2011
RECSTART: DATE:
PROGRAMID: ShowCategory:
PLOT:
PERFORMING MAINTENANCE ROUTINE...
REMOVING ORPHAN FOLDER:/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes/Mister Rogers Neighborhood/Season 7
SEARCHING: www.TheTvDb.com SHOW NAME: mister rogers EPISODE: show 1332
TESTING FOR ZAP2ITID MATCH:SH002930 Mister Rogers' Neighborhood
FOUND:Mister Rogers' Neighborhood ID#:77750 WITH CONFIDENCE:-2
DATABASE IS MAINTAINED. TIME IS:1294553025 NEXT UPDATE IS:1294622792
AIRDATE FAILED. TITLE MATCH ON ABSOLUTE EPISODE NUMBER:
TOTAL MATCHES:2 TOTAL VERIFIED FUZZY ERRORS:0
EPISODE:Show 1332 EPISODE NUMBER:462 S07E07 WITH CONFIDENCE: -1
CONFIDENCE:-3 LOGIC PATH:  Attempted Match On Zap2itID: Failed; Attemted match on Standard Logic: Failed; Attemted Title match on Fuzzy Logic: Sucessful; Attemped match on OriginalAirDate: Failed; Attemted SubTitle match on Standard logic: Sucessful;
JOB: mythicalLibrarian 'testfile.ext' 'mister rogers' 'show 1332'
TARGET SET:/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes/Mister Rogers Neighborhood/Season 7/Mister Rogers Neighborhood S07E07 (Show 1332).ext
exited with status 0
press any key to continue on to the final results...

  All tests complete           RESULTS
  MythTV Database Test:      ***PASSED***
  Installed Packages:        ***PASSED***
  Notifications Test:        ***FAILED***
  XBMC Communications:       ***FAILED***
  File System Checks:        ***PASSED***
  mythicalTesting:           ***PASSED***
  Overall mythicalReadiness: ***PASSED***
-----End of diagnostics-----
Notifications to the DBUS have failed. mythicalLibrarian relies upon
librarian-notify-send and package lib-notify to display messages on
GNOME desktop interface. These messages will not be displayed.
XBMC Communications have failed.  Likely causes: Improper IP address,
or XBMC is not equipped to handle external network commands in settings.
mythicalLibrarian will not send messages or library update requests.

Also, my .101 (XBMC) box Network settings are http://i.imgur.com/oFz0h.jpg (Do I need to open the 8080 port on router BTW?)

I added /media/myth as video source on the .101 (XBMC) box, and I can see the dir just fine, but do I set content as TV now or? I apologize for perhaps nitpicking at each step but I had a terrible loss of data some time ago with Myth and I'm overly cautious now in proceeding with anything related to it lol Big Grin Thanks very much.

- outleradam - 2011-01-10 05:54

It depends on the router. Usually, consumer routers have 4 ports which are on the LAN and 1 port which has Network Address Translations (an internet port). If your computer is on the NAT port, then you will need to open the port 8080. You should use the NAT port for a network which you consider insecure. Unless you use the NAT for a computer, you should not need to open a port.

It looks like the python bindings are not installed correctly on your computer.

Here's what you should have seen there
Code:
Establishing database connection
Connected to: XBMCLive
Db Identified as: sql://mythconverg@localhost:3306/
Diagnostics passed.
Diagnostics failed.
Please post the output of the following:
Code:
python --version
MythDataGrabber --Diagnostic=True

My python is Python 2.6.5
You are running MythTV v0.24 right?
You can check this by using the command
Code:
root@XBMCLive:~# mythbackend --version|grep Library
Library API      : 0.25.20101223-1

Code:
adam@adam-desktop:~$ mythbackend --version|grep Library
Library API      : 0.24.20101124-1


I've added better diagnostics in the SVN. You should upgrade and post the results of the database diagnostics.

I value this information as it will help me to make mythicalLibrarian more user friendly.

- colbert - 2011-01-10 06:36

outleradam Wrote:It depends on the router. Usually, consumer routers have 4 ports which are on the LAN and 1 port which has Network Address Translations (an internet port). If your computer is on the NAT port, then you will need to open the port 8080. You should use the NAT port for a network which you consider insecure. Unless you use the NAT for a computer, you should not need to open a port.

Gotcha, I will take care of that once the script is setup, good to know!

outleradam Wrote:It looks like the python bindings are not installed correctly on your computer.

Here's what you should have seen there
Code:
Establishing database connection
Connected to: XBMCLive
Db Identified as: sql://mythconverg@localhost:3306/
Diagnostics passed.
Diagnostics failed.
Please post the output of the following:
Code:
python --version
MythDataGrabber --Diagnostic=True

Yes this might be a bit tricky because Arch (my distro) did something fairly controversial in the distro's community by upgrading the main Python package to 3. We can still use 2 but it has to be specified, for example in torrentflux in my admin settings here's what I had to put: http://i.imgur.com/oAv1G.png.

Okay my output:

Code:
ââ> python --version
Python 3.1.3


Code:
ââ> MythDataGrabber --Diagnostic=True
  File "/usr/local/bin/MythDataGrabber", line 19
    """ % os.path.basename(sys.argv[0])
      ^
SyntaxError: invalid syntax

^^^ This is the same type of error I got using torrentflux before I changed it to the above image python path for python2, I'm wagering I'll have to change the script or something to use python2, lemme know where/how if so.


outleradam Wrote:My python is Python 2.6.5
You are running MythTV v0.24 right?
You can check this by using the command
Code:
root@XBMCLive:~# mythbackend --version|grep Library
Library API      : 0.25.20101223-1

Definitely, on all boxes. .100:
Code:
ââ> mythbackend --version|grep Library
Library API      : 0.24.20101028-1
.101:
Code:
ââ> mythbackend --version|grep Library
Library API      : 0.24.20101028-1

outleradam Wrote:I've added better diagnostics in the SVN. You should upgrade and post the results of the database diagnostics.

I value this information as it will help me to make mythicalLibrarian more user friendly.

Of course! Ran --update on it and here is output:

Code:
mythicalLibrarian will now conduct mythicalDiagnostics
Press any key to continue to online testing....

--You have entered diagnostic mode
mythicalLibrarian will now conduct a series of tests.
press any key to verify installed packages...

package 'curl' is installed
package 'agrep' or 'tre-agrep' is installed
package 'libnotify-bin' is installed
Installed packages check ***PASSED***
Press any key to perform Desktop Message test...

Performing librarian-notify-send test
This test verifies that the message was sent to the DBUS.
This test does not verify the user has a valid desktop open.
Desktop Notifications test ***PASSED***
Press any key to perform MythTV Database test...

Performing MythTV Python Bindings database test
  File "/usr/local/bin/MythDataGrabber", line 19
    """ % os.path.basename(sys.argv[0])
      ^
SyntaxError: invalid syntax
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named MythDB.MythTV
/usr/local/bin/mythicalLibrarian: line 1778: test: too many arguments
  File "/usr/local/bin/MythDataGrabber", line 19
    """ % os.path.basename(sys.argv[0])
      ^
SyntaxError: invalid syntax
there was a problem connecting to the database, check MythDatabase package
MythTV Database Test ***FAILED***
Press any key to perform XBMC Notification test...

Performing XBMC Notifications test
SENDING REQUESTED COMMANDS TO:192.168.1.101:8080
COMMUNICATIONS NOT ESTABLISHED: 192.168.1.101:8080
XBMC Communications ***FAILED***
Press any key to test file system permissions...

Testing file system permissions
TargetPathIsInputPath is selected. Testing with current folder
USER SETTING     |FLAG|Folder name
MoveDir          |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes
AlternateMoveDir |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes
PrimaryMovieDir  |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Movies
AlternateMovieDir|1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Movies
PrimaryShowDir   |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Showings
AlternateShowDir |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Showings
mythicalLibrarian|1   |/home/bobby2/.mythicalLibrarian
Critical Filesystem checks ***PASSED***
OVERALL FAILURE
press any key to continue on to the final results...

  All tests complete           RESULTS
  MythTV Database Test:      ***FAILED***
  Installed Packages:        ***FAILED***
  Notifications Test:        ***PASSED***
  XBMC Communications:       ***FAILED***
  File System Checks:        ***PASSED***
  mythicalTesting:           COULD NOT COMPLETE
  Overall mythicalReadiness: ***FAILURE***
-----End of diagnostics-----
When interfacing MythTV/0.24, mythicalLibrarian uses MythTV's Python
Bindings.  These must be configured and working for proper access
to data required by mythicalLibrarian.  
Could not connect to the database.  Specify a username and password
XBMC Communications have failed.  Likely causes: Improper IP address,
or XBMC is not equipped to handle external network commands in settings.
mythicalLibrarian will not send messages or library update requests.
mythicalLibrarian has encountered an error.  This may be due to
lack of network connection.   If the problem persists, contact
outleradam here: http://forum.xbmc.org/showthread.php?tid=65644

These issues must be corrected in order for mythicalLibrarian to
perform correctly
chgrp: invalid group: `mythtv'
Please try again.  If problem persists, please post here: http://forum.xbmc.org/showthread.php?tid=65644
ADDING JOB to slot 0
Could not add mythcialLibrarian MythTV UserJob because no slots were available
permissions were set for user: mythtv.
No RSS Feeds will be used on this server. Configure /var/www/mythical-rss to link to web server for access by mythicalLibrarian.
mythicalLibrarian is located in /usr/local/bin
mythicalLibrarian's log is located in ~/.mythicalLibrarian/output.log
Renamed video files will be placed in new folders within original.
  ie. /var/lib/mythtv/recordings/Episodes, /Movies and /Showings.
Check the mythicalLibrarian checkbox when setting up recordings.
For more information, type 'mythicalLibrarian --help'
Done.


- outleradam - 2011-01-10 08:05

I will try to come up with a python 3 solution.

- outleradam - 2011-01-10 08:05

For now, tell mythicalSetup that you are running 0.23.

- colbert - 2011-01-10 08:25

outleradam Wrote:I will try to come up with a python 3 solution.

That would be greatly appreciated if possible!! Cool

outleradam Wrote:For now, tell mythicalSetup that you are running 0.23.

Okay, ran --update again and chose .23:

Code:
mythicalLibrarian will now conduct mythicalDiagnostics
Press any key to continue to online testing....

--You have entered diagnostic mode
mythicalLibrarian will now conduct a series of tests.
press any key to verify installed packages...

package 'curl' is installed
package 'agrep' or 'tre-agrep' is installed
package 'libnotify-bin' is installed
Installed packages check ***PASSED***
Press any key to perform Desktop Message test...

Performing librarian-notify-send test
This test verifies that the message was sent to the DBUS.
This test does not verify the user has a valid desktop open.
Desktop Notifications test ***PASSED***
Press any key to perform MythTV Database test...

Performing MythTV Python Bindings database test
  File "/usr/local/bin/MythDataGrabber", line 19
    """ % os.path.basename(sys.argv[0])
      ^
SyntaxError: invalid syntax
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named MythDB.MythTV
/usr/local/bin/mythicalLibrarian: line 1778: test: too many arguments
  File "/usr/local/bin/MythDataGrabber", line 19
    """ % os.path.basename(sys.argv[0])
      ^
SyntaxError: invalid syntax
there was a problem connecting to the database, check MythDatabase package
MythTV Database Test ***FAILED***
Press any key to perform XBMC Notification test...

Performing XBMC Notifications test
SENDING REQUESTED COMMANDS TO:192.168.1.101:8080
COMMUNICATIONS NOT ESTABLISHED: 192.168.1.101:8080
XBMC Communications ***FAILED***
Press any key to test file system permissions...

Testing file system permissions
TargetPathIsInputPath is selected. Testing with current folder
USER SETTING     |FLAG|Folder name
MoveDir          |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes
AlternateMoveDir |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes
PrimaryMovieDir  |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Movies
AlternateMovieDir|1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Movies
PrimaryShowDir   |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Showings
AlternateShowDir |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Showings
mythicalLibrarian|1   |/home/bobby2/.mythicalLibrarian
Critical Filesystem checks ***PASSED***
OVERALL FAILURE
press any key to continue on to the final results...

  All tests complete           RESULTS
  MythTV Database Test:      ***FAILED***
  Installed Packages:        ***FAILED***
  Notifications Test:        ***PASSED***
  XBMC Communications:       ***FAILED***
  File System Checks:        ***PASSED***
  mythicalTesting:           COULD NOT COMPLETE
  Overall mythicalReadiness: ***FAILURE***
-----End of diagnostics-----
When interfacing MythTV/0.24, mythicalLibrarian uses MythTV's Python
Bindings.  These must be configured and working for proper access
to data required by mythicalLibrarian.  
Could not connect to the database.  Specify a username and password
XBMC Communications have failed.  Likely causes: Improper IP address,
or XBMC is not equipped to handle external network commands in settings.
mythicalLibrarian will not send messages or library update requests.
mythicalLibrarian has encountered an error.  This may be due to
lack of network connection.   If the problem persists, contact
outleradam here: http://forum.xbmc.org/showthread.php?tid=65644

These issues must be corrected in order for mythicalLibrarian to
perform correctly
chgrp: invalid group: `mythtv'
Please try again.  If problem persists, please post here: http://forum.xbmc.org/showthread.php?tid=65644
ADDING JOB to slot 0
Could not add mythcialLibrarian MythTV UserJob because no slots were available
permissions were set for user: mythtv.
No RSS Feeds will be used on this server. Configure /var/www/mythical-rss to link to web server for access by mythicalLibrarian.
mythicalLibrarian is located in /usr/local/bin
mythicalLibrarian's log is located in ~/.mythicalLibrarian/output.log
Renamed video files will be placed in new folders within original.
  ie. /var/lib/mythtv/recordings/Episodes, /Movies and /Showings.
Check the mythicalLibrarian checkbox when setting up recordings.
For more information, type 'mythicalLibrarian --help'
Done.


- outleradam - 2011-01-11 06:18

I keep looking for the problem. Can you try setup once again?

- colbert - 2011-01-11 07:42

Sure thing, but in case I'm doing some stupid little mistakes, here is what I'm putting at each prompt during the setup:

1. mysql username

Code:
mythtv

2. mysql password

Code:
mythtv

3. DB Host IP

Code:
192.168.1.100

4. mysql password

Code:
actual-mysql-root-password

5. XBMC IP:

Code:
'192.168.1.101:8080'

(.100 again being the myth box, and .101 being the XBMC box.)

Output doing --update and using .24:

Code:
mythicalLibrarian will now conduct mythicalDiagnostics
Press any key to continue to online testing....

--You have entered diagnostic mode
mythicalLibrarian will now conduct a series of tests.
press any key to verify installed packages...

package 'curl' is installed
package 'agrep' or 'tre-agrep' is installed
package 'libnotify-bin' is installed
Installed packages check ***PASSED***
Press any key to perform Desktop Message test...

Performing librarian-notify-send test
This test verifies that the message was sent to the DBUS.
This test does not verify the user has a valid desktop open.
Desktop Notifications test ***PASSED***
Press any key to perform MythTV Database test...

Performing MythTV Python Bindings database test
  File "/usr/local/bin/MythDataGrabber", line 19
    """ % os.path.basename(sys.argv[0])
      ^
SyntaxError: invalid syntax
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named MythDB.MythTV
/usr/local/bin/mythicalLibrarian: line 1778: test: too many arguments
  File "/usr/local/bin/MythDataGrabber", line 19
    """ % os.path.basename(sys.argv[0])
      ^
SyntaxError: invalid syntax
there was a problem connecting to the database, check MythDatabase package
MythTV Database Test ***FAILED***
Press any key to perform XBMC Notification test...

Performing XBMC Notifications test
SENDING REQUESTED COMMANDS TO:192.168.1.101:8080
COMMUNICATIONS NOT ESTABLISHED: 192.168.1.101:8080
XBMC Communications ***FAILED***
Press any key to test file system permissions...

Testing file system permissions
TargetPathIsInputPath is selected. Testing with current folder
USER SETTING     |FLAG|Folder name
MoveDir          |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes
AlternateMoveDir |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Episodes
PrimaryMovieDir  |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Movies
AlternateMovieDir|1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Movies
PrimaryShowDir   |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Showings
AlternateShowDir |1   |/home/bobby2/.mythicalLibrarian/mythicalSetup/Showings
mythicalLibrarian|1   |/home/bobby2/.mythicalLibrarian
Critical Filesystem checks ***PASSED***
OVERALL FAILURE
press any key to continue on to the final results...

  All tests complete           RESULTS
  MythTV Database Test:      ***FAILED***
  Installed Packages:        ***FAILED***
  Notifications Test:        ***PASSED***
  XBMC Communications:       ***FAILED***
  File System Checks:        ***PASSED***
  mythicalTesting:           COULD NOT COMPLETE
  Overall mythicalReadiness: ***FAILURE***
-----End of diagnostics-----
When interfacing MythTV/0.24, mythicalLibrarian uses MythTV's Python
Bindings.  These must be configured and working for proper access
to data required by mythicalLibrarian.  
Could not connect to the database.  Specify a username and password
XBMC Communications have failed.  Likely causes: Improper IP address,
or XBMC is not equipped to handle external network commands in settings.
mythicalLibrarian will not send messages or library update requests.
mythicalLibrarian has encountered an error.  This may be due to
lack of network connection.   If the problem persists, contact
outleradam here: http://forum.xbmc.org/showthread.php?tid=65644

These issues must be corrected in order for mythicalLibrarian to
perform correctly
chgrp: invalid group: `mythtv'
Please try again.  If problem persists, please post here: http://forum.xbmc.org/showthread.php?tid=65644
ADDING JOB to slot 0
Could not add mythcialLibrarian MythTV UserJob because no slots were available
permissions were set for user: mythtv.
No RSS Feeds will be used on this server. Configure /var/www/mythical-rss to link to web server for access by mythicalLibrarian.
mythicalLibrarian is located in /usr/local/bin
mythicalLibrarian's log is located in ~/.mythicalLibrarian/output.log
Renamed video files will be placed in new folders within original.
  ie. /var/lib/mythtv/recordings/Episodes, /Movies and /Showings.
Check the mythicalLibrarian checkbox when setting up recordings.
For more information, type 'mythicalLibrarian --help'
Done.


- outleradam - 2011-01-12 05:03

Alright, I hit it from both ends of your problem.

1. mysql access would never work because the command was
Code:
if [ "$DatabaseType"="MythTVPythonBindings" ]; then
which basically said "if you can define $DatabaseType as MythTVPythonBindings, then". and it should have been
Code:
if [ "$DatabaseType" = "MythTVPythonBindings" ]; then
which now correctly states "If $Database type equals MythTVPythonBindings, then"

2. MythDataGrabber now specifies runtime environment of Python2.6. Hopefully this does not cause problems in the future. I'm not sure if all distros have a /usr/bin/python2.6 or not...

You should be good to go with 0.24. It's working on my 0.25 machine just fine.