+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=26)
+---- Forum: PVR (https://forum.kodi.tv/forumdisplay.php?fid=136)
+---- Thread: MythTV PVR client Addon Developers Wanted - Developers Only! (/showthread.php?tid=82015)
- tafypz - 2010-10-07
dteirney Wrote:I've add a list of the PVR Addon API methods on the wiki page.
http://wiki.xbmc.org/index.php?title=MythTV_PVR_Addon
The raw method API can be found in the xbmc/pvrclients/mythtv/client.cpp file in the patch located at http://trac.xbmc.org/attachment/ticket/10445/mythtv-base-stripped-2.patch
If anyone is able to review the MythXML content for any of the MythTV interfaces and perhaps list what fields in the XML map to the data structures XBMC expects please let this list know and if possible update the wiki page with your name.
If anyone is able to test the HTTP streaming capabilities of the MythXML interface then also let this list know.
The C structures XBMC uses can be found in xbmc/addons/include/xbmc_pvr_types.h
The structures all appear to be doxygen'ed but I don't know if there is any documentation generated.
I updated some info in the api section of the wiki page, I will try to provide some mappings tonight when I get home.
- PhracturedBlue - 2010-10-07
I'll start working on fixing up cmyth to work with 0.24. I probably won't have time to do a 0.24 install until next week, but I've looked over their changes, and it looks trivial to support.
This whole thing is rather dumb. The myth folks explicitly added this to make it hard for code that uses old protocols (which mostly just work) to stay current with myth. There are much more intelligent ways of getting what they really want.
Instead what we're going to do is to build a list of version <-> key pairs, and send the right response for the protocol in question.
I'll verify that the functions we use behave properly and fix those that don't at the same time of course.
- outleradam - 2010-10-07
I've been working with the mythTV python bindings lately. I would like to help in any way possible.
Here is some of mine and barney_1's python code which takes an input of a filename and database information. It then tries several methods of connecting to the database. After connection, it polls all information from the database and puts it into a variable corresponding to the table name. Finally, it outputs to a file or the screen.
http://mythicallibrarian.googlecode.com/svn/trunk/pythonBindings/mythicalPythonBindings.py
The new python bindings are designed to be maintainable. The purpose of the bindings are to abstract the database. The devs at MythTV have a cool idea of a way to rearrange their database. They've provided these python bindings in advance before they make the major changes. The changes are going to separate tables from information which you would expect to see in the same table. The Python Bindings basically perform join operations and allow you to see the information in an manner which won't change with the tables in the future. Previously we were basically just doing SQL calls (like mythicalLibrarian, but in python), now the Python Bindings are going to be designed to perform tasks like join and search operations which makes it easier for us as developers because if they screw up the APIs, we can point the finger at them and tell the users to revert or use the SVN.
Please let me know if I can help. I would be happy to improve XBMC rather then building around it.
- wagnerrp - 2010-10-08
outleradam Wrote:The changes are going to separate tables from information which you would expect to see in the same table. The Python Bindings basically perform join operations and allow you to see the information in an manner which won't change with the tables in the future.
That is not the case. At its base, the MythTV python bindings function as an ORM. The DBData and DBDataWrite classes map one object directly to one entry in one database table. They abstract the raw SQL away from the user, and provide editable objects. As such, when database schema changes, so do the attributes exposed through the object.
While the intent is to try to keep the interface as consistent as possible between revisions from 0.24 forward, certain changes, such as the merging of video and recording tables planned for 0.25, will require some form of restructuring in the bindings to match.
- outleradam - 2010-10-08
It is my understanding that the rec object created in
Code:
from MythTV import MythDB
db = MythDB()
rec = db.searchRecorded(basename=options['filename']).next()
rec.itemsCode:
for x in rec.items(): print (x[0] + " = " + str(x[1]))- wagnerrp - 2010-10-08
The search functions can cross reference multiple tables through JOINs and other mechanisms for the purpose of matching, but they will only return one object, which is mapped to one table.
A better format for the below code would be:
Code:
for x in rec.items(): print "%s=%s" % xCode:
for x in rec.items(): print "{0}={1}".format(*x)- outleradam - 2010-10-08
A few things to note about this project before I go to bed.
1. Fuzzy logic can be provided bythe TRE Aproximate RegEx library on all platforms and it can be included in XBMC and compiled as source. This is pretty much required for TheTvDb matching of titles and subtitles and should be used as a backup.
2. Matching of Episodes should be done like this:
a. determine if it is a movie, generic programming or episode by mythtv programid
i. MVxxxxxxxxxxxxxx = movie
ii. SHxxxxxxxxxxxxxxx = generic showing
iii. EPxxxxxxxxxxxxxxx = Episode
b. call TVDB and send show title
c. receive a list of tvdb showids and titles
d. download all information about the showids from thetvdb (their entire database on the show)
e. compare the PROGRAMID to the ZAP2IT ID strings returned
i. mythtv schedulesdirect programid: EP007261330099
ii.thetvdb Zap2itid SH72613
iii. Remove leading 0s and leading SH EP or MVs
iv. compare the zap2it number 72613 to the mythtv program id number
v. get the series id
f. if zap2it id fails, use fuzzy logic and agrep the show title and match that to the seriesid
g. within that single seriesid database, locate the episode and season
a. try original airdate first.
b. aproximate grep the subtitle of the showing
h. determine season and airdate from the record determined from step g.
3. for generic programming, SH's a seriesid will be used, but that's all. Generate enough information to populate the database off of the show and label it S0E0.
4. For movies MV's only the title is used and the fact that it is a movie
5. wagnerrp is a developer for mythtv and one of the most helpful people I've ever run across. You can usually find him on mythtvtalk.com and don't forget that they are trying to meet a fund-raising goal.
- PhracturedBlue - 2010-10-08
Thanks, but have you tried the functionality the dtierny and I have been working on (ticket #7197)? It is all native, and works extremely well for me. Additionally, it should benefit other sources, since very little of it is Myth specific. Given this approach, I don't see much benefit to using the mythical plugin, but maybe I'm missing something here.
Our algorithm is essentially the same as yours, though I don't think we're using the zap2it id to get the title (I could be wrong, I haven't looked at that portion of the code). Also, the code needs to work with XLMLTV codes which do not use schedules-direct nomenclature, which in some cases already provides the season/episode info.
- outleradam - 2010-10-08
That's why I am exposing my methods here. I want you to put the functionality in XBMC. I dedicated quite a bit of time to figuring out the best method of categorizing television shows.
The XMLTV ids have the same numbers. They all seem to reference back to Zap2it. Given, some people make their own XMLTV data by scanning a TVGuide or something and it does not come with much information, but I have seen several xmltvs which have valid zap2itIDs. It is well worth it to utilize Zap2It id to automate the process of determining if an unknown recording is battlestar galactica (2004) or (1978).
There should be some sort of high confidence of recognition flag which is raised when matched upon Zap2ItId and original airdate and the fuzzy logic title and subtitle jive. This should be regarded as an untouchable file, where one which does not match fuzzy logic, has no airdate, or has no zap2itID should be regarded as unverified.
If zap2itID is available, then it should be checked again later as an unverified fuzzy logic match of a recording which was recorded today can be easily re-matched against current data to account for information which was not in theTvDb previously.
If a show has two subtitles, as is seen in 15 minute double showings of childrens programming like sponge-bob square pants, they are generally separated by a ";" Only the first one should be run as a fuzzy logic match. The seccond one should be disregarded.
This also triggers the guide data and tvdb data do not match, which displays the low confidence flag in mythicalLibrarian as seen here:
Code:
@@@@@@@@@@@NEW SEARCH INITIATED AT Wed Oct 6 06:44:27 CDT 2010@@@@@@@@@@@@@@
Revision 663 Tue Oct 5 21:33:54 CDT 2010 on Linux
PERFORMING MAINTENANCE ROUTINE
SEARCHING: www.TheTvDb.com SHOW NAME: SpongeBob SquarePants EPISODE: Back to the Past
DATE: 2010-02-15 FILE NAME: /home/mythtv/Videos/1024_20101006060000.mpg
Creating MythicalLibrarian Database Folder
SEARCH FOUND:SpongeBob SquarePants ID#: 75886
LOCAL DATABASE UPDATED:/home/mythtv/.mythicalLibrarian/SpongeBob SquarePants
MATCH!!!! Zap2itID:SpongeBob SquarePants ID.75886
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%% GUIDE DATA AND TVDB DATA DO NOT MATCH %%%%%%%%%%%%%%%%%
%%%%%%%%%%% Possible causes: small variations in episode name %%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%% bad guide data %%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%% bad tvdb airdate %%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%% bad tvdb episode name %%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%% bad tvdb zap2it id %%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
###################DEBUG MODE ENABLED####################
MY USER NAME:mythtv-
LISTING INTERNAL VARIABLES USED BY mythicalLibrarian.
INTERNET TIMEOUT:50- TVDB API KEY:6DF511BB2A64E0E9- mythicalLibrarian WORKING DIR:/home/mythtv/.mythicalLibrarian-
MOVE DIR:/home/mythtv/Videos/Episodes- USING SHOWNAME AS FOLDER:Enabled-
FAILSAFE MODE:Enabled- FAILSAFE DIR:/home/mythtv/FailSafe- ALTERNATE MOVE DIR:/home/mythtv/Videos/Episodes-
USE ORIGINAL DIR:Enabled NOTIFICATIONS:Enabled
INPUT SHOW NAME:SpongeBob SquarePants- LOCAL SHOW NAME TRANSLATION:- ShowName:SpongeBob SquarePants
DATABASE UPDATED:- TVDB LAST UPDATED:70000- CURRENT:1286366433-
RESOLVED SERIES ID:75886- RESOVED SHOW NAME:SpongeBob SquarePants-
INPUT EPISODE NAME:Back to the Past; The Bad Guy Club for Villains- ABSOLOUTE EPISODE NUMBER:263- RESOLVED EPISODE NAME:The Bad Guy Club for Villains-
SEASON:S07- EPISODE:E26- SYMLINK MODE:MOVE- FILESIZE: 0kB-
CREATE AND DELETE FLAGS: ORIGINALDIR:1- FREE:4290616kB- WORKINGDIR:1 FREE:4290616kB-
MOVEDIRWRITABLE:1- FREE:4290684kB- ALTERNATEMOVEDIR:1- FREE:4290684kB-
PRIMARYSHOWDIRWRITABLE:1- FREE:4290616kB-ALTERNATESHOWDIRWRITABLE:1- FREE:4290616kB-
PRIMARYMOVIEDIRWRITABLE:1- FREE:4290616kB- ALTERNATEMOVIEDIR:1- FREE:4290616kB-
DATABASE TYPE:schedulesdirect1-
RECSTART:2010-10-06 06:00:00- MOVIE YEAR:- ORIGINAL SERIES DATE:2010-02-15-
PROGRAMID:EP003077660410- CHANNEL ID:1024- CATEGORY:Children- GOFORDOOVER:1-
EXTRAPOLATED DATA DETERMINED THIS RECORDING AS A:Series With Episode Data- STARS:0 RATING:0
ZAP2IT SERIES ID:307766- MATCHED TVDB SERIES ID:75886-
PLOT: SpongeBob and Patrick go back in time to see the young Mermaid Man and Barnacle Boy; SpongeBob and Patrick watch their heroes fight bad guys.
####################END OF DEBUG LOG#####################
MOVING FILE: '/home/mythtv/Videos/1024_20101006060000.mpg' to '/home/mythtv/Videos/Episodes/SpongeBob SquarePants/Season 7/SpongeBob SquarePants S07E26 (The Bad Guy Club for Villains).mpg'
@@@@@@@@@@@@@OPERATION COMPLETE Wed Oct 6 06:44:42 CDT 2010 @@@@@@@@@@@@@@@@Data confidence markers are a very important feature to add. They serve to improve the project and they let you know, thorugh logging, where you can improve.
Traditionally, Season/Episode recognition has been considered somewhat of a black art. The data is available for a full high confidence match. Only by doing so are you going to be able to allow your program to be 100% sure of itself and be able to say that you have done the best job possible.
I'll be honest, this makes no sense to me http://trac.xbmc.org/attachment/ticket/7197/fuzzy_match_r2.patch . I do basic/vb/java/bash/some python/ I'm no good at C language past "hello world". I do, however, want to help. I am able to provide all of this information which I have performed hours upon hours upon hours of hard research to obtain since last November, and can tell you for a fact that these small details will ensure accuracy of data.
- PhracturedBlue - 2010-10-08
outleradam Wrote:That's why I am exposing my methods here. I want you to put the functionality in XBMC. I dedicated quite a bit of time to figuring out the best method of categorizing television shows.Thanks for clarifying, I didn't understand where you were coming from.
I like the idea of keeping a confidence score, though doing so will require a lot more surgery to make it happen (keeping track of it should be easy, and retrying if it is low shouldn't be hard, but displaying it to the user (so they can manually enter info for instance) would be a lot more work.
I'll i investigate how the series stuff works in xbmc now, to understand if we should do more or if it is already sufficient.
Quote:I'll be honest, this makes no sense to me http://trac.xbmc.org/attachment/ticket/7197/fuzzy_match_r2.patch .
The idea is simple:
use a fuzzy string match to find the best episode based on episode title. There are 2 match criteria:
1) more than one air date matched the recording: choose the best string match
2) no air-date matched the recording: choose the best match with a score > 80%.
That 80% was pulled out of thin air by me looking at matches across my wide range of recordings. The idea is to be conservative, because it is better (in my min) to display unmatched recordings using myth's available data than it is to do an incorrect match.
Once the xbmc devs get the current batch of code committed, we can continue to fine-tune the algorithms. Your experience with this is definitely valuable, and I'll look more closely at how our current capabilities differ. There is no question that there is a lot of black-magic in getting this right, but at the same time keeping it general enough to not be myth-specific.
- outleradam - 2010-10-08
mythicalLibrarian, although it currently only supports MythTV, is interchangable with any program that has logical guide data. It's just a matter of changing the database interpreter.
You need: filename to get Title, Subtitle, Zap2itID (or programid which is generally based on some sort of zap2itid or some other way of telling if it is a movie), original airdate, the movie airdate and that's about it.
If you cannot match a zap2itID for MV, SH, or EP:
Start off by assuming everything is an episode and disqualify those without original airdate and subtitle. Check the subtitle against theTvDb and if it is not matchable, then mark the episode as a showing (s0e0) and dump everything into the database about the file.
True showings contain only a title usually and cannot be matched against TheTVDB. They usually contain an original airdate which matches the first ever airdate of the entire series. This is where the Zap2IT is really handy because it can confirm or deny the matchability against thetvdb. The series original airdate is contained in TheTvDb's xml and can be referenced. I've noticed that the series airdate is usually different then the first show, so it is generally logical to use.
Movies require a Title and a Year (pulled from the movie airdate and is always a year without month or day).
That flowchart can be used for XBMC easily. Each new program's database can be broken down into several variables which becomes the new database. Each program's database will require a database interpreter.
As long as you keep it modular and adapt whatever information is available to whatever information you require, any new program's database should be interchangable. It's just a matter of performing checks on information which is available and not hesitating to call a program a "generic programming" if insufficient information is available. The more you have, the better the match.
- tafypz - 2010-10-10
I added 2 patches to 10445
One that creates a settings file and an english language file to the pvr.mythtv folder.
The other one creates an initial lib for mythxml.
The lib for mythxml compiles fine; I tried to use Filesystem/FileCurl.h to access mythxml (current code in the lib) but it cannot get the data from there, I guess that I will have to use curl directly. If there is another "wrapper" around curl that one of you guys know of please let me know, I'd rather use code that already exists rather than piling on stuff with the same functionnality. As always I am open to any suggestion on the code.
- dteirney - 2010-10-11
tafypz Wrote:I added 2 patches to 10445
One that creates a settings file and an english language file to the pvr.mythtv folder.
The other one creates an initial lib for mythxml.
The lib for mythxml compiles fine; I tried to use Filesystem/FileCurl.h to access mythxml (current code in the lib) but it cannot get the data from there, I guess that I will have to use curl directly. If there is another "wrapper" around curl that one of you guys know of please let me know, I'd rather use code that already exists rather than piling on stuff with the same functionnality. As always I am open to any suggestion on the code.
Might be best to ask this as a separate question in this Development forum. I'm not sure myself. Would be good if some of the wrapper code in XBMC could be used but I don't know if this is possible from an Addon.
- tafypz - 2010-10-12
I just updated the 10445 with mythxml.2.patch.
This patch queries the number of channels and get the channel list into xbmc.
I should have a full EPG tomorrow.
I did an initial test and the list of channels in the GUI was populated.
- tafypz - 2010-10-12
Thanks to dteirney mythxml.patch and mythxml.2.patch have been applied to svn.
There is no need to apply them right now.
I will provide a new patch tonight that will load the EPG.