(2014-09-10, 09:52)pecinko Wrote:

(2014-07-29, 19:09)m.savazzi Wrote: The original point was to have the possibility of using different DB servers as shared library among different XBMC.
In addition to optimize how the DB is managed (real transactions, one connection to db non thousands, etc..).

This lead to me thinking a new, simple but very flexible db structure to host all possible contents.

First of all many thanks and good luck with the project, I hope you'll manage to pull it off.

Some thoughts, in no particular order:

* It might be interesting to take a look at Plex media server (PMS) as well. HTPC client runs on XBMC fork after all and is open source. PMS is not open BUT, it uses SQLite DB and REST API is available via server_ip:32400/library. Thus, you should be to get a pretty good picture about what's going on. DB itself supports pretty much everything that is/was discussed here with exception of profiles. This is just in case you find it useful, not to say this DB is worse or better than MediaPortal/Browser's.

* Centralized server / distributed model / local content: I can see the point of local library when media is hosted on HTPC itself or attached USB disk as well as for centralized model running some kind of Kodi server.

So, how about leaving Kodi (client) local library as is with a possibility to announce it's media on LAN via UPNP or something else. In addition, for those who want it, there can be separate app, Kodi server, that could run on a separate PC or NAS. This one could offer more advanced library, manageable via web interface. For those wanting advanced library but not wanting to use server/NAS - they can install Kodi server locally, on the same HTPC.

This way we don't need to worry about library upgrade on Kodi and server app can be made and tested independently. Less savvy users can stick with what we have now if they prefer or if they find installing Kodi server app to complicated.

* Different movie versions, HD/SD etc - I would sacrifice this (and alike niche features) in a moment in order to simplify things and help library improvements get finally implemented.

Once again, thanks and good luck mate.

(2014-09-09, 07:57)Montellese Wrote:

(2014-09-08, 20:40)phate89 Wrote: But in this way don't you have 2 times the movie in the database with all scraped info with 2 different ids?

(2014-09-08, 22:17)m.savazzi Wrote: At program level you can simply add the details to the first version and copy them to the other or even not copy just leave the second record empty

IMO if a DB design results in (obvious and predictable) duplicate information there is something wrong. In the case of multiple versions of the same movie / asset IMO it makes more sense to detect this situation during library scanning and then linking the new file to the existing movie / asset instead of always retrieving both assets (with different database IDs) and then having to match multiple versions of the same asset every time they are shown in the user interface.

(2014-09-09, 07:57)Montellese Wrote: I'm actually working on this with the current DB layout and I added a table that links files to media items and during library scanning it detects "identical media items" (for movies it uses the title and year or imdbid). Then when retrieving all movies you get an item for every version of a media item (with the identical database ID and the identical metadata but with different file details like streams) which you can easily go through and put together all items with the same database ID (no additional complex logic that slows down the listing necessary) into a single item that is then presented like a movie set

(2014-09-08, 20:40)phate89 Wrote: Also about the watched status i think should be related to the episode, movie etc etc. It's the episode/movie itself that i watched not the specific file. If i have 2 versions of the file and i watch the first the 2nd should also result watched. Also if you remove a watched episode or movie and then you add it back the asset will be deleted and created so the episode will result unwatched, however since the episode entry won't be deleted the database will already have the watched status.

(2014-09-08, 22:17)m.savazzi Wrote: At program level you can simply add the details to the first version and copy them to the other or even not copy just leave the second record empty

Watched is associated to the asset so if you watch the Director's cut but not the theatrical movie it will be recorded correctly

If you watch the first the second should not be marked as watched OR if you want to have it watched the program can do it. This is a lot of freedom for Kodi

(2014-09-09, 11:08)phate89 Wrote:

(2014-09-09, 07:57)Montellese Wrote: IMO this is something that should be definable by the user i.e. the user should be able to choose whether watching the standard version of movie X should also mark the director's cut version of the same movie X as watched or not. Therefore it should be linked to the specific file and xbmc/kodi can implement the necessary logic to do it the way the user wants. That "freedom" is not possible if the watched status is linked to the media item / asset.

But what are the chances it will become an option? Usually small options like this will never go into mainline (and it's probably a good thing).
Btw i agree as i said before about the duplicate movie. Sometimes some sort of duplication is allowed to simplify the db and speed up access but in a media database movie is a major resource and it must be unique. Also it would be great if in the future kodi will not show twice the episode if we have 2 versions of it or we renamed it and not yet clean the database (it happens to me A LOT) and prompt a choice list after but that's probably part of the kodi code more than the db..

(2014-09-09, 11:27)Tolriq Wrote: Do not forget that the final plan is synchronization.

Same database ID for multiple media with different path could generate troubles for syncing watched status for the case of different watched status per media quality. (Specially with path translations or direct / indirect access ).
Not talking about remotes that will now have to deal with multiple path for a same ID, that will be complicated like hell to do offline download and sync.

But I'd also avoid the mess of Tv Shows with multiples ID per path with group by name that are well ...

Not sure there's an easy an optimal way to address this without major impacts, like for remotes Player.Open movieid 1, what file start ? Ask on the client ? Need to add the path first needing to ask the user at the remote side ?

Have not yet check the db proposal, but files / path or anything should be the object linked to physical details and description ... And we could use some kinds of collations to link the movies together like sets.

(2014-09-09, 11:27)Tolriq Wrote: Have not yet check the db proposal, but files / path or anything should be the object linked to physical details and description ... And we could use some kinds of collations to link the movies together like sets.

(2014-09-09, 14:02)jjd-uk Wrote: The support of multiple versions of one media item is certainly coming, as it's absolutely necessary for Montellese work in allowing media from multiple sources being added to the Library. For example you might have the same movie in:

1. HD version in local Library
2. SD version from UnPnP source e.g. tablet
3. Bookmarked in an addon

With Montellese's work from what I understand then all three of the above could be added to the Library.

So the question is then to what to present to the user? From what I understand concept similar to Movie Sets is seen as the best option, so you will see a single item for that movie, select it and you will then be shown the different versions. With Theatrical and Director versions I'm not sure what the plan is, I guess until there's some way for the scraper to support this then it would be similar to the moment with duplicate looking items being shown, but like now I again guess that you might be able rename how they are displayed e.g. "Film - Theatrical Cut" and "Film - Directors Cut".

(2014-09-09, 14:36)phate89 Wrote: Is it the best way to handle them as movie sets? In any view available you will have 3 items with same artwork, same title and only different stream details. Very confusing and not very immediate IMO..
Why not something similar to the bluray playlist selection with movie info? Kodi can compare them and simplify it in "sad version", "hd version" and "addon version".
About director's cut if the scraper supports it it's not hard to search "director's cut" or something like this in the file name but in this case we need a "version" attribute that the scraper can fill with the version name if they find it

(2014-09-09, 15:35)Fice Wrote: I think we should store additional information in the link from a file to an asset, as we have three situations where one movie might have several files.

A) If we have a 1 on 1 local copy from a UPnP source, there is no need to show two entries. And if one of them is watched, all of the copies should be marked as watched as well. (The link from file to asset should probably containt a foreign key to the original file?)

(2014-09-09, 15:35)Fice Wrote: B) If we have to different versions (uncut versus directors cut) then we should probably show them as a set and if one version is watched, doesn't mean we should automatically mark the other as watched as well (perhabs make it a setting). (The link from file to asset could probbly contain some metadata that is likely to differ: duration/PG rating/Year?!?)

(2014-09-09, 15:35)Fice Wrote: C) The same movie, the same version but different qualites. Is there a need for the user to select the qualities? Just play the highest quality the device can play. Would a user actually prefer SD over a HD? Not sure about 3D vs. non 3D version though. (Don't have a 3D tv)

(2014-09-09, 16:33)phate89 Wrote:

(2014-09-09, 15:35)Fice Wrote: C) The same movie, the same version but different qualites. Is there a need for the user to select the qualities? Just play the highest quality the device can play. Would a user actually prefer SD over a HD? Not sure about 3D vs. non 3D version though. (Don't have a 3D tv)

We need to le the user chose if mark as watched all the versions of the same movie but we won't let him play his sd file even if he wants to?
There are several reasons to have sd versions like bandwidth, different dubbing, different crop (i have some disney bd with it's dvd version because they did a poor job cropping only to have them in 16:9) and decoding limitations (even more now since hevc is rising and is only software).

(2014-09-09, 19:36)Tolriq Wrote: Same problem for remotes

Maybe different version needs to have some kind of a TAG representing the version difference, this would allow to play all with same tag, and remotes to play a movie ID with default TAG, or queue specific tag to not have to deal with path on client side to differentiate items.

(2014-09-10, 06:58)garbear Wrote:

(2014-08-30, 12:13)m.savazzi Wrote: Requests:
1) RetroPlayer being mainlined sooner or later might as well consider adding stand-alone games and game ROMs launched via emulators into the scope of the database model right away as well.
a. Added: Games are just an asset. Please let me know which properties we need specific for a game

from https://github.com/garbear/xbmc/blob/retroplayer-13.1/xbmc/games/tags/GameInfoTag.h

the fields I use in Game info tag:

• File path
  
• Title
  
• Platform
  
• Game Code (unique ID embedded in ROM)
  
• Region
  
• Publisher / Licensee
  
• Format (PAL/NTSC)
  
• Cartridge Type, e.g. "ROM+MBC5+RAM+BATT" or "CD"
  

I started with these fields because they can be populated directly from the ROM's data. you could get ideas about fields to add from thegamesdb.net (http://thegamesdb.net/game/1318/ ) like player count, release date, etc

keep in mind that another layout is needed for save states. the example save state database i created had fields for savestate path, rom path, rom CRC and gameclient used to create the savestate.

(2014-09-10, 13:07)Tolriq Wrote: The choice about 1 ID per real media or 1 ID by meta data is a db layout issue and this choice impact all the rest.

And yes obviously anything is possible with code, but some things are efficient some are not, and efficiency should be think about during db layout definition, since usage drives the layout.

If ID does not represent unique media then synchronization becomes complicated, and tons of things too and this should not be forgotten before stopping at the basic case when press play what happens.
(Of course could also be addressed as media ID and file ID, and with files CRC, file renaming could keep the file ID and correct adjustment to API / internals could be a solution).

But please as always in this thread think more global.

(2014-09-13, 01:39)phate89 Wrote: There is another wrong relationship.
A file could be more than one episode.
For example:
Lost.S01E01-02
In this case the file is both lost season 1 ep 1 and lost season 1 ep 2

(2014-09-13, 11:30)m.savazzi Wrote: Correct and this is NOT the case. Director's cut and Theatrical are TWO different movies, they have different poster, etc... They have different Runtime, different Year, etc...
The IMDB ID is different.

On the other side is true that they share a set of common information like Artists, director, etc...
What I mean to write is that the DB Structure is smart enough to handle this situation avoding duplicated info.

(2014-09-13, 11:41)m.savazzi Wrote: The DB fully supports this

In this case the structure will be:
File 1: Lost.S01E01-02

Asset 1: Lost Series 1 Episode 1
bound to File 1

Asset 2: Lost Series 1 Episode 2
bound to File 1

Quite straight forward.

(2014-09-13, 11:40)phate89 Wrote:

(2014-09-13, 11:30)m.savazzi Wrote: Correct and this is NOT the case. Director's cut and Theatrical are TWO different movies, they have different poster, etc... They have different Runtime, different Year, etc...
The IMDB ID is different.

On the other side is true that they share a set of common information like Artists, director, etc...
What I mean to write is that the DB Structure is smart enough to handle this situation avoding duplicated info.

It's not true.
First of all they have the same imdb id:
http://www.imdb.com/title/tt1411250/?ref_=nv_sr_1
as you can see same id SAME MOVIE 2 different lengths.
You can't say director's cut is a different movie. It's just wrong. It's the same movie..Often it's even in the same bluray and they share 90% of the video. It's only a different version of it. It's a big design flaw to let the same movie twice in the db...

(2014-09-13, 11:50)phate89 Wrote:

(2014-09-13, 11:41)m.savazzi Wrote: The DB fully supports this

In this case the structure will be:
File 1: Lost.S01E01-02

Asset 1: Lost Series 1 Episode 1
bound to File 1

Asset 2: Lost Series 1 Episode 2
bound to File 1

Quite straight forward.

Not according to the graph.
You have an asset that can have more than one file. One to many. Not the opposite..

(2014-09-13, 11:51)m.savazzi Wrote: Can you help me on IMDB. I'm not able to find the "normal" and the "director's cut" it just shows me the "normal" version.

With regards to being 2 movies is quite easy: if original movie is 120 min and directors cut is 150 min... they are different.
Also they have different votes, rating, MPAA, etc...
They are SOLD as two different movies, they have 2 different ISRC codes... etc...

If you check what I wrote there are NO duplications (or minimal) in case you save them as two assets so there is not a design flaw

(2014-09-13, 11:51)m.savazzi Wrote: No
is a M:N relationship.

One File can be bound to muliple Assets: case where one file is two episodes OR if you have an ISO where there are multiple movies (I have plenty)
One File can be bound to one Assets: easy case
Multiple File can be bound to One Assets: is a multipart movie/episode.


The structure MUST cover all this scenarios in the correct way.... and it does

(2014-09-13, 11:59)phate89 Wrote:

(2014-09-13, 11:51)m.savazzi Wrote: Can you help me on IMDB. I'm not able to find the "normal" and the "director's cut" it just shows me the "normal" version.

With regards to being 2 movies is quite easy: if original movie is 120 min and directors cut is 150 min... they are different.
Also they have different votes, rating, MPAA, etc...
They are SOLD as two different movies, they have 2 different ISRC codes... etc...

If you check what I wrote there are NO duplications (or minimal) in case you save them as two assets so there is not a design flaw

It's not true.
Actually most of blurays contains both versions. It's quite easy with bluray you split the movie into segments and have 2 different playlist combining the parts in common and the different parts. It's called seamless branching. In a bright future this could be achievable even with mkv and stacked files so you will have 2 versions of the same movie (director's and theatrical) in the same mkv file with almost the same size of a single movie (actually if we want to think very wide we should also consider this possibility)
2nd thing there are no different votes, rating, mpaa. They are the same movie. I didn't pass you the theatrical cut version. I passed the only version there is of that movie. if you open that link and go to look to the movie length you get:
Technical Specs
Runtime: 119 min | 127 min (director's cut)
There is no separate version. It's always the same movie with a different length.
OF course if they are sold separately they have a different code but also dvd->bluray version have different code, remastered has a different code. I lost count of how many star wars different versions went out. 6? 7? But we still consider them the same movie in all these cases.

(2014-09-13, 11:51)m.savazzi Wrote: No
is a M:N relationship.

One File can be bound to muliple Assets: case where one file is two episodes OR if you have an ISO where there are multiple movies (I have plenty)
One File can be bound to one Assets: easy case
Multiple File can be bound to One Assets: is a multipart movie/episode.


The structure MUST cover all this scenarios in the correct way.... and it does

(2014-09-13, 13:13)m.savazzi Wrote: Well for me if a movei has diferent Runtime is a different movie... but I'm ok with your view.
The DB supports it with no issue with 2 assets.

I think I have more than 15 Star wars (original) copies between VHS, SCD, DVD, Bluray, etc...

(2014-09-13, 13:25)phate89 Wrote:

(2014-09-13, 13:13)m.savazzi Wrote: Well for me if a movei has diferent Runtime is a different movie... but I'm ok with your view.
The DB supports it with no issue with 2 assets.

I think I have more than 15 Star wars (original) copies between VHS, SCD, DVD, Bluray, etc...

But that'S a flaw in the design that will cause inconsistency and makes harder to obtain movies only. It's not hard to split assets from movies and prevent that kind of duplication leaving the different tag (like duration) to the asset or the file.

(2014-09-13, 13:25)phate89 Wrote: Also this kind of separation is needed for montellese's work (just like the "sources" folder that is still missing) so it needs to be in the db. In this case in montellese's work you have everytime to group to find out if a movie is already in the database and it's not efficient. Also it will create inconsistency.

(2014-09-13, 13:25)phate89 Wrote: Same movie, different art, different plot, different rating and even different genre is possible.

(2014-09-13, 13:25)phate89 Wrote: Why the asset have to be the movie? They are 2 different things so they should be separate in db to bring easier access and remove inconsistencies

(2014-09-13, 14:25)m.savazzi Wrote: Asset is the parent of movie. An asset can be a movie, an episode, a game, a fanart, a game, a picture... etc...

I think you made a little confusion on that.

(2014-09-13, 15:26)phate89 Wrote: Again it doesn't matter what is the path and what is the file name. Kodi needs to store somewhere if the user wants to automatically import media from a specific kodi instance or not. If you have 3 other kodi pcs in your network and you want to automatically import from "k1" only movies, from "k2" only music and import nothing from "k3" you need a table to store it where kodi will look. Something like:

Btw theatrical IS the same movie of director's. You can look everywhere and it's always one movie. Maybe they split it in versions or something else but is still logically a unique thing with eventually 2 child items. It will always have same main title, same plot, same genre, same rating between all versions and you need consistency between them and make it easy to access.