• 1
  • 3
  • 4
  • 5
  • 6(current)
  • 7
DB - SQLLite/MySQL - Performance issues
#76
1) profile is on db so all xbmc read the info from there, in the config file or UI just the db address is needed.
2) if you use transactions the data on db is always statefull and users will be saved correctly. Like every system the first user you create is the "admin" able to create and/or modify other users. The other users can be admin or user (no mod on users). Admin can decide who sees what's.
This management is very flexible so you can associate content,menu items, etc...to a profile. Its more on the UI side than on the db. This is another advantage in terms of flexibility on the UI
3) once we build a structure that is correct and flexible as a first step probably we can simply "migrate" actual behavior and then evolve.
4) totally agree with you, no dreams but challenging objectives Wink. I do not think is a trivial change but is mandatory to bring xbmc to the next level and the more we wait the more it will be bad/impactfull because will continue to grow in time. A "design flaw" (aka requirement changed over time, not saying it was wrong) need to be fixed as soon as possible.

anyway started to download all code, trying to get a successful run and this thread and other info is helping me a lot. Will start to analyze queries and data then will try to propose something to the team, starting from design...
If you think I'm useful please use the +/- button to raise my reputation
Reply
#77
all the syncing logic is IMO not related to the DB abstraction itself but rather a new/altered db schema. The abstraction doesn't have to care how the schema looks like, it only has to be able to work with a schema description. But ofc, these things have to be ironed out on the long run and AFAIK some team devs already had some ideas on the syncing stuff.
Reply
#78
Well abstraction needs a model and the model is designed from the need.

Starting to build an abstraction layer when you do not known the model is like building a tower without being sure the foundation is at the correct place.
And to move the tower to the new place unless you have some giant robots to do the work you have no choice than building new foundations and restart over it Wink

And as said before if the discussion moved a little from performance to synchronization is that the need for Mysql might be irrelevant in the new architecture, as a powerful sync system on a native database tool will always perform better than an abstraction layer that will be also tied to each database "addon" performance.

And using energy and time to build correct things is better than doing stuff multiple times.

I will not come back again on my attempts to code for XBMC to always finish in code then we discuss instead of discuss then code. I just can't understand how it can works in the long term.

But letting m.savazzi use his energy on this without a proper target validated is a pure loss that could be used for a pure win.
Reply
#79
who has to validate the target? is there a "God in a bottle"? Smile

In any case am I right saying this?
1) current db implementation has issues
2) current db structure has issues and is not flexible
3) no effective centralized db is really possible now
4) no multi db support is possible today
5) a real shared library cannot be implemented right now

So I can try to see if I can find a feasible solution for the above... then we will discuss framework, code etc...

I do not want to waste my time (or yours) but we need to fix those points in XBMC

M
If you think I'm useful please use the +/- button to raise my reputation
Reply
#80
Tolriq IMO abstraction has to be indepented from DB schema. Ofc there has to be a schema, but it should be some XML description or something and thus be interchangeable. It's the communication between XBMC and the DB that has to be abstracted. But whatever.
Reply
#81
I forgot
6) performance issues on library generation and update


M
If you think I'm useful please use the +/- button to raise my reputation
Reply
#82
I've found a free tool for data modeling:
https://www.youtube.com/watch?v=Xj52wxV0314#t=40

here to get:
http://www.casestudio.com/enu/download.aspx

Will work on the model

Note: there is a totally free edition for students and Teachers: https://www.toadworld.com/?Redirected=true
If you think I'm useful please use the +/- button to raise my reputation
Reply
#83
@da-anda : I do not speak about DB schema but object model Smile If you abstract the DB layer you need to talk in pure object at XBMC level. And this will more or less limits the possible DB schema but this is not the main point.

Knowing what object you deal with and how they are related is the basic of abstraction. Then you can code the database access part that will obviously be more or less dependent of your objects or could be autogenerated from them.
Reply
#84
sure, but how objects are related would IMO be defined in the XML schema, and the abstraction layer will be using this object model schema to build the required SQL data structure.

@m.savazzi you know about the GSOC project that was about a new DB architecture/structure? Just mentioning in case you haven't seen this yet.
Reply
#85
Well the model must define how object are related to each others.

If the model does not know that Users and Files are related and if 1-1, 1-n or n-n, you cannot say I want all files from that users. The abstraction layer will then use this information to build a schema and choose to have a PK or a link table but underlying schema do not have to be known from the object model.

But the base is still the object model with relations. Then the rest will just follow easily, doing it in reverse order can not work
Reply
#86
Da-anda
found Little information on the forum but not the outcome of the gsoc Project.
if you can help m gather their info I will look into it.

in any case I was already working on a db model for ember media manager and has quite some resemblances with xbmc data Wink
If you think I'm useful please use the +/- button to raise my reputation
Reply
#87
Much looks to have been discussed in great detail before and in a few cases some work even started here:

http://forum.xbmc.org/showthread.php?tid=165024

http://forum.xbmc.org/showthread.php?tid=141169
https://github.com/garbear/xbmc/commits/photolibrary

http://forum.xbmc.org/showthread.php?tid=155304
https://github.com/declankenny/Video_Database_Proposal

http://forum.xbmc.org/showthread.php?tid=73831
http://wiki.xbmc.org/index.php?title=Dat...Schema_4.0
http://wiki.xbmc.org/index.php?title=Dat...hema_4.0/a

http://forum.xbmc.org/showthread.php?tid=165929

But as far as I understand none of these previous ideas and concepts have yet made it into mainline, or?
Reply
#88
(2014-07-20, 14:02)RockerC Wrote: Much looks to have been discussed in great detail before and in a few cases some work even started here:

Will read and study....

Thank you a lot
If you think I'm useful please use the +/- button to raise my reputation
Reply
#89
Thanks for the info... I think I will propose a slightly different approach that should be more flexible and future proof. Also I'm looking at one db for everything not just the video part
If you think I'm useful please use the +/- button to raise my reputation
Reply
#90
Ok here is the first part of the model:
http://1drv.ms/1Aj0KCz

The movies table still miss a lot of fields and I have not added yet the season-series, also there is a separate are for the TV-EPG

but the core is there...

Some of the key concepts:
1) there is a file table that includes all files independently from what they are
2) files are grouped in libraries
3) users can filter libraries
4) an asset is composed by different files (i.e. multipart movie)
5) assets can be: movie, episode, pictures, music, thumbnail, module, etc...
6) each specific asset does have it's own set of specific properties
7) rating/stars are linked to assets
8) series/seasons are a grouping of assets
9) watched is linked to assets
10) there is a path table where we can store any path format and the format is speficied in path/type. In this way we can have Linux based path/samba/URI etc... also path substitution becomes very easy

CRC32 is used to identify univokely a file

I think this base structure takes in account everything I've read so far in the different posts GSOC and in the code.

For pictures I've already added the EXIF information and we can easily add an Album level as a grouping of pictures like a season/series is a grouping of episodes.
For movies Sets are just a grouping of movies.
If you think I'm useful please use the +/- button to raise my reputation
Reply
  • 1
  • 3
  • 4
  • 5
  • 6(current)
  • 7

Logout Mark Read Team Forum Stats Members Help
DB - SQLLite/MySQL - Performance issues0