[spiff]

they are accessed through http - so whatever url you use... which prob needs manual configuring

[ABarbaccia]

from CouchDB.org:

CouchDB is a peer-based distributed database system, it allows for users and servers to access and update the same shared data while disconnected and then bi-directionally replicate those changes later.

I just don't understand if there is a master or always-on shared file store somewhere that all machines are aware of.

I saw some performance metrics between CouchDB and mysql. On a dataset of 50,000 records, query performance was identical between the two. Index and table creation was 50x slower, but that is only seen during the first query to the view.

[jmarshall]

The music library will stay as-is simply as I don't have the time to commit to looking at it. Assuming the video library rewrite goes well, there's no reason why couchdb can't be used for everything else (it makes sense to do so, after all).

To get a feel for how couch works, pop over here:

http://labs.mudynamics.com/wp-content/up...couch.html

Cheers,
Jonathan

[wstewart]

Assuming the video library rewrite goes well...

Could you provide more info about the "video library rewrite"? I've read the other thread where you are asking for idea, but I don't have a clear idea of the objectives.

Besides an overview of the re-write, I am most interested in the planned changes to the video database code.

[jmarshall]

The other thread sums things up quite well.

Each "root node" in the library is essentially an independent source. Think of how files mode works, but change it so that when you click on a source you can have either folder browsing or browsing by your library. There is no longer a files and library view. It's all in one.

Automatic updating as you browse is supported - essentially the database is pre-filled with filename information, and then local or online sources are queried for the metadata info.

Arbitrary information will be allowed (key/value pairs) beyond the basic info to retrieve and sort a listing. If we stick with SQL, then this additional info is retrieved in a second pass of the database, so we'll have a fast initial listing, and metadata info (thumbs etc.) will then be background filled. Obviously it'll be cached so that next time you go in you'll get the metadata straight away. Not sure exactly how we'll do it with CouchDb.

Nodes at the root level can be virtual folders of nodes, or can be filter nodes (eg smartplaylists), category nodes (eg genres) or can be actual sources. Obviously the entire hierarchy is user-definable - it'll basically be a filesystem on disc.

In short a much more flexible system for the power users, while still having things work simply for those that just wanna add a folder of videos and play 'em.

As for changes in the database code: Essentially it's a rewrite from scratch.

Cheers,
Jonathan

[wstewart]

Much appreciated for the explanation, I hope my suggestion for custom categories (ie. children, mom, dad, exercise, home vids) will be considered.

I'm just getting into XBMC database code, if it works out, I hope to leave as much of the existing video/music/program database code as is and just translate it at the database abstraction level. If it works, this should work well for your code re-write.

The main changes I see are adding host name, password and userid and perhaps the table names may need to change. I think that each profile will need its own database and all tables would be in the profile database. But since the table names conflict between music, video, etc., I am leaning towards prefixing the table names with video_, music_, etc.

[jmarshall]

"Each root node is a separate source". You can put whatever you like in each of them. They're completely independent, so you can have all your kiddie movies nicely separated out.

Furthermore, the idea is that Category nodes or Filter nodes can filter or categorize across multiple source nodes if you wish to combine them in some way.

Cheers,
Jonathan

[slash]

If anyone wants to help out at the exploratory phase, give me a shout. Essentially we need to:

1. Setup an example db with a bunch of documents with whatever fields we want.
2. Setup a bunch of views to give us what we have today.
3. Look at how easy/hard it is to generate dynamic views for smartplaylists and the like - perhaps they can simply be generated directly based on input from the UI and be kept in the db rather than hanging on to their XML descriptions?

Cheers,
Jonathan

The concept sound very interesting. I would be glad to help on this.

I've been looking a way to distribute the library and managing offline medias.

Let me know how I can contribute.

Cheers

[Gamester17]

I saw some performance metrics between CouchDB and mysql. On a dataset of 50,000 records, query performance was identical between the two. Index and table creation was 50x slower, but that is only seen during the first query to the view.

...and PostgreSQL is twice as fast as MySQL on multi-processor CPUs?
http://www.wikivs.com/wiki/MySQL_vs_PostgreSQL

[ABarbaccia]

I think that each profile will need its own database and all tables would be in the profile database. But since the table names conflict between music, video, etc., I am leaning towards prefixing the table names with video_, music_, etc.

This could be done with a single database.

Add a profile table and a profile_id field to the music, video, etc tables. Then you can query only the rows that correspond to a specific profile. The database abstraction layer will have to be made aware of the active profile_id and sql modified to include it. you're sql will be something like this: select artist from music where profile_id = 1.

Setting up multiple databases would require granting access to multiple databases and lots of ugly other stuff too.

[wstewart]

This could be done with a single database.

Add a profile table and a profile_id field to the music, video, etc tables. Then you can query only the rows that correspond to a specific profile. The database abstraction layer will have to be made aware of the active profile_id and sql modified to include it. you're sql will be something like this: select artist from music where profile_id = 1.

Setting up multiple databases would require granting access to multiple databases and lots of ugly other stuff too.

I agree that would be idea, but I am looking to maintain compatibility with the existing sqlite databases and minimize the code impact at this stage. Perhaps I should give some more thought though.

[Gamester17]

This could be done with a single database.

Add a profile table and a profile_id field to the music, video, etc tables. Then you can query only the rows that correspond to a specific profile. The database abstraction layer will have to be made aware of the active profile_id and sql modified to include it. you're sql will be something like this: select artist from music where profile_id = 1.

Setting up multiple databases would require granting access to multiple databases and lots of ugly other stuff too.

I agree that would be idea, but I am looking to maintain compatibility with the existing sqlite databases and minimize the code impact at this stage. Perhaps I should give some more thought though.

Perhaps keep it in mind now already for a possible merge in the future?
So keep compatibility today but you are prepared for a future merge

[firnsy]

G'day,

I've been lurking the forums for a long time and with my 3rd XBMC client being built, complete centralised management of my media is moving high on my priority list.

I have been doing a lot of browsing of the codebase lately and agree that it is going to be a sizeable task with:

• 2400+ SQL calls (most of which are "portable"),
  
• separate databases for music, video, programs and views, and
  
• the current user profile implementation.

However, there are a lot of fruits to bear by going down this path and with the UMM project in its relative infancy, the video DB getting an overhaul it would be a good time to get this under way.

Of all projects described in this thread, I'm of the opinion that OpenDBX[1] is the most appropriate with it's preference for speed whilst still remaining very lightweight. It supports the most likely of choices for DB backends (mysql, postgresql) as well as the current DB backend (sqlite3). The project has a very tight API, and is still reasonably maintained.

I believe the appropriate steps for pushing this forward include, but are not limited to:

• Determine who will be contributing (including dev platforms) to this effort
  
• Examine the 2400+ SQL calls and ensure their "portability"[2]
  
• Integrate the OpenDBX library (perhaps initially coexisting with dbiplus and incrementally phased out)
  
• Coordinate and align all DB calls to a the common API
  
• Document the code and API (join the "commenting" revolution)
  
• Let the good times roll ...

Looking forward to give back to something I've taken SO much from

Footnotes:
[1] http://www.linuxnetworks.de/doc/index.php/OpenDBX
[2] http://www.linuxnetworks.de/doc/index.php/Portable_SQL

[jmarshall]

Looks reasonable to me - it's available for all platforms we serve.

[CapnBry]

I'm all for replacing the DB backend with something client/server, but isn't OpenDBX kind of ...light on functionality? Sure it provides a common API to a bunch of databases, but it doesn't seem to add anything beyond very basic string-based query/result. Notably missing:
-- Retrieving result values in native types (every value is char *)
-- Parameterized queries

The first we have with the current dbiplus, the second would be nice to have instead of using the workaround "escape the SQL" method.

Again I will also bring up the fact that SQL itself is the tricky bit. How will this line of code look in a backend-agnostic manner?

Code:

UPDATE song SET iTimesPlayed=iTimesPlayed+1, lastplayed=CURRENT_TIMESTAMP where idSong=%ld

Things like CURRENT_TIMESTAMP, LIMIT, ALTER/CREATE TABLE are specific their backends. Ideally the abstraction layer chosen should help reduce these issues. I've never been happy with any of the SQL abstraction layers though, so maybe this point is moot.

At the very least I'd expect the chosen dbiplus replacement to support 1 (if not both) of the issues I listed above.