[DaveBlake]

As to EnTity Relationships Diagrams... that just sounds like someone's pr0n habits.

Or is it that slightly annoying side of any job where complex words are made up to explain a handful of lines scribbled on a pad of paper showing something obvious... I expect it will be as useful as that wiki page where it is just a list of field names written out in a slightly more verbose form without any attempt to actually explain anything

Yeap, you got it. Nice thinking scribbles, or a visual summary/memory jogger for someone that knows a design, but alone not very informative. Another design snapshot is CRUD tables - what processes Create, Read, Update or Delete records from each table, but again "when" and "why" is often as important as "who dunit". Don't get me wrong, they all have uses, but alone none of them are a full spec.

[BatterPudding]

As to EnTity Relationships Diagrams... that just sounds like someone's pr0n habits.

Or is it that slightly annoying side of any job where complex words are made up to explain a handful of lines scribbled on a pad of paper showing something obvious... I expect it will be as useful as that wiki page where it is just a list of field names written out in a slightly more verbose form without any attempt to actually explain anything

Yeap, you got it. Nice thinking scribbles, or a visual summary/memory jogger for someone that knows a design, but alone not very informative. Another design snapshot is CRUD tables - what processes Create, Read, Update or Delete records from each table, but again "when" and "why" is often as important as "who dunit". Don't get me wrong, they all have uses, but alone none of them are a full spec.

It is all clues to the puzzle... and a CRUD table does kind of explain some of the weird stuff I am seeing... tables with random CRUD in them.

[scott967]

IIRC there are 3 types of bookmarks -- resume points created by Kodi, user created bookmarks (visible in boookmarks window) and episode bookmarks (used when multiple episodes exist in a single file).

scott s.
.

[black_eagle]

IIRC there are 3 types of bookmarks -- resume points created by Kodi, user created bookmarks (visible in boookmarks window) and episode bookmarks (used when multiple episodes exist in a single file).

scott s.
.

Yes, I need to study this a bit more although I have a working add-on that can currently skip the bookmarks so if a user has any, they won't be deleted. It also can keep or remove pvr files. Any file that is related to a source in a users sources.xml is automatically kept and the script will produce an SQL statement which you can approve, in which case it gets executed, or you can turn down, in which case the script just quits. By default, the script calls the built in 'clean library' routine if it deletes anything, but this can be turned off if desired (not recommended though).

Needs some more testing but it's getting near to a beta release. MySQL windows users will have to wait a little but I'm working on an installer for the needed python library. Linux users can install it with 'apt-get' or build it from source. Instructions will be in the readme. Cross-platform SQLite works fine.

~~~~~EDIT~~~~~

Figured the Windows stuff. Just need to drop 3 files in the correct places.

[grumbler]

Hello there
I am following this thread for a while now and wanted to ask a question because I have the feeling I am missing something:
You are working on a method to delete all the references to old video files, streams and whatnot that were once watched in XBMC. This sounds like a really good idea, because I second the opinion that its a questionable idea to store all this stuff into the database and have no way of cleaning it after a while.

But:
Wouldn't it be easier to do an "Export database to separate files", then drop the whole database and let Kodi create it new from the stored .nfo files?
Maybe I am wrong, but I suspect this would also result in a clean database with all necessary data restored.
Or am I missing something here?

[black_eagle]

Hello there
I am following this thread for a while now and wanted to ask a question because I have the feeling I am missing something:
You are working on a method to delete all the references to old video files, streams and whatnot that were once watched in XBMC. This sounds like a really good idea, because I second the opinion that its a questionable idea to store all this stuff into the database and have no way of cleaning it after a while.

But:
Wouldn't it be easier to do an "Export database to separate files", then drop the whole database and let Kodi create it new from the stored .nfo files?
Maybe I am wrong, but I suspect this would also result in a clean database with all necessary data restored.
Or am I missing something here?

It would indeed result in a clean database, but this is a faff that probably most people can't/don't want to be bothered with. My own library is totally built from local files as I use a 3rd party manager for creating them but it is still much easier to click on an addon and have it do all the work for you. It is also much faster. The slowest bit is once the addon finishes, it calls Kodi's built in clean library routine, which cleans all the other tables.

The other thing is that most people probably don't realise the amount of stuff stored in there - I certainly didn't until I looked. An addon provides an easy, convenient method to remove it.

Also, it's been fun to write and collaborate on. Both BatterPudding and I have learnt a lot.

I finally realised I don't need any 3rd party support for MySQL (thanks to the wiki), just a dependency in the addon.xml file. This means the addon should run on all the platforms that Kodi runs on, regardless of which type of database the user is running.

[pr0xZen]

I see work is coming along greatly. the community spirit is very much alive and thriving. Great work so far guys / gals! Let me know if something should be changed or ammended to #1 post.

[grumbler]

Also, it's been fun to write and collaborate on. Both BatterPudding and I have learnt a lot.

I finally realised I don't need any 3rd party support for MySQL (thanks to the wiki), just a dependency in the addon.xml file. This means the addon should run on all the platforms that Kodi runs on, regardless of which type of database the user is running.

Don't misunderstand me, I think its great what you are doing. I just wanted to be sure that there isn't some important stuff in the database that I don't know of and that isn't stored in the nfo files. Because I dropped my DB already multiple times thinking that I can create it with everything important from the files.

I am also a big fan of the texturecache cleanup utility. Everything that discards old unnecessary stuff is great

[black_eagle]

Yes, texturecache is a great bit of work and can do lots of things. I'm also a big fan of it.

Our addon aims to remove rtmp://, rtmpe://, http:// and plugin references from the files table. If it is run using the default settings, it builds an SQL query that excludes all the path sources defined in your sources.xml file and deletes everything else. This behaviour can be changed as BatterPudding has found during testing that some people don't have any sources defined as such and just use filemanager to navigate with. It's now possible to turn off using sources.xml and use a default set of rules to select and delete the aforementioned paths.

When the addon has removed the required paths from the files table, the built in 'clean library' routine is called which scans the file table and removes any associated references in the rest of the tables to the links we have just deleted.

The addon will prompt you before deleting anything by showing you the SQL statement and asking if you want to execute it. This prompt can be turned off if required.

I just wanted to be sure that there isn't some important stuff in the database that I don't know of and that isn't stored in the nfo files. Because I dropped my DB already multiple times thinking that I can create it with everything important from the files.

I would always recommend backing up your database(s) before using something like this. In fact, I can probably make that an option - to auto back-up the database before executing the SQL although I have run it several times on my own database with seemingly no ill effects.

[black_eagle]

Some taster images



Settings







If anybody has any comments or suggestions as to what should be added/changed please feel free to post them !!

[Scott00007]

Looks awesome!! Cannot wait to have the ability to keep things much cleaner!!

Thanks for the work.

[ashlar]

Wow! Thanks so much for your work on this. Shouldn't the discussion be moved under add-on development considering its... well... development?

[ReplayHarry]

Looks great so far. I will be glad to be one of the early adopters. I know I have plenty of stuff needing cleaning up.

[scott967]

Looking good. My suggestions are:
1. ensure capable of handing UTF-8 character encoding
2. ensure capable of using in profiles

scott s.
.

[BatterPudding]

Wow! Thanks so much for your work on this. Shouldn't the discussion be moved under add-on development considering its... well... development?

This is the discussion that started it all... but we will be creating an add-on thread for it when it is released (shortly). When the discussion started none of us had really planned to be creating the fix like this. KODI is just so darn flexible that it is fun diving in and fixing things. The whole point of a community project - if you spot something you don't like, you gather together with some other like minded strangers, and fix it.