Pictures database and library mode; scan EXIF and IPTC metadata tags into a SQL db

Running with newest version 0.6.6 on Windows Server 2008 R2 and scanning speed has drastically improved. As mentioned previously building the database with about 7000 images from scratch took between 10 and 12 hours to complete. Version 0.6.6 builds almost the same database in 45-50 minutes.

I do still experience two issues.

One is that I still can't select a person with many images attached. In the log a python error is mentioned. Relevant log is posted here: http://pastebin.com/EdMYbcjQ

The other issue is somewhat stranger. Some keywords don't show up - no matter which picture is tagged with them. For example - I'm tagging images with the car featured, but the tag "Hyundai" will not show up - no matter which image is tagged. I'm still testing this issue though.

FFY

EDIT: Updated with relevant info.

Can't add a new folder with the 0.6.6
I can't browse

(2012-04-09 08:49)mgmartin Wrote:  Still seems to have issues under Linux.
I have to trap exceptcions in the picentry={...stat...} code to successfully scan the folders.

Which directory permission is missing x or r?
I need it to know in order to test the plugin within a vmware linux.

---

(2012-04-09 13:54)fmronan Wrote:  Can't add a new folder with the 0.6.6
I can't browse

Did you add picture sources?
The browse dialog only shows directories which are added as sources.

(2012-04-09 13:21)fungify Wrote:  Running with newest version 0.6.6 on Windows Server 2008 R2 and scanning speed has drastically improved. As mentioned previously building the database with about 7000 images from scratch took between 10 and 12 hours to complete. Version 0.6.6 builds almost the same database in 45-50 minutes.

I do still experience two issues.

One is that I still can't select a person with many images attached. In the log a python error is mentioned. Relevant log is posted here: http://pastebin.com/EdMYbcjQ

The other issue is somewhat stranger. Some keywords don't show up - no matter which picture is tagged with them. For example - I'm tagging images with the car featured, but the tag "Hyundai" will not show up - no matter which image is tagged. I'm still testing this issue though.

FFY

EDIT: Updated with relevant info.

1) Seems (I'm not sure) to be a problem with GPS coordinates.
Can you PM me the picture "På høreklinikken 27-02-2012\IMAG0029.jpg" or a similar one because I don't own any GPS tagged pictures.

2) Is it possible to PM the database C:\Users\Administrator\AppData\Roaming\XBMC\userdata\Database\MyPicture.db.
I already found some problems when deleting and rescanning pictures. Then sometimes old keywords aren't deleted and the new pictures doesn't get keywords due to the "old not deleted" ones.
You can rename or delete the db and try to rescan your pictures, too.

ok, sometime I'm stupid
Added 0.6.6 in passion repo

(2012-04-10 11:47)Xycl Wrote:  1) Seems (I'm not sure) to be a problem with GPS coordinates.
Can you PM me the picture "På høreklinikken 27-02-2012\IMAG0029.jpg" or a similar one because I don't own any GPS tagged pictures.

PM send.

(2012-04-10 11:47)Xycl Wrote:  2) Is it possible to PM the database C:\Users\Administrator\AppData\Roaming\XBMC\userdata\Database\MyPicture.db.
I already found some problems when deleting and rescanning pictures. Then sometimes old keywords aren't deleted and the new pictures doesn't get keywords due to the "old not deleted" ones.
You can rename or delete the db and try to rescan your pictures, too.

I am actually right now rebuilding the database from scratch (deleted the old one) to see if that might help. Will update if tags / keywords are still not added.

FFY

(2012-04-10 12:40)fungify Wrote:  

(2012-04-10 11:47)Xycl Wrote:  1) Seems (I'm not sure) to be a problem with GPS coordinates.
Can you PM me the picture "På høreklinikken 27-02-2012\IMAG0029.jpg" or a similar one because I don't own any GPS tagged pictures.

PM send.

FFY

I sent you a PM.

---

(2012-04-08 21:45)mikebzh44 Wrote:  How I set Home property with RandomAndLastItems script :

WINDOW = xbmcgui.Window( 10000 )
_setProperty( "PictureDB.%d.Path" % ( _count ), path )

So skinners can use PictureDB.1.Path to PictureDB.10.Path for example.

There is a problem.
The MyPicDB Addon cannot set the properties because it doesn't run when XBMC shows the main menu

Two quick comments / bug reports.

* When adding a folder and the moving to the topfolder (by clickinging ..) the addon sometimes delete the folder that has just been added. I even get the message as if I actually chose this option. This is a new bug that was introduced in 0.6.6 and might be related to the fix you made to dialoges that didn't show?

* When scanning for keywords, it would be great if you would expand the fields you scan. Right now the only keyword field that is scanned seems to be [IPTC]Keywords but that one is far from a standard tag. Different programs seems to use different tags, and IPTC seems to lose users to XMP.

Looking through my images that I've managed with a selection of utilities, I can identify the following tags with keywords:

[IPTC]Keywords
[XMP-microsoft]LastKeywordIPTC
[XMP-microsoft]LastKeywordXMP
[XMP-dc]Subject
[IFD0]XPKeywords

Might I suggest some sort of prioritized reading of tags, so that if one tag is empty, the next one is read?

FFY

(2012-04-11 19:59)fungify Wrote:  Two quick comments / bug reports.

* When adding a folder and the moving to the topfolder (by clickinging ..) the addon sometimes delete the folder that has just been added. I even get the message as if I actually chose this option. This is a new bug that was introduced in 0.6.6 and might be related to the fix you made to dialoges that didn't show?

FFY

The complete "Add path" menu must be rewritten.
I got the same message after deleting a folder and then adding one or more folders but never when I only add folders.

(2012-04-11 19:59)fungify Wrote:  * When scanning for keywords, it would be great if you would expand the fields you scan. Right now the only keyword field that is scanned seems to be [IPTC]Keywords but that one is far from a standard tag. Different programs seems to use different tags, and IPTC seems to lose users to XMP.

Looking through my images that I've managed with a selection of utilities, I can identify the following tags with keywords:

[IPTC]Keywords
[XMP-microsoft]LastKeywordIPTC
[XMP-microsoft]LastKeywordXMP
[XMP-dc]Subject
[IFD0]XPKeywords

Might I suggest some sort of prioritized reading of tags, so that if one tag is empty, the next one is read?

FFY

(2012-03-26 22:16)Xycl Wrote:  Suggestion for further development:

1)
There are two tables e.g. Cities & CitiesInFiles for each supported exif, iptc or xmp tag.
Therefore adding new kinds of tags is quite time consuming.
2 tables, 3 database class methods and the menu handling must be added.

Instead of using a table like Cities (key, content) or Categories (key, content) a meta table (key, tag-type, content) would be better because then adding new tags types is only done in file parser. Selecting the tag types from meta table and adding menu items can be done programmatically.
There would be only one menu item "keyword" or "tag" which opens a sub menu displaying all available tag types from meta table.

2)
There should be a mapping table. Then we can join e.g. iptc keyword tag with exif xpkeyword tag. The user would only see one menu item called keyword.

3)
An advanced filter. Perhaps a wizard.
Select pictures from country abc with keyword xyz containing person def. Needs 1) otherwise it's not maintainable.

Any opinions?

I already started implementing part 1).

Oh - that's right. I did actually think we had this conversation before, but looking back in the thread I couldn't find the evidence. Good to hear that you are on the case.

Just spend around 10 hours copying tags from [XMP-microsoft]LastKeywordXMP (which all of my pictures had) to [IPTC]Keywords (which only around a third had). MyPicsDB only reads the last tag, and this was why I did not see the correct amount of images under each tag and why some tags where not read at all.

Now it works and I'm a happy camper. I'm just not sure what happens when I add new images, as I still don't know if WLPG writes one or the other tag - though as previously mentioned I actually thought I did.

FFY