XBMC Community Forum
Pictures database and library mode; scan EXIF and IPTC metadata tags into a SQL db - Printable Version

+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Development (/forumdisplay.php?fid=32)
+--- Forum: Python Add-on Development (/forumdisplay.php?fid=26)
+--- Thread: Pictures database and library mode; scan EXIF and IPTC metadata tags into a SQL db (/showthread.php?tid=80845)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

Pictures database and library mode; scan EXIF and IPTC metadata tags into a SQL db - solexalex - 2010-09-10 21:53

2006, October 4th, whodini expose something very interesting as a new feature for XBMC. He asks for a picture management like videos and musics have, storing informations inside a database and accessing at will.

Many of us, and surely a lot of people that never write on forums, are asking themselves "Why the picture section is not used through a database ?" but no answer from team.

Again now I still think it's a must have, but as it is not included inside xbmc itself, I made a plugin for that. It is called "MyPicture Database"

Before continuing, you may want/need to have a look to the original conversation :

As the plugin is now released on the official new XBMC dharma addon repository, I opened a topic in the python developpement section for debugging or just for feature requests.

- Recursive jpg picture scanning (gif, png, bmp, tif ... may be added soon)
- EXIF / IPTC pictures metadata databasing
- By date filtering
- By folders filtering
- By keywords filtering
- local or distant files scanning (need to mount a virtual drive to use network folders)
- A python library behind the system so that developpers can write their own plugins/scripts

- solexalex - 2010-09-11 00:48

If everything goes well, you can expect a new version soon (maybe Sunday night in France)
-more file extension discovered and added in the database (but IPTC/EXIF will be only for jpg's)
-Number of pics for each filter
ex.: date : 2009-12 (142 pics)
ex.: keyword : Tour Eiffel (231 pics)

- tslayer - 2010-09-11 18:47

I tried the plugin on linux but get this:

19:09:58 T:140552301729552 M:1088495616 INFO: -->Python script returned the following error<--
19:09:58 T:140552301729552 M:1088495616 ERROR: Error Type: exceptions.ImportError
19:09:58 T:140552301729552 M:1088495616 ERROR: Error Contents: No module named sqlite3
19:09:58 T:140552301729552 M:1088495616 ERROR: Traceback (most recent call last):
File "/home/eric/.xbmc/addons/plugin.image.mypicsdb/default.py", line 31, in ?
import MypicsDB as MPDB
File "/home/eric/.xbmc/addons/plugin.image.mypicsdb/resources/lib/MypicsDB.py", line 51, in ?
from sqlite3 import dbapi2 as sqlite
ImportError: No module named sqlite3
19:09:58 T:140552301729552 M:1088495616 INFO: -->End of Python script error report<--


- solexalex - 2010-09-12 13:08

It can t load sqlite library wich should be taken from native xbmc dharma.
Dont really know why it does not load in you case.
I will have a look later when i will be back home, but you may try to know if it is the real xbmc dharma beta 1 or another home made compilation in which sqlite would not be compiled...

- tslayer - 2010-09-12 15:48

It's just from Dharma branch itself. Maybe something isn't being built compared to ppa or perhaps ppa includes something not in the Dharma branch itself?

Good question.

- tslayer - 2010-09-12 16:37

Looks like this thread: http://forum.xbmc.org/showthread.php?tid=77708

- tslayer - 2010-09-12 16:45

I guess I should have read the README.linux Smile

.3 $ make -C lib/addons/script.module.pil

.4 $ make -C lib/addons/script.module.pysqlite

- solexalex - 2010-09-12 19:29

Are you telling me it is working now ? Does any xbmc for linux need something special ?

- tslayer - 2010-09-12 20:07

It works now.. nothing else needed except proper reading of readme.linux Smile

Though, a lot of my pictures didn't scan in... I'll have to check log later. Lots of the folders (within the plugin) show no pics.

- tslayer - 2010-09-12 20:08

Okay, just looked quickly at the logs:

Filled with:

10:50:51 T:140232956843792 M:1131466752 NOTICE: MyPicsDB >> /home/eric/mounts/Volume_1/Pictures/0 RamirezGross06/2010-06-01
10:50:51 T:140232956843792 M:1131466752 NOTICE: MyPicsDB >> Ce dossier ne contient pas d'images :
10:50:51 T:140232956843792 M:1131466752 NOTICE: MyPicsDB >> /home/eric/mounts/Volume_1/Pictures/0 RamirezGross06/2010-06-01

But I clearly have files in there:

eric@htpc:~/mounts/Volume_1/Pictures/0 RamirezGross06/2010-06-01$ ls
IMG_6140.JPG IMG_6141.JPG IMG_6142.JPG IMG_6143.JPG IMG_6144.JPG IMG_6145.JPG IMG_6146.JPG

- solexalex - 2010-09-12 21:02

Okay i know. The problem is known.
It is due to the fact that my lib only scan for 'jpg' down case not JPG upper case. This is not a problem for windows users but uggly for linux users.
This is already repaired and i just need to send the version to the repository. I will not at home to do that before tomorrow.
This new version will make possible the scan of any extension you like. Anyway, only jpg will got iptc and exif infos if available.
Wait til tomorrow and tell me what do you think

- tslayer - 2010-09-12 23:06

Thanks Smile

feature request - mhdebokx - 2010-09-13 13:45

feature request to implement more EXIF.IPTCdata keywords, like
- Category
- SupplementalCategories
- CountryCode
- Country
- City

thanks in advance

- solexalex - 2010-09-13 18:49

hi mhdebokx
All of these are already scanned for and included in the pictures database. This is the job of my library mypicsdb.py

Unfortunately I don't really know how to include all of these within a plugin. That's why it is a library and a plugin using my library. I hope other devs could take advantage of my lib and create some scripts with full interface.

I'm thinking right now about your request, maybe can you be more explicit about what you want ? Maybe I can create some other folders like now we have 'by date, by fodler, by keywords, I may add 'by categories', 'by countrycode', ....
(You may be the only one interrested by this but I'm ready to code it for you (maybe can I think about a way to create any filters like the user want.
I.E you want a 'sort by City' folder.
The plugin show you all available fields (ie exif info and.or iptc info) and then create all the subfolders for you (like keywords filter)
City /
..../ Paris
..../ Cape Tribulation
..../ Sydney
......../Darling harbour.jpg
......../Circular Quay.jpg

- Robotica - 2010-09-13 19:20

In the previous topic I reported 2 bugs:
- Strange char in the foldername caused "ERROR: PICTURE: Error loading image F:\Foto's\100ÊSIO\CIMG0276.JPG"
- Time/date values from scan with TWAIN drivers causes problems:
03:23:34 M: 28311552 WARNING: -->Python Initialized<--
03:23:35 M: 25894912 NOTICE: using pysqlite2
03:23:36 M: 23371776 NOTICE: Q:\plugins\pictures\MyPictures (beta1)\MyPictures.db
03:23:36 M: 23330816 NOTICE: Scan folder :
03:23:36 M: 23347200 NOTICE: ["F:\\Foto's\\xx\\xxxxx\\"]
03:23:36 M: 23330816 NOTICE: F:\Foto's\xx\xxx\
03:23:37 M: 23306240 ERROR: Error Type: exceptions.ValueError
03:23:37 M: 23285760 ERROR: Error Contents: time data did not match format: data=2010.03.24 13.48.00 fmt=%Y:%m:%d %H:%M:%S
03:23:37 M: 23285760 ERROR: Traceback (most recent call last):
File "Q:\plugins\pictures\MyPictures (beta1)\default.py", line 427, in ?
ok = scan_my_pics()#scan lorsque le plugin n'a pas de paramètres
File "Q:\plugins\pictures\MyPictures (beta1)\default.py", line 404, in scan_my_pics
MPDB.browse_folder(chemin,parentfolderID=None,recu rsive=xbmcplugin.getSetting('recursive')=="true",u pdate=False,updatefunc = pupdate)
File "Q:\plugins\pictures\MyPictures (beta1)\resources\lib\MypicsDB.py", line 426, in browse_folder
exif = get_exif(os.path.join(dirname,picfile).encode('utf 8'))
File "Q:\plugins\pictures\MyPictures (beta1)\resources\lib\MypicsDB.py", line 528, in get_exif
tagvalue = time.strftime("%Y-%m-%d %H:%M:%S",time.strptime(tags[tag].__str__(),"%Y:%m:%d %H:%M:%S"))
File "Q:\system\python\python24.zlib\_strptime.py", line 292, in strptime
ValueError: time data did not match format: data=2010.03.24 13.48.00 fmt=%Y:%m:%d %H:%M:%S

Picture rating from GUI (probably needs XMP to store)