[bmillham]

I've been trying to get Persistence working with a MySQL connection. Sadly, it doesn't look like Persistence is up to the task yet.

Code:

import ClassHandler

handler = ClassHandler.ClassHandler()
db = MySQLdb.connect(host='tardis', user='root', passwd='', db='mynewmovies')
handler.save(db)

Results in

Code:

22:10:06 T:140196460296256 M:846225408   DEBUG: ------ Window Init (DialogKaiToast.xml) ------
22:10:06 T:140196460296256 M:846225408   DEBUG: Alloc resources: 0.01ms (0.00 ms skin load)
22:10:07 T:140196033116432 M:846225408    INFO: Python script stopped
22:10:07 T:140196033116432 M:846225408   DEBUG: Thread 140196033116432 terminating
22:10:07 T:140196149074192 M:846225408  NOTICE: Traceback (most recent call last):
22:10:07 T:140196149074192 M:846225408  NOTICE:   File "/home/brian/.xbmc/scripts/Persistence/Persistence.py", line 65, in ?
22:10:07 T:140196149074192 M:846225408  NOTICE: server = ClassServer()
22:10:07 T:140196149074192 M:846225408  NOTICE:   File "/home/brian/.xbmc/scripts/Persistence/Persistence.py", line 12, in __init__
22:10:07 T:140196149074192 M:846225408  NOTICE: self.connect()
22:10:07 T:140196149074192 M:846225408  NOTICE:   File "/home/brian/.xbmc/scripts/Persistence/Persistence.py", line 27, in connect
22:10:07 T:140196460296256 M:846225408   DEBUG: python thread 8 destructed
22:10:07 T:140196149074192 M:846225408  NOTICE: self.processCommand(clientsock, clientaddr)
22:10:07 T:140196149074192 M:846225408  NOTICE:   File "/home/brian/.xbmc/scripts/Persistence/Persistence.py", line 43, in processCommand
22:10:07 T:140196149074192 M:846225408  NOTICE: classInstance = comm.recv(clientsock)
22:10:07 T:140196149074192 M:846225408  NOTICE:   File "/home/brian/.xbmc/scripts/Persistence/comm.py", line 24, in recv
22:10:07 T:140196149074192 M:846225408  NOTICE: datasize = unpack("<i", sock.recv(4))[0]
22:10:07 T:140196149074192 M:846225408  NOTICE: struct
22:10:07 T:140196149074192 M:846225408  NOTICE: .
22:10:07 T:140196149074192 M:846225408  NOTICE: error
22:10:07 T:140196149074192 M:846225408  NOTICE: :
22:10:07 T:140196149074192 M:846225408  NOTICE: unpack str size does not match format
22:10:07 T:140196149074192 M:846225408   ERROR: Scriptresult: Error
22:10:07 T:140196460296256 M:846225408   DEBUG:  WaitOnScriptResult - plugin exited prematurely - terminating

I've tried various different ways like:

Code:

classInstance = handler.exists(MySQLdb.connect) or handler.new(MySQLdb.connect, ('tardis', '3306', 'root', '', 'mynewmovies'))

with similar results. (one strange difference is the error would vary from expecting a str when using 3306 as the port to expecting an int when using '3306'

Oh well, looked like a nice idea, but I'm going to continue on working on my script instead of a plugin.

(BTW: I doubt that it matters, but I'm running 9.11alpha1. I did try the example supplied with Persistence, and it works fine)

Brian

I've looked at this some more, and it looks like the problem is with cPickle.

I tried a script from command line:

Code:

>>> c = MySQLdb.connect(host='tardis', port=3306, user='root', passwd='', db='mynewmovies')
>>> p = cPickle.dumps(c)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/copy_reg.py", line 71, in _reduce_ex
    state = base(self)
TypeError: connect() argument 1 must be string, not Connection
>>>

From the error, it looks like cPickle is attempting to make the MySQL connection instead of pickling it.

Can someone give me a pointer about how to solve this, or pointers to where in the Python world where I may get some help?

Thanks again!
Brian

[rwparris2]

I've looked at PyDbLite, and it's real strange. Looks like it takes the database and creates it's own copy of it. That (to me at least) defeats the purpose of a database... It also seems that startup of a plugin using this would be REAL slow if you have a good size database.

I haven't used the mysql or sqllite bindings, but it won't create a copy of its native "pdl".
You would execute pydblite outside of your plugin and it just would just keep running, keeping the db in memory. It really isn't for handling large amounts of data.

If you have a db big enough to make pydblite REAL slow, you need to look at using a real solution like sqllite.

Also, don't forget you can create an index if you're constantly refering to the same dataset to help speed things up

• to speed up selections, an index can be created on a field : db.create_index('age')
  When an index is created, the database instance has an attribute (here _age : note the heading underscore, to avoid name conflicts with internal names). This attribute is a dictionary-like object, where keys are the values taken by the field, and values are the records whose field values are egal to the key :
  records = db._age[23] returns the list of records with age == 23
  If no record has this value, lookup by this value returns an empty list
  The index supports iteration on the field values, and the keys() method returns all existing values for the field
  

[bmillham]

Thanks rwparris2, you just gave me some inspiration.

PDL won't really do what I want. The MySQL database that I read my movie info from can change at any time, and it doesn't look like PDL will see changes to the database. From looking at the code, it makes a snapshot of the database at startup.

The other problem for me is I have some VERY complex SQL queries that I don't see how PDL can handle.

The idea that you just gave me is to do something similar to what Persistence does, but instead of using cPickle, to have the sql logic in the script, and the plugin will just make canned requests to the script.

I think that will do what I'm looking for.

BTW, is there a way to make sure that a script starts when XBMC is started, and to close gracefully when XBMC exits?

Thanks much again!
Brian