Database issues
#1
I am having an issue with the core DB code while working on the Fanart code. In order to allow selection of Fanart, I have to store several pieces of information on each of the available fanart. I'm currently packing these all into one XML formatted string so they can be saved into the DB in one column.

Unfortunately i'm having an issue where the DB code is truncating my string. If I examine the DB the entire stringn IS stored inthe table, but the code that retrieves it apparently is limiting it. I don't know if this is an intentional thing or some bug or limitation, but I was hoping someone with more familiarity with this code could shed some light on things.

The odd thing is it seems to be truncating at around 1006 characters which seems a very odd number.

Now I can re-write my code to pack things tighter, but that doesn't really solve the issue. If a show has a lot of fanarts then it will just crop up again.
Reply
#2
I would personally seriously consider not storing all that info in the db. It holds little value to us - we have to download the images to allow selection, so what's an extra lookup to get the info. I'm not sure whether the scraper allows this ofcourse, and I'd be tempted to do it for thumbs as well.

As for the truncation, I personally don't believe it - some of the plots returned from IMDb are much longer than 1000 chars. If you want somewhere to look, however, you may try sqlite_dataset.cpp and see whether somewhere there it's using a fixed column width. If it's not there, it may be within the sqlite lib itself, the source of which is in trunk/docs/ IIRC.

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#3
we need the color info somewhere. and i know scott will NOT fancy us hitting the server every time a thumb is selected, atleast he did not back when the interfaces used tons of cpu. not sure if the bw really matters now since the zip is << than the pictures by several magnitudes anyways
Reply
#4
The only way to not store all this info is to rescrape before choosing fanart. I thought about that, and it was suggested in the IRC, but I didin't think the system was really setup to do that. It also doesn't seem, as spiff said, very service friendly. CPU cycles cost more than bandwidth after all.

I know the database isn't truncating it since sqlitespy shows the entire string inside the DB, which was why I suspected some code somewhere. I didin't even consider that plots are workign fine. Good point.
Reply
#5
the system not being setup for it isnt really an argument - we can change anything we want and the scraper system surely has the flexibility needed if we choose to do it that way.
Reply
#6
spiff Wrote:i know scott will NOT fancy us hitting the server every time a thumb is selected

This. Smile Agathorn... store 3 colors instead of 6. I'll be updating the site soon so it's all 3.
Reply
#7
szsori Wrote:This. Smile Agathorn... store 3 colors instead of 6. I'll be updating the site soon so it's all 3.

Sweet!
Reply
#8
agathorn: I'll be around on IRC today, so may have time to have a quick look if you have a diff for me.

Obviously regarding rescraping thumbs/fanart would only occur on manual selection of "Get Thumb" and "Get Fanart", the result of which is stored in the db.

If we do decide to store the XML locally in the db as we do with thumbs, then it may make sense to tag the db columns with an additional bool as to whether we need to retrieve that column or not for listing purposes, in order to reduce the amount of data grabbed from the db on a folder listing. Whether or not it'll make any difference will ofcourse require profiling.

Cheers,
Jonathan
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply

Logout Mark Read Team Forum Stats Members Help
Database issues0