XBMC MAL Updater Plugin
#1
With XBMC knowing what we have watched and playback progress and being able to create queue lists ect

is their any way to create a plugin that will have XBMC update myanimelist when we have completed watching a file / import a show into a plan to watch que

MAL has an API for being able to change / read our profiles and search db for show details ect

I am just unsure why no ones even bothered trying this yet ?

http://myanimelist.net/modules.php?go=api#

http://mal-api.com/docs/
Reply
#2
Queue* sorry, that was annoying me (I really try not to be a spelling troll, honest! I can barely spell myself! It's just one of those little things, you know...)

More importantly!

I ran into this want myself recently, as I have always trouble remembering to update my account. So, I took a crack at it. "plugin" is a little bit kind, to say the least, right now, it's actually independent of XBMC other than calling the json rpc api, but I do plan to make it a plugin, eventually. If you want to help test it and play with it, here's the github: https://github.com/kuruoujou/XBMal

I've only tested it in ubuntu 11.10 with Eden, so your results may (and probably will, particularly with windows and mac) vary. I'll be happy to entertain feedback, though.
Reply
#3
Just used the branch script

FreeBSD# cd /files/Downloads/kuruoujou-XBMal-f7da154/

FreeBSD# python xbmal.py -c config
Traceback (most recent call last):
File "xbmal.py", line 219, in <module>
generateConfig(sys.argv[2])
File "xbmal.py", line 138, in generateConfig
tvshows = xbmc.VideoLibrary.GetTVShows()[u'tvshows']
File "/usr/local/lib/python2.7/site-packages/jsonrpclib/jsonrpc.py", line 276, in __call__
return self.__send(self.__name, kwargs)
File "/usr/local/lib/python2.7/site-packages/jsonrpclib/jsonrpc.py", line 224, in _request
response = self._run_request(request)
File "/usr/local/lib/python2.7/site-packages/jsonrpclib/jsonrpc.py", line 242, in _run_request
verbose=self.__verbose
File "/usr/local/lib/python2.7/xmlrpclib.py", line 1264, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/local/lib/python2.7/xmlrpclib.py", line 1292, in single_request
self.send_content(h, request_body)
File "/usr/local/lib/python2.7/site-packages/jsonrpclib/jsonrpc.py", line 122, in send_content
connection.endheaders()
File "/usr/local/lib/python2.7/httplib.py", line 954, in endheaders
self._send_output(message_body)
File "/usr/local/lib/python2.7/httplib.py", line 814, in _send_output
self.send(msg)
File "/usr/local/lib/python2.7/httplib.py", line 776, in send
self.connect()
File "/usr/local/lib/python2.7/httplib.py", line 757, in connect
self.timeout, self.source_address)
File "/usr/local/lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error: [Errno 61] Connection refused
FreeBSD# python xbmal.py -c config
Traceback (most recent call last):
File "xbmal.py", line 219, in <module>
generateConfig(sys.argv[2])
File "xbmal.py", line 138, in generateConfig
tvshows = xbmc.VideoLibrary.GetTVShows()[u'tvshows']
TypeError: 'NoneType' object has no attribute '__getitem__'
Reply
#4
ok I used a fresh xbmc

looks like it was not liking my mysql data base

added my fail safe shiki and

FreeBSD# python xbmal.py -c config
Which show is Shiki Season 1?
0) Ashiaraiyashiki no Juunintachi
1) Genji Monogatari
2) Prince of Tennis: Eikokushiki Teikyuu Shiro Kessen!
3) Steel Angel Kurumi 2
4) Gakuen: Chijoku No Zushiki
5) Gosick Recap
6) Level-C
7) Detective Conan: Black History
8) Shiki
9) Utsukushiki Emono-tachi no Gakuen
10) Detective Conan: Black History 2
11) Immorality
12) Kouchuu Ouja Mushiking: Mori no Tami no Densetsu
13) Shiki Specials
14) Kusatta Kyoushi no Houteishiki
15) Genji Tsuushin Agedama
16) Hinomaru Hatanosuke: Bakemonoyashiki no Maki
17) Kouchuu Ouja Mushiking Super Battle Movie: Yami no Kaizou Kouchuu
18) Queen's Blade: Utsukushiki Toushitachi
19) Renketsu Houshiki
20) None of the above
21) Manual Search
Your Selection >8


FreeBSD# python xbmal.py
Shiki season 1 updated.



woohoo it worked !


Great Job



Reply
#5
Great! Glad to hear it. I'm a little interested in why it disliked your mysql database, but I'm not sure why that would have been.

Good luck on your car!
Reply
#6
it might have been my extra profiles
verified it working with fresh external mysql database

any way to use an exact naming match

https://github.com/Manromen/script.Trakt...i/Download

might have some useful bits of code as well

if only they had an option to export / import to/from MAL Tongue
Reply
#7
That could be useful, though I don't know how well trakt matches the tvdb. The biggest issue with MAL is that it's not very structured. If it had a consistent naming scheme with second seasons, third seasons, etc. or some way to filter out those seasons, /and/ all of the specials were combined (or, alternatively, the tvdb separates the specials appropriately), then it wouldn't be too bad at all. Getting the two to work together isn't an easy thing to do.

I'll certainly take a look at it though, once I get a v0.2 out, which should be today or tomorrow (and should be an actual XBMC plugin!)

Quick edit: v0.2 is out, and is now an /actual/ XBMC plugin! Whoo! Lemme know if you have any problems.
Reply
#8
A new post for an important update?

Maybe I should make a new thread in addon development or something. Ah, well...

I've released a new version of xbmal, along with a repository to go with it. The new version uses an xml file instead of a flat file to store configuration, so you'll need to run the setup script again. However, the xml file allows for faster future additions.

For the time being, I can't update the XBMC wiki due to some weird akismet error that's happening left and right. So, I can't add the addon or the new repository to it, which means the repository auto-installer can't pick up my repository (how sad *sniff*), at least until it's fixed. However, if you want to add the repository, it's available on my github: https://github.com/kuruoujou/kuroshi-xbmc-repo, and a direct link to the zip file: https://github.com/kuruoujou/kuroshi-xbm...-1.0.0.zip

Once you add the repository, you can install xbmal from the "get addons" screen, along with any other addons I may eventually make. You can also install the add-on directly using it's zip file, which is available from the same github.

Enjoy ^_^

Update: Thanks to Ned Scott, I can now edit the wiki! As such, the Repository Installer addon can now install my repository! Enjoy!
Reply
#9
sadly not much in the logs to help solve this but I can not get into MAL with this new setup username / pass is correct and also manually added it into the settings.xml file default="" and it still did not work

14:36:24 T:34891237376 NOTICE: -->Python Interpreter Initialized<--
14:36:24 T:34891237376 DEBUG: Process - The source file to load is /root/.xbmc/addons/script.artwork.downloader/service.py
14:36:24 T:34891238400 NOTICE: -->Python Interpreter Initialized<--
14:36:24 T:34891238400 DEBUG: Process - The source file to load is /root/.xbmc/addons/script.xbmal/default.py
14:36:24 T:34891237376 DEBUG: Process - Setting the Python path to /root/.xbmc/addons/script.artwork.downloader:/usr/local/share/xbmc/addons/script.module.simplejson/lib:/usr/local/share/xbmc/addons/script.module.pil/lib:/root/.xbmc/addons/script.common.plugin.cache/lib:/root/.xbmc/addons/script.module.elementtree/lib:/usr/local/share/xbmc/addons/script.module.pysqlite/lib:/usr/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg:/usr/local/lib/python2.7/site-packages/Beaker-1.6.3-py2.7.egg:/usr/local/lib/python2.7/site-packages/MarkupSafe-0.15-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/Babel-0.9.6-py2.7.egg:/usr/local/lib/python2.7/site-packages/Mako-0.7.0-py2.7.egg:/usr/local/lib/python2.7/site-packages/zope.interface-3.8.0-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/deluge-1.3.5-py2.7.egg:/usr/local/lib/python2.7/site-packages/cssutils-0.9.8-py2.7.egg:/usr/local/lib/python2.7/site-packages/python_dateutil-1.5-py2.7.egg:/usr/local/lib/python2.7/site-packages/lxml-2.3.4-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/mechanize-0.2.5-py2.7.egg:/usr/local/lib/python27.zip:/usr/local/lib/python2.7:/usr/local/lib/python2.7/plat-freebsd9:/usr/local/lib/python2.7/lib-tk:/usr/local/lib/python2.7/lib-old:/usr/local/lib/python2.7/lib-dynload:/usr/local/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages/PIL:/usr/local/lib/python2.7/site-packages/gst-0.10:/usr/local/lib/python2.7/site-packages/gtk-2.0:
14:36:24 T:34891237376 DEBUG: Process - Entering source directory /root/.xbmc/addons/script.artwork.downloader
14:36:24 T:34891237376 DEBUG: Instantiating addon using automatically obtained id of "script.artwork.downloader" dependent on version 2.0 of the xbmc.python api
14:36:24 T:34891238400 DEBUG: Process - Setting the Python path to /root/.xbmc/addons/script.xbmal:/usr/local/share/xbmc/addons/script.module.simplejson/lib:/usr/local/share/xbmc/addons/script.module.pil/lib:/root/.xbmc/addons/script.common.plugin.cache/lib:/root/.xbmc/addons/script.module.elementtree/lib:/usr/local/share/xbmc/addons/script.module.pysqlite/lib:/usr/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg:/usr/local/lib/python2.7/site-packages/Beaker-1.6.3-py2.7.egg:/usr/local/lib/python2.7/site-packages/MarkupSafe-0.15-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/Babel-0.9.6-py2.7.egg:/usr/local/lib/python2.7/site-packages/Mako-0.7.0-py2.7.egg:/usr/local/lib/python2.7/site-packages/zope.interface-3.8.0-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/deluge-1.3.5-py2.7.egg:/usr/local/lib/python2.7/site-packages/cssutils-0.9.8-py2.7.egg:/usr/local/lib/python2.7/site-packages/python_dateutil-1.5-py2.7.egg:/usr/local/lib/python2.7/site-packages/lxml-2.3.4-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/mechanize-0.2.5-py2.7.egg:/usr/local/lib/python27.zip:/usr/local/lib/python2.7:/usr/local/lib/python2.7/plat-freebsd9:/usr/local/lib/python2.7/lib-tk:/usr/local/lib/python2.7/lib-old:/usr/local/lib/python2.7/lib-dynload:/usr/local/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages/PIL:/usr/local/lib/python2.7/site-packages/gst-0.10:/usr/local/lib/python2.7/site-packages/gtk-2.0:
14:36:24 T:34891238400 DEBUG: Process - Entering source directory /root/.xbmc/addons/script.xbmal
14:36:24 T:34891244544 NOTICE: -->Python Interpreter Initialized<--
14:36:24 T:34891244544 DEBUG: Process - The source file to load is /root/.xbmc/addons/script.common.plugin.cache/default.py
14:36:24 T:34891238400 DEBUG: Instantiating addon using automatically obtained id of "script.xbmal" dependent on version 2.0 of the xbmc.python api
14:36:24 T:34891244544 DEBUG: Process - Setting the Python path to /root/.xbmc/addons/script.common.plugin.cache:/usr/local/share/xbmc/addons/script.module.simplejson/lib:/usr/local/share/xbmc/addons/script.module.pil/lib:/root/.xbmc/addons/script.common.plugin.cache/lib:/root/.xbmc/addons/script.module.elementtree/lib:/usr/local/share/xbmc/addons/script.module.pysqlite/lib:/usr/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg:/usr/local/lib/python2.7/site-packages/Beaker-1.6.3-py2.7.egg:/usr/local/lib/python2.7/site-packages/MarkupSafe-0.15-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/Babel-0.9.6-py2.7.egg:/usr/local/lib/python2.7/site-packages/Mako-0.7.0-py2.7.egg:/usr/local/lib/python2.7/site-packages/zope.interface-3.8.0-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/deluge-1.3.5-py2.7.egg:/usr/local/lib/python2.7/site-packages/cssutils-0.9.8-py2.7.egg:/usr/local/lib/python2.7/site-packages/python_dateutil-1.5-py2.7.egg:/usr/local/lib/python2.7/site-packages/lxml-2.3.4-py2.7-freebsd-9.0-RELEASE-amd64.egg:/usr/local/lib/python2.7/site-packages/mechanize-0.2.5-py2.7.egg:/usr/local/lib/python27.zip:/usr/local/lib/python2.7:/usr/local/lib/python2.7/plat-freebsd9:/usr/local/lib/python2.7/lib-tk:/usr/local/lib/python2.7/lib-old:/usr/local/lib/python2.7/lib-dynload:/usr/local/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages/PIL:/usr/local/lib/python2.7/site-packages/gst-0.10:/usr/local/lib/python2.7/site-packages/gtk-2.0:
14:36:24 T:34891244544 DEBUG: Process - Entering source directory /root/.xbmc/addons/script.common.plugin.cache
14:36:24 T:34891244544 DEBUG: Instantiating addon using automatically obtained id of "script.common.plugin.cache" dependent on version 2.0 of the xbmc.python api
14:36:24 T:34891237376 NOTICE: Artwork Downloader: ## Service - Run at startup: False
14:36:24 T:34891237376 NOTICE: Artwork Downloader: ## Service - Delayed startup: -- minutes
14:36:24 T:34891237376 NOTICE: Artwork Downloader: ## Service - Run as service: False
14:36:24 T:34891237376 NOTICE: Artwork Downloader: ## Service - Time: 00:00
14:36:24 T:34891237376 INFO: Scriptresult: Success
14:36:24 T:34891237376 INFO: Python script stopped
14:36:24 T:34891237376 DEBUG: Thread XBPyThread 34891237376 terminating
14:36:24 T:34891244544 NOTICE: StorageServer Module loaded RUN
14:36:24 T:34891244544 NOTICE: StorageClient-0.9.2 Starting server
14:36:24 T:34891244544 NOTICE: StorageServer-0.9.2 Storage Server starting /root/.xbmc/userdata/Database/commoncache.db
14:36:25 T:34891238400 FATAL: ### [XBMAL] - MAL Username or Password Invalid.
Reply
#10
Hmmm...Well, let's see what we can do here to fix this.

Since you say you edited the default attribute in the settings file, I assume you mean the one in [addons directory]/script.xbmal/resources/settings.xml. The first thing you should do is double check your username and password in [userdata]/addon_data/script.xbmal/settings.xml. If those are wrong, then you've found the problem.

If those are correct, try replacing [addons directory]/script.xbmal/resources/lib/request.py with this one: https://github.com/kuruoujou/XBMal/raw/m...request.py and restarting XBMC. Assuming MAL still doesn't log in (it will sometimes refuse to log in if you try to do so in rapid succession, within a minute or so of a previous login), post your xbmc log to http://xbmclogs.com/ and put the link here.

Hopefully I can get it running for you soon!
Reply
#11
enabled

import xbmc
xbmc.log("### [%s] - %s" % ("XBMAL",str(response.status) + ": " + str(response_content)), level=xbmc.LOGERROR)

only showed this extra part in log
ERROR: ### [XBMAL] - 401: Authorization Required


So I am going to try waiting a while with the rapid succession in mind ( with manually added in usr / pass )

as I noticed MAL tries to log in when XBMC first starts up ( at the time it will be blank password )
also as soon as MAL kicks in PC starts grinding away and I have to close XBMC so I can try and do things

http://xbmclogs.com/show.php?id=3121


and yes its running as root cause it the only clean profile / sql data base running

*edit*

still no luck for me

I did not notice it bringing up auth failed on XBMC startup with manually added usr / pass ( neither saw it in the log )

I then waited 10 mins before clicking on setup .. auth still failed

http://xbmclogs.com/show.php?id=3122

skipping on the setup process with manually added usr/pass

the MAL updater seems to try updating but assuming no season episode support
( just caved in and left absolute numbering due to regex issues and now just renaming all my anime files via symlinking )
can not see any thing about failed auth in logs .. but i am yet to be able to get setup working with out auth failing

http://xbmclogs.com/show.php?id=3124


after adding some anime into XBMC and adding a few into myanimelist for testing the script finally allowed me to run setup

http://xbmclogs.com/show.php?id=3125

I also noticed it tried updating even files played from browsed folders ( shame it keeps going 0 updated tho )

also XBMC uses 100% CPU when not playing back a file
Reply
#12
Meh, being root shouldn't matter, other than it being a mildly dangerous decision to run anything as root (which I'm sure you already know).

It's designed to attempt a login when it first starts up, and if it fails to keep trying until it succeeds, once every 5 minutes, a maximum of 6 times. If that works, then great! it'll go about it's merry way. If it doesn't, then meh, it'll stop trying and quit.

I did notice, though, that it's much more complex than it seems to get the add-on running as it is now, so I need to re-think a few things. Most notably, the setup script. Here's the step-by-step guide, which I'll throw on the wiki page eventually:

Step 1) Download and install the add-on.
Step 2) Configure your username and password. Don't run setup just yet.
Step 3) Restart XBMC.
Step 4) Now go in and run setup.
Step 5) It should work now.
(Let's see if strikeout code works! See edit below...)

Ideally, what I should do is error out /and stop trying/ if no username and password is found on startup, or if it's invalid, requiring the user to go into the configuration screen and edit their user and password. Then I need to move the setup script somewhere other than the configuration screen so that the new user and pass are saved by hitting OK before the setup script runs. Then, after the setup script runs, it needs to check to see if xbmal is already running, and if not, start it. That would probably work best.

Looks like I'm working on 0.4 tonight. Incidently, I just committed version 0.3.3 to the xbmal repository, and will post the zip in the repo in a few minutes. Thanks for helping me out with this.

UPDATE!

Well, version 0.4 is out and in the repository. You might need to force an update for the repository to see it, otherwise it should update on it's own. I would recommend uninstalling and reinstalling, though, as the new version changes things up a little bit. Most notably, the convoluted setup procedure is now fixed, to some extent! To setup xbmal now:
1) Install from the zip or the repository (use the repo if you want automatic updates)
2) Configure your username and password.
3) Run the addon like you would any other - by finding it in the appropriate menu (in Alaska Revisited, it's under Apps). This will run the setup script and generate the XML file.
You're done! Once you exit the setup script, XBMAL will automatically start listening. In addition, it will automatically start listening every time xbmc is started.

If you still notice any issues (particularly with logging in), let me know. Also, I'm not sure why xbmc was running so hard for you with xbmal running. I don't notice anything, but it might just be me.

Finally, it seems I forgot to mention it earlier, but the 401 error code we were receiving was what I was looking for, it indicates a bad username or password from MAL (with a 503 being a rate-limiting issue). However, it may have been trying to submit a blank user and password, which would obviously fail. Hopefully the blank-string checks added in 0.3.3 will help.
Reply
#13
Ah! I didn't notice your updates, so I didn't refer to them in my previous post. I'm posting again just so it pings you and so when you reply it pings me Wink

Anyway, I don't know what could be causing the 100% CPU issue. Have you tried running xbmc without xbmal (but with everything else) to be sure it's the problem? I'm not sure how it would be, as when a video isn't playing, all it does is sit there in a wait loop, doing literally nothing but sleeping, until XBMC fires an 'OnVideoStarted' or 'OnPlaybackStopped' or 'OnPlaybackEnded' event. You might try updating to 0.4 and deleting your /userdata/addon_settings/xbmal.script/settings.xml file and checking CPU usage before and after running the setup script, just to rule it out. I don't want to go on a wild goose chase ^_^

Since the MAL Updater uses calls to the XBMC library, it assumes all of your shows are indexed in that library. It can only update what the xbmc library sees. When you run the setup script, it should report back all of your xbmc shows, and it's best guesses to a mapped MAL show. The guess is almost guaranteed to be wrong for each show (well, 50%, anyway), and that's my next major fix goal. It'll only update shows in your 'watching' and 'plan to watch' list, based on stats that are only available in the xbmc library (that is, I can't read them from the file alone), and right now it only works for the tv library (I imagine the video library will be a future release, way down the line). So basically, unless you populate the xbmc library, this addon is sadly useless, due to (understandable) limitations in the way xbmc stores it's database.

Just to test to be sure I didn't just say complete idiocy to you because I misunderstood, could you run https://github.com/kuruoujou/XBMal/blob/...pisodes.py (outside of XBMC, just a simple python script) which will get a list of all the episodes recognized by the XBMC library in exactly the same format as the script? You can post the output to xbmclogs.com, if you think it's too long. Also, if the result is not empty, could you post your /userdata/addon_data/script.xbmal/config.xml file? DON'T post settings.xml, that contains some sensitive info and I really don't need it, as it's generated by XBMC anyway and there's not much I can do with it.

Thank you, sir!
Reply
#14
I will give it ago in 12 hrs time when I get back from work

but what has got me is I have not changed my password and now its trying to work ...


could you give an example of Anime / file names that are working for you ? s1e22 1x22 do not seem to be working for me .. yet ..
I will keep trying with single season ones until I see this working how ever , just a matter of getting all the correct options ticked I guess

any how I will make another fresh start tonight

thanks for all your work Big Grin

where's your donation link Tongue
Reply
#15
That is odd. Perhaps there was a stray bug in there that got fixed during one of the updates, or maybe when you were messing around with settings? Who knows. If it crops up again, I'll definitely need to look into it.

All of my shows are automatically named by my computer based on some really simple heuristics (I got really, really bored one day, and indexing it seemed tedious). Everything is in one of two /TV Shows/ folders, with each show in a separate sub folder that's named after it's title on the tvdb, and each season (labeled Season 0, Season 1, etc.) in a subfolder beneath that (even if it's only one season). The shows themselves are fairly loosely named, so long as 'S01E22', in your example, show up in them.

Using Lucky Star as an example, Season 1 Episode 22 is in "~/TV Shows/Lucky Star/Season 01/Lucky Star - S01E22.mkv". The way XBMC requires the directory structure to be is very specific, and I've also added some specific regexes in my advancedsettings.xml file, though I don't know how much use they're actually getting: http://xbmclogs.com/show.php?id=3127

Good luck, and as always just post here with your problems (and successes!). I hope we can get you running with this soon!

It's my pleasure! After all, I needed it too!

As for a donation link...I should probably make one, shouldn't I? Hahaha. I'll look into it, maybe I'll have one up somewhere this weekend! Although I'm not expecting anything, of course. ^_^
Reply

Logout Mark Read Team Forum Stats Members Help
XBMC MAL Updater Plugin0