TIP: Use python webserver in scripts to be able to communicate with them
#1
Hi,

I wanted to share an insight I had with everyone, regarding communication with a running script/plugin.

Problem: You have a script that launches a custom UI and stays running until you specifically exit it. At the same time, you need to run callable instances of the script that do not launch a UI, and only make use of "setResolvedURL" so that the Player can get a time-limited URL from a server. The problem is that the instances of the plugin that run to get the current content URL don't have access to the variables or objects that are in the instance of the script that is currently running the GUI.

I recently realized there is a very easy way to make the functions and variables of the main instance of the script available to other instances of the script. There is an HTTP server built into Python, and by running an instance of this server in the main instance of the script, then you have the ability to get and put data to and from the main script instance from any other instance.

An example: I'm building a script with a full UI for a streaming music service. The main script is launched and the GUI stays running until you quit. The music streams that the service uses are time-limited URLs that also require a temporary login token to be able to fetch.

Using the webserver that runs with the main instance of the script, now the plugin instances that run with sole purpose of getting a streaming URL and sending it back to the player, the script instance can hit a URL that my little http server responds to, such as http://localhost:9001/tra.456789, and the main running instance can do the processing I need to have done, and then return the freshly generated content URL to the plugin instance. This lets the plugin code be very small, and it doesn't have to duplicate a bunch of code (such as the API calls themselves, and all the logic to deal with errors, session management, password management, etc) that is already running in the main instance of the script.

It looks like you can roll your own server pretty easily, or use a mini-server framework python module such as 'bottle' (which is what I'm using) to be able to parse the URLs that are sent to the webserver and then run whatever functions and return whatever data you need.

Here's a god little tutorial on how to roll your own (which is really all you need if you only need to process a couple different URLs):

http://www.acmesystems.it/python_httpserver

And here's the 'bottle' python module, which is contained in one file and can be very easily integrated into your script:
http://bottlepy.org/docs/dev/index.html


Basically what you're doing here is making your script have its own little http-based API that can be used by any other program. The possibilities are really endless. You could implement your own http-based remote control for your script, for example. You could implement JSON-RPC-style APIs for your plugin that follow the style of the XBMC ones. You can use it to make your plugin respond to external home automation events.

Anyway, just wanted to share this concept in case people are looking to solve this particular problem. It was a head scratcher for me until I realized how easy it was to make a http server in python.
Reply
#2
For a little followup to this, I tried using 'bottle', but it introduced a lot of threading issues for me. It generally worked, but it always left a bunch of crap in memory when I quit my script. It was also a real pain to get it to recognize my own app's data structures. It really is more geared for having an entire self contained website run under it, with nothing else above it. I'm sure a more skilled coder than me could have figured it out, but I decided to just write my own little WSGIRef server instance. I'm only using this web interface as a sort of private api offered by my script's main instance to other instances of my xbmc plugin to be able to call. This turned out to be much simpler, and cleaner, and I've got better control over all the threads.

In case anybody is interested, here's what my server class looks like. I borrowed heavily from other WSGIREF examples I found on the web. Everything it uses is in the python standard library. :-)
Code:
#file httpd.py

import threading
import thread
import sys
from cgi import parse_qs
from wsgiref.simple_server import make_server
if sys.version_info >=  (2, 7):
    import json as json
else:
    import simplejson as json


class TinyWebServer(object):
    def __init__(self, app):
        self.app = app

    def simple_app(self, environ, start_response):
        status = '200 OK'
        headers = [('Content-Type', 'application/json')]
        start_response(status, headers)
        if environ['REQUEST_METHOD'] == 'POST':
            request_body_size = int(environ.get('CONTENT_LENGTH', 0))
            request_body = environ['wsgi.input'].read(request_body_size)
            d = parse_qs(request_body)  # turns the qs to a dict
            return 'From POST: %s' % ''.join('%s: %s' % (k, v) for k, v in d.iteritems())
        else:  # GET
            d = parse_qs(environ['QUERY_STRING'])  # turns the qs to a dict
            #return 'From GET: %s' % ''.join('%s: %s' % (k, v) for k, v in d.iteritems())
            try:
                track_id = str(d['track'])
                url = self.app.api.get_playable_url(track_id[2:-2])
                response = json.dumps([{"track": track_id, "url": url}], indent=4, separators=(',', ': '))
                return response
            except:
                return "Hi"


    def create(self, ip_addr, port):
        self.httpd = make_server(ip_addr, port, self.simple_app)

    def start(self):
        """
        start the web server on a new thread
        """
        self._webserver_died = threading.Event()
        self._webserver_thread = threading.Thread(
                target=self._run_webserver_thread)
        self._webserver_thread.start()

    def _run_webserver_thread(self):
        self.httpd.serve_forever()
        self._webserver_died.set()

    def stop(self):
        if not self._webserver_thread:
            return

        thread.start_new_thread(self.httpd.shutdown, () )
        #self.httpd.server_close()

        # wait for thread to die for a bit, then give up raising an exception.
        #if not self._webserver_died.wait(5):
            #raise ValueError("couldn't kill webserver")
        print "Shutting down internal webserver"


I start the server in my main loop like this:
Code:
#file main.py
import httpd

self.srv = httpd.TinyWebServer(self)
self.srv.create("localhost", 8090)
self.srv.start()

And I stop it like this, during a controlled shutdown:
Code:
self.srv.stop()

In this case, I realized, using any kind of framework was just overkill. :-) I'm only using it to respond to one kind of call, "http://localhost:8090/?track=Tra.123456789" and my srv instance parses the track ID, and returns a playable URL for the XBMC media player to begin to play as a streaming music file. I really wanted my main script instance to handle this logic because it already has access to the user's login info, access token, and also knows how to handle things like token expiration and renewal. This way my playlist items don't have to contain any data except for the track ID for the music service.

I hope this helps anybody interested in using this technique.

-Jerimiah
Reply
#3
Why a httpserver and not socket?
Reply
#4
So I can eventually add more functionality to it and control the plugin from any browser on the network. HTTP seemed like the most universal protocol to me. But I am pretty new to many aspects of coding and IPC in general, so I don't actually know if this is a defensible reason yet. :-)
Reply
#5
This seems OK if you only have one add-on using an HTTP server. Can you imagine what a pain it will be when you have 10 add-ons using this strategy? You'll have to have a sheet to keep track of all the different port numbers you've used.
Reply
#6
Port number manager addon!
Reply
#7
Ha! Totally. I will make the port number configurable, at least. :-)
Reply

Logout Mark Read Team Forum Stats Members Help
TIP: Use python webserver in scripts to be able to communicate with them0