KinEmote - Kinect Driven Media Control (OpenNI based open source project for Windows) - Printable Version
+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for XBMC (/forumdisplay.php?fid=116)
+--- Thread: KinEmote - Kinect Driven Media Control (OpenNI based open source project for Windows) (/showthread.php?tid=87663)
KinEmote - Kinect Driven Media Control (OpenNI based open source project for Windows) - Johnsel - 2010-12-15 13:05
KinEmote - Kinect Driven Media Control
KinEmote is an easy-to-use application that allows Windows users to navigate the menus of Boxee and XBMC media portals using nothing but hand gestures that are captured by MicroSoft's Kinect camera. It's built around the OpenNI framework and NITE Middleware provided by PrimeSense Inc.
KinEmote Public Beta is available for Windows 32 and 64 bit systems and can be downloaded for free at http://code.google.com/p/kinemote/
Please note if you're upgrading from Kinect Remote for XBMC alpha you'll have to install the new version of the OpenNI framework
Will be released after some much needed cleaning.
Latest public beta in action
First demo http://www.youtube.com/watch?v=BKyFL3CgIXc
Second demo http://www.youtube.com/watch?v=1h5OP-FKhOo
Third demo showing v0.02 ALPHA in action http://www.youtube.com/watch?v=AV3jffH7jPg
- alex84 - 2010-12-15 14:09
Damn that looks sleek mate, great work
Can i buy an kenetic kit and get this working on my asrock 330ht ?
Thums upp for your great effort. I'm hoping to see an linux version in the future
- mrioa - 2010-12-15 14:26
Awesome! That's exactly what I was thinking of when I first read that the official drivers have been released
I don't have a Kinect yet but I will follow this project for sure and maybe going to get myself one later on.
- Johnsel - 2010-12-15 15:09
Yes i will eventually rewrite it to something usable under linux as well, but for now i use C# since i have most experience with that. That way i can rapidly prototype different techniques for controlling xbmc. The swipe gestures look neat, but are absolutely unusable in a library with more than 10 movies. I'm currently looking into a virtual slider solution that enables you to scroll at different speeds, but maybe a push and swipe technique would work even better. Also, context sensitivity has to be integrated, when playing a movie you want different actions than in your movie library.
Ideas on navigation techniques are very welcome, as very people have created something like this before. Xbox kinect dash & the opensense media center demo are my only references right now.
alex84 Wrote:Can i buy an kenetic kit and get this working on my asrock 330ht ?
I'd have to test the performance, but as long as it runs windows it _should_ work.
- Gotcha007 - 2010-12-15 16:24
Impressive man, can't wait to have it released
- rausch101 - 2010-12-15 16:47
Johnsel Wrote:Ideas on navigation techniques are very welcome, as very people have created something like this before.
I don't know how possible it is, but the obvious one would be trying to emulate the touch-swipe motion of smartphones. You'd probably need position, velocity and acceleration (with the latter two being derivatives of position of course). I think this would involve the entire hand and maybe some use of the forearm/arm.
Practically, I think it would get tiring waving your arm around that much though. Assuming the user is close enough to the Kinect so that it can distinguish individual fingers, an entirely different route could be this for basic navigation (which I think I like more):
This method can be combined with whole hand movements or fist movements for a wider range of inputs, but I think it's good in general to stick to the least effort possible for the most common inputs.
- Johnsel - 2010-12-15 17:04
Seperate finger tracking is another interesting technique, will look into that too. Only problem with the vanilla kinect driver is the low resolution (640x480), which obviously complicates tracking very slight changes in hand position. I did see an example that increases the resolution to 1280x1024 at the cost of framerate, which goes down to 10fps iirc. Don't know whether that would be enough.
Anyways, thanks for providing input.
- kizer - 2010-12-15 17:27
That is so Geeky that I love it. Good job man.
Microsoft seems to be embracing the fact that they can actually make money off their stuff when they let others hack on it.
- i*windows - 2010-12-15 22:27
very nice, would it be able to have a look at the code you have created - pls?
- rflores2323 - 2010-12-15 23:33
this is awesome. linux would be great also.
- tungmeister - 2010-12-15 23:52
I was thinking about how to go about implementing scrolling last night. I came up with as follows:
To initiate scrolling the user would put their hand out and show their palm to kinect.
This would start the scrolling functionally by setting the hands current position as a start point.
The user would then move their hand up,down,left,right depending on the required direction still pointing their palm at kinect.
The further the user moves their hand from the start point the faster scrolling would go probably using an exponential scale so that it would be easy to move one or two items at a time without overshooting.
The user would then close their hand to stop scrolling.
haven't really considered how to implement other features but I'll have a think
- Johnsel - 2010-12-16 00:48
That was actually one of the first things i tried. Scrolling that is, not the palm gesture. I've thought about it though, but in a grab and "continuous swipe" style. I guess i'll wait with releasing a binary until i've create a demo app that lets you choose between a couple of different control techniques.
For now, another video. Using my hand as a virtual cursor this time
Find it on Youtube
In this video I control XBMC using my hand as a cursor. Push and pull are respectively click and go back. It seems that using the xy coordinates with a subtle smoothing is very accurate, but the push used to click will almost certain move the cursor too much. I think the best solution for this is speed (and maybe z axis) dependent sensitivity.
- oOWallaceOo - 2010-12-16 05:42
I am not entirely sure what information you are able to get or are using from the kinect but after watching your video these are my control scheme suggestions:
My idea for a control scheme would be to have your palm facing the screen and rotating it counterclockwise to scroll left and clockwise to scroll right with the degree of rotation determining how quickly it scrolls. With this added to the single swipe gestures from your first video for more precise navigation I think it would be very easy to quickly get to an area of your Video Library with my gesture then select the exact movie with yours.
Ideas for additional gestures:
- htpc guy - 2010-12-16 06:15
Congrats Johnsel, you made it on Engadget!
- natethomas - 2010-12-16 06:50
Johnsel Wrote:In this video I control XBMC using my hand as a cursor. Push and pull are respectively click and go back. It seems that using the xy coordinates with a subtle smoothing is very accurate, but the push used to click will almost certain move the cursor too much. I think the best solution for this is speed (and maybe z axis) dependent sensitivity.
I remember reading an article about the Dance Revolution game and how they spent months trying to get something that worked AND wasn't insanely hard to accurately select something with. I'm guessing the mouse is just never going to be the best kind of kinect controller. But I'd love to be proven wrong.