• 1(current)
  • 2
  • 3
  • 4
  • 5
  • 12
KinEmote - Kinect Driven Media Control (OpenNI based open source project for Windows)
#1
Thumbs Up 
KinEmote - Kinect Driven Media Control

KinEmote is an easy-to-use application that allows Windows users to navigate the menus of Boxee and XBMC media portals using nothing but hand gestures that are captured by MicroSoft's Kinect camera. It's built around the OpenNI framework and NITE Middleware provided by PrimeSense Inc.

KinEmote Public Beta is available for Windows 32 and 64 bit systems and can be downloaded for free at http://code.google.com/p/kinemote/

Please note if you're upgrading from Kinect Remote for XBMC alpha you'll have to install the new version of the OpenNI framework

Source:
Will be released after some much needed cleaning.

Latest public beta in action
http://www.youtube.com/watch?v=3y0JT0xR_a4

Old Videos:
First demo http://www.youtube.com/watch?v=BKyFL3CgIXc
Second demo http://www.youtube.com/watch?v=1h5OP-FKhOo
Third demo showing v0.02 ALPHA in action http://www.youtube.com/watch?v=AV3jffH7jPg
Reply
#2
Damn that looks sleek mate, great work Smile

Can i buy an kenetic kit and get this working on my asrock 330ht ?

Thums upp for your great effort. I'm hoping to see an linux version in the future Wink
Cheers
---------------------------------------------------
Intel NUC Haswell D34010WYK | ATV2 | Logitech Harmony One | Onkyo TX-NR808 Receiver | QNAP 809 | APC Back-UPS RS 550
Reply
#3
Awesome! That's exactly what I was thinking of when I first read that the official drivers have been released Smile

I don't have a Kinect yet but I will follow this project for sure and maybe going to get myself one later on.

Thumbs upp!
Reply
#4
Yes i will eventually rewrite it to something usable under linux as well, but for now i use C# since i have most experience with that. That way i can rapidly prototype different techniques for controlling xbmc. The swipe gestures look neat, but are absolutely unusable in a library with more than 10 movies. I'm currently looking into a virtual slider solution that enables you to scroll at different speeds, but maybe a push and swipe technique would work even better. Also, context sensitivity has to be integrated, when playing a movie you want different actions than in your movie library.

Ideas on navigation techniques are very welcome, as very people have created something like this before. Xbox kinect dash & the opensense media center demo are my only references right now.


alex84 Wrote:Can i buy an kenetic kit and get this working on my asrock 330ht ?

I'd have to test the performance, but as long as it runs windows it _should_ work.
Reply
#5
Impressive man, can't wait to have it released Cool
Reply
#6
Johnsel Wrote:Ideas on navigation techniques are very welcome, as very people have created something like this before.

I don't know how possible it is, but the obvious one would be trying to emulate the touch-swipe motion of smartphones. You'd probably need position, velocity and acceleration (with the latter two being derivatives of position of course). I think this would involve the entire hand and maybe some use of the forearm/arm.


Practically, I think it would get tiring waving your arm around that much though. Assuming the user is close enough to the Kinect so that it can distinguish individual fingers, an entirely different route could be this for basic navigation (which I think I like more):

  1. Have the nominal state of scrolling (e.g. no movement) occur when the user has a clenched fist (thumb pointing upwards). I think this makes sense because it's relaxed and natural

  2. Point the index finger out to the Kinect and have this be one keystroke of 'down arrow'

  3. Hold this for, say, .5 seconds to start inducing an accelerated scroll (easily done in EventGhost, not sure about XBMC, but probably that as well)

  4. Curl the finger back in to stop the scroll. Repeat as needed.

  5. Point your thumb upwards for one keystroke of 'up arrow'

  6. Hold this for, say, .5 seconds to start inducing an accelerated scroll

  7. Curl the thumb back down to stop the scroll. Repeat as needed.


This method can be combined with whole hand movements or fist movements for a wider range of inputs, but I think it's good in general to stick to the least effort possible for the most common inputs.
Have a question? First try the XBMC online-manual and FAQ. Also: How to submit a debug log
Reply
#7
Seperate finger tracking is another interesting technique, will look into that too. Only problem with the vanilla kinect driver is the low resolution (640x480), which obviously complicates tracking very slight changes in hand position. I did see an example that increases the resolution to 1280x1024 at the cost of framerate, which goes down to 10fps iirc. Don't know whether that would be enough.

Anyways, thanks for providing input.
Reply
#8
That is so Geeky that I love it. Good job man.

Microsoft seems to be embracing the fact that they can actually make money off their stuff when they let others hack on it. Laugh
Reply
#9
very nice, would it be able to have a look at the code you have created - pls?
Reply
#10
this is awesome. linux would be great also.

Good luck!
Nvidia Shield with Kodi 18
Reply
#11
I was thinking about how to go about implementing scrolling last night. I came up with as follows:

To initiate scrolling the user would put their hand out and show their palm to kinect.
This would start the scrolling functionally by setting the hands current position as a start point.
The user would then move their hand up,down,left,right depending on the required direction still pointing their palm at kinect.
The further the user moves their hand from the start point the faster scrolling would go probably using an exponential scale so that it would be easy to move one or two items at a time without overshooting.
The user would then close their hand to stop scrolling.

haven't really considered how to implement other features but I'll have a think :p
Reply
#12
That was actually one of the first things i tried. Scrolling that is, not the palm gesture. I've thought about it though, but in a grab and "continuous swipe" style. I guess i'll wait with releasing a binary until i've create a demo app that lets you choose between a couple of different control techniques.

For now, another video. Using my hand as a virtual cursor this time Smile

Find it on Youtube

In this video I control XBMC using my hand as a cursor. Push and pull are respectively click and go back. It seems that using the xy coordinates with a subtle smoothing is very accurate, but the push used to click will almost certain move the cursor too much. I think the best solution for this is speed (and maybe z axis) dependent sensitivity.
Reply
#13
I am not entirely sure what information you are able to get or are using from the kinect but after watching your video these are my control scheme suggestions:

My idea for a control scheme would be to have your palm facing the screen and rotating it counterclockwise to scroll left and clockwise to scroll right with the degree of rotation determining how quickly it scrolls. With this added to the single swipe gestures from your first video for more precise navigation I think it would be very easy to quickly get to an area of your Video Library with my gesture then select the exact movie with yours.

Ideas for additional gestures:
  • Putting hands together starts the movie
  • Showing the palm of left hand brings up Movie Info
  • Right and Left hands up to ears turns on subtitles
Reply
#14
Congrats Johnsel, you made it on Engadget!
http://www.engadget.com/2010/12/15/kinec...l-xbmc-th/
How to use Git
| AMD Athlon II X3 Triple Core Processor 2.9 GHz |GIGABYTE GA-MA785GM-US2H Mobo 2GB DDR2 Ram | MSI N430GT |
| Logitec Harmony Smart Control Remote| 52" Sharp Aquos LED TV | Denon AVR-X1000 |
| Freenas Server with 18TB ASRock Intel Avoton C2750 |
Reply
#15
Johnsel Wrote:In this video I control XBMC using my hand as a cursor. Push and pull are respectively click and go back. It seems that using the xy coordinates with a subtle smoothing is very accurate, but the push used to click will almost certain move the cursor too much. I think the best solution for this is speed (and maybe z axis) dependent sensitivity.

I remember reading an article about the Dance Revolution game and how they spent months trying to get something that worked AND wasn't insanely hard to accurately select something with. I'm guessing the mouse is just never going to be the best kind of kinect controller. But I'd love to be proven wrong.
Reply
  • 1(current)
  • 2
  • 3
  • 4
  • 5
  • 12

Logout Mark Read Team Forum Stats Members Help
KinEmote - Kinect Driven Media Control (OpenNI based open source project for Windows)1