KinEmote - Kinect Driven Media Control (OpenNI based open source project for Windows)

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
Johnsel Offline
KinEmote Co-Founder
Posts: 77
Joined: Oct 2009
Reputation: 10
Location: Maastricht, Netherlands
Thumbs Up  KinEmote - Kinect Driven Media Control (OpenNI based open source project for Windows)
Post: #1
KinEmote - Kinect Driven Media Control

KinEmote is an easy-to-use application that allows Windows users to navigate the menus of Boxee and XBMC media portals using nothing but hand gestures that are captured by MicroSoft's Kinect camera. It's built around the OpenNI framework and NITE Middleware provided by PrimeSense Inc.

KinEmote Public Beta is available for Windows 32 and 64 bit systems and can be downloaded for free at http://code.google.com/p/kinemote/

Please note if you're upgrading from Kinect Remote for XBMC alpha you'll have to install the new version of the OpenNI framework

Source:
Will be released after some much needed cleaning.

Latest public beta in action
http://www.youtube.com/watch?v=3y0JT0xR_a4

Old Videos:
First demo http://www.youtube.com/watch?v=BKyFL3CgIXc
Second demo http://www.youtube.com/watch?v=1h5OP-FKhOo
Third demo showing v0.02 ALPHA in action http://www.youtube.com/watch?v=AV3jffH7jPg
(This post was last modified: 2010-12-24 05:31 by Johnsel.)
find quote
alex84 Offline
Senior Member
Posts: 180
Joined: Jan 2010
Reputation: 2
Location: Sweden
Post: #2
Damn that looks sleek mate, great work Smile

Can i buy an kenetic kit and get this working on my asrock 330ht ?

Thums upp for your great effort. I'm hoping to see an linux version in the future Wink
Cheers

---------------------------------------------------
Intel NUC Haswell D34010WYK | ATV2 | Logitech Harmony One | Onkyo TX-NR808 Receiver | QNAP 809 | APC Back-UPS RS 550
find quote
mrioa Offline
Junior Member
Posts: 3
Joined: Jul 2010
Reputation: 0
Post: #3
Awesome! That's exactly what I was thinking of when I first read that the official drivers have been released Smile

I don't have a Kinect yet but I will follow this project for sure and maybe going to get myself one later on.

Thumbs upp!
find quote
Johnsel Offline
KinEmote Co-Founder
Posts: 77
Joined: Oct 2009
Reputation: 10
Location: Maastricht, Netherlands
Post: #4
Yes i will eventually rewrite it to something usable under linux as well, but for now i use C# since i have most experience with that. That way i can rapidly prototype different techniques for controlling xbmc. The swipe gestures look neat, but are absolutely unusable in a library with more than 10 movies. I'm currently looking into a virtual slider solution that enables you to scroll at different speeds, but maybe a push and swipe technique would work even better. Also, context sensitivity has to be integrated, when playing a movie you want different actions than in your movie library.

Ideas on navigation techniques are very welcome, as very people have created something like this before. Xbox kinect dash & the opensense media center demo are my only references right now.


alex84 Wrote:Can i buy an kenetic kit and get this working on my asrock 330ht ?

I'd have to test the performance, but as long as it runs windows it _should_ work.
find quote
Gotcha007 Offline
Senior Member
Posts: 125
Joined: Dec 2010
Reputation: 0
Post: #5
Impressive man, can't wait to have it released Cool
find quote
rausch101 Offline
Fan
Posts: 582
Joined: Sep 2006
Reputation: 1
Location: Washington D.C.
Post: #6
Johnsel Wrote:Ideas on navigation techniques are very welcome, as very people have created something like this before.

I don't know how possible it is, but the obvious one would be trying to emulate the touch-swipe motion of smartphones. You'd probably need position, velocity and acceleration (with the latter two being derivatives of position of course). I think this would involve the entire hand and maybe some use of the forearm/arm.


Practically, I think it would get tiring waving your arm around that much though. Assuming the user is close enough to the Kinect so that it can distinguish individual fingers, an entirely different route could be this for basic navigation (which I think I like more):

  1. Have the nominal state of scrolling (e.g. no movement) occur when the user has a clenched fist (thumb pointing upwards). I think this makes sense because it's relaxed and natural
  2. Point the index finger out to the Kinect and have this be one keystroke of 'down arrow'
  3. Hold this for, say, .5 seconds to start inducing an accelerated scroll (easily done in EventGhost, not sure about XBMC, but probably that as well)
  4. Curl the finger back in to stop the scroll. Repeat as needed.
  5. Point your thumb upwards for one keystroke of 'up arrow'
  6. Hold this for, say, .5 seconds to start inducing an accelerated scroll
  7. Curl the thumb back down to stop the scroll. Repeat as needed.


This method can be combined with whole hand movements or fist movements for a wider range of inputs, but I think it's good in general to stick to the least effort possible for the most common inputs.

Have a question? First try the XBMC online-manual and FAQ. Also: How to submit a debug log
find quote
Johnsel Offline
KinEmote Co-Founder
Posts: 77
Joined: Oct 2009
Reputation: 10
Location: Maastricht, Netherlands
Post: #7
Seperate finger tracking is another interesting technique, will look into that too. Only problem with the vanilla kinect driver is the low resolution (640x480), which obviously complicates tracking very slight changes in hand position. I did see an example that increases the resolution to 1280x1024 at the cost of framerate, which goes down to 10fps iirc. Don't know whether that would be enough.

Anyways, thanks for providing input.
find quote
kizer Offline
Fan
Posts: 701
Joined: Jul 2008
Reputation: 0
Location: Seattle WA, US
Post: #8
That is so Geeky that I love it. Good job man.

Microsoft seems to be embracing the fact that they can actually make money off their stuff when they let others hack on it. Laugh
find quote
i*windows Offline
Junior Member
Posts: 28
Joined: Oct 2009
Reputation: 0
Post: #9
very nice, would it be able to have a look at the code you have created - pls?
find quote
rflores2323 Online
Posting Freak
Posts: 1,918
Joined: Jan 2009
Reputation: 2
Post: #10
this is awesome. linux would be great also.

Good luck!

ATV1 with crystalbuntu (12.2 frodo) as main theater. Theater remote URC MX-810, droidx with xbmc remote, or transformer tf101 with YATSE. 3 x ATV2 (V12.2 frodo).

find quote
tungmeister Offline
Fan
Posts: 466
Joined: Jan 2009
Reputation: 8
Location: Sheffield, UK
Post: #11
I was thinking about how to go about implementing scrolling last night. I came up with as follows:

To initiate scrolling the user would put their hand out and show their palm to kinect.
This would start the scrolling functionally by setting the hands current position as a start point.
The user would then move their hand up,down,left,right depending on the required direction still pointing their palm at kinect.
The further the user moves their hand from the start point the faster scrolling would go probably using an exponential scale so that it would be easy to move one or two items at a time without overshooting.
The user would then close their hand to stop scrolling.

haven't really considered how to implement other features but I'll have a think Tongue
find quote
Johnsel Offline
KinEmote Co-Founder
Posts: 77
Joined: Oct 2009
Reputation: 10
Location: Maastricht, Netherlands
Post: #12
That was actually one of the first things i tried. Scrolling that is, not the palm gesture. I've thought about it though, but in a grab and "continuous swipe" style. I guess i'll wait with releasing a binary until i've create a demo app that lets you choose between a couple of different control techniques.

For now, another video. Using my hand as a virtual cursor this time Smile

Find it on Youtube

In this video I control XBMC using my hand as a cursor. Push and pull are respectively click and go back. It seems that using the xy coordinates with a subtle smoothing is very accurate, but the push used to click will almost certain move the cursor too much. I think the best solution for this is speed (and maybe z axis) dependent sensitivity.
find quote
oOWallaceOo Offline
Junior Member
Posts: 3
Joined: Dec 2010
Reputation: 0
Post: #13
I am not entirely sure what information you are able to get or are using from the kinect but after watching your video these are my control scheme suggestions:

My idea for a control scheme would be to have your palm facing the screen and rotating it counterclockwise to scroll left and clockwise to scroll right with the degree of rotation determining how quickly it scrolls. With this added to the single swipe gestures from your first video for more precise navigation I think it would be very easy to quickly get to an area of your Video Library with my gesture then select the exact movie with yours.

Ideas for additional gestures:
  • Putting hands together starts the movie
  • Showing the palm of left hand brings up Movie Info
  • Right and Left hands up to ears turns on subtitles
find quote
htpc guy Offline
Posting Freak
Posts: 1,182
Joined: Oct 2010
Reputation: 7
Location: Near St. Louis Mo
Post: #14
Congrats Johnsel, you made it on Engadget!
http://www.engadget.com/2010/12/15/kinec...l-xbmc-th/

How to use Git
| AMD Athlon II X3 Triple Core Processor 2.9 GHz |GIGABYTE GA-MA785GM-US2H Mobo 2GB DDR2 Ram | MSI N430GT |
| Logitec Harmony One Remote| 52" Sharp Aquos LED TV | Demon AVR-X1000 |
find quote
natethomas Offline
Team Kodi Community Manager
Posts: 3,729
Joined: Apr 2008
Reputation: 63
Location: Kansas
Post: #15
Johnsel Wrote:In this video I control XBMC using my hand as a cursor. Push and pull are respectively click and go back. It seems that using the xy coordinates with a subtle smoothing is very accurate, but the push used to click will almost certain move the cursor too much. I think the best solution for this is speed (and maybe z axis) dependent sensitivity.

I remember reading an article about the Dance Revolution game and how they spent months trying to get something that worked AND wasn't insanely hard to accurately select something with. I'm guessing the mouse is just never going to be the best kind of kinect controller. But I'd love to be proven wrong.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.

[Image: badge.gif]
find quote
Post Reply