[strateego]

Microsoft unveiled its newest control scheme for the Xbox 360: full-body motion control that doesn't require a controller of any kind called Project Natal.

YouTube Demo Video (watch the media center navigation towards the end of this video):
http://www.youtube.com/watch?v=g_txF7iETX0

http://gizmodo.com/5274319/xbox-360-full...ject-natal

Project Natal is a bar that sits above or below your TV, much like the Wii's sensor bar. But instead of sensing controllers, this thing has a camera, sensors and a microphone inside that lets it get an accurate sense of your space and you in it. This lets you control games just by moving around; kicking, jumping, whatever. But that's really just the start.

What makes this so damned cool is that it lets you do more than just play games with it. You can also move through menus by swiping your hands back and forth. The camera allows for fun features like facial and voice recognition. When you have it running, your Xbox will recognize your face and sign you in automatically

You can start to imagine the possibilities already. Everything from Minority Report style menu control to Dance Dance Revolution games that require you to actually dance rather than just hit buttons to videoconferencing in your living room, the thing just feels like the future. And it already makes the Wii seem like a gimpy and outdated piece of tech. There's no way it can touch what this will do.

http://www.xbox.com/en-US/live/projectnatal/

Compatible with any Xbox 360 system, the "Project Natal" sensor is the world's first to combine an RGB camera, depth sensor, multi-array microphone and custom processor running proprietary software all in one device. Unlike 2-D cameras and controllers, "Project Natal" tracks your full body movement in 3-D, while responding to commands, directions and even a shift of emotion in your voice.

In addition, unlike other devices, the "Project Natal" sensor is not light-dependent. It can recognize you just by looking at your face, and it doesn't just react to key words but understands what you're saying. Call a play in a football game, and players will actually respond.

I know it is way too early for to even tell if this is hackable (or revered-engineered) for Windows, Mac, or Linux.

What if someone comes out with an open source library similar to this project http://johnnylee.net/projects/wii/

How would you incorporate this with XBMC if that was ever made possible?

[natethomas]

I, personally, would use magic.

[althekiller]

webcam + OpenCV + some of natethomas' magic

[rwparris2]

xbmc isn't a game?

Anyways, it sounds like Microsoft wrote that review... I'll remain skeptical until a friend buys one.

[Gamester17]

I believe that strateego is mostly thinking about a kind of Minority Report style input(?)

[EDIT]: Checkout FluidTunes for a such implementation for a media center application:

http://fluidtunes.com

There is a few demo videos which has hands filmed as an overlay to explain how it works:
http://vimeo.com/moogaloop.swf?clip_id=2376525
http://www.youtube.com/watch?v=oCU0axCNEJI
http://www.youtube.com/watch?v=oCU0axCNEJI
http://www.youtube.com/watch?v=ylgnMtub7hU

Search YouTube with keyword "fluidtunes" for even more videos:
http://www.youtube.com/results?search_ty...fluidtunes

FluidTunes



FluidTunes gives you control over your music using only a camera, and your hands, head or feet.

Swipe your hand from left to right or right to left over the screen center to scrub. Wave quickly at buttons to activate them.

Without using a keyboard or mouse, you can navigate through your music and play or pause songs.

Contact: [email protected]

If you like FluidTunes, you might also like iSight Screensavers

Meaning waving your arms in the air as an alternative input to browse and pressing buttons?

So for a media center interface this would be similar to the multi touch input of the iPhone?
...imagine having a enormous 'virtual' iPhone in front of you and controlling it this way

http://www.google.com/search?q=Minority+...+interface

http://images.google.com/images?q=Minori...+interface

http://www.youtube.com/watch?v=7CoJGrtVs4c
http://www.youtube.com/watch?v=dyMVZqJk8s4
http://www.youtube.com/watch?v=3bn-zZX9kdc
http://www.youtube.com/watch?v=vhf8dNnzC9w
http://www.youtube.com/watch?v=ywh9vKjYBus

I imagine that this would be very tiresome input method for your arms in the long run?

Might be fun at a party though, or impress friends by showing of your holiday photos, hehe

[althekiller]

This is pretty far outside the scope of XBMC. Once someone has a stable cross-platform input library that does this reasonably accurately (maybe 5-10yrs from now at the earliest) it *might* be feasible to try to integrate into XBMC. Even at that point it would take HEAVY modification to the skinning engine and specially designed skins. Your best bet would be to lobby groups like SDL to add support first.

[AnalogKid]

I have to say....

the reason I have a remote control is to avoid having to move my ass in the first place...

if I wanted to wave my arms around and jump.... I'd simply insert my Elle McPherson DVD and have my wife do the work for me.


Microsoft's hype on this particular technology has more in common with a Bowel Motion.

[natethomas]

Just to be a little less smart alec, for XBMC for Windows, at least, Project Natal seems likely never to be programmed in, unless EventGhost or something similar is programmed in first.

In fact, if there is going to be any kind of Natal for Windows future, I have to think it'll begin right there, with some kind of driver support for Windows Media Center. Then EventGhost or something similar will work with that driver. And then we XBMC users will simply work with EventGhost.

In order for Natal to be useful to XBMC for Windows in a more exciting way, Windows would need to start accepting mouse-like input in a 3D environment, using either two mice to simulate a 3D plane or something like Natal to do something likewise.

If we actually reach that point in the next 10 years, you can color me impressed.

[Tariella]

Any of you ever looked into opencv?

Here are some examples for it:
a blog entry with video of headtracking that eveolved into a google code project
youtube video with markerless human tacking

Here is a wiki for it.

[AnalogKid]

The technology isn't Microsoft's or Sony's.... it will be some IP owned by some startup, or a chip manufacturer somewhere... adapted / purchased / licensed by them.... regardless, here is what will happen...

The 'Motion Detection' component will become a commodity within a couple of years (just like accelerometers in phones have become in the last couple of years). There will be PC versions hooked up to USB and act like any other HID, so XBMC (if it's still around in a couple of years) will get in on the act too.

However, I will say this much.... we went through this hype with the EyeToy. It worked... but not to reliably enough to do anything critical with.... just a bit of fun..... there are too many subtleties in human motion to be detected by tiny CCD's... even a Wii controller is about as subtle as a brick.

[EMK0]

from what i heard Microsoft bought the company that made the tech for the Natal. Its called 3dv Zcam here is a demo of it http://www.youtube.com/watch?v=kT5QNisqDE0 it looks like it would be fun for gaming but using it as a way to control your media i still think a remote would be much easier and faster then waving your hands around.

[jballard]

Natal is more than just the motion capture though. It also includes voice commands and natural language processing. The end of the trailer demos using it to watch a movie. The gestures were used to navigate the marketplace, swiping your hand left and right to scroll, but playing and stopping were voice commands.

... the “Project Natal” sensor is the world’s first to combine an RGB camera, depth sensor, multi-array microphone and custom processor running proprietary software all in one device. Unlike 2-D cameras and controllers, “Project Natal” tracks your full body movement in 3-D, while responding to commands, directions and even a shift of emotion in your voice.

In addition, unlike other devices, the “Project Natal” sensor is not light-dependent. It can recognize you just by looking at your face, and it doesn’t just react to key words but understands what you’re saying...

http://www.microsoft.com/Presspass/press...1E3PR.mspx

I wouldn't be surprised if we see them do a PC release of this in some form. Many of the 360 peripherals have been PC compatible, the controller and headset come to mind. Also this was developed using much of the same technology as is in the Surface systems, so maybe there will be some tie in with the retail release of those.

[therain93]

http://news.cnet.com/8301-13860_3-102863...BlogArea.0

Microsoft doesn't just want to bring gesture recognition to the Xbox with Project Natal. It also wants the technology in Windows, according to a very good source--Microsoft Chairman Bill Gates.

In an interview with CNET News this week, Gates talked about a world in which depth-sensing cameras such as the one Microsoft is adding to the Xbox allow people to control their PCs, game devices, and televisions. (See a video from the E3 conference below.)

Speaking about all of the technology Microsoft has cooking in its labs, Gates said: "I'd say a cool example of that, that you'll see... in a little over a year, is this (depth) camera thing." Gates said it was not just for games, "but for media consumption as a whole, and even if they connect it up to Windows PCs for interacting in terms of meetings, and collaboration, and communication."

Gates said it is an example where the project started in Microsoft research but is now being commercialized by both the Xbox and Windows units. "Both the Xbox guys and the Windows guys latched onto that and now even since they latched onto it the idea of how it can be used in the office is getting much more concrete, and is pretty exciting."

Using your body to control devices makes a lot of sense, Gates said. "I think the value is as great for if you're in the home, as you want to manage your movies, music, home system type stuff, it's very cool there," he said. "And I think there's incredible value as we use that in the office connected to a Windows PC. So Microsoft research and the product groups have a lot going on there, because you can use the cost reduction that will take place over the years to say, why shouldn't that be in most office environments."