November 4, 2016 2:41 pm

Kinect demo code and new driver for UWP now available

By

Here’s a little memory test: Do you recall this blog, which posted back in May and promised to soon begin integrating Kinect for Windows into the Universal Windows Platform? Of course you do! Now we are pleased to announce two important developments in the quest to make Kinect functionality available to UWP apps.

First, by popular demand, the code that Alex Turner used during his Channel 9 video (above) is now available on GitHub as part of the Windows universal samples. With this sample, you can use Windows.Media.Capture.Frames APIs to enumerate the Kinect sensor’s RGB/IR/depth cameras and then use MediaFrameReader to stream frames. This API lets you access pixels of each individual frame directly in a highly efficient way.

These new functionalities debuted in the Windows 10 Anniversary Update, and structure of the APIs should be familiar to those who’ve been using the Kinect SDK for years. But these new APIs are designed to work not only with the Kinect sensor but with any other sensors capable of delivering rich data streams—provided you have a matching device driver.

Which brings us to our second announcement: We have now enabled the Kinect driver on Windows Update. So if you’d like try out this new functionality now, simply go to the Device Manager and update the driver for the Kinect sensor. In addition to enabling the new UWP APIs described above, the new driver also lets you use the Kinect color camera as a normal webcam. This means that apps which use a webcam, such as Skype, can now employ the Kinect sensor as their source. It also means that you can use the Kinect sensor to enable Windows Hello for authentication via facial recognition.

picture1

Another GitHub sample demonstrates how to use new special-correlation APIs, such as CameraIntrinsics or DepthCorrelatedCoordinateMapper, to process RGB and depth camera frames for background removal. These APIs take advantage of the fact that the Kinect sensor’s color and depth cameras are spatially correlated by calibration and depth frame data. This sample also shows how to access the Kinect sensor’s skeletal tracking data through a custom media stream in UWP apps with newly introduced APIs.

Finally, we should note that the Xbox summary update also enables these Kinect features through Windows.Media.Capture.Frames for UWP apps. Thus, apps that use the Kinect sensor’s RGB, infrared, and/or depth cameras will run on Xbox with same code, and Xbox can also use the Kinect RGB camera as a normal webcam for Skype-like scenarios

Judging from requests, we’re confident that many of you are eager to explore both the demo code and download the new driver. When you do, we want to hear about your experiences—what you liked, what you didn’t, and what enhancements you want to see. So send us your feedback!

Please note that, if you have technical questions about this post or would like to discuss Kinect with other developers and Microsoft engineers, we hope you will join the conversation on the Kinect for Windows v2 SDK forum. You can browse existing topics or ask a new question by clicking the Ask a question button on the forum webpage.

The Kinect for Windows Team

Key links

Join the conversation

  1. One feedback i can give you … provide c# sample for skeleton tracking (already available in cpp) thank you fabio

  2. Really great news. But what about the hardware ? Kinect is getting old now, it’s not fast enough, nor accurate enough, and how does it handle the PC situation? (ie 2 feet from the screen). I think Kinect has a long way to go, and a really useful and widely used Kinect will look a lot different to what it is today. But unless we see iterative development and expansion of the systems capabilities, well, Kinect isn’t going to be a part of anybodies player-aware setup.

  3. What one or two files can I download to use Kinect as webcam for my skype? I just cant find right answers anywhere. Note that I am not a developer and I have Kinect for windows and would like to connect my PC with Kinect and use skype Kinect camera.

  4. What about a good book/tutorial explaining things and teaching the sdk for Kinect v2? There are few (if none) good tutorial about how to get-start with Kinect V2. Personally I can’t find a tutorial explaining how to do things… All I can see is a course on MVA about usnig Kinect with Scratch, but…. REALLY?! Scratch????!!!!

  5. The new UWP API can be used to access Body frames on a PC. However, when the app is deployed on an XBOX console, Body frames are not available. The XBOX API only has Color, Depth, and Infrared frames. No Custom frames, unlike the PC.

    The problem has been reported in the official Kinect forums:
    https://social.msdn.microsoft.com/Forums/silverlight/en-US/3eb710d0-8801-4a5c-965a-51e2e7577e9d/using-uwp-mediacapture-cannot-access-custom-framesources-on-xbox?forum=kinectv2sdk&prof=required

    It is also mentioned on GitHub:
    https://github.com/Microsoft/Windows-universal-samples/issues/568

    There is no reply from Microsoft so far.

    Is Microsoft planning to support Custom/Body frames on the XBOX?