Augmented Reality with Kinect: If you know C/C++ programming, then this book will give you the ability to develop augmented reality applications with Microsoft's Kinect. By the end of the course you will have created a complete game.
By Rui Wang
()
About this ebook
Microsoft Kinect changes the notion of user interface design. It differs from most other user input controllers as it enables users to interact with the program without touching the mouse or a trackpad. It utilizes motion sensing technology and all it needs is a real-time cameras, tracked skeletons, and gestures.
Augmented Reality with Kinect will help you get into the world of Microsoft Kinect programming with the C/C++ language. The book will cover the installation, image streaming, skeleton and face tracking, multi-touch cursors and gesture emulation. Finally, you will end up with a complete Kinect-based game.
Augmented Reality with Kinect will help you get into the world of Kinect programming, with a few interesting recipes and a relatively complete example. The book will introduce the following topics: the installation and initialization of Kinect applications; capturing color and depth images; obtaining skeleton and face tracking data; emulating multi-touch cursors and gestures; and developing a complete game using Kinect features.
The book is divided in such a way so as to ensure that each topic is given the right amount of focus. Beginners will start from the first chapter and build up to developing their own applications.
Related to Augmented Reality with Kinect
Related ebooks
Augmented Reality with Kinect Rating: 0 out of 5 stars0 ratingsKinect in Motion – Audio and Visual Tracking by Example Rating: 0 out of 5 stars0 ratingsUnity 2018 By Example: Learn about game and virtual reality development by creating five engaging projects, 2nd Edition Rating: 0 out of 5 stars0 ratingsCocos2D Game Development Essentials Rating: 0 out of 5 stars0 ratingsSmart Internet of Things Projects Rating: 4 out of 5 stars4/5Hands-On Blockchain for Python Developers: Gain blockchain programming skills to build decentralized applications using Python Rating: 0 out of 5 stars0 ratingsMicroPython Projects: A do-it-yourself guide for embedded developers to build a range of applications using Python Rating: 0 out of 5 stars0 ratingsHands-On Internet of Things with Blynk: Build on the power of Blynk to configure smart devices and build exciting IoT projects Rating: 0 out of 5 stars0 ratingsC++ Application Development with Code::Blocks Rating: 0 out of 5 stars0 ratingsAugmented Reality for Android Application Development Rating: 0 out of 5 stars0 ratingsGetting Started with SpriteKit Rating: 0 out of 5 stars0 ratingsKivy – Interactive Applications and Games in Python - Second Edition Rating: 0 out of 5 stars0 ratingsAugmented Reality for Developers: Build practical augmented reality applications with Unity, ARCore, ARKit, and Vuforia Rating: 0 out of 5 stars0 ratingsHands-On Artificial Intelligence with Unreal Engine: Everything you want to know about Game AI using Blueprints or C++ Rating: 0 out of 5 stars0 ratingsQt5 C++ GUI Programming Cookbook Rating: 0 out of 5 stars0 ratingsUnity Virtual Reality Projects: Learn Virtual Reality by developing more than 10 engaging projects with Unity 2018, 2nd Edition Rating: 0 out of 5 stars0 ratingsBlueprints Visual Scripting for Unreal Engine: The faster way to build games using UE4 Blueprints, 2nd Edition Rating: 0 out of 5 stars0 ratingsRaspberry Pi LED Blueprints Rating: 0 out of 5 stars0 ratingsLearn OpenCV 4 by Building Projects,: Build real-world computer vision and image processing applications with OpenCV and C++ Rating: 0 out of 5 stars0 ratingsHands-On Internet of Things with MQTT: Build connected IoT devices with Arduino and MQ Telemetry Transport (MQTT) Rating: 0 out of 5 stars0 ratingsBuilding an RPG with Unity 2018: Leverage the power of Unity 2018 to build elements of an RPG., 2nd Edition Rating: 0 out of 5 stars0 ratingsArduino Networking Rating: 4 out of 5 stars4/5C++ Game Development By Example: Learn to build games and graphics with SFML, OpenGL, and Vulkan using C++ programming Rating: 0 out of 5 stars0 ratings
Information Technology For You
Summary of Super-Intelligence From Nick Bostrom Rating: 4 out of 5 stars4/5Creating Online Courses with ChatGPT | A Step-by-Step Guide with Prompt Templates Rating: 4 out of 5 stars4/5Supercommunicator: Explaining the Complicated So Anyone Can Understand Rating: 3 out of 5 stars3/5Data Analytics for Beginners: Introduction to Data Analytics Rating: 4 out of 5 stars4/5Self-Sovereign Identity Rating: 0 out of 5 stars0 ratingsCompTIA Network+ CertMike: Prepare. Practice. Pass the Test! Get Certified!: Exam N10-008 Rating: 0 out of 5 stars0 ratingsCompTia Security 701: Fundamentals of Security Rating: 0 out of 5 stars0 ratingsLearning Microsoft Endpoint Manager: Unified Endpoint Management with Intune and the Enterprise Mobility + Security Suite Rating: 0 out of 5 stars0 ratingsCyber Security Consultants Playbook Rating: 0 out of 5 stars0 ratingsCompTIA A+ CertMike: Prepare. Practice. Pass the Test! Get Certified!: Core 1 Exam 220-1101 Rating: 0 out of 5 stars0 ratingsHow to Write Effective Emails at Work Rating: 4 out of 5 stars4/5An Ultimate Guide to Kali Linux for Beginners Rating: 3 out of 5 stars3/5A Mind at Play: How Claude Shannon Invented the Information Age Rating: 4 out of 5 stars4/5COMPUTER SCIENCE FOR ROOKIES Rating: 0 out of 5 stars0 ratingsMachine Learning Interview Questions Rating: 5 out of 5 stars5/5Google SketchUp for Game Design: Beginner's Guide Rating: 5 out of 5 stars5/5Cybersecurity for Beginners : Learn the Fundamentals of Cybersecurity in an Easy, Step-by-Step Guide: 1 Rating: 0 out of 5 stars0 ratingsGetting started with Audacity 1.3 Rating: 5 out of 5 stars5/5CompTIA ITF+ CertMike: Prepare. Practice. Pass the Test! Get Certified!: Exam FC0-U61 Rating: 5 out of 5 stars5/5Linux Command Line and Shell Scripting Bible Rating: 3 out of 5 stars3/5WordPress Plugin Development: Beginner's Guide Rating: 0 out of 5 stars0 ratingsGet Programming with Haskell Rating: 0 out of 5 stars0 ratingsInkscape Beginner’s Guide Rating: 5 out of 5 stars5/5Unity Game Development Essentials Rating: 5 out of 5 stars5/5Understanding TCP/IP Rating: 4 out of 5 stars4/5CODING INTERVIEW: Advanced Methods to Learn and Excel in Coding Interview Rating: 0 out of 5 stars0 ratings
Reviews for Augmented Reality with Kinect
0 ratings0 reviews
Book preview
Augmented Reality with Kinect - Rui Wang
Table of Contents
Augmented Reality with Kinect
Credits
About the Author
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers and more
Why Subscribe?
Free Access for Packt account holders
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
1. Getting Started with Kinect
Installation of Kinect
Setting up your Kinect software on PCs
Why did I do that?
The idea of the AR-based Fruit Ninja game
Summary
2. Creating Your First Program
Preparing the development environment
Building the Visual Studio project
Starting the device
Initializing and using Kinect in C++
Understanding the code
Additional information
Summary
3. Rendering the Player
Choosing image stream types
Obtaining color and depth images
Drawing color and depth as textures
Understanding the code
An incorrect way to combine depth and color
A traditional way for background subtraction
Understanding the code
Aligning color with depth
Generating a color image from depth
Understanding the code
Additional information
Using a green screen with Kinect
Making a magic photographer
Understanding the code
Additional information
Summary
4. Skeletal Motion and Face Tracking
Understanding skeletal mapping
Obtaining joint positions
Drawing the skeleton
Understanding the code
Drawing the linetrails following the hands
Drawing the path for specified joints
Understanding the code
Face tracking in Kinect
Detecting a face from the camera
Detecting and drawing the face rectangle
Understanding the code
Constructing the face model
Drawing the parametric face model
Understanding the code
Summary
5. Designing a Touchable User Interface
Multitouch systems
Locating the cursors
Drawing cursors from two hands
Understanding the code
Additional information
Common touching gestures
Recognizing holding and swiping gestures
Drawing cursors using two hands
Understanding the code
Additional information
Sending cursors to external applications
Emulating Windows mouse with cursors
Understanding the code
Summary
6. Implementing the Scene and Gameplay
Integrating the current code
Integrating existing elements in a scene
Understanding the code
Cutting the fruits
Adding and handling fruit objects
Understanding the code
Playing the game
Adding simple game logic
Understanding the code
Additional information
Summary
A. Where to Go from Here
libfreenect – the pioneer of Kinect middleware
OpenNI – a complete and powerful Kinect middleware
Free and open source resources
Commercial products using Kinect
Index
Augmented Reality with Kinect
Augmented Reality with Kinect
Copyright © 2013 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: July 2013
Production Reference: 1040713
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-84969-438-4
www.packtpub.com
Cover Image by Suresh Mogre (<[email protected]>)
Credits
Author
Rui Wang
Reviewers
Ricard Borràs Navarra
Vangos Pterneas
Acquisition Editor
Kartikey Pandey
Commissioning Editors
Llewellyn Rozario
Priyanka Shah
Technical Editors
Sumedh Patil
Aniruddha Vanage
Copy Editors
Insiya Morbiwala
Alfida Paiva
Laxmi Subramanian
Project Coordinator
Amigya Khurana
Proofreader
Maria Gould
Indexer
Rekha Nair
Graphics
Abhinash Sahu
Production Coordinator
Conidon Miranda
Cover Work
Conidon Miranda
About the Author
Rui Wang is a Software Engineer at Beijing Crystal Digital Technology Co., Ltd. (Crystal CG), in charge of the new media interactive application design and development. He wrote a Chinese book called OpenSceneGraph Design and Implementation in 2009. He also wrote the book OpenSceneGraph 3.0 Beginner's Guide in 2010 and OpenSceneGraph 3.0 Cookbook in 2012, both of which are published by Packt Publishing and co-authored by Xuelei Qian. In his spare time he also writes novels and is a guitar lover.
I must express my deep gratitude to the entire Packt Publishing team for their great work in producing a series of high-quality Mini books, including this one, which introduces the cutting-edge Kinect-based development. Many thanks to Zhao Yang and my other colleagues at Crystal CG, for their talented ideas on some of the recipes in this book. And last but not the least, I'll extend my heartfelt gratitude to my family for their love and support.
About the Reviewers
Ricard Borràs Navarra has always been working around computer vision and machine learning. He started creating machines that apply pattern recognition to quality assortment in the cork production industry. Later, he applied these techniques for human tracking in complex scenarios, creating audience measurement, and people-counter solutions for retail stores.
With the eruption of Kinect, he started working on and deploying augmented reality interactive applications based on this great device. These applications were targeted at the marketing and retail sectors.
All these projects were developed by him as an Inspecta (www.inspecta.es) employee. Also, Ricard has developed several freelance projects based on augmented reality.
Vangos Pterneas is a professional Software Engineer, passionate about natural user interfaces. He has been a Kinect enthusiast ever since the release of the very first unofficial SDKs and has already published a couple of commercial Kinect applications.
He has worked for Microsoft Innovation Center as a .NET developer and consultant and he's now running his own company named LightBuzz. LightBuzz has been awarded the first place in Microsoft's worldwide innovation competition, held in New York.
When he is not coding, he loves blogging about technical stuff and providing the community with open source utilities.
www.PacktPub.com
Support files, eBooks, discount offers and more
You might want to visit www.PacktPub.com for support files and downloads related to your book.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
Support files, eBooks, discount offers and morehttp://PacktLib.PacktPub.com
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books.
Why Subscribe?
Fully searchable across every book published by Packt
Copy and paste, print and bookmark content
On demand and accessible via web browser
Free Access for Packt account holders
If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access.
Preface
Microsoft Kinect was released in the winter of 2010. As one of the first civil handsfree motion input devices, it brings a lot of fun to end users of Xbox 360 and Windows PCs. And because Kinect is very useful for designing interactive methods in user applications, new media artists and VJs (video jockeys) are also interested in this new technology as it makes their performances more dramatic and mystical.
In this book, we will focus on introducing how to develop C/C++ applications with the Microsoft Kinect SDK, as well as the FreeGLUT library for OpenGL support, and the FreeImage library for image loading. We will cover the topics of Kinect initialization, color and depth image streaming, and skeleton motion and face tracking, and discuss how to implement common gestures with Kinect inputs. A simple but interesting Fruit Ninja-like game will be implemented in the last chapter of this book. Some alternative middlewares and