Kinect Home Automation goes HTML5

It's taken me longer than I expected, but I finally have a new video ready, complete with music this time.  This video covers my latest work with the Kinect: an HTML5(ish) web interface and low-level protocol for configuring and using the Kinect with a home automation system.  I also demonstrate my own home automation software.  I'm sure you'll all enjoy this video.

Watch in fullscreen to see the screen details, and be sure to continue reading below the video for more information.

video summary

The video is split into two parts, preceded by a deliberately-cheesy title animation.

Part 1

I explain the web interface I've created.  The interface allows the creation of trigger zones (not unlike trigger boxes in a video game), which can later be assigned to home automation logic.

Next I discuss the JSON interface and low-level TCP/IP protocol used to communicate between the Kinect server and a home automation system.

Part 2

In Part 2 I present my home automation software.  It uses simple building blocks, connected with virtual wires on a graphical canvas, to perform automation logic.  The concept is loosely based on Blender's composite nodes, Pure Data, etc.

technical details

For those who like to know how everything works:

  • The Kinect back-end server is written in C using libevent and libfreenect.
  • The web interface server is in Ruby using eventmachine plus a couple of custom extensions in C.
  • The web interface client, obviously, is written in HTML, CSS, and JavaScript, using jQuery UI.
  • The automation software backend is lots of very-well-documented C code.
  • The automation software frontend is C++/Qt4 (Linux only), with plans to reimplement as a web interface.

Source code will not be released, though individual components and other open source contributions have been and will continue to be made.


I prefer comments right here on the blog, but if you must send a private inquiry, you can use the contact form at