Hi, it’s been a while. So I actually uploaded this next component from Nitrogen Logic, knd, quite a while ago (October 2018). But things got busy and I never wrote the blog post. So now that we’re all on quarantine lockdowns and have extra time, here’s a very late announcement that knd is now open source!

knd provides the core Kinect backend interface for all Nitrogen Logic controllers. It’s a background daemon written in C with a text interface over TCP. knd provides image, position, and zone data to the rest of the Nitrogen Logic system, and performs the heavy lifting of all 3D distance and location calculations.

A few highlights:

• Optimized approximated integer math functions using reciprocal multiplication to get 30fps on SheevaPlugs, which lack floating point hardware.
• Human- and machine-friendly TCP text-based command line interface.
• Cross-compilation support through CMake.
• Automatic stack trace printout of all threads when there is an error (not always easy in C, but made possible by nlutils and glibc).

Preparing for release

Even back in 2018 when I was getting this code ready to release, I hadn’t touched the project in a few years. Here’s what I had to do to tidy things up and get everything working on then-current systems (it’s possible further distro drift and bit rot has occurred since):

• Fix library search to find libfreenect on newer Ubuntu.
• Reorganize the source tree to match other Nitrogen Logic C projects.
• Convert from a hand-rolled Makefile to CMake.
• Fix cross-compilation with newer versions of CMake.
• Fix warnings reported by newer versions of GCC and Clang.
• Update for compatibility with the open source release of nlutils.
• Update for compatibility with newer versions of libevent.
• Add AGPLv3 license text.

Building and running

This is just a quick glance at the app. Full documentation is available in the knd project README.

You’ll need Linux (this was tested on Ubuntu 18.04 LTS), but it’s pretty straightforward:

1
2
3
4

sudo apt-get install libfreenect-dev libusb-1.0.0-dev libevent-dev
make
mkdir -p ~/.knd
KND_SAVEDIR=~/.knd ./build-$(uname -m)/src/knd

Using the TCP/IP command line

The command line interface is sort of based on my old cliserver example for libevent. A constant connection has much lower overhead than something like HTTP.

To test the command line yourself, you can connect with netcat to port 14308. Any number of clients can connect at the same time. An event-driven reactor loop handles all client connections.

1

nc localhost 14308

There’s a help command that describes all other commands:

Commands always respond with a status line, which usually contains the type and size of the response. Text commands like zones will indicate the number of lines to read, binary commands like getvideo will indicate the number of bytes to read.

1
2
3
4

zones
OK - 2 zones - Version 3, 0 occupied, peak zone is -1 "[none]"
xmin=1015 ymin=-487 zmin=5327 xmax=1478 ymax=204 zmax=5765 px_xmin=154 px_ymin=217 px_zmin=1027 px_xmax=215 px_ymax=295 px_zmax=1032 negate=0 param=pop on_level=300 off_level=140 on_delay=2 off_delay=30 occupied=0 pop=0 maxpop=4758 xc=-1 yc=-1 zc=-1 sa=0 name="Entry"
xmin=1427 ymin=-774 zmin=3948 xmax=2338 ymax=227 zmax=4480 px_xmin=0 px_ymin=206 px_zmin=1003 px_xmax=129 px_ymax=357 px_zmax=1014 negate=0 param=pop on_level=300 off_level=140 on_delay=1 off_delay=1 occupied=0 pop=0 maxpop=19479 xc=-1 yc=-1 zc=-1 sa=0 name="Kitchen"

1
2
3
4

getvideo
OK - Requested delivery of a video frame
VIDEO - 307200 bytes of video data follow newline
...

1
2
3
4

getdepth
OK - Requested a single depth frame for delivery as a DEPTH message
DEPTH - 422400 bytes of raw data follow newline
...

Some commands, like sub and subdepth, send messages periodically. Clients are able to distinguish between subscription messages and responses to commands by the first word (OK, SUB, DEPTH, etc.).

1
2
3
4
5
6

sub
OK - Subscribed to global zone updates
SUB - xmin=1015 ymin=-487 zmin=5327 xmax=1478 ymax=204 zmax=5765 px_xmin=154 px_ymin=217 px_zmin=1027 px_xmax=215 px_ymax=295 px_zmax=1032 negate=0 param=pop on_level=300 off_level=140 on_delay=2 off_delay=30 occupied=0 pop=0 maxpop=4758 xc=-1 yc=-1 zc=-1 sa=0 name="Entry"
SUB - xmin=1427 ymin=-774 zmin=3948 xmax=2338 ymax=227 zmax=4480 px_xmin=0 px_ymin=206 px_zmin=1003 px_xmax=129 px_ymax=357 px_zmax=1014 negate=0 param=pop on_level=300 off_level=140 on_delay=1 off_delay=1 occupied=0 pop=0 maxpop=19479 xc=-1 yc=-1 zc=-1 sa=0 name="Kitchen"
unsub
OK - Unsubscribed from global zone updates

Enjoy

So that’s all for now. If time permits I’ll eventually go into more detail about how knd maintains high framerates without an FPU, or how data is passed between the processing threads and the event loop, or how error recovery and timeouts work. There’s also lots of room for improvement.

But until then feel free to explore the knd source!