Once again it’s time for another episode of my video series, Code, Sound & Surround! This series is all about using 2D and 3D visualizations, code, and sound to explore the relationships between math, sound, visuals, and problem solving from a unique angle. My goal is not necessarily to be a one-stop source to learn these concepts, but rather to both provide an overview of things you might not have heard of, and to present familiar concepts in an unfamiliar way, to help unlock new insights and reveal the connections between ideas.

This time we’re reviewing how sound is digitized, quickly recapping filters, and looking at the Fast Fourier Transform (FFT) with some never-before-seen 3D visualizations. The video also briefly introduces analytic signals and sets the stage for future videos that will dive further into analytic signals, synthesizers, and surround sound. And don’t miss the synthesizer demo song at the end, showing just how much can be accomplished with a few stages of simple signal processing.

As part of working on this video, I’ve made hundreds of commits and added thousands of lines of code to mb-math, mb-util, and mb-sound, so check those out as well. Also, all music from all of my videos is original, either composed for this video series, or repurposed from my previous work.

Keep reading below for the video’s backstory, a list of the concepts covered in the video, and a transcript of the video.

My latest video is live on YouTube! In this beautifully animated tour of Voronoi diagrams, Delaunay triangulation, and natural neighbor interpolation, I explain the algorithms used in my mb-geometry rubygem for generating Delaunay triangulations and Voronoi partitions. Then interpolation is given as a practical application of these algorithms.

This video is part of my ongoing video series, Code, Sound & Surround.

You can keep reading for a partial transcript of the video, and I’ve written more about Voronoi partitions, Delaunay triangulations, and mb-geometry previously.

I’ve been writing a lot of code for my next video in Code, Sound & Surround. One small part of that has turned out to be interesting enough to show off on its own. I built some tools for plotting and animating Voronoi diagrams in Ruby, and now I’ve published them on GitHub.

Update: I’ve also made a YouTube video showing these geometric algorithms.

A Voronoi diagram (Voronoi rhymes with “enjoy”) or Voronoi partition shows which areas are closest to a set of starting points. Choose one or more points on a 2D plane (you can also do 3D or higher, but I haven’t done that). Draw a convex polygon around each point, showing the area that is closest to that point. That’s a Voronoi diagram.

Each edge of a polygon should be half-way between two points, and each corner of a polygon is equally distant to three or more points. Every polygon has one point inside it, and every point has one polygon around it, with no overlap. Give each surrounding polygon its own color, and you get something like this:

Ruby is one of those programming languages that seems to get slotted into a specific use-case in people’s minds. You use Ruby for web development, C or C++ for high-performance code, Java for banks and insurance companies, Python for machine learning, Rust for trendily secure code, Typescript for web frontends. Everyone just “knows” these things. But why?

For a long time I had wanted to make my own A/V receiver with my own audio algorithms. Most people just buy a TV and call it done. Some people will buy a TV and a sound bar and call it done. A few more will buy a TV, a receiver, and some speakers and call it done. And the super dedicated (or super rich) will buy an expensive hi-fi system and/or home theater. But not me. Instead of just buying a receiver and pushing that “Dolby” or “DTS” button, why not do things the hard way, and make my own?

Put those two things together – my love of Ruby, and my love of sound – and mix in a bit of 2020 lockdown. I just had to prove that Ruby is good for more than web development. I ended up writing my own stereo-to-surround-sound decoder (more than one, in fact), and a bunch of other audio code, all in Ruby. Not content with that, I’ve released a lot of my code on GitHub and I’m making videos to teach others about sound, and about Ruby.

This series of videos and posts is all about my journey to find out just what Ruby is capable of, and to make the best-sounding surround decoder ever written in Ruby.

I’ve been working hard on a YouTube project lately, which I’ll blog about later. For now, I’m pleased to announce that I’ve opened up the source code for Nitrogen Logic’s Depth Camera Controller browser-based UI, called knc.

knc is a Ruby- and EventMachine-based service that serves a dynamic browser-based UI (HTML, CSS, JS) for controlling Nitrogen Logic automation systems and Philips Hue lights using the Kinect.

Hi, it’s been a while. So I actually uploaded this next component from Nitrogen Logic, knd, quite a while ago (October 2018). But things got busy and I never wrote the blog post. So now that we’re all on quarantine lockdowns and have extra time, here’s a very late announcement that knd is now open source!

knd provides the core Kinect backend interface for all Nitrogen Logic controllers. It’s a background daemon written in C with a text interface over TCP. knd provides image, position, and zone data to the rest of the Nitrogen Logic system, and performs the heavy lifting of all 3D distance and location calculations.

A few highlights:

• Optimized approximated integer math functions using reciprocal multiplication to get 30fps on SheevaPlugs, which lack floating point hardware.
• Human- and machine-friendly TCP text-based command line interface.
• Cross-compilation support through CMake.
• Automatic stack trace printout of all threads when there is an error (not always easy in C, but made possible by nlutils and glibc).

If you’ve read my previous post, you know that I’m opening up some of the source code from my old startup, Nitrogen Logic. Here’s the first major piece of the Nitrogen Logic source code to go open source. It’s a core utility library used by all of Nitrogen Logic’s C projects, called nlutils.

Because of C’s very limited standard library compared to more modern and full-featured languages, it’s incredibly common for developers to write their own core data structures and helper functions that they use over and over. These include better string manipulation functions, linked lists, associative arrays, etc. – things that we take for granted in languages like Ruby.

nlutils is the set of such tools for Nitrogen Logic’s C projects. Where possible the code adheres to C99 and POSIX.1-2008. The header files are thoroughly documented, so browse around the include/ directory to see what nlutils provides. After meticulously reviewing the code and tests, I’ve pushed it to GitHub under the GNU AGPLv3 license for all to enjoy.

It’s been quite some time since my last post. I’ve been meaning to write more about my old startup, Nitrogen Logic, but as they say life gets busy. But now seems like a good time to start opening up more of the source code that powered Nitrogen Logic. It’ll be slow going, but the first big C release, nlutils, is already on GitHub.

Since Nitrogen Logic was always a one-person show, there’s nobody but me stopping me from releasing, but also nobody pushing me onward. I figured I should finally take advantage of the rare opportunity to add 5+ years of my career to my public portfolio and to the shared knowledge of our industry.

Any time you open source old code there’s a lot of work involved. In Nitrogen Logic’s case there are several major components to open up, each with its own unique needs, and built on a plethora of different technologies:

Lately I’ve been reading about the Rust programming language, just for fun. One of the ways I (and others) enjoy learning a new language or framework is porting an old project into the new language, while trying to take advantage of the language’s features.

So to learn Rust, I’ll be reimplementing my old cliserver example for libevent using Rust’s Tokio crate. At the risk of public embarrassment, I will be tracking my progress using GitHub issues.

The web interfaces for the Nitrogen Logic depth and automation controllers are written in Ruby. To talk to the Philips Hue lighting system and some other DIY home automation systems, I created two Ruby libraries, nlhue and xap_ruby. I recently converted these libraries to Ruby Gems available from RubyGems.org.

Both of them depend on EventMachine for fast network event handling, but that also makes them a bit difficult to use. They were developed in a bit of a hurry, follow C rather than Ruby indentation, lack automated tests, and don’t represent the best code quality. Nevertheless they have been battle tested in real products working well for real people, and they may yet be useful.

Continue reading for more information about these gems.