Rendering IR Sensor Data in Unity

In the world of robotics, a common goal is for the robot to “know” what is going on around it.  Often, sensors are employed in achieving this task.  Sensors come in all sorts of shapes, sizes and specifications.  Which sensor you choose depends completely on what exactly you want your robot to “know”.

Section I: The Idea

For this particular endeavor, my colleague and I were interested in, and hoped to allow a robot to “know”, the placement of the objects in the space around it.  This is by no means a new problem in robotics.

The above video is an excellent example of a robot that knows just enough about its environment to avoid colliding with it.  But how much does it actually know?  Very little.  This robot is using an ultrasonic sensor to check if there are readings in certain directions and simply avoiding those directions.  We wanted to go for something a bit more informative.

This next video displays the impressive abilities of the Velodyne HDL-32E Laser LiDAR system when mounted on a UAV.  The difference in the amount this robot is able to know about its environment from the amount the previous robot knows is enormous.  So precise and extensive is the data from the sensor in the second video that a surprisingly helpful realtime rendering of the environment is available and 3D models can be made afterwards.  I would love to have one of these, but seeing as how I don’t have a spare $30K lying around (price listed in 2010), we’ll have to make due with what we have and, hopefully, we will be able to land somewhere in the middle.

Section II: Our Plan

The basic design we decided to go with is a system in which distance data of some sort is fed into Unity, along with some way of knowing the position of the datapoint, and subsequently rendered as an object, or piece of an object.  With the high-level attack laid out, we proceeded to the hardware specifics.

The Hardware:

The equipment we used for this project was fairly simple. We had a Sharp IR sensor with a range of 100cm – 550cm. This range was quite long and it would have been easier to do testing with a shorter ranged sensor. We used an Arduino Uno to collect the data from the sensor and transfer it to the computer. We also had two R/C servo motors to rotate the sensor around.

Section III: Bumps in the Road

Hardware Issues:

We hit some bumps almost immediately in getting our hardware up and working. At first, our IR sensor was not outputting stable output, but then it was discovered that our wiring was wrong.  After connecting the inputs and outputs for our sensor correctly we achieved stable output that was detecting objects within the proper range.  In hindsight we wish that we used a shorter range sensor to for data collection. Long range sensors do not have extensive documentation or examples available.

Coding Journey:

Our first thought was that it would be somewhat trivial to read in the serial port, to which the Arduino was writing, from within Unity.  I researched serial port interaction in C#, one of Unity’s coding options, and found pretty dismal news.  I came upon a forum dedicated to how serial port interaction in C# is apparently finicky on Mac.  Considering we were running this project on a Mac, we were forced in another direction.  With a bit more research, I found that serial port communication is quite easy in Python, as long as you have the pyserial (the serial module) installed.  I quickly installed the needed module and whipped out the following code:

Screen Shot 2015-12-11 at 11.19.42 AM

Quite simply, this code reads a line from the serial port, does a tiny bit of input verification and writes that line to a file.  From here, we had only to read in that file from a C# script in Unity and the information was received.  It turned out to be much harder than anticipated to run the Python script on a reliable repetition.  Also, there were some issues with writing to and reading from a file “at the same time”.  We had come to  a solution that sort of worked but was grossly unpredictable.

Back to research.

This time around, I decided to go straight to documentation.  Lo and behold, serial port communication in C# (on Mac!) is so difficult it takes only two lines of code.  Apparently, the people that developed the code know better than the people on forums.  Who knew?  We now had a quick, one-stage solution for receiving serial port data that was vastly more dependable.

Section IV: It Actually Works, Sort Of

Our final, and working, implementation went something like this:

  • Scan environment horizontally and vertically while taking incremental distance measurements and returning the data
    • Data returned included
      1. Voltage from IR sensor
      2. Horizontal angle of servo 1 position
      3. Vertical angle of servo 2 position
  • Write captured data to a serial port
  • From Unity, read in a line from the serial port at a regular interval
  • Parse the data from the read-in string
  • Display datapoint in rendering by positioning a block at points
    • x: voltage * Mathf.Cos (Mathf.PI / 180 * horizontalAngle)
    • y: voltage * Mathf.Sin (Mathf.PI / 180 * verticalAngle)
    • z: voltage * Mathf.Sin (Mathf.PI / 180 * horizontalAngle)

With the hardware correctly set up and the code in place to implement these steps, we had a working system.  Observe:

Scanning motion:

Example of rendering one swipe of data in Unity:

The result of a full scan:

Screen Shot 2015-12-18 at 3.41.31 PM

Front View

Screen Shot 2015-12-18 at 3.42.09 PM

Side View

Section V: Looking Back

Our results were not as pretty as we initially desired.  As mentioned earlier, we chose (in all honesty, by accident) a long range IR sensor to obtain our distance data.  This caused us a few problems.  The data points were required to start at 1 meter out, due to the limitations of the sensor.  This made testing a bit difficult.  Also, the input voltages, by which the distances were calculated, were very inconsistent.  Readings were all over the place.  The data sheet for the sensor suggested using a capacitor to regulate the input power supply (in turn dampening the jumps in output voltage), but we were unable, due to time constraints, to pursue this avenue.  If doing this project a second time, we would definitely choose a much closer range sensor in order to obtain much more precise results.


The Unity project and Arduino code can be viewed and downloaded at



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s