<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=752538731515435&amp;ev=PageView&amp;noscript=1">

Self-Driving Cars: Inside the Machine

Instead of convincing you to trust or avoid a self-driving car, or debating timeline predictions, let's leave that drama to the thousands of other blogs for now.

Let's get a little adventurous and pop the hood instead... Can you spot the AI Brain in these pictures?

Car Engine 1.jpeg Car Engine 2.jpeg Car Engine 3.jpeg

That was a trick question. Despite the mesmerizing sound of the idea that there's an 'AI brain' inside the car driving it around like some sort of friendly terminator robot that's been tricked into being a chauffeur, the reality is that self-driving cars today are just putting together many simple little tricks. However, it will have a few extra computers to help with some of the simple math and added hardware.

Let's look at just a few of these, so that the next time you go to happy hour and your awesomely geeky programmer friend starts word-vomiting tech-heavy shizzle, you'll be able to carry the conversation. Overview of the simple tech and tricks we will look at:

LIDAR: Think SONAR, but instead of sound, it uses freaky spinning laser beams.
COMPUTER VISION: Detecting objects with the cameras that your self-driving car is decorated with.
PREDICTION: Little tricks to guess where the car next to you is going to be in about 1 second from now.


Uses light in the form of a pulsed laser to measure ranges - noaa.gov

Car makers and self-driving car software companies are now attaching spinning laser beams onto their cars! I know, pretty cool right?

Here's some pictures showing how beautiful this can make your car look, very steampunk, so much awesome going on here:

Uber_car_with_lidar (2).jpg 1280px-Hands-free_Driving.jpg Google's_Lexus_RX_450h_Self-Driving_Car.jpg

Pictures showing collected data points of distances.

Road 1.jpg Road 2.jpg Road 3.jpg

This means that your car can know, without using any image recognition or fuzzy logic, that there's a treeline or a pedestrian right in front of you, and that it needs to stop! It also helps your car know where the flat area of the road is, and where other cars around it are located.

Tech Crunch has a nice post that complements this topic: WTF is Lidar?


Even though the GPS is off by a few feet, it's still not ok to smash into oncoming traffic. By combining GPS with LIDAR and computer vision techniques to detect lane lines, we can keep the car in the lane.

One thing the cameras can do is look for yellow and white lines, and fit a simple math curve with what it sees, to predict where the lane is, and help the car add information to the GPS and LIDAR data about where it is in the world. Here is a quick homework assignment I did for my Udacity course that shows a really poor attempt at detecting the lanes with cameras only: CurvyMcLaneFace Project

Another cool technique is using the camera and some machine learning tricks to detect different objects and what they are, so that with the LIDAR and Camera together, we can have even higher confidence that there is a car in front of us and that it's not, in fact, Godzilla! It's always useful to know that the thing in front of you is a car or that the sign in front of you means stop. Fast cascade classifiers can easily pick out the cars in a picture, just like your phone knows where the faces are in an image. And neural networks can easily tell what type of street sign is in front of you!

This YouTube video from Davis King shows an example of a simple vehicle detector using the camera.

Neural networks can do other fun things besides telling the difference between a car and Godzilla, or classifying a 20MPH vs 30MPH street sign, like playing video games:

Building Neural Nets in Ryu's Dojo



When we see a bunch of points in the LIDAR map moving together, and we detect a car at that location in the image, we may want to predict where that object will be in a couple seconds, so that we can slow down if we need to, you know, to avoid crashing into the car in front of us!

Some math can be used to predict where objects will be based on their recent speed and direction. One trick that can be used is a Kalman Filter. Basically, it takes the current info, and calculates the next likely position and speed. It's a math formula that says this is probably where the car's going to be in about 1 second from now, and this is probably how fast the car is going, and this is what curve the car is probably turning on, etc...

Here is a fun video that introduces the Kalman Filter using a Pikachu:


Thanks for learning about some simple tech that goes into your next self-driving car. Now you probably know more about it than your local car salesman!

If you want to know more about self-driving cars, then I suggest you check out this awesome Udacity program:

Udacity Self-Driving Car Nanodegree

A car that's not very smart:

Smart Car.jpeg

And if you want to geek out more and continue the conversation, reach out so we can connect you with the right technical consultant today.

Hans Gronewold

About Author Hans Gronewold

Former Solutions Consultant at Omni.


Omni’s blog is intended for informational purposes only. Any views or opinions expressed on this site belong to the authors, and do not represent those held by people or organizations with which Omni is affiliated, unless explicitly stated.

Although we try to the best of our ability to make sure the content of this blog is original, accurate and up-to-date, we make no claims of complete accuracy or completeness of the information on this site/s to which we link. Omni is not liable for any unintended errors or omissions, or for any losses, injuries, or damages from the display or use of this information. We encourage readers to conduct additional research before making decisions based on the information in this blog.