# Advanced Lane-Detection for Self-Driving Cars



## Maven (Feb 9, 2017)

Release the Transhumanists! Bwahahahahahahahahahahaha 
____________________

becominghuman May 24 2017 By David Clark
*Advanced Lane-Detection for Self-Driving Cars*
*Project #4 in Udacity's Self-Driving Car Nanodegree*

Project #4 in the Udacity self-driving car program involves using computer vision and machine learning techniques to identify road lanes and lane features from a vehicle's dash-cam.

My code for this project is publicly available.

_This is part of a series. You can also read about my first, second, and third projects if you missed them!_

*Overview: The Pipeline*
The goal of this project is to develop a pipeline. Given an image of the road, the pipeline should draw _onto_ the image the features it identifies about the road's lane and lines. The lane-detection pipeline I developed consists of the following steps:

Pre-Step: Calibrate the camera
Undistort the image
Threshold the image using gradients and colors
Apply a perspective transform (warp) to view the image from top-down
Identify the lane lines in the warped image
Draw features back onto the original image
The pre-step, calibrating the camera, is performed only once (assuming all the images are taken from the same camera). All the other steps are performed on each image.

Let's look at each of these steps one-by-one.

*Calibrate the Camera*
"Calibrating the camera" really means accounting for the distortion in an image introduced by the camera's lens. This is done using multiple images of checkerboard patters, which _should_ have straight lines. Examining how the checkerboard patterns are distorted (not straight) allows us to precisely identify *how* the camera lens is distorting images - which means we can undistort them.









A distorted image of a checkerboard pattern placed on a wall. These lines should be straight&#8230;.
*Undistort the Image*
Once the camera is "calibrated," we simply need to apply the knowledge gained to undistort images. This is important because it will restore the straightness of lines, helping to identify lane lines later in the pipeline. The difference between the distorted and undistorted images is clear. The bent lines are now straight.









While the very top and left still don't seem perfect, it's a whole lot better now!
*Threshold the Image*
Thresholding is a method of isolating the pixels we are interested in. This can be done using a combination of gradient and color filters. Here's what a thresholded image looks like next to the original.

















The original road image and a thresholded image. I applied pixel-gradient and color threshold filters to narrow down the pixels of interest (lane lines).
*Perspective Transform*
While undistorting and thresholding help isolate the important information, we can _further_ isolate that information by looking only at the portion of the image we care about - the road. To focus in on the road-portion of the image we shift our perspective to a top-down view of the road in front of the car. While we don't gain any extra information from this step, it's much easier to isolate lane lines and measure things like curvature from this perspective.

















The thresholded image (left), and the same image with a top-down perspective shift (right).
*Identify the Lane Lines*
From the top-down perspective we can easily identify the lane lines. Below you can see the lane lines identified by a sliding window search. The green boxes represent the windows where we colored the lane lines. As the windows search upward they recenter to the average pixel position so they "follow" the lines. The colored lines will be drawn back onto the original image.









A sliding window search identifies each of the lane lines.
*Drawing onto the Original Image*
Finally, we take all this information we gathered and draw the results back onto the original image. The blue and red lines we identified above are present, and the space between them is colored green to show the lane. The calculated right/left lane curvature and center-lane offset are shown in the top-left of the image as well. (These values would be useful when telling a self-driving car how to steer.)









Final output - lane and lines drawn onto original image along with curvature and offset measures.

*Applying the Pipeline to Videos*
While the pipeline processes single images, it can easily be applied to processing videos by separating the video into frames and processing each frame individually.

I applied my pipeline to three different videos: the normal project video, a challenge video, and a _harder_ challenge video.





 Normal





 Challenge





 Harder


----------



## Gung-Ho (Jun 2, 2015)

That's all fascinating.

But what annoys me is when they do roadwork and change the lane patterns but you can still see the faint old lines in the road with the newer lines painted over them.


----------



## Maven (Feb 9, 2017)

Gung-Ho said:


> That's all fascinating, but what annoys me is when they do roadwork and change the lane patterns but you can still see the faint old lines in the road with the newer lines painted over them.


What annoys me is driving on highways at night in the rain where the lines, that have not been redrawn in about 10 years, can barely be seen. I agree with your example, but feel mine is far more dangerous, traveling at 50-60 mph in poor visibility. Unfortunately, the article talks about improving lane detection for self-driving cars, not humans. There is no reason to believe both will improve concurrently. It might even get worse for us humans.


----------



## Mars Troll Number 4 (Oct 30, 2015)

Gung-Ho said:


> That's all fascinating.
> 
> But what annoys me is when they do roadwork and change the lane patterns but you can still see the faint old lines in the road with the newer lines painted over them.












Good luck robo car....


----------

