Building OpenCV Stereo Vision - Getting Depth Images and 3D Point Cloud

No comments

Building OpenCV Stereo Vision - Getting Depth Images and 3D Point Cloud

At this point I'll assume entire stereo vision rig is setup and calibrated with calibration error small enough to get anything useful. If not, you can take a look how to calibrate or most common calibration mistakes for help with calibration.

Depth Image

OpenCV already provides four different algorithms to construct our depth image:

  • BM - block matching
  • SGBM - semi-global block matching
  • HH - H. Hirschmuller's algoritm
  • SGBM3way - semi-global block matching

I won't go into details, but based on my experience, HH algorithms is by far the fastest and gives the worst results. BM is slightly better, but SGBM and SGBM3way produce the best results. Image above is processed using SGBM method. None of the algorithms mentioned isn't anywhere near real time processing, so don't get your hopes to high. On an i7 CPU BM with everything turned off took about 10s to generate, while SGMB a couple of seconds more.

For generating depth images, I used provided OpenCV example program and slightly modified it. Didn't really see the need for rewriting. The usage is simple:

stereo_depth.exe left.png right.png -i intrinsics.yml -o extrinsics.yml--algorithm=sgbm -o depth_image.png -p point_cloud.obj

The program requires a couple of inputs. Left and right images are required, everything else is optional. I used SGBM algorithm therefore specified --algorithm program flag and I also wanted to create a point cloud file. To generate one OpenCV needs to have intrinsics (-i) and extrinsics (-e) projection matrices in order to calculate distances and generate points.

You might notice that OpenCV prints X, Y and Z coordinates for points, one per line and I specified Wavefront .obj file ending which isn't exactly what OpenCV prints. Here's where my slight modification comes in. I changed the following line (should be around line 38):

fprintf(fp, "%f %f %f\n", point[0], point[1], point[2]);

and added v in front of it for a valid .obj file. v in front means vertex making the file now completely compatible with any 3D programs such as Blender, SketchUp and others.

fprintf(fp, "v %f %f %f\n", point[0], point[1], point[2]);


Leave a Reply

Your email address will not be published.

Time limit is exhausted. Please reload CAPTCHA.