OpenNI + OpenCV + PCL + Mountain Lion

This is a setup guide for getting a Kinect Sensor to work partially with OpenCV and PCL at the same time under Mountain Lion. THIS GUIDE USES THE NEWLY OUTDATED VERSIONS OF THESE FRAMEWORKS. IT WILL BE UPDATED TO KEEP UP. Keep in mind that visualization with VTK(PCLs default visualization kit) and its own OpenNI module will not function properly or at all. As a workaround I am using the following pipeline:

  1. Capture with OpenNI module of OpenCV
    • Capture PointCloud maps and BGR color maps, and others if needed
  2. Apply OpenCV functions if needed
    • Calibration
    • Feature extraction
    • etc.
  3. Pipe OpenCV matrices to pcl::<PointCloud> types
  4. Apply PCL functions if needed
    • Filters
    • Normal estimation
    • Registration
    • etc.
  5. Visualize with OpenGL

Disclaimer: I have a fine arts degree, not computer science. Kindly point my mistakes, redundancies, my overall stupidity, but most importantly your comments and improvements to muratg[at]gmail if you would like to. Thank you.

Before anything is installed, it is best to clean the system from any remnants of previous installed packages. Third party installations of MacPorts, OpenNi, QHull, Flann, Boost, Eigen, VTK, GLEW, GLUT, OpenCV, PCL, etc has to be removed for a smooth setup. Please take care not to accidentally uninstall any system packages.

Homebrew

This guide uses the Homebrew package manager. Please install its dependencies before anything else and install itself right after.

Homebrew will throw warnings and errors during use. Please take these into account before continuing to avoid bigger problems later on.

XCode dependency

XCode or at least XCode Command Line Tools is required. XCode is now available as a free download at the Mac App Store. XCode itself is a heavy piece of software, but fortunately Apple made the Command Line Tools available independent of Xcode. It can be downloaded from https://developer.apple.com/downloads/index.action with a free Apple Developer Connection account.

Install Homebrew

Installing Homebrew is very simple, just copy and paste the following command in a terminal window running bash:

ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"

You can also check their homepage http://mxcl.github.io/homebrew/ for any changes to install command in the future or for more information.

If you have MacPorts it is best to get rid of everything it installed and itself before installing Homebrew. Following listing will first uninstall everything installed by MacPorts and uninstall MacPorts itself:

sudo port -fp uninstall installed
sudo rm -rf \
/opt/local \
/Applications/DarwinPorts \
/Applications/MacPorts \
/Library/LaunchDaemons/org.macports.* \
/Library/Receipts/DarwinPorts*.pkg \
/Library/Receipts/MacPorts*.pkg \
/Library/StartupItems/DarwinPortsStartup \
/Library/Tcl/darwinports1.0 \
/Library/Tcl/macports1.0 \
~/.macports

You can also check MacPorts official uninstall documentation for changes at http://guide.macports.org/chunked/installing.macports.uninstalling.html.

If you have installed anything in /usr/local/* it is best to take a prior look in there and make sure that what Homebrew is about to install won't conflict your software development. In this case if there are any remnants of OpenNI, OpenCV, PCL, and their dependencies, they have to be cleared for a clean install. For example, following command will display all files in the system that have pcl in their name:

sudo find / -name "*pcl*"

Slow network mounts and USB drives will cause find command to perform slow. Each line of the output has to be checked and deleted if it belongs to PCL.

After Homebrew succesfully installs brew doctor command checks the system and points out any current and potential problems. It helps to run brew doctor once in a while to keep an eye on the development environment.

Homebrew install formulas are kept up-to-date rigorously to keep up with daily updated open source software and to iron out bugs. Running brew update will synchronize the Homebrew package installer ruby scripts with their GitHub repository. Keep in mind that update command will only update the installer scripts and not the already installed packages. brew upgrade will upgrade all packages installed and their dependencies. upgrade command might break the developer environment due to changing library files or other internal calls.

Homebrew is very capable, please check its GitHub page at https://github.com/mxcl/homebrew/wiki for a lot more.

Install OpenNI

Homebrew itself does not have OpenNI installer scripts. However, it is possible to tap to repositories that do.

Following command will tap the local Homebrew installation to download an OpenNI installer repository:

brew tap totakke/openni

Documentation specific to this repository can be found at https://github.com/totakke/homebrew-openni.

After the installer scripts are downloaded, install OpenNI with the following commands:

brew install openni
brew install sensor-kinect
brew install nite

Please run these one by one and keep in mind that openni package will fail to install the first time if you don't have Java Developer Kit installed. If a dialog box asking if you wish to install JDK appears, please say yes and try brew install openni again before continuing with the other lines.

If another ONI compatible depth camera will be used other than Kinect, and this includes the PrimeSense Carmine 1.08 too which is identical to Kinect, please run brew install sensor instead of sensor-kinect. Unfortunately, both Kinect and another ONI compatible sensor cannot be used with OpenNI simultaneously.

To test the installation and your device, copy and paste the following listing into terminal:

cd /usr/local/Cellar/openni/unstable-1.5.4.0/sample/Bin/x64-Release
./Sample-NiSimpleViewer

You may have to change the unstable-1.5.4.0 to the version you have installed.

If you have OpenKinect it is best to get rid of it prior to installing OpenNI since they will both attempt to use the same libusb library and each have different libusb libraries. Please remove libfreenect with its libusb properly.

Install OpenCV

OpenCV installer script is available in the official Homebrew repository, however it is located in the science section. So first tap in with:

brew tap homebrew/science

Homebrew does have a solid formula to install OpenCV within its science tap. However, the formula won't find the OpenNI directories installed by the previous openni tap by default. Luckily, a commit request at GitHub solves this issue.

Until this commit is merged to the master branch, you have to open /usr/local/Library/Taps/homebrew-science/opencv.rb and replaces its contents with the contents of the commit request at https://github.com/Homebrew/homebrew-science/pull/224/commits. It is also best to make sure that ffmpeg is not installed in the system prior to installing OpenCV. Alternatively you can add the following to the args within def install block to to both disable ffmpeg support and to build the OpenCV world library:

-DWITH_FFMPEG=OFF

Following listing will install OpenCV and its dependencies:

brew install openexr
brew install opencv --with-eigen --with-libtiff --with-qt --with-tbb --with-openni 

This will attempt to compile OpenCV with OpenCL if OpenCL is available. Unfortunately, the script doesn't compile the examples so there is no way to test if OpenCV functions before actual development.

If you already have OpenCV already installed, remove each and every file that belongs to it and its dependencies. If you used an installer package, its "bill of materials" or bom file is located in /var/db/receipts/. You can see all the files installed by the installer package with the following command:

lsbom -l /var/db/receipts/*opencv*.bom

Please note that all the paths are relative. It is best to check the entire output of lsbom before proceeding with:

lsbom -fls /var/db/receipts/*opencv*.bom | (cd /; sudo xargs rm)
lsbom -dls /var/db/receipts/*opencv*.bom | (cd /; sudo xargs rmdir -p)
sudo rm *opencv*.bom *opencv*.plist

Listing above will first delete the files, then the empty directories, and finally the bom and plist files in the receipts folder. Replace *opencv* with the actual package name.

If the previous OpenCV was compiled from source, go to its build folder and run make uninstall. If the cmake generated files are still in place make uninstall will run smoothly. If not, or if the build folder doesn't exist, the previous version of OpenCV will have to be built again for a clean uninstall. Following listing will compile OpenCV and uninstall it from the system:

cmake ..
make
make uninstall

Install PCL

Official Homebrew repository does not have a formula to install PCL, and regrettably the formula PCL website provides is out of date and not updated as often as it should. As a result, the best way to install PCL is to compile it from source.

Following listing will clone the PCL git repository for the latest unstable version:

mkdir ~/source
cd ~/source
git clone https://github.com/PointCloudLibrary/pcl pcl-trunk

Following listing will prepare the build folder and compile PCL:

mkdir build
cd build
cmake -DOPENNI_INCLUDE_DIR=/usr/local/include/ni -DGLEW_INCLUDE_DIR=/usr/X11R6/lib -DCMAKE_BUILD_TYPE=Release ..
make

Although the arguments passed in to cmake should fix most errors and warnings, make might throw other errors and these have to be fixed before proceeeding. Once PCL is compiled, install it with

make install

It is a good practice to keep the build folder intact for future proofing. If you are running out of precious disk space, run make clean instead of removing it altogether. This way you will still be able run make uninstall smoothly and re-compile if needed.

If you installed PCL and its dependencies before from packages, you will have to remove all of it before compiling the new PCL. Packages found at http://pointclouds.org/downloads/macosx.html are prepared with MacPorts, and the system should be clean after uninstalling MacPorts. However, some files might have been left over. System can be checked for any left overs with the first line of the folloing listing:

sudo find / -name "*pcl*"
sudo find / -name "*pcl*" -exec rm -i {} \;

Second line of the above listing will ask the user each time to delete a found file. Please consider that not all files in your system that contain the string pcl actually belong to PCL. Read twice, delete once. Keep in mind that the command above is case sensitive and it will only look for "pcl" and not "PCL". Previous installer might have installed PCL under a different name such as "PointClouds" as well. A through search is required to make sure every remnant of PCL and its dependencies are uninstalled.

The above steps and instructions should create the development environment for the aforementioned pipeline. However, the versions of the software change daily and independent from each other. It is very likely that one package may break the installation or functionality of another over time. Fortunately, developers of these packages are very dedicated and attentive that a broken installer might be fixed in a very small amount of time.