OpenCV Face Recognition

tl;dr

OpenCV is an Open Source computer vision library that, among other things, can recognize faces, including eyes and smiles.

I set out a few weeks ago to build some prototypes with the library, to get an idea of how it works. I have created 3 prototypes, one that detects faces, one that detects eyes and one that detects smiles. All the projects are on GitHub here and they all use Python 3.

Installation

The following are steps to install OpenCV libraries and Python 3 for macOS and Linux. Sorry Windows users, you are on your own here.

macOS

For macOS, the most straight forward way to install OpenCV and Python 3 is to use Homebrew. The following steps will install Xcode, Homebrew, Python 3 and OpenCV.

Software Versions

Testing was done with macOS High Sierra.

OpenCV Version

The OpenCV face recognition demos do not require the latest version of OpenCV, so the following steps will be using the current packaged version.

To use the latest version of OpenCV install from source, see the Reference links at the end of the macOS and Linux sections.

First, install Xcode. Open the macOS App Store app, search for Xcode and install. Downloading and installing Xcode will take a while, it is a large application.

After Xcode has been downloaded and installed, open Xcode and accept the license. If prompted to install additional components, install them.

Next, install Homebrew; Homebrew is a package manager for macOS, it will be used to install Python and OpenCV.

Visit the Homebrew home page and run the ruby script to install:

https://brew.sh/

The command should be something like:

Existing Homebrew

If you have an existing Homebrew installation, be sure to run update, doctor and upgrade:

Finally, install Python 3 and OpenCV:

Homebrew installs to /usr/local/bin, so make sure that it is on the path (you may have to open a new terminal window):

Response should include the /usr/local/bin, for example:

Reference:
https://www.learnopencv.com/install-opencv3-on-macos/

Linux

These steps will be specific to Ubuntu, but other distributions will be similar.

Software Versions

Testing was done with Ubuntu 18.04 Desktop.

First, install all updates, patches and reboot if necessary:

Ubuntu 18.04 Desktop includes Python 3 by default, so only install OpenCV Python libraries and git:

Reference:
https://www.learnopencv.com/install-opencv3-on-ubuntu/

Confirm Installation

At this point, confirm that Python 3 and OpenCV are installed correctly, run Python in interactive mode and get the version number:

It should look like:

Hit control d to exit.

Face Recognition

What are Haar Cascade Functions?

OpenCV includes functions (Haar Casacades) that can be trained to identify images. These images can be anything, like cars, animals, or faces. The training is done using examples of positive images (e.g. images containing a face) and negative images (e.g. images that don’t contain a face).

The exact details of how Haar Cascade functions work is pretty much magic. To borrow from the Wikipedia entry on Haar-like features:

A Haar-like feature considers adjacent rectangular regions at a specific location in a detection window, sums up the pixel intensities in each region and calculates the difference between these sums.

The key advantage of a Haar-like feature over most other features is its calculation speed. Due to the use of integral images, a Haar-like feature of any size can be calculated in constant time.

So magic. But magic you can train. Also, the constant time calculation is important, it allows a predictable response to image recognition.

The OpenCV distribution also contains some pre-trained classifiers for face, eyes and smiles. We will be using those pre-trained classifiers in the following demos.

Reference:
https://docs.opencv.org/3.4.1/d7/d8b/tutorial_py_face_detection.html

General Approach

The demo apps are very bare bones Python apps, meant to be simple to understand the showcase the OpenCV functions.

The demo apps all do the following:

  • Event loop
  • Read frame from camera
  • Convert to gray scale
  • Use classifier
  • Drawing frame with face overlays

Some of the parameters need to be adjusted for the camera, as the subject may be larger or smaller in the frame depending on hardware and the subject(s) distance from the camera.

Those parameters to adjust are:

  • scaleFactor
  • minNeighbors
  • minSize
  • flags

Demo Code

The demo code can be cloned from:

https://github.com/gordonturner/opencv-face-recognition-demos.git

In a working directory, run the following to clone the projects:

Detecting Faces

The first demo is opencv-detect-faces, it is the simplest of the demos and only uses the haarcascade_frontalface_default.xml classifier.

Change to the demo directory and run app.py:

A new window should open with the output of the camera and a blue box around any detected faces.

If there are falsely reported faces, or not enough reported faces, adjust the parameters for your camera.

Hit the Esc key to exit.

Detecting Eyes

The second demo is opencv-detect-eyes, it is only slightly more complicated, it relies on the face being detected first and then within that region, looks for eyes. If face detection isn’t run first, there can be many false positives with ‘eyes’ everywhere. Spooky. It uses the haarcascade_frontalface_default.xml and haarcascade_eye.xml classifiers.

Change to the demo directory and run app.py:

A new window should open with the output of the camera and a blue box around any detected faces and red boxes around the eyes in the box.

As before, if there are falsely reported faces, or not enough reported faces, adjust the parameters for your camera.

Hit the Esc key to exit.

Detecting Smiles

The final demo is opencv-detect-smiles, it is similar to the second demo, using the face detection, but rather then eyes it will detect if the user is smiling. It uses the haarcascade_frontalface_default.xml and haarcascade_smile.xml classifiers.

Change to the demo directory and run app.py:

A new window should open with the output of the camera and a blue box around any detected faces and red boxes around smiles in the box.

As before, if there are falsely reported faces, or not enough reported faces, adjust the parameters for your camera.

Hit the Esc key to exit.

Summary

Hopefully these simple demos provide a bit of an insight into how face detection works. I hope to explore other features of OpenCV in the future.

Comments are closed.