Hand gesture detection and recognition using openCV 2

hand gesture detect using openCV - demo

hand gesture detect using openCV - demo

Before the example – why haar classifiers for hand gesture detection

As mentioned in one of early articles such like hand gesture detection and recognition using openCV, there are two main ways to detect hand gestures, including skin colour segmentation and haar classifier training, although recently some feature-detecting algorithms are used to detect hand gestures such like swift. Skin colour segmentation is a dead end – hand colours can be white, or black, or any other colours – depends on the varieties of environmental light conditions. So using a skin colour model (as in The colour range for HSV skin extraction) with a specific colour range can do hand gesture recognition quite well in strict scenarios but this still faces tough challenges for other use.

The source codes

// Name : opencv_handdetect.cpp
// Author : andol li, andol@andol.me
// Version : 0.1
// Copyright : 2012
// Description : using haartraining results to detect the hand gesture of FIST in video stream.
#include <opencv/cv.h>
#include <opencv/cxcore.h>
#include <opencv/highgui.h>#include

using namespace cv;
using namespace std;

const double scale = 1.1;

//1.0 api version
CvMemStorage* storage = 0;
CvHaarClassifierCascade* cascade = 0;
void detectAndDraw(IplImage *input_image);
const char* cascade_name = “fist.xml”;

//define the path to cascade file
string cascadeName = “fist.xml”; /*ROBUST-fist detection haartraining file*/

int main()
//1.0 api version
CvCapture *capture =0;
IplImage *frame, *frame_copy = 0;
cascade = (CvHaarClassifierCascade*)cvLoad( cascade_name, 0, 0, 0 );
if( !cascade ){
fprintf( stderr, “ERROR: Could not load classifier cascade\n” );
return -1;
storage = cvCreateMemStorage(0);
capture = cvCaptureFromCAM(0);
cvNamedWindow(“result”, 1);
if(!cvGrabFrame(capture)) break;
frame = cvRetrieveFrame( capture);
if(!frame) break;
if(!frame_copy) frame_copy = cvCreateImage(cvSize(frame->width, frame->height), IPL_DEPTH_8U, frame->nChannels);
if(frame->origin == IPL_ORIGIN_TL)
cvCopy(frame, frame_copy, 0);
cvFlip(frame, frame_copy, 0);
if(cvWaitKey(10) >= 0) break;
cvReleaseImage( &frame_copy );
cvReleaseCapture( &capture );

return 0;

void detectAndDraw(IplImage *img)
double scale = 1.1;
IplImage* temp = cvCreateImage( cvSize(img->width/scale,img->height/scale), 8, 3 );
CvPoint pt1, pt2;
int i;

cvClearMemStorage( storage );
CvSeq* faces = cvHaarDetectObjects(
cvSize(24, 24) );
for( i = 0; i < (faces ? faces->total : 0); i++ )
CvRect* r = (CvRect*)cvGetSeqElem( faces, i );
pt1.x = r->x*scale;
pt2.x = (r->x+r->width)*scale;
pt1.y = r->y*scale;
pt2.y = (r->y+r->height)*scale;
cvRectangle( img, pt1, pt2, CV_RGB(200, 0, 0), 1, 8, 0 );
cvShowImage(“result”, img);
cvReleaseImage( &temp );

it can be downloaded from here SOURCE CODES OF HAND GESTURE DETECTION AND RECOGNITION USING OPENCV, relevant files/codes are for requests.

Author: Andol Li

A HCI researcher, a digital media lecturer, an information product designer, and a python/php/java coder.

38 Comments On “ Hand gesture detection and recognition using openCV 2”

    • @sergi
      thanks for your interests, the current haar xml file is still not robust enough for open palm detect in chaotic backgrounds – guess this is due to the lack of training samples – but it will be published as soon as it achieves satisfied results.

    • @ryan
      Hi there, nice to have your arguments here – I have downloaded and read that paper – the adaptive skin colour detection still face some limitations – correct me if I am wrong:
      1. there must be a colour image, a gray image may not have enough colour information to shape the boundaries.
      2. the hand must be within a narrow range of normal light scenarios – a hand under blue light does not fit with the hisgram range of normal hand colours. if it is, then the hand colour range, particularly the Hue value, is far toooo broad.
      3. the background with the hand must be static, otherwise the motion object’s colour extraction would be inaccurate.

      ps- i like the board game project you did recently.

      • Yes your right with point 1 and 2, the hand has to be in the ‘normal’ colour ranges for the initial detection but I think after that as long as changes are gradual then the system would adapt to it, although I could be wrong, it’s been a while since I read the paper myself!

        I think moving objects in the background would only be a problem is they are skin coloured, so people walking passed or maybe blowing branches, but so long as they aren’t skin coloured it shouldn’t be a problem. Also this would only be a problem if they are constantly moving and stay on the scene.

        Hoping to have a implementation of this done at some point so will let you know how it goes.

        Really need to get my C and OpenCV skills built up so thought it was a good thing to take on a project in my spare time. Just setting the scene just now it’ll get more interesting soon!

  1. Hey Andol,
    I am looking for a tool that can recognise body gesture from video(.avi file). Please share the link with me if u have one.

    Thank you,

  2. libv4l2: error setting pixformat: Device or resource busy
    HIGHGUI ERROR: libv4l unable to ioctl S_FMT
    libv4l2: error setting pixformat: Device or resource busy
    libv4l1: error setting pixformat: Device or resource busy
    HIGHGUI ERROR: libv4l unable to ioctl VIDIOCSPICT

    what are all these errors mean? how to solve it?

  3. main.cpp:22: error: stray ‘\224’ in program
    main.cpp:25: error: stray ‘\223’ in program
    main.cpp:25: error: stray ‘\224’ in program
    main.cpp:34: error: stray ‘\223’ in program
    main.cpp:34: error: stray ‘\’ in program
    main.cpp:34: error: stray ‘\224’ in program
    main.cpp:39: error: stray ‘\223’ in program
    main.cpp:39: error: stray ‘\224’ in program
    main.cpp:85: error: stray ‘\223’ in program
    main.cpp:85: error: stray ‘\224’ in program
    main.cpp:20: error: expected primary-expression before ‘*’ token
    main.cpp:22: error: ‘fist’ was not declared in this scope
    main.cpp:25: error: ‘fist’ was not declared in this scope
    main.cpp: In function ‘int main()’:
    main.cpp:34: error: expected ‘)’ before ‘:’ token
    main.cpp:34: warning: null argument where non-null required (argument 2)
    main.cpp:39: error: ‘result’ was not declared in this scope
    main.cpp: In function ‘void detectAndDraw(IplImage*)’:
    main.cpp:85: error: ‘result’ was not declared in this scope

    I’m getting this kind of errors while compiling, what can be the reason?

  4. OpenCV Error: Parsing error (haar/fist.xml(4): Valid XML should start with ”) in icvXMLParse, file /home/frankie/Downloads/opencv-2.4.4/modules/core/src/persistence.cpp, line 2258
    terminate called after throwing an instance of ‘cv::Exception’
    what(): /home/frankie/Downloads/opencv-2.4.4/modules/core/src/persistence.cpp:2258: error: (-212) haar/fist.xml(4): Valid XML should start with ” in function icvXMLParse
    Hi sir,
    after “run” opencv_handdetect.cpp with the “fist.xml”&”palm.xml”, both result the above error. Do u have any idea why? thx.

    • @pengin
      be aware of openCV versions – as the fist.xml and palm.xml worked fine in 2.3.1 with many times of test.

      openCV may have made some upgrades to the xml handling functions in new versions, such as 2.4.4, that may/will cause unexpected compatability problem.

      to get over this issue, you can either try change the xml file to fit your xml file parser, such like the ” start of valid XML file as the error indicates, or try find and use another XML parser which can work well.


  5. Hey Andol, I’m student, I’m working on a project using (OpenCV 2.4.1) that can detect hand gestures, but I’ve to recognize the hand by the shape, not by the color, your page It’s amazing. Do you have more Gestures?.

    Thanks for share.

  6. Hi,Andol.i am a student,i want to do a project that should be able to “convert sign language to text for deaf and dumb people”,can i have source code for that,it will be very helpful to me.
    Thank You.

  7. hi,Andol. Im working on a project where we want to track d path of moving objects, infact our hand or finger using webcam. actually we want to track d digits shown using our hands like 1, 2,3….,0. the language used is python. cn u help me to do it..pls…

  8. I am using Visual Studio 2010 to compile your code.

    I want to ask, where should I put the cascade file, because I always only get ERROR: Could not load classifier cascade

    Where should I put the resource file? On the Resource Files / at the Debug folder?

    I even tried to put the xml file at both but still, no result.

    Thank you very much.

    • @Raymond
      technically the cascade file is located in debug folder, but to ensure the programme can read the file you can put the cascade file in release folder and src folder as well.

Leave a comment
Due to technical adjustments, the comment function is shortly closed and will be re-openning soon. Thanks.

Copyrights 2006-2017 © All rights reserved
Theme Tree2, re-designed by Andol Li, powered by WordPress and Bootsrap
WWW.ANDOL.ME | 浙ICP备15040508号-1
公安备案图标 浙公网安备33010602004018号
Back to top