Using hand gestures to control mouse pointer in c sharp



Why mapping hand gestures to system mouse pointer controls?

Hand gesture recognition, including hand gestures and movements, is becoming increasingly attractive. Many people are currently working on it to achieve natural interaction. One important step between hand gesture recognition and natural-interaction applications is to map detected hand gestures to system mouse pointer control. In one of previous articles here.

In this article an simple example is provided to demonstrate how to map detected hand gestures’ movements to system mouse pointer position using c#.

An example

Because my current work is focusing on c#, this example is written in c#. For other programme languages, such as c++ and c, the principle is similar, the detail functions may be different though.

Below is the source codes of this example, comments are given to explain what does each line mean.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Runtime.InteropServices;

namespace pointerCtrl
class Program
//load system dll file user32.dll
//define a static function as entrance of user32.dll
static extern int SetCursorPos(int x, int y);

//the main function
static void Main(string[] args)
//a 1000 loop
for (int i = 0; i < 1000; i++) { //key part: set mouse position to specific position SetCursorPos(i, i); //press any key to continue to next position Console.ReadKey(true); } } } }

Step further – more operations with mouse pointer control

There are other functions specified to take over system mouse pointer control.

static extern void mouse_event(MouseEventFlag flags, int x, int y, unit data, UIntPrt extraInfo);
enum MouseEventFlag : Unit
Move = 0x0001,
LeftDown = 0x0002,
LeftUp = 0x0004,
RightDown = 0x0008,
RightUp = 0x0010,
MiddleDown = 0x0020,
MiddleUp = 0x0040,
XDown = 0x0080,
XUp = 0x0100,
Wheel = 0x0800,
VirtualDesk = 0x4000,
Absolute = 0x8000

It is not working to use
mouse_event(MouseEventFlag.LeftDown, 100,100,0,UIntPtr.Zero)
to trigger a mouse left button click at [10,10].

To realise that, there needs two steps.
1) moving pointer to right position [10,10].
2) triggering a left button click event, by using
mouse_event(MouseEventFlag.LeftDown, 0, 0, 0, UIntPtr.Zero).

Author: Andol Li

A HCI researcher, a digital media lecturer, an information product designer, and a python/php/java coder.

6 Comments On “ Using hand gestures to control mouse pointer in c sharp”

  1. hello,I am curious about hands gestures to control mouse pointer.I am confused about left mouse or right mouse is distinguished. Thanks for reading my confusion.

    • @David
      as posted in the code example, there are several parameters used to identify specific mouse operations, including mouse left/right click, hold and drag, and so on.

  2. hello, can hep me my final year project based on the controlling mouse pointer using hand gestures is one of my project module and can u help me this module because i m more time try to work this module but not working it . please help other wise m not able to go next year. please share c code for this in c# . or C/C++ m already work in opencv library . so plese help me .

Leave a comment
Due to technical adjustments, the comment function is shortly closed and will be re-openning soon. Thanks.

Copyrights 2006-2018 © All rights reserved
Theme Tree2, re-designed by Andol Li, powered by WordPress and Bootsrap
WWW.ANDOL.ME | 浙ICP备15040508号-1
公安备案图标 浙公网安备33010602004018号
Back to top