In this paper we present an idea for Human computer Interaction (HCI), where we have tried to control the cursor operation of the mouse using hand fingers (Red Green Blue). Hand gestures were collected using a camera based on color detection technique. This method mainly concerned on the use of a Web Camera to develop a virtual HCI device in a cost effective manner. The system that we aim to develop is interacting with humans i.e. it creates a human computer interface for Performing Mouse operations using RGB plains. The geometrical operation and integral image processes are performed on the input. System will compare both coordinates and perform the action i.e. it will allow the user to control the mouse operations using webcam based on which color tape is used in front of the web camera.