Sixth sense device is a gesture recognition device which uses hand gestures for recognition and performing various operations. Color segmentation is an important task for sixth sense device after color recognition. A RGB color space is an additive color space based on RGB color model. This proposed color model is used to recognize and segment various colors from real time images. A RGB color space model consists of three chromaticity?s i.e. Red, Green & Blue additive primaries, and can produce any chromaticity that is defined by those colors [1]. The continuous frames of video are converted to RGB color space, where the color information is extracted for color recognition. Here we capture video through HD webcam. After color recognition, the color is being segmented. The color segmentation involves RGB color normalization, centroid calculation. The Euclidean distance formula is being used for color recognition & centroid location. The coding for all the above steps is being done using MATLAB 7.1 version.