This paper studies the effect of colour space on the performance of tracking algorithms. The colour spaces that were investigated were grayscale, RGB, YCbCr and HSV. The performance of a normalised cross correlation tracking algorithm was measured to determine robustness and accuracy in the different colour spaces. Track Detection Rate (TDR) and Object Tracking Standard Deviation (OTStd) were used to provide quantitative measures of tracking performance. The combined results indicate that the colour spaces of YCbCr and HSV give more accurate and more robust tracking results compared to grayscale and RGB images. The results also show that the information stored in the chrominance layers of CbCr in the YCbCr colour space and chromaticity layers HS in the HSV colour space, were sufficient for robust tracking. The TDR results range from 93.7 to 97.1 for grayscale and RGB, and 98 to 100 for the YCbCr and HSV colour spaces respectively. A similar trend in the OTStd was observed with a range of 17.0 pixels to 23.9 pixels for grayscale and RGB, and 7.56 pixels to 20.5 pixels for YCbCr and HSV. ©2008 IEEE.