The complexity of a scene in addition to the need for real-time processing are the main challenges that face any background/foreground separation approach for maritime environment. Recent studies on Low-rank and Sparse Separation (LSS) achieved good performance when compared to traditional background subtraction techniques in segregating the foreground from a complex background. However, the issue of maintaining this type of a separation via an updating mechanism is not well addressed by the majority of LSS approaches. The study presents a tensor based singular value decomposition approach for background/foreground separation. The approach is uniquely designed to deal with most challenges related to a maritime environment such as sea dynamics, boat wakes, variety of foreground objects, and camera jitter. Furthermore, the proposed approach operates incrementally via updating the separation components as opposed to reperforming the decomposition on the entire video sequence when a set of frames arrives. Additionally, a forgetting mechanism is employed in the proposed approach to efficiently handle challenges such as Stationary Foreground Objects (SFOs) and ghost effects. The performance of the proposed method with several state-of-the-art LSS and Non-LSS techniques on videos with complex maritime scenarios are evaluated. The results exhibit better performance over most of the tested challenges and also demonstrate the capability of the proposed method to perform the separation in less computational time. © 2000-2011 IEEE.