Learning and recognizing 3-dimension (3D) adaptive features are important for crowd scene understanding in video surveillance research. Deep learning architectures such as Convolutional Neural Networks (CNN) have recently shown much success in various computer vision applications. Existing approaches such as hand-crafted method and 2D-CNN architectures are widely used in adaptive feature representations on image data. However, learning dynamic and temporal features in 3D scale features in videos remains an open problem. In this study, we proposed a novel technique 3D-scale Convolutional Neural Network (3DS-CNN), based on the decomposition of 3D feature maps into 2D spatio and 2D temporal feature representations. Extensive experiments on hundreds of video scene were demonstrated on publicly available crowd datasets. Quantitative and qualitative evaluations indicate that the proposed model display superior performance when compared to baseline approaches. The mean average precision of 95.30 was recorded on WWW crowd dataset. © 2017 IEEE.