Radial Basis Function networks (RBF) is one form of feed forward neural network architecture which is popular besides Multi Layer Preceptor (MLP). It is widely used in identifying a black box system. Finite Difference approach is used to improve the learning performance especially in the non-linear learning parameter update for identifying system with lost packet in online manner. Since initializing of non-linear learning's parameters is crucial in RBF networks' learning, some unsupervised learning methods such as, K-means clustering and Fuzzy C-means clustering are used besides random initialization. All the possible combination methods in the initialization and updating process try to improve the whole performance of the learning process in system identification with lost packet compared to Extreme Learning Machine as the latest improved learning method in RBF network. It can be shown that Finite difference approach with dynamic step size on Decomposed RBF network with Recursive Prediction Error for the non-linear parameter update with appropriate initialization method succeed to perform better performance compared to ELM. © 2009 IEEE.