This study presents the deep insight and comprehensive analysis of evolving spiking Neural network (eSNN) development in recent years (last eight years). eSNN has been used to a vast number of optimization problems. It has several advantages: Computationally inexpensive, knowledge-based, on-line learning method, and we have analyzed the improvements of eSNN in different application zones. This review paper discusses eSNN optimization done by researchers using distinct optimization techniques to achieve the possible best accuracy. In this inclusive study, few publications using eSNN have been gathered and summarized. First, we introduce eSNN. Then, we characterized the current versions of eSNN into 5 variants mainly Hybridization, Modifications, Multi-objective, Dynamic, and Integration. Afterwards, the results of the studied eSNN models being evaluated. The review paper is summed up by giving a conclusion of the optimized eSNN model's fundamentals and providing thinkable future directions that can be explored in the current works on the Hyper-parameter optimization of eSNN. © 2005 - ongoing JATIT & LLS.