Long Short-Term Memory (LSTM) is a popular Recurrent Neural Network (RNN) algorithm known for its ability to effectively analyze and process sequential data with long-term dependencies. Despite its popularity, the challenge of effectively initializing and optimizing RNN-LSTM models persists, often hindering their performance and accuracy. This study presents a systematic literature review (SLR) using an in-depth four-step approach based on the PRISMA methodology, incorporating peer-reviewed articles spanning 2018�2023. It aims to address how weight initialization and optimization techniques can bolster RNN-LSTM performance. This SLR offers a detailed overview across various applications and domains, and stands out by comprehensively analyzing modeling techniques, datasets, evaluation metrics, and programming languages associated with these networks. The findings of this SLR provide a roadmap for researchers and practitioners to enhance RNN-LSTM networks and achieve superior results. © 2024 The Author(s)