Currently, anomaly detection is receiving a great deal of attention due to its importance in many real-world applications especially in data analytics. Anomaly detection has two important purposes in data pre-processing stages: one is to detect and attempts to eliminate them if and only if it is really outlier, while the other requires attention be paid to valid outlier (extreme value) because extreme value themselves carry the significant and critical information, such as extreme weather events. It will lead to significant harm if not detected and treated properly. This paper begins with the summary of anomaly detection, challenges, type of anomalies, extreme value followed by machine learning algorithms approach that deal with anomaly detection. Finally, we provide a conclusion that discusses about the important of detecting clearly between outlier or extreme value in data pre-processing stage, so that an effective and efficient prediction can be achieved later in modelling stage. © 2020 IEEE.