The evaluation metric is an essential part to evaluate text generation tasks such as news headline generation. The ROUGE evaluation metric is still the standard evaluation metric for most text summarization tasks. Nevertheless, this metric has its weakness due to its nature of measuring exact lexical overlapping between the candidate text and reference text. In this paper, we would like to attempt to propose a word embedding-based ROUGE named EmbeddingROUGE. The proposed design of this embedding-based evaluation metric aim to overcome the weakness of the ROUGE metric. Generally, the experiment shows that the EmbeddingROUGE demonstrates superior evaluation score over the original ROUGE. © 2022 IEEE.