With the widespread of blockchain technology, preserving the anonymity and confidentiality of transactions have become crucial. An enormous portion of blockchain research is dedicated to the design and development of privacy protocols but not much has been achieved for proper assessment of these solutions. To mitigate the gap, we have first comprehensively classified the existing solutions based on blockchain fundamental building blocks (i.e., smart contracts, cryptog-raphy, and hashing). Next, we investigated the evaluation criteria used for validating these techniques. The findings depict that the majority of privacy solutions are validated based on computing resources i.e., memory, time, storage, throughput, etc., only, which is not sufficient. Hence, we have additionally identified and presented various other factors that strengthen or weaken blockchain privacy. Based on those factors, we have formulated an evaluation framework to analyze the efficiency of blockchain privacy solutions. Further, we have introduced a concept of privacy precision that is a quantifiable measure to empirically assess privacy efficiency in blockchains. The calculation of privacy precision will be based on the effectiveness and strength of various privacy protecting attributes of a solution and the associated risks. Finally, we conclude the paper with some open research challenges and future directions. Our study can serve as a benchmark for empirical assessment of blockchain privacy. © by the authors. Licensee MDPI, Basel, Switzerland.