Enhanced oil recovery (EOR) has been offered as an alternative to declining crude oil production. EOR using nanotechnology is one of the most innovative trends in the petroleum industry. In order to determine the maximum oil recovery, the effect of a 3D rectangular prism shape is numerically investigated in this study. Using ANSYS Fluent software(2022R1), we develop a two-phase mathematical model based on 3D geometry. This research examines the following parameters: flow rate Q = 0.01�0.05 mL/min, volume fractions = 0.01�0.04, and the effect of nanomaterials on relative permeability. The result of the model is verified with published studies. In this study, the finite volume method is used to simulate the problem, and we run simulations at different flow rates while keeping other variables constant. The findings show that the nanomaterials have an important effect on water and oil permeability, increasing oil mobility and lowering IFT, which increases the recovery process. Additionally, it has been noted that a reduction in the flow rate improves oil recovery. Maximum oil recovery was attained at a 0.05 mL/min flow rate. Based on the findings, it is also demonstrated that Si (Formula presented.) provides better oil recovery compared to A (Formula presented.). When the volume fraction concentration increases, oil recovery ultimately increases. © 2023 by the authors.