Waste management is becoming increasingly important around the world, and incorporating agro-waste into the pavement industry represents a promising strategy for achieving sustainability while improving mixture properties. In this study, we optimize and determine the optimum asphalt binder content of asphalt concrete mixtures modified with waste palm oil clinker powder (WPOCP) and waste rice straw ash (WRSA) to improve their engineering properties. To optimize the interactions between three independent variables (asphalt binder, WPOCP, and WRSA content) on mixture bulk unit weight (BUW), void in the total mix (VTM), Marshall stability, and flow values, the Marshall mix design approach and response surface methodology (RSM) with a Box-Behnken design were used. WPOCP samples containing 2, 4, 6, and 8 by weight of asphalt mixtures were prepared, as were WRSA samples containing 25, 50, 75, and 100 by weight of filler, with asphalt binder content ranging from 4 to 6 by weight of the mix. The statistical model results show that all responses were significant, with high coefficients of correlation (R2) of 0.9840, 0.9971, 0.9920, and 0.9891 for the BUW, VTM, Marshall stability, and flow, respectively. Individual effects of the input variables and synergistic interactions between the three variables were observed to influence all of the responses. Numerical optimization produced optimum WPOCP, WRSA, and asphalt content values of 8, 74, and 5, respectively. The mean error for all responses was less than 5, indicating that predicted values agree well with experimental data and that generated models accurately reflect experimental results. Based on the findings of the study, it can be concluded that RSM is an effective method for determining the optimal asphalt binder and modifier content in asphalt mixtures. It enables the identification of the most important variables influencing the response of the asphalt mixture and enables mixture optimization for improved performance. Furthermore, incorporating WPOCP and WRSA into asphalt mixtures was found to improve both volumetric and Marshall properties, resulting in a more sustainable approach in the pavement industry. © 2023 The Author(s)