One of the missions of mobile edge computing (MEC) is to develop innovative applications in closer proximity while preserving latency constraints. Integration of MEC with the emerging wireless power transfer (WPT) approach extends the battery lifetime of end devices. Fading channels with time-varying constraints limit this procedure. We propose a joint deep reinforcement learning based framework that effectively initiates the worthy offloading solution with optimal WPT duration. To solve the joint optimization non-convex problem, we decompose the original problem into two sub-problems including the selection of an efficient offloading solution and allocation of optimal WPT duration. We consider orthogonal frequency division multiple access (OFDMA) as a channel access method. As the hyperparameters of a neural network largely influence its performance, we also simulate and analyze different variations of these parameters for system performance. Furthermore, we also reduce the overhead of task partitioning which usually involves in partial offloading schemes with DNNs. Simulation results show the effectiveness and applicability of the proposed framework in dynamic networks. For example, the proposed framework provides an average execution latency of 372.3 ms per channel in a 30-user network, making the system truly applicable in large scale dynamic networks. © (2023), (Korea Information Processing Society). All Rights Reserved.