Magnetic Resonance Imaging, being a harmless, non-invasive and highly informative modality, has proved to be one of the most widely accepted and used neuroimaging modality for visualizing human brain. Brain MR images possess similar features specially in case of neurodegenerative disorders containing very subtle and intricate changes rendering the diagnosis process very challenging. Manual inspection by experts of such images results in diagnoses based on their expertise, with a probability of misdiagnosis due to subtle changes in such images depending on disease stage, overlapping features, among other factors. In addition, in case of unavailability of experts in remote areas, accurate and timely diagnosis can be a problem. The advent of humongous multi-modal data and Deep Learning techniques has enabled researchers to develop intelligent classification methods with adequate performance accuracies. A review of the literature suggests that a lot of research has been carried out in the direction of automatic diagnosis of neurological disorders, but to date, no consolidated framework has been developed with the capabilities to classify multiple diseases and their sub-types with adequate accuracy from structural and functional MR images of varying types and planes of orientation. The contributions of this research include the design of a unified framework for multiple neurological disease diagnosis resulting in the development of a generic assistive tool for hospitals and neurologists to precisely and briskly diagnose disorders that might result in saving lives in addition to increasing the quality of life of patients suffering from neurodegenerative disorders. To materialize this idea, Deep Learning has been deployed to train a three class model to classify Brain Tumors, Parkinson's disease and normal subjects. A test accuracy of 83.69 has been achieved even with limited dataset used for training, thereby encouraging the idea of a unified framework to diagnose neurodegenerative and other disorders. © 2023 IEEE.