With over 18,000 species, butterflies account for nearly one-quarter of all identified species on the planet. The images of different butterfly species can be utilized to train Deep Convolutional Neural Networks (CNNs) for the automatic detection and classification of butterflies. This work proposes an end-to-end system for automatically detecting butterflies in given images and predicting their respective species. To achieve butterfly detection, we utilized the YOLOv3 object detection model, which was trained on the Beautiful Butterflies dataset. This dataset comprises 832 photos of butterflies from 10 different species, captured from various angles. For species classification, we designed a deep convolutional neural network-based architecture named Efficient Convolutional Neural Network (Effi-CNN), employing multiple CNN layers and trained on the custom dataset. To benchmark the performance of Effi-CNN, we compared three versions: Effi-CNN-1, Effi-CNN-2, and Effi-CNN-3, with five other transfer learning CNN models, including VGG16, VGG19, ResNet50, MobileNetV2, and Inception-v3 models. Evaluation of the models was conducted using a separate test dataset. The YOLOv3 object detection model exhibited a promising result, achieving a mean Average Precision (mAP) of 0.98. Among the classification models, Effi-CNN-3 demonstrated the highest accuracy, reaching 98.20. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023.