Recent advances in artificial intelligence (AI), especially convolutional neural networks (CNNs), alongside the digitization of histopathological images, have made the computer-aided diagnosis of breast cancer a reality. However, deep learning-based approaches are computationally expensive and have huge parameters, which makes them less affordable for edge devices. In order to make them affordable for edge devices, the whole classification model needs to be compressed while maintaining accuracy. Providing a low-cost solution for histopathological diagnosis in the recent edge-computing world is of utmost importance. Therefore, in this study, we propose “MobiHisNet,” an efficient and lightweight CNN model for histopathological image classification (HIC) based on MobileNet. MobiHisNet was successfully deployed on a Raspberry Pi, as well as three mobile devices, demonstrating its ability to run on a lightweight and portable processor. Our studies indicated that a depth parameter ( ) and a 16-bit quantization are the optimum parameters for the proposed model while balancing the accuracy, inference time, and memory peak requirements. Compared to the state-of-the-art, pretrained models, MobiHisNet has fewer parameters and calculations, resulting in faster image classification. This renders it more viable for production purposes and applications on edge devices. In addition, MobiHisNet is computationally faster than VGG16, ResNet50, Xception, and InceptionV3 by twenty-seven, eight, six, and five times, respectively. This also outperforms all the baseline models with the moderate model size and FLOP counts. Experiments on breast cancer HIC (BreakHis) data sets show superior performance of MobiHisNet on edge devices in terms of higher accuracy, lesser complexity, and lesser memory requirements. Thus, it has a high potential for deployment in mobile edge devices.