Medical image classification is not only a complex task but also a challenging one due to the heterogeneous nature of medical data. Deep transfer learning has proven to be a viable technique for medical image classification throughout the years, mostly because it is able to leverage knowledge from pre-trained models learned from large-scale datasets, improved performance, minimal training and overcoming the disadvantage of small data sets. This paper offers a succinct review of the cutting-edge deep transfer learning optimization approaches for medical image classification. The paper begins with an overview of convolutional neural networks (CNN) and transfer learning techniques, such as relation-based, feature, parameter and instance-based transfer learning. Then, the study examines classical classifiers, such as Resnet, VGG, Alexnet, Googlenet, and Inception, and compare their performance on medical image classification tasks. The study also presents optimization techniques, including batch normalization, regularization, and weight initialization, data augmentation and the kernel mathematical formulations. Finally, the study unearths  various challenges that arise when using deep transfer learning for medical image classification as well as potential future approaches for this field.