Histopathology for tumor margin assessment is time-consuming and expensive. High-resolution full-field optical coherence tomography (FF-OCT) images fresh tissues rapidly at cellular resolution, facilitating evaluation. We imaged fresh ex vivo skin tissues (normal and neoplastic) from Mohs surgery. FF-OCT features were defined and diagnostic accuracy for malignancies was performed by the two experts OCT readers via a blinded analysis. A convolutional neural network was built to distinguish and outline normal structures and tumors. Of the 113 tissues imaged, 95 (84%) had a tumor (75 BCCs and 17 SCCs). The average reader diagnostic accuracy was 88.1%, a sensitivity of 93.7%, and a specificity of 58.3%. The AI model achieved a diagnostic accuracy of 87.6%±5.9%, sensitivity of 93.2%±2.1%, and specificity of 81.2%±9.2%. A mean intersection-over-union of 60.3%±10.1% was achieved delineating nodular BCC from normal. We envision FF-OCT for rapid evaluation of surgical margins and AI tumor detection leading to widespread technique integration.