Cellular heterogeneity is one of the main hallmarks of cancer, referring to the coexistence of different phenotypes with very distinct biological behaviors in single isolates. Automatically detecting single-cell heterogeneity is therefore critical, and can provide important information on cancer initiation, tumour aggressiveness or drug-related resistance. Here we introduce He2Cl, a novel machinelearning algorithm able to detect single cell heterogeneity from the morpho-dynamic analysis of 2D cell cultures. Built on label-free quantitative time-lapse imaging, He2Cl performs a 2-step clustering algorithm for unsupervised classification of sub-phenotypes within a cell population. He2Cl outperforms state-of-the-art clustering models and allows the simultaneous classification of thousands of cell trajectories from their birth to their division, paving the way towards AI-enhanced microscopy for live-cell analysis.