This paper proposes an epidemic model-based approach to inference the incubation period distribution of COVID-19 just utilizing the publicly reported confirmed case number. This method can reduce the biases of traditional survey-based methods, and it can be practiced without the assumption of the shape of the incubation period’s distribution. The most commonly studied metrics (e.g., expectation, median, and 95th percentile) obtained from our estimated incubation period distribution of COVID-19 are in line with current studies’ results. However, due to the mathematical-model-based nature of this method, the inference results are somewhat sensitive to the setting of parameters. Therefore, this method should be practiced reasonably on the basis of a certain understanding of the studied epidemic.