With the rapid development of Internet of Things (IoT) technologies, traditional disease diagnosis carried out in medical institutions can now be performed remotely at home or even ambient environments, yielding the concept of Internet of Health Things (IoHT). Among the diverse IoHT applications, inertial measurement units (IMUs) based systems play a significant role in the detection of diseases in many fields, such as neurological, musculoskeletal, and mental. However, traditional numerical interpretation methods have proven to be challenging to provide satisfying detection accuracies owing to the low quality of raw data especially under strong electromagnetic interference (EMI). To address this issue, in recent years, machine learning (ML) based techniques have been proposed to smartly map IMU captured data on disease detection and progress. After a decade of development, the combination of IMUs and ML algorithms for assistive disease diagnosis has become a hot topic, with an increasing number of studies reported yearly. To help readers comprehensively understand the fundamentals and state-of-the-art techniques, this article systematically reviews this field, by introducing and explaining relevant application scenarios, discussing challenges, and predicting foreseeable future trends.