Effective fault diagnosis methods for dynamic systems are crucial in various industrial applications. As systems become increasingly complex, traditional diagnostic frameworks exhibit certain limitations, particularly in handling changes in working conditions, unknown faults, and performance degradation. With the advancement of artificial intelligence technologies, fault diagnosis methods inspired by incremental learning have received increasing attention over the past decade and are considered highly promising. The primary objective of this paper is to systematically review, analyze, and discuss existing research in this field. Firstly, basic concepts and technical taxonomies are briefly introduced. Subsequently, recent developments are reviewed from two perspectives, providing a more comprehensive understanding of the state-of-the-art advancements. Key evaluation metrics and representative application scenarios are also discussed. Finally, the analysis of main challenges and prospects is thoroughly provided.