A differential derivation method is demonstrated to high-accurately calculate the refractive index curves by solving the mode coupling problems in theoretical calculations. Because of the existence of mode coupling, the finite element method cannot lock the calculator in the interested mode, especially at the crossing part of two refractive index curves. Either imaginary crossing or real crossing will result in an inaccurate calculation of the refractive index curve. The calculation accuracy in the crossing parts is improved by using the differential derivation method, and the effectiveness is verified by calculating the refractive index curves for the HE11, HE12, TE01, and EH11 modes of a custom-designed fiber.