The angle of the polarization (AOP) and the degree of polarization (DOP) of the scattered skylight are symmetrically distributed concerning the solar meridian. Based on the symmetry of the skylight polarization distribution pattern, this paper proposes a novel skylight orientation sensor consists of a camera, an S-waveplate, and a linear polarizer. The skylight orientation sensor is using the image polarization encoding capability of the S-waveplate and the linear polarizer to convert the skylight polarization information into the image’s symmetry axis extraction, which has the advantages of no resolution loss and instantaneous field of view error. The symmetry axis in the image is consistent with the solar meridian. Therefore, the angle between the solar meridian and the skylight orientation sensor reference axis can be obtained without calculating the polarization information, which is also beneficial for real-time performance. The angle measurement accuracy and uncertainty of the skylight orientation sensor are verified by numerical simulation and outdoor experiments. The results demonstrate that the skylight orientation sensor has good application potential in autonomous navigation.