Horses and breeders need to be safe on the farm and the riding club. On account of the great value of the horse, the breeder needs to protect it from theft and disease. In this context, it is important to detect and to recognize the identity of each horse for security reasons. In fact, this paper proposes a Smart Riding Club Biometric System (SRCBS) consisting in automatically detecting and recognizing horses as well as humans. The proposed system is based on the facial biometrics for a horse and the gait biometrics for a human due to their simplicity and intuitiveness in an uncontrolled environment. The present work focuses mainly on horse face detection and recognition. Animal face detection is still extremely difficult given the fact that face textures and shapes are grossly diverse. In addition, recent detectors require a huge dataset for training and represent a huge number of parameters and layers, leading to so much training time. For resolving these problems and also for a useful detection system, this paper proposes a Sparse Neural Network (SNN) based on sparse features for horse face detection. Different global and local features were performed to identify horses and humans for the recognition process. Due to the unavailability of horse databases, this paper presents a new benchmark for horse detection and recognition in order to evaluate our proposed system. This system achieved an average precision equal to 90% for horse face detection and a recognition rate equal to 99.89% for horse face identification.