Geometric problem-solving remains a challenging area for artificial intelligence due to the necessity for precise rule application and spatial reasoning. A novel approach is introduced in this research that incorporates rule-based alignment within the architecture of an open-source language model, Llama, to enhance its geometric reasoning capabilities. Through the embedding of explicit geometric rules into the model's neural network, the modified Llama demonstrates improved accuracy and efficiency in solving a wide range of geometric problems, from basic shape recognition to complex theorem application. The study employs a geometry-focused curriculum for training, which progressively increases in complexity, enabling the model to develop a robust understanding of geometric principles. Experimental results, compared with a baseline model, reveal significant improvements in problem-solving accuracy, consistency, and adherence to geometric rules, highlighting the efficacy of the rule-based alignment strategy. The findings suggest that integrating structured knowledge into language models can lead to substantial advancements in their ability to perform specialized mathematical tasks, thereby broadening the scope of applications for artificial intelligence in scientific and technical domains.