The increasing complexity of regulatory frameworks across industries presents significant challenges in ensuring compliance, especially as legal and ethical guidelines evolve rapidly. A novel approach to enhancing automated systems' ability to generate legally and ethically sound responses is presented, focusing on the modification of GPT-Neo to incorporate real-time, compliance-driven information retrieval. Through the integration of an external retrieval system, the model dynamically accesses domain-specific legal documents and generates responses that align with complex regulatory requirements. Key metrics such as precision, recall, and compliance score demonstrate significant improvements in the model's capacity to handle diverse compliance scenarios across healthcare, finance, and data privacy domains. The system's ability to retrieve and apply relevant documents during the response generation process allows for a high level of adaptability in meeting evolving legal standards, offering promising implications for compliance automation in highly regulated industries. The findings of this study highlight the effectiveness of combining information retrieval with language model architectures, achieving substantial advancements in accuracy and coherence for compliance-sensitive tasks.