The potential of AI in medication prediction through Electronic Health Records (EHRs) is a subject of substantial interest and ongoing research. This review critically examines the challenges and future directions needed to transform AIbased medication prediction, which is a hype in its current state rather than a practical tool for real-world application. Unlike previous literature that primarily addresses broad AI challenges, this paper focuses on the intricacies of medication prediction. We structure our analysis around three research questions. RQ1 investigates the prevailing and emerging challenges, highlighting the extensive scope for improvement in model interpretability, scalability, transferability, and domain-specific issues like the cold-start problem, drug interactions, personalization, dosage optimization, etc. RQ2 synthesizes insights from existing research, noting that performance, personalization, and domain-knowledge integration are the most frequently tackled problems, with typical metrics used for evaluation. RQ3 explores future developments required to move from theoretical potential to practical implementation, emphasizing the necessity of addressing data standardization, scalability, interpretability, and other domainspecific challenges. We also discuss the implications of these advancements for various stakeholders, including AI practitioners, medical professionals, and interdisciplinary collaborators.