The development of autonomous vehicles (AVs) has seen rapid advancements over the past decade, driven by significant improvements in artificial intelligence, sensor technologies, and computational power. This survey paper provides a comprehensive review of end-to-end perception and prediction systems for autonomous vehicles, which are critical components for enabling safe and efficient navigation in complex environments. We systematically examine the state-of-the-art approaches, focusing on deep learningbased methods that integrate perception (e.g., object detection, semantic segmentation) with prediction (e.g., trajectory forecasting) in a unified framework. The paper explores the key challenges associated with end-to-end learning, such as data fusion from multiple sensors, the tradeoffs between accuracy and computational efficiency, and the interpretability of learned models. Additionally, we review publicly available datasets and benchmarks that have catalyzed progress in this domain. Finally, we discuss the current limitations and potential future research directions, highlighting the need for more robust, scalable, and generalizable solutions to achieve full autonomy in diverse driving scenarios. This survey aims to serve as a valuable resource for researchers and practitioners working towards the next generation of autonomous vehicle technologies.