As standardization work is ongoing for the upcoming generation of cellular networking (6G), exploring new technologies for low-latency, high-bandwidth fronthauling technologies becomes a crucial aspect for the overall system performance. Free Space Optics (FSO) has emerged as a light beam-based wireless communication technology that can be fulfill such requirements with the possibility of surpassing certain radio technologies. Consequently, in this paper, we investigate the possibility of employing FSO as a feasible solution for 6G fronthaul networks because of its promising characteristics in providing high bandwidth, low latency, and high flexibility. We review the main features of FSO and its primary use-cases in the 6G fronthaul. Building on these, we highlight the challenges that face the implementation of FSO in a 6G fronthaul environment, such as the influence of atmospheric conditions on signal dependability. In the form of recent solutions and techniques, we enlist possibilities to overcome these challenges. Furthermore, we outline potential future research directions including network slicing and resource allocation strategies for FSO-based 6G fronthaul networks to offer a complete picture. This way our comprehensive analysis aims to provide valuable insights into the prospects of employing FSO in the development of robust and efficient 6G fronthaul networks.