Increasing the implication of growing data generated by the Internet of Things (IoT) brings the focus toward extracting knowledge from sensors’ raw data. In the current cloud computing architecture, all the IoT raw data is transmitted to the cloud for processing, storage, and control things. Nevertheless, the scenario of sending all raw data to the cloud is inefficient as it wastes the bandwidth and increases the network load. This problem can be solved by Providing IoT Gateway at the edge layer with the required intelligence to gain the Knowledge from raw data to decide to actuate or offload complicated tasks to the cloud. This collaboration between cloud and edge called distributed intelligence. This work highlights the distributed intelligence concept in IoT. It presents a deep investigation of distributed intelligence between cloud and edge layers under IoT architecture, with an emphasis on its vision, applications, and research challenges. This work aims to bring the attention of IoT specialists to distributed intelligence and its role to deduce current IoT challenges such as availability, mobility, energy efficiency, security, scalability, interoperability, and reliability.