Trust models play an important role in Internet of Things (IoT) as it provides a means of finding whether a given device can provide a service to a satisfactory level as well as a means for identifying potentially malicious devices in the network. Context awareness in trust models allows a trustor to filter and aggregate evidence by their relevance to the current situation. Context awareness is important in the formulation of trust in IoT networks due to their heterogeneity and due to the dynamic changes in the capabilities of IoT devices. In this paper,we have proposed a new type of context-based attack on context aware trust models for IoT systems. An adversary is able to manipulate the context and impact a target group of IoT devices, while other devices in non-targeted groups are not even aware of the attack. We have demonstrated the effectiveness of this new type of attack on six previously proposed trust models. Through practical simulations and theoretical proofs, we show that the adversaries can launch such context-based attacks against a targeted group of IoT devices in the network. The paper also proposes a new trust management system that can mitigate such context-based attacks.