We review the transformer architectures that have potentially paved the way toward providing it with a better cognitive ability. This has led these architectures to think and query themselves with higher levels of intelligent questions and provide relevant answers by evolving subject-context relationships in the transformer layers. This is very similar to the cognitive ability of the human mind which can potentially think creatively by asking more relevant questions to the problem at hand which we frequently call “thinking out loud”. These transformer architectures are potentially thought for a more creative and deep thinking approach by doing repeated queries and monitoring generated responses for cognitive behavior. Cognition and thinking for creating better answers in a generative AI system are evolving rapidly. In this article, we are trying to summarize the research conducted and the systems that evolved over time for cognitive thinking (after the basic architecture was proposed).