In this tutorial, we will leverage artificial intelligence (AI) techniques to facilitate the reverse engineering of an integrated circuit based on scanning electron microspcopy, contributing to hardware assurance. This work encompasses various subjects, including image processing, computer vision, and machine learning, providing a comprehensive learning experience in these specialized domains. We encourage you to actively participate in this tutorial to execute your own project. The skills and knowledge acquired in this activity can bolster your resume, enhancing your prospects when seeking employment, pursuing further education, or advancing in your current professional role.

Abstract-As reinforcement learning agents become more popular in critical applications across both the public and private sectors, the need to reverse engineer, comprehend, and audit their decisions becomes crucial. While there are many traditional explainable AI methods to aid in this endeavor, many approaches assume direct access to the agent model, which is not available in many reverse engineering applications such as in-house recovery of legacy models or analysis of competitor, adversarial, or other external agents. To address this research gap, this paper presents LfRLD, a framework for reverse engineering reinforcement learning (RL) agents using learning from demonstrations (LfD) approaches. Empirical results demonstrated the proposed framework's potential to aid in generalizing, predicting, and summarizing agent behaviors using only observed demonstrations, and revealed many opportunities for future research. Within the wider scope of AI, this paper's findings have many implications for auditable, and therefore, trustworthy AI, aiding applications in a variety of areas such as business and finance, criminal justice, cybersecurity and defense, and Internet of Things (IoT). Impact Statement-Reinforcement learning agents are becoming increasingly popular in a variety of applications such as chatbots, economics, healthcare, and autonomous driving. Their ability to learn consequences through trial-and-error interaction enables them to explore potentially optimal routes with longterm gains despite possible short-term losses. However, over the years, such agents have become increasingly scrutinized as their decisions are seldom interpretable by others. Although recent advances in explainable AI have aided in addressing this challenge, they typically assume access to the agent models, which are often not public. The framework introduced in this paper helps address this research gap. While the proposed technology has shown promising results for reverse-engineering agent behaviors using only observed demonstrations, there are many opportunities for future developments. In addition to enhancing agent auditability and trustworthiness, the proposed work also offers an approach for reverse-engineering agents to gain an intelligence or resource advantage over external competitors or adversaries.

Abstract—Reinforcement learning (RL) has become more popular due to promising results in applications such as chat-bots, healthcare, and autonomous driving. However, one significant challenge in current RL research is the difficulty in understanding which RL algorithms, if any, are practical for a given use case. Few RL algorithms are rigorously tested, and hence understood, for their practical implications. Although there are a number of performance comparisons in literature, many use few environments and do not consider real-world limitations such as run-time and memory usage. Furthermore, many works do not make their code publicly accessible for others to use. This paper addresses this gap by presenting the most comprehensive performance comparison on the practicality of RL algorithms known to date. Specifically, this paper focuses on discrete, model-free deep RL algorithms for their practicality in real-world problems where efficient implementations are necessary. In total, fourteen RL algorithms were trained on twenty-three environments (468 environment instances), which collectively required 224 GB and 766 days CPU time to run all experiments, and 1.7 GB to store all models. Overall, the results indicate several shortcomings in RL algorithms’ exploration efficiency, memory/sample efficiency, and space/time complexity. Based on these shortcomings, numerous opportunities for future works were identified to improve the capabilities of modern algorithms. This paper’s findings will help researchers and practitioners improve and employ RL algorithms in time-sensitive and resource-constrained applications such as economics, cybersecurity, and Internet of Things (IoT). Impact Statement—Reinforcement learning (RL) technologies are commonly used in autonomous driving, chat-bot, and business analytic applications. They learn how to adapt to unforeseen situations, reducing the load on human drivers, support teams, and analysts. Although there are a variety of theoretical works in RL literature, very few algorithms are tested and evaluated to facilitate their use in real-life scenarios. The performance comparison introduced in this paper addresses these limitations. The performance analysis framework, re-implemented source code, and findings identified in this study could increase the adoption and speed development of RL technologies in more real-life applications. Moreover, the open challenges, recommendations, and practical implications identified in this paper could facilitate collaboration and development of new technologies among researchers and practitioners in industry and academia.