In Reinforcement Learning (RL), Monte Carlo methods teach the agents how to make decisions based on the principle of averaging the sample returns. Monte Carlo Reinforcement Learning (MCRL) methods interact with an environment and estimate the value functions of states based on experiences gained from those interactions. Monte Carlo RL methods need to collect many full trajectories (sequences of states, actions, and rewards) for accurate estimation of value functions and for achieving convergence. Even if the state space of the Reinforcement Learning problem is moderately large, the number of trajectories required to estimate value functions accurately can become significantly large. Since the length of each trajectory increases as well with the increase in the size of the state space, collecting and processing each trajectory also becomes computationally expensive and time-consuming. To address this problem, in this work, we proposed a new methodology for making Monte Carlo RL methods more efficient and converge faster by accelerating the learning process. The proposed methodology performs state space abstraction by clustering the state space, which causes a significant reduction in the lengths of trajectories. A rough clustering-based approach is considered for the abstraction of state space. An agent is allowed to explore the environment and collect the trajectories for some number of episodes. Based on the collected trajectories, the preliminary topological structure of the environment is obtained and initial estimates of state value functions are calculated. Using the obtained preliminary topological structure, available estimates of state value functions, group average hierarchical agglomerative clustering is performed and the state space is divided into clusters of states. The initial estimates of state value functions and preliminary topological structure are again used for the reshuffling of states in clustered state space via rough clustering. By performing several iterations of rough clustering, better-quality clusters of states are obtained. Options (macro-actions) for traversing between different clusters of the state space are learnt using dynamic programming techniques. Once the options are learnt, at each time step of Monte Carlo learning, the agent can choose between a primitive action or an option if available. These options significantly accelerate the learning process. Options reduce the lengths of trajectories. Trajectory length reduction results in faster convergence of Monte Carlo methods, and the same is shown with the support of experimental results in this paper.

Recent trends in emerging applications have motivated researchers to design advanced wireless systems to meet their evolving requirements. These emerging applications include digital healthcare, intelligent transportation systems, Industry 5.0, and more. To address the evolving requirements, leveraging a metaverse to empower $6$G wireless systems is a viable solution. A metaverse-empowered $6$G wireless system can offer numerous benefits, but it may also be vulnerable to a wide variety of security attacks. In this survey, we discuss potential security attacks in metaverse-empowered $6$G wireless systems. We introduce an architecture designed to enhance security within metaverse-empowered $6$G wireless systems. This architecture comprises two key spaces: the meta space and the physical space. We present physical space attacks and outline effective solutions to secure metaverse-empowered $6$G wireless systems. We provide invaluable insights and discussions on meta space attacks, along with promising solutions. Finally, we discuss open challenges and provide future recommendations.

A Magnetic field Aided Inertial Navigation System (MAINS) for indoor navigation is proposed in this paper. MAINS leverages an array of magnetometers to measure spatial variations in the magnetic field, which are then used to estimate the displacement and orientation changes of the system, thereby aiding the inertial navigation system (INS). Experiments show that MAINS significantly outperforms the stand-alone INS, demonstrating a remarkable two orders of magnitude reduction in position error. Furthermore, when compared to the state-of-the-art magnetic-field-aided navigation approach, the proposed method exhibits slightly improved horizontal position accuracy. On the other hand, it has noticeably larger vertical error on datasets with large magnetic field variations. However, one of the main advantages of MAINS compared to the state-of-the-art is that it enables flexible sensor configurations. The experimental results show that the position error after 2 minutes of navigation in most cases is less than 3 meters when using an array of 30 magnetometers. Thus, the proposed navigation solution has the potential to solve one of the key challenges faced with current magnetic-field simultaneous localization and mapping (SLAM) solutions â\euro” the very limited allowable length of the exploration phase during which unvisited areas are mapped.

The main goal of Design for Testability (DFT) is to offer a way to test each node in the design (Netlist) for different kinds of defects in the chip. These nodes that are to be checked by using the intended number of patterns, the more thoroughly the design has been tested. Every node in the architecture must be visible(observable) and controlled for this to be achievable. These two concepts may be thought of as the cornerstones of DFT, which must be adhered to in order to get the most test coverage with the fewest possible patterns. This work is applied in the field of IC manufacturing before IC gets fabricated. The parameters like multi-voltages and temperature of IC will be determined with the help of Scan Insertion, ATPG pattern generation, EDT test compression and simulation using mentor graphics tool. In this script we are mainly concentrated on the compression process of the data volume and test time by generating automatic test patterns according to it and simulating.

State-of-the-art object detection methods applied to satellite and drone imagery largely fail to identify small and dense objects. One reason is the high variability of content in the overhead imagery due to the terrestrial region captured and the high variability of acquisition conditions. Another reason is that the number and size of objects in aerial imagery are very different than in the consumer data. In this work, we propose a small object detection pipeline that improves the feature extraction process by spatial pyramid pooling, cross-stage partial networks, heatmap-based region proposal network, and object localization and identification through a novel image difficulty score that adapts the overall focal loss measure based on the image difficulty. Next, we propose novel contrastive learning with progressive domain adaptation to produce domain-invariant features across aerial datasets using local and global components. We show we can alleviate the degradation of object identification in previously unseen datasets. We create a first-ever domain adaptation benchmark using contrastive learning for the object detection task in highly imbalanced satellite datasets with significant domain gaps and dominant small objects from existing satellite benchmarksâ\euro”the proposed method results in up to a 7.4% increase in mAP performance measure over the best state-of-art.Â

Secure storage and sharing of patientsâ\euro™ medical data over the Internet are part of the challenges for emerging healthcare systems. The use of blockchain technology in medical Internet of Things systems can be considered a safe and novel solution to overcome such challenges. Patients with movement disorders require multi-disciplinary management and must continuously receive medical care from a specialist. Due to the increasing costs of face-to-face treatment, especially during the pandemic, patients would highly benefit from remote monitoring and management. The proposed work presents a model for blockchain-based remote management systems for patients with movement disorders, especially those with Parkinsonâ\euro™s disease. The model ensures a high level of integrity and decreases the security risks of medical data sharing.

In this study, we focus on an MCN where the direct links towards a shore BS are not available, due to excessive fading conditions. For this case, we use a UAV swarm to provide improved wireless connectivity, adopting non-orthogonal multiple access (NOMA) for high resource efficiency. In downlink communication, UAVs take into consideration the desired service rate and the channel quality of their links towards the maritime nodes. In the uplink, UAVs employ dynamic decoding ordering to enhance the performance of successive interference cancellation, avoiding fixed ordering of the maritime nodesâ\euro™ signals. Moreover, to ensure highly flexible UAV selection, UAVs have buffers to store data. Performance comparisons show that the UAV swarm-aided MCN enjoys increased average sum-rate by relying on multi-criteria-based interference cancellation and buffer-aided UAVs, over other benchmark schemes in the downlink and uplink.

The paper covers topics ranging from hyperspectral imaging applications to innovative algorithms that have enhanced the analysis of hyperspectral data. We delve into the practical applications, including Agriculture, Food Quality and Safety, Earth Sciences, Exploration and Monitoring, Healthcare, Pharmaceuticals, Medical Imaging, Industrial Manufacturing, Management, Conservation, Safety and Security. Our goal is to provide a holistic understanding of how hyperspectral imaging is transforming these fields and driving new possibilities. We also discussed detailed algorithmic development in Hyperspectral Imaging including Supervised and Unsupervised algorithms. We reviewed algorithmic development from the early 1980â\euro™s to recent developments including Artificial Intelligence involving algorithms from Machine Learning to Deep Learning. We believe our paper will be of particular interest to your readers for those who wish to study algorithm development from simple image processing procedures to AI based approaches and how these algorithms are used in diverse applications to make intelligent decisions for increasing productivity and efficiency in Agriculture, Exploration and Management and identification of diseases and natural resources. Our paper is well organized into various sections as per algorithms and applications. We reviewed more than 250 papers with proper citations and authors’ research work.

This study investigated the utilization of memory forensics and the Large Language Model LLaMA-7B for the purpose of detecting and analyzing contemporary ransomware. It articulates a shift from traditional encryption-focused ransomware attacks to more sophisticated strategies, like data exfiltration, underscoring the evolving nature of these cyber threats. The methodology involves an integrated approach, combining memory forensic techniques with the advanced pattern recognition capabilities of LLaMA-7B, to identify and analyze ransomware signatures within system memory. The results demonstrate the efficacy of this combination in accurately distinguishing between ransomware and benign software, with a particular focus on identifying data exfiltration activities. Discussions revolve around the challenges of keeping pace with the evolving ransomware tactics and the ethical considerations in applying AI in cybersecurity. The study concludes by underscoring the importance of continuous innovation in cybersecurity strategies and the potential of AI integration in developing dynamic defense mechanisms against ransomware.

Given an undirected, unweighted graph with n vertices and m edges, the maximum cut problem is to find a partition of the n vertices into disjoint subsets V1 and V2 such that the number of edges between them is as large as possible. Classically, it is an NP-complete problem, which has potential applications ranging from circuit layout design, statistical physics, computer vision, machine learning and network science to clustering. In this paper, we propose a biomolecular and a quantum algorithm to solve the maximum cut problem for any graph G. The quantum algorithm is inspired by the biomolecular algorithm and has a quadratic speedup over its classical counterparts, where the temporal and spatial complexities are reduced to, respectively, \(O(\sqrt{2^n/r})\) and \(O(m^2)\). With respect to oracle-related quantum algorithms for NP-complete problems, we identify our algorithm as optimal. Furthermore, to justify the feasibility of the proposed algorithm, we successfully solve a typical maximum cut problem for a graph with three vertices and two edges by carrying out experiments on IBM’s quantum simulator.

The escalating sophistication of ransomware has catalyzed the transition from traditional encryption-focused attacks to more pernicious data exfiltration techniques. This research proposes a Decentralized Identity Verification System to combat the amplified threats of data theft and extortion. The system integrates blockchain technology to establish a resilient identity management framework reinforced by multi-factor authentication and smart contract-based access policies. By addressing fundamental vulnerabilities enabling unauthorized data access, the system proactively reduces attack surfaces vulnerable to ransomware infiltration. Simulated attack scenarios reflect promising security enhancement outcomes across metrics including prevention of data breaches, access control efficiency and system responsiveness to threats. While scope remains for continued innovation, this exploration highlights the synergetic potential of cryptography, access governance decentralization for reshaping data protection against contemporary cyber risks.

The transition from 5G to 6G will result in an explosion of Internet of Things (IoT) devices that provide pervasive and constant connection across all spheres of human activities. In this regard, non-terrestrial networks (NTNs) will play a crucial role in supporting and supplementing terrestrial systems in order to cope with such a vast number of IoT devices and to fulfill the massive capacity needs of the most sophisticated of them. This study investigates the implication of several multiple access techniques for 6G enabled NTN-assisted IoT technologies. First, general architecture of NTN, NTN-assisted IoT technologies, and key features and challenges of NTNs are presented. Then, different types of multiple access schemes are discussed and compared in the context of NTN-assisted IoT systems. Simulation results are presented and important performance parameters such as energy efficiency and spectrum efficiency are examined for the discussed multiple access schemes. Furthermore, challenges and future research directions are discussed.

This study explores the efficacy of software simulation in refining control system design, focusing on the implementation of a fuzzy controller within a Ball & Beam system. The proposed fuzzy controller, validated through SIMULINK simulation experiments, demonstrated robust performance in handling disturbances, inaccuracies, and noise. Employing a cascade structure for dual control, managing both beam angle and ball position, enhanced disturbance rejection and overall system performance.

In anatomic pathology laboratories, several processes are still manual. Automated solutions can help standardizing the fabrication/processing of paraffin-embedded tissue blocks (PETBs) for a reliable and more effective diagnosis. Here, we propose a novel and automated strategy to introduce a fiducial marker within PETBs, serving as a reliable reference point. This would assist the clinician in automatically identifying specific regions of biological tissue in paraffin-embedded tissue blocks and paraffin-free tissue slices, in case the patient needs further laboratory analysis on that tissue portion to indicate a tailored treatment (e.g., oncological treatment). The proposed strategy implies the introduction of two automated platforms integrated into the conventional anatomic pathology workflow. The first one, named â\euroœIndexingâ\euro?, involves the insertion of a fiducial marker in tissue-free areas within PETBs. The second one, named â\euroœVirtual marker reconstructionâ\euro?, grounds on image analysis and virtually reconstructs the fiducial marker on the paraffin-free tissue slice. An algorithm that analyzes the similarity of the tissue was also developed to assist the traceability of the biological tissue along its processing from the embedding to the post-staining phase. Together, these platforms could assist the work of anatomopathologists, helping avoiding errors and supporting the final diagnosis in a future automated laboratory.

Federated Learning (FL) enables training Artificial Intelligence (AI) models over end devices without compromising their privacy. As computing tasks are increasingly performed by a combination of cloud, edge, and end devices, FL can benefit from this End-Edge-Cloud Collaboration (EECC) paradigm to achieve collaborative device-scale expansion with real-time access. Although Hierarchical Federated Learning (HFL) supports multi-tier model aggregation suitable for EECC, prior works assume the same model structure on all computing nodes, constraining the model scale by the weakest end devices. To address this issue, we propose Agglomerative Federated Learning (FedAgg), which is a novel EECC-empowered FL framework that allows the trained models from end, edge, to cloud to grow larger in size and stronger in generalization ability. FedAgg recursively organizes computing nodes among all tiers based on Bridge Sample Based Online Distillation Protocol (BSBODP), which enables every pair of parent-child computing nodes to mutually transfer and distill knowledge extracted from generated bridge samples. This design enhances the performance by exploiting the potential of larger models, with privacy constraints of FL and flexibility requirements of EECC both satisfied. Experiments under various settings demonstrate that FedAgg outperforms state-of-the-art methods by an average of 4.53% accuracy gains and remarkable improvements in convergence rate. Our code is available at https://github.com/wuzhiyuan2000/FedAgg.

A novel SiC trench MOSFET with a P-type half-wrapped shield (HW-TMOS) is proposed and analyzed by the TCAD simulator. HW-TMOS could better shield the gate oxide from the high electric field in the off-state by extending the P shield below the gate to the side wall of the trench gate. The breakdown voltage of the device can be enhanced without causing significant degradation of the output characteristics. The peak electric field value of the gate oxide decreases by 64.9% compared with the conventional SiC trench MOSFET, in addition, the breakdown voltage increases by 13.7%. Meanwhile, the capacitance of the HW-TMOS has been optimized by converting the gate-drain capacitance of the trench gate side wall into the gate-source capacitance. The Cgd of HW-TMOS is 67pF/cm2 and the high-frequency FOM of the device is 51 mΩ·nC, which is 80.1% and 36.4% better than the traditional structure. The HW-TMOS has great advantages in reducing switching power due to the low gate-drain capacitance. Overall, the performance of the new structure is significantly improved, which can meet the high requirements of modern electronics for power devices.

Unmanned Aerial Vehicles play a crucial role in various operations especially where human life must be protected. This work presents an adaptable intelligent system suitable to enhance the efficiency and effectiveness of drone swarm operations in a 3D dynamic environment. The system incorporates several modules, including an Ant Colony Optimization (ACO)-based path planning algorithm, collision avoidance mechanism, messaging system, and a hybrid navigation approach, which evaluates the application requirements to decide to prioritize the desired formation of the swarm or the path length and flight time. The proposed system is adaptable and can optimize to several optimization parameters, including solution quality, time consumption, mission completeness, and average divergence. The experiments show that the system consistently provides high-quality paths, achieving around 97% path quality in most cases, and never declines below 90%, even in challenging scenarios. The collision avoidance module ensures 100% mission completeness successfully navigating drones around obstacles and maintaining an optimal path. Moreover, the hybrid navigation approach demonstrates the ability to maintain desired formations while dynamically adapting to obstacles. The systemâ\euro™s performance shows its potential for real-world applications, ensuring efficient and autonomous operations in different missions.

We introduce a new construction method of diffusion layers for Substitution Permutation Network (SPN) structures along with its security proofs. The new method can be used in block ciphers, stream ciphers, hash functions, and sponge constructions. Moreover, we define a new stream cipher mode of operation through a fixed pseudorandom permutation and provide its security proofs in the indistinguishability model. We refer to a stream cipher as a Small Internal State Stream (SISS) cipher if its internal state size is less than twice its key size. There are not many studies about how to design and analyze SISS ciphers due to the criterion on the internal state sizes, resulting from the classical tradeoff attacks. We utilize our new mode and diffusion layer construction to design an SISS cipher having two versions, which we call DIZY. We further provide security analyses and hardware implementations  of DIZY. In terms of area cost, power, and energy consumption, the hardware performance is among the best when compared to some prominent stream ciphers, especially for frame-based encryptions that need frequent initialization. Unlike recent SISS ciphers such as Sprout, Plantlet, LILLE, and Fruit; DIZY does not have a keyed update function, enabling efficient key changing.Â