Using data augmentation techniques, unsupervised representation learning methods extract features from data by training artificial neural networks to recognize that different views of an object are just different instances of the same object. We extend current unsupervised representation learning methods to networks that can self-organize data representations into two-dimensional (2D) maps. The proposed method combines ideas from Kohonen’s original self-organizing maps (SOM) and recent development in unsupervised representation learning. A ResNet backbone with an added 2D Softmax output layer is used to organize the data representations. A new loss function with linear complexity is proposed to enforce SOM requirements of winner-take-all (WTA) and competition between neurons while explicitly avoiding collapse into trivial solutions. We show that enforcing SOM topological neighborhood requirement can be achieved by a fixed radial convolution at the 2D output layer without having to resort to actual radial activation functions which prevented the original SOM algorithm from being extended to nowadays neural network architectures. We demonstrate that when combined with data augmentation techniques, self-organization is a simple emergent property of the 2D output layer because of neighborhood recruitment combined with WTA competition between neurons. The proposed methodology is demonstrated on SVHN and CIFAR10 data sets. The proposed algorithm is the first end-to-end unsupervised learning method that combines data self-organization and visualization as integral parts of unsupervised representation learning.

The given work is devoted to designing and implementing different dynamic offset cancellation techniques for 50 nm technology CMOS operational amplifiers. The goal is to minimize or get rid of the effects of the offset voltage. Offset voltage exists in all differential amplifiers due to the fact that no pair of transistors can be fabricated with the same size, there is always a slight difference in their dimensions (length or width), this gives rise to an undesirable effect called offset, the value of offset voltage for cheap commercial amplifiers are in the range of 1 to 10 mV, de-spite the fact that this isn’t a significant value, due to the high gain of such amplifiers, this voltage is amplified by tens or hundreds of times, this results in clipping of the output signal and this further limits the amplifier’s maximum allowable input voltage within the given dynamic range, hence its of great importance to take this small voltage into consideration, low-offset amplifiers find applications in mixers, analog to digital converters, instrumentation devices, etc. In this thesis, by using two different techniques for removing offset voltage (chopping and auto-zeroing), five low offset operational amplifiers were designed. The implemented methods reduced the flicker noise by more than 457 times (from 9.4 nV/√Hz to 20 pV/√Hz) at 1 Hz. All the simulations were done using Cadence Virtuoso.

With the increasing complexity of simulation studies, and thus increasing complexity of simulation experiments, there is a high demand for better support for their conduction. Recently, model-driven approaches have been explored for facilitating the specification, execution, and reproducibility of simulation experiments. However, a more general approach that is suited for a variety of modeling and simulation areas, experiment types, and tools, which also allows for further automation, is still missing. Therefore, we present a novel model-driven engineering (MDE) framework for simulation studies that extends the state-of-the-art by means for knowledge sharing across domains, increased productivity and quality of complex simulation experiments, as well as reusability and automation. We demonstrate the practicality of our approach using case studies from three different fields of simulation (stochastic discrete-event simulation of a cell signaling pathway, virtual prototyping of a neurostimulator, and finite element analysis of electric fields), and various experiment types (global sensitivity analysis, time course analysis, and convergence testing). The proposed framework can be the starting point for further automation of simulation experiments, and therefore can assist in conducting simulation studies in a more systematic and effective manner. For example, based on this MDE framework, approaches for automatically selecting and parametrizing experimentation methods, or for planning following activities depending on the context of the simulation study, could be developed.

Our manuscript proposing a deep learning approach for egocentric upper limb segmentation in unconstrained real-life scenarios and a video demo of our work.

The implementation of the Internet of Things (IoT) is mostly done through cellular networks which do not cover the whole world. In addition, the explosive growth of global Internet access demand introduces the need for integrating satellites with cellular IoT networks for coverage extension and backhaul offloading. Operating hybrid satellite-IoT (SIoT) networks, however, might incur excessive service latency and high satellite bandwidth consumption. To tackle these issues, edge caching technology has been considered in SIoT. This article reviews existing research on edge caching-based SIoT networks with illustrative performance evaluation. Various caching design criteria with a focus on two-tier cache-enabled SIoT are discussed. In addition, open research problems on edge caching in SIoT are identified as future research directions and opportunities.

Maintaining the balance between electricity production and consumption is an essential task in the operations of modern power grids. In recent years, battery energy storage system (BESS) has been gaining more and more attention owning to its decreasing capital cost, high flexibility and short response time. However, there exist several technical challenges in developing accurate models and effective algorithms for the operations of BESS. One major challenge is that in order to precisely describe the changing dynamics of battery status, usually highly non-linear functions and integer variables must be used in the model, leading to optimization models with nonlinear and non-convex objective function/constraints that are difficult to handle. To address the above challenges, in this paper we first explore the physical law in the battery charging and discharging process to develop a new model which naturally addresses both the charging and discharging processes through a single current decision variable. Then we propose to approximate the dependency relationship between the open circuit voltage (OCV) and the state of charge (SOC) by some linear functions. This leads to a new non-convex quadratic programming model with linear and bilinear constraints (BLCQP) for the identification of the optimal operational strategy for the BESS. To cope with the bilinear constraints in the BLCQP, we introduce a novel transformation technique to transfer the original BLCQP into another equivalent exponential optimization problem with linear constraints (LCEO). A new sequential linear and quadratic programming approach (SLQP) for the LCEO is developed and its convergence is established. Preliminary experiments are conducted to demonstrate the efficacy of the new model and the efficiency of the new algorithm.

The focus of this paper is to provide an authentic approach to solving bi-objective optimization problems. The target problem is a novel extension of a multi-period $p$-mobile hub location problem, which takes into account the impact of the traveling time at the hubs’ network, the time spent at each hub for processing the flows, and the delay caused by congestion at hubs with specific capacities. We first develop a mixed-integer mathematical model corresponding to the context problem. Afterward, a hybrid meta-heuristic algorithm will be proposed to solve the unveiled model that operates based on simultaneously employing a novel evaluation procedure, a clustering technique, and a genetic approach. The experiments validate that the proposed algorithm performs significantly better than several state-of-the-art algorithms. Furthermore, the decisive effect of two considerable factors: congestion and service time, are also analyzed.

Textbook Question Answering (TQA) is the task of answering diagram and non-diagram questions given large multi-modal contexts consisting of abundant text and diagrams. Deep text understandings and effective learning of diagram semantics are important for this task due to its specificity. In this paper, we propose a Weakly Supervised learning method for TQA (WSTQ), which regards the incompletely accurate results of essential intermediate procedures for this task as supervision to develop Text Matching (TM) and Relation Detection (RD) tasks and then employs the tasks to motivate itself to learn strong text comprehension and excellent diagram semantics respectively. Specifically, we apply the result of text retrieval to build positive as well as negative text pairs. In order to learn deep text understandings, we first pre-train the text understanding module of WSTQ on TM and then fine-tune it on TQA. We build positive as well as negative relation pairs by checking whether there is any overlap between the items/regions detected from diagrams using object detection. The RD task forces our method to learn the relationships between regions, which are crucial to express the diagram semantics. We train WSTQ on RD and TQA simultaneously, \emph{i.e.}, multitask learning, to obtain effective diagram semantics and then improve the TQA performance. Extensive experiments are carried out on CK12-QA and AI2D to verify the effectiveness of WSTQ. Experimental results show that our method achieves significant accuracy improvements of $5.02\%$ and $4.12\%$ on test splits of the above datasets respectively than the current state-of-the-art baseline. We have released our code on \url{https://github.com/dr-majie/WSTQ}.

Abstract---An increasing number of industrial assets are equipped with IoT sensor platforms and the industry now expects data-driven maintenance strategies with minimal deployment costs. However, gathering labeled training data for supervised tasks such as anomaly detection is costly and often difficult to implement in operational environments. Therefore, this work aims to design and implement a solution that reduces the required amount of data for training anomaly classification models on time series sensor data and thereby brings down the overall deployment effort of IoT anomaly detection sensors. We set up several in-lab experiments using three peristaltic pumps and investigated approaches for transferring trained anomaly detection models across assets of the same type. Our experiments achieved promising effectiveness and provide initial evidence that transfer learning could be a suitable strategy for using pretrained anomaly classification models across industrial assets of the same type with minimal prior labeling and training effort. This work could serve as a starting point for more general, pretrained sensor data embeddings, applicable to a wide range of assets.

As the use of unmanned aerial vehicles (UAVs) becomes ever more widespread there is a growing need to develop traffic management and flight rules, in particular for autonomous UAVs or where the predicted traffic densities far exceed those of traditional manned aviation. Inspired by the current rules of the air and multi-agent systems (e.g., pedestrians and swarm robotics) we outline a set of flight rules for autonomous UAVs that consist of waypoint following and conflict avoidance schemes. These flight rules are then explored in small,pairwise simulations and thus refined to allow a UAV to choose from three potential avoidance behaviors based on it and its neighbors velocities and positions. Finally we compare the original and modified flight rules in larger scale simulations modelling two streams of UAV traffic crossing at a point. We show that the modified rules significantly reduce the mean transit time by reducing the impact of UAVs avoiding other UAVs from the same stream.

A method to assess the performance of control loops, based on closed-loop system identification is proposed. This method allows to take into account the trade-off between process variable and manipulated variable energy, thus over-coming one of the most important criticisms to Harris’ index. To illustrate the proposed approach, a numerical example is given, for which the proposed index is 8% greater than Harris’ index.

The paper deals with the subject of the prediction of useful energy during the cycling of a lithium-ion cell (LIC), using machine learning-based techniques. It was demonstrated that depending on the combination of cycling parameters, the useful energy (RUEc) that can be transfered during a full cycle is variable, and also three different types of evolution of changes in RUEc were identified. The paper presents a new non-parametric RUEc prediction model based on Gaussian process regression. It was proven that the proposed methodology enables the RUEc prediction for LICs discharged, above the depth of discharge, at a level of 70% with an acceptable error, which is confirmed for new load profiles. Furthermore, techniques associated with explainable artificial intelligence were applied, for the first time, to determine the significance of model input parameters – the variable importance method – and to determine the quantitative effect of individual model parameters (their reciprocal interaction) on RUEc – the accumulated local effects model of the first and second order. Not only is the RUEc prediction methodology presented in the paper characterised by high prediction accuracy when using small learning datasets, but it also shows high application potential in all kinds of battery management systems.

We present a hybrid analog/digital computing circuit to solve a selective harmonic minimization problem. The approach leverages favorable attributes of digital and analog controllers to yield a fast and scalable optimization solver. A digital microcontroller programs the cost function and other user-defined inputs to the optimization. Voltages in the circuit represent switching angles in the optimization problem. In steady state, the voltages converge to Karush–Kuhn–Tucker (KKT) points of the problem. We present a specific realization of the computing circuit that solves for eight independent switching angles for a quarter-wave symmetric PWM driven two-level single-phase inverter. Seven undesired harmonics are minimized while retaining control over the modulation index. The proposed computing circuit is verified with simulations and a PCB hardware implementation. The experimental results demonstrate that the proposed circuit can converge to the optimal solution in less than 5.0 ms, which is substantially faster than existing methods and facilitates real-time implementation. Moreover, the steady-state power consumption of the PCB implementation is approximately 750 mW, which is also significantly lower than published methods for comparable applications. The computing circuit is utilized to generate the PWM for a 2 kW single-phase inverter, which validates its feasibility in practical applications.

Smart grid is an essential concept in the transformation of the electricity sector into an intelligent digitalized energy network that can deliver optimal energy from the source to the consumers. Smart grids being self-sufficient systems are constructed through the integration of information, telecommunication, and advanced power technologies with the existing electricity systems. Artificial Intelligence (AI) is an important technology driver in smart grids. The application of AI techniques in smart grid is becoming more apparent because the traditional modelling optimization and control techniques have their own limitations. Machine Learning (ML) being a sub-set of AI enables intelligent decision-making and response to sudden changes in the customer energy demands, unexpected disruption of power supply, sudden variations in renewable energy output or any other catastrophic events in a smart grid. This paper presents the comparison among some of the state-of-the-art ML algorithms for predicting smart grid stability. The dataset that has been selected contains results from simulations of smart grid stability. Enhanced ML algorithms such as Support Vector Machine (SVM), Logistic Regression, K-Nearest Neighbour (KNN), Naïve Bayes (NB), Decision Tree (DT), Random Forest (RF), Stochastic Gradient Descent (SGD) classifier, XGBoost and Gradient Boosting classifiers have been implemented to forecast smart grid stability. A comparative analysis among the different ML models has been performed based on the following evaluation metrics such as accuracy, precision, recall, F1-score, AUC-ROC, and AUC-PR curves. The test results that have been obtained have been quite promising with the XGBoost classifier outperforming all the other models with an accuracy of 97.5%, recall of 98.4%, precision of 97.6%, F1-score of 97.9%, AUC-ROC of 99.8% and AUC-PR of 99.9%.

The improvement of Artificial Intelligence (AI) and Machine Learning (ML) can help radiologists in tumor diagnostics without invasive measures. Magnetic resonance imaging (MRI) is a very useful method for diagnosis of tumors in human brain. In this paper, brain MRI images have been analyzed to detect the regions containing tumors and classify these regions into three different tumor categories: meningioma, glioma, and pituitary. This paper presents the implementation and comparison of various enhanced ML algorithms for the detection and classification of brain tumors. A brain tumor is the growth of abnormal cells in the human brain. Brain tumors can be cancerous or non-cancerous. Cancerous or malignant brain tumors can be life threatening. Hence, detection and classification of brain tumors at an early stage is extremely important. In this paper, enhanced ML algorithms have been implemented to predict the presence or the absence of brain tumors using binary classification and to predict whether a patient has brain tumor or not and if he does, detect the type of brain tumor using multi-class classification. The dataset that has been used to perform the binary classification task comprises of two types of brain MRI images with tumor and without tumor. Here nine ML algorithms namely, Support Vector Machine (SVM), Logistic Regression, K-Nearest Neighbor (KNN), Naïve Bayes (NB), Decision Tree (DT) classifier, Random Forest classifier, XGBoost classifier, Stochastic Gradient Descent (SGD) classifier and Gradient Boosting classifier have been used to classify the MRI images. A comparative analysis of the ML algorithms has been performed based on a few performance metrics such as accuracy, recall, and precision, F1-score, AUC-ROC curve and AUC-PR curve. Gradient Boosting classifier has outperformed all the other algorithms with an accuracy of 92.4%, recall of 94.4%, precision of 85%, F1-score of 89.5%, AUC-ROC of 97.2% and an AUC-PR of 91.4%. To address the multi-class classification problem, four ML algorithms namely, SVM, KNN, Random Forest classifier and XGBoost classifier have been employed. In this case, the dataset that has been used consists of four types of brain MRI images with glioma tumor, meningioma tumor, and pituitary tumor and with no tumor. The performances of the ML algorithms have been compared based on accuracy, recall, precision and the F1-score. XGBoost classifier has surpassed all the other algorithms in terms of accuracy, precision, recall and F1-score. XGBoost has produced an accuracy of 90%, precision of 90%, and recall of 90% and F1-score of 90%.

A new state observer-based current balancing method for Modular Multilevel Converters with Interleaved half-bridge Sub-Modules (ISM-MMC) is presented in this paper. The developed observer allows estimating currents through interleaved half-bridge legs in each submodule of ISM-MMC basing only on arm current and submodule’s capacitor voltage measurements. Then, the interleaved current balancing control uses the estimated currents to reduce the interleaved currents imbalance caused by upstream control actions. This technique minimizes the number of required current sensors in ISM-MMC, thereby reducing the converter’s cost, weight, and volume. Capabilities of the proposed interleaved currents sensorless balancing control has been tested against standard parameter tolerances of the composing passive elements. The feasibility of the proposed method is verified by extensive simulation and experimental tests.

The sudden short-circuit is considered the gold-standard parameter measurement method for wound-field synchronous machines (WFSMs) as it enables the recording of the characteristic quantities in near-to-real conditions. However, the test needs huge pieces of equipment, but even worse, it reduces the lifetime of the electrical components by up to 10 years due to the high winding overhang mechanical forces. The DC-Decay tests are low-power alternatives to obtain the characteristic quantities without damaging the machinery. To allow wider use of this method, there are a couple of challenges left that are tackled by this paper. The two main open challenges are, firstly, the number of measurements needed to reach a particular precision, and secondly, a comparison of the DC-Decay with the sudden short-circuit test to allow the validation against the gold standard. More detailed, this paper explores the main challenges associated with the practical use of the DC decay method, which is a non-conventional and detailed-level approach to characterize WFSMs. We provide replies and recommendations regarding the number of measurements, suggesting the minimum number of recorded tests needed to obtain the equivalent diagram with a given accuracy, which has been further validated with an experimental case study. Moreover, the potential enhancement and precision of the parameter identification algorithm are studied in detail. Finally, the equivalent parameters of the DC decay method are compared to the gold standard, which concludes on what the characterization means in terms of predicting accurate transient short-circuit currents for WFSMs.

We present a data-driven approach for probabilistic wind power forecasting based on conditional normalizing flow~(CNF). In contrast with the existing, this approach is distribution-free (as for non-parametric and quantile-based approaches) and can directly yield continuous probability densities, hence avoiding quantile crossing. It relies on a base distribution and a set of bijective mappings. Both the shape parameters of the base distribution and the bijective mappings are approximated with neural networks. Spline-based conditional normalizing flow is considered owing to its universal approximation capability. Over the training phase, the model sequentially maps input examples onto samples of base distribution, where parameters are estimated through maximum likelihood. To issue probabilistic forecasts, one eventually map samples of the base distribution into samples of a desired distribution. Case studies based on open datasets validate the effectiveness of the proposed model, and allows us to discuss its advantages and caveats with respect to the state of the art. Code will be released upon publication.