Attention maps, a popular heatmap-based explanation method for Visual Question Answering (VQA), are supposed to help users understand the model by highlighting portions of the image/question used by the model to infer answers. However, we see that users are often misled by current attention map visualizations that point to relevant regions despite the model producing an incorrect answer. Hence, we propose Error Maps that clarify the error by highlighting image regions where the model is prone to err. Error maps can indicate when a correctly attended region may be processed incorrectly leading to an incorrect answer, and hence, improve users’ understanding of those cases. To evaluate our new explanations, we further introduce a metric that simulates users’ interpretation of explanations to evaluate their potential helpfulness to understand model correctness. We finally conduct user studies to see that our new explanations help users understand model correctness better than baselines by an expected 30% and that our proxy helpfulness metrics correlate strongly (rho>0.97) with how well users can predict model correctness.

How would you describe the features that a deep learning model composes if you were restricted to measuring observable behaviours? Explainable artificial intelligence (XAI) methods rely on privileged access to model architecture and parameters that is not always feasible for most users, practitioners, and regulators. Inspired by cognitive psychology research on humans, we present a case for measuring response times (RTs) of a forward pass using only the system clock as a technique for XAI. Our method applies to the growing class of models that use input-adaptive dynamic inference and we also extend our approach to standard models that are converted to dynamic inference post hoc. The experimental logic is simple: If the researcher can contrive a stimulus set where variability among input features is tightly controlled, differences in response time for those inputs can be attributed to the way the model composes those features. First, we show that RT is sensitive to difficult, complex features by comparing RTs from ObjectNet and ImageNet. Next, we make specific a priori predictions about RT for abstract features present in the SCEGRAM dataset, where object recognition in humans depends on complex intra-scene object-object relationships. Finally, we show that RT profiles bear specificity for class identity, and therefore the features that define classes. These results cast light on the model’s feature space without opening the black box.

Purpose: The field of cancer nanomedicine has made significant progress, but its clinical translation is impeded by many challenges, such as the difficulty in analysing intracellular anticancer drug release by the nanocarriers due to the lack of suitable tools. Here, we propose the development of a combinatorial imaging and analysis technique to evaluate anticancer drug such as doxorubicin HCl (DOX) released by a nanocarrier inside the HCT116 colon cancer cells and its subsequent intracellular accumulation. Procedure: Fluorescent cell images were captured and subjected to combined image analysis and machine learning based procedures to assess and quantify the delivery and retention rate of DOX inside the cancer cells by multifunctional CNT-DOX-Fe3O4nanocarrier. Results: We show that DOX in HCT116 cells was higher for multifunctional CNT-DOX-Fe3O4nanocarrierthan free DOX, indicating efficient and steady release of DOX as well as superior retentive property of the nanocarrier. Initially (1 h and 4 h) the luminance intensity of DOX in the cell cytoplasm delivered by CNT-DOX-Fe3O4nanocarrier was ~0.34 times and ~0.42 times lesser than that of free DOX delivered normally. However, at 24 h and 48 h post treatment the luminance intensity of DOX for CNT-DOX-Fe3O4nanocarrier was ~1.98 times and 1.92 times higher than that of free DOX. Furthermore, the luminance intensity of DOX for CNT-DOX-Fe3O4in the whole cell was ~1.35 times and ~1.62 times higher than that of free DOX at 24h and 48 h, respectively. Conclusions: The high-throughput nature of our image analysis workflow allowed us to automate the process of DOX retention analysis, and enabled us to devise machine learning-based modeling to predict the percentage of anticancer drug retention in cells. The development of models to automatically quantify and predict intra-cellular drug release in cancer cells could benefit personalized treatments by optimizing the design of nanocarriers.

This letter provides a retrospective analysis of our team’s research performed under the DARPA Explainable Artificial Intelligence (XAI) project. We began by exploring salience maps, English sentences, and lists of feature names for explaining the behavior of deep-learning-based discriminative systems, especially visual question answering systems. We demonstrated limited positive effects from statically presenting explanations along with system answers – for example when teaching people to identify bird species. Many XAI performers were getting better results when users interacted with explanations. This motivated us to evolve the notion of explanation as an interactive medium – usually, between humans and AI systems but sometimes within the software system. We realized that interacting via explanations could enable people to task and adapt ML agents. We added affordances for editing explanations and modified the ML system to act in accordance with the edits to produce an interpretable interface to the agent. Through this interface, editing an explanation can adapt a system’s performance to new, modified purposes. This deep tasking, wherein the agent knows its objective and the explanation for that objective will be critical to enable higher levels of autonomy.

COGLE (COmmon Ground Learning and Explanation) is an explainable artificial intelligence (XAI) system for autonomous drones that deliver supplies in mountainous areas to field units. The drone missions have risks that vary with topography, flight decisions, and mission goals in a simulated environment. Users must determine which AI-controlled drone is better for a mission. Narrative explanations identify the advantages of a drone’s plan (“What?”) and reasons that the better drone is able to do them (“Why?”). Visual explanations highlight risks from obstacles that users may have overlooked (“Where?”). A model induction user study showed that post-decision explanations produced a small effect on the participants’ abilities to identify the better of two imperfect drones and their plans for a mission, but they did not teach participants to judge the multiple success factors in complex missions as well as the AI pilots. In a decision support variation of the task, users would receive pre-decision explanations to help them to decide when to trust the XAI’s decision. In a fielded XAI application, every drone available for a mission may lack some competencies. We created a proof-of-concept demonstration of automatic ways to combine knowledge from multiple imperfect AIs to get better solutions that the individual AIs do not find on their own. This paper reports on the research challenges, technical approach, and findings of the project and also reflects on the multidisciplinary journey that we took.

We present a focused analysis of user studies in explainable artificial intelligence (XAI) entailing qualitative investigation. We draw on social science corpora to suggest ways for improving the rigor of studies where XAI researchers use observations, interviews, focus groups, and/or questionnaires to capture qualitative data. We contextualize the presentation of the XAI papers included in our analysis according to the components of rigor described in the qualitative research literature: 1) underlying theories or frameworks, 2) methodological approaches, 3) data collection methods, and 4) data analysis processes. The results of our analysis support calls from others in the XAI community advocating for collaboration with experts from social disciplines to bolster rigor and effectiveness in user studies.

The ease with which mobile money is used to facilitate cross-border payments presents a global threat to law enforcement in the fight against laundering and terrorist financing. This paper aims to use machine learning classifiers to predict transactions flagged as a fraud in mobile money transfers. Data for this paper came from real-time transactions that stimulate a  well-known mobile transfer fraud scheme. This paper uses logistic regression as the baseline model and compares it with ensembles and gradient descent models. The results indicate that the established logistic regression model did not perform too poorly compared to the other models. The random forest classifier had the most outstanding performance among all measures. The amount of money transferred was the top feature to predict money laundering transactions through mobile money transfers. These findings suggest that more research is needed to improve the logistic regression model. The random forest classifier should be further explored as a potential tool for law enforcement and financial institutions to detect money laundering activities in mobile money transfers.