With organizations seeking faster, cheaper, and smarter ways of delivering higher quality software, many are looking towards generative artificial intelligence (AI) to drive efficiencies and innovation throughout the software development lifecycle. However, generative AI can suffer from several fundamental issues, including a lack of traceability in concept generation and decision making, the potential for making incorrect inferences (hallucinations), shortcomings in response quality, and bias. Quality engineering (QE) has long been utilized to enable more efficient and effective delivery of higher quality software. A core aspect of QE is adopting quality models to support various lifecycle practices, including requirements definition, quality risk assessments and testing. In this position paper, we introduce the application of QE to AI systems, consider shortcomings in existing AI quality models from the International Organization for Standardization (ISO), and propose extensions to ISO models based on the results of a survey. We also reflect on skills that IT graduates may need in the future, to support delivery of better-quality AI.

Large language models (LLMs) have been touted to enable increased productivity in many areas of today’s work life. Scientific research as an area of work is no exception: the potential of LLM-based tools to assist in the daily work of scientists has become a highly discussed topic across disciplines. However, we are only at the very onset of this subject of study. It is still unclear how the potential of LLMs will materialise in research practice. With this study, we give first empirical evidence on the use of LLMs in the research process. We have investigated a set of use cases for LLM-based tools in scientific research, and conducted a first study to assess to which degree current tools are helpful. In this paper we report specifically on use cases related to software engineering; specifically, on generating application code and developing scripts for data analytics and visualisation. While we studied seemingly simple use cases, results across tools differ significantly. Our results highlight the promise of LLM-based tools in general, yet we also observe various issues, particularly regarding the integrity of the output these tools provide.

Finding software developers with expertise in specific technologies that align with industry domains is an increasingly critical requirement. However, due to the ever-changing nature of the technology industry, locating these professionals has become a significant challenge for companies and institutions. This research presents a comprehensive overview of studies exploring suitable recommendation systems that can assist companies in addressing this pressing need. To conduct this study, we employ a hybrid systematic mapping approach with an initial number of 1251 studies and a final selection of 21 studies. Our work focuses on collecting data on key technologies, methodologies, and data sets utilized in proposed recommendation systems, to design a new recommendation system that can effectively identify specialists capable of aligning specific technical knowledge with industry domains. The outcomes of this study include insights into the current research trends in this field, alongside a practical overview of considerations necessary for developing a recommendation system that successfully meets the criteria for aligning technical skills with industry domains. By following a hybrid Systematic Mapping methodology and presenting the outcomes in the form of insights, this research addresses the challenge of finding software developers with domain-specific expertise in a rapidly changing technology industry, laying the groundwork for aligning technical skills with industry domains.

Testing is one of the most time-consuming and unpredictable processes within the software development life cycle. As a result, many Test Case Optimisation (TCO) techniques have been proposed to make this process more scalable. Object Constraint Language (OCL) was initially introduced as a constraint language to provide additional details to UML models. However, as OCL continues to evolve, an increasing number of systems are being expressed by this language. Despite this growth, a noticeable research gap exists for the testing of systems whose specifications are expressed in OCL. In our previous work, we verified the effectiveness and efficiency of performing the Test Case Prioritisation (TCP) process for these systems. In this study, we extend our previous work by integrating the Test Case Minimisation (TCM) process to determine whether TCM can also benefit the testing process under the context of OCL. The evaluation of TCO approaches often relies on well-established metrics such as the Average Percentage of Fault Detection (APFD). However, the suitability of APFD for Model-Based Testing (MBT) is not ideal. This paper addresses this limitation by proposing a modification to the APFD metric to enhance its viability for MBT scenarios. We conducted four case studies to evaluate the feasibility of integrating the TCM and TCP processes in our proposed approach. In these studies, we applied the multi-objective optimisation algorithm NSGA-II and the genetic algorithm independently to the TCM and TCP processes. The objective was to assess the effectiveness and efficiency of combining TCM and TCP in enhancing the testing phase. Through experimental analysis, the results highlight the benefits of integrating TCM and TCP in the context of OCL-based testing, providing valuable insights for practitioners and researchers aiming to optimise their testing efforts. Specifically, the main contributions of this work include: 1). We introduce the integration of the TCM process into the TCO process for systems expressed by OCL. This integration benefits the testing process further by reducing redundant test cases while ensuring sufficient coverage. 2). We comprehensively analyse the limitations associated with the commonly used metric, APFD, and then a modified version of the APFD metric has been proposed to overcome these weaknesses. 3). We systematically evaluate the effectiveness and efficiency of OCL-based TCO processes on four real-world case studies with different complexities.

Deep learning (DL) techniques have grown in leaps and bounds in both academia and industry over the past few years. Despite the growth of DL projects, there has been little study on how DL projects evolve, whether maintainers in this domain encounter a dramatic increase in workload, and whether or not existing maintainers can guarantee the sustained development of projects. To address this gap, we perform an empirical study to investigate the sustainability of DL projects, understand maintainers’ workloads and workloads growth in DL projects, and compare them with traditional OSS projects. In this regard, we first investigate how DL projects grow, then, understand maintainers’ workload in DL projects, and explore the workload growth of maintainers as DL projects evolve. After that, we mine the relationships between maintainers’ activities and the sustainability of DL projects. Eventually, we compare it with traditional OSS projects. Our study unveils that although DL projects show increasing trends in most activities, maintainers’ workloads present a decreasing trend. Meanwhile, the proportion of workload maintainers conducted in DL projects is significantly lower than in traditional OSS projects. Moreover, there are positive and moderate correlations between the sustainability of DL projects and the number of maintainers’ releases, pushes, and merged pull requests. Our findings shed lights that help understand maintainers’ workload and growth trends in DL and traditional OSS projects, and also highlight actionable directions for organizations, maintainers, and researchers.

Bug localization, which is used to help programmers identify the location of bugs in source code, is an essential task in software development. Researchers have already made efforts to harness the powerful deep learning (DL) techniques to automate it. However, training bug localization model is usually challenging because it requires a large quantity of data labeled with the bug’s exact location, which is difficult and time-consuming to collect. By contrast, obtaining bug detection data with binary labels of whether there is a bug in the source code is much simpler. This paper proposes a WEakly supervised bug LocaLization (WELL) method, which only uses the bug detection data with binary labels to train a bug localization model. With CodeBERT finetuned on the buggy-or-not binary labeled data, WELL can address bug localization in a weakly supervised manner. The evaluations on three method-level synthetic datasets and one file-level real-world dataset show that WELL is significantly better than the existing SOTA model in typical bug localization tasks such as variable misuse and other programming bugs.

Cybersecurity is an ongoing and critical concern because of the constant and persistent threats from malicious actors such as hackers and crackers. The widespread use of software systems has revolutionized modern society in various aspects, but it has also brought forth new challenges in safeguarding sensitive and confidential information with the evolution of information and communication technology (ICT). Quantifying security measures can provide evidence to support decision-making in software security, especially when it comes to evaluating the security performance of software systems. This involves understanding the main quality criteria of security metrics, which can aid in building security metrology models based on practical requirements. To further explore this topic, this study conducted a systematic literature review of security metrics and measures in the context of Secure Software Development (SSD). The study selected 61 research studies based on specific inclusion and exclusion criteria and extracted data from the selected articles. The study identified 215 software security metrics, which were then categorized based on Software Development Life Cycle (SDLC) phases. To evaluate the effectiveness of the most commonly cited metrics in each phase, the study applied a SWOT analysis to highlight their strengths, weaknesses, opportunities, and threats. The findings of this study offer valuable guidance to diligent and motivated researchers to investigate emerging research trends and address existing gaps in Secure Software Development. Furthermore, this investigation provides software professionals with a more comprehensive understanding of security measurements, constraints, and open-ended specific and general issues.

On June 26-28, 2020, the International Conference on Software and Systems Processes (ICSSP 2020) and the International Conference on Global Software Engineering (ICGSE 2020) were held in virtual settings during the first year of the COVID pandemic. Several submissions to the joint event have been selected for inclusion in this special issue, focusing on impactful and timely contributions to Machine Learning (ML). At present, many in our field are enthusiastic about the potential of ML, yet some risks should not be casually overlooked or summarily dismissed. Each ML implementation is subtly different from any other implementation, and the risk profile varies greatly based on the approach adopted and the implementation context. The ICSSP/ICGSE 2020 Program Committees have encouraged submissions that explore the risks and benefits associated with ML so that the important discussion regarding ML efficacy and advocacy can be further elaborated. Four contributions have been included in this special issue.

The extit{instanceof} pattern matching is periodically previewed in the latest published JDK by removing the redundant casting and optimizing its usage in different scenarios, which improves the code quality and readability. However, the existing integrated development environment (IDE) does not provide sufficient support for refactoring of extit{instanceof} pattern matching in all situations. This paper first identifies several cases that cannot be handled by existing IDEs, and then proposes a novel approach called extit{ReInstancer} to refactor extit{instanceof} pattern matching automatically. extit{ReInstancer} conducts visitor pattern analysis for extit{if} statement blocks with extit{instanceof}. For the extit{instanceof} expression, a pattern variable is obtained after removing the redundant type casting by pattern analysis. For multi-branch statements, extit{ReInstancer} employs class hierarchy analysis and control flow analysis to infer the relationship of instance type in each branch and then optimize these branches. Finally, the multi-branch statements are converted into switch statements or switch expressions. extit{ReInstancer} is evaluated with 20 real-world applications. The experimental results demonstrate a total of 3558 extit{instanceof} expressions and 228 multi-branch statements applying the refactoring, and each application takes an average of 10.8s. extit{ReInstancer} not only reduces the redundant type casting but also improves the code quality.

The Physics of Notations (PoN) is a framework to design cognitively effective notations. The term cognitive effectiveness here refers to the ease (speed and accuracy) by which model readers read the models. In 2019 a PoN-Enabled notation for use case diagrams was proposed. It has been empirically proven that the cognitive effectiveness of this new notation is superior to the original use case notation. The new notation however relies in part on the use of color. The accessibility of this new design however has not been validated for use by the color-blind community, or any users who need to use a greyscale version of the models created. The PoN framework was not designed to explicitly account for the color-blind community or users who can only access greyscale versions of the models. Inclusivity should not be an afterthought in software engineering practice or research. To this end, this paper presents an empirical evaluation of the cognitive effectiveness of the PoN-enabled use case notation in the situation it was viewed in greyscale.

Developers spend significant time locating and fixing bugs, which is often performed manually. Although spectrum-based fault localization (SFL) techniques aim at helping developers to locate faults, they are not yet used in practice. Recent studies have investigated how developers use SFL, presenting different conclusions about their effectiveness and usefulness. We carried out a user study to further enhance the understanding of SFL. We assessed whether SFL improves the developers’ performance and to what extent SFL leads developers to inspect faulty code excerpts. We also investigated the intention of the developers to use SFL and how they interact with SFL. Twenty-six participants performed debugging tasks using real programs, with and without using the Jaguar SFL tool. Using SFL, more developers located and fixed the bugs. SFL also led more developers to inspect the faulty code and locate the faulty method. However, they did not spend less time locating the faults. SFL was well accepted by the participants, who showed intention to use it in their daily activities. Our results indicate that SFL is useful even when the fault is not ranked among the first positions, leading developers to reach faulty code regions and find the bugs.

To enable the people involved in a software development process to communicate and reason close to their area of knowledge, we are investigating a method to formalize and integrate knowledge of multiple domains into domain models and into specifications in terms of those domain models. For this purpose, we have previously defined a set of method objectives, and an initial version of the method –called MuDForM. This paper reports on the methodical support for using a domain model as terminology to define other specifications, and feature specifications in particular. We performed a case study to validate how well the method helps in the specification of processes and to realize the case-specific objectives of the customer. The case study pertains to the formalization of the ISO26262 standard for functional safety in the automotive domain. We found that our method is suitable to systematically formalize a process that is described in natural language, such that there is a clear separation of domain-specific concepts, unambiguous process specifications, and concepts from outside the domain and process of interest. We have extended our method with concepts, steps, and guidelines for grammatical analysis, for the formalization of constraints, and for the specification of processes. The case-specific results are the unambiguous specification of a part of the ISO26262 processes.