This paper applies the Design of Experiments approach for detecting detrimental weak emergent behavior of an Autonomous Surface Vessel operating in a dynamic environment on a Search and Rescue mission. The research utilizes Orthogonal Arrays in combination with regression analysis to systematically test the parameter space of an engineered system function. We explored the parameter space of interest and detected where the system model does not comply with a defined Measure of Effectiveness. The findings from this case study suggest that these methods enable a systematic exploration of the system’s parameter space, allowing for effective detection of detrimental weak emergent behavior. This approach potentially enhances test coverage, expands system operating knowledge, and facilitates mitigation efforts more efficiently.

This paper investigated the effect of automation processes in an industrial company engineering complex cyber-physical systems. The authors used industry-as-laboratory as research method, exploring an ongoing development project. The automation efforts focused on four areas, being 1) test setup, 2) test execution, 3) test result analysis, and 4) documentation. All four areas showed promising results on increased effectiveness and/or efficiency. Especially automation of test result analysis will help the industrial company, KONGSBERG, reduce their main bottleneck in the test process, as well as reduce the risk of costly project delays. An automated system integration test process, facilitating iterative regression testing, will leverage the efficiency of the verification test process.