A document by Christoph Funda . Click on the document to view its contents.

A document by Christoph Funda . Click on the document to view its contents.

Our goal is to solve the challenge of quantifying the performance of Hardware-in-the-Loop (HIL) computer systems that are used for data re-injection. In such a system, there are multiple queues and a server system operating on a First-In, First-Out (FIFO) basis. Here, the challenge lies in establishing tight bounds on end-to-end delay and system backlog. With that, buffer and pre-buffer time configurations can be optimized. To achieve this, Network Calculus (NC) is chosen as the basic analytical framework. For NC calculations, different techniques of estimating arrival and service curves from measurement data can be used from the literature. We have chosen four of these methods, which can be applied to data sets of industrial Timestamp Logging (TL). However, here the problem is that these conventional methods could produce too large bounds (by factor of 1000 or more) than the measured maximum values. This can lead to an ineffective design of HIL system parameters and inefficient resource usage. In our proposed approach, called local TBASCEM, we introduce a local reverse engineering approach. It derives from the global TBASCEM and relies on linear NC equations for estimating the parameters of arrival and service curves. For test purposes, we imposed constraints on the equation variables and employed non-linear optimization. So that, we achieve tighter bounds on service curves in comparison to four other state-of-the-art methods. Furthermore, TBASCEM in general eases the run-time measurement process. This is done by supporting real-time data acquisition to evaluate and optimize HIL system performance. It also enhances observability for the adaption of the HIL configuration to new sensor data. Efficient performance logging of arrival and service curve parameters and the derivation of tighter bounds in HIL systems make TBASCEM a strong tool for optimizing and monitoring applications in non-hard-real-time environments.

This paper surveys different arrival and service curve estimation methods for basic linear functions from experimental data. We experiment with methods we found in literature and discuss the benefit and deficit of these methods. We propose two own developed simple methods inspired by literature, to estimate rate-latency service curves matching to our use case to analyze a soft real-time streaming system with Network Calculus. We furthermore apply three methods to a novel field for performance evaluation with network calculus: hardware-in-the-loop test benches. The performance evaluation of these test systems and the two estimation methods are our contribution to the field of network calculus applicated to soft real-time streaming systems. The performance metric playback buffer and the minimum needed pre-buffer time are dimensioned with the help of adapted and mathematically proven network calculus solutions for streaming devices from our last paper. Linear network calculus elements are generated in three different ways based on measurements of software processing latencies and network latencies. In the next step, playback buffer backlog and delay bounds are calculated based upon the three service curve estimation methods by applying network calculus to the measurements. These calculated bounds are compared to each other and discussed and furthermore verified by a discrete-event simulation model with trace-driven input and with distribution fits of the processing latencies. The simulation model uses a variation of the workload to prove the system, while being operated in soft real-time operating point, with the mean arrival-rate near the mean service-rate and above the minimum service-rate. We recommend using the delay metrics by NC with a safety-factor to design the pre-buffer time.