An initial study of quality assurance techniques for automated water level control systems

. This study aims to formulate requirements for models and programs of cyber-physical systems through an investigation of existing approaches to dynamic verification of control programs. Building on this research, we propose a formal model for dynamic verification of process-oriented control programs in cyber-physical systems. Additionally, our goal is to develop a software package based on the proposed methods and models. The research focuses on hyperprocesses, which involve multiple interacting processes with executable states. The complexity of the models being created and the prevalence of routine operations present challenges in applying formal methods to their verification. Neglecting the verification of process-oriented software poses significant risks in system development. Therefore, addressing this challenge involves research and development of dynamic software verification methods that combine testing and simulation techniques.


Introduction
The current landscape witnesses a pervasive trend of digitalization, encompassing diverse domains of human activity. The widespread integration of cyber-physical systems (CPS) into manufacturing and other industries is recognized as the fourth industrial revolution. Various countries and major corporations are actively driving these advancements, exemplified by Germany's high-tech development strategy, which emphasizes the establishment of "smart" production through a global industrial network powered by the Internet of Things (IoT) and related services [1]. CPS has permeated not only industrial sectors but also user devices like the Internet of Things and embedded systems. The algorithms governing such systems exhibit traits such as significant uncertainty in duration, reliance on environmental events, and dynamic characteristics of the external environment [2]. Consequently, specialized programming tools such as Mathlab, LabVIEW, and others [3] have emerged to address these intricacies.
Given this context, the integration of cyber-physical systems (CPS) into various industries represents a significant advancement in the digitalization landscape. This transformative shift is propelled by the concerted efforts of nations and major corporations, with EU's high-tech development strategy serving as a notable example. The proliferation of CPS in diverse domains has the potential to revolutionize industries by enabling increased automation, enhanced connectivity, and improved decision-making processes. However, with the increasing complexity and interdependencies within CPS, ensuring their reliable and secure operation becomes paramount. Robust algorithm design, thorough risk assessment, and effective verification techniques are crucial to mitigating potential challenges and guaranteeing the safety and reliability of CPS.
Obviously, the ongoing digitalization trend, coupled with the integration of cyberphysical systems, also represents a transformative force driving the AI revolution. This paradigm shift necessitates specialized programming tools and techniques to address the unique challenges posed by AI models. By embracing these advancements and prioritizing robustness and security, industries can harness the full potential of the new models to drive innovation and achieve sustainable growth in an increasingly interconnected world.

Related works
The software of modern CPS has reached such a level of complexity that its verification has become a separate field of study [4]. The current trend for verification of control programs is to integrate dynamic verification methods into the iterative software development process. Approaches to software verification are being actively developed that combine various types of testing (system, modular, regression, integration) with the use of physical or software models of the control object [5]. Another classic approach to dynamic software verification is the method of monitoring system performance during regular running [6]. However, it does not allow the behavior of the software to be tested under different conditions.
On the other hand, the work is also useful for solving a number of problems related to the completeness of software analysis and the identification of high-risk functionalities in it. The most effective methods in terms of vulnerability detection accuracy based on the use of machine learning algorithms. The most common methods include the use of information derived from static analysis. The efficiency of vulnerability detection methods based on dynamic analysis is determined by the possibility of providing high coverage of executable code. Most of existing dynamic analysis methods are based on investigation of a single execution path, which does not provide complete analysis of software with multiple conditions of risky actions [7].
The development of models and methods of verification of control algorithms is interesting not only from the theoretical but also from the practical side. The methods being developed in the field of formal verification of CPS at the current stage involve serious limitations on the complexity of verifiable algorithms, which make these methods difficult to apply to practical problems. One of our objective is to integrate methods for dynamic verification of CPS programs into the iterative software development process using software environmental simulators [8]. One of the key advantages of using software simulators is the ability to adjust physical environmental parameters and to simulate situations that would be impossible or dangerous to verify on a real facility. Synchronizing CPS programs with physical environmental events can lead to longer development times when verifying at a real site. In dynamic verification, the use of a software simulator can minimize these delays. Therefore, dynamic verification of CPS software properties requires modeling: behavior of the physical component of the CPS; behavior of the CPS control algorithm; behavior of the CPS operator; planning of work scenarios on the control object.

Formal analysis
The main problem with prioritizing the analyzed execution paths based only on a preliminary risk assessment of the called system functions is the inaccuracy of the assessment for paths containing a large number of indirect function calls. In order to improve the efficiency of the dynamic path analysis process, a method based on the application of a machine learning algorithm with reinforcement is proposed to accumulate the available information on the called functions and their a posteriori assessment using the proposed risk criteria of the executed operations, taking into account the generated information flows [9].
We have experimentally developed a general scheme for verification of CPS programs, which consists of the following steps: the control algorithm to be verified is implemented into a separate block; the model of physical environment is implemented as a virtual object; verification is performed through creation of test scenarios; the algorithm code is corrected based on verification results [10].  (1) By analyzing the vector Vn, we can gain insights into the performance and compliance of the software with the requirements (R). The weights assigned to the requirements (W) can be utilized to prioritize and assess the impact of each requirement on the overall evaluation.
The verification process is based on the interaction of the program with the virtual object (computer model). Control of work scenarios, simulation of system operator's actions and control of algorithm's reaction are performed automatically. In this case both normal operation of the object and failures of its elements are emulated. The proposed approach for dynamic verification of control algorithms was tested on the task of automating a water level control system in a tank based on available open data [11].
The main problem identified in the research is the limitation of prioritizing execution paths(E) based solely on the preliminary risk assessment(R) of system functions (E → P → R). The challenge arises in accurately assessing paths involving a significant number of indirect function calls (E ∩ C), where the indirect nature of the calls reduces assessment precision. To address this issue and enhance the efficiency of the dynamic path analysis process, our research proposes a novel method leveraging machine learning algorithms with reinforcement (M). This method aims to accumulate information about the called functions (E → F) and their subsequent assessment (F → A) using the proposed risk criteria of executed operations. Crucially, our approach takes into account the generated information flows (E → I).
The utilization of machine learning algorithms with reinforcement offers several advantages. Firstly, it enables the extraction of patterns and insights from the available information, aiding in the prioritization of execution paths. Secondly, the iterative nature of reinforcement learning allows the algorithm to refine its assessment over time based on feedback and accumulated knowledge. This adaptability enhances the accuracy of the risk assessment for paths involving indirect function calls. By incorporating the proposed method into the dynamic path analysis process, we anticipate significant improvements in the efficiency and accuracy of prioritizing execution paths. The combination of machine learning algorithms with reinforcement and the consideration of risk criteria and information flows empowers the analysis to better handle complex scenarios with indirect function calls. Ultimately, this approach contributes to more reliable and robust software assessment in the context of cyber-physical systems.

Experiments
The experiments were conducted on a high-performance server (Windows 10, 64-bit, Intel i5 12400f, 32GB RAM, Nvidia RTX 3060 12GB) to evaluate the processing capacity of incremental verification in handling large dynamic CPS. As a demonstration, we deployed the technique on a water level control system dataset. To capture the system behavior, we generated a script that provides detailed information for each cycle. This includes the list of active processes and their corresponding parameters. The script encompasses data from 2000 cycles, allowing for a comprehensive analysis of system performance. Given that the communication period of the system is 600 ms, the script effectively describes the system's behavior over a duration of 600 seconds. This duration provides a representative and meaningful snapshot of the system's operation, enabling the evaluation of the incremental verification technique's effectiveness in handling real-world scenarios.

Fig1. Water level control system
The chart presented below evaluates the results of the software for a cyber-physical system. It provides insights into the performance of the control algorithms through the verification process. It visualizes the occurrence of passed and failed scenarios based on the interaction of the program with a virtual object, which emulates both normal operation and failures of system elements. The chart allows us to analyze the performance of the control algorithms in different scenarios. The y-axis represents the number of occurrences, while the x-axis represents the range of executed scenarios. The green line represents the passed scenarios, indicating successful execution and desired system behavior. The red line represents the failed scenarios, indicating situations where the system did not respond as expected.
By examining the failed tests, we can gain valuable insights into potential weaknesses or areas of improvement within the control algorithms. The occurrences of failed scenarios can be further analyzed to identify patterns, root causes, or specific elements of the system that require attention. This analysis can guide the refinement of the control algorithms to enhance their robustness, reliability, and effectiveness.

Fig.3. Failed test scenarios analysis
Understanding the reasons behind the failed scenarios is crucial for achieving a verifiable software for cyber-physical systems [12]. It enables the identification of critical edge cases, vulnerabilities, or limitations in the control algorithms. Our analysis focused on distinguishing between occasional and sporadic failures versus consistent and systematic issues [13]. By addressing these shortcomings, the software can be iteratively improved, leading to a more dependable and resilient cyber-physical system that ensures the desired behavior and safety in real-world scenarios.

Discussion
The experimental evaluation conducted on a high-performance server provided valuable insights into the processing capacity of incremental verification in handling large dynamic Cyber-Physical Systems (CPS). By deploying the technique on a water level control system dataset, we aimed to assess the system's behavior and evaluate the effectiveness of the incremental verification approach.
To accurately capture the system's behavior, we generated a comprehensive script that detailed the information for each cycle. This script included the list of active processes and their corresponding parameters, allowing for a thorough analysis of the system's performance. The script encompassed data from 2000 cycles, providing a significant sample size for evaluation purposes. The choice of a 600-second duration for the script was based on the system's communication period of 600 ms. This duration offered a representative and meaningful snapshot of the system's operation, enabling us to assess the incremental verification technique's effectiveness in handling real-world scenarios.
By examining the failed tests, we were able to gain valuable insights into potential weaknesses or areas of improvement within the control algorithms. The occurrence of failed scenarios served as a crucial indicator of the algorithm's performance and highlighted areas that required attention. This analysis allowed us to identify patterns, root causes, or specific elements of the system that needed further refinement. Refining the control algorithms based on these insights became imperative to enhance their robustness, reliability, and overall effectiveness. By addressing the identified weaknesses and areas of improvement, we aimed to optimize the system's performance and ensure its ability to handle complex and dynamic CPS environments. The evaluation of the incremental verification technique's effectiveness was a significant step towards advancing the field of CPS software verification. By successfully handling large dynamic systems and providing insights into algorithm performance, this technique showcased its potential to contribute to the development of reliable and efficient CPS solutions.
The verification process in this study relies on the program's interaction with a virtual object, specifically a computer model. Through automated control of work scenarios, simulation of system operator actions, and monitoring of algorithm responses, the verification process ensures the comprehensive assessment of the program's behavior. This approach encompasses both normal operation and the emulation of failures in system elements. To validate the proposed approach for dynamic verification of control algorithms, the study focused on the automation of a water level control system in a tank. The evaluation utilized available open data, ensuring transparency and reproducibility in the research. By leveraging the virtual object and automated processes, the study successfully demonstrated the effectiveness of the dynamic verification approach. The comprehensive testing on the water level control system showcased the method's capability to assess the performance of control algorithms in real-world scenarios.
However, it is important to acknowledge certain limitations of the study. The evaluation was conducted on a specific water level control system dataset, and the results may not be directly generalizable to other CPS domains. Additionally, while the analysis of failed scenarios provided valuable insights, further investigations are necessary to fully understand the underlying patterns and root causes.

Conclusion
As a conclusion, we propose a formal model for the dynamic verification of CPS control programs on a control object simulator. The experimental results give hope for the effectiveness of the proposed approach and the developed methods, demonstrating both an increase in the quality of verifiable programs and an expansion of functionality relative to the tools previously developed for program analysis.
The findings of this research hold promise for the advancement of CPS software verification. By leveraging the formal model and dynamic verification techniques, the quality and reliability of CPS control programs can be enhanced. The use of a control object simulator provides a realistic environment for evaluation and validation, ensuring the applicability of the proposed approach to real-world CPS scenarios.
Future research directions could include expanding the evaluation to different CPS domains, incorporating additional metrics to assess algorithm performance, and conducting extensive fault injection experiments to validate the robustness of the refined control algorithms. The analysis of failed scenarios guided the refinement of control algorithms, enhancing their robustness, reliability, and effectiveness. The findings of this study contribute to the ongoing efforts in advancing CPS software verification and pave the way for further research in this field.