Automated Testing: Higher Efficiency and Improved Quality of Testing Command, Control and Signaling Systems by Automation

Introduction

Railway operation is the procedure to run trains on railway tracks. While the engine driver cannot oversee the full braking distance at higher speed those trains must be protected against passing a signal at danger or overspeeding by an automatic train protection system (ATP). Those safety-related systems in railways must be highly reliable and safe, which means that they cause hazards at most at an extremely low rate. The relevant standards mandate that safety and functionality of systems and components have to be proven. Therefore, they have to be tested comprehensively before being taken into operation. Tests are performed on a railway line or in a specifically equipped laboratory. Increasing the functionality of the systems while keeping the rate of hazards low leads to an increasing number and complexity of those tests. On the other hand the open, worldwide market requires to lower costs and to reduce the time to market. So the challenge is to improve the quality of the tests and reduce redundancy in them while speeding up and reducing cost. There are tests with different aims: they can be used to show that a system fulfils the relevant specification, the foreseen operational tasks and/or safety requirements. All these different tests need to be described and specified to be performed in the field or to be formalized to be executed in a lab. Field and lab testing require different levels of formal definition and description. The ideas presented here aim to use basically the same test case formalization and lab automation for different kinds of lab tests, and to relocate some kinds of field tests to the laboratory.

The specific objectives of this chapter are approaches for automation of different steps in the testing process: specifically tailored approaches can be applied to automate the generation, execution and evaluation of different kinds of lab tests. The generation can be done by e.g. software tools, and checking by web-based evaluations which implement suitable checking automations. The test automation requires real-time interaction, so the suitable approach is based on real-time event triggering and logging software as well as the use of tailored robots for manual inputs. The analysis can be automated by using software for automatic comparison and analysis of the logged data as well as report generators.

This chapter presents ways to improve the test process from automations of test campaigns to conceptual approaches which are still in an experimental stage. First, as an example of automation, it is shown how tests involving a graphical human machine interface can be performed without manual interaction, and how test results can be evaluated mechanically even in complicated cases. These two techniques have already been applied successfully. Following that, a prototype tool for constructing ETCS test sequences is presented, which addresses the problem of parameterizing generic test cases. And lastly, it is discussed how tests can be generated systematically and partly automatic from system specifications given in the form of semi-formal models. A full-scale application of the latter still constitutes a scientific challenge.

The structure of this chapter follows a shell-like approach. The innermost idea to improve the testing effort and duration is to execute tests as far as possible automatically. More or less in the same shell is the automation of the analysis of the test results. The next section discussing this aspect is presenting results on high readiness level. The next outer shell is the automation of the specification and documentation of the test spec. In the associated section a method and a tool are shown, which are available as a prototype. The outermost shell is a complete model-based system and test specification, which is still a scientific challenge. Current results and further work are discussed in the corresponding section.