Automated Test Generation for IEC 61131-3 ST
Programs via Dynamic Symbolic Execution
Li Hao
†
, Jianqi Shi
†‡∗
, Ting Su
¶
, Yanhong Huang
†§
†
National Trusted Embedded Software Engineering Technology Research Center
East China Normal University, Shanghai, China
‡
Hardware/software Co-Design Technology and Application Engineering Research Center, Shanghai, China
¶
Nanyang Technological University, Singapore
§
Shanghai Key Laboratory of Trustworthy Computing, Shanghai, China
Abstract—A programmable logic controller (PLC) is essentially
a computer dedicated to industrial control which is widely used
in the field of global automation control. However, PLC software
bugs can result in economic losses and even personal safety issues.
PLC software must be thoroughly tested regarding function,
structure, safety, and other aspects to avoid accidents. Existing
PLC tools are mainly based on the manual setting of input data,
which is not only unable to be well automated but also cannot
provide information about code coverage. This paper presents
an automated test case generation approach for a Structured
Text (ST) language to reduce the cost of testing, using dynamic
symbolic execution. We apply this method to implement the
coverage-based automated test case generation tool STAutoTester.
We have evaluated STAutoTester on 21 programs. The experi-
mental results show that STAutoTester can effectively handle
these programs. For 11 ST programs, STAutoTester reduces, on
average, 87.5% of generated test cases compared to S
YM
PLC.
I. INTRODUCTION
Programmable logic controllers (PLCs) are the core equip-
ment of industrial automation control and have been widely
used in various control systems. PLC software, particularly
used on military hardware, is increasingly demanding in terms
of correctness and security. IEC 61508 is an international
safety standard and applies to many industries. It specifies
techniques for each phase of the life-cycle to prevent errors.
Software testing is an effective means of ensuring the quality
of software products. Unit testing is required in software
testing and depends on the target SIL (Safety Integrity Level)
capabilities of the software being tested. Unit testing also
covers specific criteria such as all statements, branches, or
conditions of the program.
At present, the most popular PLC software testing methods
are testing in the whole digital simulation test environment
and the hardware environment. These methods are mainly
concerned with whether the various needs of the user are met,
such as function, performance, interface, and so on. However,
compared with structured testing, functional testing needs
analysis of requirements to generate test cases and the process
of writing a requirement specification is time-consuming and
is not fully automatic. The method of structured testing is a
white box test including model checking [1], [2], symbolic
execution [3], and so on. Model checking is more suitable
*Corresponding Author: jqshi@sei.ecnu.edu.cn
for checking design errors, as it is a vital but complicated
process to convert to an intermediate formal model higher than
the actual code abstraction level. In addition, model checking
suffers from the state-space explosion problem. Therefore, we
use dynamic symbolic execution techniques to automate test
case generation.
Dynamic symbolic execution (DSE) is a technique that
combines symbolic execution with concrete execution and is
widely used in source code-based automated software testing.
Many automated testing tools based on DSE (e.g. CUTE [4],
KLEE [5], and SmartUnit [6]) have appeared in academia and
industry.
There has been little work on directly applying dynamic
symbolic execution technology to automatic test case gener-
ation of ST, which is one of the IEC 61131-3 PLC standard
languages. One reason is that the CPU of the PLC sequentially
scans the running user program. That is, the PLC program has
the characteristics of cyclic execution. Another reason is that
generating a small number of test cases with high coverage is
not an easy task.
In this paper, we present an automated test case genera-
tion framework for ST using dynamic symbolic execution.
The framework takes the ST program and a search strategy
selection as input and outputs a test case set and coverage
information for the program. First, the ST source program
is preprocessed and parsed to get the abstract syntax tree.
Second, we generate an intermediate model by traversing the
abstract syntax tree. The intermediate model contains infor-
mation such as variables, assignment constraints, execution
condition constraints, and order constraints. Then, the dynamic
symbolic execution engine generates a set of test cases for the
program based on the intermediate model. The Satisfiability
modulo theories (SMT) solver is called multiple times to solve
the constraints. Finally, a test case analysis report is output.
We have designed a general intermediate model named
ST
CFG (Control Flow Graph for ST). ST CFG is a combi-
nation of control flow graphs and PLC language features. The
operation of the DSE engine and the statistics of the test cases
are inseparable from ST
CFG. ST CFG is the unified input
structure of the DSE engine, which keeps the DSE engine
independent of the source program.
Additionally, we propose a test case generation algorithm
200
2019 International Symposium on Theoretical Aspects of Software Engineering (TASE)
978-1-7281-3342-3/19/$31.00 ©2019 IEEE
DOI 10.1109/TASE.2019.00034