A Multi-Objective Approach for Test Suite Reduction During Testing of Web Applications: A Search-Based Approach

1. Introduction

Today, the internet is spreading to almost every corner of the globe; the use of web applications is fast becoming a necessity for internet user’s right from search to purchase. To attract more and more users, developers of applications update them frequently for different requirements such as, to give offers on special events/festivals, incorporate innovative ideas, and update services for making the applications more interactive. All such operations have to be performed without interrupting the services provided. Many previously published studies have revealed that the stoppage (or malfunctioning) of services leads to substantial monetary loss and depletion of the customer database.

To validate updates/upgradations, regression testing is exercised with the primary objective of ensuring that recent amendments in the system do not obstruct the functionality of the unchanged parts and do not initiate any new fault. Hence it can be said that regression testing is one of the vital processes exercised during the maintenance phase of the Software Development Life Cycle (Mathur, 2012; Chauhan, 2010; Singh, 2012).

Test case prioritization, Test case selection, and Test suite reduction (also called test suite minimization)are classical strategies followed to implement regression testing; however, the present work primarily focuses on test suite reduction. Rerunning the whole test suite is not an inventive step and that motivates us for a wiser strategy which is test suite reduction. Test suite reduction techniques aim to decrease the cost of regression testing by removing the redundant test cases from the test suite and then obtaining a representative set of test cases that still yield a high level of fault coverage. The motivation of this study is to create a reduced representative set of the original test suite which has to satisfy all the objectives(in multi-objective environment) of the testing criteria without the loss of fault detection capability and to help the testers community during regression testing phase of the evolving software. Rather than running all the test cases of the suite tester can run this representative set according to their objective(out of three considered objectives)and will save an ample amount of time, labor and resources.

The test suite reduction problem can be defined as follows:

• •

  Consider a set of test cases T= {t₁, t_2,…, t_n}, consisting of n elements, known as original test suite(or universal test suite).

• •

  There are a set of testing requirements R = {r₁,r₂,…,r_m}, consisting of m elements, such that each of the test requirements must be covered by at least one of the test cases belonging to T.

• •

  There is a binary relationship between T and R: S={ (t,r)|t satisfies r, t∈T and r∈ R}.

• •

  There are subsets {T₁,T₂,…T_m} of T named test sets where each test set is associated with r_i such that anyone test case(s) belonging to T_i satisfies r_i.

The objective of this research study is to develop a practical test suite reduction approach for regression testing. This process aims to create a subset of T, usually known as representative set T_RS, that can satisfy all the requirements met initially by the universal test suite T. In the case of evolving software, size of the test suite grows to validate new functionalities; during this process, some redundant test cases may be generated. Test suite reduction intends to minimize the size of the test suite and remove unnecessary test cases. This follows specific criteria, without compromising coverage capability (fault coverage capability in this study). Two prior published studies contradict each other regarding the loss of fault detection capability of a reduced test suite(Wong et al., 1999; Elbaum et al., 2003).

Since primarily the presented study focuses on test suite minimization in a multi-objective environment, a formal narration in brief (Yoo and Harman, 2010) about multi-objective test suite minimization is given below. A detailed explanation is also offered later where the description on the NSGA-II algorithm is presented.