Publications

Automatic Generation of Search-Based Algorithms Applied to the Feature Testing of Software Product Lines

Published in Proceedings of the 31st Brazilian Symposium on Software Engineering (SBES) , 2017

The selection of products for the variability testing of Feature Models (FMs) is a complex task impacted by many factors. To solve this problem, Multi-Objective Evolutionary Algorithms (MOEAs) have been successfully used in the field known as Search-Based Software Engineering (SBSE). However, the design of a search-based approach is not an easy task for the software engineer, who can find some difficulties such as: the choice and configuration of the best MOEAs, the choice of the best search operators to be implemented, and so on. In addition to this, existing approaches are dependent on the problem domain and do not allow reuse. In this way the use of Hyper-Heuristic (HH) can help to obtain more generic and reusable search-based approaches, and because of this is considered a trend in the SBSE field. Following this trend and to contribute to reduce the software engineer's efforts, this work explores the use of a hyper-heuristic for automatic generation of MOEAs to select test products from the FM, considering three factors: pairwise coverage, mutation score and cost, given by the number of products. The HH is based on a grammar that represents the elements, parameters and components of existing MOEAs and implements evolutionary operators, such as crossover and mutation, suitable for selection problems. In this way, it can be reused for other similar software engineering problems. Evaluation results show that the proposed approach obtains results that are better or statistically equivalent than similar approaches found in the literature.

Recommended citation: Helson L. Jakubovski Filho, Jackson A. Prado Lima, and Silvia R. Vergilion, Automatic Generation of Search-Based Algorithms Applied to the Feature Testing of Software Product Lines, in: Proceedings of the 31st Brazilian Symposium on Software Engineering, SBES, ACM, Maringa, Brazil, 2017, pp. 114-123. https://dl.acm.org/citation.cfm?doid=3131151.3131152

A Multi-objective optimization approach for selection of second order mutant generation strategies

Published in Proceedings of the 2nd Brazilian Symposium on Systematic and Automated Software Testing (SAST), 2017

The use of Higher-Order Mutants (HOMs) presents some advantages concerning the traditional use of First-Order Mutants (FOMs). HOMs can better simulate real and subtle faults, reduce the number of generated mutants and test cases, and so on. However, the HOM space is potentially huge, and an efficient strategy to generate the best HOMs is fundamental. In the literature different strategies were proposed and evaluated, mainly to generate Second-Order Mutants (SOMs), but none has been proved to perform better in different situations. Due to this, the selection of the best strategy is an important task. Most times a lot of experiments need to be conducted. To help the tester in this task and to allow the use of HOMs in practice, this paper proposes a hyper-heuristic approach. Such approach is based on NSGA-II and uses the selection method Choice Function to automatically choose among different Low-Level Heuristics (LLHs), which, in this case, are search-operators related to existing SOM generation strategies. The performance of each LLH is related to some objectives such as the number of SOMs generated, the capacity to capture subtler faults and replace the constituent FOMs. In comparison with existing strategies, our approach obtained better results considering the used objectives, and statistically equivalent results considering mutation score with respect to the FOMs.

Recommended citation: J. A. Prado Lima, S. R. Vergilio, A Multi-objective optimization approach for selection of second order mutant generation strategies, in: Proceedings of the 2nd Brazilian Symposium on Systematic and Automated Software Testing, SAST, ACM, Maringa, Brazil, 2017, pp. 6:1–6:10. https://dl.acm.org/citation.cfm?doid=3128473.3128479

Hyper-Heuristic Based Product Selection for Software Product Line Testing

Published in IEEE Computational Intelligence Magazine, 2017

A Software Product Line (SPL) is defined as a set of software systems that share a common and managed set of features satisfying specific needs of a particular market segment or domain [1]. The SPL offers a number of common artifacts for building products, including mandatory and variable elements. SPL approaches have been adopted by many software companies1 to ease reuse and reduce time and production costs. A feature represents a functionality that is visible to the user and can be designed as a variability, which represents a variable functionality that may or may not be present in a product. On the other hand, mandatory features are common to all SPL products. To facilitate feature management, most SPL methodologies use the Feature Model (FM) [2] to represent all the SPL variabilities and commonalities.

Recommended citation: T. N. Ferreira, J. A. Prado Lima, A. Strickler, J. N. Kuk, S. R. Vergilio, and A. Pozo. 2017. Hyper-heuristic Based Product Selection for Software Product Line Testing. IEEE Computational Intelligence Magazine 12, 2 (May 2017), 34–45. https://ieeexplore.ieee.org/document/7895294/

Evaluating Different Strategies for Reduction of Mutation Testing Costs

Published in Proceedings of the 1st Brazilian Symposium on Systematic and Automated Software Testing (SAST), 2016

Mutation testing presents high efficacy in terms of revealed faults, but with high computational costs, because test cases must be executed against a great number of mutants. To reduce such costs several strategies exist. In general, they select a reduced number of mutants that maintain a great overall mutation score. Recently, Higher-Order Mutation Testing (HOM) was proposed. HOM introduces more than one fault at a time in the mutants such that they are harder to kill. This kind of testing can also be used as a cost reduction strategy. However, few works in literature evaluate the cost reduction potential of HOM or compare it to other existing strategies. In this work, we evaluate HOM as a mutation testing cost reduction strategy. We conducted an experiment to compare four HOM-based strategies (First to Last, Random Mix, Different Operators, and Each-Choice) and three conventional ones (Random Mutant Selection, Selective Mutation, and Search-Based Mutation by using Genetic Algorithm). The analysis considers the number of selected mutants, number of test cases, mutation score and efficiency measures. Selective Mutation presented the best results overall. Among the HOM-based strategies, Each-Choice was the best.

Recommended citation: J. A. Prado Lima, G. Guizzo, S. R. Vergilio, A. P. C. Silva, H. L. J. Filho, H. V. Ehrenfried, Evaluating different strategies for reduction of mutation testing costs, in: Proceedings of the 1st Brazilian Symposium on Systematic and Automated Software Testing, SAST, ACM, Maringa, Brazil, 2016, pp. 4:1–4:10. https://dl.acm.org/citation.cfm?id=2993292

Deriving Products for Variability Test of Feature Models with a Hyper-Heuristic Approach

Published in Applied Soft Computing, 2016

Deriving products from a Feature Model (FM) for testing Software Product Lines (SPLs) is a hard task. It is important to select a minimum number of products but, at the same time, to consider the coverage of testing criteria such as pairwise, among other factors. To solve such problems Multi-Objective Evolutionary Algorithms (MOEAs) have been successfully applied. However, to design a solution for this and other software engineering problems can be very difficult, because it is necessary to choose among different search operators and parameters. Hyper-heuristics can help in this task, and have raised interest in the Search-Based Software Engineering (SBSE) field. Considering the growing adoption of SPL in the industry and crescent demand for SPL testing approaches, this paper introduces a hyper-heuristic approach to automatically derive products to variability testing of SPLs. The approach works with MOEAs and two selection methods, random and based on FRR-MAB (Fitness Rate Rank based Multi-Armed Bandit). It was evaluated with real FMs and the results show that the proposed approach outperforms the traditional algorithms used in the literature, and that both selection methods present similar performance.

Recommended citation: A. Strickler, J. A. Prado Lima, S. R. Vergilio, and A. T. R. Pozo. 2016. Deriving products for variability test of Feature Models with a hyper-heuristic approach. Applied Soft Computing 49 (Dec. 2016), 1232–1242, https://www.sciencedirect.com/science/article/pii/S1568494616303994?via%3Dihub