Context: The alignment of different software engineering activities for coordinated functioning and optimized product development is of great importance, particularly in industrial-scale development. The link between intermediate activities has been researched extensively, but the link between requirements engineering (RE) and software testing (ST) is a relatively less explored area.
Objective: The objective of this study is to aggregate, structure, and classify all existing research regarding alignment of RE and ST published by the end of 2015.
Method: We conducted a systematic mapping study (SMS) and aggregated all studies relevant to our scope. The primary studies are analyzed in terms of publication trend, focus area, i.e., how alignment is supported, the application domain and benefits and challenges, methodological data, and scientific rigor and industrial relevance.
Results: There is a growing interest towards the topic. Several different techniques have been identified to improve RE and ST alignment. Test generation from requirements specification has received most attention. Alignment of RE and ST is particularly important for large safety-critical domains. While many challenges have been reported, the supporting evidence for benefits is scarce. Frameworks/methods/techniques is the most frequent contribution type. Solution proposal and evaluation research were the most frequently applied research type. Case study research was the most frequently applied research method, however, almost half of the studies did not clearly report any research method.
Conclusion: Despite the numerous approaches that are proposed, it is not clear what approach is suitable in what context and why. To support industry in RE and ST alignment, guidelines and tool support are needed. The supporting evidence for claimed benefits is very limited. Overall, the research area is in its early stages and an increase in both the number and rigor of empirical studies are required.

As agile methods become more pervasive, agile practices are applied to more large-scale systems with a scope that goes beyond pure software. The expansion of agile in these contexts provides benefits, but creates new challenges. Widespread use of agile has changed the way we must think about practices both in Requirements Engineering (RE) and in System Testing (ST). Our experience shows that many challenges in the application of large-scale agile development relate to either RE or ST, and in particular to the alignment between these areas. In this paper we present large-scale agile-related challenges from a multiple case study which relate to REST alignment. We map our challenges to an existing framework for REST alignment, and make an initial attempt to suggest agile RE practices from the literature which may alleviate these challenges. Our results show that the interviewed companies need to first adopt more agile RE practices to enhance REST alignment and then leverage agile testing. Future work will look more towards evaluating these best practices.

Requirements-based testing has become a critical quality assurance technique designed to ensure a sufficiently high degree of product quality. However, the quality of the test cases depends on the quality of the requirements specification. In a preliminary experiment, we analyze potential links between the quality of the requirements and of the test cases.

Software requirements specifications (SRS) serve as a source of communication and information for a variety of roles involved in development activities. From the viewpoint of these SRS consumers, which includes testers as one of the key customers, the analysis of requirements specifications is often frustrating as it is time consuming and often requiring a lot of cognitive effort due to the increasing complexity of the documented information. Filtering the large amount of information by generating views that fit role-specific demands of SRS consumers is a promising solution approach for tackling this problem. This paper discusses concepts and key functionalities of an initial tool implementation of our proposed solution that is based on detailed knowledge about information needs that we gained in a series of empirical studies. Furthermore, we present potential usage scenarios illustrating its application in industry from the viewpoint of a tester.

The ongoing trend on distributed development activities of system integrators managing global development teams causes a growing need for specification activities and techniques. Each component leads to a large number of specification documents being exchanged, change managed and committed between system integrator and component supplier. The quality of the specifications exchanged influences significantly the timing, costs and success of the development task. However, the quality of such specifications is often far from optimal, exhibiting gaps, inconsistencies, redundancies, and unbalanced structures. At every release or delivery milestone, acceptance and integration testing take place at the system integrator. Therefore, testcases have to be created from the requirements exchanged. This paper presents a model-based approach for improving the quality of comprehensive requirements sets. The presented solution is based on a combination of a graphical notation and natural language and can be used to improve existing specifications or to create sound requirements structures from scratch. Furthermore, a model-based test-case specification approach is presented. The approach has been implemented using state-of-the-art tools. We present experience from field application of methods and tools in large industrial projects.

The coverage of requirements is a fundamental need throughout the software life cycle. It gives project managers an indication how well the software meets expected requirements. A precondition for the process is to link requirements with project artifacts, like test cases. There are various (semi-) automated methods deriving traceable relations between requirements and test scenarios aiming to counteract time consuming and error-prone manual approaches. However, even if traceability links are correctly established coverage is calculated based on passed test scenarios without taking into account the overall code base written to realize the requirement in the first place.
In this paper we introduce the so called "Requirements-Testing-Coverage" (ReTeCo) approach that automatically establishes links between requirements and test cases through source code lines which a) have been written in the context of an issue as part of a linked requirement, b) have been committed into a version control system, and c) produce code coverage results. Since the approach takes into account source code lines it is able to calculate coverage reports on a fine-grained contextual level rather than on the result of high-level artifacts.
We show the feasibility of the approach and initial evaluation results using the code base and test scenarios of large open source projects.

Keeping requirements and acceptance test documents aligned and up-to-date plays an important role in the success of software projects. In practice, these documents are not always aligned with each other, nor with the actual system behaviour. A previous study showed that even when requirements are updated, acceptance tests might stay outdated, which often leads to quality problems and unintended costs. In order to keep the requirements and test documents in a consistent state, we are developing an approach that automatically generates guidance on how to change impacted acceptance tests when changes in requirements occur. In this paper, we briefly present our approach and a prototype tool that implements it. A preliminary evaluation of our approach yielded encouraging results.

Web applications pervade our life, being crucial for a multitude of economic, social and educational activities. For this reason, their quality has become a top-priority problem. End-to-End testing aims at improving the quality of a web application by exercising it as a whole, and by adopting its Requirements Specification as a reference for the expected behaviour. In this paper, we outline a novel approach aimed at generating test scripts for web applications from either Textual or UML-based Requirements Specifications. A set of automated transformations are employed to keep Textual and UML-based Requirements Specifications synchronized and, more importantly, to generate End-to-End test scripts from UML artefacts.

This paper presents ReqCap, an implementation of a new method that articulates hierarchical requirements modeling and test generation to assist in the process of capturing requirements for PLC-based control systems. ReqCap is based on a semi-formal graphical model that supports hierarchical modeling, thus enabling compositional specifications. The tool supports automated generation of test cases according to different coverage criteria. It can also import requirements directly from ReqIF files and automatically generate Sequential Function Charts (SFCs).
We use a real-world case study to show how ReqCap can be used to model realistic system requirements. We show how the automated generation of SFCs and test cases can support engineers (and clients) in visualizing and reviewing requirements. Moreover, all the tests listed in the original test document of the case study are also generated automatically by ReqCap, demonstrating that the tool can be used to effectively capture requirements and generate valid and useful test cases.