The workshop takes place in the Palazzo dei Congressi, Room 6 (click here for a map). For more information about the workshop venue, visit the ICSE 2015 conference page.
Each paper session starts with brief presentations of each paper (by one of its authors) and conclude with a 15-minute panel where the session topic is discussed by the session chair, the presenters and the participants. The presentations are scheduled for 10 minutes for positions papers and 15 minutes for technical papers including 2 minutes for quick clarifying questions.
|Monday, May 18, 2015
Welcome! Who's who
Mats Heimdahl (University of Minnesota, USA)
When using requirements as a basis for testing, the tester creates a set of tests to exercise a system's required behavior without regard to the internal structure of the system under test; an approach commonly referred to as black-box testing.
Generally, no objective metric is used to measure the adequacy of black-box tests—when do we have enough black box-tests to perform adequate testing? Of course, one could also ask, when do we have enough requirements to perform adequate black-box testing? In fact, maybe we should not have any requirements at all and only develop test scenarios? Currently, there are no mature techniques to help us answer any of those questions.
In this talk, we will discuss the challenges of assessing the adequacy of test suited designed to cover the required behavior of a system. We will explore how coverage measures could potentially be used to address the questions of black-box test adequacy as well as the adequacy of a set of requirements. Since the issues related to Requirements Engineering and Testing are just starting to be explored, we will also discuss many open challenges lying ahead for the RET community.
Timeline perspective by workshop chairs: RET14 → RET15 → Future
Mapping RET: Intermediate Report from an Evolving Overview
|Coffee Break (10:30-11:00)
Positions on Quality
Session Chair: Michael Felderer
Richard Berntsson Svensson, Björn Regnell (Lund University, Sweden)
Weak alignment of requirements engineering (RE) and Software Testing (ST) may lead to problems in delivering the required products in time with the right quality. Despite the importance of aligning RE and ST, the research have mainly been focused on one or the other of RE and ST. Previous work has identiﬁed several challenges in aligning RE and ST related to Quality Requirements, for example, the need for tool support for QR, deﬁning and managing QR, and veriﬁcation of QR. In this position paper, we introduce a new view of the QUPER model, namely aligning QR targets and test results in QUPERs roadmap view. The alignment view was evaluated at a case company with 13 practitioners. The results show that the alignment of QR and test results using QUPER’s roadmap view was viewed as an important input to the decision-making process.
Anila Mjeda, Mike Hinchey (Lero - the Irish Software Research Centre, Ireland)
We propose a test reactivity taxonomy for embedded automotive software that a tester can use to align the process of test design with the requirements specification and verification of the project at hand. The proposed test reactivity taxonomy can be used to explore, identify and design the right types of test reactivity which target a specific functional or non-functional requirement or an application-specific situation. The taxonomy can also be used to document software testing decisions and enable discussions between the relevant stakeholders. The ordering of information provided by the taxonomy could also be a starting point into identifying relationships, patterns and factors that influence the relevance of different reactivity dimensions.
Maya Daneva (University of Twente, The Netherlands)
In Requirements Engineering (RE) for large scale game systems, play-testing is an important activity that is used to validate requirements from players’ perspective. Play-testers are not professionals that are involved in the process of RE. They are not professional testers, either. Yet, their feedback in terms of perceptions and experiences of the early prototypes of a game, have a decisive impact on what the RE professionals do next in the RE process. This position paper presents a qualitative study that sought to discover who takes the role of play-testers and what kind of feedback play-testers generate in the early stages of RE for games. The case study responds to the observation that no textbook on RE or software engineering addresses play-testing as a phenomenon, and that classic Computer Science programs at universities teach testing techniques mostly in the context of embedded systems, hence students often are left with little opportunity to develop testing skills that build upon play-testing practices happening in the game development sector. The study therefore was aimed at identifying important implications that play-testing may have for research and teaching.
Research and Practice
Session Chair: Mirko Morandini
Henning Femmer, Jakob Mund, Daniel Méndez Fernández (Technische Universität München, Germany)
[Context] Requirements Engineering (RE) artifacts are central items in software development: Their quality is of essential importance for development, testing and other software engineering activities. However, as requirements artifacts are used differently in different processes, the proper definition of what is good quality depends on the context under consideration. [Problem] So far, no methodology exists that enables to define context-specific RE artifact quality in a precise manner. [Principle Idea] We define context-specific RE artifact quality by how quality attributes of an RE artifact impact on the activities of the software development process in which this artifact is used. [Contribution] In this paper, we introduce a methodology to define RE artifact quality specifically to a project- or process context. Furthermore, we provide a preliminary technical validation and a first industrial case study on the application of our approach. Our studies indicate that the activity-based approach enables defining and validating RE quality in a precise and systematic manner. The industrial case study furthermore suggests the applicability of the approach in practical use.
Paolo Tonella, Roberto Tiella (Fondazione Bruno Kessler, Italy)
Since the SEAC software handles business activities that are highly dependent on norms, such as the contracts of employees, the taxation of incomes and salaries, the pension contributions, one of the key challenges is to support a smooth transformation of norms into requirements, into code and eventually into test cases used to verify that norms have been implemented as prescribed by the law. The SE research unit at FBK has been involved to introduce a set of practices aimed at supporting such transformation, so as to improve the current process. We report the experience made during the project in this paper.
Session Chair: Michael Unterkalmsteiner
Sebastian Eder, Henning Femmer, Benedikt Hauptmann, Maximilian Junker (Technische Universität München, Germany)
Latent Semantic Indexing (LSI) is an accepted technique for information retrieval that is used in requirements tracing, to recover links between artifacts, e.g., between requirements documents and test cases. However, configuring LSI is difficult, because the number of possible configurations is huge. The configuration of LSI, which depends on the underlying dataset, greatly influences the accuracy of the results. Therefore, one of the key challenges for applying LSI-based methods is finding an appropriate configuration. Evaluating results for each configuration is time consuming, and therefore, automatically determining an appropriate configuration for LSI improves the applicability of LSI based methods. We propose a fully automated technique to determine appropriate configurations for LSI to recover links between requirements artifacts. We evaluate our technique on six sets of industrial requirements artifacts and show that the configurations selected by our approach yield nearly optimal results.
Hiroyuki Nakagawa, Tatsuhiro Tsuchiya (Osaka University, Japan)
This paper focuses on the constraints elicitation in the combinatorial test design. Pair-wise testing, a common combinatorial testing approach, is an effective test planning technique to find interaction faults from the relatively small set of test cases. The test space is modeled by a set of parameters, their individual values, and constraints on the value combinations. Constraints reduce the test space; however, little consideration has been given to the process of finding them. The goal of our study is to establish an automatic constraints acquisition from the requirements document. We try to establish a constraints elicitation mechanism that helps to extract constraints from the requirements document based on a linguistic approach. In this paper, as the first step, we attempt to extract the coupling strength between parameters from the requirements document. We conducted a preliminary experiment on an ATM system example and evaluated the feasibility of our approach.
Kapil Singi, Vikrant Kaulgud, Dipin Era (Accenture Technology Labs, India)
Digital, an amalgamation of social, cloud, mobility and analytics, is being widely used by enterprises for building innovative business applications. In digital applications, there is a healthy tension between 'business' and 'IT', leading to a rapidly evolving digital application. Often, digital applications have many stakeholders, requiring a role-based access and visualization of data. There is an emphasis on user experience. Due to such characteristics, a visual approach, such as 'Pretotyping', to documenting and validating requirements is required. While this creates a better connect between business users and business analysts, it makes testing more difficult. In this paper, we present a rationale for visual requirements of digital applications, and then present a graph-model based approach for automated generation of test cases from visual requirements.
Introduce exercise & start working in groups
|Coffee Break (15:30-16:00)
Working Session including summary
Paper Submission: January
Author Notification: February 18, 2015
Camera-Ready Due: February 27, 2015
Early-Bird Registration by: April 6, 2015
Workshop Date: May 18, 2015