One of the most difficult issues in the test efficiency is that of achieving a balance between testing goals and testing resources. Test execution scheduling is one way to save time and budget, where a set of test cases are grouped and tested at the same time. To have an optimal test execution schedule, all additional information of a test case (e.g. execution time, functionally, dependency and similarity with other test cases) need to be analyzed. Test scheduling problem becomes more complicated in the high-level testing, such as integration testing and especially in a manual testing procedure. Test specifications are generally written in natural text by humans and usually contain ambiguity and uncertainty. Therefore, analyzing a test specification is demanding a strong learning algorithm. In this position paper, we propose an NLP based approach that, given a test specification in the integration level, allows automatic detection of test cases semantic dependencies. The proposed approach utilizes the Doc2Vec algorithm and converts each test case into a vector in a n-dimensional space. These vectors are then grouped using the HDBSCAN algorithm into semantic clusters. Finally, a set of cluster-based test scheduling strategies will be proposed for execution. The proposed approach has been applied in a sub-system from the railway domain by analyzing an ongoing testing project at BOMBARDIER Transportation Sweden.

Designing a small set of tests that nonetheless cover the requirements sufficiently is tantamount to keep costs for testing at bay while still maintaining the necessary quality. Engineering an optimal test-suite requires both, insight into the domain and the system under test, but also carefully examining the combinatorics inherent in the requirements. Especially the second part is a cognitive challenge and systematic methods are cumbersome when performed by hand. In this paper, we present Specmate, a tool that supports and partly automates the design of tests from requirements. It provides light-weight modeling techniques to capture requirements, test generation facilities to create test specifications and further supporting functions to derive test procedures or test-scripts from specifications. Specmate has been developed an evaluated in the context of one of the core business systems of Allianz Deutschland, a large insurance company. The source code is freely available at GitHub and an online-demo of Specmate is available at Specmate TUM.

In order to reach an acceptable level of quality in a software product, testing of the software is paramount. Testing can be requirement driven or test-driven, which is translated into requirement-driven test case development or test-driven test case development. The focus of the study in this paper is on the former. But no matter what type of test case development implemented, to reach "good" software quality it is necessary to rely on "good" test cases. To define the criteria that make up for a good test case is not an easy task. Some intents have been presented over the years but none of the publications have studied the individual ranking of these criteria. This paper presents the results from a study undertaken in Sweden that indicate how experts in the Swedish software industry think about good test cases and rank a set of test case criteria.

Testing in the active sense is the most common way to perform verification and validation of systems, but testing in the passive sense has one compelling property: independence. Independence from test stimuli and other passive tests opens up for parallel testing and off-line analysis. However, the tests can be difficult to develop since the complete testable state must be expressed using some formalism. We argue that a carefully chosen language together with an interactive work flow, providing immediate feedback, can enable testers to approach passive testing. We have conducted a case study in the automotive domain, interviewing experienced testers. The testers have been introduced to, and had hands-on practice with a tool. The tool is based on Easy Approach to Requirements Syntax (EARS) and provides an interactive work flow for developing and evaluating test results. The case study shows that i) the testers believe passive testing is useful for many of their tests, ii) they see benefits in parallelism and off-line analysis, iii) the interactive work flow is necessary for writing the testable state expression, but iv) when the testable state becomes too complex, then the proposed language is a limitation. However, the language contributes to concise tests, resembling executable requirements.

Researchers have proposed a number of prioritization techniques to help decision makers select an optimal combination of (non-) functional requirements to implement. However, most of them are defined based on an ordinal or nominal scale, which are not reliable because they are limited to simple operations of ranked or ordered requirements. We argue that the importance of certain requirements could be determined by their criticality level, which can be assessed using a ratio scale. The main contribution of the paper is the new strategy proposed for prioritizing functional requirements, using early mutation testing and dependency analysis.

There are two basic constraints in testing: cost and quality. The cost depends on efficiency of testing activities and quality and testability. The practical experience of the author in large-scale systems shows that if requirements are adapted iteratively or the architecture is altered, the testability decreases. However, what is often lacking is a root cause analysis of testability degradations and the introduction of improvement measures during the development of a software. In order to introduce agile practices in the rigid strategy of the V-model good testability of software artifacts is vital. So, testability is also the bridgehead towards agility. In this paper we report on a case study where we measure and improve testability based on the Goal Question Metric Approach.

Software requirement specification (SRS) document is the most crucial document in software development process. All subsequent steps in software development are influenced by the requirements. This implies that the quality of SRS influences the quality of the software product. However, issues in requirement, such as ambiguities or incomplete specification may lead to misinterpretation of requirements which consequently, higher the risk of time and cost overrun of the project. Finding defects in the initial phase is crucial since the defect that found late is more expensive than if it was found early. Thus, the requirement should be tested before moving to other development phases. This study describes an automated approach for detecting ambiguous software requirement specification. To this end, we propose the combination of text mining and machine learning. Since the dataset is derived from Malaysian industrial SRSs and the respondents’ that evaluated our result are from Malaysia, we focus this study on Malaysian context for a pragmatic reason. We used text mining for feature extraction and for preparing the training set. Based on this training set, the method ’learns’ to classify ambiguous and non-ambiguous requirement specification. In this paper, we study a set of nine classification algorithms from the machine learning community and evaluate which algorithms perform best to detect ambiguous software requirement specification. We take a step forward to develop a working prototype to evaluate the reliability of our approach. The validation of the working prototype shows that the classification result is reasonably acceptable. Even though this study is an experimental benchmark, we optimist that the result of this study may contribute to enhance the quality of SRS and as well as assisting the requirement testing or review.