Quality requirements is a difficult concept in software projects, and testing software qualities is a well-known challenge. Without proper management of quality requirements, there is an increased risk that the software product under development will not meet the expectations of its future users. In this paper, we share experiences from testing quality requirements when developing a large system-of-systems in the public sector in Sweden. We complement the experience reporting by analyzing documents from the case under study. As a final step, we match the identified challenges with solution proposals from the literature. We report five main challenges covering inadequate requirements engineering and disconnected test managers. Finally, we match the challenges to solutions proposed in the scientific literature, including integrated requirements engineering, the twin peaks model, virtual plumblines, the QUPER model, and architecturally significant requirements. Our experiences are valuable to other large development projects struggling with testing of quality requirements. Furthermore, the report could be used by as input to process improvement activities in the case under study.

Improvement and evaluation of software quality is a recurring and crucial activity in the software development life-cycle. During software development, software artifacts such as requirement documents, comments in source code, design documents, and change requests are created containing natural language text. For analyzing natural text, specialized text analysis techniques are available. However, a consolidated body of knowledge about research using text analysis techniques to improve and evaluate software quality still needs to be established. To contribute to the establishment of such a body of knowledge, we aimed at extracting relevant information from the scientific literature about data sources, research contributions, and the usage of text analysis techniques for the improvement and evaluation of software quality. We conducted a mapping study by performing the following steps: define research questions, prepare search string and find relevant literature, apply screening process, classify, and extract data. Bug classification and bug severity assignment activities within defect management are frequently improved using the text analysis techniques of classification and concept extraction. Non-functional requirements elicitation activity of requirements engineering is mostly improved using the technique of concept extraction. The quality characteristic which is frequently evaluated for the product quality model is operability. The most commonly used data sources are: bug report, requirement documents, and software reviews. The dominant type of research contributions are solution proposals and validation research. In our mapping study we identified 81 relevant primary studies. We pointed out research directions to improve and evaluate software quality and future research directions for using natural language text analysis techniques in the context of software quality improvement.

When specifying user requirements, not only it is critical to ensure correct and unambiguous specification of functional requirements, but also that of non-functional requirements (NFRs). In fact, resolving ambiguities from user specified natural language NFRs and specifying the correct ones in a formal language have attracted significant attention. Our current research focuses on the issues pertaining the same. We observe that it is a usual practice for a user to narrate the NFRs in natural language and the requirement engineers manually try to express the same, using some semi-formal or formal language notations. However, inaccurate and the laborious manual approach may fail to detect all the NFRs and correctly remove the ambiguities in those detected. Hence, current research attempts have focused on automating the conversion of natural language NFRs to formal notations. In literature, there exist numerous approaches that take requirements as input and output the extended UML counterpart including NFRs. However, majority of the approaches do not support ambiguity resolution and verification of the extracted NFRs that are fairly essential. In this paper, we propose and discuss a hybrid approach viz. NFRs-Specifier, that attempts to resolve ambiguities, extract NFR’s, perform verification and generate NFRs specification by means of the extended UML model.

Coordination is an important success factor for a development project. Communication gaps, e.g. between product owners shaping the requirements and testers verifying the developed software can result in wasted effort and unsuccessful products. We propose improving the communication between project members with recommendations of whom to interact with and what to discuss based on link prediction in multi-layered proximity-based social graphs based on data mined from project repositories. We plan to explore and validate these ideas through prototyping and by applying a design-science approach in collaboration with an industrial partner.

In this paper, we discuss the problem of transforming a natural language requirements specification into a formal specification. We present several methods how to support the process and implemented them in a commercial tool, the Requirements Quality Suite. We achieve this by enriching the requirement text with additional structure (using a knowledge base) and asking the requirement engineer to formulate his requirements in Boilerplates. The additional structure is used to analyze the requirements automatically or semi-automatically leading finally to a formal specification. The formal specification then enables verification activities, such as testing or formal analysis. We discuss our methods by examples from an industrial case study and report on our experiences.