WER Overview: Retrospective, Trends and Relevance

Joselaine Valaski, Wilian Stancke, Sheila Reinehr, Andreia Malucelli

Pontifícia Universidade Católica do Paraná, Programa de Pós-Graduação em Informática (PPGIa), Escola Politécnica, Curitiba, Brazil, 80.215-901

joselaine.valaski@pucpr.br, stancke@ieee.org, sheila.reinehr@pucpr.br, malu@ppgia.pucpr.br

Abstract

This work refers to the review of 258 papers published in the WER throughout 15 editions. This review´s goal was to identify the most active research groups within this workshop, the most debated topics and the trends in the Requirements Engineering area. Furthermore, the aim of this review was to analyze the relevance of WER. It was made identifying where WER papers have been cited. The results showed that Brazil, Argentina and Spain hold the most active groups. The requirements modeling is one of the most discussed topics in this event. Moreover, the results pointed out the international conference as one type of publication that more referenced WER papers, and the Requirements Engineering as one of the journals that more referenced WER papers.

Portuguese Abstract

Este trabalho se refere a revisão de 258 artigos publicados no WER ao longo de 15 edições. O objetivo desta revisão foi identificar os grupos de pesquisa mais ativos nesta conferência, os assuntos mais debatidos e as tendências na área de Engenharia de Requisitos. Além disso, o objetivo desta revisão foi analisar a relevância do WER, cuja análise foi realizada identificando as citações recebidas pelos artigos publicados no WER. Os resultados apontam que o Brasil, Argentina e Espanha representam os grupos de pesquisa mais ativos. A modelagem de requisitos é um dos temas mais discutidos no evento. Além disso, os resultados demonstram que conferências internacionais é o tipo de publicação que mais referencia artigos publicados no WER, e um dos journals que mais citam os artigos publicados no WER, é o Requirements Engineering.

Keywords: Retrospective, Trends, Influences, WER, Requirement Engineering.

Portuguese Keywords: Retrospectiva, Tendências, Influências, WER, Engenharia de Requisitos.

Received: 2013-09-09, Revised: 2014-02-05, Accepted: 2014-02-05

1 Introduction

The requirements identification is an extremely important activity, since it is the basis for the planning, development follow up and acceptance of the software project results [1]. In order to support these results, Requirements Engineering provides appropriate mechanisms to understand the client´s needs, analyzing necessities, checking feasibilities, negotiating a reasonable solution, specifying a solution without ambiguities, validating a specification and managing the needs as they are transformed into a system [2].

Some of the main activities related to Requirements Engineering are as follows: elicitation, analysis, specification, validation and requirements management [3]. These activities main goal is to support understanding and to formalize the client´s main needs in a way that it decreases the problems throughout the software development.

However, despite the advances in the Requirements Engineering area, as software become much more complex and bigger, new problems emerge and new solutions are proposed [4]. Thus, it is important to follow the evolution of the topics related to this area. Some mappings and systematic reviews have been performed in order to provide a better idea of what has been produced in this area. These works have offered a better vision for specific topics from the Requirements Engineering area, such as elicitation techniques [5] [6], specification techniques [7] and requirements writing standards [8].

However, it is also important to consider providing a broad vision of the main discussed topics among research groups, as well as identifying these groups. This information may bring several benefits to the scientific area, such as the identification of new information sources, establishment of new partnerships and the orientation of researches toward more relevant topics at a given time. The scientific works are a quite important source to find this information. The Requirements Engineering area has several means of publishing these works, such as specialized journals and international events.

In this context, this work aims to identify some of this information and as an initial source of research, the Workshop on Requirements Engineering (WER) was chosen. The WER is a workshop that has been taking place since 1998 with the goal of consolidating the Iberoamerican Requirements Engineering research community. The event has predominantly happened in Argentina and Brazil. However, in the last few years, other countries from South and North America and Europe have hosted this event as seen in Table 1.

Throughout 15 editions of the WER, 258 papers have been published. The published topics discuss issues involving the main activities of Requirements Engineering. The WER is considered a mature and consolidated event in the Requirements Engineering area and for this reason was considered a feasible source to obtain relevant answers in this area.

Table 1: Host cities of WER

Section 2 describes the method used in this research, followed by Section 3, which details and discusses the results. Finally Section 4 presents the final considerations of this work.

2 Method

This section describes the method used in this research, including the questions and the steps performed in order to make the extractions and classification of the analyzed papers.

2.1 Development of the research questions

With the goal of identifying the most active countries and institutions at the WER, as well as the most discussed topics, the trends in Requirements Engineering and the relevance of the WER event, the following questions have been made:

1. What are the main countries that published in WER?

2. What are the main institutions that published in WER?

3. What are the main topics discussed in WER?

4. Which institutions have been discussed the main topics?

5. Which topics have been discussed for the main institutions?

6. What are the trends in Requirements Engineering?

7. Where have WER papers been more referenced?

8. What are the conferences that have most referenced WER papers?

9. What are the journals that have most referenced WER papers?

2.2 Extraction of information

In order to answer the previous questions, each one of the 258 papers has been analyzed with the goal of extracting the following information: institutions involved in research, home country of the institution and main discussed topics. The access to the papers was made through the website http://wer.inf.puc-rio.br.

The institution identification and home country was manually made in each one of the papers. More than one institution could have been related to the paper through their researchers, including the ones from different countries.

In order to define the topics to be classified, first was analyzed the classification used in the previous WER´s editions. However, it was noted that in back 1998, 2000, 2002, 2006, 2007 and 2012, no classification was proposed by the event. Moreover, it was also noted that the topics had a variation in their nomenclature throughout the years, as well as the unification of topics, as it can be seen in Table 2. Considering these limitations, it has been decided to propose a classification based on the classifications already made unifying or subdividing determined topics. The result of this new classification is presented in Table 3.

After the topics classification definition, two students (one master and one doctoral) read the papers with the goal of identifying the main discussed topics. It was defined that the paper should be related to at least one topic presented in Table 3 but new topics not listed in Table 3 could be included and related. The reading and identification of the topics was performed in an independent way between the students. For some of the papers, the reading of the abstracts was enough in order to identify their topics. In other cases, the papers had to be read thoroughly. After the individual classification was performed, the students made together ​​the consolidation of issues where there was divergence of classification.

Table 2: Topics of the WER

Table 3: Proposed topics for classification

In order to identify where the WER papers have been most referenced, 21 of the most cited papers up to July 2013 have been selected. The list of 21 papers, on Table 4, was obtained on the website that hosts WER publications (http://wer.inf.puc-rio.br). A search was performed for each one of the 21 papers using the website http://scholar.google.com.br in order to obtain corresponding citations. For each citation, the type of publication was identified, according to the categories defined on Table 5. The conferences have also been classified as international and regional.

Table 4: WER most cited papers

Table 5: Type of publication used to classify the papers

As a support to align the names and types of conferences, as well as the identification of the journals, the WebQualis (http://qualis.capes.gov.br/webqualis/principal.seam) portal was used. The WebQualis is a Brazilian portal used to qualify journals and conferences in different knowledge areas. The results obtained after the papers’ classification are presented and discussed below.

3 Results and Discussion

The results will be presented and organized according to the research questions presented in Section 2.

3.1 What are the main countries that published in WER?

In all, 20 countries had at least one publication at WER through 121 institutions. In Table 6 it is shown the amount of institutions involved, the number of published papers and the percentage of publications considering the total amount of papers published up to 2012.

Table 6: Numbers of papers per country

As it has been pointed out in Table 6, the countries with the most publications at WER are Brazil, Argentina, Spain and Canada. Brazil has had an 80% participation ratio, as far as publication is concerned. Once the identification of the most active countries was done, this was also applied to the institutions (of education or not) with the most publications at WER.

3.2 What are the main institutions that published in WER?

As it has already been mentioned, 121 institutions have had at least one publication at WER. However, due to space limitation, in Table 7 only the institutions that have had at least 10 publications since the beginning of the WER (15 years ago) are presented. In this same table, the home country, amount of published papers and publishing ration for each institution are shown, considering the total of published papers up to 2012. The institutions with the most number of publications, due to logical reasons, are located in the countries pointed out in Table 6. Among the 9 institutions with the most number of publications, 4 of them are located in Brazil, 2 in Argentina, 2 in Spain and 1 in Canada. After the identification of the most active countries and institution, a verification of the most discussed topics and their relation with the research groups was performed.

Table 7: Number of papers per institution

3.3 What are the main topics discussed in WER?

In Table 8 it is shown for each one of the topics defined in Table 3, the amount of papers that were related to the topic and the corresponding percentage in an array of 258 papers.

As shown in Table 8, the 3 most related topics were: Requirements modeling, Requirements elicitation and Process/Method. In order to have a better idea of the intensity that these topics were discussed throughout the editions, the illustrative graphic in Fig. 1 was used. Through this graphic it is possible to observe that the topic Requirements Elicitation was highlighted up to 2008, with some peaks.

As far as the topic Requirements modeling, since 2009, it started to be the more discussed than the topic Requirements Elicitation. It is important to observe that in general is difficult to separate the both terms, they are much related. Usually the elicitation activity is supported by the modeling activity and vice-versa.

However, the modeling activity in the last years may has been seen as the main activity in this relationship. Process/Method is a topic that has had many peaks and starting on 2009, it presented a significant reduction in the number of published papers. This oscillation may be related to the period of the creation and dissemination of quality maturity models.

Table 8: The most discussed topics in WER

Figure 1: Number of papers related to the topic per year

As it has been previously mentioned, the topic relation was based in Table 3, but the students who made the classification could also identify other topics that they judged relevant in the paper. These other topics are presented in Table 9.

Table 9: Other relevant topics discussed in WER

These topics can also be understood as subtopics, since each one of them is related to at least one topic from Table 3. Out of these subtopics, the most related ones were: i*, Models transformation and Oriented goals, also known as GORE (Goal Oriented Requirement Engineering).

The i* is an objective oriented approach used to describe not only social and intentional needs in the organizational environment, but also functional and non functional Requirements [9]. These three topics appear in conjunction in many situations, since they are strongly related.

With the mapping of the main topics and institutions, the relation of these topics with the institutions and vice versa has been also performed.

3.4 Which institutions have been discussed the main topics?

According to the 3 highlighted topics in Table 8 and the most related subtopic in Table 9, it was possible to identify which institutions had the most publications related to these topics. In Table 10 it is shown the 3 institutions with the most papers related to the 4 topics selected for evaluation.

The Requirements elicitation topic was the most discussed one by the Pontifícia Universidade Católica do Rio de Janeiro, whereas the Requirements modeling, Process/Method and i* was the most discussed by Universidade Federal de Pernambuco (UFPE). The UFPE appears as one of the 3 most active institutions as far as the 4 topics analyzed are concerned.

The topics that main institutions have been discussing have also been identified. The results are presented as follows.

Table 10: The main topics and related institutions

3.5 Which topics have been discussed for the main institutions?

Based on the 5 most active institutions highlighted in Table 7, the most discussed topics by them have been analyzed. Table 11 presents the institutions, the 3 most discussed topics and the amount of papers related to this topic.

Both the Universidade Federal de Pernambuco and the Universidad Politécnica de Valencia have been discussing with more emphasis the Requirements modeling topic, whereas the Pontifícia Universidade Católica do Rio de Janeiro and the Universidad Nacional de La Plata have been discussing the Requirements elicitation topic. The Universidad Nacional del Centro de la Provincia de Buenos Aires has been emphasizing the Scenario subtopic discussion. With the mapping of the most discussed topics at WER, it is observed that some topics were more discussed in the past, whereas other topics gained more importance in the last few years.

Based on this observation, some topics were extracted where these variations are more meaningful. These variations may indicate trends in Requirements Engineering and the results are shown in section 3.6.

Table 11: The main institutions and the related topics

3.6 Which are the trends in Requirements Engineering?

According to the variation on the amount of published papers related to the analyzed topics, a trend analysis in the Engineering Requirements area was performed. In order to facilitate the visualization of this analysis, the publications were gathered into triennium groups, coming to a total of 5 groups, according to what is shown in Table 12. Between the second and fourth triennium there has been an increase in the amount of publications, when compared to the first and fifth triennium. It is important to observe this detail so that a correct analysis in the variation of publications can be performed.

Table 12: Numbers of paper published per triennium

Among the evaluated topics, it has been noted some significant variations. These topics are represented in Table 13. The amount of publications for each triennium and its corresponding percentage of the total amount of published papers are shown on it. This percentage was used to balance the difference in the number of publications that has happed among the trienniums.

Table 13: Number of publications per topic/year

Based on these percentages the graph was created (Fig. 2) where it is possible to observe these variations in a clearer manner.

Through Fig. 2 it is possible to observe an increase in the past few years of the following topics: i*, Business modeling and Software transparency. On the other hand, it has been observed a decrease in the number of publication of the following topics: Process/Method, Requirements Management, Measurement/Metrics and Agent-oriented paradigm. The Quality software topic had its pinnacle in the 2004-2006 trienniums, but it still bears certain stability.

The following questions had the objective of identifying the relevance of WER. For that purpose, it has been identified where the papers published on WER have been referenced on.

Figure 2: Publications’ variation of some topics

3.7 Where have the WER papers been most reference?

As previously mentioned, in order to answer this question, 21 of the most referenced papers from WER were selected. In all, 511 references have been identified. However, the duplicated references to a single paper were excluded, resulting in a total of 502 analyzed references.

Two analyses were performed. The first one considered all of the 21 papers and the second one considering the 16 papers written in English language only. This separation was performed due to the observation of different results between papers in English and in other languages. Figure 3 presents the results of these differences.

Figure 3: Where WER papers have been most referenced

According to the two analyses presented in Fig. 3, the conferences, the academic documents and the journals are types of publications that possess most references to papers published on WER. However, if only the 16 papers written in English are observed, the average of journal citations increases. In case 21 of the selected papers are observed, not taking into account the language they are written, there is an average increase in the citation of academic documents.

In order to verify if the obtained results may be considered as a standard to others papers published in the WER, the standard deviation was calculated and the result is showed on the Table 14.

Table 14: Standard deviation

The standard deviation is greater for the following publication types: conference and academic document. However, if it is only considered the results of papers written in English, the standard deviation percentage decreases, mainly for conferences, journals and academic document. Even if the standard deviation is considered, it is possible to conclude that a conference is the type of publication that most references papers published on WER.

Considering that, conferences and journals are the types of publications that most spread scientific works, the particular conferences and journals that most reference papers from WER have also been identified.

3.8 What are the conferences that have most referenced WER papers?

Among the 502 analyzed papers, 95 different conferences have been identified. Due to a space constraint, Table 15 only shows the conferences that have had at least two WER papers citations.

Table 15: Conferences that have referenced WER papers

According to the results, it is possible to observe that citations are distributed among several conferences. The main concentration of references has been identified on WER itself (36); the following ones are REFSQ (9), WICC (8) e IDEAS (7), respectively. In order to verify the conferences’ coverage, they were classified into international and regional. Fig. 4 presents the results.

Figure 4: Percentage of publication on international and regional conferences

Most citations (54%) have been made on international conferences. On Fig. 5, it is shown that if the number of citations made on WER (36), which is not considered an international conference, is excluded, the percentage of international conferences citations (66%) increases.

Figure 5: Percentage of publications on international and regional conferences (excluding WER)

The results conclude that although the WER is not considered an international conference, most of the citations made in conferences, are related to international ones. This shows the relevance and influence of the WER event on international research results.

It is important to emphasize that some conferences have changed their names as years past, and this alignment was not done in this research. Thus, it is possible that some results have not been grouped on the same conference.

Next section shows the journals in which the results published on WER have been referenced.

3.9 What are the journals that have most referenced WER papers?

As regards to journals, 41 different occurrences have been identified. Table 6 shows the relation of all identified journals.

As in the conferences, the citations are also distributed among several journals. The greatest concentration of references has been identified on Requirement Engineering Journal (10), followed by Software & Systems Modeling Journal (5) and the Journal of Systems and Software (4).

Table 16: Journals that have referenced WER papers

4 Conclusion

When studying a research area, it is important to identify the answers to some questions, such as: the most active groups, the most discussed topics, the identified trends and so forth. This information may bring some benefits as the identification of new information sources, establishment of new partnerships, the orientation of researches toward more relevant topics at a given time. We can also mention the relevance of the event in the international scene.

The revision of 258 papers published at WER had the goal of answering these questions as far as this event is concerned. Some of the main obtained results were the identification of Brazil, Spain and Argentina as the home countries of the main institutions that have published at WER so far. The Universidade Federal de Pernambuco located in Brazil, is a reference as the most active in this event. The Requirements elicitation and Requirements modeling topics are the most referenced in the published papers and the Requirements modeling is the most cited in the past few years. It has been observed an increase in the reference to the following topics: i*, Business modeling and Software transparency.

The majority of references published on WER were made on conferences, journals and academic documents. Among conferences, WER itself is the one that most references the works published. Considering all conferences’ publications, the international publications are the ones that most reference works published on WER. As regards to journals, the Requirements Engineering is the one that most references the works published on WER.

This is a preliminary study, since the issues exposed here may be broaden and explored in a deeper fashion in future works. Other analysis may be performed in the database mentioned above, allowing distinct points of view, other than the one presented in this work. A new paper classification may be performed in order to refine the obtained results and it may also amplify the events to be analyzed. A similar but more refined review may be performed, including the main events or journals in the Requirements Engineering area.

In the same event and year where the paper [10] that triggered this work was published, a systematic mapping in the Requirement Engineering Area was published [11] as well, motivated by similar reasons as this research, however, it considered Brazilian Community publications only. These works emphasize the importance of analyzing what has been done in the past in order to decide how to act in the future.

References

[1] S. T. Fiorini., J. C. S. P. Leite, and C. J. P. Lucena, “Organizando Processos de Requisitos,” in Workshop on Requirements Engineering, Maringa, Brazil, pp 1-8, 1998

[2] R.Thayer and M. Dorfman, “System and Software Requirements Engineering,” IEEE Computer Society Press Tutorial, 718p, 2000

[3] R. S. Pressman, Engenharia de Software – Uma Abordagem Profissional, McGraw-Hill, 2011.

[4] J. Dominguez, The Curious Case of the CHAOS Report, 2009

[5] A. Davis, O. Dieste, A. Hickey, N. Juristo, and A. M. Moreno, “Effectiveness of Requirements Elicitation Techniques: Empirical Results Derived from a Systematic Review,” in Proceedings of the 14th IEEE International Requirements Engineering Conference. IEEE Computer Society, 2006

[6] O. Dieste, N. Juristo, “Systematic Review and Aggregation of Empirical Studies on Elicitation Techniques”, IEEE Transactions on Software Engineering, IEEE computer Society Digital Library. IEEE Computer Society, 2010

[7] N. Condori-Fernández, M. Daneva, K. Sikkel, R. Wieringa, O. Dieste, and O. A, Pastor, “A systematic mapping study on empirical evaluation of software requirements specifications techniques”, third International Symposium on Empirical Software Engineering and Measurement, ESEM 2009, IEEE Computer Society, Florida, USA, pp 502-505, 2009

[8] R. Cezario, F. Barreto, and V. Benitti, “Padrões de Escrita de Requisitos : um mapeamento sistemático da literatura,” in Workshop on Requirements Engineering, Rio de Janeiro, Brazil, pp 259-272, 2011

[9] M. Lucena, C. Silva, E. Santos, F. Alencar, and J. Castro, “Modularizando Modelos i *: uma Abordagem baseada em Transformação de Modelos,” in Workshop on Requirements Engineering, Valparaiso, Chile, pp 33-44, 2009

[10] J. Valaski, W. Stancke, A. Malucelli, S. Reinehr, “Retrospective and Trends in Requirements Engineering” in Workshop on Requirements Engineering, Montevideo, Uruguay, pp 135-148, 2013

[11] K. Oliveira, J. Pimentel, E. Santos, D. Dermeval, G. Guedes, C. Souza, M. Soares, J. Castro, F. Alencar, C. Silva, “25 years of Requirements Engineering in Brazil: a systematic mapping,”. in Workshop on Requirements Engineering, Montevideo, Uruguay, pp 118-134, 2013