Monotonic and Linear Relations between Growth of Quality vs Growth in

Quantity in Open-Source Software Projects

EKBAL RASHID , NIKOS MASTORAKIS

Abstract: - Mathematical models have been developed to study the relation between the growth of quantity and the

growth of quality of open-source collaborative software repositories of GitHub. GitHub events triggering the

growth of quantity and quality have been identified. Linear regression analysis, Pearson’s, Spearman’s, and

Kendall’s correlation coefficients have been used. Hypothesis testing has led to the conclusion that there may be a

linear relation between quality and quantity within a certain range of values. Positive monotonic relations and

dependency between quantity and quality have been strongly established. Scripts for automated testing have been

1 Introduction

The noted statistician George Box had said, “All

models are wrong, but some are useful.” And so,

when we go on to try to model the quality vs quantity

factor for open-source software repositories, we are

facing a myriad of issues to be tackled. The foremost

being how to look at quality and quantity, how to

represent them, and how to cope with the ever-

open-source world with its collaborative nature is also

full of people who are learning and at the same time

contributing. There are communities of developers,

writers, artists, and designers, working in tandem,

with many of them also involved in other day jobs,

contributing to making big projects work. The projects

evolve beautifully unfolding amidst their seemingly

chaotic but homogeneous path of development, giving

rise to extremely useful engineering products. In this

paper, the authors have attempted to try and identify

events that lead to quantitative changes and those that

lead to qualitative changes, then data related to those

events can be mined and analyzed. This is exactly

what the authors have attempted to do in the present

study. Parameters have been identified, normalized

and bundled into two variables – quality and quantity.

Two models have been suggested here. Many more

can be designed likewise. One of the models suggests

a strong positive linear relationship between quality

and quantity. Both suggest a strong positive

The field of mining software repositories is still an

evolving area of research and hence needs to be

further probed and studied. The present research is

motivated by the following works. [1], has discussed

about the differences between two versions of

software and has shown the effect of changes in

source code. This will assist the developers in

an attempt to understand the quality of the software in

tool has been used here to get identifier names from

the source code of Java projects. They have also

discussed which type of names of identifiers lead to

which type of problems. Again, here the changes in

the code are affecting the software quality. [3], as

explained the effect of code refactoring on the quality

of the software. They have developed techniques to

improve the quality of the software by refactoring

code. The authors have tried to formalize the method

of refactoring code in this paper. [4], have in a similar

relation between the two and this may constitute an

interesting study. However, these works are looking

into the software projects only from a static point of

view. It is necessary to look at these projects from a

dynamic perspective, from the viewpoint of their ever-

changing nature, from the viewpoint of their coming

into being and going out of being. This has been the

primary focus of the present paper.

Several studies are there that have tried to

understand features of open-source projects and

at popularity. There has to be a judgement of quality

too using suitable metrics based on events of open-

source projects.

There are other works such as [7] and [8] that

analyze the data in datasets related to the GitHub

repositories. In such studies, mainly the stars, forks,

and issues are considered. Many have also included

code and outline of how the data has been retrieved,

that is they have elaborated in detail the mining

methods involved. The study also did a random word

Out of the list, some are selected at random and

does motivate the present study to think in similar

terms.

In [9], authors have adequately described that a

large number of GitHub repositories are personal and

not active. This may have a large effect on the

conclusions that one may draw from a dataset of

GitHub repositories. For this, the authors analysed

parts of GHTorrent datasets and sent surveys to users

of GitHub. They also highlighted the fact that there

was a substantial number of projects that had very few

successful. It presented a picture of the number of

commits and the number of lines of code being

changed in each file and a comparison between the

number of commits and file changes in different

versions etc. All these works are pointing towards the

attempt to design a software quality vs quantity model

for understanding the relation between them.

than 170 queries were performed on the dataset to

extract the data. Since the process is cost-consuming,

it could be performed for only a single time. This

extracted data was cross-checked from Click House

[11] and all the data was tabulated. It needs to be

mentioned here that GH Archive has made available

the data from GitHub for the last eleven years, that is,

There are many actions involved with it such as

opened, closed, reopened, assigned, etc.

activity of any member of the repository

repository

pull request is made and this also includes many

actions like opened, closed, reopened, assigned,

etc.

release, this event is triggered

listing of sponsorship

For this study, some GitHub projects have been

randomly selected based on popularity. Three

types of popularity measures have been

considered as follows (Table 1):

selected by running queries as per GitHub docs, [12],

and then listing them out. Only one criterion has been

considered while selecting the repositories, that is, the

repositories should hold coded projects. There are

collections of images, or books, or other resources as

GitHub repositories, but they have not been

considered for this study.

The raw data was then processed to properly

ascertain the quantitative, qualitative, and popularity

parameters. Although there may be many ways in

collaborative projects like GitHub, this research has

is normalized. The reason for this is that different

types of repositories are taken up for data

collection. A less active repository may have the

same value of closed pull requests to opened pull

requests ratio with a far lesser number of pull

requests. Hence, this ratio has been normalized by

multiplying it by a factor proportional to the closed

number of pull requests. The final normalized pull

request activity is as follows:

where (closed pull requests/100) is the normalizing

factor. There would be not much difference in the

final results had we simply multiplied the original

ratio with the number of closed requests without

dividing it by 100, but it has been done to keep the

values small. This normalized pull request activity has

been calculated for all the listed projects for a period

of seven years (2015-2021).

Hence, any update in the form of pushes or the

The data for seven years (2015-2021) have been

calculated for all the projects listed above. After this

has been done the value showing the growth of quality

is calculated for each year and each project using the

following formula:

Where the term (no. of pushes)/10 is the

correlation coefficients have been evaluated for

understanding the relation between them. The detailed

methods to find out these coefficients have been

explained in several noted works, [13], [14], [15],

[16], which have been consulted meticulously. The

details have been skipped with the assumption that the

reader can easily access these works or any other

related literature.

Several models have been framed with different

upper and lower limits of parameters. The most

to estimate the mathematical relation between the

growth in quantity and the growth of quality. For this

entire activity, an entire Python script has been

developed for automated testing. The script has been

executed using Jupyter. It takes values from the Excel

sheets of different models and generates graphs and

necessary analytical material for drawing inferences.

We know from the principles of Hypothesis

testing as elucidated in many noted works such as

[17], [18], [19], [20] that the null hypothesis is

4 Mathematical Models

the statistician to reject the null hypothesis. However,

the residuals of the OLS fit do not satisfy the

condition of being normally distributed. Besides the

studentized residuals also do not fall within the range

from +2 to -2. We therefore cannot accept the

regression model. We may not conclude anything

about the linear relationship between quality and

quantity based on this model. This is probably

because of the outliers existing in the model. We can

see from the graph of predicted mean and predicted

quality and quantity. Kendall’s coefficient also does

not rely on any assumption and it has a high value

suggesting that there is a strong dependence of quality

on quantity. Therefore, although this model does not

succeed in terms of establishing a linear relationship,

it does suggest strong dependence and strong positive

monotonic relationship between quantity and quality.

Model 1: In this model, an attempt has been made to

assuming that pure collaborative work may result in

some upper limit of quality. Projects achieving higher

quality values may be artificially achieved by using

the non-random involvement of developers. Hence

there are no zero values for this model. The Python

script generates the following regression analysis See

Table 4 and Table 5:

Table 5. Regression summary (model1) with constant

Fig. 10: predicted mean and predicted points interval

for model 1

Fig. 11: Quality vs residuals for model 1

Fig. 12: studentized residuals for model 1

Fig. 16: Kendall correlation for model 1

The correlation coefficient without constant is

fairly high. The correlation coefficient with the

coefficient is also sufficiently high. In both cases, the

P-values are way below the 0.05 value. Almost all the

residuals fall within the range from +2 to -2. Almost

all the residuals are along the 45-degree line and thus,

and quality for this model. The relation between

quality and quantity can be expressed using the

equation:

y = 5.3543 x + 60.5005

Suggesting that for every unit quantitative

dependence between quality and quantity. Based on

this model, we may say that within the given range of

quality values selected, there exists a strong positive

monotonic and linear relationship between growth in

quantity and growth of quality of collaborative

software projects.

The predicted mean interval shown in the graph of

this model is also very encouraging. It suggests that

the mean quality may be maintained in this interval if

the quantitative changes are monitored accordingly.

5 Conclusions

The relation between the growth of quality vs growth

in quantity can be modeled in the manner

demonstrated above. For both the models shown

above, we may safely conclude based on Kendall’s

coefficient values as seen in both models. Both

Spearman’s and Kendall’s coefficients do not rely on

any assumptions and can be used for drawing these

relationships. The second model has passed the test

assumptions of a linear regression model. Hence, we

can use it. The correlation coefficient in this model is

significantly high. The P-values are much lower than

0.05. This is true for both cases – the OLS regression

without a constant and with a constant. Due to this, we

can reject the null hypothesis and accept the alternate

hypothesis. We therefore conclude based on the

second model that there exists a strong positive linear

relationship between growth in quantity and growth of

quality of collaborative software projects within a

suitable range of values as presented by the model.

may suggest that the quality of the software project

may be estimated or even predicted by measuring the

growth in quantity. This may lead to the development

of newer software development models so far as the

open-source collaborative software world is

concerned. There may be situations where the

direction of development may be controlled based on

such quality vs quantity mathematical models.

Supervised learning of such types also may give us an

insight into the nature of the changing world of

Quantity and quality have been defined based on four

GitHub events, namely – pullRequest, issue, push, and

release. However, there are many other events in the

lifecycle of open-source collaborative projects.

Interpretation of such other events may lead to more

interesting results. This paper has discussed two

important models. The objective has been to

demonstrate the technique of mathematical modeling

in this field. Further situations may be modeled and

used to arrive at conclusions. A particular range of

Source Code Changes on Software Run

Behaviour. Journal of Software Engineering

and Applications, 2013, vol. 6, no 4.

Exploring the Influence of Identifier Names on

European Conference on Software maintenance

and Reengineering, Madrid, Spain, 2010.

4503-2863-0.

it? A large-scale investigation of issue trackers

Symposium on Software Reliability Engineering

(ISSRE). Nov. 2013 pp. 189-196.

Quality Analysis of Open-Source Software.

Social Informatics Springer, Cham, Nov. 2014

pp. 81-89.

Perils of Mining GitHub. Proceedings of the

Repositories MSR 2014 ACM, pp. 92-101.

GitHub: Why Commit Stops Exploring the

Relationship between Developer’s Commit

Pacific Software Engineering Conference, Vol.

2 Dec. 2013.

Date: April 30, 2022).

Theory of Evolution. V. On the Reconstruction

of the Stature of Prehistoric Races. London:

Philosophical Transactions of the Royal Society

of London, 1898. First Edition.

[14] Chen P. Y., Popovich P.M., Correlation,

Parametric and Non-Parametric Measures. A

Sage University Publication, 2002. --14

[15] Bobko P., Correlation and Regression