An Oversampling Technique with Descriptive Statistics

HYONTAI SUG

Department of Computer Engineering,

Dongseo University,

47 Jurye-ro, Sasang-gu, Busan, 47011,

REPUBLIC OF KOREA

Abstract: - Oversampling is often applied as a means to win a better knowledge model. Several oversampling

representative dataset called wine data in the UCI machine learning repository, which is one of the datasets that

has been experimented with by many researchers in research for knowledge discovery models. I generated

synthetic data iteratively using SMOTE, and I compared the synthetic data with the original data of wine to see

if it was statistically reliable using a box plot and t-test. Moreover, since training a machine learning model by

supplying more high-quality training instances increases the probability of obtaining a machine learning model

with higher accuracy, it was also checked whether a better machine learning model of random forests can be

obtained by generating much more synthetic data than the original data and using it for training the random

forests. The results of the experiment showed that small-scale oversampling produced synthetic data with

statistical characteristics that were statistically slightly different from the original data, but when the

oversampling rate was relatively high, it was possible to generate data with statistical characteristics similar to

box plot, t-test.

Received: July 24, 2023. Revised: May 11, 2024. Accepted: June 24, 2024. Published: July 17, 2024.

1 Introduction

Discovering knowledge models with high accuracy

on a given dataset is one of the most important

issues in the field of machine learning and data

mining. Since the accuracy of a machine learning

data, so, to encourage researchers to find the best

machine learning algorithm for each data set, there

are several public data sites, and the UCI machine

learning repository, [2], is one of the most

prominent sites. The site has several data sets and

related research results that have been done using

those data sets. Among the data in the UCI machine

learning repository, the wine data attracted

particular attention because we can achieve the high

accuracy of several machine learning models even

reported to have a high accuracy of 95%±α, [3]. On

the other hand, according to experiments, some

machine learning algorithms do not have a high

accuracy for the data. For example, locally weighted

learning (LWL) has an accuracy of 88.8%, while

zeroR has a low accuracy of 39.9% in 10-fold cross-

validation. If the quality of the data is good and the

for training, the higher the accuracy of machine

learning algorithms. Of course, the data supplied for

this must be of good quality and may represent the

entire population well. To obtain more training data,

fields such as image recognition make it easier for

humans to label new image data that can be added to

train machine learning models because it makes it

easier for humans to recognize what an object is,

[4], while data from science and social experiments

rely on statistical methods because it is difficult for

can generate more synthetic data than the original

data, we want to see if feeding a lot of synthetic data

into the training of a machine learning model can

lead to a better machine learning model. Of course,

we want to conduct a statistical analysis of how

statistically reliable such training data is, because

until now, the oversampled data has been used

conventionally to train machine learning models

without statistical analysis, so it is not certain that

the machine learning models will be fine for unseen

Wine is one of the most consumed alcoholic

beverages, and as a result, the analysis of wine-

related public data has attracted the attention of

many researchers. For example, there are two kinds

of wine data on the UCI machine learning repository,

one called ‘wine’ and the other called ‘wine quality’.

Of these two types of data, we note that it is worth

oversampling for ‘wine’ data that is relatively small-

given, and neural networks and support vector

machine classifiers were made to generate accurate

classifiers achieving the accuracy of 94.4% ~ 97.8%

in 5-fold cross-validation, [6]. An ensemble-based

method called the ensemble learning method for

spectral clustering reported achieving a performance

of 75.8% in clustering, [7]. Graph neural networks

achieved an accuracy of 97.5% to 98% in 10-fold

cross-validation for the data set, [8]. The multi-

output neural tree that has a tree structure whose

nodes are neural achieved an error rate of 15% to

16% where 20% of the data is used for testing, [9].

may rely on oversampling. Simple oversampling

oversamples a specific class to give more attention

simple oversampling increases the likelihood of

overfitting by introducing replicated samples,

data instance by multiplying an interval value of

continuous attributes of the k-nearest neighbors with

a random number between 0 and 1, [12]. There are

several variant algorithms of SMOTE. Among them,

borderline-SMOTE and ADSYN (Adaptive

Synthetic Sampling Approach) may be two

representatives. Borderline-SMOTE uses a synthetic

data generation oversampling method that focuses

on the hard-to-classify parts of the sample by

increasing the number of samples that are borderline

selection of root attributes for each subtree and no

pruning is performed, and classes are determined by

majority vote by the decision trees in the forest so

that overfitting can be avoided. The most important

property of random forests is that it has randomness

when creating the trees that make up the forest so

that the prediction of each tree is de-correlated with

each other, and as a result, the generalization

performance is improved. Random forests are

known to be one of the most reasonable machine

dimensionality of the data affects the size of the

sample required to generate good random forests,

[19].

oversampling as a method to improve the

knowledge model has been studied, and there are

two methods of oversampling; Simple oversampling

and synthetic data-based oversampling like SMOTE.

In this paper, we want to apply SMOTE to generate

new synthetic data to avoid overfitting, and by

identifying the statistical properties of the new

synthetic data concerning the original data, we aim

same population.

whether the difference in the mean values between

two groups is statistically the same or different. In

other words, in a two-sample t-test, the t-value is the

mean difference between the two groups divided by

the mean standard error. Before the t-test, the equal

variance test will be performed first, and the reason

for the equal variance test is that it can indirectly

confirm whether the target of statistical analysis is

than 0.05, then equal variance can be assumed so

that Student’s t-test will be applied. If the p-value is

less than or equal to 0.05, then equal variance

cannot be assumed, so Welch’s t-test will be applied.

If we use IBM SPSS for the t-test, [21], the top line

shows the t-test result when it is equal variance, and

the bottom line shows the t-test result when it is not

equal variance. Finally, in the t-test, the smaller the

p-value, the more significant the difference between

the two groups is, so the p-value of 0.05 or less is

the test results of the original data.

grapevines. Total phenols in wine are important in

estimating the taste and health benefits of wine.

Flavonoids are a type of antioxidant that is high in

red wines. Nonflavonoid phenols can include

several subclasses of importance to wine.

Proanthocyanins are condensed tannins and the most

abundant class of phenols in wine. OD280/OD315

of diluted wines determine the protein content of

various wines. Proline is an amino acid in wine.

Table 2 shows random forests for the original

Table 2. The result of random forests for the original

highest number of misclassifications in the

confusion matrix was repeated until there was no

further improvement in misclassification using

SMOTE. Default parameters of SMOTE of k = 5

which is the number of nearest neighbors and seed =

1 which is the seed for a random number between 0

to 1 were used. The misclassification in 10-fold

cross-validation is considered when original data

and oversampled data are used together for training

and testing, and also the number of

misclassification in the test with the original data

still needs to be improved, we do oversampling

further. Table 3 and Table 4 show the results of the

experiment, and Table 3 shows the result of the

experiment using 10-fold cross-validation.

Table 4. The corresponding result of progressive

oversampling when oversampled data are used for

training and the original data are used for testing for

Statistical tests are done to see the properties of

oversampled data for each attribute. Data at

iterations 4 and 9 are tested because all three classes

are oversampled for the first time at iteration 4 and

iteration 9 is the final iteration.

oversampled data. We can see the final

oversampling generated the data of a similar but

slightly narrow box. The oversampling at iteration 4

generated lower Q1(the first quartile), lower Q2(the

second quartile), lower Q3(the third quartile), and

lower mean.

original data, the data of 10-fold CV and

oversampled only at iteration 4, and the data of 10-

fold CV and oversampled only at iteration 9 from

left to right respectively.

synthetic data for the attribute alcohol at iterations 4

and 9. From the test equal variance was assumed,

because F=5.136 with significance=0.24 for the data

at iteration 4 and F=3.307 with significance=0.69

see that the difference in the means is statistically

significant, so we can say that the oversampled data

only for attribute alcohol are not good at iteration 4.

Table 6 shows the result of the t-test for the

oversampled data at iteration 9.

Table 6. Result of the t-test for the oversampled data

at iteration 9 for the attribute alcohol

can see that the difference in the means is

statistically insignificant, so we can say that the

oversampled data for attribute alcohol are good at

iteration 9.

4.1.2.2 Statistical Test for Attribute Malic Acid

Figure 2 shows the 5 box plots for the attribute

the data at iteration 9, we cannot assume equal

statistically insignificant, so we can say that the

can see that the difference in the means is

result of the t-test for the oversampled data at

fold CV and oversampled only at iteration 9 from

can see that the difference in the means is

statistically insignificant, so we can say that the

can see that the difference in the means is

statistically insignificant, so we can say that the

oversampled data only for attribute alkalinity of ash

are good at iteration 9.

4.1.2.5 Statistical Test for Attribute Magnesium

fold CV and oversampled only at iteration 9 from

synthetic data for the attribute magnesium from

iterations 4 and 9. From the test equal variance

significance=0.013 for the data at iteration 4 and

F=8.827 with significance=0.003 for the data at

iteration 9. Table 13 and Table 14 summarize the

results. Table 13 shows the result of the t-test for the

oversampled data at iteration 4.

Table 13. Result of the t-test for the oversampled

can see that the difference in the means is

statistically significant, so we can say that the

oversampled data only for attribute magnesium are

not good at iteration 4. Table 14 shows the result of

the t-test for the oversampled data at iteration 9.

Table 14. Result of the t-test for the oversampled

good at iteration 9.

4.1.2.6 Statistical Test for Attribute Total

Phenols

boundary. However, the oversampling at iteration 4

generated a larger Q1, lower Q2, and lower Q3.

Fig. 6: Box plots for attribute ‘total phenols’ in the

original data, the data of 10-fold CV and

oversampled only at iteration 4, and the data of 10-

fold CV and oversampled only at iteration 9 from

left to right respectively.

15 shows the result of the t-test for the oversampled

data at iteration 4.

the t-test for the oversampled data at iteration 9.

Figure 7 shows the 5 box plots for the attribute

‘flavonoids’. We can see the final oversampling

generated the data of a similar but slightly narrow

box as we can see the Q1, Q3, upper, and lower

boundary. But, the oversampling at iteration 4

generated larger Q1, lower Q2, lower Q3, lower

mean and lower upper boundary.

iterations 4 and 9. From the test equal variance

significance=0.0 for the data at iteration 4. Table 17

shows the result of the t-test for the oversampled

data at iteration 4.

Table 17. Result of the t-test for the oversampled

data at iteration 4 for the attribute flavonoids

good at iteration 4. Because F=1.06 with

significance=0.303 for the data at iteration 9, we can

assume equal variance. Table 18 shows the result of

the t-test for the oversampled data at iteration 9.

can see that the difference in the means is

statistically insignificant, so we can say that the

oversampled data only for attribute flavonoids are

good at iteration 9.

4.1.2.8 Statistical Test for Attribute Nonflavonoid

Figure 8 shows the 5 box plots for the attribute

Fig. 8: Box plots for attribute ‘nonflavonoid phenols’

oversampled only at iteration 4, and the data of 10-

cannot be assumed, because F=14.807 with

significance=0.0 for the data at iteration 4. Table 19

shows the result of the t-test for the oversampled

data at iteration 4.

phenols