Ozone Day Classification using Random Forests with Oversampling

and Statistical Tests

HYONTAI SUG

Department of Computer Engineering,

Dongseo University,

47 Jurye-ro, Sasang-gu, Busan, 47011,

REPUBLIC OF KOREA

resulting in the 72 attributes being reduced to 20 attributes, and generating slightly better random forests, but

the classification for ozone days is still poor due to insufficient data. To solve the insufficient data problem for

the minor class which is 6.3% of the total, SMOTE which is one of the representative oversampling methods is

applied to a minor class at very high rates repeatedly. It was also checked whether a better machine learning

model of random forests can be obtained after applying oversampling at the same very high rate for each class,

generating much more synthetic data than the original data and using it to train the random forests. In addition,

to ensure the reliability of the synthetic data generated by SMOTE statistical test has been done for each

attribute to see if it is statistically reliable. The results of the experiment showed that when the oversampling

rate was relatively high with the suggested oversampling and statistical tests, it could be possible to generate

synthetic data with statistical characteristics similar to the original data, and by using it to train the random

oversampling, box plot, t-test, random forests.

Received: June 26, 2024. Revised: November 11, 2024. Accepted: December 10, 2024. Published: December 30, 2024.

1 Introduction

At the top 20 kilometers of the Earth's atmosphere,

the ozone layer surrounds the Earth. The ozone

layer absorbs ultraviolet rays from the sun and

prevents adverse effects on life on Earth, so it is

oxygen molecules are combined with oxygen in the

air. Ozone, which is produced near the earth's

surface, can irritate the respiratory organs, irritate

the eyes, and pose a health risk to respiratory

patients. Normally, the ozone concentration on a

normal day is maintained at the level of 0.02 ppm,

the dataset might have multicollinearity between

attributes and contain several irrelevant features

among the 72 conditional features in the original

dataset, and there are some missing data also which

is common in real-world data. Therefore, finding

highly accurate knowledge models for the dataset is

a challenging task, and we may expect that the

dataset may offer a chance to explore new data

mining techniques, and to provide guidance for

similar problems.

such as the ozone dataset.

In addition, in classification problems, when the

number of data instances markedly differs by class,

it is common to apply oversampling to ensure that

minority classes are not discriminated against

classification. Meanwhile, it is generally true that

the larger the high-quality data used for training, the

higher the accuracy of knowledge models by

machine learning algorithms can be. Considering

that oversampling can generate more synthetic data

ozone dataset.

From now on section 2 covers related work,

section 3 deals with problem formation, section 4

covers experimentation, and section 5 presents the

conclusion.

Ozone day prediction analysis in the original paper

was performed with a precision-recall curve only by

recall = TP/(TP+FN), where TP stands for the

number of True Positives, FP stands for the number

of False Positives, and FN stands for the number of

the proportion of samples that are never sampled out

of n samples is about 36.8%. In other words, when

each decision tree is generated, some data is

sampled several times and some data is not sampled

at all, so only about 63.2% of the total data is

randomly selected to generate the decision tree. As a

result, as each decision tree is generated slightly

differently for the overall data, it ends up with a

slightly biased decision tree, so that it can be a

knowledge model that represents the data from a

for each subtree, and no pruning is performed. The

default value for the size of the set is INT(log2(the

number of attributes) + 1) or one can give an

appropriate size of the set. By randomly selecting

the root attribute of each subtree, we may avoid

local optimization due to the greedy search. Random

forests are known to be one of the most reasonable

machine-learning algorithms across a wide range of

data, [10]. Other factors that affect the performance

of random forests are the size and dimensionality of

Random forests are likely to be more accurate

with a larger number of samples, so when a given

number of data is fixed and not large, oversampling

can be a means of obtaining a large amount of data.

give more attention to training a machine learning

model, [13]. Because simple oversampling increases

the likelihood of overfitting by introducing

replicated samples, oversampling based on synthetic

samples was invented. SMOTE(Synthetic Minority

Over-sampling Technique) is one of the

representative oversampling methods that generate

synthetic instances of a minor class. SMOTE selects

k-nearest neighbors where k can be given by users,

variance, if the significance level p-value is greater

than 0.05, then equal variance can be assumed so

that Student’s t-test will be applied. If the p-value is

less than or equal to 0.05, then equal variance

cannot be assumed, so Welch’s t-test will be applied.

If we use IBM SPSS for the t-test, [17], the top line

shows the t-test result when it is equal variance, and

the bottom line shows the t-test result when it is not

equal variance. Finally, in the t-test, the smaller the

p-value, the more significant the difference between

open-source tool called Weka will be used. [19].

When principal components analysis and

transformation of the data are performed, a ranker

search based on eigenvalues is adapted in Weka.

Dimensionality reduction is accomplished by

choosing enough eigenvectors to account for some

percentage of the variance (default = 0.95) in the

original data.

Note that 2374 records represent no ozone day

and 160 records represent ozone day, so the trained

model using the original dataset cannot classify

ozone day at all. It just classifies ‘no ozone day’

with an accuracy of 93.6464%.

4.1.1 Principle Component Analysis

Because the original data set contains many similar

attributes, principle component analysis is

performed. The principle component analysis

module in Weka is used with an attribute ranker

the ozone data set

The above attributes are the abbreviations of the

Table 4. Mean and SD of each attribute for each

class

Figure 2 shows the box plot of attribute values

class depending on the rank of the attribute by PCA.

Table 5 shows random forests for the

transformed data of ozone data by PCA.

Table 5. The result of random forests for the

transformed data of the ozone data set by PCA

4.1.2 Progressive and Repetitive Oversampling

The target data of oversampling is the data

generated by the PCA. The following is the

progressive and repetitive oversampling procedure.

highest number of misclassifications for the

next oversampling in the random forests of

the original data;

highest number of misclassifications

for the next oversampling in the 10-

the original and oversampled data

together and test based on 10-CV;

oversampling when oversampled data are used for

training and the original data are used for testing for

the ozone data set

Table 9. The result of random forests of the ozone

data set with the equal oversampling rate of 800%

for each class when oversampled data only is used

oversampled data of the oversampling rate of 800%

and 7200% are used for box plot, and 7200% are

used for t-test.

the original data, the data of 10-fold CV and

oversampled only at the oversampling rate of 800%,

and the data of 10-fold CV and oversampled only at

the oversampling rate of 7200% from left to right

Because p=0.406 which is greater than 0.05, we

Because p=0.373 which is greater than 0.05, we

We can see the oversampling generated the data of a

slightly narrow distribution.

A t-test for the mean was carried out on the data

7200%. From the test equal variance was not

assumed, because F=10.222 with

significance=0.001 for the data. Table 14 shows the

result of the t-test for the oversampled data at the

oversampling rate of 7200%.

Table 14. The result of the t-test for the oversampled

Because p=0.368 which is greater than 0.05, we

can see that the difference in the means is

statistically insignificant, so we can say that the

oversampling rate of 7200%.

4.1.3.4 Statistical Test for the 4th Attribute

We can see the oversampling generated the data of a

attribute