Predicting Success for Web Product through Key Performance
Indicators based on Balanced Scorecard with the Use of Machine
Learning
ELENI TAGKOUTA, PANAGIOTIS – NIKOLAOS PSYCHARIS, ALKINOOS PSARRAS,
THEODOROS ANAGNOSTOPOULOS, IOANNIS SALMON
Department of Business Administration,
University of West Attica,
12243 Athens,
GREECE
Abstract: - Machine Learning (ML) can be proved as an important tool in planning better business strategies.
For the purposes of the present study, the prospect for the development of an electronic platform by a
technology firm providing financial services is explored. The purpose of this article is to demonstrate the ways
in which a start-up can predict the success of an online platform prior to its market launch. The prediction is
achieved by applying Artificial Intelligence (AI) on Key Performance Indicators (KPIs) derived from the
customers’ perspective, as shown in the Balanced Scorecard (BSC). The research methodology was
quantitative and online questionnaires were used to collect empirical quantitative data related to bank loans.
Subsequently, KPIs were created based on the collected data, to measure and assess the success of the platform.
The effectiveness of the model was calculated up to 91.89%, and thus, it is estimated that the online platform
will be of great success with 91.89% validity. In conclusion, prediction was found to be crucial for businesses
to prevent a dire economic situation. Finally, the necessity for businesses to keep up with technological
advances is highlighted.
Key-Words: - Artificial Intelligence, Machine Learning, Business plan, Business strategy, Change management,
Balanced Scorecard, Product Success, E-Business, Start-ups, Artificial Neural Networks
Received: January 16, 2023. Revised: February 28, 2023. Accepted: March 14, 2023. Published: March 15, 2023.
1 Introduction
A business strategy is a specific plan with basic
principles about the routes in which a company can
achieve its goals [1]. The importance of a strategy is
to provide businesses with the time to get a sense of
their performance, their capabilities, and their
potential growth. In the digital era, the number of
businesses that provide their services in the form of
web applications is constantly increasing and lately,
many of those appear to have a web presence since
their inception [2]. As a result, the advances that
have taken place in the web technologies have
significantly altered the way that businesses
collaborate and compete [3].
E-business can be defined as information
systems for acquiring, processing, and transmitting
information for more effective decision making, in
relation to competitive standards [4]. On the other
hand, digital networks and communication
infrastructures are also a part of the broader
economic framework that is accountable for the
drastic changes in business that e-business
represents [5]. Due to the growing number of online
businesses, it is becoming more and more
challenging for them to stand out in the web market.
Therefore, it is important to have the right business
strategy because without it, even the best idea
cannot be successfully implemented [6].
In terms of business strategy, start-ups
frequently deal with disruptive technological
developments as they seek to develop innovative
services for emerging markets or enter an existing
market using a new strategic mentality or business
model. It is well understood that start-ups must
continuously learn from customer behavior, because
quick and continuous organizational learning
provides a competitive advantage for them.
Therefore, they should be tactfully dynamic since
they function in a continuously changing and
volatile operating environment. As a result, their
strategic development is a process of learning [7].
As the market environment tends to change
more rapidly than in past decades, numerous
organizations have embraced emerging technologies
created to improve performance and acquired a
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
646
Volume 20, 2023
competitive advantage. AI holds a key role in these
advancements and has caught the interest of both
academics and the industrial sector [8].
Data is being collected in massive amounts,
various forms and faster than ever before. This has
resulted in the emergence of new technologies,
culminating in the acceleration of technological
developments that include computational processing
capabilities, as well as the development of new AI
methods [9].
As a result of the exponential growth in online
platforms occurred in recent years, some of them
utilize ML technologies to improve the services they
provide and to support the decision-making process
of the individuals using them [10].
Change management is crucial for a company's
survival, operation, and success in a competitive
market in light of the substantial change that the use
of AI has brought to the practice of decision-making
in businesses. To accomplish the objectives it has
set, a company should be able to alter its
management and organizational structure as needed
[11]. The BSC, which outlines the means for a
coherent strategic plan that will lead in the success
of the business, is a methodology used in modern
businesses for change management and, by
extension, decision making. More specifically, it
takes into account four factors: 1) Financial, 2)
Customers, 3) Internal processes and 4) Learning
and innovation [12].
Thus, in order to describe the key components
of these four aspects of a business, a BSC employs
an array of measurements, ratios, indexes of
significant accomplishments, and target goals [13].
As a result, the importance of the BSC is about
determining the company strategy, breaking it
down, and implementing it, all of which are critical
phases for a startup looking to introduce new
products [14].
KPIs are indicators used by organizations to
measure, manage, and compare their performance.
These indicators can focus on a variety of aspects
and can be implemented to monitor monetary
efficiency, customer satisfaction and overall
operational quality, among others [15]. Thus, this
study is using KPIs from the customers’ perspective
of the BSC in order to estimate the success of a new
product. Specifically, the estimation was achieved
by applying an Artificial Neural Networks (ANNs)
algorithm using the Python programming language.
2 Problem Formulation
In more detail, this paper examines the success of an
online product (financial services platform) using
KPIs and an ANN model to determine whether it is
likely for a potential customer to use the platform.
The data collection was carried out using a
questionnaire. The sample was obtained using
snow-ball sampling (by sharing it on social media),
in an effort to be as large and representative as
possible. Usually, in snowball sampling, researchers
contact a small number of individuals who then pass
the questionnaire along to their own group of
people, and so on. Furthermore, the questionnaire
that was created consisted of two (2) groups of
questions: 1) demographic questions and 2)
questions related to the platform's scope of services.
The purpose of this article is to demonstrate how a
start-up can estimate the KPI that is responsible for
the success of an online platform prior to its market
launch. The estimation is performed by utilizing AI,
using the KPIs derived from the customers’
perspective of the BSC.
The contribution of this research to the literature
focuses on how ML can be used to design a strategy
for optimal decision making about a new web
product. For the purposes of this study, the potential
development of an online platform by a technology
company providing financial services is explored.
In this paper, the results for the estimation of the
KPI were extracted after the literature review and
quantitative analysis using ML with the use of an
ANN model. Then, some comments were made, and
the research's scope and future steps were also
mentioned. Finally, conclusions were drawn.
3 Literature Review
3.1 Change Management and Balanced
Scorecard
According to [16], the advancement of technology
has resulted in a highly competitive global market.
This has brought changes in the management of
businesses where companies are no longer
concerned only with their production and financial
investments but also with the quality of their
products in order to differentiate themselves from
similar companies. Therefore, there is an obvious
need for product evaluation in a business to ensure
that the product is profitable.
On the other hand, [17], in their article on
measuring the performance of public administration
bodies, mention the importance of applying not only
financial indicators but also non-financial indicators
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
647
Volume 20, 2023
in the analysis models. This led them to the
conclusion that the BSC is the most effective tool
for measuring a company's performance, especially
when it is tailored to the specifics of the
organization. Thus, it is clear that the appropriate
use of BSC should be carried out following
sufficient research pertaining to its four aspects
mentioned in the previous section.
Through their investigation into the
Multicriteria Decision Aid WAP method's
integration into the BSC, [13], arrived at the
conclusion that this method's application is highly
beneficial for a business due to its many advantages.
This research employs both the theoretical
framework of change management and
mathematical models via linear programming. It is,
thus, important, because applying a theory via
models validates the theory.
According to [18], the application of AI in
business has revolutionized decision making.
Particularly in their article they make use of the
BSC tool in combination with an ANN model to
analyze the available data of a co-financed European
Union program resulting in optimal decision
making.
The necessity of managing internal company
changes is discussed by [19]. In particular, they
claim that a company finds it difficult to adapt to
changes that take place within the organization
because the environment in which it operates is
altering quickly and unpredictably. These changes
have both positive and negative effects, so managers
should make the necessary adjustments in the
business to maximize positive results while
minimizing negative ones. Therefore, for the
business's overall well-being, change management
should offer the most appropriate balance between
positive and negative changes.
Similarly, in their article, [20], describe a study
they conducted that identifies the factors that
influence the management of a change in a
particular firm. They conclude by generalizing that
the factors that influence change management need
further research by managers and are the ones that
contribute to the implementation of effective change
in a company. Consequently, prior to
implementation, change management in a business
requires extensive research and pilot testing.
The research conducted by [21] that studies the
ways to use a BSC in planning the launch of a new
product, is also important. With a new product in
mind, they examine all four aspects of the BSC.
According to them, in the financial perspective, a
company can assess not only the effectiveness of all
the perspectives but also the effectiveness of
previous strategies to achieve economic success. It
also pursues to define the financial performance of
revenue growth and cost-cutting processes. The
Internal perspective is used to identify and better
satisfy customer demand through innovation and
process improvement, as well as to follow up with
excellent customer management service.
Additionally, they demonstrate how effectively
business resources are used to provide value to
customers. The Customers’ perspective evaluates
the degree of target market penetration. The level of
customer performance underlines how much the
customer market and service have improved.
Organizations must enhance their new product
development process and product quality, modify
their products to customers' requirements, hasten the
commercialization process, and stay one step ahead
of their rivals if they are willing to continue growing
further. Finally, the Learning and Innovation
perspective is related to management of routine
processes, employee training, and skills
development. In other words, to align them with the
strategic goals of the organization, this perspective
focuses on internal skills and capabilities. This
element is essential for the development of new
products and serves as the foundation for
management. To use their minds and creative
abilities to accomplish organizational and customer
goals, employees must be motivated.
The importance of coordinating people,
processes, and technology to support strategic goals
is acknowledged in the study performed by [22].
Therefore, organizations should consider making
investments in coordinating these three aspects that
encourage learning and development.
3.2 Business Management with AI
[23] describe in great depth the ML benefits in
business. They discuss how both ML and deep
learning affect positively a company's increased
output, customer retention and growth in the context
of today's rapid technological advancements. They
also mention the difficulties analysts face and their
inability to interpret the outcomes of the algorithms.
Therefore, ML algorithms are viewed as a tool that
aids in better managing the various issues that may
arise in business by speeding up analysis and
increasing output.
As demonstrated by [24], the abundance of services
deriving from the application of ML opens new
opportunities for many technology companies and
contributes to the quick expansion of these types of
web platforms on the internet. In particular, it is
stated that AI is part of many companies today,
facilitating business activities, increasing
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
648
Volume 20, 2023
productivity and offering a variety of ways to speed
up communication processes. Additionally,
automation using AI systems carries many of the
tasks that were once performed by employees and
results in company’s reduced costs, significant time
savings and a gradual increase in profits [25].
In addition to ML, a subfield of AI is Deep
Learning. Deep Learning is based on ANNs, which
consist of a large number of simple units, called
neurons, that exchange signals with each other and
form a model used to solve computational problems,
[26]. Due to its ability to process information in a
similar way that humans acquire certain types of
knowledge and the higher accuracy it offers
compared to conventional ML methods, it was
chosen to be used in this research work. The Deep
Learning method with an ANN that was used in this
paper is based on the Supervised Learning approach.
In this approach, a computer algorithm is trained
using labeled data as input in order to accurately
classify data or predict outcomes [27].
Moving on to the role of KPIs in this study, [28]
mention the identification of KPIs in project-based
organizations based on their organizational and
functional needs. Their research is primarily
concerned with the categorization of KPIs using a
qualitative approach to project success.
4 Research Methodology
4.1 Balanced Scorecard
For the change management of the technology
company in the example of this research, a BSC, as
shown in Figure 1, was constructed including the
four aforementioned aspects.
Fig. 1: The balanced scorecard.
Source: Authors
Additionally, the information for these four aspects
is summarized in Tables 1, 2, 3, and 4. The four
KPIs, which are the measures of the customer’s
perspective of the BSC (KPI2, KPI3, KP4 and
KPI5), were used with the combination of another
key performance indicator (KPI1) that was
constructed from the quantitative survey and
estimated. The KPI1 ultimately determines whether
the online product will be a success, allowing the
business strategy and its development to proceed.
Specifically, KPI1 is the dependent variable y, and it
is the one that is estimated. Furthermore, KPI1
demonstrates the success of the web product and
indicates the Percentage (%) of potential users of the
platform, with a range [1, 5], (1: Not at all, 2: Not
Really, 3: Undecided, 4: Somewhat, 5: Very much).
Also, KPI2 up to KPI5 are the four independent
variables X2,…,X5. All of the five KPIs are
presented in Table 5.
Moreover, the BSC initially starts from the
objectives which represent the strategic targets. On
the basis of that, the corresponding measures that
quantify the objectives emerge. Next to that, the
targets pertinent to the goals that need to be
achieved are listed, and in the rightmost part, the
steps that the business will take to accomplish the
strategic goals are presented [18]. Specifically,
Table 1 contains information relevant to financial
goals, Table 2 showcases the customer’s goals,
Table 3 is about goals relevant to internal process
and Table 4 lists the learning and innovation goals.
All of these perspectives have been developed with
an understanding of the requirements of a company
that strives to be successful and remains competitive
over time.
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
649
Volume 20, 2023
Table 1. The financial perspective.
Objectives
Measures
Targets
Profit
increase
Earnings
ratio
Finding net
profits
Reduction
of
operating
costs
Average
cost
Reduction
of operating
Costs /
Expenses
Cash
liquidity
Sales
growth
rate
Acquisition
of cash /
Equity
capital
Increase in
market
share
Market
share
index
Increase in
market
share
Table 2. The customers perspective.
Objectives
Measures
Targets
Actions
Customer
retention
Total
number of
users of
bank
websites
Maintaining
customer
trust
Frequent update
of
product/website
Customer
satisfaction
Total
satisfied
and not
satisfied
citizens
who have
used bank
websites
Increase
customer
confidence
Direct line of
communication
via chat, email,
phone etc.
Customer
knowledge
related to
the
product
Proportion
of their
education
level and
computer
literacy
Introducing
the product
to potential
customers
Product related
campaigns
Financial
comfort of
customers
Total
employees
per
household
Customer
segmentation
Targeted
marketing
Table 3. The internal processes perspective.
Objectives
Measures
Targets
Actions
Research
and
innovation
Number
of
product
creation
methods
Support for
scientific
research
Participation in
research
products
Product
quality
control
Quality
control
index
Ensuring
the quality
of the
products
Research and
development
Cooperation
of
departments
Internal
problem
resolution
index
Correct
operation of
the
departments
Network
modifications
for enhanced
communication
between
departments
Productivity
Increase
Number
of
working
hours
spent /
task
Personnel
training
A merit-based
evaluation
system
Table 4. The learning and innovation perspective.
Objectives
Measures
Targets
Actions
Employee
satisfaction
Employee
satisfaction
score
Reduction of
employee
attrition
Assessment
of personnel
needs
Continuous
training of
employees
Staff
productivity
index
Highly
skilled
colleagues
Business
tools
training
program
Business
process
development
Internal
operations
index
Smooth
operation of
all
departments
Investigation
of needs in
each
department
separately
Personnel
performance
Personnel
efficiency
index
Staff
performance
improvement
Staff training
program and
assessment
of their
competences
4.1 Methodology and Dataset
The data used for this survey is anonymous personal
data on bank loans and was collected with the use of
a questionnaire that is presented in Appendix A. For
its proper structure and design, bibliographic
research was considered necessary. Specifically,
[29], states that a questionnaire is regarded as a tool
for gathering data for an analysis. Therefore, when
designing it, researchers should consider the larger
context in which the questionnaire will be used. For
instance, the survey's goals, the questions that will
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
650
Volume 20, 2023
be asked, the number of participants, the study
sample, the delivery and completion methods, as
well as the format design, which should adhere to
international standards. Based on this, it is clear that
the design of a questionnaire should be of top
priority in order to avoid discouraging respondents
and also ensuring that the data collected is as
suitable as possible for analysis.
To interpret the analysis performed on the
gathered data and how they can improve a business
promoting a platform of this kind, the research
conducted by [30] was utilized. Their findings cover
general strategies and recommendations as well as
the foundational ideas for business problem
analysis, with an emphasis on data and the
development and evaluation of solutions.
The findings of [31] cover both theory and
applications of the survey and were used for the
proper organization of the questionnaire, which was
essential for the collection of appropriate data. They
address issues regarding data collection, analysis,
sampling, questionnaire design and statistical
estimation. Consequently, the research part of the
questionnaire was based on this book, which focuses
on conducting sampling and collecting data to create
the KPIs.
Furthermore, the methodology that is used is
empirical research, which means that the theory is
linked to reality [32] and is therefore inductive. It is
also a case study. Then, a quantitative analysis was
performed using ML, specifically by applying an
ANN model with the use of Python programming
language. Afterwards, the results were extracted to
estimate the KPI1.
In particular, the data were obtained using
online questionnaires, where the number of
completed questionnaires is 122 and each of them
consists of 32 questions. Thus, for their analysis, the
original raw data were arranged in a table with
dimensional structure of (122, 32). The sampling
method used is Snowball sampling and the majority
of the sample's basic demographic characteristics
are people who are adequately educated, have a
moderate income, and have fairly good computer
literacy.
Following the data collection, the raw data were
transformed in order to create KPIs. First, they were
interpreted from text data to numerical data with the
scope to analyse them quantitatively. Then the data
were combined and thus created the five KPIS
(KP1, KPI2, KP3, KP4 and KP5) which are
presented in Table 5 below.
Table 5. The five KPIs of the research
Afterwards, the data were separated into
Training data (70%) and Test data (30%), using the
empirical rule [33]. The separation process was
carried out in order for the algorithm to learn the
characteristics of the data from the model (Model
Training) and then for verifying the effectiveness of
the model using the test data (Model Testing).
The next step was the encoding of the
dependent variable y (KPI 1), which involves
converting the codes of the classes: (1: Not at all, 2:
Not Really, 3: Undecided, 4: Somewhat, 5: Very
much) from the number of each category to a
specific code that can be used in the algorithm,
consisting entirely of zeros and ones. On the other
hand, the independent variables Xi, ..., XN, with i =
2 and N = 5 (KPI 2 – KPI 5) were normalized, i.e.
they were transformed in a way that centers them
around zero in order to follow the normal
distribution (Gaussian) with zero mean and unit
variance within the range [0, 1].
The next step was to develop an ANN model
using the training data and the appropriate
parameters, which are given in Table 6 and detailed
in paragraph 3.3.1. The model was then trained in
order to learn the characteristics of the training data.
KPI
Objectives
Measures
Variable
KPI 1
Product
success
Percentage
(%) of
potential
users of the
platform
Dependent
variable y
KPI 2
Customer
retention
Total
number of
users of
bank
websites
Independent
variable X2
KPI 3
Customer
satisfaction
Total
satisfied and
not satisfied
citizens who
have used
bank
websites
Independent
variable X3
KPI 4
Customer
knowledge
related to
the product
Proportion
of their
education
level and
computer
literacy
Independent
variable X4
KPI 5
Financial
comfort of
customers
Total
employees
per
household
Independent
variable X5
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
651
Volume 20, 2023
Finally, the ANN model was evaluated with the use
of testing data and the confusion matrix tool. As a
result, the overall effectiveness of the ANN model
was calculated, and the misclassifications—areas in
which the model is confused between the classes—
were identified.
4.3 Model
This research addresses the contribution of ML in
designing a strategy for optimal decision making.
This occurs as a result of ANNs' ability for pattern
recognition and their strength to imitate the neural
networks in the human brain. This procedure is
currently a lot more efficient thanks to faster
computer processing [34].
In terms of structure and function, the abstract
model of an ANN, which is called a perceptron
model in its general form, is made up of neurons
which are also known as nodes. Nodes can receive
information from the outside world or from other
nodes, transmit it to other nodes, and then extract it
as final result. Thus, three types of nodes can be
distinguished: Input nodes, Hidden nodes, and
Output nodes.
Another part of the structure of ANNs is their edges,
which link nodes to one another. The edge has a
specific weighting depending on the significance
and strength of the connection. The higher the
weighting, the greater the influence a node can exert
on another connected node. Furthermore, bias is
added in the model and a threshold decides the
binary output of the model [35]. If the weighted sum
of the inputs is greater than the threshold, then the
value of the output will be '1', otherwise it will be
'0'. Finally, in order for the output to be within the
specified range [0,1], the activation function is
employed in conjunction with the inputs, after they
have been transformed into a weighted sum to a
certain output according to a set of rules.
A supervised learning algorithm called a multi-layer
perceptron was used in this study. The key
distinction between it and the simple perceptron is
that this model's hidden layers are non-linear, and as
a result, the multi-layer perceptron is non-linear.
Also, the multilayer perceptron uses the
backpropagation technique to iteratively change the
network's weights [36].
4.3.1 Experimental Parameters
The total number of records in this study was 122.
Therefore, the ANN was decided to have four layers
as according to the literature, the less data a problem
has, the fewer levels it should have. The first layer is
the input layer, the second and third are the two
hidden layers and the fourth is the output layer.
Furthermore, in this research there are five
categories in the output, i.e. it is a multiclass
classification. In Figure 2 below, the structure of the
ANN of this research is illustrated in a diagram.
Fig. 2: Structure diagram of research’s levels.
Source: Authors
In terms of functions, two activation functions were
used initially, the ReLu function and the Softmax
function. The choice of the former was made since
this function allows the models to learn faster and
perform better. According to the literature it is
commonly used as an activation function not only
for the input layer but also for the hidden layers
[37]. The second function was chosen because it is
appropriate for categorical variables with more than
two categories and is commonly used for the output
layer in the literature [38]. In the first three layers
(the input layer and the two hidden layers), the
ReLu function was used, while in the Output layer,
the Softmax function was used for the reasons
mentioned above.
Another important element of the Neural Model
is its type. The type of model chosen is the
Sequential model. The Sequential model was
applied in this paper in order to insert data into the
ANN model sequentially. This means that the output
data from one layer is introduced as input data to the
next layer. According to the literature this type of
model is used for classification problems [39].
The Loss function is another function used in
this ANN model for the purpose of measuring its
performance. The Loss function used in this study is
the Categorical Cross Entropy function, which was
chosen because it is applicable to multi-class
classification problems.
In order to be able to measure the performance
of the ANN model through the Performance
Function, an Optimizer algorithm is applied. In this
case the 'Adam' Optimizer was used.
Another parameter applied is the epochs, i.e. the
number of iterations of the training of the ANN
model. According to the literature, there is no fixed
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
652
Volume 20, 2023
number of epochs that will give the optimal model,
and thus tests on the ANN must be performed to
determine the ideal number [40]. In this paper, after
several tests, the Optimizer was given 1000 epochs
or iterations, to find the minimum point. This
parameter along with Training data and Test data
were used during the Model fitting of the ANN.
Table 6 summarizes all the parameters used in this
research.
Table 6. Experimental neural network parameters.
4.4 Evaluation
To evaluate the model and in particular to find the
correct and incorrect classifications of the categories
of the dependent variable y, the Confusion Matrix
was used. The Confusion Matrix is a tool that
verifies the classification results, i.e., verifies the
overall accuracy of the classification and finds the
correct and incorrect classifications of the variable y
[41].
5 Problem Solution
From the analysis of the data using the ANN, the
following results were obtained. As shown in Figure
3, out of 122 participants, 48 chose option number
five (5), making it the most popular, indicating that
they would use a financial service platform to the
extent of 5: Very much (with a range of 1: Not at all
- 5: Very Much). However, a small percentage of
the population would not use it at all, having
selected the option 1: Not at all (18/122). Moreover,
the mean of 3.8 demonstrates that a majority of
respondents, specifically 7/10, would use the
platform.
Fig. 3: Bar chart with the dependent variable y
Source: Authors
On the Confusion matrix, Table 7, it is observed that
the Overall Accuracy is 91.89%. It is also observed
that in all categories (1: Not at all, 2: Not Really, 3:
Undecided, 4: Somewhat, 5: Very much) the data
was classified with a very high success rate. Also, in
category 4 the most misclassifications are found.
Table 7. Confusion matrix of the neural network
model
6 Discussion
According to the literature, ML models are applied
to automatically analyze data, find patterns, and
estimate them with the ultimate goal of making
decisions about a problem. By developing a better
strategy, the above can help businesses improve
their production and economic growth. Furthermore,
automation frees up more time for analysts to focus
on other aspects of the business plan, allowing them
to make better decisions. It also lowers costs and
improves the customer experience.
In this research work, the evaluation of the
success of an online product using KPIs was carried
out by applying ML methods. Eventually, the ML
algorithm showed a high validity rate regarding
whether the majority of the population interested in
obtaining a loan would use this platform to find the
most suitable one.
Predicted Class
Row
Total
Actual Class
1
2
3
4
5
1
6
0
0
0
0
100
%
2
0
2
0
0
0
100
%
3
0
0
4
0
0
100
%
4
0
1
1
8
0
80%
5
0
0
0
1
14
93.3
3%
Column
Total
100%
66.
66%
80%
88.
88%
100
%
91.89
%
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
653
Volume 20, 2023
Furthermore, the threshold that would determine
whether the new product will finally be created or
not was set to 60%. The specific threshold was
selected on the basis that it is marginally higher than
the average (50%). Given that the findings revealed
that 7 out of 10 people (70%) would use this online
platform, it is deduced that it is worthwhile to
proceed with its development if just over half of
respondents reply favourably to its implementation.
Hence, this study demonstrated that it is preferable
to develop the new platform.
According to this research, since it is necessary
for a company to adapt to changes in its
environment, a company's change management
should be appropriate for the company in question.
This means that on the one hand it should follow
some general market rules but on the other hand it
should be personalized. By having an appropriate
change management and applying the BSC tool, this
can be achieved because this tool presents a
comprehensive overview of how to define, measure
and implement its objectives.
Therefore, the significance of this study is to
apply ML to the survey’s results while viewing
them through the lens of BSC. Generally, the vast
majority of current studies on the subject use BSC
in a conventional manner, i.e. without the use of
ML. In comparison to them, the current study
integrates the ML procedures and the business
component of a BSC in order to contribute to the
success of the new product before its market launch,
which also reduces the financial risk. Also,
companies can use this approach because the study's
findings indicated that machine learning is a useful
tool for predicting various aspects of a business,
especially when combined with the information
contained in the BSC.
A company's change management process is a
crucial component. If a company does not properly
and promptly manage it, it may stray from its
objectives and have disastrous consequences for its
future wellbeing. Given that a company must
effectively coordinate all BSCs’ aspects in order to
achieve its goals, additional information can be
derived from these aspects and applied to improve
its products, especially when it wants to introduce a
new product to the market. As a result, the business
plan will be stronger, but since the market and the
environment are always changing, research and
BSC improvement are integral.
Therefore, given the results of this study, the
use of ML in business has proven to have the
potential to improve many processes by offering
speed, accuracy, and creativity by discovering
patterns in the data used in the analysis. The
combination of these capabilities is particularly
useful for decision making in a business where
many parameters need to be taken into account so as
to make a decision that is personalized and will lead
to the optimal solution for customer satisfaction and
thus increase profits.
Similarly to the majority of the past studies on
the subject, this research also presents some
limitations. The research's sample size is one of
them. Although snowball sampling is an effective
way to gather data, it would be preferable if the
sample size was larger and more random i.e., more
representative. Another limitation is the absence of
cross validation, which evaluates how the prediction
model performs with an unknown dataset, i.e., it
examines the model's ability to generalize to an
independent dataset. Cross validation is a
resampling method that tests and trains a model on
different iterations using different segments of the
data [42] that can be used in future work. Another
task for the future is to perform requirements
analysis of this new product, in order to define the
functional and non-functional requirements that the
product needs to fulfill. Eventually, a future work
that can be done after the product is implemented is
to involve the actual customers of the platform in
this research, not just the potential ones. In addition,
regarding the business plan, it can be further
explored in the other three aspects of the BSC:
financial, internal processes and learning and
innovation.
7 Conclusion
In this research, the BSC was presented in a way
that not only takes into account the most important
changes that should be made in the company, but
also how its use helps to predict and prevent certain
behaviors. This is achieved with the help of AI and
in this case with the deep learning algorithm ANN.
Therefore, the BSC can be combined with
technology and go a step beyond its traditional uses.
From the collected data, five KPIs were created, and
an ANN model was used to classify the categories
of the dependent variable y (KPI1) using the
independent variables Xi (KPI2 – KPI5).
Furthermore, the questionnaires revealed that 7/10
people would use this online platform, indicating
that it would be successful. However, using ML, it
was discovered that with 91.89% accuracy, the
company's product will be successful. Additionally,
category 4 (4: Somewhat) was found to be the
category that the algorithm misunderstood the most,
which is probably due to its shared traits with the
other categories.
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
654
Volume 20, 2023
Furthermore, given that the threshold that would
determine whether the new product would be built
or not was set at 60%, and the results revealed that
70% of people would use this online platform, it is
concluded that the technology firm of this study
would proceed with the development of the new
product. It is also worth noting that the presence of
high accuracy in the ML model indicates that the
development of the new product will have less
financial risk compared to not using ML for this
analysis.
In conclusion and based on all of the findings of this
study, the application of ML in business is very
important. For optimal results, businesses should
strive to combine accurate results interpretation by
qualified analysts and proper data management
techniques that should be applied before building
the models. Finally, it is important for a business to
keep up with technological advances in order to reap
the benefits and gain a competitive advantage.
References:
[1]
Kitsios, F.; Kamariotou, M., “Digital Business
Strategy and Information Systems Planning:
Determinants of Success,” in Paper presented at the
14th European Conference on Innovation and
Entrepreneurship (ECIE19), Kalamata, Greece, vol.
1, 2019.
[2]
Magumba, M. A., “WEB 2.O For Small And
Medium Sized Companies: A practical Case
Study.,” arXiv, p. arXiv:1611.02421, 2016.
[3]
Bakker, E.; Zheng, J.; Knight, L., “Putting e-
commerce adoption in a supply chain context.,”
International Journal of Operations & Production
Management 28(4):, p. 313–330, 2008.
[4]
Ismail, M.; Khater, M.; Zaki, M., “Digital Business
Transformation and Strategy: What Do We Know
So Far? Working Paper,” University of Cambridge:
Cambridge Service Alliance., 2018.
[5]
Borges, M.; Hoppen, N.; Luce, F., “Information
technology impact on market orientation in e-
business.,” Journal of Business Research 62, p. 883–
890, 2009.
[6]
Brzozowska, A., “E-business as a new trend in the
economy,” in International Conference on
Communication, Management and Information
Technology (ICCMIT), 2015.
[7]
Stolz, L.; Sternberg, R., “Do the Winners Really
Take It all? Exploring Entrepreneurial Learning in
Start-UpCompetitions,” Entrepreneurship Education
and Pedagogy, pp. 5: 599-626, 2022.
[8]
Balog, K., “The concept and competitiveness of
agile organization in the fourth industrial
revolution’s drift,” in International Scientific
Conference Strategic Management and Decision
Support Systems in Strategic Management., 2020.
[9]
Božič, K.; Dimovski, V., “Business intelligence and
analytics use, innovation ambidexterity, and firm
performance: A dynamic capabilities perspective,”
The Journal of Strategic Information Systems, p. 28,
2019.
[10]
Weber, M.; Moritz, B.; Weking, J.; Böhm, M.;
Krcmar , H., “AI Startup Business Models: Key
Characteristics and Directions for Entrepreneurship
Research,” Business & Information Systems
Engineering, pp. 64: 91-109, 2022.
[11]
Psarras, A.; Anagnostopoulos, A.; Tsotsolas, N.;
Salmon, I.; Vryzidis, L., “Applying the Balanced
Scorecard and Predictive Analytics in the
Administration of a European Funding Program,”
Administrative Sciences, p. 10(4):102, 2020.
[12]
Kaplan, R. S.; Norton, D. P., “Using the Balanced
Scorecard as a Strategic Management System,”
Harvard Business Review, pp. 74: 75-78, 1996.
[13]
Salmon, I.; Pappas, I. O.; Spyridakos, A.; Vryzidis,
I., “Applying multicriteria decision aid in a weighted
balanced scorecard method for supporting decision
making in change management,” Journal of Applied
Research Review, 2019.
[14]
Sevilla-Bernardo, J.; Sanchez-Robles , B.; Herrador-
Alcaide, T. C., “Success Factors of Startups in
Research Literature within the Entrepreneurial
Ecosystem,” Administrative Sciences, pp. 12, 102,
2022.
[15]
Gilsing, R.; Wilbik, A.; Grefen, P.l; Turetken, O.;
Ozkan, B.; Adali, Onat E.; Berkers, F., “Defining
business model key performance indicators using
intentional linguistic summaries,” Software and
Systems Modeling, pp. 20: 965-996, 2021.
[16]
Quesado, P.; Marques, S.; Silva, R.; Ribeiro, A.,
“The Balanced Scorecard as a Strategic Management
Tool in the Textile Sector,” Administrative Sciences,
p. 12(1):38, 2022.
[17]
Gębczyńska, A.; Brajer-Marczak, R., “Review of
Selected Performance Measurement Models Used in
Public Administration,” Administrative Sciences, p.
10(4):99, 2020.
[18]
Psarras, A.; Anagnostopoulos, A.; Salmon, I.;
Psaromiligkos, Y.; Vryzidis, Y., “A Change
Management Approach with the Support of the
Balanced Scorecard and the Utilization of Artificial
Neural Networks,” Administrative Sciences, p.
12(2):63, 2022.
[19]
Ramosaj, B.; Karaxha, H.; Karaxha, H., “Change
Management and Its Influence in the Business
Environment,” ILIRIA International Review, vol. 4,
no. 2, 2014.
[20]
Errida, A.; Lotfi, B., “The determinants of
organizational change management success:
Literature review and case study,” International
Journal of Engineering Business Management,
2021.
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
655
Volume 20, 2023
[21]
Zhengxiaoming, A.; Fernando, Y., “A Concept
Paper of Balanced Scorecard for New Product
Development,” KnE Social Sciences, 2019.
[22]
Kaplan, R. S.; Norton, D. P., “The strategy-focused
organization,” Strategy & Leadership, 2001.
[23]
Lee, I.; Shin, Y. J., “Machine learning for
enterprises: Applications, algorithm selection, and
challenges,” Business Horizons, pp. 157-170, 2020.
[24]
Whig, P., “Artificial Intelligence and Machine
Learning In Business,” International Journal on
Integrated Education, 2019.
[25]
Punitha, N. J.; Preethi, D. M. D.;, “Artificial
Intelligence and Machine Learning Techniques for
COVID-19 Prediction,” in 2022 International
Conference on Advanced Computing Technologies
and Applications (ICACTA), IEEE, Coimbatore,
India, 2022.
[26]
Goodfellow, I., “Deep Learning,” Cambridge, MIT
Press, pp. 1-16, 2016.
[27]
Chellapa, R.; Theodoridis, S.; van Schaik, A.;,
“Advances in Machine Learning and Deep Neural
Networks,” IEEE, vol. 109, pp. 607-611, 2021.
[28]
Villazón, C. C.; Pinilla, L. S.; Olaso, J. R. O.;
Gandarias, N. T.; de Lacalle, N. L., “dentification of
Key Performance Indicators in Project-Based
Organisations through the Lean Approach,”
Sustainability, pp. 12(15), 5977, 24 July 2020.
[29]
Ekinci, Y., Designing Research Questionnaires for
Business and Management Students, Sage, 2015.
[30]
Fawcett, T.; Provost, F., Data Science for Business,
O'Reilly Media, Inc., 2013.
[31]
Davino, C.; Fabbris, L., Survey Data Collection and
Integration, Berlin: Springer, 2013.
[32]
Sauce, B.; Matzel, L. D., “Inductive Reasoning,” in
Encyclopedia of Animal Cognition and Behavior,
Springer, 2017, pp. 1-8.
[33]
Gholamy, A.; Kreinovich, V.; Kosheleva, O., “Why
70/30 or 80/20 Relation Between Training and
Testing Sets: A Pedagogical Explanation,”
Departmental Technical Reports Computer Science
at ScholarWorks@UTEP., 2018.
[34]
Fausett, Laurene V., Fundamentals of Neural
Networks: Architectures, Algorithms And
Applications, Pearson, 1994.
[35]
Cantareira, G. D.; Etemad, E.; Paulovich, Fernando
V., “Exploring Neural Network Hidden Layer
Activity Using Vector Fields,” Information, p. 11:
426, 2020.
[36]
Nielsen, M., Neural Networks and Deep Learning,
Determination Press. pp. 1-12, 2018.
[37]
Agarap, M., “Deep Learning using Rectified Linear
Units,” ArXiv, p. arXiv:1803.08375, 2019.
[38]
Hagan, M.; Demuth, H.; Beale, M., “Neural
Network Design (2nd Edition),” Martin Hagan, pp.
889-890, 2014.
[39]
Zargar, S. A., Introduction to Sequence Learning
Models: RNN, LSTM, GRU, 2021.
[40]
Afaq, S., “Significance Of Epochs On Training A
Neural Network,” International Journal of Scientific
& Technology Research:, pp. 485-488, 2020.
[41]
Markoulidakis, I.; Rallis, I.; Georgoulas, I.;
Kopsiaftis, G.; Doulamis, A.; Doulamis, N.,
“Multiclass Confusion Matrix Reduction Method
and Its Application on Net Promoter Score
Classification Problem,” in The 14th Pervasive
Technologies Related to Assistive Environments
Conference, Corfu, Greece, 2021.
[42]
Xu, Y.; Goodacre, R., “n Splitting Training and
Validation Set: A Comparative Study of Cross-
Validation, Bootstrap and Systematic Sampling for
Estimating the Generalization Performance of
Supervised Learning,” Journal of Analysis and
Testing, pp. 2: 249-262, 2018.
Contribution of Individual Authors to the
Creation of a Scientific Article (Ghostwriting
Policy)
-Eleni Tagkouta, Panagiotis – Nikolaos Psycharis
were the main authors and were responsible for the
conceptualization, the writing of the original project,
the investigation, the statistics, the formal analysis,
the resources, the software, the methodology and the
visualization.
-Alkinoos Psarras was responsible for the
conceptualization, the methodology, the resources
and the writing review & editing.
-Theodoros Anagnostopoulos was responsible for
the review of the statistics, the methodology, the
validation, the supervision and the writing review &
editing.
-Ioannis Salmon was responsible for the
methodology, the supervision, the project
administration, the writing review & editing and the
funding acquisition.
WSEAS TRANSACTIONS on BUSINESS and ECONOMICS
DOI: 10.37394/23207.2023.20.59
Eleni Tagkouta, Panagiotis – Nikolaos Psycharis,
Alkinoos Psarras, Theodoros Anagnostopoulos,
Ioannis Salmon
E-ISSN: 2224-2899
656
Volume 20, 2023
Sources of Funding for Research Presented in a
Scientific Article or Scientific Article Itself
No funding was received for conducting this study.
Conflict of Interest
The authors have no conflicts of interest to declare
that are relevant to the content of this article.
Creative Commons Attribution License 4.0
(Attribution 4.0 International, CC BY 4.0)
This article is published under the terms of the
Creative Commons Attribution License 4.0
https://creativecommons.org/licenses/by/4.0/deed.en
_US