The Cognitive Engineering in Manufacturing Process

through Reinforcement Learning

DASCHIEVICI LUIZA, GHELASE DANIELA

Faculty of Engineering and Agronomy, Braila,

“Dunarea de Jos” University of Galati,

ROMANIA

Abstract: The reinforcement learning (RL), through its role, in business integrated manufacturing, means the

manufacturing system capacity to 'learn' in permanent interaction with the economic environment, to inform

and update the information about the auctions and to anticipate, before deciding to conclude a contract, the

level of costs, profit and what is the best way to act. In other words this means that the manufacturing system

'learns' what actions to take in certain situations, based on the data supplied by the economic environment, so

that such actions increase the possibilities of achieving the aim proposed. The business integrated

manufacturing should 'exploit' what it already knows to obtain profit, but at the same time it must 'explore' the

possibility of finding other suitable actions for the future. The manufacturing system should try a variety of

actions and then choose those that seem best. This study shows the potential of RL for application to the

business integrated manufacturing.

Key-Words: - Competitiveness, Cognitive Engineering, Reinforcement Learning.

Received: May 19, 2022. Revised: January 27, 2023. Accepted: February 23, 2023. Published: March 7, 2023.

1 Introduction

On a world wide plan, enterprises are confronted

with a dynamic more and more and unpredictable

changes. This is influenced by the technical and

scientific progress, dynamic requirements of the

customers, science of management and

mathematical economy “[1]”. These changes enforce

an aggressive competition to the global scale which

assumes the request of a new settlement equilibrium

between economy, technology and society. At this

changes, the scientific community proposes to

answer new paradigm: Knowledge-based Economy

Competitiveness characterizes synthetic and

complete the viability of the enterprises. In

economic literature, competitiveness is analyzed in

particular from an economic and managerial

viewpoint, entering or not at all in analyzing the role

of the technology in the assurance and increase of

competitiveness. The necessity to manage the

manufacturing systems based on cognitive

modeling.

The term cognitive engineering is connected

directly to the area of the cognitive sciences and

emphasizes the existence of a stock of techniques

and technologists of processing knowledge. The

principal idea of this area is considered the mental

from perspective processing of the knowledge. The

cognitive, the conception which unites the cognitive

sciences, gives a capital importance for processing

knowledge, in explication working the human mind.

More, is done hypothesis that the faculty of human

mind, such as intelligence, can be obtained of

mechanisms, don't is can done a functional

distinction (from viewpoint of the complete

functions) between the human mind and the

adequate machine.

The cognitive approach is based on continuing

conscientiously of the situations and the decisions in

real-time about activities.

The equivalence between the human mind and

the computer preached of cognitive research, bears

at two interpretations of cognitive engineering,

accordingly the implications which build this

equivalence. First acceptance is named knowledge

engineering, the engineering area which pertains to

artificial engineering and which proposes to build

structures which purchases, manages, processes,

expanding and exploits the human cognition. Here,

are integrated the expert systems, the systems based

on knowledge.

There is a second possible implication, from the

cognitive sciences (informatics, artificial

intelligence, psychology, just neurology and

sociology) to man.

WSEAS TRANSACTIONS on SYSTEMS

DOI: 10.37394/23202.2023.22.19

Daschievici Luiza, Ghelase Daniela

E-ISSN: 2224-2678

194

Volume 22, 2023

2 Application

The establishment of tight connections between the

technical variables (the manufacturing parts) and

economical variables (modeling as entities of costs)

is realized through the estimation of the cost.

For most industrial companies, the estimation

method of the cost determines in particular the

performances of two strategic functions: product

design and the offer (the price of product). In

general, it is commonly admitted that product design

can engage up to 70-80% of the total product cost.

The recent progress achieved in Integrated

Engineering such as concurrent engineering or

integrated design opens a new field for cost

estimating during the design stage.

In a competitive market, the incapacity of the

company to quickly and adequately successfully

request for quotation can severely affect its capacity

to survive economically. Indeed, an underestimated

cost will result in losses while an overestimated cost

will prevent the company from remaining

competitive. So, there is a strong need expressed by

industry to have sound cost estimating solutions,

both in terms of design and quotation that can

improve the performance of these strategic

functions.

To face this need, and to replace the analytical-

based methods commonly used in manufacturing

process planning, many companies apply parametric

and analogous cost estimation methods. These

methods are really fast because they are essentially

synthetic, and provide the total cost of the product

according to some of its characteristics.

After a detailed study of the cost estimating

problem in mechanical engineering, it can be

concluded that two support models are required: a

knowledge model and a reasoning model.

In manufacturing, cost estimating is the art of

predicting what it will cost to make a given product

or batch of products. Various techniques exist for

cost estimating. The manufacturing cost of a part can

be estimated using one of four basic methods:

intuitive, analogous, parametric and analytical.

Based on the theories “[1]”, “[6]”, “[7]”,

“[8]“about cognition and complexity, it is a design

of a cognitive and adaptive mechanism that manages

processes by responding flexibly to the demands of

the economical environment (figure 1). This

mechanism is characterized by an ability to perceive

the economical process environment and make real-

time decisions about interactions among the

manufacturing system and the economical

environment.

The cognitive approach is characterized by an

ability to perceive the economical environment and

make real-time decisions about tasks.

In general, anything that learns a problem through

interaction can be reduced to three signals which are

transmitted between the agent and the environment

actions, the states and the rewards (figure 2).

In function of interaction between the agent and the

environment distinguished the next types of learn:

1. supervised learning: the environment offers the

problems on which the agent has solved and the

correct answered at this problems;

WSEAS TRANSACTIONS on SYSTEMS

DOI: 10.37394/23202.2023.22.19

Daschievici Luiza, Ghelase Daniela

E-ISSN: 2224-2678

195

Volume 22, 2023

2. reinforcement learning: the environment offers

the dates about the correctness actions undertaker of

agent, but don’t says which are the correct action;

3. unsupervised learning: the environment doesn't

offer the information about the correctness of the

actions undertaken by the agent.

In reinforcement learning “[2]”, “[3]”, “[4]”,

“[5]”, “[9]” the machine interacts with its

environment by producing actions a1, a2,…..These

actions affect the state of environment, which in turn

results in the machine receiving some scalar rewards

r1, r2,…. The goal of the machine is to learn to act in

a way that maximizes the future rewards it receives

(or minimizes the punishments) over its

lifetime. Reinforcement learning is closely related

to the fields of decision theory (in statistics and

management science), and control theory (in

engineering).

In general, the learning process, is an action, in

abaft whom, a manufacturing system improves the

capacity to react, so that, in temporally of a

subsequent solicitations, this undertakes actions with

efficient increase. Conception of a methodologies of

modelling in the real-time, based on reinforcement

learning, for relation of the manufacturing system

with economical environment, it means, that the

manufacturing system "learn" what to do in certain

situations, on the based of given data of economical

environment, so that the actions undertake to lead

increase possibility of touches the suggested aim.

The system must to "exploit" what it knows has

already obtained the profit, but must at the same

time to "explore" the possibility of finding other

future actions. The manufacturing system must try a

variety of actions and then to choose them on those

which are even optimal.

Is done an evaluation of the evolution of the

state economical environment, while, and gives an

ensemble modeling based on the past events.

Through reinforcement learning is understanding the

capacity of the manufacturing system to learn

permanently in interaction with the economical

environment, to inform and update the info about

auctions and anticipate the statement, the level

profited, and how to act well. The relation modeling

of the market – manufacturing system simulates, on

the basis of environment states and one action of the

manufacturing system, the behavior ensemble and

can predict which will be the next state and the

result obtained. The relation is used for planning,

that is, for taking decisions about cognitive

modeling of ensemble the manufacturing system –

market, and considering possible future situations

before these states are experienced. After each

possible situation the manufacturing system will

adapt the cognitive models, so that it can learn

towards his next states values most probable.

Through the process of learning, the manufacturing

system will be left to execute a series of actions

according to the instructions of the cognitive model

of the ensemble and will select the act in which it

will go in the state with maximum competitiveness.

3 Conception of a Methodologies of

Modelling, in Real-time based on

Reinforcement Learning, of the

Relation of the Manufacturing System

with Economic Environment

The research about learning in the word pointed out

the crucial role it plays in the interaction with the

environment. The practice of sensory connection

with the environment produces a big quantity of info

of type cause-effect about the consequences of the

actions and keeping with decisions for touching the

aims. These interactions are a major source of

knowledge about the environment. In each moment,

we are conscious of the manner in which the

environment reacts to our actions and we search to

influence this thing through our behavior. The

interaction is the fundamental cause of theories

about learning and intelligence.

In general, the learning process is a process in

which the agent (it who learns) improves the

capacity of act, so that, temporally of next

solicitations, the agent carries on actions which are

very efficient.

In reinforcement learning, the environment

offers the dates about the correctness actions of the

agent, but doesn't say which is the correct action.

We will develop at conceptual level a

methodology of modelling based on reinforcement

learning of relation manufacturing systems –

economical environment for a real system of

manufacturing of an enterprise which works on a

real market with values of the parameters taken from

the economical reality. The values economical

parameters unite with values technical parameters,

accordingly product achieved will be used to

generate a relation which describes dependent the

manufacturing system - market. It will analyze the

details about the methodology of learning based on

reinforcement learning that can be applied for the

elaboration and modelling of the relation between

the market and the manufacturing system. The

activities investigatory afferent are:

a. Extract through the data mining of an

information concerning the situation of the auctions,

WSEAS TRANSACTIONS on SYSTEMS

DOI: 10.37394/23202.2023.22.19

Daschievici Luiza, Ghelase Daniela

E-ISSN: 2224-2678

196

Volume 22, 2023

from database derived from the marketing

compartment of enterprises and the definiteness of

an evaluation functions;

b. The elaboration of the cognitive model of the

manufacturing system on the base of information

from the data mining;

c. The elaboration of algorithm of

reinforcement learning and this applied to the

operation of the manufacturing system in relation

with economic environment to obtain the maxim

profit;

d. The integration of the model with the algorithm

in the modelling of methodology, in real-time based

on reinforcement learning of the relation of the

manufacturing system with the economic

environment.

4 Devising a Real-time Modeling

Methodology based on Reinforcement

Learning, of the Manufacturing

System Relationship with the

Economic Environment

The learning process, in general, is an action in

which the manufacturing system can improve its

ability to react so that, during subsequent requests, it

should take actions more efficiently.

Devising a real-time modeling methodology,

based on reinforcement learning (which is a specific

non supervised learning technique) of the

manufacturing system relationship with the

economic environment means that the

manufacturing system 'learns' what actions to

perform in certain situations, based on the data

supplied by the economic environment , so that

such actions increase the possibilities of achieving

the aim pursued.The system should 'exploit' what it

already knows to get profit, but at the same time it

must 'explore' the possibility of finding other

suitable actions for the future. The manufacturing

system should try a variety of actions and then

choose those that seem best.

According to the competitive management

algorithm presented in Figure 3, regarding the

market-manufacturing system relationship

by reinforcement learning, from the data supplied

by the marketing section of the enterprise (auctions

situation) , an evolution of the economic

environment for a period of time is carried out

and an overall modeling is provided on the basis

of past events.

Reinforcement learning is to be understood as the

manufacturing system capacity to 'learn' in

permanent interaction with the economic

environment, to inform and update the information

about the auctions and to anticipate, before deciding

to conclude a contract, the level of costs , profit and

what is the best way to act. Modeling the market-

manufacturing system relationship simulates, based

on a state of the environment and an action of the

manufacturing system, the behavior of the assembly

and can predict what will be the next state and the

result obtained.

The relationship is used for planning, i.e. to take

decisions regarding the behavioral modeling of the

manufacturing system – market assembly while

considering possible future cases before such

situations are experimented.

After each possible situation, the manufacturing

system will adapt its behavior, so that it tends

towards its next most favorable state. By the

learning process, the manufacturing system will be

allowed to execute a number of actions in

accordance with the instructions from the behavioral

model operation of the assembly and that action will

be selected likely to bring it to the maximum

competitiveness state.

5 Conclusion

At conceptual level a modeling methodology based

on a reinforcement learning of the manufacturing

system - economic environment relationship will

be developed . The methodology will be tested on an

actual manufacturing system from an enterprise

working on a real market and the parameter values

taken from economic reality. The values of the

economic parameters, together with the values of the

technical parameters corresponding to the product

WSEAS TRANSACTIONS on SYSTEMS

DOI: 10.37394/23202.2023.22.19

Daschievici Luiza, Ghelase Daniela

E-ISSN: 2224-2678

197

Volume 22, 2023

developed, will be used to generate a relationship

that describes the dependence of the manufacturing

system on the market. It will analyze the details of

how the reinforcement learning based methodology

can be applied to develop and shape the relationship

between market and manufacturing system. The

research activities include:

a) extraction by data mining of information on the

status of the auctions database from the

marketing department of the company and

defining an evaluation function

b) developing the behavioral model of the

manufacturing system based on the data mining

information

c) develop a reinforcement learning algorithm and

its application to the manufacturing system

operation in relation to the economic environment in

order to obtain maximum profit

d) integration of the model algorithm into the

methodology for modeling in real-time, based on

reinforcement learning, the relationship of the

manufacturing system with the economic

environment.

This approach opens new horizons in imagining

how management systems can operate cognitively

with technical appearances, economically,

commercial, managerial.

The applications of cognitive engineering for a

manufacturing system leads to the appearance of the

new generations of enterprises which will achieve

the products to the level of quality solicited of the

market. In this paper is developed the new concept

of management for the manufacturing system, the

concept of competitive management.

The elaboration of a new concept of managing

the manufacturing systems based on

cognitive modeling of ensemble manufacturing

systems – market and the implementation of

management to the level of the manufacturing

system which is generally applicable and proper to

current requirements of the market.

In this paper is described the utilization of the

method reinforcement learning in the assurance

adaptability of the enterprise at the requirements

market.

References:

[1] Minhong, Wang, Huaiquing, Wang– From

process logic to business logic – A cognitive

approach to business process management,

(electronic Elsevier), 2006

[2] Zoubin Grahramani – Unsupervised learning,

Gatsby Computational Neuroscience Unit,

University College London, UK, September,

2006

[3] Ghaeli M, Bahri P., Lee P - Scheduling of a

mixed batch/continuous sugar milling plant

using Petri nets, in Computers & Chemical

Engineering, Volume 32, Issue 3, 24 March

2008, 580-589 (electronic in Science direct ),

2008

[4] Gi-Tae Yeo, Roe M.and Dinwoodie J. -

Evaluating the competitiveness of container

ports in Korea and China Transportation

Research Part A: Policy and Practice, In Press,

Corrected Proof, Available online 14 February

2008

[5] Seong Kon Lee, Gento Mogi and Jong Wook

Kim - The competitiveness of Korea as a

developer of hydrogen energy technology: The

AHP approach Energy Policy, In Press,

Corrected Proof, Available online 28 January,

2008

[6] George F. Georgakopoulos - Chain-splay trees,

or, how to achieve and prove loglogN-

competitiveness by splaying, in Information

Processing Letters, Volume 106, Issue 1, 31

March 2008, 37-43, 2008.

[7] Toly Chen - Evaluating the mid-term

competitiveness of a product in a

semiconductor fabrication factory with a

systematic procedure, Computers & Industrial

Engineering, Volume 53, Issue 3, October

2007, 499-513, 2007

[8] Christoph H. Loch, Stephen Chick and Arnd

Huchzermeier - Can European Manufacturing

Companies Compete?: Industrial

Competitiveness, Employment and Growth in

Europe, in European Management

Journal, Volume 25, Issue 4, August 2007,

251-265, 2007

[9] V. Samsonov, K.B.Hicham, T.Meisen-

Reinforcement Learning in Manufacturing Control:

Baselines, challenges and ways forward,

Engineering Applications of Artificial

Intelligence, Volume 112, June 2022, 104868,

Elsevier.

WSEAS TRANSACTIONS on SYSTEMS

DOI: 10.37394/23202.2023.22.19

Daschievici Luiza, Ghelase Daniela

E-ISSN: 2224-2678

198

Volume 22, 2023

Contribution of Individual Authors to the

Creation of a Scientific Article (Ghostwriting

Policy)

The authors equally contributed in the present

research, at all stages from the formulation of the

problem to the final findings and solution.

Sources of Funding for Research Presented in a

Scientific Article or Scientific Article Itself

No funding was received for conducting this study.

Conflict of Interest

The authors have no conflicts of interest to declare

that are relevant to the content of this article.

Creative Commons Attribution License 4.0

(Attribution 4.0 International, CC BY 4.0)

This article is published under the terms of the

Creative Commons Attribution License 4.0

https://creativecommons.org/licenses/by/4.0/deed.en

_US

Contribution of Individual Authors to the

Creation of a Scientific Article (Ghostwriting

Policy)