Enhancing Condition Monitoring with Virtual Reality Visualization for
Industrial Application
ALI MOHAMMED RIDHA1, WESSAM SHEHIEB2
1Beckhoff Automation FZE,
Dubai,
UAE
2Research and Development Department,
Analytica FZE,
Dubai,
UAE
Abstract: - Monitoring the machines' health in an industrial setting is crucial to maintaining the workflow
without disturbance. There are multiple methods commonly used for this purpose that go under conditional
monitoring. Conventional conditional monitoring can be time-consuming and difficult to interpret by a non-
experienced operator, which increases the risks in such critical applications. Virtual Reality (VR) has been in
use for multiple applications to ease the usage of complex tasks via proper visualization. In this paper, a
visualization approach of industrial application that monitors the motor’s health connected with a vibration
sensor via a programmable logic controller (PLC) is presented. The Beckhoff PLC was programmed using
TwinCAT 3 software and linked with Unity VR headsets via MQTT protocol. The user is given the ability to
select the number of connected sensors and place them to mimic the actual environment, import their own 3D
machine design, test for MQTT broker connectivity, and visualize the machine's health as per the ISO 10826-3
standard. The proposed system has been tested successfully and it is expected to ease visualization in an
industrial setup.
Key-Words: - Condition Monitoring; IoT; MQTT; Programmable Logic Controller; TwinCAT 3; Virtual
Reality.
Received: February 19, 2023. Revised: December 4, 2023. Accepted: December 17, 2023. Published: March 6, 2024.
1 Introduction
Continuous production output in industrial settings
is a requirement, where any failure of the equipment
can result in high losses. This is why monitoring the
machine operation parameters has always been
important to assess the maintenance requirement of
a machine before a failure can occur. This will allow
the user to optimize the equipment performance,
reduce cost, and ensure the continuous operation of
the machine, [1].
Condition monitoring, unlike traditional time-
based maintenance, can estimate the health of a
machine, and provide information on the
performance and maintenance required based on
real-time data. There are many condition monitoring
methods, such as thermal monitoring, current
analysis, signal processing, vibration analysis, or
artificial intelligence (AI), [2], [3], [4], [5].
Recent research [4], has proposed a technique
that employs motor current signature analysis and
fuzzy logic to detect various types of faults in the
stator winding of the motor. The system can also
classify the driving system fault into two categories
(inverter and motor faults), with the motor fault
having 2 subcategories (Mechanical and electrical
faults).
Another research [5], outlines predictive
maintenance tools that support fault detection, it
explores different tools such as time series, decision
trees, and artificial intelligence. Additionally, it
identifies AI tools that can be used to assist in
decision-making for management, to increase the
availability of the asset, and decrease the
maintenance cost.
These techniques can then be used to obtain
results that can be presented in a graphical view.
By utilizing technologies like virtual reality (VR)
and augmented reality (AR) in condition
monitoring, can create an immersive experience in
monitoring equipment in real time. This could
enable the maintenance personnel to identify the
WSEAS TRANSACTIONS on POWER SYSTEMS
DOI: 10.37394/232016.2024.19.4
Ali Mohammed Ridha, Wessam Shehieb
E-ISSN: 2224-350X
26
Volume 19, 2024
problems and resolve them. VR and AR can also be
used in remote monitoring of equipment, this can be
extremely useful in situations where it is not safe for
people to be physically present.
Haptic technologies, which create the experience
of 3D touch by applying vibrations and motion, [6],
are one of the key advantages of VR. This can
enhance the effectiveness of training for the
maintenance personnel, by familiarizing them with
the equipment without the equipment being
physically present.
VR and AR technologies have been widely
researched and used in several different fields, such
as education [7] and landscape design [8], but the
usage of VR in industrial settings is still limited.
In this paper, a novel system is proposed that
utilizes a Beckhoff programmable logic controller
(PLC) and Meta Quest 2 VR headset to enable the
user to visualize, interact, and monitor their
equipment in real time. The structure of this paper is
as follows: The design and implementation of the
hardware and software are described first. Then, the
test results are presented, and followed by a
discussion of the results. Finally, the conclusion and
future work are discussed.
2 Design and Implementation
The conceptual diagram of the proposed system can
be seen in Figure 1. The system consists of three
main parts: data acquisition, visualization, and an
MQTT broker. Both data acquisition PLC and VR
headset are connected to the MQTT broker through
the internet.
The first step is to read the data, a Beckhoff PLC,
which is an industrial control system that can be
programmed to perform automation tasks is utilized
to acquire data from one vibration sensor, then
perform analysis on the data and publish it to the
MQTT broker.
Fig. 1: Conceptual Diagram
After the results of the analysis are published to
the MQTT broker, the VR headset that has a unity-
developed application will visualize the users’
equipment health in real-time using the analysis
results.
2.1 Hardware Setup
The hardware part of the system consists of the
PLC, MQTT broker (computer), and VR headset.
2.1.1 Data Acquisition Setup
The system consists of two main components: the
PLC and the sensors, as depicted in Figure 2. The
PLC utilized is a Beckhoff CX5130 Embedded PC.
That uses an Intel Atom E3827 processor (1.75
GHz), with 4 GB DDR3 RAM, and is running
Windows 10. The PLC is powered by a Beckhoff
PS1111 24v power supply.
Fig. 2: System hardware design
An ELM3602 IEPE measurement terminal with a
max sampling rate of 50,000 samples per second is
also used. The system additionally includes one
SPM IEPE SLD144TB vibration transducer, which
is mounted on a single-phase motor (0.37kW, 2.7
A). To test the concept of the proposed system, an
eccentric load was mounted on the shaft of the
motor, to simulate vibrations that can be picked up
by the vibration transducer.
2.1.2 MQTT Broker
A computer connected to the network will have
Eclipse Mosquitto [9], an open-source MQTT
broker running. It will allow both the PLC and VR
to communicate with it, to transfer data.
2.1.3 VR Headset
Meta Quest 2, [10], VR headset was used to
visualize the equipment in an immersive 3D
environment, to provide an interactive experience
for the users to analyze and interpret the data in real-
time.
2.2 Software Design
The software part of the system consists of the PLC
program, and the VR application developed for
visualization.
2.2.1 PLC Software
The PLC will utilize TwinCAT 3, a real-time
control software developed by Beckhoff that is
WSEAS TRANSACTIONS on POWER SYSTEMS
DOI: 10.37394/232016.2024.19.4
Ali Mohammed Ridha, Wessam Shehieb
E-ISSN: 2224-350X
27
Volume 19, 2024
based on Visual Studio. The data acquisition and
analysis program was developed on TwinCAT 3
using Structured Text programming language, and
then it was deployed to the CX5120 PLC. TwinCAT
3 Analytics Workbench, a user interface that was
developed by Beckhoff for the design and
deployment of analytics programs, along with
TwinCAT 3 Condition Monitoring library, were
utilized for the vibration assessment program. Some
parameters such as machine group, and machine
substructure, which are crucial for accurate
vibration assessment are read through MQTT from
the VR application. Additionally, after receiving the
start signal for the real-time analysis from the VR
application, the PLC will start the acquisition and
publish the results to the MQTT broker.
2.2.2 VR Visualization
Unity 3D, which is a game engine that is widely
used for creating interactive applications, was
utilized to create the VR application. M2MQTT
library, [11], was used in Unity to enable
communication between the VR application and the
MQTT broker. The 3D model of the motor was
designed using Sketchup, [12] and then added to the
VR application to provide a realistic representation
of the motor’s health.
The VR application will require the user to
configure the MQTT communication settings, and
the equipment configuration according to ISO
10826-3 standard. The VR application also allows
the user to import their custom-designed equipment
and set up the location of the vibration sensors, and
the 3D model will have a Unity Shader that creates a
heatmap to visually represent motor health. The
Shader will use the sensor locations and vibration
velocity values to dynamically generate the
heatmap, providing an overview of the equipment’s
condition, which would enhance the accessibility
and provide real-time assessment of the equipment
for effective condition monitoring.
3 Results
The results of this study are presented in three
sections: the hardware implementation, the
developed PLC analytics program, and the VR
application implementation.
3.1 Hardware Implementation
The data acquisition setup is shown in Figure 3 and
the single-phase motor with the vibration transducer
can be seen in Figure 4.
Fig. 3: Data acquisition setup
After reading the accelerometer data, the PLC
program will perform analysis according to the ISO
10816-3 standard, and classify the equipment’s
health, then publish the results of the vibration
assessment to the MQTT broker.
Fig. 4: Hardware setup consisting of a single-phase
motor and a vibration sensor
3.2 PLC Software
To perform the vibration assessment according to
the ISO 10816-3 standard, which involves
measuring and evaluating vibrations in machinery,
the assessment criteria is based on the root mean
squared (RMS) value of the vibration velocity or
displacement. At the same time, it is sufficient only
to measure the velocity of vibrations. According to
the ISO 10816-3 standard, the minimum output
range for the motor must be between 15 kW to 300
kW. Considering that the motor used in this
experiment is 0.37kW it cannot be categorized using
the standard, Therefore, it will be considered that
it’s in Group 2 for simulation.
The PLC will first buffer the input data from the
sensor. Since the test motor has a speed above 600
rpm, complying with the ISO 10816-3 standard, the
frequencies in the range of 10 Hz to 1000 Hz will be
evaluated. Fast Fourier Transform (FFT) is used to
analyze only the specified frequency range, then the
RMS values for the acceleration data input are
calculated, additionally, the RMS value of vibration
WSEAS TRANSACTIONS on POWER SYSTEMS
DOI: 10.37394/232016.2024.19.4
Ali Mohammed Ridha, Wessam Shehieb
E-ISSN: 2224-350X
28
Volume 19, 2024
velocity and displacement is calculated from the
acceleration data. The RMS values of both the
vibration velocity and displacement are then
checked against the ISO standard limits for machine
group 2 tabulated in Table 1 to determine the
classification.
Table 1. Evaluation Zones Limit Values for
Machine Group 2 From ISO Standard 10816-3
Installation
Rigid
Elastic
11.00 - inf
D
D
07.10 - 11.00
D
D
04.50 - 07.10
D
C
03.50 - 04.50
C
B
02.80 - 03.50
C
B
02.30 - 02.80
B
B
01.40 - 02.30
B
A
00.00 - 01.40
A
A
There are in total 4 zones described in ISO
10816-3, zone A describes a recently commissioned
machine, and Zone B is for a machine suitable for
continuous operations without any restrictions.
Finally, unsuitable machines for continuous
operation and machines regarded as dangerous that
may damage machines are described as zones C and
D respectively. The limits described by the standard
are only intended as a general guideline, and in
some cases cannot apply to certain machinery or
operating conditions.
Fig. 5: Acceleration, Vibration, and displacement
data from PLC
This is due to the variety of machinery in an
industrial setting, where the machine groups
described in the standard try to categorize the
machine while accounting for the variations in
design. However, due to differences in
environmental factors, machinery types, or the
application of the machine, the users are advised to
interpret the limits only as a foundational
framework when assessing the vibrations of a
machine.
The worst-case classification is then determined
by comparing the two possible classifications
(velocity and displacement), adhering to ISO 10816-
3 guidelines. Finally, the classification results are
then published to the MQTT broker for the VR
application.
Figure 5 showcases a sample of the acceleration
data acquired from the PLC. The part in red
showcases the normal operations of the motor, while
the green was produced by simulating vibrations on
the motor to visualize in VR.
3.3 VR Application
The Unity application was developed and installed
on the Quest 2 Headset, enabling the user to move
around in a virtual world and interact with a 3D
representation of the motor. After starting the VR
application, the user will be presented with the main
menu as shown in Figure 6. The first step is to set
up the MQTT broker configuration i.e., broker
address, port, subscription topic, and
Username/password if applicable. After that, the
user can test the communication connection to
check if the VR application can communicate with
the broker.
Fig. 6: Main menu of VR application
The user then must set the number of sensors,
and the MQTT topic of each sensor. They are also
able to test the MQTT communication for the
sensors to see if data is being received. The user will
also be able to set the motor configuration as per the
ISO 10826-3 standard and publish it to the broker,
where the PLC will be subscribed to read and set the
values for analytics.
WSEAS TRANSACTIONS on POWER SYSTEMS
DOI: 10.37394/232016.2024.19.4
Ali Mohammed Ridha, Wessam Shehieb
E-ISSN: 2224-350X
29
Volume 19, 2024
Optionally, the user can import their own
motor/machine 3D model if required, instead of the
default 3D motor model, and then set the sensor
points as shown in Figure 7.
Fig. 7: Sensor placement in the VR application
Finally, after starting the measurements, the user
will be able to see the motor overview with the
details of the sensor and evaluation zones as shown
in Figure 8.
Fig. 8: Motor overview in the VR application
As per the vibration velocity and evaluation zone
received from the PLC, the shaders of the 3D motor
will start to display a heatmap based on the
previously placed sensor point. Alternatively, if
there is more than one sensor, touching the sensor
placeholder or selecting it will show an overview of
the sensor with its values as shown in Figure 9.
Fig. 9: Sensor evaluation in the VR application
4 Discussion
While the developed system was only tested on a
small motor, the same concept can be scaled to
existing machines. By integrating a data acquisition
system, and the developed VR application, users
will have an immersive platform on their VR device
to interact with the machine in real-time, and
utilizing the analyzed vibration data from the
Beckhoff TwinCAT system will enhance the
diagnostics of machines. Additionally, the
developed VR application allows custom
configurations of sensors and user-designed 3D
machine models to be used, which makes it flexible
to be used with a diverse range of machines. Further
research can be focused on extending real-time data
acquisition to create a fully functioning digital twin
of the machine in VR.
5 Conclusion
The proposed system integrated a Beckhoff PLC for
data acquisition and a Meta Quest 2 VR Headset for
visualization was successfully implemented. The
system allowed the user to view and interact with
the device in real-time to evaluate the equipment’s
health. The use of VR for condition monitoring has
the potential to significantly improve the efficiency
and effectiveness of maintenance operations in
industrial settings. It can help to reduce costs,
improve productivity, and ensure the continuous
operation of equipment. As VR technologies
continue to advance and become more widely
adopted, an increasing number of applications in the
industrial sector will likely start employing VR.
Further work can include mapping the results with
multiple sensors and applying artificial intelligence
predictions on the equipment to assist the operators
and engineers in the field.
WSEAS TRANSACTIONS on POWER SYSTEMS
DOI: 10.37394/232016.2024.19.4
Ali Mohammed Ridha, Wessam Shehieb
E-ISSN: 2224-350X
30
Volume 19, 2024
References:
[1] P. Kumar, A. Namdeo, and A. Dixit,
"Condition Monitoring of 3-Φ A.C Induction
Motor Using PLC," Semantic Scholar
Engineering, 2015, [Online].
https://api.semanticscholar.org/CorpusID:321
53976 (Accessed Date: November 27, 2023).
[2] S. H. Chetwani, M. K. Shah, and M.
Ramamoorty, “Online condition monitoring
of induction motors through Signal
Processing,” 2005 International Conference
on Electrical Machines and Systems, 2005.
doi: 10.1109/icems.2005.202952.
[3] R. C. Bhavsar and R. A. Patel, "Various
Techniques for Condition Monitoring of
Three Phase Induction Motor-A Review,"
International Journal of Engineering
Inventions, vol. 3, no. 4, pp. 22-26, Nov.
2013.
[4] Joao Rodrigues, Jose Torres Farinha, Antonio
Marques Cardoso, "Predictive Maintenance
Tools - A Global Survey," WSEAS
Transactions on Systems and Control, vol. 16,
pp. 96-109, 2021,
https://doi.org/10.37394/23203.2021.16.7.
[5] Mohamed I. Abu El-Sebah, Faeka M.H.
Khater, "A Proposed Fault Diagnostics
Technique for Induction Motor Stator
Winding," International Journal of Electrical
Engineering and Computer Science, vol. 5,
pp. 33-40, 2023, doi:
10.37394/232027.2023.5.5.
[6] S. Biswas and Y. Visell, “Emerging material
technologies for haptics,” Advanced Materials
Technologies, vol. 4, no. 4, 2019. doi:
10.1002/admt.201900042.
[7] A. M. Ridha and W. Shehieb, “Assistive
technology for hearing-impaired and deaf
students utilizing augmented reality,” 2021
IEEE Canadian Conference on Electrical and
Computer Engineering (CCECE), 2021. doi:
10.1109/ccece53047.2021.9569193.
[8] S. Sun, Q. Meng, Y. Ma, and Z. Ren,
“Application of virtual reality technology in
Landscape Design,” 2021 International
Symposium on Artificial Intelligence and its
Application on Media (ISAIAM), 2021. doi:
10.1109/isaiam53259.2021.00026.
[9] “Eclipse mosquitto,” Eclipse Mosquitto,
[Online]. https://mosquitto.org/ (Accessed
Date: November 27, 2023).
[10] “Meta Quest 2,” Meta, [Online].
https://www.meta.com/quest/products/quest-
2/ (Accessed Date: November 27, 2023).
[11] “M2mqtt & GnatMQ,” M2Mqtt & GnatMQ,
[Online]. https://m2mqtt.wordpress.com/
(Accessed Date: November 27, 2023).
[12] SketchUp, Trimple, [Online].
https://www.sketchup.com/ (Accessed Date:
November 27, 2023).
Contribution of Individual Authors to the
Creation of a Scientific Article (Ghostwriting
Policy)
The authors equally contributed to the present
research, at all stages from the formulation of the
problem to the final findings and solution.
Sources of Funding for Research Presented in a
Scientific Article or Scientific Article Itself
This research was funded by Beckhoff Automation
FZE, Dubai, UAE and Analytica FZE, Dubai, UAE.
Conflict of Interest
The authors have no conflicts of interest to declare.
Creative Commons Attribution License 4.0
(Attribution 4.0 International, CC BY 4.0)
This article is published under the terms of the
Creative Commons Attribution License 4.0
https://creativecommons.org/licenses/by/4.0/deed.en
_US
WSEAS TRANSACTIONS on POWER SYSTEMS
DOI: 10.37394/232016.2024.19.4
Ali Mohammed Ridha, Wessam Shehieb
E-ISSN: 2224-350X
31
Volume 19, 2024