WSEAS Transactions on Systems and Control
Print ISSN: 1991-8763, E-ISSN: 2224-2856
Volume 18, 2023
Synchronized Control of Robotic Arm based on Virtual Reality and IMU Sensing
Authors: , ,
Abstract: This study introduces a robotic control system that combines virtual reality integration and inertial measurement units (IMU) using mixed reality (MR) devices. The system integrates three main modules: (1) virtual reality (VR), which simulates remote reality by the Hololens2, (2) the wearable IMUs device, which captures the operator's hand movements; and (3) the robotic arm UR5, which is controlled by the user and the VR environment. The virtual and physical systems communicate via a Message Queuing Telemetry Transport (MQQT) communication architecture to establish communication between modules. To introduce a closed-loop control system for the robotic arm, model predictive control (MPC) was achieved with precise path planning to provide a flexible, intuitive, and reliable method to operate the remote-controlled manipulator. To validate the system integration and functions, two demonstrations were conducted: (a) the offline mode, where the VR module of the robotic arm was controlled by the IMUs device to check correctness, and (b) the online mode, where the control command was transferred to UR5 to complete a target mission via artificial potential field (APF) adjustment. The primary outcome of this study was the development of virtual and real industrial robotic arms to test the developed model in VR and shop floor labs.
Search Articles
Keywords: model predictive control (MPC), MQTT, Hololens2, artificial potential field (APF), virtual reality, path planning, visual localization, inertial measurement units (IMU)
Pages: 460-468
DOI: 10.37394/23203.2023.18.49