LSTM-Based Visual Control for Complex Robot Interactions

Intelligent robots can interact safely with their environment and perform tasks that resemble human actions, thanks to the utilization of exteroceptive sensors such as cameras. Cameras provide abundant information about the environment, and algorithms extract visual features that can be used to synthesize a vision control law. In recent years, researchers have addressed the problems in visual servoing [1]. Most of these control laws are based on regulating a task function that minimizes the error between the actual and measured visual data. The traditional approach involves calculating the interaction matrix (visual Jacobian) and its pseudo-inverse for each visual feature point. However, these conventional controllers have large interaction matrices and pseudo-inverses, which significantly increase calculation time, particularly for complex visual trajectories. Additionally, conventional controllers are unable to compensate for lost visual information.

Visual control is inherently challenging due to the complex and ever-changing nature of visual data in real-world settings. Images and video frames contain a lot of information, and environmental conditions, object positions, and scene configurations can vary unpredictably. Traditional control methods often struggle to adapt to this complexity and variability. LSTM, a type of neural network, is a favored solution for visual control due to its unique attributes. LSTMs excel at handling non-linear relationships between visual input and control actions, capturing temporal dependencies in data, and adapting to different tasks through training. They are well-suited for real-time processing, making them valuable in dynamic environments.

Furthermore, LSTMs have a well-established history of success in various domains, prompting their exploration in visual control applications where complex, high-dimensional, and dynamic visual data requires effective management.

Shi et al. [2] proposed a fuzzy adaptive method for decoupled image-based visual servoing (IBVS) to enhance the performance of a mobile robot's control. This method enables efficient control by addressing the under-actuated dynamics through the decoupling of translation and rotation, achieved by using two independent servoing gains, as opposed to the single servoing gain used in traditional IBVS. To improve convergence, the study incorporates an improved Q-learning approach to adaptively adjust the parameters of the image Jacobian matrix. Furthermore, the fuzzy method is utilized to tune the learning rate of the Q-learning technique.

Wang [3] have introduced innovative techniques for vision-based robotic manipulation tasks, specifically visual servoing. These tasks involve positioning the object at specific angles or shapes for further operations. The authors proposed a method that incorporates Bezier curves and non-uniform rational b-spline (NURBS) curves to accurately fit the object's contour and extract its image contour features. These features are then utilized to provide visual feedback on the object's shape to the eye-in-hand visual servoing system. By using this visual feedback, the system controls the robot's movements and guides it to converge the image features to the desired values, allowing precise positioning of the object for subsequent operations. Notably, their proposed method does not rely on prior knowledge of the object's 3D position, as it employs adaptive laws to estimate curve parameters online.

In examining the limitations of various existing methods, we find that feedforward neural networks (FNNs) are designed primarily for static data processing, lacking inherent temporal awareness needed for tasks involving time-series data or dynamic visual control. Their training demands intricate model tuning with a multitude of hyperparameters, potentially time-consuming and requiring expertise. Furthermore, FNNs often require a huge amount of labeled data, presenting challenges in scenarios where obtaining such data is costly or challenging. Fuzzy logic, another approach, relies on manually defined rules and membership functions, posing difficulties in crafting accurate rules for complex systems. It also lacks the autonomous learning capabilities essential for dynamic environments. While renowned for interpretability, achieving high performance in fuzzy logic often entails complex rule sets, compromising the system's interpretability-performance balance. Hybrid systems that merge fuzzy logic and neural networks encounter challenges in integrating these components seamlessly, potentially diminishing interpretability and straightforward learning. Additionally, generating effective fuzzy rules from data remains a complex task, and suboptimal rules may adversely affect system performance. It's essential to recognize that these limitations vary with context, and these methods, while constrained, continue to have value, often complementing other techniques in response to specific problem requirements and constraints. The LSMT method overcomes the limitations of the FFNs and fuzzy logic.

The Long Short-Term Memory is a specific type of neural network architecture designed to model complex sequences and patterns using a recursive structure. Unlike other neural networks, the LSTM can retain information over extended periods, making it adept at learning and predicting complex temporal relationships. In the literature, the most commonly used variant of the LSTM is known as the vanilla LSTM, which was initially proposed by Graves and Schmidhuber. This variant typically comprises a single memory unit and only one recursive connection, a configuration that has proven to be effective for a wide range of sequence modeling tasks. Examples of such tasks include speech recognition, natural language processing, and image captioning [4-6].

Despite its success, the vanilla LSTM model has limitations in terms of its ability to capture long-term dependencies and handle noisy data. To address these issues, researchers have developed a variety of modified LSTM architectures, such as the peephole LSTM, the gated recurrent unit (GRU) and the attention-based LSTM. These variants have been shown to outperform the vanilla LSTM on certain tasks and are widely used in both academic research and industry applications.

Liu et al. [7] utilized deep neural networks (DNNs) and LSTM to develop models for robot inverse dynamics, demonstrating the application of deep learning techniques. Rueckert et al. [8] also presented an inverse dynamics prediction model for manipulators based on LSTM, and their model exhibited satisfactory predictive performance. This increasing trend in using deep learning techniques for developing robot inverse dynamics models emphasizes the growing importance of artificial intelligence in robotics research and development.

Motivated by the previously proposed techniques, in this paper, we introduce a new approach, leveraging LSTM for adept control and optimization in complex systems with refined precision and adaptability. It not only inherits the robust memory and learning capabilities of LSTM in system control, producing a methodology that is both innovative and practically astute. Subsequent sections will delve into a detailed discourse on the various advantages brought forth by our LSTM-based approach, exploring its potential and applicability in control of robotics systems, which can offer several advantages that can be outlined as follows:

• It does not present any problems when dealing with high-dimensional data.
• The solution obtained is always unique.
• It uses few tuning parameters.
• Its implementation is stable.
• It allows obtaining precise results even with a small training set.
• It is suitable for complex situations where the number of variables exceeds the number of observations.
• It enables the elimination of non-significant data.
• It is robust to changes in the system's parameters.
• It allows for the construction of non-linear model.

The outline of the paper is as follows: We start by modeling a six DOF arm robot. Next, an overview of the conventional vision control law will be provided, and an enhanced robust visual controller will be suggested. Finally, the results obtained from the implementation of the proposed robust vision control law will be evaluated and discussed.

Robotic vision is a critical task performed by mobile, manipulator and humanoid robots, as it enables safe interaction with the environment. The vision task and visual servoing, which involve the performance of a control law, are strongly linked to the selected criterion or task function. Additionally, if some visual features are lost, there may be of a weakness effect on the control signal. On the other hand, the ability of the neural network is to make generalizations and to reduce the time of calculation.

In order to satisfy all these constraints, a control scheme based on LSTM is proposed as a specific type of recurrent neural network widely utilized in the field of deep learning. They are distinguished for their ability to learn and remember dependencies in sequence data, adaptation them particularly useful for time-series prediction, it can be used to calculate the inverse kinematic model. Moreover, LSTMs can also handle complex systems and nonlinear models, addressing unmodeled parameters such as friction, compliance, and environmental factors.

The vision control law intervenes to make a mapping from the error in visual features (ΔS) and the camera screw.

The visual information is given as follows:

$S=S(r(t))$                   (11)

where, S and r(t) are the visual features and the element of the space.

The differential of S allows knowing how the variations in the visual features are related to the relative velocity of the camera.

$\dot{S}=\frac{\partial S}{\partial r} \dot{r}=L_x T_c$                   (12)

L_x is a k×6 matrix, called interaction matrix related to S

$T_c$ is the relative instantaneous velocity of the camera (or kinematic screw vector).

The second step consists of defining a criterion related to the task at hand [16-18] and to minimize this criterion. The general form of a task function e is given by:

$e(r(t))={\hat{L_x}}^{+}\left(S(r(t))-S_d\right)$                   (13)

where, $S(r(t))$ is the current value of the visual information, $S_d$ the desired value and ${\hat{L_s}}^{+}$the pseudo-inverse matrix of the visual Jacobian which is given by:

$L_x(X, Z)=\left(\begin{array}{cccccc}-1 / Z & 0 & x / Z & x y & -\left(1+x^2\right) & y \\ 0 & -1 / Z & y / Z & 1+y^2 & -x y & -x\end{array}\right)$                   (14)

where, x and y are the coordinates of a given point in the image frame.

Note that this interaction matrix matches in this case.

$\dot{S}=L_x(X, Z) \cdot T_c$              (15)

where, $\dot{S}$ represents the velocities of the visual features and $T_c$ is the kinematic screw of the camera, which is attached to the end effector of the robot.

Because the camera is attached in the end effector, and from Eq. (10), we can deduce this equality:

$\underline{\dot{\theta}}={ }^0 J_6^{+} \underline{\dot{X}}={ }^0 J_6^{+} T_c$              (16)

By combining the Eq. (15) and Eq. (16):

$\underline{\dot{\theta}}=\left(L_x(X, Z)^{+}\left(J_6^{+}\right) \dot{S}\right.$              (17)

where, $\left(L_x(X, Z)^{+}\right.$is the pseudo inverse of the interaction matrix.

Eq. (17) describes the connection of visual feature velocities to joint velocities via the inverse kinematic model of the UR5 robot. Unfortunately, there are singularities and redundancies in the robot, and some visual features are lost. To overcome these drawbacks, we propose the LSTM method as a suitable solution.

During a tracking task performed by a robot, the process is typically non-linear, coupled and time-varying, which means that it can be challenging to achieve the desired behavior. In order to achieve satisfactory performance during such tasks, it is often necessary to regulate the control parameters, which may require tuning or adjusting the control law to adapt to changing conditions.

In order to implement the control law that involves using a LSTM controller, it typically requires the following steps:

(1) Initialization: The LSTM is initialized off-line from the identification of a classical visual control law. This involves training the parameters of LSTM using data obtained from the classical visual control law in order to establish an initial set of weights and biases (Figure 2).

(2) Online adaptation: An on-line adaptation of the LSTM parameters is implemented to regulate the LSTM's parameters according to the task being performed. This involves updating the weights and biases of the LSTM in real-time based on feedback from the system being controlled, in order to improve the performance of the system as depicted on Figure 3.

Overall, the two steps are essential to implement a control law that involves using a LSTM to regulate the behavior of the system.

where, $S$ is the current value of the visual information, $S_d$ is the desired value, $J^{+}$the pseudo inverse Jacobian matrix of the robot and $T_C$ the camera screw.

2.png

Figure 2. Vision control law identification

3.png

Figure 3. Revision control law

3.1 LSTM for identification and control

LSTM is suitable for system identification; it models complex system dynamics by capturing temporal input-output dependencies and managing non-linearities [19, 20]. For control, it predicts future states, detects anomalies, and adapts actions based on learned complex system dynamics, involving a comprehensive process from data preparation to real-time implementation. In applications such as robotics control, LSTMs facilitate trajectory prediction, though they require meticulous attention to data quality, computational resources, and model robustness for effective deployment.

In our study, the input to the LSTM is the error between the desired and actual visual features, while the output is the kinematic screw, as illustrated in Figure 2.

The principle idea of the LSTM is illustrated on Figure 4.

LSTM can remember information for long time, but more previous information may disturb the model accuracy. Typical LSTM module can have three gate activation functions σ. Two output activation functions are hyperbolic tangent functions (tanh) as illustrated in Figure 5.

4.png

Figure 4. LSTM principle scheme

5.png

Figure 5. LSTM mechanism

where, $h_t$ is the output of the current layer, $h_{t-1}$ denotes the output of the previous layer, $x_t$ the input, $C_t$ the cell state of the current layer, $C_{t-1}$ the cell state of the previous layer, $f_t$ the forget gate layer and $\tilde{C}_t$ the candidate vector.

One can define the system by introducing the forget gate layer given by:

$f_t=\sigma\left(W_{f \cdot}\left[h_{t-1}, x_t\right]+b_f\right)$             (18)

where, $\sigma$ is the sigmoid function, $W_f$ the weights of the forget layer, $x_t$ is the input vector and $b_f$ the bias of the forget layer.

Then, the input layer is updated using the following equations:

$i_t=\sigma\left(W_i \cdot\left[h_{t-1}, x_t\right]+b_i\right)$      (19)

$i_t$ : Is the output of the input gate layer'.

$W_i$ : Is the weights of the input gate layer'.

$b_i$ : Is the bias of the input gate layer'.

$\tilde{C}_t=\tanh \left(W_C \cdot\left[h_{t-1}, x_t\right]+b_C\right)$       (20)

$\tilde{C}_t$: Is the output of the input candidate layer’.

$\tanh$: Is the output of the input candidate layer’.

$W_C$: Is the hyperbolic tangent function.

$b_C$: Is the bias of the candidate layer’.

The output layer is given as follows:

$o_t=\sigma\left(W_o \cdot\left[h_{t-1}, x_t\right]+b_o\right)$                  (21)

$o_t$: Is the output of the output gate layer’.

$W_o$: Is the weights of the output gate layer’.

$b_o$: Is the bias of the output gate layer’.

The output of the output gate layer is given by:

$h_t=o_t * \tanh \left(C_t\right)$         (22)

The current cell state can be expressed by:

$C_t=f_t * C_{t-1}+i_t * \tilde{C}_t$        (23)

The proposed configuration for a recurrent neural network (RNN) consists of 3 layers of LSTM cells, with a total of 120 LSTM neurons. The activation function used for the LSTM cell state is set to 'tanh', which is a common choice for RNNs. The activation function used for the LSTM cell gate is set to 'sigmoid', which is also commonly used for this purpose [7, 15].

• The input weights are initialized using the Glorot initializer, which is a technique for initializing neural network weights, based on the size of the input and output layers. This can help improve network convergence and performance.
• The forget gate bias is initialized with ones, which means that the network will initially be biased towards remembering previous inputs. The remaining biases are initialized with zeros, which means that the network will initially be unbiased towards new inputs.
• The training algorithm used is back-propagation through time (BPTT), which is a popular algorithm for training RNNs. This algorithm uses the chain rule of differentiation to propagate errors backwards through time in the network.

The hyperbolic tangent function is used as an activation function as depicted by Eq. (24):

$\tanh (x)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$                (24)

where, $e^x$ is the exponential of the input vector.