Multi-Source Data Fusion and Target Tracking of Heterogeneous Network Based on Data Mining

2.1 Target tracking of heterogenous network

Traditionally, researchers of target tracking regarded sketch as the expression of shape contour, and tried to describe sketch features with geometric relationship [11]. Features are usually extracted with the feature descriptors specially designed for sketches, e.g., edge direction histogram of binary shape feature descriptor, key shape learning, gradient field as natural image feature descriptor, and scale invariant feature. Then, the similarity between the grass image and the edge image is measured by Euclidean distance and other metrics, and the candidate image is selected through similarity matching. Finally, the output is sorted and retrieved, completing the image retrieval process. However, the sketch is so abstract that the generated feature descriptors cannot effectively fit the content of the sketch, and thus cannot be easily used in real scenes [12]. In addition, it is difficult to realize an end-to-end retrieval system by traditional methods, causing a substantial growth in workload.

In 2012, Hinton’s team won an image recognition competition with the AlexNet, which set off a new wave of heterogeneous networks [13]. Heterogenous networks like CNN and deep neural network (DNN) become the primary tools to solve relevant problems [14]. Unlike traditional manual approach of feature extraction, heterogenous networks adopt a layer-by-layer design. Mimicking human perception, a heterogenous network can learn depth features of different levels (low, medium, or high levels) from hand-drawn sketches or natural images, and learn the abstract pixel information hidden in the target image [15]. In many fields, the fusion algorithm based on heterogeneous network has achieved good results. Studies have shown the immense value of using artificial neural network (ANN) fusion to handle nonlinear problems. For instance, ANN has an outstanding performance in obstacle detection [16].

2.2 Data fusion of bidirectional long short-term memory (BLSTM)

As an improved version of recurrent neural network (RNN), BLSTM overcomes the difficulty in processing long-term dependent information, and effectively acquires the context of temporal data [17]. Ilić et al. [18] proposed a deep CNN (DCNN) fusion framework, which fuses input features excellently layer by layer, but ignores the time features of data. To improve the fusion performance, two-layer CNN was combined with the LSTM, and single-layer CNN with BLSTM.

The coupling of DCNN with BLSTM can extract the temporal and spatial information of eye tracking data. The literature shows that, when the time window is longer than 1s, eye movement can be captured well. Since the sampling frame rate of the test data is between 24 and 25 frame per second (fps), the time window was set to 25. Specifically, each convolution layer performs a convolution operation, and the window length is reduced by 2. Thus, 6 data need to be filled for the three convolution layers.

Image filling is a strategy involving the followings steps: Read the input image, decode the JPEG content into RGB grid pixels, convert the results into floating-point tensor, and rescale the pixel values from (0, 255) to [0, 1]. Then, multiple images could be imported, and fused into a new image of the same class. The new image contains the information of the multiple input images. After that, linear interpolation is carried out in the potential space, and image fusion is conducted by a certain proportion. Finally, the length of input vector (time window) is extended from 25 to 31 through image filling, such as to match the length between input and output vectors, and retain the temporal features of eye tracking data [19].

For eye-CNN-BLSTM, the parameters are initialized empirically, and optimized through repeated experiments. The optimal scheme is as follows: First, one DCNN is adopted to extract the spatial information from the input data. The three convolution layers have 16, 8, and 4 convolution kernels, respectively. Before each convolution, the data are subjected to batch normalization (BN), because the data distribution can be changed through network training [20]. The convolution part adopts the rectified linear unit (ReLU) as the activation function. This sparse activation function can speed up network training [21]. Besides, the fully connected layer is adopted to connect all the features extracted from the convolution layer, and a dropout operation is added to enhance the generalization ability of the model [22]. Furthermore, the tanh function is taken as the activation function, to prevent the excessively large output of ReLU [23]. Finally, a fully connected layer is selected as the output layer of the fusion value.

2.3 Heterogenous network model for target tracking

Ground base station (G-BS) and unmanned aerial vehicle (UAV)-mounted base station (U-BS) are two millimeter wave heterogenous networks built for real hot scenes. The G-BS and U-BS are composed of macro- and micro- cell, respectively [24]. 

In order to reduce the traffic load of G-BS, it is assumed that U-BSs in the air are clustered and projected around G-BS. On the Euclidean plane, the position of a G-BS is modeled as a dense, homogeneous network of G-BSs, while that of a U-BS is modeled as a PCP with parent point process of G-BS, for the U-BS is deployed in hot spots. Therefore, the U-BS is further modeled as U-BS/G-BS, and its projection is scattered around the G-BS, following the symmetrical Gaussian distribution with a variance of 2 [25].

A multi-relay multi-user millimeter wave hybrid heterogeneous network consists of a macro cell source node s, K small cell relay nodes R_k(k = 1, 2, 3,..., K), and M ground destination nodes D_M (M = 1, 2, 3,..., m), which correspond to each relay node. Each relay node has a receiving antenna (RN) and a transmitting antenna (TN). Each other node has a single RN and a single TN. Each relay node is far from other relay nodes, and the M destination nodes of each RN form a cluster. Therefore, the following mathematical expression can be derived:

$\begin{aligned}

&\text { G-BS }(\mathrm{x})=2 n \ln (\sigma)+n \ln (2 \pi) \\

&+n\left\{\frac{n+t r(S)}{n-2-t r(S)}\right\} * \mathrm{DV}

\end{aligned}$     (1)

where, x is position; s is density. Moreover, we have:

$y_{i}=\mathrm{IR} * \beta\left(u_{i}, v_{i}\right)+\sum_{j=1}^{p} \beta_{j}\left(u_{i}, v_{i}\right) x_{i j}+\varepsilon_{j} \beta_{j}$      (2)

where, IR is the trade-off coefficient of the objective function; DV is a soft max cross entropy loss function. In addition, we have:

$\sigma_{\text {ikj1 }}=\left\{\begin{array}{l}

\frac{\mathrm{n}}{\Delta_{\text {ikj1 }}} \sqrt{\sum_{\mathrm{s}=1}^{\mathrm{n}}\left(\mathrm{x}_{\mathrm{ik}}(\varepsilon)-\mathrm{x}_{\mathrm{j} 1}(\varepsilon)\right)^{2} \Delta_{\mathrm{ikj} 1}(\varepsilon)} \quad \Delta_{\mathrm{ikg} 1}>0 \\

0 \quad \Delta_{\mathrm{ikjl}}<0

\end{array}\right.$     (3)

$y_{i}=f\left(\sum_{j=i}^{k} \omega_{i j} y_{j}-\theta_{i}\right)-\mathrm{C}^{*} \mathrm{~N}, i \neq j$     (4)

where, n is the total number of initial training samples in X; K is the total number of occlusion transformations in each x; C is the total class. After the above steps, the relaxed nonconvex minimization problem can be written as:

$L_{r}=\left\|D_{I}-D_{O}\right\|_{2}$     (5)

$I R=\sum_{S-1}^{U} \sum_{d-1}^{K} f_{s}, D V_{s}, d$     (6)

$\begin{aligned}

&W(T)= \\

&K(y(T-1), \ldots, y(t-n), u(T-d-1), \ldots, u(k-d-n))

\end{aligned}$     (7)

$W(T)=K(y(T-1), u(T-d-1))$     (8)

In addition, two additional layers can be constructed: one layer of one U-BS, and three layers of three G-BSs. The two layers are composed of inter-cluster U-BS and inter-cluster G-BS, respectively. Therefore, the network model can be extended as follows: Layer 0 of intra-cluster U-BS, Layer 1 of inter-cluster U-BS, layer 2 of intra-cluster G-BS, and layer 3 of inter-cluster G-BS. Then, the logarithmic hyperbolic cosine function can be compared with other functions:

$\mathrm{U}=\sum_{\mathrm{i}=1}^{\mathrm{g}}\left\{\mathrm{P}_{\mathrm{i}} \mid \sum_{\mathrm{j}=1}^{\mathrm{k}} \mathrm{p}_{\mathrm{j}}^{(\mathrm{i})}\right\}$     (9)

$\text { overlap }=\frac{\left|R^{g} \cap R^{r} \| Z_{0}-Z\right|}{R^{g} \cup R^{r}}$     (10)

$\begin{aligned}

&P\left(D_{i}, w_{j}\right)=P\left(d_{i}\right) P\left(w_{j} \mid d_{i}\right) \\

&P\left(w_{j} \mid d_{i}\right)=\sum_{k=1}^{K} P\left(w_{j} \mid z_{k}\right) P\left(z_{k} \mid d_{i}\right)

\end{aligned}$     (11)

where, P is the position of G-BS in the cluster; Z₀, Z are the locations of service U-BS and interference U-BS in the air cluster, respectively; R is a typical ground user (GUE); K is the distance from the horizontal projection of the air service U-BS of the typical GUE to the representative cluster center and the typical GUE; D is the distance from the horizontal projection of the air interference U-BS to the typical GUE and the representative cluster center; G-BSx is the position of the G-BS between clusters; v = x is the distance from the typical GUE to the G-BSx of the cluster center; V = x is the distance from the plane projection of the air interference U-BS between clusters to the typical GUE and the cluster center G-BSx. The heterogenous network of G-BS and U-BSs can be described as:

$U=\left\{P_{1}\left|D, L, f_{2}, Q, d, l \quad P_{2}\right| f_{1}, \mu \quad P_{3} \mid N, M, I\right\}$     (12)

where, D is the constraint of the feature change of the objective function. From the constraint, it is possible to derive the feature error of clear and occluded samples, and reduce the error through optimization. When the error is reduced to a certain extent, the feature of the clear sample can be approximated to the mean feature of the occluded sample. Furthermore, we have:

$L\left(Y_{i}, y_{i}\right)=-\frac{1}{n} \sum_{i=1}^{n}\left[Y_{i} \log \left(y_{i}\right)+\left(1-Y_{i}\right) \log \left(1-y_{i}\right)\right]$     (13)

$H(x)=F(x)+x$     (14)

where, H is the transmitting antenna at G-BS; t = u is the transmitting antenna at U-BS; f is the receiving antenna at GUE. The maximum transmission gain received by a typical GUE from the layer I transmitter can be described by:

$\Delta_{\text {ikil }}=\sum_{\delta=1}^{n} \Delta_{\text {ikjl }}(\varepsilon)$     (15)

According to the total transmission gain that could be received by a typical GUE from the layer I transmitter T, the objective function can be obtained by:

$\Delta_{\mathrm{ikjl}}(\varepsilon)= \begin{cases}0, & x_{i k}(\varepsilon)=N / A \quad \text { or } \quad x_{j l}(\varepsilon)=N / A; \\ 1, & x_{i k}(\varepsilon)=N / A \quad \text { and } \quad x_{j l}(\varepsilon)=N / A\end{cases}$     (16)

$x_{i}=\sum_{j=1}^{n} \omega_{i j} y_{j}-\theta_{i}$     (17)

The advantages of the multi-layer network can be highlighted through cascading. Therefore, two cascading schemes, namely, two-layer cascading scheme, and our four-layer cascading scheme, were compared with each other. The former only considers the typical GUE cascading with GBS/UBS within the cluster, while latter fully considers all cases of typical GUE cascading with GBS/UBS within and between clusters. In other words, the 4-layer cascading scheme helps to quantify the impact of GBS/UBS on the performance of UAV-assisted multi-layer millimeter wave heterogeneous network. In addition, both schemes follow the maximum binary recursive portioning (BRP) criterion, that is, the typical GUE is cascaded with the G-BS/U-BS which provides the strongest mean BRP in the long term. Therefore, when a typical GUE is cascaded with the nearest G-BS/U-BS at x in layer I, the following problem will occur:

$\begin{aligned}

&P\left(d_{i}, w_{j}\right)=P\left(d_{i}\right) P\left(w_{j} \mid d_{i}\right) ; \\

&P\left(w_{j} \mid d_{i}\right)=\sum_{k=1}^{K} P\left(w_{j} \mid z_{k}\right) P\left(z_{k} \mid d_{i}\right)

\end{aligned}$     (18)

where, P(d_i, w_j) is the minimum path loss from a typical GUE to layer K. According to the cascading standard, the cascading probability refers to the probability of typical GUE cascading with G-BS/U-BS in LOS/NLOS state in layer I. Since the G-BS/U-BS of layers 0 and 2 are in the representative cluster, and only one G-BS/U-BS exists in the cluster nearest to the typical GUE, the link state between the typical GUE and the G-BS/U-BS in the cluster can generally be changed by adjusting the position and height of the UAV. Therefore, the link state is either LOS or NLOS. Treating the path losses of layers 0 and 2 as two states, the probability of the typical GUE cascading with G-BS/U-BS can be defined as:

$\mathrm{UBS}=f\left(W^{e} D_{1}+\delta^{e}\right)$     (19)

Taking U-BS as the input, the feature constraint can be added through the objective function to make the occluded sample close to the eigenvector of the clear sample. Then, the deviation between the target value and the actual output, and the deviation and characteristic error between the calculated target value and the actual output, were used as the characteristic parameters. In addition, the forward propagation of the network was completed through the primary and secondary approximation of the front and back terms. After the training, the weights and thresholds were determined, completing network modeling.

4.1 Energy efficiency of multi-target heterogenous network

As shown in Figure 1, the energy efficiency of the network changes with target density, at the user rate demand of 0.1, 0.5, or 1. When the user rate demand remained constant, network interference and energy consumption increased with target density, causing a gradual decline in network energy efficiency. When the target density remained constant, the network needed more transmission power to meet the rising user rate demand, which also leads to a downward trend of network energy efficiency. Therefore, the proposed DM-DQN can dynamically adjust the network state according to user QoS and optimize the energy efficiency of the network.

1.png

Figure 1. Advantages of image classification algorithms

As shown in Table 1, the higher the target density, the greater its influence on network performance. In DM-DQN, agents constantly interact with the environment. The RB allocation strategy and the corresponding power allocation strategy are taken as network actions to optimize the network performance, and their interaction is considered in a comprehensive manner. Through continuous trial and exploration, agents gradually approach the best strategy. After DNN training, the agents can adaptively adjust the strategy for data fusion and target tracking of the heterogenous network, according to the changing environment of the network. That is why our network performs better than data mining algorithm and two-stage algorithm.

As shown in Figure 2, the total energy efficiency of the network changed with target density, under the user rate demand of 0.5m. The growing density of nano targets in the network suppressed the overall network energy efficiency of every algorithm. The reason is that more nano targets push up network interference and energy consumption, resulting in poorer network performance.

As shown in Table 2, DM-DQN was more energy efficient than the typical data mining algorithm and two-stage algorithm, and achieved comparable effect as the efficiency traversal algorithm with the optimal energy efficiency. This is because the two-stage algorithm optimizes RB allocation and power control in two separate stages. The RB allocation stage could avoid some network interference, but the allocation strategy is predetermined during the power control. Thus, the overall performance of this algorithm can only be improved by a limited margin.

2.png

Figure 2. Variation of total energy efficiency of the network

4.2 Influence of link cascading on target tracking

As shown in Figure 3, the probability of a typical GUE and G-BS/U-BS in LOS link state was generally higher than that in NLOS link state. At a small Rb value, the probability of typical GUE cascading with G-BS/U-BS within the representative cluster was usually greater than that of typical GUE cascading with G-BS/U-BS between clusters.

Table 1. Continuous interaction between agents and environment

Table 2. Superiority of DM-DQN in energy efficiency

3.png

Figure 3. Performance of small sample image recognition

Table 3. Relationship between cascading probability and G-BS density

As shown in Table 3, the cascading probability A1 and A3 of typical GUE and G-BS/U-BS between clusters increased with G-BS density, while the cascading probability NA of typical GUE and G-BS/U-BS within representative clusters decreased with the increase of G-BS density.

4.png

Figure 4. Model of image target detection

As shown in Figure 4, under the user QoS constraint in DM-DQN, the higher the target density, the greater the network interference, and the more transmission power is needed to ensure user rate. Therefore, the lower the target density, the smaller the gap between DM-DQN and the enumeration algorithm. When the user rate demand was 0.5m, the total user delay in the image increased with target density.

Table 4. Effectiveness of DM-DQN on ensuring user QoS

As shown in Table 4, after 160s of tracking, the modified mean error of the data mining based on the proposed model was only 1.5m, a sign of ideal tracking effect. Therefore, DM-DQN performs better than other algorithms in total user delay. DM-DQN can improve energy efficiency of the network, and guarantee the user QoS. With the increase of target density, the number of users in the network increased, the network interference intensified, and the total user delay gradually lengthened. As for the enumeration algorithm, the optimal energy efficiency is taken as the optimization objective to maximize the energy efficiency of the whole system. When the target density increased, the rate of individual users declined, and the total network delay grew, resulting in a longer delay of the enumeration algorithm. The proposed DM-DQN effectively reduces the network interference and ensures the user rate by taking the total user delay as a part of the return function, and optimizing the RB and power jointly, with the RB allocation and power allocation strategies as executive actions.

Table 5. Iterative convergence of algorithms

As shown in Table 5, DM-DQN gradually converged after nearly 100 iterations. In the first 50 iterations, DM-DQN was outperformed by structured data mining. In the early phase, structured data mining learns from the initial feedback, while DM-DQN only randomly selects actions and stores the feedback in the playback experience pool. After 50 iterations, DM-DQN began to learn from the stored feedback. After 100 iterations, both DM-DQN and structured data mining tended to be stable, with the former having an edge in performance. Overall, DM-DQN converged faster and performed better than the typical structured data mining algorithm.

5.png

Figure 5. Joint RB allocation and power control

The optimization problem of joint RB allocation and power control was proposed for the ultra-dense heterogeneous network, in the light of user QoS, aiming to reduce the same layer and cross layer interference and improve the energy efficiency of the network. To avoid the high complexity of traditional algorithms, the DM-DQN framework was introduced, and a reward function was defined to optimize network energy efficiency and ensure user QoS. As shown in Figure 5, our DM-DQN algorithm guaranteed the QoS of users, and achieved closer-to-optimal performance than typical data mining, two-stage algorithm, and enumeration algorithm. Next, multi-agent-based distributed resource management would be studied, trying to reduce network interference and improve energy efficiency through multi-agent cooperation.

As shown in Figure 6, 5G/B5G network architecture could be improved from fixed ground infrastructure to air mobile connection. Millimeter wave communication is the preferred way to realize 5G/B5G ultra reliable and low delay communication. In the UAV-assisted wireless communication system, UAVs can fly out of the blocking area to establish LOS links, which can overcome the penetration loss and facilitate the transmission of millimeter wave signals. Hence, a multi-layer heterogeneous network was formed with the coexistence of ground cellular network and UAV network. Due to the irregularity of the location for ground target (G-BS)/UAV target (U-BS), it is important to realize accurate modeling and simplified analysis through mathematical methods, such as random geometry and spatial point process. In addition, the spatial coupling capacity is insufficient between G-BS/U-BS and UE.

6.png

Figure 6. Air mobile connection of 5G/B5G network architecture

7.png

Figure 7. Recognition accuracy and time

As shown in Figure 7, with the increase of SINR threshold, the AAT could reach the maximum, and the optimal SINR threshold stood at 0. When the SINR threshold was above zero, the AAT decreased with the rise of the threshold. From the AAT corresponding to the SINR threshold of 0 in Figure 7, it can be found that the available AAT decreased with the growing m value.

Table 6. Downlink available to G-BS/U-BS in 4-layer cascade scheme

As shown in Table 6, when the SINR threshold remained the same, the downlink AAT that can be obtained by G-BS/U-BS in the 4-layer cascade scheme was greater than that of the 2-layer cascade scheme. As a result, the total AAT that can be obtained by the 4-layer cascade scheme was higher than that of the 2-layer cascade scheme. This confirms that the proposed 4-layer cascade scheme helps to increase the available AAT and optimize the performance of the UAV-assisted multi-layer millimeter wave heterogeneous network.

8.png

Figure 8. Dynamic recognition accuracy of image tracking

Figure 8 compares the basic network, model structure, usage method, loss function, test dataset, retrieval accuracy, and pixel loss (which ensures that the pixel information is preserved in the acquired embedding) of four representative generalization methods of depth target tracking classes. Ideally, the sketch domain and the image domain can be fully aligned in the common pixel space. Under the ideal situation, better target tracking results can be obtained by properly retaining more effective pixels in the learning process, thereby completing the retrieval of detail pixels.

9.png

Figure 9. Quantization error introduced through binarization

As shown in Figure 9, the quantization error introduced by the binarization of hash algorithm destroyed the domain-invariant information and the pixel consistency across domains. Therefore, this paper also proposes a generating hash algorithm to reconstruct the zero-learning knowledge representation. However, a high retrieval cost was incurred by the inefficiency of Kronecker fusion layer. It is often necessary to use expensive sketch image pairs, e.g., sketchy dataset, to better align the pixels in the common mapping space between the sketch domain and the image domain.

As shown in Figure 10, a typical GUE and G-BS/U-BS was much more likely to be in LOS link state than to be in NLOS link state, that is, LA was usually greater than 0. In addition, the density G-BS did not greatly affect the cascading probability, especially when the typical GUE and the G-BS/U-BS in the representative cluster were in the LOS link state. In this case, the cascading probability was basically invariant with the rising G-BS density. When the mean number of cluster members m was different, the authors discussed the relationship between AAT and SINR threshold, and compared the AATs between different cascading schemes. The discussion shows that the rise of SINR threshold does not always benefits AAT improvement. When the SINR threshold is relatively small, the AAT increases with SINR threshold.

10.png

Figure 10. Typical GUE and G-BS/U-BS in LOS link state