Adaptive Graph Convolution Algorithm Based on 3D Vision Selectivity and Its Application in Scene Segmentation

Breakthroughs in 3D sensor technology [1] that overcome the precision bottleneck of point cloud collection can capture accurate point cloud datasets, providing a data foundation for the rapid advancement of 3D point cloud data processing algorithms. These advancements have achieved good research and application results in fields such as drone control [2], autonomous driving [3], augmented reality [4], and medical image processing [5].

Conventional convolution algorithms based on 2D structured spaces cannot be directly applied to semi-structured 3D point cloud spaces. Based on this, researchers have proposed point cloud space structuring algorithms, direct point cloud processing algorithms, and direct structuring algorithms to address the issue of spatial feature extraction in semi-structured spaces.

Point cloud space structuring algorithms essentially solve the problem of structuring 3D point cloud data by structuring 3D point cloud space, facilitating the direct or expanded application of two-dimensional convolution methods to this 3D structured space. The main methods include: voxel-based approaches [6-8] and multi-view methods [9-11]. However, the structuring algorithms for 3D point cloud data have the following shortcomings: (1) Structuring of 3D space results in overlap of some data, compromising the purity and completeness of the original data points. (2) The process of structuring 3D space increases the complexity of the algorithms and reduces their performance. (3) Polarized structuring (such as the view method) loses three-dimensional topological structure information, affecting the recognition performance of the algorithms.

Therefore, direct point cloud processing algorithms have been proposed, which do not require structuring of 3D point cloud datasets and perform feature learning directly on the original datasets, attempting to address the aforementioned shortcomings. These include: PointNet [12], PointNet++ [13], and subsequent improvements to these algorithms [14-16].

Practice has shown that structured information expressed by spatial topological structure is an essential feature of 3D point clouds, and neither spatial structuring algorithms nor direct point cloud processing methods can effectively solve the learning and representation of 3D point cloud spatial structuring information. Graph structure is an excellent way to learn and represent spatial structuring information. Thus, the graph convolution method has become a direction for 3D point cloud research [17-20], achieving certain research and application progress.

The Dynamic Graph Convolutional Neural Network (DGCNN) algorithm [21] constructs local graph structures by identifying the nearest neighbors of 3D points in feature space and then performing edge convolution operations to extract features. Shen et al. [22] extended this idea and further learned spatial topological information during the feature aggregation process. Relation-Shape Convolutional Neural Network (RS-CNN) [23] applies the weighted sum of neighboring node features, where each weight is learned using an MLP based on the geometric relationship between the two points. These efforts aim to learn the local topological features of three-dimensional point clouds. 3D-GCN [24] uses a deformable 3D kernel to learn information from 3D point clouds and addresses the disorder and unstructured nature of point cloud data through graph-based max pooling methods, proposing a universal model concept. The GCN algorithm based on visual selectivity [25] proposes a method that combines point cloud information with point cloud topological structure features, establishing a graph convolution computation method. It effectively solves the problem of learning and integrating global and local features. The multi-view depth-adaptive GCN [26] method, integrating visual computing techniques with GCN algorithms, addresses the inability of existing GCNs to effectively learn the correlation between different scale visual features under multi-view (multi-domain) conditions. Point cloud transformer [27] presents a novel framework named Point Cloud Transformer (PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. TransNet [28] proposes a novel downsampling model based on the transformer-based point cloud sampling network (TransNet) to efficiently perform downsampling tasks. The proposed TransNet utilizes self-attention and fully connected layers to extract meaningful features from input sequences and perform downsampling.

Despite the theoretical and practical research achievements in 3D point cloud processing through GCN methods, there are still shortcomings: (1) The selection of neighboring space for graph convolution and spatial selection for information aggregation lacks theoretical support. (2) How to integrate features at different scales in deep graph convolution algorithms. (3) The algorithms do not utilize multi-view information for parallel fusion models, achieving parallel extraction and fusion of visual features, which results in single visual feature structures, weak correlations, and low classification accuracy.

Based on this, this study proposes an adaptive GCN algorithm based on visual computing theory and discusses its application in scene segmentation. The paper expands the theory of 2D visual selectivity to form a theory of 3D visual selectivity and proposes an adaptive graph convolution algorithm based on 3D visual selectivity (abbreviated as AGCNTDVS). Combining this algorithm with deep learning theory, we form a single-linkage depth-adaptive GCN algorithm based on 3D visual selectivity (abbreviated as SD-AGCNTDVS), which extracts visual features at different depths and effectively integrates them. Inspired by the parallel visual information processing pattern of primates, building on the single-linkage algorithm, we construct a multi-linkage deep adaptive GCN algorithm based on 3D visual selectivity (abbreviated as MD-AGCNTDVS), extracting and optimizing visual features from different linkages.

Therefore, the innovations of this paper include:

(1) Theoretically, inspired by the 3D vision of primates [29], we propose a theory of 3D visual computation. Based on 3D spatial visual selectivity, we propose the AGCNTDVS algorithm, which addresses the issues of spatial partitioning and information aggregation in graph convolution, and adaptively learns the global and local features of 3D point clouds.

(2) Inspired by the serial visual information processing pattern of primates, we integrate the AGCNTDVS algorithm with deep graph convolution learning theory to construct a SD-AGCNTDVS algorithm, which learns deep abstract features of 3D point clouds at various depths and achieves feature integration.

(3) Inspired by the parallel visual information processing pattern of primates [30], and using viewpoints as a standard, we expand the SD-AGCNTDVS algorithm to construct the MD-AGCNTDVS algorithm to learn multi-linkage features, forming a feature fusion matrix, and representing the optimal visual area.

(4) We apply this algorithm to the custom Mortise_and_Tenon_DB dataset for the digital reconstruction of the joint structure, achieving adaptive segmentation of mortise and tenon structures. This validates the correctness of the structural segmentation and provides effective support for the intelligent design of ancient Chinese buildings.

Based on the theoretical analysis mentioned above, this paper proposes and implements three algorithms: AGCNTDVS, SD-AGCNTDVS, and MD-AGCNTDVS.

3.1 The AGCNTDVS

Inspired by the theory of 3D visual space selectivity, this algorithm integrates 3D visual selectivity with graph convolution algorithms, proposing the AGCNTDVS. The main steps of AGCNTDVS include spatial selection, information aggregation, and activation, detailed as follows.

030201.png

(a) K-nearest neighbor clustering space division diagram

030202.png

(b) Graph convolution information aggregation diagram

Figure 2. Information aggregation method of graph convolution based on visual selectivity

(1) Spatial Selection

Inspired by the theory of 3D visual selectivity, the selection process mainly targets the global visual space $\Omega$, utilizing the K-means clustering method to achieve the subdivision of the global space into subspaces, denoted as $\Omega=\bigcup_{k=1}^{k=K} \Omega_k\left(o_k, R_k\right)$ Elements within $\Omega_k$ have the same or similar functions (receptive fields).

(2) Information Aggregation

In the visual subspace $\Omega_k\left(o_k, R_k\right)$, for $\forall q \in \Omega_k$, the spatial topological information of observation point $q$ and its Lnearest neighborhood $\Omega_{\text {Neighbor }}\left(R_L, q\right), L=0,1,2, \ldots,, K$ is represented as vector $\mu=\left(\alpha, \beta, \gamma, R, w_i^{j k}\right)$. In this paper, the function $\Phi=$ MLP, meaning a MLP is used to learn the spatial topological structure between neighbors (Figure 2). Therefore, the information aggregation of observation point $q$ and its neighborhood $\Omega_{\text {Neighbor }}\left(R_L, q\right)$ are represented as:

$\begin{gathered}\text { topological_Feature }\left(R_i, q\right)= \\ \operatorname{MLP}\left(\operatorname{CONCAT}\left(\left(\operatorname{MAP}\left(R_i, N_0^j\right), j=\right.\right.\right. \\ \left.\left.\left.1,2,3 \ldots, \text { nember }\left(\Omega_{\text {Neighbor }}\left(R_L, q\right)\right),\right)\right)\right)\end{gathered}$     (2)

$\begin{gathered}\operatorname{MAP}\left(R_i, N_i^j\right)= \\ \operatorname{MLP}\left(\operatorname{CONCAT}\left(\left(\operatorname{MAP}\left(R_{i-1}, N_{i-1}^j\right), j=\right.\right.\right. \\ \left.\left.\left.1,2,3 \ldots, \operatorname{nember}\left(\Omega_{\text {Neighbor }}\left(R_L, q\right)\right)\right)\right)\right)\end{gathered}$     (3)

Inspired by visual selectivity theory, topological_Feature has a stable number of points and stable spatial structure, and for $\forall \mathrm{q} \in \Omega_k\left(o_k, R_k\right)$, point $q$ has a stable number of points and structure, which is independent of the input point order.

(3) Activation

To increase the non-linearity of the features, the algorithm uses the ReLU(R) activation function, with different pooling radii used for different visual areas.

The pseudocode for the AGCNTDVS algorithm is shown in Table 1.

Table 1. Pseudocode for the AGCNTDVS algorithm

3.2 The SD-AGCNTDVS

Table 2. Pseudocode for the SD-AGCNTDVS algorithm

Input: point cloud data $(\Omega)$; hyperparameter R; observation point $P_i$; Network depth parameter L.

Produce: S1: Parameter Initialization. Initialize the network depth parameter Net_Deep_Len $=1$; select observation set $\Omega_k\left(R, p_i\right) \subset \Omega$ S2: 3D Visual Depth Calculation. In the 3D visual set $\Omega_k\left(R, P_i\right)$, sequentially execute the AGCNTDVS. The process involves: AGCNTDVS ${ }^{\text {Net_Deep_Len }+1}=$ AGCNTDVS ${ }^{\text {Net_Deep_Len }}\left(\Omega_k\left(R, P_i\right)\right) \quad, \quad$ incrementing Net_Deep_Len by 1, and linking the generated features of different layers in sequence. S3: If Net_Deep_ Len $\leq \mathrm{L}$, the algorithm loops back to step S2. Otherwise, the SD- AGCNTDVS algorithm ends, and deep feature extraction is complete.

Output: Output features formed by linking visual features at different depths.

Inspired by the serial processing mode of 3D visual information, this paper integrates the AGCNTDVS algorithm with deep learning theory to construct the SD-AGCNTDVS. The AGCNTDVS algorithm extracts abstract visual features of the 3D point cloud from different depths and merges these features to form complex visual features, enhancing the capability of the algorithm to extract visual features and enriching the expression of visual features. Thus, the pseudocode for the SD-AGCNTDVS is shown in Table 2.

3.3 Semantic segmentation algorithm based on SD-AGCNTDVS

Use the SD-AGCNTDVS algorithm for semantic segmentation of three-dimensional point cloud data, achieving multi-scale feature aggregation of the $3 \mathrm{D}$ point cloud collection and using a per-point classification shared MLP algorithm. Since the algorithm uses a pooling mechanism, the feature lengths of three-dimensional point cloud data between different layers do not match. Based on this, the following feature filling operation, the Fill_algorithm, is defined:

$\text { Fill }_{\text {algorithm }}=T D E F_q(p, R)= \\

\operatorname{argmin}\left(T D E F_{\bar{q}, r}(p, R)\right), \text { and } \bar{q} \in \dot{U} U(q, r)$     (4)

Here, $q$ is the point to be filled, and $\dot{U}(q, r)$ represents the punctured r-neighborhood of point $q$.

Thus, the model for the semantic segmentation algorithm based on the SD-AGCNTDVS (abbreviated as SSMAGCNTDVS) is given below (Figure 3).

(1) Traditional segmentation algorithms

030303.png

Figure 3. The 3D SSM-AGCNTDVS model based on the AGCNTDVS algorithm

3.4 The MD-AGCNTDVS

The AGCNTDVS extracts deep visual features of the 3D point cloud; it has not studied the breadth visual features of the 3D point cloud and the correlation of its breadth features. The clustering radius r of the AGCNTDVS algorithm determines the learning of breadth features and the analysis and application of feature correlations, with a microscopic analysis as shown in Figure 4.

The above figure shows that different visual neighborhoods $R$ correspond to different neighborhoods of point $p$; on different neighborhoods, the features of point $p$ aggregated by the AGCNTDVS algorithm vary; when $R$ is too small, the visual region $\Omega_q(R, p)$ of point $p$ is too small, leading to insufficient expressive regions for the graph convolution, resulting in poor completeness; when $R$ is too large, the visual region $\Omega_q(R, p)$ is too large, containing different functional cells, and the visual information within the region does not satisfy visual selectivity, resulting in poor purity of visual features extracted from the $\Omega_k\left(R, P_i\right)$ collection; therefore, the hyperparameter $R$ is set during the experimental process.

Based on this, this paper proposes the MD-AGCNTDVS, which adaptively extracts features within several consecutive visual regions corresponding to different $R$ values, to learn the optimal or suboptimal global and local spatial features, addressing the shortcomings of the SD-AGCNTDVS algorithm. The pseudocode for the MD-AGCNTDVS is shown in Table 3.

030401.png

a) $R_{i-1}$ order visual region

030402.png

b) $R_i$ order visual region

43.png

c) $R_{i+1}$ order visual region

Figure 4. Microscopic relationship between visual regions and feature extraction by the 3D GCN algorithm

Table 3. Pseudocode for the MD-AGCNTDVS algorithm

Input: point cloud data $(\Omega(R, p)$; hyperparameter set $\mathrm{R}$; observation point $p$ (set $p$ is the triplet of continuous natural number); Network depth parameter L.

Produce: S1: Algorithm Initialization: Based on the hyperparameter $R$, generate neighborhoods $\Omega\left(R_1, p\right), \Omega\left(R_2, p\right), \ldots, \Omega\left(R_{N 0}, p\right)$, where $R_1 \leq R_2 \leq \cdots \leq R_{N 0}$ S2: Multi-feature learning: Start multiple GPUs or threads to parallelly learn visual features from different visual regions $\Omega\left(R_i, p\right)$ : $\operatorname{TDEF}\left(p, R_i\right)=$ Single link deep AGCNTDVS $\left(\Omega\left(R_i, p\right)\right)$ S3: Multi-domain feature fusion learning: $H\left(T D E F\left(p, R_{-} i\right)\right)$, where $H(\cdot)$ is a feature fusion function.

Output: Output different depth visual features and their fusion features.

3.5 Application model of MD-AGCNTDVS for 3D point cloud segmentation

The function H(·) fuses the segmentation results of the multi-domain adaptive GCN algorithm based on visual computing theory. In this paper's experiments, it can be implemented using an MLP algorithm, or through algorithms like MaxPool.

$\begin{aligned} & H(.)=\operatorname{MLP}\left(\operatorname{MDDVSA3DGCN}\left(\mathrm{R}_i\right)\right) \\ & \text { or } \operatorname{MaxPool}\left(\operatorname{MDDVSA3DGCN}\left(R_i\right)\right)\end{aligned}$     (5)

Here, the MaxPool algorithm learns the global maximum of different visual region features, representing global features and ignoring the details of features, with a lower time complexity. Meanwhile, the MLP algorithm learns the fused features of different visual regions, including the correlation between various features, hence, the features are more representative, but the feature fusion has higher time complexity.

3.6 Algorithm invariance

Existing works such as [14, 21, 23, 26, 31-34] have reported good geometric invariance performance, but they typically consider global coordinates or require point cloud normalization to mitigate this data variation, which could limit their invariance. The algorithm in this paper learns directional information and its directional selectivity within local receptive fields through 3D convolution kernels, and these features are formed independently of specific positions and distances, further enhanced by pooling mechanisms to improve the algorithm’s geometric invariance. Therefore, the algorithm exhibits good translational, scale, and rotational invariance.

This paper proposes "Multi-domain adaptive graph convolution algorithm based on visual computing theory and its scene segmentation application". Inspired by the 3D vision of primates, a 3D vision computing theory is proposed; based on this, the algorithm of nearest neighbor space range of graph convolution is definition, and an adaptive graph convolution algorithm based on 3D vision selectivity is proposed. Inspired by the serial/parallel processing model of primate visual information, a single-link/multi-link depth adaptive graph convolution algorithm based on 3D visual selectivity is constructed to learn and integrate the depth and breadth visual features of point clouds. and global and local visual features; the MLP algorithm with shared weights is used to achieve object segmentation. On ShapeNetPart and custom Mortise_and_Tenon_DB, compare with algorithms such as PointNet, PointNet++, KPConv deform, 3D GCN and My algorithm to verify the segmentation performance and geometric invariance of this algorithm. The experimental results show that the algorithm of this paper has good segmentation performance, and the segmentation success rate reaches 90.9%; the algorithm in this paper has strong geometric invariance, rotation and translation transformations have geometric invariance, and in the interval [-0.15,0.15], the telescopic transformation has limited geometric invariance. transsexual.

Future optimizations of the algorithm will focus on two main aspects to improve performance:

(1) Verify the feasibility and advantages of this algorithm on more data sets (especially large scene data sets), and prove the universality of the algorithm.

(2) The multi-link parallel depth vision algorithm involves many parameters, the parameter settings are more complex, and parameter optimization and algorithm problems are prominent. Parameter optimization and adaptive setting are important follow-up research directions.

(3) The algorithm parameter setting determines the computational complexity of the algorithm, and determines the time and space complexity of the algorithm. With the improvement of algorithm optimization methods, the time and space complexity of the algorithm under optimization conditions will be further explored.