Attributed Graph Convolutional Network for Enhanced Social Recommendation Through Hybrid Feedback Integration

Recommendation systems have been recognized as instrumental tools in addressing the challenges of information overload, guiding users towards valuable insights and predicting preferences across an array of items in alignment with individual tastes. In the evolving landscape of social commerce, a profusion of social connections, such as friendships, is generated within social networks. These connections are believed to significantly augment social engagements amongst individuals, whether they be acquaintances, colleagues, or even strangers [1]. In this digital realm, avenues for communication and the exchange of ideas are presented, fostering rich social interactions. Recommendations, derived from these social interactions, underscore the foundational importance of these social ties. An enhancement in the potency of social recommendations is, therefore, postulated to be closely tethered to these social connections. The principle of social correlation posits that within a given network, individuals are predisposed to manifest similar preferences or exhibit mutual influences, culminating in analogous choices within their interconnected web [1]. CF, which predicates user preferences on the notion that individuals with converging tastes are inclined towards analogous items, emerges as a quintessential methodology in the realm of social recommendation [2]. For the actualization of this principle, user-item interactions are reconstituted through a process that necessitates the parameterization of both users and items. Conventionally, most CF models can be demarcated into two predominant facets: embedding and interaction modelling. In initial methodologies, entities were projected into a shared latent space, and hidden vectors were employed to represent either entity. Subsequently, the emphasis shifted towards reconstructing user-item interactions leveraging these embeddings, as evidenced by techniques such as matrix factorization (MF) [3]. Advances like the collaborative topic regression (CTR) [4] amalgamated with deep feature representation learning further refined these interactions. A paradigm shift was observed with the advent of Neural Collaborative Filtering (NCF) and its derivatives [5], which merged linear MF and nonlinear neural networks, forsaking the traditional MF interaction function.

While the aforementioned methodologies have demonstrated efficacy, inherent limitations emerge when confronted with data paucity, inhibiting the generation of optimal embeddings for CF. In response to the challenges posed by data scarcity, attribute data has been integrated into traditional CF methods by several researchers [6]. To more accurately distil the essential elements from attribute data, techniques such as latent Dirichlet allocation, Bayesian personalized ranking (BPR), and autoencoders, among others, have been employed in previous studies [7]. However, a predominant reliance on inner products to emulate interactions between users and items has been identified, restricting these methodologies' capability to delineate non-linear relationships [8]. In a bid to rectify this, various approaches, inclusive of NCF, DeepFM [9], and Neural Factorization Machine [10], have harnessed deep neural networks, resulting in enhanced modelling of non-linear interactions. Notwithstanding their successes, the intricate ambiguities of latent representations pertaining to users and items have not been entirely captured by these profound neural architectures. Recently, the adoption of deep generative models, such as the Variational Autoencoder (VAE) [11], in CF tasks has been noted. Characterized by its probabilistic nature, VAE can encapsulate uncertainty, facilitating the exploration of non-linear probabilistic latent-variable models on extensive recommendation datasets, as evidenced by models like collaborative variational autoencoder (CVAE) [12] and VAE-based CF (VAECF) [13]. Nonetheless, certain caveats accompany these VAE-based CF methods. For instance, the dependency of CVAE on inner products for interaction representation has been observed to constrict its scope in capturing intricate non-linear interactions between users and items. Conversely, VAECF's exclusive reliance on rating matrices for modelling user behaviors and prediction generation is found to be suboptimal on exceedingly sparse rating matrices, further compromising its ability to furnish new users with accurate recommendations [14].

In the quest to ameliorate the shortcomings of insufficient embeddings, recent analyses have been undertaken [7, 15]. It was discerned that a preponderance of studies formulated embedding functions without the integration of implicit characteristics. As a result, the capability to efficaciously seize the fundamental collaborative signals, indicative of user-item interactions, was compromised. Furthermore, the employment of user-item interactions was discerned to be predominantly constrained to delineating training objectives, leading to the insufficiency in embeddings' capability to capture pertinent signals [8]. Such an observed disconnect between embedding and interaction modelling suggests that latent representations of users and items may not always be accurately discerned. Predominantly, information pertaining to users in social recommendation systems is gleaned from both social interactions and direct user-item interactions. However, this multifaceted information is not always holistically leveraged, often due to a lack of integrated analysis from diverse vantage points [1]. It should be emphasized that interactions between users and items not only denote direct exchanges but also encapsulate users' intrinsic item preferences. The nature of interactions within social networks is subject to the strength of user connections; stronger ties often manifest in more aligned preferences compared to weaker connections. A uniform treatment of these varied social interactions might inadvertently attenuate the efficacy of recommendation systems [16]. Hence, there emerges an intrinsic imperative to distinctly categorize these varied social ties, concurrently assimilating insights about both interactions and preferences. This comprehensive approach would involve amalgamating user information sourced from both social dynamics and direct user-item engagements, ensuring the embedding function is optimally primed to extrapolate the requisite collaborative signals for enriched representation.

To address the outlined challenges, a hybrid attributive GCN structure tailored for recommendation in the realm of social e-commerce has been proposed. Within this structure, explicit collaborative signals in the user-item interaction graph are discerned, leveraging two supplementary VAEs to derive non-linear latent representations and establish high-order connectivity via attentive embedding propagation. A unified neural variational model has been employed to encapsulate the generative processes of both users and items, thereby facilitating the efficient extraction of non-linear latent representations for CF. By embedding the attribute information of users and items into their latent factors through a deep graph neural networks (GNN) for CF, data sparsity issues can be effectively addressed and the latent representations of users and items can be enhanced. Drawing from findings in GNN research [17, 18], an embedding propagation layer has been integrated to augment user/item embeddings. This augmentation is achieved by aggregating embeddings from interacting items or users. Sequentially, these embeddings are optimized to discern cooperative indications in higher-order connectivity, achieved through layer stacking for embedding propagation. Through data-driven training, the adopted neural network has demonstrated the capacity to incorporate both user preferences and item attributes into the latent factors of users and items. This manuscript is structured as follows: Section 2 offers a review of extant literature on CF models. In Section 3, the proposed models are elucidated, accompanied by a discussion on parameter learning methodologies. Section 4 presents the empirical outcomes and associated discussions, with Section 5 concluding the paper and suggesting avenues for future research.

The forthcoming section elucidates the architecture of the proposed Hybrid Attributive GCN (HAGCN), as visually represented in Figure 1. This model comprises three pivotal components: the initial feature extraction and embedding, an attentive GCN fortified with a dual attention mechanism, and the subsequent rating prediction. Initially, two ancillary VAEs are employed to discern users' and items' characteristics through a coherent deep generative framework. Nodes’ embeddings are initialized employing both ID and feature embeddings. Thereafter, the node embeddings undergo enhancement by a refined GCN, incorporating dual attentive propagation mechanisms. Such a mechanism facilitates the assimilation of collaborative signals through attentive embedding propagation atop the user-item relation graph. The culminating stage involves generating predictions via the inner product of users' and items' final representations.

11.png

Figure 1. The architecture of the proposed HAGCN model

3.1 Notations

For clarity, a compendium of employed symbols and notations is presented, with comprehensive details delineated in Table 1.

Table 1. Symbols and notations used in this paper

The sets $\mathrm{U}=\left\{u_i \mid i=1, \ldots, M\right\} \in \mathbb{R}^{K \times M}$ and $\mathrm{V}=\left\{v_j \mid j=1, \ldots, N\right\} \in \mathbb{R}^{K \times N}$ are posited to symbolize the latent factors of users and items, respectively, with K representing the latent factors' dimensions. In scenarios involving implicit feedback, both the rating matrix and its predictive counterpart are denominated as $\mathrm{R} \in \mathbb{R}^{M \times N}$ and $\mathrm{R}^* \in \mathbb{R}^{M \times N}$, respectively. Within this framework, R_ij = 1 is indicative of engagement between the i-th user and the j-th item, whereas R_ij = 0 denotes a lack thereof. Concurrently, rating vectors for users and items are represented as $\mathrm{R}_u \in \mathbb{R}^{K^{\times M}}$ and $\mathrm{R}_v \in \mathbb{R}^{M \times N}$, respectively.

Vectors $\mathrm{X}_u=\left\{\mathrm{X}_{u i} \mid i=1, \ldots, M\right\} \in \mathbb{R}^{P \times M}$ and $\mathrm{X}_v=\left\{\mathrm{X}_{v j} j=1, \ldots\right.$ $N\} \in \mathbb{R}^{Q \times M}$ encapsulate attribute data pertinent to users and items, respectively. Dimensions of these attributes are demarcated by P for users and Q for items. Within this context, terms latent profile representation and latent content representation correspond to X_u and X_v, respectively. Embedding representations for users and items after l propagation steps are delineated as $e_u \in \mathbb{R}^D$ and $e_y \in \mathbb{R}^D$, respectively. The overarching endeavour is to deduce the latent factors u_i and v_j for users and items, predicated upon the extant variables R, U, V, X_u, and X_v, with the terminal objective being the prediction of hitherto uncharted ratings, R^*.

3.2 Feature extraction and node embedding

Traditionally, most model-based CF methodologies are dependent exclusively on user-item interactions for rating predictions. However, when certain techniques are employed to amalgamate user or item attribute data into rating prediction through linear regression, an observed limitation in accuracy emerges. In addressing this constraint, attribute information pertaining to both users and items is integrated into the feature learning process in the proposed model. This integration is surmised to enhance the precision in discerning latent factors germane to users and items. Subsequently, the ID and feature embeddings pertaining to users and items are amalgamated into two distinct embedding lookup tables. Utilizing the features of previously unobserved nodes, embeddings for such nodes are generated, a feat unattainable with conventional ID embedding. Furthermore, this method contributes to a reduction in the count of learnable parameters.

3.2.1 Generative model

In the pursuit of deriving robust user and item features, a dual modeling structure fortified with two supplementary VAEs is devised. This structure is primed to incorporate user profile and item content information, along with associated tag data. By this approach, the effective derivation of concealed user and item representations becomes feasible. The generative process espoused by the proposed model is analogical to the deep latent Gaussian model paradigm. Tag information for user profiles and item contents are symbolized by $S \in \mathbb{R}^{N \times S}$ and $T \in \mathbb{R}^{N \times T}$ respectively, being represented as binary matrices. The condition S_is = T_jt = 1 connotes the association of the s-th tag with user u_i and the t-th tag with item v_j. Conversely, S_is = T_jt = 0 denotes a void in association. For each user u_i, a K-dimensional latent representation $z_u \sim N\left(0, \mathbb{I}^K\right)$ is sampled from a standard Gaussian prior. Post this sampling, a conditional sample variable $X \sim p_\theta\left(X \mid z_u\right)$ is generated via a decoder, where θ represents the generative parameter. Depending on data type, $p_\theta\left(X \mid z_u\right)$ might originate from a multivariate Bernoulli distribution (binary data) or a Gaussian distribution (real-valued data).

The latent representation $z_u \sim N\left(0, \mathbb{I}^K\right)$ is drawn from a Gaussian prior distribution with an identity covariance matrix. The latent representation of user u_i amalgamates the latent user offset with the latent user profile vector: u_i = ε_i + z_ui. The methodology employed for generating item content mirrors that of the user profile. Thus, the latent representation of item v_j is the fusion of the latent item offset and the latent item content vector: v_j = ε_j + z_vj.

3.2.2 Inference model

The inference framework encompasses an encoder network that bears resemblance to the generative model. Within this construct, the generative network prescribes the posterior distribution p_θ(z_u|X), which, due to its intractable nature, necessitates approximation during the inference phase for each user. By deploying the SGVB estimator, the posterior of the latent user profile variable z_u can be effectively approximated through a tractable variational distribution q_ф(z_u|X). The variational parameters are subsequently derived by optimizing the evidence lower bound (ELBO) objective:

$q_\phi\left(z_u \mid X_i\right)=N\left(\mu_\phi(X)_i, \operatorname{diag}\left(\sigma_\phi^2\left(X_i\right)\right)\right)$                 (1)

where, the mean and standard deviation of the approximate posterior are denoted as $\mu_\phi \in \mathbb{R}^K$ and $\sigma_\phi^2 \in \mathbb{R}^K$, respectively.

Outputs generated by the inference mode were identified as nonlinear functions of $X_i$ and the variational parameter $\phi$. The representation $z_u \sim N\left(\mu_i, \operatorname{diag}\left(\sigma_i^2\right)\right)$ was elucidated in a manner aligned with the findings presented by Liang et al. [13]. Drawing on the work of Deng et al. [14], it is posited that the SGVB estimator holds potential for approximating the ELBO of X_i.

$\begin{aligned} & \mathcal{L}\left(\theta, \phi ; X_i, S_s\right) \\ = & \mathrm{E}_{q_\phi\left(z_u \mid X_i, S_s\right)}\left[\log p\left(u_i \mid z_u\right)+\log p_\theta\left(X_i, S_s \mid z_u\right)\right]  -\beta \mathrm{KL}\left(q_\phi\left(z_u \mid X_i, S_s\right) \| p\left(z_u\right)\right) \\ \approx & \log p\left(u_i \mid z_{u, l}\right)+\frac{1}{L} \sum_{l=1}^L \log p_\theta\left(X_i, S_s \mid z_{u, l}\right)  -\beta \cdot \operatorname{KL}\left(q_\phi\left(z_u \mid X_i, S_s\right) \| p\left(z_u\right)\right)\end{aligned}$              (2)

$\begin{aligned} \mathrm{KL} & =\frac{1}{2} \sum_{i=1}^M\left(\mu_i^2+\sigma_i^2-\log \sigma_i^2-1\right), \\ z_{u_i, l} & =\mu_i+\sigma_i \odot \varepsilon_{i, l}\end{aligned}$                       (3)

The term KL is understood to denote the Kullback-Leibler divergence. In this context, the parameter $\beta \in[0,1]$ has been identified as a pivotal component, employed to modulate the robustness of regularization, a strategy adopted to address the challenge of posterior collapse, as documented by Lee et al. [11]. It was also observed that ε_i,l adheres to the normal distribution $N\left(0, \mathbb{I}^K\right)$, and $\odot$ is interpreted to symbolize the element-wise multiplication. Inferences related to items were drawn in a manner mirroring the inferences pertaining to users. Furthermore, the derivation of the ELBO for the item-oriented network was found to emulate established protocols.

$\begin{aligned} & \mathcal{L}\left(\theta, \phi ; Y_j, T_t\right) \\ = & \mathrm{E}_{q_\phi\left(z_v \mid Y_j, T_t\right)}\left[\log p\left(v_j \mid z_v\right)+\log p_\theta\left(Y_j, T_t \mid z_v\right)\right] \\ & -\beta \operatorname{KL}\left(q_\phi\left(z_v \mid Y_j, T_t\right) \| p\left(z_v\right)\right) \\ \approx & \log p\left(v_j \mid z_{v_j, l^{\prime}}\right)+\frac{1}{L} \sum_{l=1}^L \log p_\theta\left(Y_j, T_t \mid z_{v_j, l^{\prime}}\right) \\ & -\beta \cdot \operatorname{KL}\left(q_\phi\left(z_v \mid Y_j, T_t\right) \| p\left(z_v\right)\right)\end{aligned}$                  (4)

$\begin{aligned} & \mathrm{KL}=\frac{1}{2} \sum_{j=1}^N\left(\mu_j^2+\sigma_j^2-\log \sigma_j^2-1\right), z_{v_j, l^{\prime}}=\mu_j+\sigma_j \odot \varepsilon_{j, l^{\prime}}\end{aligned}$               (5)

3.2.3 Node embedding

In the absence of supplementary features such as user profiles or item attributes, ID embeddings were traditionally employed. To augment the precision of rating predictions, the integration of attribute data pertaining to users and items into node embeddings was proposed, as such an approach was observed to enhance the inference of latent factors for users and items. Two embedding look-up tables, denoted as E_u={e_u1,…, e_uM} and E_v={e_v1,…, e_vN}, were utilized, providing an initial state for both user and item embeddings.

3.3 Simplified GCN with dual attentive propagation

Guided by the foundational principles presented in Light GCN, an innovative mechanism for the transmission of information was incorporated, with the primary objective being the capture of collaborative signals intricately woven within the graph structure. Such an endeavour was undertaken to fortify the quality and precision of user and item embeddings.

3.3.1 First-order propagation

As postulated by Wang et al. [23], an observable interaction between a user and an item often provides valuable insights into the user's predilection for said item. Furthermore, an engagement of a user with a particular item can be interpreted, not merely as an isolated event, but as an intrinsic attribute of that item. This attribute, in turn, can be instrumental in evaluating the shared collaborative inclinations between that item and others. During the embedding propagation phase, attention-driven mechanisms were deployed to determine the relevance of data exchanged across a consistent user and varying items. In the proposed model, a meticulous propagation of embedding was executed, predicated predominantly on the documented interactions between users and items. This structured propagation strategy encompassed three core procedures: the initial dissemination of information, the duality of attentive transmission, and the final synthesis of the amassed information.

Information Propagation. The information transmitted from v to u is defined as follows for a connected user-item pair (u, v).

$\operatorname{IP}(u, v)=f\left(e_u, e_v, p_{u, v}\right)=\frac{e_v}{\sqrt{\left|\mathbb{N}_u\right|\left|\mathbb{N}_v\right|}}$             (6)

where, function IP(∙) is delineated as the conduit for the propagation of information embeddings, and the role of function f is to facilitate the encoding of this information. The primary inputs to this function are represented by the embeddings, namely e_u and e_v. To modulate the decay factor of each propagation occurring on the edge (u, v), the parameter p_u,v is introduced. This parameter essentially quantifies the historical influence items exert on user preferences. The set of items and users interacting with user u and item v are symbolized by $\mathbb{N}_u$ and $\mathbb{N}_v$, respectively. These denote the firsthop neighbors for both $u$ and $v$. Intriguingly, $p_{u x x}$ is conceptualized as the square root of the graph Laplacian $\operatorname{norm}\left(\left|\mathbb{N}_u\right|\left|\mathbb{N}_v\right|\right)^{-1 / 2}$, offering further insights into the extent of the influence of historical items on user inclinations. In the pursuit of amplifying both performance and representational capability, the distinction of NGCF lies in its opting for an identity matrix over a feature transformation matrix during the information aggregation phase. Such an approach has been observed to enhance recommendation performance.

Dual Attentive Propagation. Deficiencies in user-item interaction signals are noted to impact the precision of the CF task, as underscored by Shi et al. [29]. Given such challenges, a dual attentive network, geared towards assimilating both user and item attentions, has been integrated to enhance the acquisition of their embeddings. The embeddings within this network are iteratively updated based on calculated attentions corresponding to diverse items. Subsequent to this, predictions are formulated by amalgamating the embeddings corresponding to self-connection information with those linked to interactive data.

Upon procuring the node embeddings, attention mechanisms are deployed to discern a user's inclination towards an assortment of items, with subsequent updates applied to the embeddings. The ensuing step involves a fusion of the self-connection message embedding with the interactive message embedding to proffer the concluding prediction. In this study, the dual attention mechanism is articulated through the subsequent equations:

$\begin{aligned} & H_j=\text { LeakyRelu }\left(W_a e_{v_j}+b_a\right), \\ & \operatorname{AT}_j=\exp \left(e_u^{\top} H_j\right) / \sum_{k \in|\mathrm{V}|} \exp \left(e_u^{\top} H_k\right)\end{aligned}$               (7)

where, Hj denotes the concealed representation procured from the dense, low-dimensional embedding associated with item $v_j \in|\mathrm{V}|$. The LeakyReLU function has been enlisted to bolster the nonlinear potential of the attention model, complemented by the designated hyper-parameters $W_a \in \mathbb{R}^{K \times K}$ and $b_a \in \mathbb{R}^{K \times 1}$. Given a sequential labelling of items from 1 to N, the dense embedding vector corresponding to item v_j is represented as e_vj.

Diverging from traditional attention models which uniformly employ context vectors for each input, the adopted context vector in this methodology is identified as the embedding e_u. The attention score AT_j, indicative of the significance of v_j for u, is discerned by harnessing the Softmax function. This assists in computing the normalized similarity between H_1j and e_u. The representation of the user, denoted as e_u, is then derived by integrating the item embeddings, each weighed by their corresponding attention scores. Such a computational approach is designed to encapsulate both inherent traits of u_i and the interactions noted between the user and the item.

$e_u=e_{u_i}+\sum_{j \in|\mathcal{V}|} \operatorname{AT}_j e_{v_j}$              (8)

Information Aggregation. During the aggregation phase, information from the neighborhoods of u is assimilated to refine its representation. The accuracy of predictions is purported to be augmented when information propagation and aggregation are undertaken using a straightforward weighted sum aggregator. Notably, there is no reliance on feature transformation or nonlinear activation functions in this context [17]. The corresponding aggregation functions are depicted as:

$\begin{aligned} & e_u^{(1)}=\sum_{v \in \mathbb{N}_u} \operatorname{IP}(u, v)=\sum_{v \in \mathbb{N}_u} \frac{1}{\sqrt{\left|\mathbb{N}_u\right|\left|\mathbb{N}_v\right|}} e_v^{(0)}, \\ & e_v^{(1)}=\sum_{u \in \mathbb{N}_u} \operatorname{IP}(v, u)=\sum_{u \in \mathbb{N}_v} \frac{1}{\sqrt{\left|\mathbb{N}_v\right|\left|\mathbb{N}_u\right|}} e_u^{(0)}\end{aligned}$                (9)

In the presented Eq. (9), the representations of user u and item v, symbolized by e_u⁽¹⁾ and e_v⁽¹⁾, are procured by channelling information from interconnected users and items during the foundational embedding propagation stages. The representations at the 0-th layer for the user and item are depicted as e_u⁽⁰⁾ and e_v⁽⁰⁾ respectively. As highlighted by He et al. [18], superior outcomes can be achieved through the application of the square root of the symmetric graph Laplacian normalization in contrast to other norms. This method also serves to circumvent any escalation in the embedding scale attributable to graph convolution operations.

Within the model described, the aggregation functions solely take into account direct connections of users and items, deliberately overlooking their self-connection. It is posited that layer combination operations can fulfil the roles typically ascribed to self-connections. Such an approach presents a deviation from a majority of extant graph convolution operations [17, 23] that encompass extended neighbors and acknowledge self-connections. Upon the culmination of the K-th embedding propagation layers, the final representations for users and items are synthesized by amalgamating the embeddings discerned at each successive layer, as illustrated:

 $\begin{aligned} & e_u=\xi_0 e_u^{(0)}+\xi_1 e_u^{(1)}+\cdots+\xi_K e_u^{(K)}=\sum_{k=0}^K \xi_k e_u^{(k)}, \\ & e_v=\xi_0 e_v^{(0)}+\xi_1 e_v^{(1)}+\cdots+\xi_K e_v^{(K)}=\sum_{k=0}^K \xi_k e_v^{(k)}\end{aligned}$                (10)

where, ξ_k stands as a modifiable hyper-parameter reflecting the importance of the embedding in the k-th layer for the synthesis of the concluding embedding. Disparate embedding layers are understood to encapsulate varied representations. For instance, the foundational layer is tailored for fostering unimpeded interactions between users and items. Subsequent layers accentuate the congruity between users and items exhibiting shared interactions. Elevated layers, in contrast, are adept at capturing higher-order proximities. Such a conglomerate approach ensures a more encompassing final representation. The infusion of the attention mechanism into the embedding propagation is believed to amplify the dynamics of user-item interactions.

3.3.2 High-order propagation

Upon completion of the primary information propagation, representations of both users and items are ascertained. To further delve into higher-order interactions between users and items, supplementary layers of embedding propagation are introduced. It is posited that these interactions are instrumental in encapsulating collaborative signals and discerning the affiliations between users and items. By invoking multiple layers of embedding propagation, both users and items stand to assimilate information conveyed by their neighbors within a defined range of hops. Within the model, the recursion formulas are employed to determine the representations of the user and item at the l-th iteration.

$\begin{aligned} e_u^{(l)} & =\sum_{v \in \mathbb{N}_u} \frac{1}{\sqrt{\left|\mathbb{N}_u\right|\left|\mathbb{N}_v\right|}} e_v^{(l-1)}, &  e_v^{(l)} =\sum_{u \in \mathbb{N}_v} \frac{1}{\sqrt{\left|\mathbb{N}_v\right|\left|\mathbb{N}_u\right|}} e_u^{(l-1)}, l \geq 1\end{aligned}$           (11)

By iteratively implementing multiple layers of embedding propagation, the user and item representations, delineated as e_u^(l−1) and e_v^(l−1) from the (l-1)-th propagation phase, are believed to inculcate collaborative signals from their respective (l-1)-hop neighbors. Such an integration is purported to augment representation learning and thereby bolster performance.

3.3.3 Propagation in matrix form

To facilitate the execution of HAGCN, the matrix formulation is introduced. Let $\mathrm{E}^{(0)} \in \mathbb{R}^{(M+N) \times d_l}$ stand as the embedding matrix at the initial layer, and let d_l < min(M,N) represent the size of the embedding. The corresponding matrix form is delineated as:

$\begin{aligned} & \mathrm{E}^{(l)}=\left(\mathrm{D}^{-\frac{1}{2}} \mathbb{A} \mathrm{D}^{-\frac{1}{2}}\right) \mathrm{E}^{(l-1)}, \mathrm{D}=\left(\begin{array}{cc}0 & \mathrm{R} \\ \mathrm{R}^{\top} & 0\end{array}\right), \\ & \mathrm{E}=\sum_{k=0}^l \xi_k \mathrm{E}^{(k)}=\xi_0 \mathrm{E}^{(0)}+\xi_1 \mathrm{E}^{(1)}+\ldots+\xi_l \mathrm{E}^{(l)}\end{aligned}$             (12)

where, $\mathbb{A}$ serves to represent the adjacency matrix pertaining to the user-item graph. Furthermore, the diagonal degree matrix D encapsulates the count of non-zero elements present within each row of the matrix $\mathbb{A}$. The parameter ξ_k is posited to demarcate the significance of the embedding at the k-th layer within the holistic embedding.

3.4 Prediction and optimization

Upon the culmination of the attentive embedding propagation, a series of user representations is acquired from multiple layers. These representations underscore the nuances of information propagation through distinct connections. These sets of user representations, represented as {e_u⁽¹⁾, e_u⁽²⁾, … , e_u^(l)}, signify varied importances concerning user preferences. The eventual user embeddings are formulated by collating e_u⁽¹⁾, e_u⁽²⁾, … , e_u^(l). Analogously, the item representations are derived by amalgamating the item representations e_v⁽¹⁾, e_v⁽²⁾, … , e_v^(l) amassed across diverse layers. The ultimate representation for both users and items is established as:

$\begin{aligned} & e_u^*=e_u^{(0)} \otimes e_u^{(1)} \otimes \cdots \otimes e_u^{(l)}, \\ & e_v^*=e_v^{(0)} \otimes e_v^{(1)} \otimes \cdots \otimes e_v^{(l)}\end{aligned}$             (13)

where, $\otimes$ symbolizes a concatenation operation, absent of learning any ancillary parameters. Such an operation has been demonstrated as efficacious in GCN [22]. Through the deployment of concatenation operations, initial embeddings are enhanced via attentive embedding propagation, with the propagation breadth modulated by the parameter l. The focal point remains on embedding learning, with a fundamental inner product interaction function utilized for deriving the ranking score for recommendations. This implies that the preference of a user for a specific item is evaluated through $\mathrm{R}^*=e_u^{* \mathrm{~T}} e_u^*$. Future investigations are poised to delve into more sophisticated mechanisms, encompassing VAE-based interaction functions.

Following the prediction of ratings, optimization of the loss functions is conducted to achieve optimal performance. Typically, these loss functions encompass both the feature reconstructing error and the rating prediction error. Training of parameters in HAGCN is confined to the embeddings of the initial layer, making its complexity comparable to that of the standard MF. Therefore, the pairwise BPR loss is adopted as the principal loss function for HAGCN, in line with findings by He et al. [18]. This loss function accounts for the relative order of both observed and unobserved user-item interactions. Notably, the pairwise BPR loss tends to favour predictions of observed interactions with elevated values in comparison to their unobserved counterparts. The formulation of HAGCN's loss function is delineated as:

$\begin{aligned} & \mathcal{L}_{\mathrm{BPR}}=  \sum_{u=1}^M \sum_{i \in \Psi^{+}} \sum_{j \in \Psi^{-}}-\ln \sigma\left(\mathrm{R}_{u i}^*-\mathrm{R}_{u j}^*\right)+\lambda_{\mathrm{L} 2}\left\|\mathrm{E}^{(0)}\right\|^2\end{aligned}$              (14)

In Eq. (15), $\Psi^{+}$ signifies the set of observed interactions, while Ψ^- denotes the set of unobserved instances. The sigmoid function is represented as σ(·). All trainable parameters within the 0-th layer are encapsulated by E⁽⁰⁾, with λ_L2 being employed to modulate the intensity of L2 regularization, thereby mitigating overfitting.

Drawing parallels with SGCN and Light GCN, a minimal increase in parameters is observed in the model to facilitate the modelling of high-order connectivity. Given that HAGCN's propagation layers encompass a limited number of parameters, and that the size of the parameters matrix is defined by d_l×d_l−1 (where d_l represents the embedding size and d_l <min(M, N)), the model's size is approximated as 2C×d_l×d_l−1. Herein, C typically remains a minor integer, seldom surpassing four, as these embedding matrices are derived from the underlying user-item graph structure and corresponding weight matrices.

In the study presented, a hybrid attributive neural graph structure was introduced for enhanced social recommendation. Dual AVAEs structure was employed for feature extraction, adeptly capturing attribute details from both users and items. Through this method, linear and non-linear latent representations of users and items for feature embedding were derived. Initial embeddings were inferred from shared distributions anchored on Bayesian inference, allowing cold-start embeddings to derive benefit from statistical strength among users and items.

A simplified GCN, embedded with attentive feature propagation across layers, was integrated to ascertain a profound high-order connectivity, unveiling explicit collaborative signals within the interaction graph. By amalgamating user and item attribute data into latent factors via a deep neural graph network, data sparsity challenges were addressed, and enhanced user and item representation was achieved. Attention was given to refining the embeddings of users and items, by including a layer of attentive embedding propagation that amalgamates the embeddings of interactive users and items. Multiple embedding propagation layers, layered with weighted sum, were further augmented to harness collaborative indications in advanced connectivity. This provision was invaluable in ameliorating over-smoothing.

As a result, HAGCN was postulated to effectively tackle the cold-start problem and augment the learning of user and item latent factors. This was achieved by synergising the intrinsic Bayesian probabilistic perspective evident in VAE and GCN. Experimental results validated the superiority of HAGCN over contemporaneous methods such as NGCF, KGAT, LRGCCF, and Light GCN. An improvement of at least 2.9% on MAE/RMSE for rating prediction performance was observed, while top-ranking performance witnessed an upliftment of at least 2.2% on Recall/NDCG. The efficacy of HAGCN in addressing challenges in cold-start scenarios, particularly those bereft of preliminary data, was further corroborated through extensive experimentation.

Given the mounting inclination towards graph-based models in social recommendations, harnessing supplementary data from social commerce or media platforms has emerged as a novel trajectory. Yet, extant GCN models grapple with complexities in their GCN design. Prospective efforts will pivot on probing into the simplified graph convolutional layers within these GCNs to amplify recommendation outputs. The focus might shift towards harnessing limited rather than infinite message passing and reducing regularization for expedited training. Emphasis will also be placed on refining the variational graph architecture to deduce users/items embeddings from attribute distributions via Bayesian inference, coupled with the application of VAE-based interaction functions for reconstructing elusive preference embeddings.