Energy-Efficient Node Placement and Routing in Dynamic WSNs Using Adaptive Nest Competition and Deep Reinforcement Learning

1.png

Figure 1. System flow

2.png

Figure 2. DQL for WSN

Figure 2 illustrates the interaction between the DQL model and the WSN environment. The DRL model (DQL) agent observes the network state, selects an optimal routing action, and receives feedback based on network performance. This feedback is used to update the Q-values, enabling the model to learn better routing policies over time. The loop ensures continuous adaptation to dynamic WSN conditions, enhancing energy efficiency and reliability.

2.2 ANCA for node placement

The ANCA draws inspiration from the natural reproductive strategy of cuckoo birds, particularly their unique approach of laying eggs in the nests of other bird species. In this biological process, if a host bird detects that an egg does not belong to it, it either discards the egg or abandons the nest entirely to construct a new one. Analogously, in the context of optimization, each nest represents a candidate solution, and each cuckoo egg symbolizes a promising or improved solution. The optimization process evolves by refining these solutions iteratively to identify the optimal outcome. In this model, a population of nests, each containing a potential solution, is maintained. The selection of the nest for laying the egg mimics the stochastic behavior of cuckoos and is governed by Levy flight, a random walk strategy that ensures exploration across a wide solution space. To further refine the search capability and enhance exploitation, an adaptive competition-based learning mechanism is introduced. This strategy incorporates an elite selection and rivalry mechanism, in which members with superior performance are engaged in a competitive learning framework to generate more efficient solutions. This process promotes solution refinement without requiring entirely new individuals, thus improving convergence while preserving diversity.

Key Concepts of ANCA Competitive Strategy:

(1). Elite selection: Two high-performing candidates are selected from the top-performing subset of the population (top 5% based on fitness).

(2). Competition and replacement: The two candidates undergo a competitive evaluation, and based on the outcome, the weaker candidate is modified using the traits of the stronger one.

(3). Fitness evaluation: After modification, both solutions are evaluated. The one with better fitness may replace the current global best if it outperforms it.

2.2.1 Algorithm steps: ANCA

Step 1:

Select two individuals (C1 and C2) at random from the top 5% elite set of the population.

C1 ← Random selection from elite pool

C2 ← Another random selection from the elite pool

Step 2:

Perform a competitive learning phase between C1 and C2 to generate modified versions.

[C1', C2'] ← Competitive_Update (C1, C2)

Step 3:

Evaluate the fitness of updated candidates.

fitness_C1' ← Evaluate (C1')

fitness_C2' ← Evaluate (C2')

Step 4:

Update the global best solution if either C1' or C2' has better fitness.

If fitness_C1' > Global_Best_Fitness:

        Global_Best ← C1'

If fitness_C2' > Global_Best_Fitness:

        Global_Best ← C2'

Step 5:

Repeat this competitive update across the population until convergence or a termination condition is met.

This approach significantly enhances the exploitation ability of the search process by continuously reusing and refining individuals near the global optimum. It also ensures rapid convergence through focused competition among elite candidates, making it particularly suitable for high-dimensional optimization tasks such as sensor node deployment in dynamic WSNs.

2.3 FCM clustering

In the proposed hybrid architecture for dynamic WSNs, FCM clustering plays a central role in managing energy-efficient data transmission by organizing sensor nodes into flexible, overlapping clusters. Unlike hard clustering methods, FCM minimizes intra-cluster distances while maintaining flexibility in cluster formation, which is critical in dynamic WSN environments. In the proposed system, FCM is employed after optimal node placement to form energy-aware clusters. This soft clustering strategy enhances load balancing and improves local data aggregation, thereby reducing overall energy consumption and communication overhead. This characteristic is advantageous in dynamic or mobile environments where node energy levels and topology change frequently.

2.4 DQL-based routing

To accompany the clustering framework, DQL is employed to dynamically manage routing both within and between clusters. DQL empowers sensor nodes to act as intelligent agents that learn optimal data forwarding paths by interacting with the environment and receiving feedback in the form of rewards or penalties. DQL is a value-based reinforcement learning algorithm that leverages deep neural networks to approximate the optimal action-value function. In the proposed framework, DQL is utilized to optimize the routing process within the clustered Wireless Sensor Network. Each sensor node acts as an agent that learns to select the most energy-efficient routing path based on a reward mechanism that considers factors such as residual energy, hop count, and link reliability.

The DQL agent interacts with the dynamic WSN environment, continuously updating its Q-values to adapt to node failures, mobility, and energy depletion. This learning-based routing approach ensures robust and adaptive communication from cluster members to the base station, effectively minimizing End-to-End Delay, Routing Overhead, and energy consumption. By integrating DQL, the system achieves intelligent decision-making capabilities that enhance the overall network lifetime and performance.

2.5 Mathematical models

(1). FCM clustering

Objective: Partition a set of sensor nodes into c clusters with soft membership, allowing each node to belong to multiple clusters.

Let:

N = {n₁, n_c, ..., n_k}: Set of sensor nodes

C = {c₁, c_2, ..., c_m}: Set of initial cluster centers from FCM

CH: Set of refined Cluster Heads selected by DQL

BS: Base Station location

E(n_i): Residual energy of node n_i

d_ij: Distance between nodes n_i and n_j

D: End-to-End Delay

E_avg: Average Energy Consumed

L: Network Lifetime

R: Set of possible routes

(2). Initial clustering using FCM (soft assignment)

The membership matrix U=[u_ij] is computed as:

$u_{i j}=\frac{1}{\sum_{k=1}^m\left(\frac{| | x_i-c_j| |}{| | x_i-c_k| |}\right)^{\frac{2}{m-1}}}$             (1)

where:

x_i: Feature vector of node n_i

c_j: Cluster center

m: Fuzziness factor (typically 2)

Update cluster centers:

$c_j=\frac{\sum_{i=1}^n u_{i j}^m x_i}{\sum_{i=1}^n u_{i j}^m}$              (2)

(3). DQL state-space definition for clustering and routing

Each state s is defined as:

$s=\left[\begin{array}{c}E\left(n_i\right), d\left(n_i, B S\right), P D R\left(n_i\right), \text { hop count, } \\ \text { buffer size, cluster assignment }\end{array}\right]$             (3)

Each action a $\in$ A can be:

Clustering: Elect node n_i as cluster head

Routing: Forward packet to neighbor n_j

The Q-value update is:

$\begin{gathered}Q\left(s_t, a_t\right) \leftarrow Q\left(s_t, a_t\right) \\ +\alpha\left[r_t+\gamma \max _{a^{\prime}} Q\left(s_{t+1}, a^{\prime}\right)-Q\left(s_t, a_t\right)\right]\end{gathered}$              (4)

where:

$s_t$: Current state

$a_t$: Action taken

$r_t$: Reward received

$\alpha$: Learning rate

$\gamma$: Discount factor for future rewards

$\max _{a^{\prime}} Q\left(s_{t+1}, a^{\prime}\right)$: Maximum expected future reward from the next state

(4). Reward function $r_t$

In the proposed DQL-based routing mechanism, each sensor node (agent) learns to choose the most suitable next-hop node based on the current state of the network. The state-space is defined using key network parameters that reflect the current condition of a node and its neighbors. These parameters include:

• Residual Energy (E) of the node
• PDR of the link
• Distance (D) to the destination or cluster head
• Hop Count (H) from the current node to the sink

The agent evaluates these states to decide the best action, i.e., selecting the next-hop node for forwarding the data packet. The reward function guides the learning process by providing feedback after each action. It is designed to encourage energy-efficient and reliable routing. The reward at time t is calculated as:

$r_t=w_1 \cdot \Delta E+w_2 \cdot P D R+w_3 \cdot \frac{1}{D}+w_4 \cdot \frac{1}{H}$             (5)

where:

ΔE: Change in residual energy (preferably low)

PDR: Packet Delivery Ratio (higher is better)

D: Distance to destination (shorter is preferred)

H: Hop count to sink (fewer hops are ideal)

$w_1, w_2, w_3, w_4$: Weight factors controlling the influence of each metric:

$w_1+w_2+w_3+w_4=1$

PDR = "Total packets received at destination" / "Total packets sent by sources"

Let:

$P_{\text {recv}}$: Total successfully received packets.

$P_{\text {sent}}$: Total packets sent.

So:

$P D R=\frac{P_{\text {recv }}}{P_{\text {sent }}}$              (6)

End-to-End Delay $\mathrm{D}=\frac{\sum_{i=1}^N\left(t_{\text {recv }, i}-t_{\text {sent }, i}\right)}{P_{\text {recv }}}$              (7)

where:

$t_{\text {recv}, i}$: Time packet i was received.

$t_{\text {sent}, i}$: Time packet i was sent.

$P_{\text {recv}}$: Number of packets successfully received.

Hop Count $\boldsymbol{H}=\frac{\sum_{i=1}^{P_{\text {recv }}} h_i}{P_{\text {recv }}}$            (8)

where:

$h_i$: Number of hops taken by packet i.

$P_{\text {recv}}$: Total packets received.

In this research, we proposed a novel, energy-efficient, and intelligent framework for WSNs by integrating the ANCA for optimal node placement, FCM for clustering, and DQL for dynamic routing. The hybrid ANCA–FCM–DQL approach effectively addresses challenges such as uneven energy depletion, suboptimal cluster formation, and inefficient routing under dynamic network conditions.

Comprehensive simulation experiments demonstrated that the proposed method significantly outperforms benchmark algorithms like ABC, PSO, and GWO in multiple performance metrics:

• Up to 20–35% improvement in network lifetime.
• 15–25% reduction in Average Energy Consumption.
• Enhanced PDR by 10–18%.
• Lower End-to-End Delay and reduced Routing Overhead.

These improvements are attributed to the synergistic combination of ANCA’s global optimization for node placement, FCM’s adaptive clustering capabilities, and DQL’s intelligent, feedback-driven routing decisions.

Implications

• The framework demonstrates strong adaptability in dynamic WSN environments, with performance scaling effectively as node density increases.
• The integration of machine learning and bio-inspired algorithms opens new avenues for real-time decision-making in distributed sensor deployments.
• The proposed system architecture is generalizable and can be adapted for IoT-based monitoring, smart agriculture, and disaster-response systems, where energy efficiency and reliability are critical.

Future scope

• Amalgamation with real-time hardware testbeds (e.g., Arduino, Raspberry Pi with XBee modules) will be pursued to validate practical performance under real deployment conditions.
• Exploration of transfer learning or meta-reinforcement learning can reduce DQL training time in dynamic environments.