Latency Optimization for UAV Networks via Latency-Aware Long-Term Deep Deterministic Policy Gradient in Slicing-Enabled MEC Systems

2.1 Unmanned aerial vehicles communication challenges

UAVs have become more important in recent years across a wide range of wireless uses. This is because they are flexible, don't cost much to run, and are easy to set up. These traits make UAVs appealing, but adding them to communication networks causes a number of technical issues that have a direct effect on the stability and performance of the system [1]. Because a UAV is so mobile, especially because it can move easily in three dimensions, the network topology around it is almost never fixed. When the drone changes height or direction, the wireless channel changes too. This makes the signal power and link reliability go up and down. Such variations may lead to intermittent delays or temporary link failures, which pose serious limitations for latency-sensitive applications such as real-time monitoring, emergency response, and delivery missions [9]. As a result, UAV-based services require communication links that can sustain low-latency performance under rapidly changing conditions. In practical deployments, UAVs often operate in environments with sparse or heterogeneous infrastructure, where network conditions vary frequently and unpredictably. Traditional ground-based communication systems are generally optimized for static or slowly varying traffic patterns and are therefore ill-suited to support highly dynamic, on-demand UAV workloads [1]. Addressing these challenges requires scalable communication mechanisms and adaptive resource-management strategies capable of responding to continuous changes in UAV network topology and traffic demand [10, 11].

2.2 Mobile edge computing

Modern wireless networks are putting pressure on traditional cloud systems, and many of today’s applications simply expect more than what distant cloud servers can deliver. Tasks such as smart surveillance, AR/VR, and various autonomous services produce large data streams and depend on very tight latency requirements. Because of that, the classical cloud model often becomes too slow to keep up. MEC, introduced by ETSI, was meant to close this gap by shifting computation and storage closer to the radio access network, which naturally results in faster and more responsive service [6].

Compared to mobile cloud computing (MCC), MEC keeps the processing near the point where the data is generated. This not only reduces end-to-end delay but also removes part of the load from the backhaul network. UAVs operate with limited onboard computational and energy resources. For this reason, offloading support is required [6]. MEC servers can be deployed at multiple locations. MEC servers can be deployed at different locations within the network. These locations include cellular base stations, roadside units, and mobile platforms such as UAVs and ground vehicles [2].

A typical MEC architecture consists of several functional components, including virtualization layers, orchestration mechanisms, traffic steering, and local breakout functions. These components enable computational tasks to be executed at the network edge, closer to data sources and end users. Within the 5G service-based architecture (SBA), MEC operates in conjunction with NS. Multiple services share the same physical infrastructure while maintaining performance isolation between slices [8]. In addition, hybrid MEC (H-MEC) configurations combine fixed edge nodes with mobile computing resources. Such configurations are applied in environments where coverage conditions change frequently, including disaster-response scenarios, dense urban areas, and UAV-based delivery networks [2]. Recent MEC deployments also integrate complementary technologies such as non-orthogonal multiple access (NOMA), machine-learning-based resource optimization, and wireless power transfer. These technologies aim to improve resource utilization and energy efficiency at the edge [2]. Despite these capabilities, MEC introduces several challenges. The distributed nature of edge resources increases coordination complexity. Mobility may interrupt both computation and communication processes. Furthermore, decentralized architectures raise additional security and reliability concerns that must be carefully managed [6].

Still, MEC is not without challenges. Wide distribution of resources complicates coordination, mobility can interrupt both computation and communication, and the decentralized nature of the architecture raises additional security concerns that must be addressed.

2.3 Network slicing

NS is defined as a function of 5G networks that enables a physical infrastructure to be partitioned into multiple logical slices. Each slice is configured to meet specific service requirements [12]. This approach is applied in several domains, including IoT systems, automotive communication, and UAV networks.

The 3GPP standard specifies three main slice categories: massive Machine-Type Communication (mMTC), Ultra-Reliable Low-Latency Communication (URLLC), and enhanced Mobile Broadband (eMBB) [4]. In UAV-related applications, URLLC is commonly associated with control and command signaling, mMTC with sensor data transmission, and eMBB with high-data-rate services such as image or video delivery [13].

NS enables differentiated allocation of network resources according to quality-of-service (QoS) requirements. Resource allocation is performed based on task characteristics and service constraints. Architectures such as AirSlice apply QoS-based allocation mechanisms for UAV traffic, assigning low-latency resources to control-related tasks and higher bandwidth to data-intensive transmissions [12]. Slice configuration and management are supported through software-defined networking (SDN) and network function virtualization (NFV). These technologies allow slice parameters to be adjusted during system operation without relying on fixed configurations.

The slicing workflow consists of several functional stages. These stages include slice admission control, resource allocation, traffic scheduling, and orchestration [14]. Each stage operates as part of the overall resource management process, as shown in Figure 2.

Figure 2. Phases of resource management in 5G/6G network slicing (NS) admission control, resource allocation, scheduling, and orchestration [14]

UAV missions exhibit heterogeneous requirements. Some missions require strict latency constraints. Others require sustained bandwidth or basic connectivity. When UAVs operate as temporary network nodes, slicing supports coverage extension and link stability in areas with limited signal quality [4].

Despite these advantages, NS introduces several unresolved challenges. In this work, NS is discussed to provide background context on its role in modern MEC-enabled networks. The proposed framework does not aim to resolve these open challenges; instead, slicing is assumed as an infrastructure-level capability that enables differentiated task handling in the baseline configuration.

Ensuring isolation and security among slices remains difficult, especially under high mobility conditions where UAVs frequently change positions, channels, and network associations [4]. As a result, efficient mobility management and slice continuity remain active research problems in UAV-enabled networks [13].

4.1 Task modeling and dataset preparations

An initial configuration was evaluated using size-based task assignment. In this configuration, tasks were assigned according to file size only. Slice S3 recorded the highest processing delay under this setting. A load-balancing mechanism was then applied. Task distribution across MEC nodes changed after this step. Overall processing delay decreased compared to the initial configuration.

However, the role of NS is applied in the baseline configuration by grouping tasks into slices (S1, S2, S3) with predefined processing characteristics. These slice-based rules define the baseline behavior used for evaluation. The proposed LLDDPG does not perform slice selection or inter-slice resource optimization; instead, it learns task offloading decisions under the same slicing-enabled MEC environment.

The LLDDPG-based offloading policy was evaluated next. Under this configuration, total processing time decreased from 10,479 s to 3,795 s. Slice-level processing times were recorded as follows: 425 s for S1, 320 s for S2, and 3,050 s for S3. This configuration was selected for subsequent experiments. A second experimental phase was conducted using a dataset of more than 28,000 UAV images. The dataset was obtained from previous studies [27, 28]. Each image was treated as an independent task. Additional task parameters were assigned. These parameters included deadline constraints, priority labels, and estimated transmission energy values.

Task metadata was extracted during preprocessing. The extracted information included file path, image resolution, and file size. This metadata was used for dataset organization and simulation input preparation. The retrieved information was organized and stored for later stages of processing.

The information that was extracted was put together using pandas into an organized CSV file. There are now more factors for tasks, like randomly chosen deadlines (100-1000 ms), priority levels based on directory labels, and energy estimates based on data size and communication cost. The Latency-aware LLDDPG model was trained and tested in a simulated MEC setting using this task-level dataset.

By switching from static size-based categorization to dynamic task modeling, the better framework made offloading choices that were more flexible and aware of the situation. This made LLDDPG even better at working in real UAV–MEC situations. Table 2 presents sample entries from the synthesized task dataset used in the offloading experiments.

Table 2. Synthesized an unmanned aerial vehicle (UAV) task dataset used for offloading simulation

4.2 The proposed system architecture

The proposed architecture enables intelligent task offloading in UAV-based delivery systems by integrating MEC, NS, and a Latency-aware LLDDPG agent. It consists of four layers:

(1) UAVs that generate and offload tasks,

(2) A communication layer using 5G with variable latency,

(3) A distributed MEC Layer with heterogeneous edge nodes,

(4) A decision layer powered by RL for latency-aware.

Figure 3 shows the interaction among components for dynamic, real-time offloading.

The proposed model integrates several modular components to enable intelligent task offloading in UAV-based smart delivery networks. These components include:

• UAV Agent: Captures tasks from delivery missions and initiates the offloading process based on task characteristics and UAV status.
• Task Generator: Transforms real-world UAV image data into structured tasks containing file size, deadline, and energy estimates.
• MEC Node Simulator: Emulates heterogeneous edge servers with dynamic load levels and response times.
• LLDDPG Controller: This is the reinforcement-learning agent that watches what’s going on in the system task details, the UAV’s status, and which MEC nodes are available. Based on this information, it chooses where to send each task to reduce delay and save energy.
• All parts of the system keep exchanging information in a continuous loop, which helps the UAV make fast, context-aware decisions as conditions change.

Figure 3. The proposed system architecture

4.2.1 The proposed model components

The LLDDPG is an extension of the standard deep deterministic policy gradient (DDPG) algorithm, designed for task offloading in latency-sensitive UAV–MEC environments. Unlike conventional DDPG, which optimizes a generic long-term reward, LLDDPG is tailored to account for task-level latency characteristics and sequential decision-making under dynamic network and workload conditions. The model applies a reinforcement-learning approach to task offloading in UAV delivery scenarios. The operating environment changes during execution. Offloading decisions are generated using a LLDDPG algorithm. Static decision rules are not used. Task assignment is updated at each decision step. The update depends on the current network state and available computation resources. Communication conditions and MEC server availability vary over time. Offloading decisions are recalculated accordingly. The choice changes depending on whatever is happening at that moment: how close a task is to missing its deadline, how busy the MEC servers look, how much battery the UAV still has, and even how good or weak the wireless link is at that instant.

The operating environment changes during execution. Offloading decisions are generated using a LLDDPG algorithm. Static decision rules are not used. Task assignment is updated at each decision step. The update depends on the current network state and available computation resources. Communication conditions and MEC server availability vary over time. Offloading decisions are recalculated accordingly. The choice changes depending on whatever is happening at that moment: how close a task is to missing its deadline, how busy the MEC servers look, how much battery the UAV still has, and even how good or weak the wireless link is at that instant.

• Model overview

Each UAV creates its own set of tasks while it moves through its delivery route. These tasks don’t all look the same some are larger, some have tighter deadlines, and some need to be handled more urgently than others.

When a new task appears, the UAV has to make a quick decision:

• Either try to process it on its own with its limited onboard resources;
• Or send it to one of the MEC servers available, where each server has a different amount of computing power.

This decision is not made heuristically, but instead through an RL-based controller trained to optimize task distribution across the network.

• LLDDPG-based offloading

The decision-making process is modeled as a continuous-state, continuous-action reinforcement learning problem. The LLDDPG agent observes a state vector that includes:

• UAV battery level
• Task file size
• Deadline in milliseconds
• Link quality between UAV and MEC nodes
• Current processing load on each MEC server

Based on this input, the agent selects an action from a continuous action space representing offloading probabilities to each MEC node (or staying local). The policy is optimized using an actor–critic architecture.

• The actor proposes an action (offloading decision);
• The critic evaluates its expected cumulative reward.

The reward function is designed to minimize end-to-end latency, avoid missed deadlines, and reduce energy consumption, encouraging the agent to learn efficient, real-time decisions under uncertainty. Figure 4 shows the LLDDPG-based offloading workflow. The agent observes UAV and task-related parameters and learns to make optimal offloading decisions using a reinforcement learning loop.

Figure 4. LLDDPG-based offloading workflow

Note: LLDDPG: latency-aware long-term deep deterministic policy gradient.

To explicitly incorporate latency awareness into the learning process, the reward function is designed to penalize end-to-end task delay and deadline violations. The immediate reward at decision step $t~$is defined as:

${{r}_{t}}=-(\alpha \cdot {{L}_{t}}+\beta \cdot {{V}_{t}})$                    (1)

where, ${{L}_{t}}$ denotes the end-to-end latency experienced by task $t,$ ${{V}_{t}}~$represents the deadline violation indicator (equal to 1 if the task misses its deadline and 0 otherwise), and α and β are weighting coefficients controlling the relative importance of latency minimization and deadline satisfaction.

The long-term optimization aspect of LLDDPG is achieved by maximizing the expected cumulative discounted reward over sequential task arrivals. Instead of optimizing each offloading decision independently, the agent learns policies that account for the long-term impact of current actions on future system states, such as MEC load evolution, resource availability, and task congestion. This enables more stable and efficient offloading behavior under continuously changing UAV–MEC conditions.

Compared to the standard DDPG algorithm, which optimizes a generic long-term reward without domain-specific awareness, LLDDPG integrates latency-sensitive objectives and task-level constraints into the learning process. This allows the agent to prioritize time-critical tasks and adapt its offloading decisions based on dynamic MEC load and wireless link conditions. As a result, LLDDPG is better suited for latency-sensitive UAV–MEC scenarios than conventional DDPG. The overall workflow of the proposed LLDDPG-based task offloading process is summarized in Algorithm 1.

• Training procedure

The LLDDPG model was trained over multiple episodes using the task dataset described in Section 4.1.

We split the synthesized UAV task dataset into 20% for training and 80% for testing (seed = 42). To preserve distributional balance, we apply an approximate stratified split by file size, deadline, and priority. During training, the LLDDPG agent interacts with the environment over multiple episodes using mini-batches drawn exclusively from the 20% training subset. At each step, the agent selects offloading actions (local vs. MEC nodes) conditioned on the state vector (UAV energy, task size, deadline, MEC load, and link quality), receives rewards shaped to minimize end-to-end latency and deadline violations, and updates actor–critic parameters with target networks and prioritized experience replay. All model selection and hyperparameter tuning are performed on the training subset only; the 80% test subset remains strictly unseen until final evaluation and figure generation.

• Integration with MEC and NS

The MEC infrastructure in the system includes heterogeneous servers with support for NS. Each server hosts multiple virtual slices configured with different latency and resource guarantees. The LLDDPG agent implicitly learns to assign tasks to the most appropriate slice by associating observed performance feedback with the action taken, without explicit slice labeling.

This model is illustrated in Figure 5, which shows the interaction between UAVs, MEC nodes, and the LLDDPG controller operating in a closed-loop feedback system.

Figure 5. The interaction between UAVs, MEC nodes, and the LLDDPG

Note: UAVs: unmanned aerial vehicles; MEC: mobile edge computing; LLDDPG: latency-aware long-term deep deterministic policy gradient.

4.3 Simulation strategy

4.3.1 Simulation environment and parameter settings

The simulation environment is implemented in Python using task-level UAV image data. Each image is treated as an independent task with attributes including file size (MB), deadline, priority, and an estimated energy cost. Tasks are categorized into three slices based on size: S1 (≤ 1 MB), S2 (1-3 MB), and S3 (> 3 MB). MEC nodes are modeled as heterogeneous servers with predefined processing rates assigned per slice. Specifically, MEC processing rates are set to 5 MB/s (S1), 3 MB/s (S2), and 1 MB/s (S3). Local processing is modeled as slower execution with a rate of 0.5 MB/s. For each task, the processing delay is computed as size divided by the selected processing rate (local or MEC).

Deadline constraints are generated within the range 100-1000 ms using a fixed random seed (42). Priority labels are generated as {High, Medium, Low} with probabilities {0.2, 0.5, 0.3}. The task energy estimate is modeled as proportional to task size using an energy factor of 0.5 (energy_est = 0.5 × size_MB). The learning agent observes a state vector that includes task size, normalized deadline, normalized energy estimate, UAV battery level, a normalized link-quality coefficient, and MEC load indicators. For reproducibility, the battery level is set to 0.7 and link quality is represented by a fixed normalized coefficient (link_q = 0.8) during simulation runs. MEC load indicators are initialized as (0.3, 0.5, 0.2).

The dataset split follows a fixed 20/80 train–test ratio with seed = 42. During training, a subset of up to 8000 tasks is sampled with random_state = 7, and the model is trained for 100 epochs. Performance is evaluated using end-to-end latency and deadline satisfaction rate under the same parameter settings. Table 3 presents the main simulation parameters used in the evaluation, including task slicing settings, MEC processing capabilities, wireless link assumptions, and training configuration. These parameters define the experimental environment and ensure the reproducibility of the simulation results.

Table 3. Simulation parameters and experimental settings used in the UAV–MEC offloading evaluation

Figure 6. Simulation strategy overview