A Reinforcement Learning-Driven Co-Simulation Framework for Cyber-Resilient MPPT Control under Stealthy Cyber Attacks

Since the introduction of photovoltaic (PV) systems, they have become one of the most promising renewable energy technologies for energy generation [1]. PV installations are often equipped with DC-DC converters and Maximum Power Point Tracking (MPPT) algorithms in order to maximize the harvested energy under different environmental conditions [2]. Irradiance, temperature and load fluctuations are the most important factors affecting the tracking efficiency and hence, proper MPPT control is necessary for a successful tracking operation [3].

The conventional MPPT techniques have been widely explored in the past decade. Perturb and Observe (P&O) is a very popular algorithm to implement, while Incremental Conductance (InC) is known to give better tracking performance under rapidly changing operating conditions [4]. The MPPT efficiency has been further improved by several optimization approaches such as genetic algorithm–proportional–integral–derivative controller [5], Particle Swarm Optimization (PSO), Artificial Neural Networks (ANN) [6] and duty-cycle estimation using machine learning [7].

While these methods can enhance energy harvesting, their primary focus is on electrical aspects with relatively little attention given to cybersecurity [8]. In recent years, deep learning has been explored for anomaly detection [9], irradiance-aware neural networks [8] and converter-state monitoring [10] with most of these studies concentrating on attack detection but not providing closed-loop control during cyberattacks.

As a result, the co-simulation of cyber and physical systems is now a useful tool for analyzing and verifying the cyber-physical interdependence of communication networks and power systems [11]. Previous research showed how communication latency and packet loss affect the operation of the power system [12] and how a denial-of-service attack had a significant impact on power system coordination [13]. The same was reported for inverter-dominated microgrids when they were under communication degradation [14, 15]. The reinforcement learning (RL) approach of [16] is, however, focused on a traditional intrusion detection system for IT and does not consider photovoltaic cyber-physical control loops. Recent surveys have pointed out how deep reinforcement learning (DRL) and adversarial RL are emerging to become significant tools in adaptive cyber-defense [17, 18] and recent RL frameworks have shown that they can be used to dynamically mitigate new Distributed Denial-of-Service (DDoS) attacks, instead of just statically detecting intrusions. RL-based IDS methods have also shown the effectiveness of DQN agents for adapting to malicious network traffic classification through experience replay and reward-driven learning in recent years [19]. RL has also been studied recently to enhance the cyber resiliency of photovoltaic energy systems. For instance, in the study [20] a soft RL algorithm was fused with observer-based control to maintain stability of a PV/fuel cell system against cyberattacks. Recently, DRL has been suggested for autonomous cyber defense in smart solar systems. A DQN-LSTM framework was proposed to identify FDI and DoS attacks and adjust the mitigation policies according to the PV-IoT environment [21].

Other recent research works have examined intelligent solutions for improving cybersecurity in PV energy systems. Supervised ANN-based models, which are typically trained with normal data, have been used to detect False Data Injection Attacks (FDIA) in PV-powered energy systems [22] and graph-based one-class learning has been studied for physics-aware anomaly detection for mainly normal operating data [23]. More recently, an integrated cyber-resilience framework has integrated digital twins, RL, and blockchain authentication, offering adaptive protection for Smart Grid (SG) and PV arrays, against evolving cyber threats [24]. Therefore, this study proposes a synchronized cyber-physical co-simulation framework, based on the identified limitations, which integrates OMNeT++ [25] with a physical control environment implemented in Python. The proposed agent, Double Deep Q-Network (DDQN) agent, recognizes coordinated DDoS and FDIA attacks and recovers the MPPT operating point by applying PV operating baselines, which were determined experimentally.

However, previous works have not yet fully considered the simultaneous effect of communication degradation, physical PV dynamics, and DDQN-based mitigation in a single cyber-physical MPPT system. This deficit encourages the proposed hybrid approach of network simulation, physical modeling, and adaptive RL for resilient MPPT operation in the presence of coordinated cyberattacks. The main scientific contributions of this work are summarized in the following section.

The methodology is organized around a physical development layer, which is validated using empirical field data and photos of an operational solar PV asset, and a cyber-physical co-simulation platform to test dynamic network resilience against adversarial data injection.

3.1 Proposed circuit diagram of solar Photovoltaic system

The solar cell is a major energy source, since it is a renewable energy source. It is a semiconductor device that can transform sunlight energy into electric power [26, 27]. A PV array is a collection of PV cells that are connected in series and parallel. The voltage of the module is boosted by placing two or more modules in series and the current of the array is increased by connecting two or more arrays in parallel. In general, the modelling of a solar cell will be sought in the form of an inverted diode in parallel with a current source and resistance in parallel and series, as shown in Figure 1. There is an n-p junction with obstacles to the movement of electrons, which causes a series resistance. The leakage current is the cause of the creation of parallel resistance [28, 29].

tu_pian_1.png

Figure 1. DC-DC boost converter circuit diagram with Maximum Power Point Tracking (MPPT) control

One way to increase the efficiency of output from a PV panel is to use converter control schemes, especially MPPT. This includes measuring the voltage and current out of the PV module and changing the duty cycle of the switching device (typically an IGBT or MOSFET) to get maximum power from the PV module. Some important factors influencing maximum PV power are irradiation, the angle of incidence of Sun's rays and temperature. As for the linear methods for sliding mode control, there are various MPPT algorithms, in which P&O algorithm is superior to others [30, 31]. The most common algorithm is P&O, which has simple circuitry and is easy to implement, but it does not work well when the environment changes rapidly, such as partial shadowing, and needs to be combined with both voltage and current sensors [32].

3.2 Experimental setup and hardware validation

The aim of this section is to describe the experimental hardware setup and the empirical operational baselines in detail so as to maintain the physical fidelity of the simulation framework. The physical infrastructure consists of a PV array containing 16 high-efficiency modules connected in two parallel strings, each of which has 8 modules connected in series as shown in Figure 2.

tu_pian_2.png

Figure 2. Experimental hardware components including the 16 Photovoltaic (PV) modules, storage batteries, and hybrid inverter

The installation has a local Battery Energy Storage System (BESS) and a hybrid power inverter. A numerical modelling approach was not employed, but rather a systematic extraction of the nominal electrical parameters and the dynamic asset boundaries were carried out during peak irradiance periods from both the manufacturer datasheets and real-world logging files. These measurements measured in the field are integrated with system logic, enabling a high-fidelity cyber-physical benchmark that is needed to effectively evaluate tracking control loops and network resilience following data injection attacks. Table 1 shows the manufacturer's nominal values for a single module, compared to the telemetry values measured during operation of the array, collected dynamically through SolarMan, the smart monitoring platform. The tight match of this alignment shows that the physical characteristics of the system design are well matched to the operating conditions that are measured in the field.

Table 1. Manufacturer's nominal parameter values and actual operating data for a single Photovoltaic (PV) module

The total array power is 5760 W (16 × 360 W). Theoretically, the open-circuit voltage is 348.8 V (8 × 43.6 V) in the 8 × 2 series-parallel string configuration and the total input current is (9.8 A) at the inverter channels. The increased operational voltage recorded (328.7 V) from the nominal maximum power voltage baseline (292 V) is solely due to commercial manufacturing tolerance for positive cells (+5%) and to reduced internal voltage losses under peak irradiance conditions due to increased cell conductivity, which provides superior dynamic MPPT tracking. For the power conversion stage, a hybrid inverter with a power rating of 5.5/6.0 kW and an operating range of MPPT of 120-450 V is used. It is coupled with a 48 V Lithium-ion/Lead-Acid BESS dedicated to controlling energy storage and load-pumping distribution. These hardware boundaries test the practicality of the power conversion layer and PV array configuration. Finally, Table 2 shows the empirical time-line data that was retrieved through the SolarMan app. The chronological data points are measured directly at the installation site and used as exact baselines for the physical layer of the cyber-physical co-simulation platform in changing environment.

The distribution of this in Table 2 agrees exactly with the thermodynamic and physical behaviour of the solar array. The open-circuit voltage (V_oc) of 348.7 V was recorded in the early morning at 07:02:41 AM when the ambient light is enough to create the maximum potential difference across the series cells, before the inverter enters into active load matching. The other operational metrics – ($I_{s c}, V_{m p}, I_{m p}$, and $P_{\max }$) – were taken at the same time the absolute peak generation was taken (10:02:44 AM). This synchronous record is the moment at which the 16-module array reached its maximum steady state output due to environmental irradiance, and serves as a time-synchronized, and structurally sound, physical profile for comparison of the cyber-physical control loops. The field measurements presented in Tables 1 and 2 were not used as performance evaluation results. Instead, they were employed exclusively to calibrate the nominal operating conditions of the cyber-physical co-simulation framework. The experimentally measured electrical parameters, including the nominal voltage, current, power, and operating limits, were directly incorporated into the OMNeT++ configuration files (CyberResilientMPPT.ned and omnetpp.ini) as baseline simulation parameters. Consequently, the co-simulation was initialized using physically validated operating conditions rather than theoretical assumptions. This data-driven calibration enables the proposed RL controller to be evaluated under realistic PV operating conditions while all cyberattack scenarios, resilience mechanisms, and performance assessments are conducted within a controlled, synchronized, and reproducible simulation environment.

Table 2. Real data table extracted from the SolarMan smart application

3.3 Cyber-physical co-simulation framework

The proposed approach is based on a high-fidelity co-simulation approach to assess the resilience of the microgrid control loops to cyber-physical disruptions.

A. Structure of the shared simulation environment

There are two distinct, mutually dependent operational layers to the framework:

•The Cyber Layer (MPPT Cyber Layer): Implemented in OMNeT++ 6.0 with INET Framework 4.5 library. It represents network nodes, sends out sensor telemetry, and calculates important network performance parameters, such as the end-to-end delay (packet delivery latency) and PLR. Importantly, embedded in this is the Cyber-Resilient Smart Client mitigation logic as a secure data gateway.

•The Physical and Control Layer: It was implemented in Python 3; this layer models the physical hardware dynamics such as PV arrays, power converters and the InC MPPT algorithm.

The communication network and the physical PV systems are tightly coupled with the RL controller, such that during the simulation, cyber events can directly affect physical systems dynamics. Using ZeroMQ over TCP/IP, the two simulation environments communicate via a request/reply protocol with a fixed pattern. Each time the OMNeT++ cyber layer receives a new telemetry message, its current state (PV voltage, PV current, PV power, duty cycle, PLR, communication latency, attack state) is sent to the Python control layer every 0.1 s. The measurements received by the RL agent are used to guide the agent to return the updated duty-cycle command to OMNeT++, and the next simulation step is performed. This deterministic synchronization provides temporal consistency between the cyber and physical layers.

B. Mathematical modeling and baselines

The physical system operating baselines are initialised with a constant PV array voltage V_pv = 185.2 V, and a nominal load current I_pv = 4.95 A, under normal environmental conditions. The base power generation (Pnormal) is controlled by [33]:

$P_{\text {normal }}=V_{p v} \times I_{p v}$           (1)

The photovoltaic current is estimated using a simplified nonlinear PV model derived from the current-voltage characteristic of the PV module in Eq. (2):

$I_{p v}=I_{S C}\left(1-\left(\frac{V_{p v}}{V_{o c}}\right)^4\right)$           (2)

where, I_sc and V_oc denote the short-circuit current and open-circuit voltage, respectively. The MPPT process is achieved by continuously adjusting the DC-DC converter duty cycle (D), while the RL agent receives synchronized telemetry from OMNeT++ and updates the control action accordingly. The RL agent is initialized using the empirical statistical baselines extracted from the OMNeT++ communication logs, ensuring consistency between the physical and cyber layers.

The tracking performance is dynamically controlled by changing the duty cycle (D) of the DC-DC converter. The internal scaling factors of the Python intelligent agent are set directly from the empirical statistical mean output from the communication log vectors generated by OMNeT++.

A. Cyber-physical attack modeling

•FDIA scenario: Threat to data integrity by altering the sensor telemetry by a scaling impact factor of α = 0.40 and artificially reducing the perceived consumption data by 40% down to 2.97 A, during peak operational periods (10:00 AM to 02:00 PM) according to Eq. (3) [34]:

$I_{\text {compromised}}=I_{\text {actual}} \times(1-\alpha)$           (3)

This deception causes the central Energy Management System (EMS) to make sub-optimal duty cycle decisions and premature disconnection of battery storage.

•DDoS attack scenario: Denies access to data by flooding with malicious traffic on the communication channel. This causes serious channel congestion, high PLR and critical end-to-end delay. The result is the control interface telemetry goes to zero, resulting in "Operational Blindness" and forcing the EMS to set up expensive auxiliary diesel generators even though the physical solar power is available.

B. Cyber resilience mitigation strategy

The cyber-resilient smart client performs an anomaly detection and data validation algorithm automatically to neutralize any active FDIAs and stabilize the microgrid:

•Anomaly Detection & Correction: If it detects an adversarial change in data profiling, the RL-based resilience algorithm segments the bad input and builds a safe operating current state based on Eq. (4):

$I_{\text {corrected}}=I_{\text {actual}} \times 0.98=4.851 \mathrm{~A}$          (4)

•Duty Cycle Protection: At the same time, the control loop resets the actuation path and ensures the converter's duty cycle is locked into a stable optimal index (D = 0.75) while preventing system degradation and microgrid instability.

E. RL implementation

The proposed cyber-resilient controller uses DDQN algorithm to learn and adaptively control the policy in both normal and cyberattack scenarios. The agent's current state of the system is passed to it through the cyber layer of OMNeT++, and the agent is able to choose which duty-cycle changes to make according to a reward function, which encourages it to extract power from PV and discourages it from degrading communication or from making inappropriate control decisions. Training is done by the repeated interaction of the communication layer (OMNeT++) and the physical layer (Python). RL was chosen over fixed-threshold or rule-based protection because PV operating conditions are constantly changing based on both sunspot intensity and load demand and the quality of the communication link, and the predefined rules would need manual tuning and would not perform well in all conditions, especially when the attack conditions change. Double DQN, on the other hand, learns the cyber-physical recovery policy from its interaction with the cyber-physical environment, so that no prior knowledge about the attacks is required.

3.4 Simulation setup and execution parameters

The cyber-physical co-simulation was performed in the following: INET Framework 4.5: communication network simulation environment, OMNeT++ 6.0: physical system modeling and RL control environment, Python 3: programming language. The two environments were synchronized by using a ZeroMQ message-based interface for exchanging data in both directions at fixed simulation time steps. In each synchronization cycle, OMNeT++ sent the network telemetry data and PV sensor data measured from the network to the Python environment, which in turn sent back the updated desired MPPT control action (duty cycle) back to OMNeT++ for implementation. In particular, OMNeT++ was used to control the dynamics in the cyber-layer and to control the network traffic. It simulated packet transmission and performed attack scenarios such as dropping packets during the DDoS attack and injecting manipulated payloads in the FDIA framework. During the co-simulation process, OMNeT++ continuously generated and exchanged network-related information, including packet transmission states, communication disturbances, and attack-induced data anomalies, which were used to represent the cyber-layer conditions affecting the physical system. At the same time, the Python 3 environment was used to implement the physical control loops and the cyber-defense mechanism. In particular, the RL algorithm that controlled the 'smart client' was run on Python, enabling the system to analyze received information and generate corresponding resilience decisions. When the RL-based resilience logic received corrupted network data from OMNeT++ because of FDIA, it was able to isolate the malicious inputs and reconstruct the actual current data by evaluating the abnormal measurements and maintaining the consistency of the control process, in real-time stabilizing it at 4.851 A. In addition, the Python environment issued direct physical control overrides to locked the duty cycle of the step-up transformer to its optimum value of 0.75 (to prevent it dropping to 0.42 during the DDoS attack). The coordinated response was effective enough to keep the MPPT tracking efficiency above 98% to overcome the adversarial goals. The co-simulation was executed via the opp_run command line environment (Cmdenv) using omnetpp.ini. The simulation network configuration parameters are summarized in Table 3.

Table 3. The OMNeT++ simulation network configuration parameters

In this paper, the vulnerabilities of the conventional MPPT control loop under advanced cyber threats were proven and the high efficacy of the proposed intelligent and cyber-resilient client framework was demonstrated. The co-simulation environment, connecting OMNeT++ and Python, showed that unmitigated DDoS and FDIA attacks cause severe performance degradation in both the communication and physical layers, resulting in a complete collapse of generated solar power to 0 kW due to severe network delays and packet drops, together with an engineered 40% degradation in physical current extraction. The framework employed a high-level RL method trained using optimal reward mechanisms, which enabled the smart agent to be able to detect telemetric anomalies and filter out communication noises dynamically in real-time. The quantitative results prove the effectiveness of the proposed algorithm in absorbing the cyber shocks and keeping the duty cycle of the boost converter within the acceptable range (0.75) and the operating current within the allowable range (4.85 A). This mitigation strategy proved the recovery of the physical system with a performance higher than 98%, creating a solid base model to secure the next-generation of photovoltaic infrastructures against hostile exploitation of the network.

The quantitative evaluation revealed an attack detection Precision of 98.3%, Recall of 95.7%, F1-Score of 97.0%, and ROC-AUC of 99.51%. A comparison of the Double DQN and PPO training-reward measurements was performed using the paired t-test, which resulted in the values t = −34.602 and p = 4.2665 × 10⁻³⁶, representing a significant difference between training-reward measurements (paired t test). The robustness analysis also revealed the consistency of the detection performance with respect to the various FDIA intensities tested. A validated cyber-physical co-simulation scenario was also developed and calibrated with the proposed framework; future work could focus on exploring hardware-in-the-loop implementation and more diverse adaptive cyber-attack scenarios.

Unlike conventional rule-based methods based on preset thresholds, the controller designed by the proposed Double DQN learns the control policy as a function of the observed cyber-physical state. This allows for adaptive recovery under the evaluated communication conditions without the need for predefined decision rules to be changed.