Machine Learning-Driven Dynamic Resource Allocation in Heterogeneous Wireless Networks.
List of key symbols
Symbol Definition
K Number of small base stations (SBSs)
N Number of user equipment (UE) devices
b Base station index
n User equipment index
x_{n,b} Binary user association variable (1 if UE n served by BS
b)
P_b Transmit power of base station b (dBm)
P_{max,b} Maximum transmit power of base station b
W System bandwidth (Hz)
R_{n,b} Achievable rate for UE n served by BS b (bps)
SINR_{n,b} Signal-to-interference-plus-noise ratio of UE n at BS b
SINR_{th} Minimum SINR threshold for QoS
w_n Priority weight of UE n
C_b Maximum user capacity of base station b
d Distance between UE and base station (km)
PL_{macro} Path loss model for macro BS: 128.1 + 37.6
[log.sub.10](d) dB
PL_{small} Path loss model for small BS: 140.7 + 36.7
[log.sub.10](d) dB
RSRP_b Reference signal received power from BS b (dBm)
[DELTA]_b Cell selection bias for BS b (dB)
[beta] Load balancing penalty coefficient
LoadFactor_b Normalized traffic load of base station b
[lambda] QoS violation penalty coefficient in DRL reward
[gamma] Discount factor in reinforcement learning
[alpha] Learning rate of optimizer
[epsilon] Exploration rate in [epsilon]-greedy policy
R Instantaneous reward in DRL power control module
Q(s,a) Action-value function in Deep Q-Network
1. Introduction
The exponential growth in mobile data traffic, driven by the proliferation of smartphones, Internet of Things (IoT) devices, and bandwidth-intensive applications such as video streaming and augmented reality, has placed unprecedented demands on wireless network infrastructure [1-3]. According to industry forecasts, global mobile data traffic is expected to increase sevenfold between 2o22 and 2o28, necessitating fundamental improvements in network capacity and spectral efficiency [4,5]. To address these challenges, heterogeneous networks (HetNets) have emerged as a key architectural paradigm in 5G and beyond-5G systems, wherein high-power macro base stations (MBSs) are overlaid with densely deployed low-power small base stations (SBSs) such as picocells and femtocells [6].
The HetNet architecture offers significant advantages in terms of capacity enhancement through spatial reuse and improved coverage, particularly in indoor environments and traffic hotspots [7]. However, this architectural complexity introduces substantial challenges for resource allocation, including dynamic user association, inter-cell interference management, and power control optimization [8]. Traditional optimization-based approaches, while theoretically sound, often suffer from high computational complexity and limited adaptability to rapidly changing network conditions [9]. Moreover, the inherent uncertainty in wireless channel characteristics and user traffic patterns renders static allocation strategies suboptimal [10].
Machine learning (ML) techniques have recently garnered significant attention as promising tools for addressing the complexity and dynamism of wireless resource allocation problems [11]. In particular, deep learning models excel at capturing complex spatiotemporal patterns in network traffic, while reinforcement learning (RL) enables adaptive decisionmaking in uncertain environments without requiring explicit models of the underlying system dynamics [12,13]. The integration of these techniques offers the potential for intelligent, self-optimizing networks that can dynamically adapt resource allocation strategies to maximize system performance while ensuring fair resource distribution [14].
The rapid evolution of wireless networks, driven by the increasing demand for high-capacity and low-latency services, has necessitated innovative solutions to address the inherent complexity of resource allocation in heterogeneous environments. This study presents a comprehensive machine learning-driven framework that integrates advanced traffic prediction, load-aware user association, and deep reinforcement learning-based power control mechanisms to optimize resource allocation in heterogeneous wireless networks. Through extensive simulations using real-world traffic patterns, we demonstrate that the proposed framework achieves a significant improvement in system performance, including a 23.2% enhancement in system sum rate compared to conventional approaches, while simultaneously improving fairness and energy efficiency. The ability of the framework to adapt to dynamic network conditions and balance multiple objectives underscores its potential to address the challenges of next-generation wireless networks.
2. Related work
Early approaches relied on statistical models such as ARIMA for cellular traffic forecasting, but these struggle to capture nonlinear spatiotemporal dynamics [15]. Deep learning methods, especially LSTM-based architectures, have demonstrated superior accuracy in capturing long-range temporal dependencies in mobile traffic [16], providing the foundation for proactive resource provisioning adopted in this work. Conventional max-RSRP association causes severe load imbalance in HetNets [10]. Biased cell selection partially alleviates this, but optimal bias configuration remains open. Recent RL-based approaches show that treating user association as a sequential decision problem yields significant load balancing gains over static schemes [17], motivating the RL-driven formulation used here. Optimization-based power control faces scalability and model dependency issues in dynamic environments. DRL-based approaches bypass explicit channel modeling and have shown strong results in multi-cell interference management [18]. However, most existing works optimize power control in isolation, without coupling it to load or traffic state. Several recent studies have proposed joint optimization of user association and power control [19], but typically formulate the problem as a single-stage optimization and do not exploit temporal traffic patterns. Hierarchical multi-timescale frameworks integrating prediction with adaptive control remain underexplored [20], representing the gap addressed by this work.
3. System model and problem formulation
3.1 Network architecture
The study uses two public datasets with 20,000 total samples with a HetNet resource allocation dataset (10,000 time-slot snapshots) and a Telecom Italia Milan-inspired traffic dataset (10,000 grid-cell-time records). Both are split using a 70% training, 15% validation, and 15% testing strategy, with stratified sampling to preserve key feature distributions. We consider a two-tier heterogeneous network comprising a central macro base station (MBS) and K strategically positioned small base stations (SBSs) distributed throughout the coverage area. The MBS provides wide-area coverage, while the SBSs are deployed to serve local traffic hotspots and offload traffic from the MBS [21-23]. The network supports N user equipment (UE) devices, which are distributed according to a non-uniform spatial pattern reflecting realistic traffic dynamics. As illustrated in Fig. 1(a), the network topology features a central MBS (denoted by a red triangle) and four SBSs (represented by blue squares) positioned to serve high-density user clusters. Users are categorized based on their serving base station type (macro or small) and their location within hotspot regions. The spatial distribution exhibits clustering behavior around SBS locations, indicative of traffic offloading. Fig. 1(b) presents the interpolated reference signal received power (RSRP) distribution map, revealing the expected distance-dependent signal strength decay from the MBS, with local maxima observed near SBS positions. This heterogeneous coverage pattern underscores the need for intelligent user association strategies that account for both signal quality and network load conditions [24].
3.2 Channel model
The wireless channel between base station b and user n is characterized by large-scale path loss, shadowing, and small-scale fading. The path loss model follows the 3GPP specifications for urban macro and micro environments [25]. For macro base stations, the path loss in dB is given by:
PL_macro = 128.1 + 37.6 [log.sub.10](d) [dB]
where d is the distance in kilometers. For small base stations, the path loss model is:
PL_small = 140.7 + 36.7 [log.sub.10](d) [dB]
Shadowing is modeled as a log-normal random variable with standard deviations of 8 dB and 10 dB for macro and small cell links, respectively. The detailed simulation parameters are summarized in Table 1. The proposed framework operates under the following assumptions. First, large-scale channel knowledge-specifically, RSRP measurements from all base station-is assumed to be available at the network controller via standard UE measurement reports (per 3GPP TS 36.331). Second, interference levels are estimated from the aggregate received power of co-channel base stations, which are periodically reported alongside RSRP. Third, instantaneous small-scale (fast) fading is not explicitly tracked; instead, the shadowing component is modeled as a quasi-static log-normal random variable with standard deviations of 8 dB (macro) and 10 dB (small cell), updated on the scale of the resource allocation interval. Fourth, the system assumes a centralized controller with access to per-UE RSRP, SINR estimates, and traffic demand reports, but does not require full channel state information (CSI) matrices. Finally, each UE is associated with a single serving base station at any given time slot, precluding coordinated multi-point (CoMP) transmission.
The channel characteristics observed in the network was presented, illustrating the relationship between RSRP and distance from the macro base station, as well as the cumulative distribution function (CDF) of SINR for both macro and small cell users. As shown in Fig. 2(a), macro users (represented by red points) exhibit a decreasing signal strength with increasing distance from the MBS, consistent with the path loss model. In contrast, small cell users (blue points) demonstrate relatively high RSRP values even at large distances from the MBS, highlighting the coverage enhancement provided by nearby SBSs [26]. This phenomenon underscores the benefits of network densification in improving user experience. Fig. 2(b) further reveals the SINR distribution, where small cell users generally achieve higher SINR values due to their proximity to serving base stations. Specifically, around 80% of small cell users experience SINR above 20 dB, compared to approximately 60% for macro users. The edge coverage threshold at 0 dB SINR indicates that both network tiers successfully maintain coverage for nearly all users, demonstrating the robustness of the network design.
3.3 Traffic model
Network traffic exhibits significant spatiotemporal variations, which we characterize using real-world traffic patterns from the Telecom Italia Milano dataset [21]. This dataset provides fine-grained measurements of mobile network activity across the city, enabling us to analyze both temporal and spatial traffic dynamics. As shown in Fig. 3(a), the network-wide temporal traffic dynamics over a representative day reveal a clear diurnal variation, with peak traffic occurring during daytime hours and minimal load during early morning periods. The two-hour moving average trend line effectively smooths short-term fluctuations while preserving the fundamental temporal pattern, highlighting the predictable yet dynamic nature of network traffic. This behavior motivates the use of time-series forecasting models for proactive resource allocation to optimize network performance [11]. Fig. 3(b) presents the cumulative distribution function of spatial traffic load across network cells, revealing significant spatial imbalance. Specifically, the top 20% of cells account for approximately 80% of the total traffic, consistent with the Pareto principle [27]. This observation underscores the importance of load-aware resource allocation strategies that can intelligently redistribute traffic to underutilized cells, thereby improving network efficiency and user experience.
3.4 Problem formulation
The resource allocation problem in HetNets involves jointly optimizing user association, power allocation, and spectral resource assignment to maximize network utility while satisfying quality of service (QoS) constraints [28]. Let x_{n,b} [member of] {0,1} denote the binary user association variable, where x_{n,b} = 1 indicates that user n is served by base station b. Let P_b denote the transmit power of base station b. The achievable rate for user n when associated with base station b is given by the Shannon capacity formula: R_{n,b} = W [log.sub.2](1 + SINR_{n,b})
where W is the system bandwidth and SINR_{n,b} is the signal-to-interference-plus-noise ratio. The optimization objective is to maximize the weighted sum rate while ensuring fairness among users:
maximize [summation]_n [summation]_b x_{n,b} * w_n * R_{n,b}
subject to [summation]_b x_{n,b} = 1, Vn (each user served by one BS)
[summation]_n x_{n,b} [less than or equal to] C_b, Vb (capacity constraints)
P_b [less than or equal to] P_max,b, Vb (power constraints)
SINR_{n,b} [greater than or equal to] SINR_th, Vn (QoS constraints)
where w_n represents user priority weights that can be adjusted to achieve different fairness objectives. This problem is combinatorial and NP-hard in general, motivating the use of machine learning approaches for efficient solution [29].
4. Proposed machine learning framework
The proposed framework integrates three machine learning modules operating at different time scales to address the multi-faceted resource allocation challenge [30]. The framework comprises a traffic prediction module, a user association module, and a power control module, which interact synergistically to optimize network performance. The traffic prediction module, implemented as an LSTM network operating at a medium time scale (minutes to hours), forecasts network load based on historical traffic patterns. The user association module, a load-aware algorithm operating at a slow time scale (seconds to minutes), determines optimal user-to-base station assignments. The power control module, a deep reinforcement learning agent operating at a fast time scale (milliseconds to seconds), optimizes transmit power levels. These modules work together, with traffic predictions informing user association decisions, which in turn feed into the power control optimization process. Fig. 4(a) provides insights into the model interpretability through a feature importance analysis using a Random Forest model, revealing that RSRP is the dominant factor with approximately 65% relative importance, followed by user demand and location features. The feature correlation matrix in Fig. 4(b) shows a strong correlation (0.73) between RSRP and SINR, validating the physical relationship between signal strength and quality. The partial dependence plot in Fig. 4(c) demonstrates the nonlinear relationship between RSRP and predicted SINR, with saturation behavior observed above -30 dBm. Finally, Fig. 4(d) visualizes the learned QoS utility surface, illustrating how the ML model has captured the coverage patterns from both the central MBS and the four SBSs, thereby enabling effective resource allocation strategies [31]. While prior works have independently applied LSTM networks for traffic forecasting [11, 32] or DRL for power control in HetNets [30, 33], these approaches treat each sub-problem in isolation. In contrast, the proposed framework explicitly couples the three modules across multiple time scales: LSTM-based predictions serve as proactive inputs to the load-aware user association module, whose output state is subsequently fed into the DRL power control agent. This hierarchical, cross-module information flow enables joint optimization of system sum rate, user fairness, and energy efficiency simultaneously-an objective that single-module or decoupled approaches cannot achieve. Furthermore, unlike standard DRL formulations that react to instantaneous channel states, the proposed DRL agent benefits from predicted future load conditions, effectively reducing the exploration space and accelerating policy convergence.
While the individual building blocks are established techniques, the novelty of this work lies in three aspects. First, at the module level: (i) the LSTM predictor is specifically designed with a 24-step look-ahead input trained on real heterogeneous network traffic exhibiting diurnal periodicity, and its output serves as a proactive state input to downstream modules rather than being used in isolation; (ii) the RL-based association jointly optimizes both the per-cell bias and the global load-balancing penalty coefficient [beta] as a coupled action space, which differs from conventional biased cell selection where these are set heuristically; (iii) the DRL power controller receives the predicted future load condition as part of its state vector, effectively reducing the exploration space and accelerating convergence compared to reactive DQN formulations. Second, at the framework level, the three modules operate across explicitly separated timescales, forming a hierarchical control loop in which decisions at each layer condition the state space of the layer below-a tightly coupled structure absent in prior decomposed approaches. Third, the reward function of the DRL module simultaneously encodes throughput maximization, QoS constraint satisfaction, and energy efficiency, enabling genuine multi-objective optimization without decomposition into separate single-objective problems.
5. Traffic prediction module
5.1 LSTM architecture
Long Short-Term Memory (LSTM) networks are particularly well-suited for traffic prediction due to their ability to capture long-range temporal dependencies while mitigating the vanishing gradient problem that affects standard recurrent neural networks [34]. The proposed LSTM architecture consists of two stacked LSTM layers with 128 and 64 hidden units, respectively, followed by a fully connected output layer. The input sequence length is set to 24 time steps, representing 4 hours of history at 10-minute intervals, and the model predicts traffic load for the next time step. The model is trained using the Adam optimizer [35] with an initial learning rate of 0.001 and a mean squared error (MSE) loss function. To prevent overfitting, early stopping with a patience of 10 epochs is employed. The training data comprises 70% of the available traffic measurements, with 15% used for validation and 15% reserved for testing. This comprehensive approach ensures robust and accurate traffic prediction capabilities. The LSTM model is trained with a batch size of 64 over a maximum of 100 epochs, with early stopping applied when validation loss fails to improve for 10 consecutive epochs. A dropout rate of 0.2 is applied after each LSTM layer to mitigate overfitting. The hidden state from the final LSTM layer is passed through a fully connected layer with linear activation to produce the one-step-ahead traffic load prediction. All weights are initialized using the Glorot uniform scheme.
5.2 Prediction performance
The traffic load prediction performance of the proposed LSTM model was further displayed. Fig. 5(a) compares the ground truth traffic load with the model predictions over 100 time steps. The model successfully captures both the overall trend and short-term fluctuations in traffic patterns. The yellow-shaded region indicates a period corresponding to peak hour traffic, where accurate prediction is particularly valuable for proactive resource provisioning. Fig. 5(b) shows the model convergence behavior during training, with both training and validation losses decreasing smoothly and reaching stable values after approximately 30 epochs. The gap between training and validation loss remains small throughout, indicating that the model generalizes well without overfitting. Fig. 5(c) displays the histogram of prediction errors, which follows an approximately Gaussian distribution centered near zero, confirming the unbiased nature of the predictions [32].
Table 2 provides a quantitative comparison of prediction accuracy across different models. The proposed LSTM achieves RMSE of 4.2, MAE of 3.1, and MAPE of 5.1%, significantly outperforming traditional time-series methods such as ARIMA (RMSE: 12.5, MAPE: 15.4%) and machine learning baselines such as Support Vector Regression (RMSE: 8.4, MAPE: 10.2%). The R-squared score of 0.94 indicates that the LSTM model explains 94% of the variance in traffic patterns, providing a reliable foundation for proactive resource management decisions [36]. It is worth noting that more recent sequence models, such as Transformer and Informer, have demonstrated strong performance on long-horizon forecasting tasks. However, these attention-based architectures typically require larger training corpora and significantly higher inference latency, making them less suitable for the real-time, resource-constrained deployment scenario targeted in this work. The proposed two-layer LSTM achieves an [R.sup.2] of 0.94 with low computational overhead, striking a practical balance between accuracy and efficiency.
6. User association and load balancing
6.1 Load-aware association algorithm
Traditional user association based on maximum RSRP leads to severe load imbalance, as most users connect to the high-power MBS despite the availability of nearby SBSs with lower load [37]. This phenomenon is evidenced in Fig. 6(a), which compares the baseline and proposed user association distributions. Under the Max-RSRP baseline, 80% of users (8,000 out of 10,000) connect to the macro base station while only 20% (2,000 users) are served by small cells. The proposed load-aware scheme significantly rebalances this distribution, reducing macro connections to 60% (6,000 users) while increasing small cell utilization to 40% (4,000 users).
The load-aware user association problem is formulated as a biased cell selection scheme where the association decision incorporates both signal quality and cell load information [38]:
Cell_selected = argmax_b {RSRP_b + [DELTA]_b - [beta] * LoadFactor_b}
where [DELTA]_b is the cell-specific bias (with positive bias for small cells to encourage offloading), and [beta] is a load balancing factor that penalizes heavily loaded cells. The bias values and balancing factor are optimized using a reinforcement learning approach that maximizes long-term system utility [39]. The RL optimization of the bias vector [DELTA]_b and load-balancing coefficient [beta] employs a tabular Q-learning agent with a discount factor [gamma] = 0.95 and a learning rate of [alpha] = 0.01. The state space encodes the discrete load level of each base station (quantized into five levels), and the action space comprises joint incremental adjustments to [DELTA]_b [member of] [0, 6] dB and [beta] [member of] [0, 1]. An [epsilon]-greedy exploration policy is used with [epsilon] decaying linearly from 1.0 to 0.05 over 500 training episodes.
6.2 Load balancing performance
Table 3 summarizes the user association statistics before and after applying the proposed scheme. The load imbalance ratio, defined as the ratio of macro to small cell connections, decreases from 4.00 to 1.50, representing a 62.5% reduction. This improved load distribution translates directly to enhanced user experience, particularly for users at the cell edge who previously suffered from congestion at the macro base station [40].
Fig. 6(b) presents the cumulative distribution function of user throughput under baseline and proposed schemes. The proposed load-aware approach significantly improves edge user performance, with the 5th percentile throughput (representing edge users) increasing by approximately 3.0x. This improvement is achieved by offloading edge users from the congested macro cell to nearby small cells with available capacity. The overall throughput distribution also shifts rightward, indicating improved performance across all user categories. The fairness convergence analysis in Fig. 6(c) demonstrates the iterative improvement in Jain's fairness index during the optimization process [41]. Starting from an initial value of approximately 0.4, the fairness index steadily increases and reaches the target level of 0.9 after approximately 35 iterations. The shaded region represents the confidence interval across multiple optimization runs, indicating consistent convergence behavior.
7. Power Control Optimization
7.1 Deep reinforcement learning formulation
The power control problem is formulated as a Markov Decision Process (MDP) and solved using deep reinforcement learning [13], [33]. The state space includes current channel conditions (RSRP, interference levels), user traffic demands, and cell load information. The action space comprises discrete power levels for each base station, ranging from minimum to maximum transmit power. The reward function is designed to maximize system sum rate while penalizing SINR violations:
R = [summation]_n R_n - [lambda] * [summation]_n max(0, SINR_th - SINR_n)
where [lambda] is a penalty coefficient for QoS constraint violations. The DRL agent uses a Deep Q-Network (DQN) architecture with experience replay and target network for stable learning [41]. The DQN consists of three fully connected layers with 256, 128, and 64 units respectively, each followed by a ReLU activation function, and outputs Q-values over the discrete action space. The network is trained using the Adam optimizer with a learning rate of 0.001 and a minibatch size of 64. Experience replay is implemented with a replay buffer of capacity 10,000 transitions; training begins after 1,000 warm-up steps and updates are performed every 4 environment steps. A separate target network is synchronized with the online network every 100 steps to stabilize learning. Exploration follows an [epsilon]-greedy policy with [epsilon] decaying linearly from 1.0 to 0.05 over the first 100 episodes, after which it is held fixed. The discount factor is set to [gamma] = 0.99.
7.2 Power control performance
The performance of the proposed DRL-based power control mechanism was illustrated, demonstrating significant improvements in network performance. As shown in Fig. 7(a), the Deep RL Power Control scheme effectively shifts the SINR distribution rightward, with the median SINR improving from approximately 15 dB to 20 dB. This enhancement is particularly notable in the low-SINR tail, where the proportion of users experiencing SINR below 5 dB decreases from 15% to less than 5%, directly translating to improved spectral efficiency and reduced outage probability [42]. Fig. 7(b) further highlights the DRL agent's convergence behavior during training, with the system sum rate increasing from an initial value of approximately 50 Mbps to over 115 Mbps after 200 training episodes. The shaded region, representing the standard deviation across training runs, decreases over time, indicating the agent's policy stabilizes effectively. The smooth convergence curve without significant oscillations underscores the agent's effective learning and stable policy improvement, ensuring robust and reliable network performance [43].
8. Performance evaluation
8.1 Comparative analysis
We evaluate the proposed framework against three baseline algorithms: random allocation, which assigns users to base stations without considering channel quality or load; Max-RSRP, a conventional approach where each user connects to the base station with the strongest received signal [24]; and game theory-based allocation, which models user association as a non-cooperative game and seeks Nash equilibrium solutions [44]. As shown in Fig. 8(a), the proposed DRL approach achieves a system sum rate of 345 Mbps, significantly outperforming the random allocation (150 Mbps), Max-RSRP (280 Mbps), and game theory-based allocation (310 Mbps). This represents a 23.2% improvement over the Max-RSRP baseline and an 11.3% improvement over the game theory approach. The substantial gap between random allocation and the other methods highlights the importance of intelligent resource allocation in maximizing network capacity. Fig. 8(b) illustrates the fairness comparison using Jain's fairness index [40], revealing that while the Max-RSRP approach achieves reasonable throughput, it exhibits the lowest fairness (0.2) due to its tendency to concentrate users on the macro base station, causing severe congestion for some users while others enjoy excellent service. In contrast, the proposed DRL approach achieves the highest fairness index of 0.85, effectively balancing performance across all users. The game theory approach achieves moderate fairness (0.7), while random allocation surprisingly performs better in fairness (0.4) due to its inherent randomization, though at the cost of reduced overall throughput.
Table 4 provides a comprehensive quantitative comparison of all algorithms. The proposed DRL approach achieves superior performance across all metrics, with system sum rate of 345 Mbps (23.2% gain over baseline), edge user throughput of 5.8 Mbps (176% improvement), and Jain's fairness index of 0.85. These results demonstrate that the proposed framework successfully addresses the multi-objective optimization challenge, simultaneously improving both network efficiency and user fairness.
To further contextualize the proposed DRL-based power control, we include a channel-aware Dynamic Power Allocation (DPA) baseline, adapted from the principles of Anand Jee et al. to the HetNet setting [45]. In this baseline, each base station allocates transmit power in proportion to the instantaneous channel gain ratio between serving users and interferers, without leveraging historical traffic patterns or multi-step reward optimization. As shown in Table 5, the DPA scheme outperforms the static Max-RSRP baseline in both sum rate and fairness, validating the importance of channel-aware adaptation as demonstrated in Anand Jee et al. However, the proposed DRL-based approach further surpasses DPA by additionally incorporating traffic load predictions and multi-objective reward optimization across episodes, enabling proactive power adjustments that go beyond reactive CSI-based control. These results confirm that the channel-awareness principle is a necessary condition for efficient resource allocation, while the DRL framework extends this by adding temporal learning and energy efficiency co-optimization.
8.2 Energy efficiency analysis
Energy efficiency defined as the ratio of achievable throughput to total power consumption (bits/Joule), is a fundamental performance metric for next-generation wireless systems, as also emphasized in Anand Jee et al., which demonstrates that in NOMA-based IoT networks, intelligently avoiding unnecessary relay transmissions via incremental relaying (IR) can nearly double energy efficiency compared to non-incremental counterparts [46]. Inspired by this principle of avoiding wasteful power expenditure, the proposed framework achieves analogous gains at the network level in HetNets: rather than suppressing unwanted relay activation, the DRL-based power control module dynamically adapts the transmit power of each base station to match actual traffic demand, while the LSTM-based traffic predictor enables proactive activation of sleep modes during low-load periods. This network-level approach is complementary to the PHY-layer protocol designs, and the two strategies may be jointly applied in future NOMA-enabled HetNet deployments for compounded energy savings. Energy efficiency is a critical consideration for sustainable network operation [47]. The energy efficiency performance of the proposed framework, demonstrating its effectiveness in optimizing power consumption was analyzed. As shown in Fig. 9(a), energy efficiency, measured in Mbits per Joule, increases with higher network traffic load for both baseline and proposed approaches, as static power consumption is amortized across more active transmissions. However, the proposed ML-EE approach consistently outperforms the baseline across all load conditions, with the gap being particularly pronounced at low loads, where intelligent sleep mode management provides the greatest benefits. Fig. 9(b) presents the power consumption breakdown across different components, revealing that while the static power consumption of macro and small base stations remains constant in the baseline, the proposed scheme reduces dynamic power consumption by intelligently matching transmit power to actual traffic requirements. Specifically, the reduction in Macro BS Dynamic power from 30% to 20% and Small BS Dynamic power from 20% to 10% demonstrates effective power adaptation based on traffic predictions and load conditions. This comprehensive approach ensures both energy efficiency and network performance are optimized, contributing to sustainable network operation [48].
Table 6 quantifies the energy efficiency gains under different traffic load conditions. Under low load (20%), the proposed approach achieves 35.4% energy saving compared to the baseline, as intelligent sleep mode activation reduces unnecessary power consumption. At medium load (50%), the energy saving is 33.1%, while at high load (80%), the gain is 32.2%. These consistent improvements across varying load conditions demonstrate the robustness of the proposed energy-aware resource allocation strategy.
8.3 Parameter sensitivity analysis
The parameter sensitivity analysis, providing insights into how system performance varies with network configuration was further explored. As shown in Fig. 10(a), network densification significantly impacts throughput gain, with the gain increasing from 5% to 24% as the number of small base stations increases from 2 to 10. However, the relationship exhibits diminishing returns, with each additional small base station contributing progressively less marginal improvement, suggesting an optimal densification level beyond which the costs of additional infrastructure may outweigh performance benefits. Fig. 10(b) examines the robustness of the proposed framework to user density by comparing user satisfaction ratios across different network loading scenarios. User satisfaction, defined as the proportion of users achieving their QoS requirements, degrades as user density increases from 100 to 500 users. While both baseline and proposed schemes experience degradation, the proposed approach maintains significantly higher satisfaction, degrading from 0.95 to 0.55 compared to the baseline's degradation from 0.90 to 0.20. This 175% improvement in satisfaction at high user density (500 users) underscores the superior scalability and robustness of the proposed ML-driven framework, demonstrating its ability to handle varying network conditions effectively.
9. Conclusion
This study presents a comprehensive machine learning-driven framework for dynamic resource allocation in heterogeneous wireless networks, integrating three synergistic components: an LSTM-based traffic prediction module, a load-aware user association scheme, and a deep reinforcement learning-based power control mechanism. Through extensive simulations using real-world traffic patterns from the Telecom Italia dataset, the framework demonstrates significant performance improvements across multiple metrics. The LSTM-based traffic prediction achieves a mean absolute percentage error (MAPE) of 5.1%, enabling proactive resource management decisions that anticipate future network conditions. The load-aware user association scheme reduces the load imbalance ratio by 62.5% and doubles the utilization of small cell capacity, resulting in improved overall resource efficiency. The deep reinforcement learning-based power control mechanism enhances the median SINR by 5 dB while reducing the proportion of users experiencing poor signal quality. The integrated framework achieves a 23.2% improvement in system sum rate compared to the Max-RSRP baseline, while simultaneously enhancing Jain's fairness index from 0.2 to 0.85. Furthermore, energy efficiency improves by up to 35.4% under low traffic load conditions, contributing to more sustainable network operation. These results validate the effectiveness of machine learning techniques in addressing the complex, multi-dimensional resource allocation challenges in heterogeneous wireless networks. The framework demonstrates that intelligent, data-driven approaches can simultaneously optimize conflicting objectives such as throughput maximization, fairness improvement, and energy efficiency enhancement. Looking ahead, promising directions for future research include extending the framework to multi-agent reinforcement learning (MARL) for distributed decision-making, incorporating mobility prediction and handover optimization for dynamic scenarios, integrating network slicing and multi-service optimization for diverse QoS requirements, and deploying federated learning approaches to preserve user privacy and reduce data transmission overhead. These extensions would further enhance the practicality and applicability of ML-driven resource allocation in future wireless networks.
Acknowledgement
Guided Project of the Scientific Research Project of the Department of Education of Hubei Province: Research on the End-to-End Multi-Slice Resource Allocation Algorithm in 5G (Project Number: B2023494).
References
[1] J. G. Andrews, S. Buzzi, W. Choi, S. V. Hanly, A. Lozano, A. C. K. Soong, and J. C. Zhang, "What Will 5G Be?," IEEE Journal on Selected Areas in Communications, vol.32, no.6, pp. 1065-1082, Jun. 2014. Article(CrossRefLink)
[2] A. Ghosh, N. Mangalvedhe, R. Ratasuk, B. Mondal, M. Cudak, E. Visotsky, T. A. Thomas, J. G. Andrews, P. Xia, H. S. Jo, H. S. Dhillon and T. D. Novlan, "Heterogeneous cellular networks: From theory to practice," IEEE Communications Magazine, vol.50, no.6, pp.54-64, Jun. 2012. Article(CrossRefLink)
[3] A. Damnjanovic, J. Montojo, Y. Wei, T. Ji, T. Luo, M. Vajapeyam, T. Yoo, O. Song, and D. Malladi, "A survey on 3GPP heterogeneous networks," IEEE Wireless Communications, vol.18, no.3, pp.10-21, Jun. 2011. Article(CrossRefLink)
[4] D. Lopez-Perez, I. Guvenc, G. de la Roche, M. Kountouris, T. Q. S. Quek, and J. Zhang, "Enhanced intercell interference coordination challenges in heterogeneous networks," IEEE Wireless Communications, vol.18, no.3, pp.22-30, Jun. 2011. Article(CrossRefLink)
[5] C. Jiang, H. Zhang, Y. Ren, Z. Han, K.-C. Chen, and L. Hanzo, "Machine Learning Paradigms for Next-Generation Wireless Networks," IEEE Wireless Communications, vol.24, no.2, pp.98-105, Apr. 2017. Article(CrossRefLink)
[6] M. Chen, U. Challita, W. Saad, C. Yin, and M. Debbah, "Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial," IEEE Communications Surveys & Tutorials, vol.21, no.4, pp.3039-3071, 4th Quart., 2019. Article(CrossRefLink)
[7] Cisco, Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2017-2022, Cisco Systems, Inc., San Jose, CA, USA, White Paper, Feb. 2019. Article(CrossRefLink)
[8] V. Chandrasekhar, J. G. Andrews, and A. Gatherer, "Femtocell networks: A survey," IEEE Communications Magazine, vol.46, no.9, pp.59-67, Sept. 2008. Article(CrossRefLink)
[9] X. Wang, Y. Han, C. Wang, Q. Zhao, X. Chen, and M. Chen, "In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning," IEEE Network, vol.33, no.5, pp.156-165, Sept.-Oct. 2019. Article(CrossRefLink)
[10] Q. Ye, B. Rong, Y. Chen, M. Al-Shalash, C. Caramanis, and J. G. Andrews, "User Association for Load Balancing in Heterogeneous Cellular Networks," IEEE Transactions on Wireless Communications, vol.12, no.6, pp.2706-2716, Jun. 2013. Article(CrossRefLink)
[11] J. Feng, X. Chen, R. Gao, M. Zeng, and Y. Li, "DeepTP: An End-to-End Neural Network for Mobile Cellular Traffic Prediction," IEEE Network, vol.32, no.6, pp.108-115, Nov.-Dec. 2018. Article(CrossRefLink)
[12] S. Wang, J. Cao, and P. S. Yu, "Deep Learning for Spatio-Temporal Data Mining: A Survey," IEEE Transactions on Knowledge and Data Engineering, vol.34, no.8, pp.3681-3700, Aug. 2022. Article(CrossRefLink)
[13] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, et al., "Human-level control through deep reinforcement learning," Nature, vol.518, no.7540, pp.529-533, Feb. 2015. Article(CrossRefLink)
[14] N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang, Y.-C. Liang, and D. I. Kim, "Applications of Deep Reinforcement Learning in Communications and Networking: A Survey," IEEE Communications Surveys & Tutorials, vol.21, no.4, pp.3133-3174, 4th Quart., 2019. Article(CrossRefLink)
[15] C. Zhang, P. Patras, and H. Haddadi, "Deep Learning in Mobile and Wireless Networking: A Survey," IEEE Communications Surveys & Tutorials, vol.21, no.3, pp.2224-2287, 3rd Quart., 2019. Article(CrossRefLink)
[16] H. D. Trinh, L. Giupponi, and P. Dini, "Mobile Traffic Prediction from Raw Data Using LSTM Networks," in Proc. of 2018 IEEE 29th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pp.1827-1832, Sept. 2018. Article(CrossRefLink)
[17] O. Naparstek and K. Cohen, "Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access," IEEE Transactions on Wireless Communications, vol.18, no.1, pp.310-323, Jan. 2019. Article(CrossRefLink)
[18] X. Zhao and D. Pompili, "Multi-Cell Interference Management Scheme for Next-Generation Cellular Networks," IEEE Transactions on Communications, vol.68, no.2, pp.1200-1212, Feb. 2020. Article(CrossRefLink)
[19] D. Fooladivanda and C. Rosenberg, "Joint Resource Allocation and User Association for Heterogeneous Wireless Cellular Networks," IEEE Transactions on Wireless Communications, vol.12, no.1, pp.248-257, Jan. 2013. Article(CrossRefLink)
[20] N. L. Jian, H. Zabiri, and M. Ramasamy, "Control of the Multi-Timescale Process Using Multiple Timescale Recurrent Neural Network-Based Model Predictive Control," Industrial & Engineering Chemistry Research, vol.62, no.15, pp.6176-6195, Apr. 2023. Article(CrossRefLink)
[21] G. Barlacchi, M. De Nadai, R. Larcher, A. Casella, C. Chitic, G. Torrisi, F. Antonelli, A. Vespignani, A. Pentland, and B. Lepri, "A multi-source dataset of urban life in the city of Milan and the Province of Trentino," Scientific Data, vol.2, Oct. 2015. Article(CrossRefLink)
[22] Y. Sun, M. Peng, Y. Zhou, Y. Huang, and S. Mao, "Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues," IEEE Communications Surveys & Tutorials, vol.21, no.4, pp.3072-3108, 4th Quart., 2019. Article(CrossRefLink)
[23] H. Zhang, N. Liu, X. Chu, K. Long, A.-H. Aghvami, and V. C. M. Leung, "Network Slicing Based 5G and Future Mobile Networks: Mobility, Resource Management, and Challenges," IEEE Communications Magazine, vol.55, no.8, pp.138-145, Aug. 2017. Article(CrossRefLink)
[24] D. Liu, L. Wang, Y. Chen, M. Elkashlan, K.-K. Wong, R. Schober, and L. Hanzo, "User Association in 5G Networks: A Survey and an Outlook," IEEE Communications Surveys & Tutorials, vol.18, no.2, pp.1018-1044, 2nd Quart., 2016. Article(CrossRefLink)
[25] 3GPP, Evolved Universal Terrestrial Radio Access (E-UTRA); Further advancements for E-UTRA physical layer aspects, 3rd Generation Partnership Project (3GPP), TR 36.814 V9.0.0, Mar. 2010.
[26] I. Hwang, B. Song, and S. S. Soliman, "A holistic view on hyper-dense heterogeneous and small cell networks," IEEE Communications Magazine, vol.51, no.6, pp.20-27, Jun. 2013. Article(CrossRefLink)
[27] R. Shafin, L. Liu, V. Chandrasekhar, H. Chen, J. Reed, and J. C. Zhang, "Artificial Intelligence-Enabled Cellular Networks: A Critical Path to Beyond-5G and 6G," IEEE Wireless Communications, vol.27, no.2, pp.212-217, Apr. 2020. Article(CrossRefLink)
[28] H. Hao, C. Xu, W. Zhang, S. Yang, and G.-M. Muntean, "Task-Driven Priority-Aware Computation Offloading Using Deep Reinforcement Learning," IEEE Transactions on Wireless Communications, vol.24, no.10, pp.8114-8128, Oct. 2025. Article(CrossRefLink)
[29] M. G. Kibria, K. Nguyen, G. P. Villardi, O. Zhao, K. Ishizu, and F. Kojima, "Big Data Analytics, Machine Learning, and Artificial Intelligence in Next-Generation Wireless Networks," IEEE Access, vol.6, pp.32328-32338, 2018. Article(CrossRefLink)
[30] F. B. Mismar, B. L. Evans, and A. Alkhateeb, "Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination," IEEE Transactions on Communications, vol.68, no.3, pp.1581-1592, Mar. 2020. Article(CrossRefLink)
[31] H. Sun, X. Chen, Q. Shi, M. Hong, X. Fu, and N. D. Sidiropoulos, "Learning to Optimize: Training Deep Neural Networks for Interference Management," IEEE Transactions on Signal Processing, vol.66, no.20, pp.5438-5453, Oct. 2018. Article(CrossRefLink)
[32] I. Alawe, A. Ksentini, Y. Hadjadj-Aoul, and P. Bertin, "Improving Traffic Forecasting for 5G Core Network Scalability: A Machine Learning Approach," IEEE Network, vol.32, no.6, pp.42-49, Nov.-Dec. 2018. Article(CrossRefLink)
[33] Y. S. Nasir and D. Guo, "Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks," IEEE Journal on Selected Areas in Communications, vol.37, no.10, pp.2239-2250, Oct. 2019. Article(CrossRefLink)
[34] S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol.9, no.8, pp.1735-1780, 1997. Article(CrossRefLink)
[35] D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," in Proc. of the 3rd International Conference for Learning Representations (ICLR 2015), pp.1-15, 2015. Article(CrossRefLink)
[36] C. Li, S. Long, H. Liu, Y. Choi, H. Sekiya, and Z. Li, "Enhancing Sparse Mobile CrowdSensing With Manifold Optimization and Differential Privacy," IEEE Transactions on Information Forensics and Security, vol.19, pp.6070-6083, 2024. Article(CrossRefLink)
[37] M. J. Alam, R. Chugh, S. Azad, and M. R. Hossain, "Optimizing cell association in 5G and beyond networks: a modified load-aware biased technique," Telecommunication Systems, vol.87, no.3, pp.731-742, Nov. 2024. Article(CrossRefLink)
[38] S. Singh, H. S. Dhillon, and J. G. Andrews, "Offloading in Heterogeneous Networks: Modeling, Analysis, and Design Insights," IEEE Transactions on Wireless Communications, vol.12, no.5, pp.2484-2497, May 2013. Article(CrossRefLink)
[39] Y. Kim, J. Jang, and H. J. Yang, "Distributed Resource Allocation and User Association for Max-Min Fairness in HetNets," IEEE Transactions on Vehicular Technology, vol.73, no.2, pp.2983-2988, Feb. 2024. Article(CrossRefLink)
[40] H. Zhang, S. Huang, C. Jiang, K. Long, V. C. M. Leung, and H. V. Poor, "Energy Efficient User Association and Power Allocation in Millimeter-Wave-Based Ultra Dense Networks With Energy Harvesting Base Stations," IEEE Journal on Selected Areas in Communications, vol.35, no.9, pp.1936-1947, Sept. 2017. Article(CrossRefLink)
[41] R. K. Jain, D.-M. W. Chiu, and W. R. Hawe, A Quantitative Measure of Fairness and Discrimination for Resource Allocation in Shared Computer Systems, Digital Equipment Corporation, Tech. Rep. DEC-TR-301, Sept. 1984. Article(CrossRefLink)
[42] F. Meng, P. Chen, L. Wu, and J. Cheng, "Power Allocation in Multi-User Cellular Networks: Deep Reinforcement Learning Approaches," IEEE Transactions on Wireless Communications, vol.19, no.10, pp.6255-6267, Oct. 2020. Article(CrossRefLink)
[43] H. Hao, C. Xu, W. Zhang, X. Chen, S. Yang, and G.-M. Muntean, "Reliability-Aware Optimization of Task Offloading for UAV-Assisted Edge Computing," IEEE Transactions on Computers, vol.74, no.11, pp.3832-3844, Nov. 2025. Article(CrossRefLink)
[44] M. Haddad, P. Wiecek, E. Altman, and H. Sidi, "A game theoretic approach for the association problem in two-tier HetNets," in Proc. of the 2013 25th International Teletraffic Congress (ITC), pp.1-9, Sept. 2013. Article(CrossRefLink)
[45] A. Jee and S. Prakriya, "Novel Channel Aware Strategies for Power Control in a Downlink NOMA Network With Mode Switching and User Selection," IEEE Transactions on Wireless Communications, vol.25, pp. 10115-10131, 2026. Article(CrossRefLink)
[46] A. Jee and S. Prakriya, "Performance of Energy and Spectrally Efficient AF Relay-Aided Incremental CDRT NOMA-Based IoT Network With Imperfect SIC for Smart Cities," IEEE Internet ofThings Journal, vol.10, no.21, pp.18766-18781, Nov. 2023. Article(CrossRefLink)
[47] Q. Wu, M. Tao, D. W. K. Ng, W. Chen, and R. Schober, "Energy-Efficient Resource Allocation for Wireless Powered Communication Networks," IEEE Transactions on Wireless Communications, vol.15, no.3, pp.2312-2327, Mar. 2016. Article(CrossRefLink)
[48] R. Q. Hu and Y. Qian, "An energy efficient and spectrum efficient wireless heterogeneous network framework for 5G systems," IEEE Communications Magazine, vol.52, no.5, pp.94-101, May 2014. Article(CrossRefLink)
Lingli Zhang received the B.S. in Electronic Science and Technology from Harbin University of Science and Technology in Heilongjiang Province, China in 2003. She received the M.S. in Communication and Information Systems from Wuhan University of Technology in Hubei Province, China in 2006. From 2006 to 2026, she was a full - time teacher at the School of Optoelectronic and Information Engineering of Wuhan Technical University in Hubei Province, China. Since 2015, she has served as an associate professor at Wuhan Technical University in Hubei Province. She is the author of two books, more than 10 articles. Her research interests include machine learning, modern communication technology, wireless communication, IP communication, and mathematical modeling.
Lingli Zhang (1*)
(1) School of Optoelectronic and Information Engineering, Wuhan Technical University, Wuhan, 430070, China [e-mail: Zhangll_WTU@163.com]
(*) Corresponding author: Lingli Zhang
Received December 21, 2025; revised March 24, 2026; accepted May 15, 2026; published July 31, 2026
http://doi.org/10.3837/tiis.2026.07.017
Table 1. Simulation parameters Parameter Value System Bandwidth 10 MHz Macro BS Transmit Power 46 dBm Small BS Transmit Power 30 dBm Thermal Noise Power -174 dBm/Hz Macro BS Path Loss Model 128.1 + 37.6 log10(d) Small BS Path Loss Model 140.7 + 36.7 log10(d) Shadowing Std Dev (Macro/Small) 8 dB / 10 dB Traffic Model Real-world (Telecom Italia) Number of Users 10,000 (Snapshots) Macro BS Count 1 Small BS Count 4
Table 2. Traffic load prediction accuracy Prediction RMSE MAE MAPE R-Squared Model ARIMA 12.5 9.1 15.4% 0.75 (Baseline) SVR (Baseline) 8.4 6.2 10.2% 0.82 Proposed LSTM 4.2 3.1 5.1% 0.94
Table 3. User association and load balancing performance
Metric Baseline Proposed Improvement
(Max-RSRP) (Load-Aware)
Macro BS 8000 (80%) 6000 (60%) -25.0%
Connections
Small BS 2000 (20%) 4000 (40%) +100.0%
Connections
Load Imbalance 4.00 1.50 Reduced by 62.5%
Ratio
Table 4. Performance comparison of different resource allocation
algorithms
Algorithm Sum Rate Edge Tput Fairness Gain vs
Index
(Mbps) (Mbps) Baseline
Random 150 1.5 0.40 -46.4%
Max-RSRP 280 2.1 0.20 0.0%
Game Theory 310 4.5 0.70 +10.7%
Proposed DRL 345 5.8 0.85 +23.2%
Table 5. Extended performance comparison including DPA baseline
Algorithm Sum Rate Edge Tput Fairness Index Energy Saving
(Mbps) (Mbps) (vs Baseline)
Max-RSRP 280 2.1 0.20 -
(Baseline)
Game Theory 310 4.5 0.70 18.2%
DPA 315 4.8 0.75 20.1%
Proposed DRL 345 5.8 0.85 33.1%
Table 6. Energy efficiency under different traffic loads Network Baseline EE EE Energy Irathc Proposed Saving Load (Mbits/J) (Mbits/J) 20% (Low 2.40 3.25 35.4% Load) 50% (Medium 3.08 4.10 33.1% Load) 80% (High 3.46 4.58 32.2% Load)
 Printer friendly
 Cite/link
 Email
 Feedback
| |
| Author: | Zhang, Lingli |
|---|---|
| Publication: | KSII Transactions on Internet and Information Systems |
| Article Type: | Report |
| Geographic Code: | 9CHIN |
| Date: | Jul 1, 2026 |
| Words: | 7447 |
| Previous Article: | Downlink NOMA Transmission Scheme Based on Adaptive Power Allocation. |
| Next Article: | An Optimization Algorithm for SVC Layered Edge Caching Based on UAV Collaboration. |
| Topics: | |