# Differential game theoretic approach for distributed dynamic cooperative power control in cognitive radio ad hoc networks.

1. IntroductionCognitive radio (CR) or dynamic spectrum access [1] has newly emerged as a promising solution to improve the spectrum utilization by allowing unlicensed secondary users (SUs) to access idle licensed spectrum. In a CR network, SUs can periodically sense the licensed spectrum and opportunistically access the spectrum holes or spectrum opportunities (SOPs) unoccupied by primary users (PUs). In addition, SUs can further form an infrastructure based CR network or a multi-hop ad hoc network. In a cognitive radio ad hoc network (CRANET) [2], SUs can only access the SOPs by seeking to underlay, overlay, or interweave their signals with those of existing PUs without significantly impacting their communications.

The primary objective of CR networks is to achieve both quality of service (QoS) for SUs and system throughput, and also to avoid causing excessive interference to PUs by means of dynamically allocating the transmit power of SUs. In this case, power control is essential for CR networks. In conventional celluar wireless networks and ad hoc networks, many power control schemes [3]-[5] and energy-efficient routing schemes [6]-[8] have been proposed to increase the total system throughput or to improve the energy efficiency. Other scheme provides the optimal power allocation on subchannel in two-tier femtocell networks based on orthogonal frequency division multiple access (OFDMA), aiming to maximize the total capacity [9]. Recently, there has been active research efforts in power control for CR networks from different perspectives, such as imperfect channel knowledge [10], [11], rate and energy efficiency [12], worst-case robust optimization [13], joint power control and resource allocation [14]-[16], and so on. As an alternative framework for modeling, game theoretic approach has gained more attention as an economics tool to study the resource allocation in OFDMA femtocells [17] and CR networks [18], [19]. Another body of work formulates the problem of distributed channel selection in CR networks as the local interaction game [20] and the exact potential game [21]. From the point of view of the dynamic behavior of SUs, differential game as an effective method is used to study the dynamic spectrum leasing under noncooperative model [22] and the noncooperative Stackelberg model [23]. In addition, many game models for power control in CR networks are also designed as the noncooperative models [24]-[26] and the cooperative models [27]-[29]. See Section 2 for a review of related work.

Our work in this paper mainly focuses on the underlay CRANET scenario owing to its simplicity of implementation and its high spectrum utilization. Clearly, the distributed strategy needs to be used to design the power control scheme due to the lack of centralized control and global information. In CRANETs, SUs are expected to cooperate with each other to enhance their own access opportunities and achieve high spectrum utilization. To this end, cooperative game theoretic approach is better suited to the problem of power control. Moreover, both the mobility and sometimes random nature of SUs result in the dynamic change of the CRANET topology with respect to time dependency. In general, given the dynamic time-varying network topology, it is unlikely to keep the transmit power unchangeably. However, there is a need to dynamically regulate the transmit power of SUs according to the dynamic nature of time dependency. That is, it will be far more realistic to dynamically adjust the transmit power according to the current time due to the impact of the dynamic behavior of SUs on power control in practical environment. Overall, the dynamic nature of power control in our work is aimed at the time dependency in that the CRANET topology is changing dynamically over time in light of the mobility and and sometimes the erratic nature of SUs. Taking into account that differential game explores interactive decision making over time, in this paper, we present a differential game theoretic approach for the distributed dynamic cooperative power control in CRANETs. Our main contributions can be outlined as follows:

* We define a channel stability factor to identify the impact of SOP switching due to ON-OFF state of PUs on the channel conditions, and also quantify a traffic sensing factor to formulate the priorities of the different data traffic.

* We propose a differential game model for the distributed dynamic power control. In our model, a payoff function is defined by taking into account both the stock of accumulated power interference to the cellular primary networks and the dynamic adjustment of the transmit power of SUs.

* We devise a dynamic programming problem to obtain a set of optimal strategies of SUs to the proposed distributed dynamic cooperative power control model under the condition of the grand coalition. We give more insights into the impact of the channel stability factor and the traffic sensing factor on both the QoS and the total interference constraint. Moreover, we develop a distributed dynamic cooperative power control algorithm to dynamically adjust the transmit power of SUs under grand coalition.

The rest of this paper is organized as follows. In Section 2, we review related work. Section 3 describes the system model and assumptions. In Section 4, we propose the differential game formulation and cooperative solution. Section 5 presents the numerical results. Finally, Section 6 concludes the paper.

2. Related Work

Many studies on power control for CR networks have been reported from different perspectives, such as imperfect channel knowledge [10], [11], rate and energy efficiency [12], worst-case robust optimization [13], joint power control and resource allocation [14]-[16], and so on. In the literature, game theoretic approach for resource allocation [18], [19] and distributed channel selection [20], [21] in CR networks has recently been well investigated. In comparison with the coalitional games [18] and the spectrum auction games [19], the local interaction game [20] and the exact potential game [21] has been used to invetigate the problem of distributed channel selection, aiming to obtain the existence of Nash equilibrium (NE) solutions under the constraint of local information of SUs. However, these solutions are designed under noncooperative game formulation, and they cannot point out the cooperative solution in the problem of distributed channel selection. In order to deal with the impact of the dynamic behavior of SUs on the design of game theoretic framework, differential game which investigates interactive decision making over time is leveraged to study dynamic spectrum leasing problem [22], [23]. In [22], a infinite-horizon noncooperative differential game model is formulated to describe the competition of dynamic spectrum leasing among secondary service providers. The instantaneous profit of secondary service provider is defined as the difference between the instant revenue and the cost of provider by devising two weighted cost factors. Both the open-loop NE strategy and the closed-loop NE strategy to the optimal control structure are also obtained. In [23], a noncooperative Stackelberg differential game model in the upper level of the hierarchical dynamic differential game framework is proposed to investigate the incentive mechanism for spectrum sharing between small cell service providers (SSPs) and macrocell service provider (MSP). Also, an open-loop Stackelberg equilibrium is derived as the solution of the optimal dynamic pricing problem for MSP and the dynamic open access ratios for SSPs.

In addition, recent work has investigated the noncooperative game models [24]-[26] and the cooperative game models [27]-[29] for power control in CR networks. In [24], a payoff function incorporating the utility function and the pricing function is presented in game model. The utility function is devised from the throughput perspective, and the pricing function is formulated as the exponential interference function. In [25], a cost function in game model is defined as logarithmic function in view of the guarantee of adequate QoS and interference temperature, and a distributed power iterative algorithm is developed. In [26], a game theoretic framework is established to solve the distributed power control under the condition of multiple secondary source-to-destination pairs and PUs. Also, a distributed algorithm is proposed to achieve a time average performance as good as that achieved when NE is chosen in hindsight. In [27], a cooperative Nash bargaining power control game model is formulated, in which interference power constraints and minimum signal-to-interference ratio (SIR) requirements are taken into account. An SIR-based utility function is further designed to comply with all the axioms in the Nash theorem, which guarantees the uniqueness and proportional fairness of the game equilibrium. In [28], a power control problem is modeled as a cooperative game under interference temperature limit, and a distributed algorithm that converges to the optimal solution of the power control problem based on Nash bargaining solution (NBS) is presented. Different from the utility function defined in [28], a utility function based on a fairness factor is designed in [29], and a distributed power control algorithm based on NBS is also introduced.

The differences between this paper and previous works are summarized as follows:

* The existing power control models [24]-[29] using the static game theoretic approach cannot take into account the underlying constraint that CR networks in practice is subject to the dynamic time-varying network topology wherein the change of transmit power level should be continuous in time. This observation motivates us to study the differential game theoretic approach for dynamic power control that helps to cope with power regulation overtime for the underlay CRANET scenario.

* Although [22] and [23] leverage the differential game theoretic approach to study the dynamic spectrum leasing problem, they do not consider that the possible cooperation between the players in the game models will result in better system performance.

* In contrast to the weighted cost factors defined in differential game model in [22] and

3. System Model and Assumptions

We consider a distributed underlay CRANET scenario as depicted in Fig. 1. In this scenario, PUs send their data to the primary base station (PBS) through the cellular primary networks with a finite set of m cells. Let M = {1, 2, ..., m} denote the set of cells in the cellular primary networks. It is noteworthy that the spectrum bands of primary networks are divided into two sections, i. e., the uplink spectrum bands and the downlink spectrum bands. We assume that cell l[member of]M has a licensed access to a given uplink and downlink spectrum band, and different cells hold different spectrum bands for interference constraint. The uplink spectrum band of cell l[member of]M is divided into C uplink channels used for licensed PUs. We also employ the independent and identically distributed alternating ON-OFF process to model the occupation time length of PUs in uplink channels. Specifically, the OFF state indicates the idle state where the unoccupied uplink channels or so called spectrum opportunities (SOPs) can be freely occupied by SUs. By the means of collaborative spectrum sensing [30], SUs can only leverage the OFF state to access the SOPs over the authorized uplink channels.

Let [t.sub.0] and T denote the starting time and the terminal time of dynamic power control in this scenario, respectively. We assume that n SU transmitter-receiver pairs are randomly distributed in cell l [member of] M where the PUs are inactive within time interval [[t.sub.0], T]. The SU pairs in cell l [member of] M can exchange the control messages with the help of the common control channels (CCCs). Moreover, the SUs are either fixed or slowly moving in cell l[member of]M. For simplicity, the terms "SU" and "pairs" are used interchangeably henceforth. Let N = {1, 2, ..., n} denote the set of SU pairs in cell l[member of]M. Owing to the randomness of the data traffic of PUs as well as the dynamics of the behavior of PUs, the SOPs are available for useage by SU i with a probability of [[delta].sub.i], for i [member of] N. According to [31], the SOP usage probability [[delta].sub.i] by SU i is written as

[[delta].sub.i] = 1 - [[[alpha].sub.i]/[[[alpha].sub.i] + [[beta].sub.i]]] (1)

where [[alpha].sub.i] is the probability that SOP transits from OFF state to ON state, and [[beta].sub.i] is the probability that SOP transits from ON state to OFF state.

We assume that the propagation channel of SU pairs is characterized by a slow-fading channel model, in which the channel conditions remain constant throughout time interval [[t.sub.0], T]. Let [[zeta].sub.l] and L be the interference caused by the PBS in cell l [member of] M and the normalized spread sequence length, respectively. At time instant s [member of] [[t.sub.0], T], the SIR of SU i in cell l [member of] M, for i [member of] N, is given as

[[gamma].sub.i](s) = [[p.sub.i](s) x [h.sub.ii]]/[[N.sub.0] + [1/L][summation over (j[member of]N\i)][p.sub.j](s) x [h.sub.ji] + [[zeta].sub.l]] (2)

where [p.sub.i](s) is the transmit power of SU i, [h.sub.ji] is the channel gain from SU transmitter j to SU receiver i, [N.sub.0] is the SU receiver's background noise power. In the case of slow-fading channel model, the channel gain is defined as [h.sub.ji] = A[d.sup.-[theta].sub.ji], where [d.sub.ji] is the distance from SU transmitter j to SU receiver i, A > 0 is the constant gain, and d is the propagation loss factor for outdoor wireless communications.

Let [[bar.p].sub.i] be the maximum transmit power of SU i. Then we have [p.sub.i](s) [less than or equal to] [[bar.p].sub.i]. Due to the impact of propagation loss of wireless link, we state that p can be generally written as [[bar.p].sub.i] = [p.sup.ref.sub.i]/[[eta].sub.i], where [[eta].sub.i] denotes the transmission loss from SU transmitter i to SU receiver i, and [p.sup.ref.sub.i] denotes the received reference power at SU receiver i, respectively. In the analytical derivation, [[eta].sub.i] = [(c/4[pi]f[d.sub.ii]).sup.2], where [d.sub.ii] is the distance from SU transmitter i to SU receiver i, f is the carrier frequency operating under the uplink channel, and c is the speed of light. Here, we further assume that the received reference power is equal for all the SU receivers and define this power as a baseline power factor. In this case, by taking into account the constraint of propagation loss of wireless link, we assume that [[bar.p].sub.i] is a monotone increasing function of the distance from SU transmitter i to SU receiver i. To be specific, we can further have [[bar.p].sub.i] = [(4[pi]f[d.sub.ii]).sup.2][p.sub.b]/[c.sup.2], where [p.sub.b] is the predefined baseline power factor. Let [d.sub.i] be the cell radius of cell l. So we have the constraint of 0 < [d.sub.ii] [less than or equal to] 2[d.sub.l]. We also suppose that [d.sub.ii] can be acquired over time through sensing the surrounding environment by SU i with the help of CCC. and [p.sub.b] is the received reference power at SU receiver i.

Let [J.sub.i](s) denote the throughput of SU i in cell l [member of] M at time instant s [member of][[t.sub.0], T]. According to Shannon's capacity formula, [J.sub.i](s) is approximately formulated as follows

[J.sub.i](s) = [log.sub.2] [l + k x [[gamma].sub.i](s)] (3)

where k = 1.5/(-ln(5BER)) is a constant for an acceptable bit error rate (BER) requirement [32]. Note that this is a reasonable choice for a slow-fading channel, such as additive white Gaussian noise environment. Therefore, the total system throughput J of the distributed CRANET is given as

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (4)

With respect to the considered CRANET, there are two constraints that need to be taken into account.

C1) QoS constraint: let [[gamma].sup.tar.sub.i] denote the target SIR of SU i. In order to maintain a certain QoS requirement, the SIR of SU i should be subject to the QoS constraint as follows

[[gamma].sub.i](s) [greater than or equal to] [[gamma].sup.tar.sub.i] (5)

C2) Total interference constraint: in order to avoid bringing excessive interference to the cellular primary networks, it is required that the total transmit power of SUs in cell l [member of] M should not exceed the constraint of the average interference power threshold [P.sub.th]. We assume that the interference measurement point (IMP) is located in the center of cell l. Let [d.sub.i] be the distance from SU transmitter i to the IMP of PBS in cell l [member of] M. Thus, the channel gain from SU transmitter i to the IMP of PBS is defined as [g.sub.i] = A[d.sup.-[theta].sub.i]. Further, we obtain the total interference constraint as follows

[summation over (i[member of]N)][p.sub.i](s) x [g.sub.i] [less than or equal to] [P.sub.th] (6)

4. Differential Game Formulation and Cooperative Solution

4.1 Payoff Function and Differential Game Formulation

Recall the SOP usage probability [[gamma].sub.i] in (1). To descirbe the impact of SOP switching due to ON-OFF state of PUs on the channel conditions, we characterize the channel stability factor of SU i in cell l [member of] M, which should be defined as a function of some parameters such as the SOP usage probability and the channel gain. Therefor, without loss of generality, the channel stability factor [S.sub.i] of SU i in cell l [member of] M is measured by

[S.sub.i] = [[delta].sub.i] x [h.sub.ii] (7)

Remark that the channel stability factor [S.sub.i] provides the tradeoff between the requirement for quickly finding an stable available SOP and the need for better channel conditions. Taking into consideration the impact of the differentiated types of data traffic on the demand of transmission quality, we introduce the traffic sensing factor to quantify the priorities of the different data traffic. We assume that the priority factor of the data traffic is denoted by [upsilon]. Based on the Enhanced Distributed Channel Access (EDCA) mechanism [33], the traffic sensing factor [[rho].sub.i] of SU i in cell l [member of] M is defined as

[[rho].sub.i] = [[Random.sub.i]([upsilon])/[summation over (i[member of]N)][Random.sub.i]([upsilon])] x [chi]([upsilon] (8)

where [Random.sub.i]([upsilon]) is a pseudo-random integer of SU i drawn from uniform distribution over the interval [0, CW([upsilon])], x([upsilon]) is an arbitration interframe space number, and the contention window CW([upsilon]) [member of] [CW min ([upsilon]), CW max ([upsilon])] is an integer within the range of values of the contention window limits CW min ([upsilon]) and CW max ([upsilon]). The parameters measured by the EDCA mechanism [33] used in (8) are defined as in Table 1. It is noteworthy that aCWmin and aCWmax denote the minimum size and the maximum size of CW([upsilon]), respectively. According to the EDCA mechanism [33], we assume aCWmin = 31 and aCWmax = 1023 in the case of physical layer specification of direct sequence spread spectrum. From (8) and Table 1, we can observe that the higher [[rho].sub.i] announced by SU i implies the lower priority with respect to the current type of data traffic.

In general, SU i in cell l [member of] M wants to enhance the transmit power aiming at achieving the better system performance and dealing with the channel impairments. On the other hand, this higher system performance is obtained at the expense of increased unacceptable interference to PUs and other SUs. Recall that the transmit power of SU i should also satisfy the total interference constraint. Then it is also necessary to reduce the transmit power of SU i without generating excessive interference to PUs. Therefore, on one hand, SU i needs to pay for the used SOP to transfer the prioritized data traffic, and the cost is determined by the tradeoff between the power enhancement and the power decline. On the other hand, SU i needs to pay for the accumulated power interference to PUs. Taking into account the constraint of both the impact of SOP switching on the channel conditions and the priority of data traffic, we formulate the cost function of the dynamic power reduction via Definition 1.

Definition 1 : The cost function of the dynamic power reduction for SU i is given by [S.sub.i][([p.sub.i](s) - [[bar.p].sub.i]).sup.2]/[[rho].sub.i].

Let y(s) and [u.sub.i] denote the stock of accumulated power interference to PUs and the pricing factor announced by SU i, respectively. Notice that the pricing factor [u.sub.i] implies the unit cost that SU i needs to pay due to the accumulated power interference to PUs. Thus, [u.sub.i] x y(s) corresponds to the net utility function with pricing for PUs.

Definition 2: The cost function of the accumulated power interference to PUs for SU i is given by [u.sub.i] x y(s).

Therefore, based on the cost functions formulated by Definition 1 and Definition 2, the payoff function [[PHI].sub.i]([p.sub.i], y) of SU i at time instant s [member of][[t.sub.0], T] is given as follows

[[PHI].sub.i]([p.sub.i], y) = [[S.sub.i]/[[rho].sub.i]] x [([p.sub.i](s) - [[bar.p].sub.i]).sup.2] + [u.sub.i] x y(s) (9)

It is clear that the payoff function [[PHI].sub.i]([p.sub.i], y) is a continuously differentiable function of [p.sub.i](s) and y(s). Notice that the lower [[rho].sub.i] or the higher priority for the current type of data traffic results in the higher payoff borne by SU i. Depending on the differential game theory framework [34], [p.sub.i] (5) in (9) will be viewed as the strategy or the control variable, while y(s) in (9) will be regarded as the state variable in differential game. In general, the strategy means the choice of action or behavior by player in differential game. The motivation behind using differential game is that the players need to dynamically adjust the transmit power, while traditional game theoretic formulation can be used mostly for static power control. Let SU i be player i of differential game. We assume that [Q.sub.i]([p.sub.i], y) is the terminal payoff of player i at time T. To this end, when the game terminates at time T, player i will receive a terminal payment of [Q.sub.i]([p.sub.i], y). With the underlying structure of differential game in mind [34], the payoff function of player i at time instant s [member of][[t.sub.0], T] in differential game holds an explicit game structure given as

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (10)

where 0 < [sigma] < 1 is the constant discount rate. It should be noted that [[PHI].sub.i]([p.sub.i], y) and [Q.sub.i] ([p.sub.i], y) have to be discounted by the factors [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], respectively. For convenience of derivation, we relax the time interval of the game and discuss the infinite-horizon differential game (i.e., T [right arrow] [infinity]), and we also set [t.sub.0] = 0. Moreover, it is easy to verify that [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. Hence, bearing in mind the game structure of differential game in (10), the objective of SU i is to minimize the payoff function

Minimize: [[integral].sup.[infinity].sub.0]{[[S.sub.i]/[[rho].sub.i]] x [([p.sub.i](s) - [[bar.p].sub.i]).sup.2] + [u.sub.i] x y(s)} x [e.sup.-rs]ds (11)

where r > 0 is the constant discount rate. According to differential game theory [34], the state variable y(s) in (11) is assumed to satisfy the differential equation as follows

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (12)

where [tau] > 0 is the penalty factor of the stock of accumulated power interference to the cellular primary networks. Here, denote [summation over (i[member of]N)][p.sub.i](s) - [tau] x y(s) by f([p.sub.i], y). More formally, (11) and (12) constitute the differential game model for distributed dynamic power control.

4.2 Cooperative Solution

In this subsection, we proceed to identify the optimal cooperative solution to our proposed differential game model in (11) and (12), and we will solve the dynamic optimization problem. The technique of dynamic programming developed by Bellman will be exploited to obtain the optimal solution to our game model. The technique is given by Lemma 1 [34].

Lemma 1: A set of optimal strategies [b.sup.*] = [p.sup.*](y) constitutes an optimal solution to the differential game model in (11) and (12), if there exists continuously differential value function W([p.sub.i], y) defined on [R.sup.n] [right arrow] R satisfying the Bellman equation as follows

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (13)

where [[PHI].sub.i]([p.sub.i], y) and f([p.sub.i], y) are continuously differentiable functions, respectively.

Definition 3: The grand coalition N is a coalition set N ={1, 2, ..., n} containing all players which agree to cooperate according to an agreed upon payoff allocation principle.

Clearly, in view of cooperative power control, all SUs in cell l [member of] M constitute the grand coalition N to cooperatively regulate their transmit power. Hence, we formulate a dynamic programming problem to obtain a set of optimal strategies of SUs to the distributed dynamic cooperative power control as follows

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (14)

Let a set of optimal strategies {[p.sup.N.sub.1], [p.sup.N.sub.2], ..., [p.sup.N.sub.n]} be the transmit power of n SUs under the condition of the grand coalition N to the dynamic programming problem in (14). We assume that there exists continuously differentiable function W([p.sub.i], y) which satisfies the Bellman equation based on Lemma 1 as follows

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (15)

Theorem 1: A set of optimal strategies {[p.sup.N.sub.i]|[p.sup.N.sub.i] = [[bar.p].sub.i] - [[[rho].sub.i][summation over (i[member of]N)][u.sub.i]/2[S.sub.i](r + [tau])]} provides the transmit power of n SUs under the condition of grand coalition N, and the continuously differentiable function W([p.sub.i], y) is expressed as follows

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (16)

Proof: Performing the minimization operation on the right side of (15) yields

[p.sup.N.sub.i] = [[bar.p].sub.i] - [[S.sub.i]/[[rho].sub.i]] x [[partial derivative]W([p.sub.i], y)/[partial derivative]y(s)] (17)

Substituting [p.sup.N.sub.i] in (17) into (15), we obtain

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (18)

Upon solving the differential equation in (18), we obtain

[partial derivative]W([p.sub.i], y)/[partial derivative]y(s) = [[summation over (i[member of]N)][u.sub.i]/(r + [tau]] (19)

Thus, the optimal strategy of of SU i is formulated as

[p.sup.N.sub.i] = [[bar.p].sub.i] - [[[rho].sub.i][summation over (i[member of]N)][u.sub.i]/2[S.sub.i](r + [tau])] (20)

Hence, the function W([p.sub.i], y) can be also obtained by (16).

From Theorem 1, we observe that the value of the optimal strategy of SU i is inversely proportional to both the pricing factor and the traffic sensing factor. Based on Theorem 1, we assume that the pricing factor and the traffic sensing factor of SU i in cell l [member of] M are given as [u.sub.i] = [u.sub.l] and [[rho].sub.i] = [[rho].sub.l], [for all]i [member of] N. It is reasonable to give this assumption in that the pricing factor can be equal to the same value, while the same data traffic type is also leveraged by SUs in cell l [member of] M. Under this condition, we can further demonstrate the QoS constraint together with the total interference constraint in detail by Theorem 2 and Theorem 3, respectively. For analytical simplicity, we define the notations as follows

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (21)

Theorem 2: Under the condition of grand coalition N, the following QoS constraint inequality is certainly strictly guaranteed if and only if [u.sub.i] = [u.sub.l] and [[rho].sub.i] = [[rho].sub.l], [for all]i [member of] N

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (22)

Proof: Under the condition of [u.sub.i] = [u.sub.l] and [[rho].sub.i] = [[rho].sub.l], we obtain

[p.sup.N.sub.i] = [[bar.p].sub.i] - [n[[rho].sub.l][u.sub.l]/2[S.sub.i](r + [tau])] (23)

Based on [[bar.p].sub.i] = [(4[pi]f[d.sub.ii]).sup.2][p.sub.b]/[c.sup.2], substituting [p.sup.N.sub.i] in (23) and [S.sub.i] in (7) into the QoS constraint in (5), we can easily prove the inequality in (22).

Theorem 3: Under the condition of grand coalition N, the following total interference constraint inequality is certainly strictly guaranteed if and only if [u.sub.i] = [u.sub.l] and [[rho].sub.i] = [[rho].sub.l], [for all] i [member of] N

[summation over (i[member of]N)](K[d.sup.2.sub.ii][d.sup.-[theta].sub.i] - [U/[[delta].sub.i]][([d.sub.ii]/[d.sub.i]).sup.[theta]]) [less than or equal to] [P.sub.th] (24)

Proof: Based on [[bar.p].sub.i] = [(4[pi]f[d.sub.ii]).sup.2][p.sub.b]/[c.sup.2], we substitute [p.sup.N.sub.i] in (23) and [S.sub.i] in (7) into the total interference constraint in (6). The derivation is very similar to that of Theorem 2, and is therefore skipped for brevity.

Algorithm 1: Distributed Dynamic Cooperative Power Control Algorithm 1: SUs obtain SOPs by collaborative spectrum sensing Initialization at time instant [t.sub.0] = 0 : parameters of system model and differential game model, 2: initial transmit power [p.sub.i](0) = [p.sup.initial.sub.i] of SU i, for 0 [less than or equal to] [p.sup.initial.sub.i] [less than or equal to] [[bar.p].sub.i] and [summation over (i[member of]N)][p.sup.initial.sub.i][g.sub.i] [less than or equal to] [P.sub.th]; 3: Update [[bar.p].sub.i] over time within time interval [0, T]; 4: while SOPs are used by SUs do 5: If [u.sub.i] = [u.sub.l] and [[rho].sub.i] = [[rho].sub.l], [for all]i [member of] N then Regulate [p.sub.i](s) at time instant s [member of][0, T] under the constraint of the set of optimal 6: strategies {[p.sup.N.sub.i]|[p.sup.N.sub.i] = [[bar.p].sub.i] - (n[[rho].sub.l][u.sub.l])/(2[S.sub.i](r + [tau]))}; 7: if [summation over (i[member of]N)]([Kd.sup.2.sub.ii][d.sup.-[theta].sub.ii] - [U/[[delta].sub.i]][([d.sub.ii]/[d.sub.i]).sup.[theta]]) [less than or equal to] [P.sub.th] then 8: go to Step 2; 9: else if [[([Kd.sup.2-[theta].sub.ii] - R[[gamma].sup.tar.sub.i])[[delta].sub.i] - U]/[[gamma].sup.tar.sub.i][[delta].sub.i]] [greater than or equal to] [L.sup.-1] [summation over (j[member of]N\i)] ([Kd.sup.2.sub.jj] - [U/[[delta].sub.j]][d.sup.[theta].sub.jj]) x [d.sup.-[theta].sub.ji] then 10: go to Step 2; 11: else if PUs use the SOPs then 12: go to Step 1; 13: else 14: go to Step 3; 15: end if 16: else 17: Regulate [p.sub.i](s) at time instant s [member of][0, T] under the constraint of the set of optimal strategies {[p.sup.N.sub.i]|[p.sup.N.sub.i] = [[bar.p].sub.i] - [[rho].sub.i][summation over (i[member of]N)[u.sub.i]/(2[S.sub.i](r + [tau]))}; 18: if [summation over (i[member of]N)][p.sub.i](s)[g.sub.i] > [P.sub.th] then 19: go to Step 2; 20: else if [[gamma].sub.i](s) < [[gamma].sup.tar.sub.i] then 21: go to Step 2; 22: else if PS uses the SOPs then 23: go to Step 1; 24: else 25: go to Step 3; 26: end if 27: end if 28: output [p.sub.1](s) = [p.sup.N.sub.1], [p.sub.2](s) = [p.sup.N.sub.2], ..., [p.sub.n](s) = [p.sup.N.sub.n]; 29: end while

Based on the optimal cooperative solution to our proposed differential game model, we turn to propose the distributed dynamic cooperative power control (DDCPC) algorithm in Algorithm 1 to dynamically regulate the transmit power of n SUs under grand coalition N in cell l [member of] M. According to the output of Algorithm 1, we obtain the optimal transmit power of n SUs, denoted by [p.sub.1](s) = [p.sup.N.sub.1], [p.sub.2](s) = [p.sup.N.sub.2], ..., [p.sub.n](s) = [p.sup.N.sub.n].

5. Numerical Results

Consider a distributed CRANET scenario depicted in Fig. 2, involving n = 12 SU transmitter (Tx)-receiver (Rx) pairs located randomly in the range of 50m x 50m square area. The IMP is located in the center of a cell. We assume that the SU pairs employ the same best effort traffic as the type of data traffic. Thus, we choose [upsilon] = 3 with aCWmin = 31 and aCWmax = 1023. The SOP usage probability [[delta].sub.i] is assumed to be generated with [[alpha].sub.i] = 0.2 and [[beta].sub.i] = 0.8. The channel model parameters are set to [theta] = 3, A = 0.097, and [N.sub.0] = -100 dBm for slow-fading channel model. We assume that the interference caused by the PBS in the cell is given by [[zeta].sub.l] = -20 dBm and the normalized spread sequence length is defined as L = 128. We choose the carrier frequency operating under the uplink channel f = 890.4 MHz. The received reference power at SU receiver is [p.sub.b] = 20 mW, and the initial transmit power [p.sub.i](0) = 2.26 mW. We assume the target SIR [[gamma].sup.tar.sub.i] = 8 for SU i, and the average interference power threshold [P.sub.th] = -80 dBm. The constant in the throughput of SU i is k = 0.2. In the differential game model, we choose the pricing factor [u.sub.i] = [u.sub.l] = 2.5 and the penalty factor [tau] = 5. For performance comparison, we consider the existing classical distributed constrained power control (DCPC) algorithm in [14]. It should be noted that DCPC algorithm distributively and iteratively searches for the power level which is updated from the [zeta] th iteration to the ([zeta] + 1)th iteration based on the current SIR. The iterative function of power adjustment in DCPC algorithm is given as

[p.sup.([zeta]+1).sub.i] = min{[[bar.p].sub.i], [[[gamma].sup.tar.sub.i]/[[gamma].sup.([zeta]).sub.i]] x [p.sup.([zeta]).sub.i]}, [zeta] = 0, 1, 2, ... (25)

Fig. 3 shows the transmit power comparison between the proposed DDCPC algorithm under different discount factor r and DCPC algorithm. From Fig. 3, it is apparent that an increase in distance from SU Tx to SU Rx will result in an enhancement of the transmit power. In general, high transmission distance needs high average power as the expense. Moreover, it can be observed the transmit power of the proposed DDCPC algorithm is obviously lower than that of DCPC algorithm from 10m to 35m in distance from SU Tx to SU Rx. This implies that the proposed DDCPC algorithm is more adaptable to the scenario of power control in short distance between SUs. In addition, by observing the impact of discount rate r on the power regulation of the proposed DDCPC algorithm, we find that reducing discount rate r yields the lower transmit power. This can be explained by the fact that DCPC algorithm yields more power consumption for maintaining a certain SIR. However, the transmit power of the proposed DDCPC algorithm relies on the maximum transmit power of SUs and the power levels can be further reduced via the change of discount rate r. Essentially, this signifies the importance of selecting discount rate r on the transmit power control in the proposed DDCPC algorithm.

Fig. 4 compares the payoff of SUs under different discount factor r in the proposed DDCPC algorithm. From Fig. 4, we can see that the payoff of SUs will increase when the distance from SU Tx to SU Rx increases. This can be explained by the fact that more power consumption will result in more payoff that SUs need to pay with the increase of the distance from SU Tx to SU Rx. Moreover, it is clear that the payoff of SUs with lower discount factor r could be less than that with larger discount factor r. The reason for this is that the payoff function of each SU must be discounted by the factor [e.sup.-rs] at time instant s under the differential game structure. Then we find that the lower discount factor r will lead to the reduction of the factor [e.sup.-rs]. Further, the payoff will also be decrease.

Next, we examine the effect of different number of SUs n on the total throughput between the proposed DDCPC algorithm under different discount factor r and DCPC algorithm as depicted in Fig. 5. It is seen that with the increase of number of SUs, the total throughput of the proposed DDCPC algorithm will increase significantly, while the total throughput of DCPC algorithm will be growing slowly. This is a direct consequence of the design of SIR balancing used in DCPC algorithm. However, the proposed DDCPC algorithm just take into account the conditon of QoS constraint in obtaining the total throughput. In particular, the total throughput of the proposed DDCPC algorithm increases as the discount rate r decreases. This emphasizes the importance of selecting discount rate r on the total throughput in the proposed DDCPC algorithm.

Finally, to evaluate the impact of different number of SUs n on the total payoff of SUs, Fig. 6 shows the total payoff of SUs under different discount factor r. From Fig. 6, we can see that the total payoff of SUs will increase with the growing of the number of SUs. This can be easily explained by the fact that more number of SUs will result in more power consumption. Also, more power consumption will generate more payoff that SUs need to pay. Moreover, similar to the phenomenon in Fig. 4, it is seen that the payoff of SUs with lower discount factor r could also be less than that with larger discount factor r under the same number of SUs. This is due to the fact that there is lower discounted portion by the factor [e.sup.-rs] to the payoff function of each SU with lower discount factor r.

6. Conclusion

In this paper, we have developed a differential game theoretic approach for distributed dynamic cooperative power control in the underlay CRANETs. We devise the payoff function of SU, and propose a differential game model for the distributed dynamic power control. By constructing the grand coalition, we present a dynamic programming problem to the proposed distributed dynamic cooperative power control model. Moreover, we obtain a set of optimal strategies of SUs, and show the effect of the channel stability factor and the traffic sensing factor on both the QoS and the total interference constraint. Based on the set of optimal strategies of SUs, we develop a distributed dynamic cooperative power control algorithm called the DDCPC algorithm to dynamically adjust the transmit power of SUs under grand coalition.

This work was supported in part by the National Natural Science Foundation of China under Grants 61402147, 61402529 and 61440001, the Natural Science Foundation of Hebei Province of China under Grant F2013402039, the Scientific Research Foundation of the Higher Education Institutions of Hebei Province of China under Grant QN20131048, and the Research Project of High-level Talents in University of Hebei Province under Grant GCC2014062.

http://dx.doi.org/10.3837/tiis.2015.10.003

References

[1] S. Haykin, "Cognitive radio: brain-empowered wireless communications," IEEE J. Sel. Areas Commun., vol. 23, no. 2, pp. 201-220, February, 2005. Article (CrossRef Link)

[2] I. F. Akyildiz, W.-Y. Lee, and K. R. Chowdhury, "CRAHNs: cognitive radio ad hoc networks," Ad Hoc Networks, vol. 7, no. 5, pp. 810-836, July, 2009. Article (CrossRef Link)

[3] A. El-Osery and C. Abdallah, "Distributed power control in CDMA cellular systems," IEEE Antennas Propag. Mag., vol. 42, no. 4, pp. 152-159, August, 2000. Article (CrossRef Link)

[4] M. Rasti, A. R. Sharafat, and J. Zander, "Pareto and energy-efficient distributed power control with feasibility check in wireless networks," IEEE Trans. Inf. Theory, vol. 57, no. 1, pp. 245-255, January, 2010. Article (CrossRef Link)

[5] V. Kawadia and P. R. Kumar, "Principles and protocols for power control in wireless ad hoc networks," IEEE J. Sel. Areas Commun., vol. 23, no. 1, pp. 76-88, January, 2005. Article (CrossRef Link)

[6] D. Jiang, Z. Xu, W. Li, and Z. Chen, "Network coding-based energy-efficient multicast routing algorithm for multi-hop wireless networks," J. of Syst. and Softw., vol. 104, pp. 152-165, June, 2015. Article (CrossRef Link)

[7] D. Jiang, Z. Xu, W. Li, and Z. Chen, "Topology control-based collaborative multicast routing algorithm with minimum energy consumption," Int. J. of Comm. Syst., [Online], December, 2014. Article (CrossRef Link)

[8] D. Jiang, Z. Xu, J. Liu, and W. Zhao, "An optimization-based robust routing algorithm to energy-efficient networks for cloud computing," Telecomm. Syst., [Online], March, 2015. Article (CrossRef Link)

[9] H. Zhang, C. Jiang, N.C.Beaulieu, X. Chu, X. Wen, and M. Tao "Resource allocation in spectrum-sharing OFDMA femtocells with heterogeneous services," IEEE Trans. Commun., vol. 62, no. 7, pp. 2366-2377, July, 2014. Article (CrossRef Link)

[10] E. Dall'Anese, S.-J. Kim, G. B. Giannakis, and S. Pupolin, "Power control for cognitive radio networks under channel uncertainty," IEEE Trans. Wireless Commun., vol. 10, no. 10, pp. 3541-3551, October, 2011. Article (CrossRef Link)

[11] S. Gong, P. Wang, Y. Liu, and W. Zhuang, "Robust power control with distribution uncertainty in cognitive radio networks," IEEE J. Sel. Areas Commun., vol. 31, no. 11, pp. 2397-2408, November, 2013. Article (CrossRef Link)

[12] S. M. Sanchez, R. D. Souza, E. M. G. Fernandez, and V. A. Reguera, "Rate and energy efficient power control in a cognitive radio ad hoc network," IEEE Signal Process. Lett., vol. 20, no. 5, pp. 451-454, May, 2013. Article (CrossRef Link)

[13] S. Parsaeefard and A. R. Sharafat, "Robust distributed power control in cognitive radio networks," IEEE Trans. Mobile Comput., vol. 12, no. 4, pp. 609-620, April, 2013. Article (CrossRef Link)

[14] Y. Xing, C. N. Mathur, M. A. Haleem, R. Chandramouli, and K. P. Subbalakshmi, "Dynamic spectrum access with QoS and interference temperature constraints," IEEE Trans. Mobile Comput., vol. 6, no. 4, pp. 423-433, April, 2007. Article (CrossRef Link)

[15] A. T. Hoang, Y.-C. Liang, and M. H. Islam, "Power control and channel allocation in cognitive radio networks with primary users' cooperation," IEEE Trans. Mobile Comput., vol. 9, no. 3, pp. 348-360, March, 2010. Article (CrossRef Link)

[16] R. Xie, F. R. Yu, and H. Ji, "Dynamic resource allocation for heterogeneous services in cognitive radio networks with imperfect channel sensing," IEEE Trans. Veh. Technol., vol. 61, no. 2, pp. 770-780, February, 2012. Article (CrossRef Link)

[17] H. Zhang, X. Chu, W. Ma, W. Zheng, and X. Wen, "Resource allocation with interference mitigation in OFDMA femtocells for co-channel deployment," EURASIP J. Wireless Commun, andNetw., vol. 2012, 289, September, 2012. Article (CrossRef Link)

[18] J. J. Alcaraz and M. van der Schaar, "Coalitional games with intervention: application to spectrum leasing in cognitive radio," IEEE Trans. Wireless Commun., vol. 13, no. 11, pp. 6166-6179, November, 2014. Article (CrossRef Link)

[19] Y. Chen, Y. Wu, B. Wang, and K. J. R. Liu, "Spectrum auction games for multimedia streaming over cognitive radio networks," IEEE Trans. Commun., vol. 58, no. 8, pp. 2381-2390, August, 2010. Article (CrossRef Link)

[20] Y. Xu, J. Wang, Q. Wu, A. Anpalagan, and Y.-D. Yao, "Opportunistic spectrum access in cognitive radio networks: global optimization using local interaction games," IEEE J. Sel. Topics in Signal Process, vol. 6, no. 2, pp. 180-194, April, 2012. Article (CrossRef Link)

[21] Y. Xu, J. Wang, Q. Wu, A. Anpalagan, and Y.-D. Yao, "Opportunistic spectrum access in unknown dynamic environment: a game-theoretic stochastic learning solution," IEEE Trans. Wireless Commun, vol. 11, no. 4, pp. 1380-1391, April, 2012. Article (CrossRef Link)

[22] K. Zhu, D.Niyato, P. Wang, and Z. Han, "Dynamic spectrum leasing and service selection in spectrum secondary market of cognitive radio networks," IEEE Trans. Wireless Commun., vol. 11, no. 3, pp. 1136-1145, March, 2012. Article (CrossRef Link)

[23] K. Zhu, E. Hossain, and D. Niyato, "Pricing, spectrum sharing, and service selection in two-tier small cell networks: a hierarchical dynamic game approach," IEEE Trans. Mobile Comput., vol. 13, no. 8, pp. 1843-1856, August, 2014. Article (CrossRef Link)

[24] W. Wang, Y. Cui, T. Peng, and W. Wang, "Noncooperative power control game with exponential pricing for cognitive radio network," in Proc. of IEEE 65th Vehicular Technology Conference (VTC), pp. 3125-3129, April 22-25, 2007. Article (CrossRef Link)

[25] F. Li, X. Tan, and L. Wang, "A new game algorithm for power control in cognitive radio networks," IEEE Trans. Veh. Technol., vol. 60, no. 9, pp. 4384-4391, November, 2011. Article (CrossRef Link)

[26] Y. Xiao, G. Bi, and D. Niyato, "A simple distributed power control algorithm for cognitive radio networks," IEEE Trans. Wireless Commun., vol. 10, no. 11, pp. 3594-3600, November, 2011. Article (CrossRef Link)

[27] C.-G. Yang, J.-D. Li, and Z. Tian, "Optimal power control for cognitive radio networks under coupled interference constraints: a cooperative game-theoretic perspective," IEEE Trans. Veh. Technol., vol. 59, no. 4, pp. 1696-1706, May, 2010. Article (CrossRef Link)

[28] Z. Wang, L. Jiang, and C. He "A distributed power control algorithm in cognitive radio networks based on Nash bargaining solution," in Proc. of IEEE International Conference on Communications (ICC), pp. 1672-1676, June 10-15, 2012. Article (CrossRef Link)

[29] B. Chen, J. Sun, S. Shao, L. Yang, and H. Zhu "An improved power control method based on cooperative game for cognitive radio networks," in Proc. of 12th IEEE International Conference on Communication Technology (ICCT), pp. 709-712, November 11-14, 2010. Article (CrossRef Link)

[30] S. Bokharaiee, H. H. Nguyen, and E. Shwedyk, "Cooperative spectrum sensing in cognitive radio networks with noncoherent transmission," IEEE Trans. Veh. Technol., vol. 61, no. 6, pp. 2476-2489, July, 2012. Article (CrossRef Link)

[31] A. Alshamrani, X. Shen, and L.-L. Xie, "QoS provisioning for heterogeneous services in cooperative cognitive radio networks," IEEE J. Sel. Areas Commun., vol. 29, no. 4, pp. 819-830, April, 2011. Article (CrossRef Link)

[32] X. Qiu and K. Chawla, "On the performance of adaptive modulation in cellular systems," IEEE Trans. Commun., vol. 47, no. 6, pp. 884-895, June, 1999. Article (CrossRef Link)

[33] Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, IEEE Standard, June, 2007. Article (CrossRef Link)

[34] D. W. K. Yeung and L. A. Petrosyan, Cooperative Stochastic Differential Games, New York: Springer, 2005. Article (CrossRef Link)

Long Zhang received the B.S. degree in communication engineering from China University of Geosciences, Wuhan, China, in June 2006 and the Ph.D. degree in communication and information systems from University of Science and Technology Beijing (USTB), Beijing, China, in January 2012. Since February 2012, he has been with the School of Information and Electrical Engineering from Hebei University of Engineering, Handan, China. His current research interests include opportunistic Internet of Vehicles, cognitive radio ad hoc networks, satellite and space communications, delay and disruption tolerant networks, resource allocation in wireless networks, network modeling and optimization, and cross-layer design.

Wei Huang received the B.S. degree in automation from the School of Electronic and Information Engineering, University of Science and Technology Liaoning, Liaoning, China, in June 2004, and the M.S. degree in control science and engineering from the School of Automation and Electrical Engineering, University of Science and Technology Beijing (USTB), Beijing, China, in January 2010. Currently, he is an Engineer of Energy Efficiency Evaluation Center of Electric Power Research Institute, Beijing, China. His current research interests include energy efficiency evaluation, power demand management, and communication technology.

Qiwu Wu received his Ph.D. degree in communication and information systems at department of communication engineering from University of Science and Technology Beijing (USTB), Beijing, China, in July 2010. Now he is a lecturer in department of Communication Engineering, Engineering University of Armed Police Force, Xi'an, China. His current research interests are in the areas of the military Internet of things, network security and optical networks.

Wenjing Cao received the B.S degree in electronics and information engineering from Hebei University of Engineering, Handan, China in 2012. She is currently working toward the M.S. degree in computer technology at the Department of Communication Engineering, Hebei University of Engineering, Handan, China. Her current research interests include congestion control in Internet of Vehicles, wireless resource management, and power control.

Long Zhang (1), Wei Huang (2), Qiwu Wu (3), and Wenjing Cao (1)

(1) School of Information and Electrical Engineering, Hebei University of Engineering Handan 056038, China

[e-mail: zhanglong@hebeu.edu.cn, wenjingfendou@hotmail.com]

(2) Energy Efficiency Evaluation Center, China Electric Power Research Institute Beijing 100192, China

[e-mail: huangwei2 @epri.sgcc.com.cn]

(3) Department of Communication Engineering, Engineering University of Armed Police Force Xi'an 710086, China

[e-mail: wuqiwu700@163.com]

* Corresponding author: Long Zhang

Received January 30, 2015; revised May 15, 2015; revised July 17, 2015; accepted August 8, 2015; published October 31, 2015

Table 1. Data Traffic Types and Parameters Data Traffic Types Priority CWmin ([upsilon]) Voice Traffic [upsilon] = 1 (aCWmin+1)/4-1 Video Traffic [upsilon] = 2 (aCWmin+1)/2-1 Best Effort Traffic [upsilon] = 3 aCWmin Background Traffic [upsilon] = 4 aCWmin Data Traffic Types CWmax (u) x([upsilon]) Voice Traffic (aCWmin+1)/2-1 2 Video Traffic aCWmin 2 Best Effort Traffic aCWmax 3 Background Traffic aCWmax 7

Printer friendly Cite/link Email Feedback | |

Author: | Zhang, Long; Huang, Wei; Wu, Qiwu; Cao, Wenjing |
---|---|

Publication: | KSII Transactions on Internet and Information Systems |

Article Type: | Report |

Date: | Oct 1, 2015 |

Words: | 8458 |

Previous Article: | A flexible network access scheme for M2M communications in heterogeneous wireless networks. |

Next Article: | Global optimization for energy efficient resource management by game based distributed learning in internet of things. |

Topics: |