Distributed traffic signal control for maximum network throughput

We propose a distributed algorithm for controlling traffic signals. Our algorithm is adapted from backpressure routing, which has been mainly applied to communication and power networks. We formally prove that our algorithm ensures global optimality as it leads to maximum network throughput even though the controller is constructed and implemented in a completely distributed manner. Simulation results show that our algorithm significantly outperforms SCATS, an adaptive traffic signal control system that is being used in many cities.


I. INTRODUCTION
Traffic signal control is a key element in traffic management that affects the efficiency of urban transportation.Many major cities worldwide currently employ adaptive traffic signal control systems where the light timing is adjusted based on the current traffic situation.Examples of widelyused adaptive traffic signal control systems include SCATS (Sydney Coordinated Adaptive Traffic System) [1]- [3] and SCOOT (Split Cycle Offset Optimisation Technique) [4], [5].
Control variables in traffic signal control systems typically include phase, cycle length, split plan and offset.A phase specifies a combination of one or more traffic movements simultaneously receiving the right of way during a signal interval.Cycle length is the time required for one complete cycle of signal intervals.A split plan defines the percentage of the cycle length allocated to each of the phases during a signal cycle.Offset is used in coordinated traffic control systems to reduce frequent stops at a sequence of junctions.
SCATS, for example, attempts to equalize the degree of saturation (DS), i.e., the ratio of effectively used green time to the total green time, for all the approaches.The computation of cycle length and split plan is only carried out at the critical junctions.Cycle length and split plan at non-critical junctions are controlled by the critical junctions via offsets.The algorithm involves many parameters, which need to be properly calibrated for each critical junction.In addition, all the possible split plans need to be pre-specified and a voting scheme is used in order to select a split plan that leads to approximately equal DS for all the approaches.Systems and control theory has been recently applied to traffic signal control problems.In [6], a multivariable regulator is proposed based on linear-quadratic regulator methodology and the store-and-forward modeling approach [7].Robust control theory has been applied to traffic signalization in [8].Approaches based on Petri Net modeling T. Wongpiromsarn is with the Singapore-MIT Alliance for Research and Technology, Singapore nok@smart.mit.eduT. Uthaicharoenpong, Y. Wang and D. Wang are with the Nanyang Technological University, Singapore tawit@ntu.edu.sg,wangyu81@e.ntu.edu.sg,edwwang@ntu.edu.sgE. Frazzoli is with the Massachusetts Institute of Technology, Cambridge, MA, USA frazzoli@mit.edulanguage are considered in, e.g., [9], [10].Optimizationbased techniques are considered, e.g., in [11], [12].However, one of the major drawbacks of these approaches is the scalability issue, which limits their application to relatively small networks.
To address the scalability issue, in [13], a distributed algorithm is presented where the signal at each junction is locally controlled independently from other junctions.However, global optimality is no longer guaranteed, although simulation results show that it reduces the total delay compared to the fixed-time approach.Another distributed approach is considered in [14] where the constraint that each traffic flow is served once, on average, within a desired service interval T is imposed.It can be proved that their distributed algorithm stabilizes the network whenever there exists a stable fixed-time control with cycle time T .However, the knowledge of traffic arrival rates is required.In addition, multi-phase operation is not considered.
An objective of this work is to develop a traffic signal control strategy that requires minimal tuning and scales well with the size of the road network while ensuring satisfactory performance.Our algorithm is motivated by backpressure routing introduced in [15], which has been mainly applied to communication and power networks where a packet may arrive at any node in the network and can only leave the system when it reaches its destination node.One of the attractive features of backpressure routing is that it leads to maximum network throughput without requiring any knowledge about traffic arrival rates [15]- [17].
To the authors' knowledge, this is the first time backpressure routing has been adapted to solve the traffic signal control problem.Since many assumptions made in backpressure routing are not valid in our traffic signalization application, certain modifications need to be made to the original algorithm.With these modifications, we formally prove that our algorithm inherits the desired properties of backpressure routing as it leads to maximum network throughput even though the signal at each junction is determined completely independently from the signal at other junctions, and no information about traffic arrival rates is provided.Furthermore, since our controller is constructed and implemented in a completely distributed manner, it can be applied to an arbitrarily large network.Simulation results show that our algorithm significantly outperforms SCATS.
The remainder of the paper is organized as follows: We provide useful definitions and existing results concerning network stability in the following section.Section III describes the traffic signal control problem considered in this paper.Our backpressure-based traffic signal control algorithm is described in Section IV.In Section V, we formally prove that our algorithm ensures global optimality as it leads to maximum network throughput, even though the signal at each junction is determined completely independently from other junctions.Section VI presents simulation results, showing that our algorithm can significantly reduce the queue length compared to SCATS.Finally, Section VII concludes the paper and discusses future work.

II. PRELIMINARIES
In this section, we summarize existing results and definitions concerning network stabilility.We refer the reader to [15]- [17] for more details.
Consider a network modeled by a directed graph with N nodes and L links.Each node maintains an internal queue of objects to be processed by the network, while each link (a, b) represents a channel for direct transmission of objects from node a to node b.Suppose the network operates in slotted time t ∈ N 0 where N 0 is the set of natural numbers (including zero).Objects may arrive at any node in the network and can only leave the system upon reaching the their destination node.Let A i (t) represent the number of objects that exogenously arrives at source node i during slot t and U i (t) represent the queue length at node i at time t.We assume that all the queues have infinite capacity.In addition, only the objects currently at each node at the beginning of slot t can be transmitted during that slot.Our control objective is to ensure that all queues are stable as defined below.
Definition 1: where for any event X, the indicator function 1 X takes the value 1 if X is satisfied and takes the value 0 otherwise.In this paper, we restrict our attention to strong stability and use the term "stability" to refer to strong stability defined above.For a network with N queues U 1 , . . ., U N that evolve according to some probabilistic law, a sufficient condition for stability can be provided using Lyapunov drift.
Proposition 1: Suppose E{U i (0)} < ∞ for all i ∈ {1, . . ., N } and there exist constants B > 0 and > 0 such that where for any queue vector Then the network is strongly stable.Definition 2: An arrival process A(t) is admissible with rate λ if: • There exists a finite value A max such that where H(t) represents the history up to time t, i.e., all events that take place during slots τ ∈ {0, . . ., t − 1}.• For any δ > 0, there exists an interval size T (which may depend on δ) such that for any initial time t 0 , For each node i, we define λ i to be the time average rate with which A i (t) is admissible.Let λ = [λ i ] represent the arrival rate vector.
Definition 3: The capacity region Λ is the closed region of arrival rate vectors λ with the following properties: • λ ∈ Λ is a necessary condition for network stability, considering all possible strategies for choosing the control variables (including strategies that have perfect knowledge of future events).• λ ∈ int(Λ) is a sufficient condition for the network to be stabilized by a policy that does not have a-priori knowledge of future events.The capacity region essentially describes the set of all arrival rate vectors that can be stably supported by the network.A scheduling algorithm is said to maximize the network throughput if it stabilizes the network for all arrival rates in the interior of Λ.

III. THE TRAFFIC SIGNAL CONTROL PROBLEM
A road network N is defined as a collection of links and signalized junctions.Let N and L be the number of links and junctions, respectively, in N .Then, N can be written as N = (L, J ) where L = {L 1 , . . ., L N } and J = {J 1 , . . ., J L } are sets of all the links and signalized junctions, respectively, in N .Each junction J i can be described by a tuple J i = (M i , P i , Z i ) where M i ⊆ L 2 is a set of all the possible traffic movements through J i , P i ⊆ 2 Mi is a set of all the possible phases of J i and Z i is a finite set of traffic states, each of which captures factors that affect the traffic flow rate through J i such as traffic and weather conditions.Each traffic movement through junction J i is defined by a pair (L a , L b ) where L a , L b ∈ L such that a vehicle may enter and exit J i through L a and L b , respectively.Each phase p ∈ P i defines a combination p ⊆ M i of traffic movements simultaneously receiving the right-of-way.A typical set of phases of a 4-way junction is shown in Figure 1.
We assume that the traffic signal system operates in slotted time t ∈ N 0 .During each time slot, vehicles may enter the network at any link.For each a ∈ {1, . . ., N }, i ∈ {1, . . ., L}, t ∈ N 0 , we let Q a (t) ∈ N 0 and z i (t) ∈ Z i represent the number of vehicles on L a and the traffic state around J i , respectively, at the beginning of time slot t.In addition, for each i ∈ {1, . . ., L}, we define a function ξ i : gives the rate (i.e., the number of vehicles per unit time) at which vehicles that can go from L a to L b through junction J i under traffic state z if phase p is activated.By definition, number of vehicles on L a that seek the movement to L b through J i is large, ξ i (p, L a , L b , z) can be simply obtained by assuming saturated flow.
At the beginning of each time slot, the traffic signal controller determines the phase for each junction to be activated during this time slot.In this paper, we consider the traffic signal control problem as stated below.
Traffic Signal Control Problem: Design a traffic signal controller that determines the phase p i (t) ∈ P i for each junction J i , i ∈ {1, . . ., L} to be activated during each time slot t ∈ N 0 such that the network throughput is maximized.We assume that there exists a reliable traffic monitoring system that provides the queue length Q a (t) and traffic state z i (t) for each a ∈ {1, . . ., N }, i ∈ {1, . . ., L} at the beginning of each time slot t ∈ N 0 to the controller.

IV. BACKPRESSURE-BASED TRAFFIC SIGNAL CONTROLLER
In this section, we propose a distributed traffic signal control algorithm that employs the idea from backpressure routing as described in [15]- [17].Unlike most of the traffic signal controllers considered in existing literature, our controller can be constructed and implemented in a completely distributed manner.Furthermore, it does not require any knowledge about traffic arrival rates.We end the section with a discussion of some basic properties of the proposed controller.
Our traffic signal controller consists of a set of local controllers C 1 , . . ., C L where local controller C i is associated with junction J i .These local controllers are constructed and implemented independently1 of one another.Furthermore, each local controller does not require the global view of Algorithm 1: Computation of phase p * to be activated during time slot t at junction J i .
Input: z i (t) and Q a (t) for all a ∈ {1, . . ., N } such that for each pair (L a , L b ) ∈ M i .Then, for each phase p ∈ P i , we compute (line 6 of Algorithm 1) The local controller C i then activates phase p * ∈ P i such that S p * ≥ S p , ∀p ∈ P i during the time slot t (line 7-9 of Algorithm 1).If there exist multiple options of p * that satisfy the inequality, the controller can pick one arbitrarily.Note that since the number of possible phases for each junction is typically small (e.g., less than 10), the above computation and enumeration through all the possible phases can be practically performed in real time.
Our algorithm is similar in nature to backpressure routing for a single-commodity network.In [15]- [17], it has been shown that backpressure routing leads to maximum network throughput.However, it is still premature to simply conclude that our backpressure-based traffic signal control algorithm inherits this property due to the following reasons.First, backpressure routing requires that a commodity at least defines the destination of the object.Implementing the algorithm for a single-commodity network implies that we assume that all the vehicles have a common destination, which is not a valid assumption for our application.Second, backpressure routing assumes that the controller has complete control over routing of the traffic around the network whereas in our traffic signal control problem, the controller does not have control over the route picked by each driver.Third, backpressure routing assumes that the network controller has control over the flow rate of each link subject to the maximum rate imposed by the link constraint.However, the traffic signal controller can only picks a phase p i (t) to be activated at each junction J i during each time slot t but does not have control over the flow rate of each traffic movement once p i (t) is activated.To account for this lack of control authority, we slightly modify the definition of W ab (t) from that used in backpressure routing.Finally, the optimality result of backpressure routing relies on the assumption that all the queues have infinite buffer storage space.Even though it is not reasonable to assume that all the links have infinite queue capacity, for the rest of the paper, we assume that this is the case.In practice, our algorithm is expected to work well when each link can accommodate a reasonably long queue.
Before evaluating the performance of our algorithm, we first provide its basic property, which is similar to the basic property of backpressure routing.Let P = P 1 × . . .× P L and Z = Z 1 ×. ..×ZL .For each a ∈ {1, . . ., N }, we define functions V out a : P × Z → R and V in a : P × Z → R such that for any p ∈ P and z ∈ Z, where for each i ∈ {1, . . ., L}, p i ∈ P i is the element of p that corresponds to the phase of junction J i and z i ∈ Z i is the element of z that corresponds to the traffic state of junction J i .Lemma 1: Consider an arbitrary time slot t ∈ N 0 .Let z(t) ∈ Z be a vector of traffic states of all the junctions during time slot t.For each i ∈ {1, . . ., L}, let p * i (t) denote the phase determined by Algorithm 1 to be activated at junction J i during time slot t and pi (t) be the phase to be activated at junction J i determined by any other algorithm for junction J i during time slot t.Then, where p(t) = [p i (t)] and p * (t) = [p * i (t)].Proof: First, we note the following identity for all p(t) ∈ P and z(t) ∈ Z.

V. CONTROLLER PERFORMANCE EVALUATION
Let Λ be the capacity region of the road network as defined in Definition 3. Assume that z(t) = [z i (t)] evolve according to a finite state, irreducible, aperiodic Markov chain.Let π z represent the time average fraction of time that z(t) = z, i.e., with probability 1, we have lim t→∞ is an indicator function that takes the value 1 if z(τ ) = z and takes the value 0 otherwise.In addition, we let M = i M i be the set of all the possible traffic movements.For the simplicity of the presentation, we assume that M i ∩M j = ∅ for all i = j.For each p ∈ P, z ∈ Z, we define a vector ξ(p, z) whose k th element is equal to and p i and z i are the i th element of p and z, respectively.Define where for any set S, Conv {S} represents the convex hull of S.
Additionally, we assume that the process of vehicles exogenously entering the network is rate ergodic and for all for all a ∈ {1, . . ., N }, there are always enough vehicles on L a such that for all i ∈ {1, . . ., L}, b ∈ {1, . . ., N }, p ∈ P i , z ∈ Z i such that (L a , L b ) ∈ M i , vehicles can move from L a to L b through junction J i at rate ξ i (p, L a , L b , z) under traffic state z if phase p is activated at J i .For each a ∈ {1, . . ., N }, let λ a be the time average rate with which the number of new vehicles that exogenously enter the network at link L a during each time slot is admissible.Let λ = [λ a ] represent the arrival rate vector.
Before deriving the optimality result for our backpressurebased traffic signal control algorithm, we first characterize the capacity region of the road network, as formally stated in the following lemma.
Lemma 2: The capacity region of the network is given by the set Λ consisting of all the rate vectors λ such that there exists a rate vector G ∈ Γ together with flow variables f ab for all a, b ∈ {1, . . ., N } satisfying such that (L a , L b ) ∈ M, such that where G ab is the element of G that corresponds to the rate of traffic movement (L a , L b ).
Proof: First, we prove that λ ∈ Λ is a necessary condition for network stability, considering all possible strategies for choosing the control variables (including strategies that have perfect knowledge of future events).Consider an arbitrary time t.For each a ∈ {1, . . ., N }, let X a (t) denote the total number of vehicles that exogenously enters the road network at link L a during time interval [0, t].Suppose the network can be stabilized by some policy, possibly one that bases its decisions upon complete knowledge of future arrivals.For each a, b ∈ {1, . . ., N }, let Q a (t) and F ab (t) represent the number of vehicles left on L a at time t and the total number of vehicles executing the (L a , L b ) movement during time interval [0, t] under this stabilizing policy.Due to flow conservation and link constraints, we have for all a, b ∈ {1, . . ., N } where p i (τ ) and z i (τ ) are the phase and traffic state, respectively, of junction J i at time τ .
For each a, b ∈ {1, . . ., N }, define f ab F ab ( t)/ t for some arbitrarily large time t.It is clear from (14) and ( 16) that ( 10) and ( 12) are satisfied.In addition, we can follow the proof in [16] to show that there exists a sample paths F ab (t) such that f ab comes arbitrarily close to satisfying (11) and (13).As a result, it can be shown that λ is a limit point of the capacity region Λ.Since Λ is compact and hence contains its limit points, it follows that λ ∈ Λ.
Next, we show that λ strictly interior to Λ is a sufficient condition for network stability, considering only strategies that do not have a-priori knowledge of future events.Suppose the rate vector λ is such that there exists > 0 such that λ+ ∈ Λ.Let G ∈ Γ be a transmission rate vector associated with the input rate vector λ + according to the definition of Λ.It has been proved in [16] that there exists a stationary randomized policy pi (τ ) for each i ∈ {1, . . ., L} that satisfies certain convergence bounds and such that for each In addition, such a policy stabilizes the system.
Corollary 1: Suppose z(t) is i.i.d.from slot to slot.Then, λ is within the capacity region Λ if and only if there exists a stationary randomized control algorithm that makes phase decisions p(t) based only on the current traffic state z(t), and that yields for all a ∈ {1, . . ., N }, t ∈ N 0 , where the expectation is taken with respect to the random traffic state z(t) and the (potentially) random control action based on this state.Finally, based on the above corollary and the basic property of our backpressure-based traffic signal control algorithm, we can conclude that our algorithm leads to maximum network throughput.
Theorem 1: If there exists > 0 such that λ + ∈ Λ, then the proposed backpressure-based traffic signal controller stabilizes the network, provided that z(t) is i.i.d.from slot to slot.
Proof: Consider an arbitrary policy p(t).By simple manipulations, we get where A a (t) is the number of vehicle that exogenously enter the network at link L a during time slot t, However, from Lemma 1, the proposed backpressurebased traffic signal controller minimizes the final term on the right hand side of the above inequality over all possible alternative policies p(t).But since λ + ∈ Λ, according to Corollary 1, there exists a stationary randomized algorithm that makes phase decisions based only on the current traffic state z(t) and that yields for all a ∈ {1, . . ., N }, t ∈ N 0 , Hence, we get that when the proposed backpressure-based traffic signal controller is used, and from Proposition 1, we can conclude that the network is stable.

VI. SIMULATION RESULTS
First, we consider a 4-phase junction with 4 approaches and 8 links as shown in Figure 2. Vehicles exogenously entering each of the 8 links are simulated based on the data collected from the loop detectors installed at the junction between Clementi Rd and Commonwealth Ave W, Singapore.
The maximum output rate of each lane is assumed to be 4 times of the maximum arrival rate of that lane.
We implemented SCATS, which is the system currently implemented in Singapore, and our algorithm in MATLAB.The parameters used in the SCATS algorithm are obtained from [3].Based on [18], [19], the queue length on each link L a evolves as follows.
) where I a (t) is the number of vehicles arriving at link L a during time slot t and I π a is a function that describes the number of passing vehicles and is given by ) Here, R a (t) = S a (t)g a (t) is the maximum number of passing vehicles where S(t) is the saturation flow and g(t) is the green time for link L a .Assuming that all the links have infinite queue capacity, queue lengths of each lane when our algorithm and SCATS are applied are shown in Figure 3.These simulation results show that our algorithm can reduce the maximum queue length by an order of magnitude, compared to SCATS, as shown in Figure 4. Figure 5 shows that our algorithm also performs significantly better on average.
Suppose each link can actually accommodate only 100 vehicles.Figure 6 shows that SCATS can only support up to 0.9 times of the current vehicle arrival rate whereas the the backpressure-based controller can support up to 1.3 times of the current vehicle arrival rate before the queue length exceeds the link capacity.
Next, we employ a microscopic traffic simulator MIT-SIMLab [20], whose simulation models have been validated against traffic data collected from Swedish cities, to evaluate our backpressure-based traffic signal control algorithm.We consider a road network with 112 links and 14 signalized junctions as shown in Figure 7. Vehicles exogenously enter and exit the network at various links based on 46 different origin-destination pairs, with the arrival rate of 9330 vehicles/hour.We implement SCATS and our backpressure-based traffic signal control algorithm in the traffic management simulator component of MITSIMLab.Queue length (i.e., the number of vehicles) on each link when each algorithm is used is continuously recorded.Note that in this case, the rate function ξ i , which is used in our algorithm, is still derived from the macroscopic model in (19).Hence, it may not accurately give the flow rate through the corresponding junction due to a possible mismatch between the macroscopic model in (19) and the microscopic model used in MITSIMLab.In addition,  as opposed to the previous 1-junction case, all the links have finite queue capacity in this case.
The maximum and average queue lengths are shown in Figure 8 and Figure 9, respectively.These simulation results show that our algorithm can reduce the maximum queue length by a factor of 3, compared to SCATS.In addition, it performs significantly better on average.One of the reasons that the difference in the queue lengths when our algorithm and SCATS are applied is not as significant as in the previous 1-junction case is because in this case, each link has a finite capacity.Hence, the number of vehicles on each link is limited by the link capacity and therefore queue length on each link cannot grow very large.In fact, as shown in Figure 10, queue spillback, where queues extend beyond one link upstream from the junction, persists throughout the simulation, especially when SCATS is used.
e., phase p does not give the right of way to the traffic movement from L a to L b .When traffic state z represents the case where the

Fig. 3 .
Fig. 3. Simulation results showing the arrival rate (dashed line) and the resulting queue length (solid line) of each lane when (top) backpressure-based controller and (bottom) SCATS are applied.Different colors correspond to different lanes.

Fig. 4 .
Fig.4.The maximum arrival rate and the maximum queue length over all the lanes when the backpressure-based controller and SCATS are applied.
)); Instead, it only requires information that is local to the junction with which it is associated.At each time slot t, local controller C i computes the phase p * ∈ P i to be activated at junction J i during time slot t as described in Algorithm 1.Consider an arbitrary junction J i ∈ J .At the beginning of time slot t, we first compute (line 4 of Algorithm 1)