EXACT ALGORITHMS FOR THE ATSP • BranchandBound Algorithms: • LittleMurtySweeneyKarel (Operations Research, 1963); • BellmoreMalone (Operations Research, 1971); • Garfinkel (Operations Research, 1973); • SmithSrinivasanThompson (Annals Discrete Math., 1977); • CarpanetoT. (Management Science, 1980); • BalasChristofides (Mathematical Programming, 1981); • PeknyMiller (Oper. Res. Letters, 1989); • FischettiT. (Mathematical Programming, 1992); • PeknyMiller (Mathematical Programming, 1992); • CarpanetoDell’AmicoT. (ACM Trans. Math. Software, 1995); • … 1
EXACT ALGORITHMS FOR THE ATSP • BranchandCut Algorithms: • FischettiT. (Management Science, 1997); • FischettiLodiT. (LNCS Springer, 2003); • …
• Surveys: • BalasT. (The Traveling Salesman Problem, Lawler et al. eds, Wiley, 1985); • FischettiLodiT.(The TSP and its Variations, GutinPunnen eds, Kluwer, 2002); • RobertiT. (EURO Journal Transp. and Logistics, 2011); • … 2
CLASSICAL LOWER BOUNDS ASSIGNMENT PROBLEM (AP) RELAXATION (O(n3) time) min s.t.
Σ Σ cij xij i∈V j∈V Σ xij
= 1
i∈V
Σ xij
=1
j∈V
j∈V
i∈V
Σ Σ xij ≤ i∈S j∈S
xij ∈ {0, 1}
S  1
S ⊂ V, S ≥ 2
Connectivity Constraints
i ∈ V, j ∈ V
xij ≥ 0 (LP Relaxation) i ∈ V, j ∈ V
* The AP solution is given by a family of “subtours” (partial circuits) 3
Example: AP relaxation of ATSP n = V = 6
1
6
3
5
2
1
7
7
6
2
3 4
1
7
1
3
9 8 5
5
9 3
6
Optimal assignment 4
Example: AP relaxation of ATSP 1
n = V = 6
3 1 6
2
3 4
1
5
1
3
3
Optimal assignment Lower bound = v(AP) = (1 + 1 + 3) + (3 + 1 + 3) = 12 Optimal solution Cost = 16 5
r – SHORTEST SPANNING ARBORESCENCE RELAXATION
min s.t.
Σ Σ cij xij i∈V j∈V Σ xij j∈V
= 1
i∈V
outdegree constraints
Σ xij
=1
j∈V
indegree constraints
i∈V
ΣΣ
i∈S j∈V\S
xij ≥ 1
xij ∈ {0, 1}
S ⊂ V, r ∈ S (for a fixed r ∈ V) i ∈ V, j ∈ V
6
r – SHORTEST SPANNING ARBORESCENCE RELAXATION Partition the Arc Set A into two subsets: (i,j) (i,r) min s.t.
Σ Σ cij xij i∈V j∈V\r Σ xij i∈V
=1
Σ xir i∈V
=1
ΣΣ
+
i ∈ V, j ∈ V \ r i∈V
Σi∈V cir xir j∈V\r indegree constraints
i∈S j∈V\S
xij ≥ 1
xij ∈ {0, 1} xir ∈ {0, 1}
S ⊂ V, r ∈ S
(j ∈ V \ r )
i ∈ V, j ∈ V \ r i∈V
The Relaxed Problem can be split into two independent problems
7
r  SHORTEST SPANNING ARBORESCENCE RELAXATION Problem concerning the arc set: (i,j) min s.t.
i ∈ V, j ∈ V \ r
Σ Σ cij xij i∈V j∈V\r Σ xij
i∈V
ΣΣ
=1
i∈S j∈V\S
xij ≥ 1
j∈V\r
indegree constraints
S ⊂ V, r ∈ S
connectivity constraints from r
xij ∈ {0, 1} xij ≥ 0 (LP Relaxation)
i ∈ V, j ∈ V \ r i ∈ V, j ∈ V \ r
(O(n2) time) The problem calls for an rSSA: Shortest (Minimum Cost) Spanning Arborescence rooted at vertex r (involving n – 1 arcs) 8
r – SHORTEST SPANNING ARBORESCENCE RELAXATION Problem concerning the arc set: (i,r) i ∈ V min s.t.
Σ cir xir
i∈V
Σ
i∈V
xir = 1
indegree constraint for vertex r
xir ∈ {0, 1} xir ≥ 0 (LP Relaxation)
i∈V i∈V
The problem calls for an rMCA: Minimum Cost Arc entering vertex r (O(n) time)
Lower Bound = v(rSSAR) = v(rSSA) + v(rMCA) 9
Example: rSSA relaxation of ATSP (r = 1) 1
x
3
x
7
7
5
2
1
6
6
x
3
x
4
1 9
x
7
2
1
3 8
5
x
9
5
3
6
Optimal 1SSA, v(1SSA) = 1 + 1 + 2 + 3 + 1 = 8 Optimal 1MCA, v(1MCA) = 3, Lower bound = 8 + 3 = 11, Optimal solution Cost = 16
10
r – SHORTEST SPANNING ARBORESCENCE RELAXATION • Different choices of the root vertex r generally produce different values of the corresponding lower bounds. • The r – Shortest Spanning Anti – Arborescence Relaxation can be used as well (connectivity toward r). • The lower bounds can be strengthened through Lagrangian Relaxation of the outdegree constraints, and Subgradient Optimization Procedures (to determine “good” Lagrangian multipliers). 11
Example: rSSA relaxation of ATSP (r = 3) 1
6
3
7
6
5
2
1
7
x
x
3
x
4
1 9
x
7
2
1
3
8 5
9
x
x
5
3
6
Optimal 3SSA, v(3SSA) = 3 + 3 + 5 + 1 + 1 = 13 Optimal 3MCA, v(3MCA) = 1, Lower bound = 13 + 1 = 14, Optimal solution Cost =
12
Example: rAntiSSA relaxation of ATSP (r = 1) 1
x
3
1
6
x2
5
x
7
6
7
2
3 4
1 9
x
7
1
3
8 5
9
x
x 5
3
6
Optimal 1AntiSSA, v(1AntiSSA) = 3 + 1 + 5 + 3 + 1 = 13 Optimal 1AntiMCA, v(1AntiMCA) = 1, Lower bound = 13 + 1 = 14, Optimal solution Cost = 16
13
LAGRANGIAN RELAXATION OF THE r  SHORTEST SPANNING ARBORESCENCE RELAXATION (ui: “Lagrangian multiplier” associated with the ith outdegree constraint)
Σ Σ cij xij i∈V j∈V
min s.t. ui
Σ xij j∈V Σ xij
i∈V
ΣΣ
= 1
=1
i∈S j∈V\S
xij ≥ 1
xij ∈ {0, 1}
i∈V
outdegree constraints j∈V
indegree constraints
S ⊂ V, r ∈ S (for a fixed r ∈ V) i ∈ V, j ∈ V
14
LAGRANGIAN RELAXATION OF THE r – SHORTEST SPANNING ARBORESCENCE RELAXATION (ui: “Lagrangian multiplier” associated with the ith outdegree constraint) z(u) = s.t. ui
Σ Σ cij xij i∈V j∈V
min
Σ xij j∈V Σ xij
i∈V
ΣΣ
= 1
=1
i∈S j∈V\S
xij ≥ 1
xij ∈ {0, 1}
+
Σ ui ( Σ xij i∈V j∈V
 1
i∈V
) outdegree constraints
j∈V
indegree constraints
S ⊂ V, r ∈ S (for a fixed r ∈ V) i ∈ V, j ∈ V
15
LAGRANGIAN RELAXATION OF THE r – SHORTEST SPANNING ARBORESCENCE RELAXATION z(u) =
min
=

=

Σ Σ cij xij i∈V j∈V
Σ
ui + min
Σ
ui + min
i∈V
i∈V
+
Σ ui ( Σ xij i∈V j∈V
 1
)
Σ Σ (cij j∈V
+ ui) xij
Σ Σ c’ij j∈V
xij (where c’ij = cij + ui )
i∈V
i∈V
c’ij is the Lagrangian cost of arc (i,j) ( with (i,j) ∈ A)
LAGRANGIAN DUAL PROBLEM Find the optimal Lagrangian multiplier vector u* such that: z(u*) = max {z(u)} u
(with z(u) “nondifferentiable function”)
Difficult problem (heuristic solution through Subgradient Optimization Procedures) 16
SUBGRADIENT OPTIMIZATION PROCEDURE
(HeldKarp, Operations Research 1971, for STSP) Lagrangian cost: * Initially:
c’ij = cij + ui
(i,j) ∈ A
ui := 0 i ∈ V ; LB := 0
At each iteration: * Solve the rSSA Relaxation with respect to the current Lagrangian costs c’ij : (xij) is the corresponding optimal solution * LB := max (LB, v(xij)) * Subgradient vector (si): si := Σ xij  1 j∈V
i∈V
* If si = 0 for all i ∈ V then STOP ((xij) is the opt. sol. of ATSP) * ui := ui + a si i ∈ V where a := b (UB – LB) / Σ si2 ( with 0 < b ≤ 2) (a > 0) i∈V
Stopping criteria: max number of iterations, slow increase of LB,… 17
SUBGRADIENT OPTIMIZATION PROCEDURE: UPDATING OF THE LAGRANGIAN MULTIPLIERS * c’ij = cij + ui
(with (i,j) ∈ A)
* ui := ui + a si
i ∈ V (with a positive)
* si := Σ xij  1 j∈V
i∈V
* di = outdegree of vertex i (i ∈ V) (di = 0 if si = 1; di = 1 if si = 0; di ≥ 2 if si ≥ 1) * the Lagrangian cost c’ij of arc (i,j) is decreased unchanged increased Note that: Σ ui = 0
i∈V
if di = 0. if di = 1, if di ≥ 2. a Σ si = a ( Σ i∈V
i∈V
Σ xij  n) = a (n – n) = 0 j∈V
(if initially ui = 0 i ∈ V ) 18
GREEDY ALGORITHM “NEAREST NEIGHBOR” FOR ATSP (Upper Bound computation) 1. Choose any vertex h as “initial vertex” of the current “path”. Set i:= h, V’:= V \ {i} (set of the “unvisited” vertices). 2. Determine the “unvisited” vertex k “nearest” to vertex i (k : cik = min {cij : j ∈ V’}). 3. Insert vertex k just after vertex i in the current path (V’:= V’ \ {k}); set i:= k; If V’ ≠ ∅ (at least one vertex is unvisited) return to STEP 2. 4. Complete the Hamiltonian circuit with arc (i, h); STOP.
v Time complexity: O(n2).
v Different choices of the “initial vertex” lead to different solutions. 19
Example: Nearest Neighbor Heuristic Algorithm (Upper Bound computation)
Initial vertex 1
1
6
3
6
x 1
9
x
7
5
2
1
x
7
3 4
7
8
2
x x3
1
x
9
5
5
3
6
UB = 1 + 1+ 7 + 3 + 1 + 6 = 19
20
Example: rSSA relaxation of ATSP (r = 1) 1
Subgradient Procedure (b = 0.25, 6 UB = 19) 9
6
x
3
x 7
7
5
2
1
x
x
3 4
1
x
LB1 = 8 + 3 = 11 (u) = (0, 0, 0, 0, 0, 0)
7
2
1
3 8
5
x
9
5
3
6
(d) = (2, 1, 0, 1, 1, 1),
(s) = (1, 0, 1, 0, 0, 0)
a := b (UB – LB1) / Σ si2 = 0.25 (19 – 11) / 2 = 1 i∈V (u) = (1, 0, 1, 0, 0, 0) (u) = (u) + a (s)
21
Example: rSSA relaxation of ATSP (r = 1) Subgradient Procedure 3 (a = 1) 2
1
x6
x x2 1
5
3
7
6
5
7
2
3 4
1
2
7
9
1
5
4
x5
9
8
x6
x3 3
5
(c’) = (c) + (u) (u) = (1, 0, 1, 0, 0, 0) 22
Example: rSSA relaxation of ATSP (r = 1) 1
Subgradient Procedure 6 x x 3 (a = 1, 2 x 5 5 2 1 x x LB1 = 11) 6 3 7 1
9
x
x
7
3
4
2
7
8 5
2
x x3
1
x
4
x5
9 3
x6 5 LB2 = 3 + 3 + 1 + 2 + 1 + 3 = 13 (d) = (2, 1, 0, 1, 1, 1) … 23
• The AP Relaxation can be strengthened by combining the AP substructure with the SSA substructures:
• Restricted Lagrangian Relaxation (BalasChristofides, 1981): 1) Lagrangian Relaxation of a subset of Subtour Elimination Constraints (SECs); 2) Solve the corresponding AP problem (w.r.t. the current Lagrangian costs); 3) Determine good Lagrangian multipliers (through a subgradient optimization procedure) and additional SECs, to be inserted into the current Lagrangian relaxation and iterate on Steps 2) and 3). 24
Additive Bounding Procedure (FischettiT., Math.Progr.1992) 1) Solve the AP Relaxation through a “primaldual” algorithm (O(n3) time) LB := v(AP), c’ij := “reduced cost” of arc (i,j); (O(n2) time): 2) Solve the rSSA Relaxation w.r.t. costs c’ij LB := LB + v(rSSAR), c’ij := new “reduced cost” of arc (i,j); 2) time): (O(n 3) Solve the rAntiSSA Relaxation w.r.t. costs c’ij LB := LB + v(rAntiSSAR), c’ij := new “reduced cost” of arc (i,j);
Iterate Steps 2) and 3) with different root nodes r. (globally O(n3) time)
25
Example: Additive Bounding Procedure rSSA Relaxation (r = 1) 1
x3
0
0 9 x
x1 4 x7
6
6
x
x x2x
0
1
x6 x 5 2
3
6 x7 x3 0
4
1 x
x
4
5
0
x7 7 x8
LB1 = 12 (c’) = AP reduced costs Optimal assignment
x
2
1 x
x3
0
x
6 x9
2
x5
3
x6 3
v(1SSAR) = 0 + 0 + 1 + 0 + 0 + 0 = 1 LB2 = LB1 + v(1SSAR) = 12 + 1 = 13
26
Example: Additive Bounding Procedure LB1 = 13 (c’) = 1SSAR reduced costs
rAntiSSA Relaxation (r = 1) 1
x3
0 6
0 9 x
6
x
0 x x2 1
x1
x x7 4 3
4
5
x 5 2
3
x
6 x7 x3 0
2
4
1 x
x
x1
0
x6
0
x7 6
7 x8 x
6 x9
1 x
x3
x
0
x
2
x5
3
x6 3
v(1AntiSSAR) = 0 + 0 + 2 + 0 + 0 + 0 = 2 LB2 = LB1 + v(1AntiSSAR) = 13 + 2 = 15
27
BRANCHANDBOUND ALGORITHM FOR ATSP BASED ON THE AP RELAXATION (CarpanetoT., Man. Sc. 1980) • At each node of the decision tree solve the APRelaxation of the corresponding subproblem (LB= v(AP)). • If LB ≥ Z* fathom the node (Z* = cost of the best solution). • If the AP solution contains no subtour (feasible solution), update Z* (and the best solution) and “fathom” the node. • Otherwise: SUBTOURELIMINATION BRANCHING SCHEME:  Select the subtour S with the minimum number h of not imposed arcs.  Generate h descendent nodes so as to forbid subtour S for each of them (by “imposing” and “excluding” proper arc subsets). 28
BRANCHING TREE FOR ATSP AP(k) solution
level 1
arc (8, 3) “imposed”
...
...
1
X8,3 = 1 ...
2
...
8
k
4
3
...
X4,1 = 0
level 2
X1,2 = 0
X1,2 = 1 X2,8 = 0 X1,2 = 1
X3,4 = 0 X1,2 = 1
X2,8 = 1 X3,4 = 1
X2,8 = 1
k1
k2
k3
k4
29
BRANCHANDBOUND ALGORITHM by CarpanetoDell’AmicoT. (ACM TOMS, 1995) • At the root node of the decision tree:  solve the AP Relaxation;  apply the Patch Heuristic Algorithm (KarpSteele, 1985) to determine an initial tour of cost Z*;  apply a Reduction Procedure (based on the AP “reduced costs” c’ij) to fix to zero as many variables as possible (transformation of the complete graph into a sparse one): if v(AP) + c’ij ≥ Z* set xij = 0 (i.e. remove arc (i,j) from A). • At each node, the corresponding AP Relaxation is computed (by using effective parametric techniques) in O(n2) time. 30
BRANCHANDCUT ALGORITHMS (Padberg  Rinaldi for STSP, 19871991) • At each node, a Lower Bound is obtained by solving an LP Relaxation of the corresponding subproblem, containing only a subset of the constraints (degree constraints, some connectivity constraints, …). • The LP Relaxation is iteratively tightened by adding valid inequalities that are violated by the current optimal LP solution (x*ij ). • These inequalities are identified by solving the Separation Problem:  given a solution (x*ij ), find a member a x ≤ b of a given family F of valid inequalities for ATSP, such that
a x* > b holds ( maximum of d = a x*  b, with d > 0 ) . * Exact or heuristic Separation Procedures can be used. 31
SEPARATION PROBLEM FOR THE CONNECTIVITY CONSTRAINTS • Given an optimal LP solution (x*ij ) such that:
Σ x*ih = Σ x*hj
i∈V
i∈V
=1
h∈V
does exist a vertex subset S ( S ⊂ V, r ∈ S) such that:
ΣΣ i∈S j∈V\S
x*ij < 1
?
32
EXACT SEPARATION PROCEDURE FOR THE CONNECTIVITY CONSTRAINTS • For each vertex t ⊂ V \ {r}, define the NETWORK G* = (V,A), source = r, sink = t capacity of arc (i,j) = x*ij • CAPACITY of “cut” (S, V \ S) with r∈S and t∈ V \ S =
Σ Σ i∈S
x*ij
j∈V\S
33
Example NETWORK G* = (V,A)
S r
1
6
7
7
1
1
1
3
1
t
5
2
1
6
9
Only degree constraints imposed
1
3
capacity (x*)
2
4
7
3 8
5
9
1
1
1
5
3
6
CAPACITY of “cut” (S, V \ S) = 0 34
EXACT SEPARATION PROCEDURE FOR THE CONNECTIVITY CONSTRAINTS
• Minimum Capacity Cut from r to t = Maximum Flow from r to t • Determine the MAXIMUM FLOW from r to t: capacity of cut (S*, V \ S*) * If Maximum Flow < 1 then constraint
Σ Σ i∈S*
xij ≥ 1 is violated
j∈V\S*
(most violated connectivity constraint with r∈S and t∈ V \ S)
O(n3) time for each vertex t
(global time O(n4) ) 35
Example NETWORK G* = (V,A)
capacity (x*)
S* 1
1
6
7
3
r
7
6
11
1
5
2
1
2
3
1
t
4
7
13
9 8 5
11 5
9 3
6
Minimum Capacity Cut from r to t = (S*, V\S*) x =0 Maximum Flow from r to t = 0: Σ Σ i∈S* j∈V\S*
ij
36
BRANCHANDCUT ALGORITHM (FischettiT., Man. SC. 1997) • Separation procedures for the identification of the following valid inequalities:  Connectivity constraints
(exact alg.);
 Comb inequalities
(heur. alg.);
 Dk+ and Dk inequalities
(GroetschelPadberg, 1985) (exact and heur. alg.);
 ODD CAT inequalities (Balas, 1989)
(heur. alg.) 37
PRICING PROCEDURE A pricing procedure is applied to decrease the number of variables considered in the LP Relaxation: 1) 2) 3) 4)
Select a subset T of variables (other variables set to zero); Solve the LP Relaxation by considering only the variables in T; Compute the reduced cost of the variables not in T; Add to T (a subset of) the variables with negative reduced cost and return to Step 2); if no negative reduced cost is found then STOP (the current solution is the optimal solution of the original LP Relaxation).
• An effective Pricing Scheme, based on the optimal solution of an associated Assignment Problem, is used to decrease the number of variables added to T in Step 4). 38
BRANCHING SCHEME
(FischettiLodiT., 2003)
• Select a fractional variable x*ij close to 0.5 and having been “persistently” fractional in the “last” optimal LP solutions. • Generate 2 descendent nodes by imposing:
xij =1 and xij =0, respectively.
39
TRANSFORMATION OF ATSP INTO STSP Any ATSP instance with n vertices can be transformed into an equivalent STSP instance with 2n nodes (JonkerVolgenant, 1983; JungerReineltRinaldi, 1995; KumarLi, 2007). Very effective BranchandCut algorithms for STSP have been proposed:  PadbergRinaldi (Oper. Res. Letters 1987, SIAM Review 1991)  GroetschelHolland (Math. Progr. 1991)  ApplegateBixbyChvatalCook (Doc. Math., J. DMV 1998, Concorde Code 1999, Princeton Univ. Press 2006).
* Concorde Code Most effective code for STSP 40
TRANSFORMATION OF ATSP INTO STSP Given a directed graph G = (V,A) with n vertices and m arcs, build an undirected graph G` = (V`,E) with 2n nodes and (m + n) edges:  for each vertex i ∈ V consider two nodes : i and (n + i);  for each vertex i ∈ V consider an edge (i, n + i) with cost 0;  for each arc (i, j) ∈ A consider an edge (n + i, j) with cost cij + M where M is a “sufficiently” large positive value
Cost of ATSP = Cost of STSP  n M 41
TRANSFORMATION OF ATSP INTO STSP 6 arc (i, j): edge (n + i, j) 0
2 3 1
9
5 0
4
8 10 5
n=4
8 8+M
0 4
1 1
3
2
3+M
4
4+M
9+M 5+M
1+M 7 0
10+M 3 42
TRANSFORMATION OF ATSP INTO STSP 6 0x
2 3x 1
9
x
8 5
x 3
2
3+M
4
x
5 0x
4
10
x1
Optimal solution Cost = 21
4+M
x
8+M
x0 4
1
x1+M
9+M
x
8
5+M
7 0x 10+M 3
Cost = 21 + 4M 43