COS 226 Lecture 18: MSTs and shortest paths %ps /lecture 18 def Classic algorithms for solving "network" problems MINIMAL SPANNING TREE Kruskal's algorithm Prim's algorithm SHORTEST PATHS Dijkstra's algorithm Floyd's algorithm Related to generalized graph search using a priority queue Ex: telephone wiring Ex: supplies to troops ----- Generalized search (Priority-first) Use PQ, rather than stack or queue TREE vertices visited FRINGE vertices adjacent to visited UNSEEN vertices others Put all fringe vertices on PQ Search algorithm * take next vertex off PQ * change status to TREE * for each adjacent vertex UNSEEN: move to FRINGE FRINGE: update prioirity Encompasses several basic graph algorithms depth-first search (use decreasing counter for priority) breadth-first search (use increasing counter for priority) minimal spanning tree shortest path tree ----- Minimal spanning tree Weighted graph (edges have values) Ex: Vertices: points in space Edges: lines connecting them Weights: lengths of lines MST: minimal weight set of edges that connects all the vertices Theorem: Divide the vertices into two sets A and B. Shortest edge X connecting a vertex in A with a vertex in B must be in the MST Proof: by contradiction. If X not in MST, add it to MST * creates a cycle * some other edge Y from A to B must be on cyccle Delete Y and add X to get better tree. ----- sample graph a spanning tree minimal spanning tree %% 0 %ps 200 0 translate 1.5 1.5 scale %include figs/01intro/ps/ds.ps %include figs/18mstspt/mst.ps %%% ----- Priority-first search MST algorithm Set A: TREE Set B: UNSEEN+FRINGE Smallest edge in FRINGE must be in MST -- #define P t->w visit(link* adj, int k) { link t; dad[k] = k; PQreduce(k, 0); while (PQminkey() < maxNum) { k = PQdelmin(); printf("%d-%d\n", k, dad[k]); for (t = adj[k]; t != z; t = t->next) if (onPQ(t->v) && (P < val[t->v])) { PQreduce(t->v, P); dad[t->v] = k; } } } listpfs(link* adj, int V, int E) { int k; val = malloc(V*sizeof(weightType)); dad = malloc(V*sizeof(int)); PQinit(V); for (k = 0; k < V; k++) if (onPQ(k)) visit(adj, k); } --- /lines 35 def ----- MST example use line lengths as edge weights Edges Adjacency lists -- 0-6 5.3 0: 7 5 2 1 6 0-1 3.1 1: 7 0 0-2 3.6 2: 7 0 4-3 2.2 3: 5 4 5-3 2.2 4: 6 5 7 3 7-4 2.2 5: 0 4 3 5-4 3.1 6: 4 0 0-5 6.0 7: 1 2 0 4 6-4 3.6 7-0 3.6 7-2 1.4 7-1 1.0 --- %% 5 %ps 150 -18 translate 1.4 1.4 scale %include figs/01intro/ps/ds.ps %include figs/18mstspt/exTINY.ps %%% MST -- 1-0 7-1 2-7 4-7 3-4 5-3 6-4 --- %ps linesreset ----- PFS MST example -- . 0 1 2 3 4 5 6 7 min out updates . --------------------------------------- . 0 * * * * * * * 0 0-0 0-1 < 0-*, etc. . 0 0 * * 0 0 0 1 1-0 1-7 < 7-0 . 0 * * 0 0 1 7 7-1 7-2 < 2-0, 7-4 < 4-0 . 7 * 7 0 0 2 2-7 none . * 7 0 0 4 4-7 4-6 < 6-0, 4-5 < 5-0, 4-3 < 3-* . 4 4 4 3 3-4 3-5 < 5-4 . 3 4 5 5-3 none . 4 6 6-4 none --- %% 11 %ps -30 0 translate 1.2 1.2 scale %include figs/18mstspt/exPRIM.ps %%% ----- Priority queue issues Need to implement DELETE MINIMUM UPDATE Priority queue holds non-tree vertices * priority is shortest edge to tree vertex * for each vertex added to the tree each of its edges may give a shorter connection to a non-tree vertex Requires handles! given vertex, have to change its value Example: heap implementation priorities move around in heap INDIRECT heap: keep track of indices (see book for details) -- | 9.9 | 9.9 | 1.4 | 3.1 | 3.1 | 5.5 | 1.1 | 1.4 | 6.7 | 6.7 | 4.2 | 4.2 | 5.5 | --- Another approach INDEXED TOURNAMENT Separate array on top of data keep track of winners do not move data itself ----- Indexed tournament implementation -- int *pq; int pqV; void PQinit(int V) { int i; pqV = V; pq = malloc((V+V)*sizeof(link)); for (i = 0; i < V; i++) val[i] = maxNum; for (i = 0; i < V; i++) pq[V+i] = i; for (i = V-1; i > 0; i--) pq[i] = pq[i+i]; } int onPQ(int v) { return val[v] != maxNum+1; } numType PQminkey() { return val[pq[1]]; } void promote(int k) { int j, l, r; for (j = k/2; j > 0; j = j/2) if (val[pq[j+j]] < val[pq[j+j+1]]) pq[j] = pq[j+j]; else pq[j] = pq[j+j+1]; } int PQdelmin() { int k = pq[1]; val[k] = maxNum+1; promote(pqV+k); return k; } PQreduce(int v, numType x) { val[v] = x; promote(pqV+v); } --- /lines 56 def ----- Indexed tournament example -- | *** 0 | 0 | 0 | 1 | *** 1 | 3.1 1 | 0 | 1 | *** 2 | 3.6 2 | 2 | 2 | *** 3 | *** 3 | 0 | 1 | *** 4 | *** 4 | 4 | 5 | *** 5 | 6.0 5 | 4 | 7 | *** 6 | 5.3 6 | 6 | 7 | *** 7 | 3.6 7 | 0 | 0 | 1 | 0 | 1 | 1 | 2 | 2 | 3.6 2 | 1.4 2 | 2 | 2 | *** 3 | *** 3 | 7 | 2 | *** 4 | 2.2 4 | 5 | 4 | 6.0 5 | 6.0 5 | 7 | 4 | 5.3 6 | 5.3 6 | 7 | 6 | 1.0 7 | 7 | 0 | 0 | 0 | 0 | 1 | 1 | 3 | 3 | 2 | 2 | 3 | 3 | *** 3 | 2.2 3 | 4 | 3 | 2.2 4 | 4 | 4 | 4 | 6.0 5 | 3.1 5 | 4 | 4 | 5.3 6 | 3.6 6 | 6 | 6 | *** 7 | *** 7 --- %ps linesreset ----- Larger PFS MST example %% 15 %ps 0 0 translate 1.25 1.25 scale %include figs/18mstspt/pfsMST.ps %%% %% 15 %ps 300 200 translate 1.25 1.25 scale %include figs/18mstspt/pfsMSTint.ps %%% ----- Prim's algorithm Use adjacency matrix While going through the matrix to find the vertices connected to the current one, find the minimum at the same time val[k] neg: FRINGE; pos: TREE; -unseen: UNSEEN -- matrixpfs() { int k, t, min = 0; for (k = 1; k <= V; k++) { val[k] = -unseen; dad[k] = 0; } val[0] = -(unseen+1); // sentinel for (k = 1; k != 0; k = min, min = 0) { val[k] = -val[k]; if (val[k] == unseen) val[k] = 0; for (t = 1; t <= V; t++) if (val[t] < 0) { if (a[k][t] && (val[t] < -priority)) { val[t] = -priority; dad[t] = k; } if (val[t] > val[min]) min = t; } } } --- Invented in 1957 ----- Prim's algorithm vs. PFS Depends on whether graph is DENSE or SPARSE To within a constant factor PFS: (E + V) log V Prim's V*V Ex: 1 million vertices, 2 million edges PFS: 60 million steps Prims's: 1 billion steps Ex: 1 million vertices, 800 million edges PFS: 16 billion steps Prim's: 1 billion steps Clever algorithms can reduce PFS to E + V log V Bottom line: * Prim's optimal for dense graphs * don't use Prim's for sparse graphs (maybe couldn't anyway, because of space) ----- Kruskal's algorithm UF algorithms from lecture 1 find a spanning tree THEOREM: If the edges come in order of their length, the spanning tree from UF algs is a MST! Algorithm: * sort the edges * use weighted quick union In Postscript: -- /kruskal { ufwinit 0 2 E E add 1 sub { /i exch def edges i get /v1 exch def edges i 1 add get /v2 exch def v1 v2 2 copy ufw { pop pop }{ drawedge } ifelse } for } def --- Invented in 1956. ----- Kruskal MST example Take edges in order, skip those that make cycles -- 7-1 1.0 * 7-2 1.4 * 4-3 2.2 * 5-3 2.2 * 7-4 2.2 * 5-4 3.1 0-1 3.1 * 0-2 3.6 6-4 3.6 * 7-0 3.6 0-6 5.3 0-5 6.0 --- %% 11 %ps -30 0 translate 1.25 1.25 scale %include figs/18mstspt/exKRUSKAL.ps %%% ----- Larger Kruskal example %% 22 %ps 0 0 translate 1.5 1.5 scale %include figs/01intro/ps/ds.ps %include figs/18mstspt/kruskalA.ps %%% %% 0 %ps 224 0 translate 1.5 1.5 scale %include figs/18mstspt/kruskalB.ps %%% ----- Kruskal's algorithm vs PFS Depends on structure of graph To within a constant factor PFS: E log V + V log V fancy PFS: E + V log V Kruskal: E + M log E where M is the number edges in the graph shorter than the longest edge in the MST Do Kruskal with heap built from the bottom up otherwise E log E for the sort Performance differences depend upon: * dense vs. sparse * structure of graph * priority queue algorithm * priority queue implementation All require space for whole graph (UF finds a spanning tree in O(V) space) MST for huge graphs?? ----- Shortest paths SINGLE-SOURCE shortest paths: want shortest path from x to each other node Algorithm: PFS starting at x val[k]: shortest distance from x to k priority on FRINGE: shortest known distance to node smallest fringe value: shortest distance to *any* node not on tree (won't find a shorter path) -- #define priority val[k]+t->w --- DFS and BFS also immediate from PFS BFS: priority is "time since start" DFS: priority is "time to finish" ----- PFS SPT example %% 15 %ps 0 0 translate 1.25 1.25 scale %include figs/18mstspt/pfsSPT.ps %%% %% 15 %ps 300 200 translate 1.25 1.25 scale %include figs/18mstspt/pfsSPTint.ps %%% ----- PFS BFS and DFS examples %% 24 %ps 50 0 translate 1.25 1.25 scale %include figs/18mstspt/pfsBFSDFSint.ps %%% ----- Shortest paths in Euclidean graphs Problem: find shortest path from x to y Algorithm: start shortest-path PFS at x stop when reaching y val[k]: shortest distance from x to k in TREE PLUS distance from k to y as the crow flies priority on FRINGE: shortest possible way from x to y -- #define priority val[k]+t->w + distance(t_>w, y) --- ----- All shortest paths Problem: Given weighted graph, compute a table giving the shortest weighted path from each node to each other node Ex: map of New England nodes: cities edges: roads connecting cities -- . P W L N . Providence 0 53 54 48 . Westerley 53 0 18 101 . newLondon 54 18 0 12 . Norwich 48 101 12 0 --- Norwich-Westerly: 101 miles?? 12 miles Norwich-New London 18 miles New London-Westerly --------- 30 miles ----- Floyd-Warshall algorithm Another ancient algorithm (1962) Want shorter path from x to j? take x to y, then y to j, if shorter For each x-y pair, do all j -- for (y = 1; y <= V; y++) for (x = 1; x <= V; x++) for (j = 1; j <= V; j++) if (a[x][y] + a[y][j] < a[x][j]) a[x][j] = a[x][y] + a[y][j]; --- Correctness proof: induction on y Running time: O(V^3) Negative edge weights? * much more difficult to solve * could get negative cycles