Exact Algorithms and Experiments for Hierarchical Tree

Exact Algorithms and Experiments for Hierarchical Tree Clustering Jiong Guo Sepp Hartungy Christian Komusiewiczz Rolf Niedermeier Johannes Uhlmannx Un...

0 downloads 134 Views 181KB Size
Exact Algorithms and Experiments for Hierarchical Tree Clustering Jiong Guo∗

Sepp Hartung†

Christian Komusiewicz‡

Rolf Niedermeier

Johannes Uhlmann§ Universit¨at des Saarlandes, Campus E 1.4, D-66123 Saarbr¨ucken, Germany [email protected] Institut f¨ur Informatik, Friedrich-Schiller-Universit¨at Jena, Ernst-Abbe-Platz 2, D-07743 Jena, Germany {sepp.hartung,c.komus,rolf.niedermeier,johannes.uhlmann}@uni-jena.de

Abstract We perform new theoretical as well as first-time experimental studies for the NP-hard problem to find a closest ultrametric for given dissimilarity data on pairs. This is a central problem in the area of hierarchical clustering, where so far only polynomial-time approximation algorithms were known. In contrast, we develop efficient preprocessing algorithms (known as kernelization in parameterized algorithmics) with provable performance guarantees and a simple search tree algorithm. These are used to find optimal solutions. Our experiments with synthetic and biological data show the effectiveness of our algorithms and demonstrate that an approximation algorithm due to Ailon and Charikar [FOCS 2005] often gives (almost) optimal solutions.

1. Introduction Hierarchical representations of data play an important role in biology, the social sciences, and statistics [9, 10, 2, 6]. The basic idea behind hierarchical clustering is to obtain a recursive partitioning of the input data in a tree-like fashion such that the leaves one-to-one represent the single items and all inner points represent clusters of various granularity degrees. Hierarchical clusterings do not require a prior specification of the number of clusters and they allow to understand the data at many levels of fine-grainedness (the root of the tree representing the whole data set). We contribute new theoretical and experimental results for a well-studied NP-hard problem in this context, which is called M-Hierarchical Tree Clustering. The essential point of our work is that we can efficiently find provably optimal (not only approximate) solutions in cases of practical interest. Hierarchical Tree Clustering. Let X be the input set of elements to be clustered. The dissimilarity of the elements ∗ Partially supported by the DFG, research project DARE, GU 1023/1, and the DFG cluster of excellence “Multimodal Computing and Interaction”. † Partially supported by the DFG, research project DARE, GU 1023/1. ‡ Supported by a PhD fellowship of the Carl-Zeiss-Stiftung. § Supported by the DFG, research project PABI, NI 369/7.

is expressed by a positive-definite symmetric function D : X × X → {0, . . . , M + 1}, briefly called distance function. Herein, the constant M ∈ N specifies the depth of the clustering tree to be computed. We focus on the case to find a closest ultrametric that fits the given data. Definition 1. A distance function D : X × X → {0, . . . , M + 1} is called ultrametric if for all i, j, l ∈ X the following condition holds: D(i, j) ≤ max{D(i, l), D( j, l)}. The central M-Hierarchical Tree Clustering problem1 can be formulated as follows: Input: A set X of elements, a distance function D : X × X → {0, . . . , M + 1}, and k ≥ 0. Question: Is there a distance function D0 : X × X → {0, . . . , M + 1} such that D0 is an ultrametric and ||D − D0 ||1 ≤ k? P Herein, ||D − D0 ||1 := {i, j}⊆X |D0 (i, j) − D(i, j)|. In other words, given any distance function D, the goal is to modify D as little as possible to obtain an ultrametric D0 . An ultrametric one-to-one corresponds to a rooted depth-(M + 1) tree where all leaves have distance exactly M + 1 to the root and they are bijectively labeled with the elements of X [2]. This problem is closely related to the reconstruction of phylogenetic trees [7, 2]. 1-Hierarchical Tree Clustering is the same as the Correlation Clustering problem on complete graphs [4, 3], also known as Cluster Editing [8, 5]. Related Work. M-Hierarchical Tree Clustering is NPcomplete [10] and APX-hard [1], excluding any hope for polynomial-time approximation schemes. Ailon and Charikar [2] presented a randomized polynomial-time combinatorial algorithm for M-Hierarchical Tree Clustering that achieves an approximation ratio of M + 2. Moreover, there is a deterministic algorithm achieving the same approximation guarantee [13]. Numerous papers deal with 1-Hierarchical Tree Clustering and its approximability [3, 13] or fixed-parameter tractability [8, 5]. In particular, there have been encouraging experimental studies 1 Our algorithms also deal with the practically more relevant optimization problem where one wants to minimize the “perturbation value” k.

2. A Simple Search Tree Strategy The following search tree strategy solves M-Hierarchical Tree Clustering: As long as D is not an ultrametric, search for a conflict, and branch into the three cases (either decrease the max-distance pair or increase one of the other two distances) to resolve this conflict by changing the pairwise distances. Solve each branch recursively for k − 1.

based on fixed-parameter algorithms [12, 5]. In the area of phylogeny reconstruction, M-Hierarchical Tree Clustering is known as “Fitting Ultrametrics under the `1 norm”. Our Results. On the theoretical side, we provide polynomial-time preprocessing algorithms with provable performance guarantees (in the field of parameterized complexity analysis [11] known as “kernelization”). More precisely, we develop efficient data reduction rules that provably transform an original input instance of MHierarchical Tree Clustering into an equivalent instance consisting of only O(k2 ) elements or O(M · k) elements, respectively. Moreover, a straightforward exact algorithm based on a size-O(3k ) search tree is presented. On the practical side, we contribute implementations and experiments for our new data reduction rules (combined with the search tree strategy) and the known approximation algorithm: First, with our exact algorithms, we can solve a large fraction of non-trivial problem instances. Second, we observe that Ailon and Charikar’s algorithm [2] often yields optimal results. Basic Notation. Throughout this work let n := |X|. A conflict is a triple {i, j, l} of elements from the data set X that does not fulfill the condition of Definition 1. A pair {i, j} is the max-distance pair of a conflict {i, j, l} if D(i, j) > max{D(i, l), D( j, l)}. For Y ⊆ X the restriction of D to Y is denoted by D[Y] and is called the distance function induced by Y. For some of our data reduction rules we use notation from graph theory. We only consider undirected, simple graphs G = (V, E), where V is the vertex set and E ⊆ {{u, v} | u, v ∈ V}. The (open) neighborhood NG (v) of a vertex v is the set of vertices that are adjacent to v, and the closed neighborhood is defined as NG [v] S := NG (v) ∪ {v}. For a vertex set S ⊆ V, let NG (S ) := v∈S NG (v) \ S and NG [S ] := S ∪ NG (S ). With NG2 (S ) := NG (NG (S )) \ NG [S ] we denote the second neighborhood of a vertex set S . Fixed-Parameter Tractability and Kernelization. Parameterized algorithmics is a two-dimensional framework for studying the computational complexity of problems [11]. One dimension is the input size n (as in classical complexity theory), and the other one is the parameter k (usually a positive integer). A problem is called fixedparameter tractable if it can be solved in f (k) · nO(1) time, where f is a computable function only depending on k. A core tool in the development of fixed-parameter algorithms is problem kernelization, which can be viewed as polynomial-time preprocessing. Here, the goal is to transform a given problem instance x with parameter k by applying so-called data reduction rules into a new instance x0 with parameter k0 such that the size of x0 is upper-bounded by some function only depending on k, the instance (x, k) is a yes-instance if and only if (x0 , k0 ) is a yesinstance, and k0 ≤ k. The reduced instance, which must be computable in polynomial time, is called a problem kernel; the whole process is called kernelization.

Proposition 1. M-Hierarchical Tree Clustering can be solved in O(3k · n3 ) time. In the next section, we will show that by developing polynomial-time executable data reduction rules one can achieve that the above search tree no longer has to operate on sets X of size n but only needs to deal with sets X of size O(k2 ) or O(M · k), respectively. 3. Preprocessing and Data Reduction In this section, we present our main theoretical results, two kernelization algorithms for M-Hierarchical Tree Clustering. Both algorithms partition the input instance into small subinstances, and handle these subinstances independently. This partition is based on the following lemma. Lemma 1. Let D be a distance function over a set X. If there is a subset X 0 ⊆ X such that for each conflict C either C ⊆ X 0 or C ⊆ (X \ X 0 ), then there is a closest ultrametric D0 to D such that for each i ∈ X 0 and j ∈ X \ X 0 , D(i, j) = D0 (i, j). An O(k2 )-Element Problem Kernel. Our first and simpler kernelization algorithm uses two data reduction rules which handle two extremal cases concerning the elements: The first rule corrects the distance between two elements which together appear in many conflicts, while the second rule safely removes elements which are not in conflicts. Reduction Rule 1. If there is a pair {i, j} ⊆ X which is the max-distance pair (or not the max-distance pair) in at least k + 1 conflicts, then decrease (or increase) the distance D(i, j) and decrease the parameter k by one. Reduction Rule 2. Remove all elements which are not part of any conflict. Lemma 2. Rule 2 is correct. Proof. Let D be a distance function over a set X, and let x ∈ X be an element which is not part of any conflict. We show the correctness of Rule 2 by proving the following. Claim. (X, D) has an ultrametric D0 with ||D − D0 ||1 ≤ k iff (X \ {x}, D[X \ {x}]) has an ultrametric D00 with ||D[X \ {x}] − D00 ||1 ≤ k. Only the “⇐”-direction is non-trivial. The proof is organized as follows. First, we show that X \ {x} can be partitioned into M + 1 subsets X1 , . . . , X M+1 such that the maximum distance within each Xr is at most r. Then, we show that for each conflict {i, j, l} we have {i, j, l} ⊆ Xr for some r. Using these facts and Lemma 1, we then show

Several details and proofs are deferred to a full version of this article. 2

that there is a closest ultrametric D0 to D[X \ {x}] that only changes distances within the Xr and that “reinserting” x into D0 results in an ultrametric D00 within distance k to D. First, we show that there is a partition of X \ {x} into M + 1 subsets X1 , . . . , X M+1 such that the maximum distance within each Xr is at most r. For 1 ≤ r ≤ M + 1, let Xr := {y ∈ X | D(x, y) = r}. Clearly, this yields a partition of X \ {x}. Furthermore, the distance between two elements i, j ∈ Xr is at most r, because otherwise there would be a conflict {i, j, x} since D(i, x) = D( j, x) = r and D(i, j) > r. This, however, contradicts that x is not part of any conflict. Next, we show that for each conflict {i, j, l} all elements belong to the same Xr . Suppose towards a contradiction that this is not the case. Without loss of generality, assume that D(i, x) = r > D( j, x) and D(i, x) ≥ D(l, x). Since {i, j, x} is not a conflict, we have D(i, j) = r. We distinguish two cases for D(l, x): Case 1: D(l, x) = r. Since { j, l, x} is not a conflict, we also have D( j, l) = r. Then, since {i, j, l} is a conflict, we have D(i, l) > r. Since D(i, x) = D(l, x) = r the triple {i, l, x} is a conflict, a contradiction. Case 2: D(l, x) < r. Analogous to Case 1. Consequently, there are no conflicts that contain elements from different Xr ’s. Finally, let D0 be an ultrametric for D[X \ {x}]. Since there are no conflicts that contain elements from different Xr ’s and by Lemma 1, we can assume that D0 only modifies distances within the Xr and not between different Xr ’s. From D0 we can obtain a distance function D00 over X as follows: ( 0 D (i, j) i , x ∧ j , x, D00 (i, j) := D(i, j) otherwise.

of X into two subsets A and B, where A := {i ∈ X | ∃ j ∈ X : D0 (i, j) , D(i, j)} and B := X \ A. The elements in A are called affected and the elements in B are called unaffected. Note that |A| ≤ 2k since D0 has distance at most k to D. Hence, it remains to show that |B| ≤ k2 . Let S := {{i, j} ⊆ X | D0 (i, j) , D(i, j)} denote the set of pairs whose distances have been modified, and for each {i, j} ∈ S let B{i, j} denote the elements of B that are in some conflict with {i, j}. Since the inputSinstance is reduced with respect to Rule 2 we have B = {i, j}∈S B{i, j} . Furthermore, since the input instance is reduced with respect to Rule 1, we have |B{i, j} | ≤ k for all {i, j} ∈ S . The size bound |B| ≤ k2 then immediately follows from |S | ≤ k. The running time can be seen as follows. First, we calculate for each pair of elements the number of conflicts in which it is the max-distance pair and the number of conflicts in which it is not the max-distance pair in O(n3 ) time. Then we check whether Rule 1 can be applied. If this is the case, we update the number of conflicts for all pairs that contain at least one of the elements whose distance has been modified in O(n) time. This is repeated as long as a pair to which the rule can be applied has been found, at most O(M · n2 ) times. Hence, the overall running time of exhaustively applying Rule 1 is O(M · n3 ). Afterwards, we exhaustively apply Rule 2 in O(n3 ) time overall. 2

That is, D00 is obtained from D0 by reinserting x and the original distances from x to X \ {x}. Clearly, ||D − D00 ||1 ≤ k. It thus remains to show that D00 is an ultrametric. Since D0 is an ultrametric, it suffices to show that there are no conflicts containing x. In D0 the distance between two elements i ∈ Xr and j ∈ Xr is at most r since this is the maximum distance in D[Xr ] and this maximum distance will clearly not be increased by a closest ultrametric. Hence, there can be no conflicts in D00 containing x and two vertices from the same Xr . There also can be no conflicts containing x, an element i ∈ Xr , and some other element j ∈ Xr0 because the distances between these elements have not been changed from D to D00 , and these elements were not in conflict in D. Hence, D00 is an ultrametric. 2

An O(M · k)-Element Problem Kernel. Our second kernelization algorithm extends the basic idea of an O(k)element problem kernel for Cluster Editing [8]. Consider a distance function D on X := {1, . . . , n}. For an integer t with 1 ≤ t ≤ M, the t-threshold graph Gt is defined as (X, Et ) with {i, j} ∈ Et if and only if D(i, j) ≤ t. If D is an ultrametric, then, for all 1 ≤ t ≤ M, the corresponding graph Gt is a disjoint union of cliques. We call each of these cliques a t-cluster. Recall that a clique is a set of pairwisely adjacent vertices. A clique K is a critical clique if all its vertices have an identical closed neighborhood and K is maximal under this property. The kernelization algorithm employs Rule 2 and one further data reduction rule. This new rule works on the t-threshold graphs Gt , beginning with t = M until t = 1. In each Gt , this rule applies a procedure which deals with large critical cliques in Gt :

Using the standard technique of interleaving search trees with kernelization [11], one can improve the worstcase running time of the search tree algorithm from Section 2. As our experiments show (see Section 4), there is also a speed-up in practice. Corollary 1. M-Hierarchical Tree Clustering can be solved in O(3k + M · n3 ) time.

Theorem 1. M-Hierarchical Tree Clustering admits a problem kernel with k · (k + 2) elements. The running time for the kernelization is O(M · n3 ).

Procedure: Critical-Clique. Input: A set X 0 ⊆ {1, . . . , n} and an integer t. 1. Construct Gt for X 0 . 2. While Gt contains a non-isolated critical clique K with

Proof. Let I = (X, D, k) be an instance that is reduced with respect to Rules 1 and 2. Assume that I is a yesinstance, that is, there exists an ultrametric D0 on X with distance at most k to D. We show that |X| ≤ k · (k + 2). For the analysis of the kernel size we partition the elements

• |K| ≥ t · |NGt (K)| and • |K| ≥ |NGt (K)| + t · |NG2 t (K)|, do 3

3. For all x ∈ NGt [K] and y ∈ X 0 \ (NGt [K]), set D(x, y) := t + 1, and for all x, y ∈ NGt (K) with D(x, y) = t + 1, set D(x, y) := t. 4. Update Gt . 5. Decrease the parameter k correspondingly, that is, by the distance between the original and new instances. 6. If k < 0 then return “no”. 7. End while

“⇐”: After the application of Critical-Clique NG M [K] is an isolated clique in the M-threshold graph for D0 . That is, D0 (i, j) = M + 1 all i ∈ NG M [K] and j ∈ X \ NG M [K] and D0 (i, j) ≤ M for all i, j ∈ NG M [K], which implies that there is no conflict with vertices in NG M [K] and X \ NG M [K]. Let U denote an ultrametric with minimum distance to D0 . By Lemma 1, we can assume that U(i, j) = M + 1 for all i ∈ NG M [K] and j ∈ X \ NG M [K] and U(i, j) ≤ M for all i, j ∈ NG M [K]. Hence, the distance of U to D is at most k. 2

Reduction Rule 3. Recursively apply the Critical-Clique procedure to the t-threshold graphs Gt from t = M to t = 1 by calling the following procedure with parameters X and M.

Theorem 2. M-Hierarchical Tree Clustering admits a problem kernel with 2k · (M + 2) elements. The running time for the kernelization is O(M · n3 ).

Procedure: RR3 Input: A set X 0 ⊆ {1, . . . , n} and an integer t. Global variables: A distance function D on X = {1, . . . , n} and an integer k. 1. Critical-Clique(X 0 , t); 2. For each isolated clique K in Gt that does not induce an ultrametric do RR3(K, t − 1).


Experimental Results

Implementation Details. We briefly describe some notable differences between the theoretical algorithms from Sections 2 and 3 and their actual implementation.2 Main algorithm loop: We call the search tree algorithm (see Section 2) with increasing k, starting with k = 1 and aborting when an (optimal) solution has been found. Data reduction rules: We implemented all of the presented data reduction rules. However, in preliminary experiments, Rule 3 showed to be relatively slow and was thus deactivated.3 Interleaving: In the search tree we interleave branching with the application of the data reduction rules, that is, after a suitable number of branching steps the data reduction rules are invoked. In the experiments described below we performed data reduction in every second step, since this value yielded the largest speed-up. Modification flags: We use flags to mark distances that may not be decreased (or increased) anymore. There are three reasons for setting such a mark: the distance has been already increased (or decreased); decreasing (or increasing) it leads to a solution with distance more than k; decreasing (or increasing) it leads to a conflict that cannot be repaired without violating previous flags. Choice of conflicts for branching: We choose the conflict to branch on in the following order of preference: First, we choose conflicts where either both non-max-distance pairs cannot be increased or the max-distance pair cannot be decreased and one non-max-distance pair cannot be increased. In this case, no actual branching takes place since only one option to destroy the conflict remains. Second, if no such conflicts exist we choose conflicts where the max-distance pair cannot be decreased or one of the non-max-distance pairs cannot be increased. If these conflicts are also not present, we choose the smallest conflict with respect to a predetermined lexicographic order. This often creates a conflict of the first two types. We also implemented the randomized (M + 2)-factor approximation algorithm by Ailon and Charikar [2]. In our

In the following, we show that the Critical-Clique procedure is correct, that is, an instance (X, D, k) has a solution if and only if the instance (X, D0 , k0 ) resulting by one application of Critical-Clique has a solution. Then, the correctness of Rule 3 follows from the observation that every call of RR3 is on a subset K and an integer t such that K is an isolated clique in Gt+1 . Then, K can be solved independently from X \ K: Since the elements in K have distance at least t + 1 to all elements in X \ K, there is no conflict that contains vertices from both K and X \ K. By Lemma 1, we thus do not need to change the distance between an element in K and an element in X \ K. For the correctness of Critical-Clique, we consider only the case t = M; for other values of t the proof works similarly. The following lemma is essential for our proof. Lemma 3. Let K be a critical clique in G M with • |K| ≥ M · |NG M (K)| and • |K| ≥ |NG M (K)| + M · |NG2 M (K)|. Then, there exists a closest ultrametric U for D such that NG M [K] is an M-cluster in U. Lemma 4. The Critical-Clique procedure is correct. Proof. Let D denote a distance function and let K denote a critical clique of G M fulfilling the while-condition (line 2) in the Critical-Clique procedure. Furthermore, let D0 denote the distance function that results by executing lines 3-5 of Critical-Clique on K and let d := ||D − D0 ||1 . To show the correctness of Critical-Clique it suffices to show that (X, D, k) is a yes-instance if and only if (X, D0 , k − d) is a yes-instance. “⇒”: If (X, D, k) is a yes-instance, then, by Lemma 3, there exists an ultrametric U of distance at most k to D such that NG M [K] is an M-cluster of U. Hence, it must hold that U(i, j) = M + 1 for all i ∈ NG M [K] and j ∈ X \ NG M [K] and U(i, j) ≤ M for all i, j ∈ NG M [K]. Hence, the changes performed by Critical-Clique are necessary to obtain U.

2 The Java program is free software and available from http://theinf1.informatik.uni-jena.de/tree-cluster 3 For some larger instances, however, the rule is very effective because it reduces the search tree size by up to 33%.


experiments, we repeated the algorithm 1000 times and compared the best ultrametric that was found during these trials with the exact solution found by our algorithm. Experiments were run on an AMD Athlon 64 3700+ machine with 2.2 GHz, 1 M L2 cache, and 3 GB main memory running under the Debian GNU/Linux 5.0 operating system with Java version 1.6.0 12. Synthetic Data. We generate random instances to chart the border of tractability with respect to different values of n and k. We perform two studies, considering varying k for fixed values of n and considering varying n for fixed values of k. In the experiments either M = 2 or M = 4. For each value of n we generate five ultrametrics and perturb each of these instances, increasing step by step the number of perturbations k. For each pair of n and k we generate five distance functions. We thus create 25 instances for each pair of n and k. Next, we describe in detail how we generate and disturb the ultrametrics. Generation of Ultrametrics. We generate the instances by creating a random ultrametric tree of depth M + 1. We start at the root and randomly draw the number of its children under uniform distribution from {2, . . . , dln ne}. Then, the elements are randomly (again under uniform distribution) assigned to the subtrees rooted at these newly created nodes. For each child we recursively create ultrametric trees of depth M. The only difference for a node at a lower level is that we randomly draw the number of its children under uniform distribution from {1, . . . , dln ne}. That is, in contrast to the root node, we allow that an inner node has only one child. Perturbation of Generated Ultrametrics. We randomly choose a pair {i, j} of elements under uniform distribution and change the distance value D(i, j). This step is repeated until k distance values have been changed. For each chosen pair, we randomly decide whether D(i, j) will be increased or decreased (each with probability 1/2). We do not increase D(i, j) if it has been previously decreased or if D(i, j) = M + 1; we do not decrease D(i, j) if it has been previously increased or if D(i, j) = 1. Note that with this approach a generated instance may have a solutions that has distance < k to the input distance function. Experiments with fixed n. First, we study the effect of varying k for fixed values of n. As to be expected by the theoretical analysis, the running time increases for increasing k. Figure 1 shows the running times of the instances that could be solved within 5 minutes. The combinatorial explosion that is common to exponentialtime algorithms such as fixed-parameter algorithms sets in at k ≈ n. This is due to the fact that most instances with k < n could be solved without branching, just by applying the data reduction rules. Regression analysis shows that the running time is best described by exponential functions of the type αk with α ≤ 1.4. This is due to the data reduction: switching it off leads to running times with α ≈ 2.4. Experiments with fixed k. We study the effect of different input sizes n for k = 50 and k = 100, with n ≥ 10. The results are shown in Figure 2. Roughly speaking, the instances are difficult to solve when k > n. Again, this


n=50, M+1=3 n=50, M+1=5 n=100, M+1=3 n=100, M+1=5


time (s)

200 150 100 50 0 30




70 80 90 parameter k




Figure 1: Running times for fixed n and varying k. k=50, M+1=3 k=100, M+1=3

140 120

time (s)

100 80 60 40 20 0 50


150 set size n



Figure 2: Running times for fixed k and varying n.

behavior can be attributed to the data reduction rules that are very effective for k < n. Protein Similarity Data. We perform experiments on protein similarity data which have been previously used in experimental evaluation of fixed-parameter algorithms for Cluster Editing [12]. The data set contains 3964 files with pairwise similarity data of sets of proteins. The number of proteins n for each file ranges from 3 to 8836. We consider a subset of these files, where n ≤ 60, covering about 90% of the files. From each file, we create four discrete distance matrices for M = 2 as follows. We set the distance of the c% of the pairs with lowest similarity to 3, where c is a predetermined constant. From the remaining pairs the c% of the pairs with lowest similarity are set to 2, and all others to 1. In our experiments, we set c to 75, 66, 50, and 33. This approach is motivated by the following considerations. In a distance function represented by a balanced ultrametric tree of depth M + 1 at least half of all distances are M + 1 and with increasing degree of the root of the clustering tree 5

Table 1: Summary of our experiments for the protein similarity data. The second column contains the number of instances within the respective range. The next four columns provide the percentage of instances that can be solved within 2, 10, 60, ST and 600 seconds by our search tree algorithm. Further, kmS T denotes the maximum, and kavg the average distance to a closest AP ultrametric. For the approximation algorithm, kavg denotes the maximum distance and %ex is the percentage of instances which were solved optimal. Finally, d denotes the maximum difference between the distances found by the two algorithms. c = 75 range # 2s 10s n ≤ 20 2806 100 100 20 < n ≤ 40 486 63 72 40 < n ≤ 60 298 4.7 8.1 n ≤ 60 3590 87 89 c = 50 range # 2s 10s n ≤ 20 2806 97 98 20 < n ≤ 40 486 18 25 40 < n ≤ 60 298 0 1.3 n ≤ 60 3590 78 81

60s 600s kmS T 100 100 23 79 89 69 13 22 84 90 92 84 60s 600s kmS T 99 100 65 38 50 82 1.7 3.4 85 83 85 85

c = 66 ST AP d kavg kavg %ex 2s 10s 60s 600s kmS T d 3 2.4 2.4 98 99 100 100 100 35 5 6 25.7 26.2 78 57 67 74 82 74 7 6 48.3 48.8 76 2 4 8 16 76 4 6 6.4 6.5 95 86 88 89 91 76 7 c = 33 ST AP d kavg kavg %ex 2s 10s 60s 600s kmS T d 10 6.8 7 88 94 96 98 99 73 17 18 46 47 59 7.4 13 20 31 82 22 0 62 62 100 0 0 0 0 / / 18 10.1 10.4 86 74 77 79 82 82 22

the number of pairs with distance of M + 1 increases. If we assume that the ultrametric tree is more or less balanced we thus expect a large portion of pairwise distances to have maximum value making the choices of c = 75 and c = 66 the most realistic. We summarize our experimental findings (see Table 1) on these data as follows. First, our algorithm solves instances with n ≤ 20 in few seconds. Second, for n ≤ 40, many instances can be solved within 10 minutes. Third, using our exact algorithms, we can show that the approximation algorithm often yields almost optimal solutions. Finally, for decreasing values of c and increasing instance sizes the solution sizes and, hence, the running times increase. Interestingly, the approximation quality decreases simultaneously. Altogether, we conclude that both our new exact and the known approximation algorithm are useful for a significant range of practical instances.

ST kavg 2.8 27.1 53.6 6.52

AP kavg %ex 2.8 96 27.5 76 53.9 80 6.62 94

ST AP kavg kavg %ex 9.4 9.8 83 53 55 65 / / / 11.6 12.1 82

of their solutions. For instance our experiments indicate that the solution quality of the approximation algorithm [2] gets worse with growing input sizes. References [1] Agarwala, R.; Bafna, V.; Farach, M.; Narayanan, B.; Paterson, M.; and Thorup, M. 1999. On the approximability of numerical taxonomy (fitting distances by tree matrices). SIAM J. Comput. 28(3):1073– 1085. on p. 1. [2] Ailon, N., and Charikar, M. 2005. Fitting tree metrics: Hierarchical clustering and phylogeny. In Proc. 46th FOCS, 73–82. IEEE Computer Society. on pp. 1, 2, 4, and 6. [3] Ailon, N.; Charikar, M.; and Newman, A. 2008. Aggregating inconsistent information: Ranking and clustering. J. ACM 55(5). on p. 1. [4] Bansal, N.; Blum, A.; and Chawla, S. 2004. Correlation clustering. Machine Learning 56(1–3):89–113. on p. 1. [5] B¨ocker, S.; Briesemeister, S.; and Klau, G. W. 2009. Exact algorithms for cluster editing: Evaluation and experiments. Algorithmica. To appear. on pp. 1 and 2. [6] Dasgupta, S., and Long, P. M. 2005. Performance guarantees for hierarchical clustering. J. Comput. Syst. Sci. 70(4):555–569. on p. 1. [7] Farach, M.; Kannan, S.; and Warnow, T. 1995. A robust model for finding optimal evolutionary trees. Algorithmica 13:155–179. on p. 1. [8] Guo, J. 2009. A more effective linear kernelization for Cluster Editing. Theor. Comput. Sci. 410(8-10):718–726. on pp. 1 and 3. [9] Hartigan, J. 1985. Statistical theory in clustering. J. Classifi. 2(1):63– 76. on p. 1. [10] Kˇriv´anek, M., and Mor´avek, J. 1986. NP-hard problems in hierarchical-tree clustering. Acta Informatica 23(3):311–323. on p. 1. [11] Niedermeier, R. 2006. Invitation to Fixed-Parameter Algorithms. Oxford University Press. on pp. 2 and 3. [12] Rahmann, S.; Wittkop, T.; Baumbach, J.; Martin, M.; Truß, A.; and B¨ocker, S. 2007. Exact and heuristic algorithms for weighted cluster editing. In Proc. 6th CSB, 391–401. Imperial College Press. on pp. 2 and 5. [13] van Zuylen, A., and Williamson, D. P. 2009. Deterministic pivoting algorithms for constrained ranking and clustering problems. Mathematics of Operations Research 34:594–620. on p. 1.

5. Conclusion Our polynomial-time executable data reduction rules shrink the original instance to a provably smaller, equivalent one. They can be used in combination with every solving strategy for M-Hierarchical Tree Clustering. For instance, we started to explore the efficacy of combining our data reduction with Ailon and Charikar’s approximation algorithm [2]. In case k < n almost all instances could be solved exactly by data reduction within a running time that is competitive with the approximation algorithm by Ailon and Charikar [2]. Obviously, although having proven usefulness by solving biological real-word data, the sizes of the instances we can typically solve exactly are admittedly relatively small (up to around 50 vertices). In case of larger instances, one approach could be to e.g. use the approximation algorithm to create small independent subinstances, where our algorithms apply. Finally, our algorithms also serve for “benchmarking” heuristic algorithms indicating the quality 6