# How to Choose Interpolation Data in Images

data sets are identical to sets that can also be motivated using level set ideas and approximation theoretic considerations. Numerical computations ar...

How to Choose Interpolation Data in Images Zakaria BELHACHMI Dorin BUCUR Bernhard BURGETH Joachim WEICKERT Abstract We introduce and discuss shape based models for finding the best interpolation data when reconstructing missing regions in images by means of solving the Laplace equation. The shape analysis is done in the framework of Γ-convergence, from two different points of view. First, we propose a continuous PDE model and get pointwise information on the ”importance” of each pixel by a topological asymptotic method. Second, we introduce a finite dimensional setting into the continuous model based on fat pixels (balls with positive radius), and study by Γ-convergence the asymptotics when the radius vanishes. In this way, we obtain relevant information about the optimal distribution of the best interpolation pixels. We show that the resulting optimal data sets are identical to sets that can also be motivated using level set ideas and approximation theoretic considerations. Numerical computations are presented that confirm the usefulness of our theoretical findings for PDE-based image compression.

Keywords: γ-convergence, shape analysis, image interpolation, image compression

1

Introduction

In the last decade partial differential equations (PDEs) and variational techniques have been proposed for a number of interpolation problems in digital image analysis. Many of them deal with so-called inpainting problems [9, 10, 16, 32, 43, 49], where one aims at filling in missing informations in certain corrupted image areas by means of second or higher-order PDEs. To this end one regards the known image data as Dirichlet boundary conditions, and interpolates the unknown data in the inpainting regions by solving appropriate boundary value problems. Related variational and PDE methods have also been investigated for more classical interpolation problems such as zooming into an image by increasing its resolution [2, 5, 6, 15, 42, 46, 51]. Some other PDE-based interpolation strategies have been tailored to specific data sets such as level set representations for digital elevation maps [27, 48, 52]. Moreover, some variational L1 minimization ideas play an important role in recent compressed sensing concepts [13]. One of the biggest challenges for PDE-based interpolation in image analysis is image compression. While there are numerous publications that exploit the smoothing properties of PDEs as preor postprocessing tools in connection with well-established compression methods such as JPEG or wavelet thresholding, only a few attempts have been made to incorporate them actually in these methods [18, 41, 45, 53]. A more direct way, however, would be to design a pure PDE-based compression method that does not require being coupled to any existing codec. A tempting idea would be to store only a small amount of “important” pixels (say e.g. 10 %) and interpolate the others by suitable PDEs. This gives rise to two questions: 1. How can one find the most “important” pixels that give the best reconstructions? 1

2. What are the most suitable PDEs for this purpose? Intuitively one expects that one should choose more points in regions where the gray values fluctuate more rapidly, while the interpolation point density is supposed to be lower in slowly varying image areas. Galic et al. [30] have used a B-tree triangular coding strategy from [25] in combination with anisotropic PDEs of diffusion type. The B-tree triangular coding selects the interpolation points as vertices of an adaptive triangulation with a higher resolution in more fluctuating areas. Extensions to image sequences have been considered by K¨ostler et al. [38]. Parallel to these adaptation strategies, some feature-based approaches have been explored. Chan and Shen [17] considered regions around image edges and used interpolating PDEs that penalize the total variation of the image. Methods of this type are close in spirit to earlier work on image reconstruction from edges [14, 26, 35, 54] or other feature points in Gaussian scale-space [36, 37, 40]. Zimmer [55] stored corner neighborhoods and reconstructed the image using anisotropic diffusion combined with mean curvature motion. It is clear that one should not expect that these heuristic strategies give the optimal set of interpolation points, in particular since most of the before mentioned methods do not take into account that the optimal set also depends on the interpolating PDE. Interestingly, experiments indicate that even one of the simplest PDEs can give good interpolation results if the interpolation data are chosen carefully: Using a stochastic optimization strategy in conjunction with the Laplace equation for interpolation, Dell performed experiments [24] demonstrating that the most useful points indeed have a higher density near edges. Similar findings can also be observed for surface interpolation problems using the Laplace-Beltrami operator [4]. However, even with sophisticated algorithms, a stochastic optimization is still too slow for practically useful PDE-based image coding. Thus, it would be helpful to derive analytical results on how to select good interpolation points for PDE-based compression. This will be the topic of the present paper. For simplicity, we focus on the interpolants based on the Laplace equation. Most of our mathematical analysis tools stem from the theory of shape optimization. Let us now give a mathematical formalization of the problem. Let D ⊆ R2 be the support of an image (say a rectangle) and f : D → R an image which is assumed to be known only on some region K ⊆ D. There are several PDE models to interpolate f and give an approximation of the missing data. One of the simplest way is to approach f |D\K by the harmonic function on D \ K, having the Dirichlet boundary data f |K on K and homogeneous Neumann boundary conditions on ∂D, i.e. to solve   −∆u = 0 in D \ K, u = f on K, (1)  ∂u = 0 on ∂D \ K. ∂n Denoting by uK the solution of (1), the precise question is to identify the region K which gives the “best” approximation uK , in a suitable sense, for example which minimizes one of the norms Z Z p |uK − f | dx or |∇uK − ∇f |2 dx. D

D

Intuitively, the larger the set K, the better the approximation is. This is only partially true, since a small well chosen region can give better approximations than large badly chosen ones. For practical reasons, for image compression purposes, one has to search a set K that satisfies a constraint which limits its size. 2

The purpose of this paper is to introduce and discuss a model based on shape analysis tools which is intended to obtain information about those regions K which give the best approximation of the image, under a constraint on their size. The best approximation is to be understood in the sense of a norm (Lp or H 1 ), while the constraint on the size of K is to be understood in the sense of a suitable measure (Lebesgue measure, Hausdorff measure, counting measure or capacity). We refer the reader to [11] for an introduction to shape analysis techniques and a detailed exposition of the main tools used throughout the paper. Two directions will be taken. The first idea is to set a continuous PDE model and search pointwise information by a topological asymptotics method, in order to evaluate the influence of each pixel in the reconstruction. If m is one of the measures above, roughly speaking the model we consider is equivalent to Z |∇u|2 dx − m(K). max min K u∈H 1 (D),u=f on K D The second way is to simulate into the continuous frame a finite dimensional shape optimization problem by imposing K to be the union of a finite number of fat pixels. Performing the asymptotic analysis by Γ-convergence when the number of pixels is increasing (in the same time that the fatness vanishes), we obtain useful information about the optimal distribution of the best interpolation pixels. The remainder of our paper is organized as follows. In Section 2 we review a number of useful concepts. A continuous shape optimization model is analyzed in Section 3, and finite dimensional considerations are presented in Section 4. These shape analysis results are complemented by a mathematical motivation for using a more fuzzy point selection strategy in Section 5. Similar results are obtained in Section 6 where the data selection problem is treated from an approximation theoretic viewpoint. In Section 7 we present numerical experiments where our data selection strategies are applied to a real-world image. Our paper is concluded with a summary in Section 8.

2

Γ-Convergence, Capacity and Measures

Let D ⊆ R2 be a smooth bounded open set and α > 0. The α-capacity of a subset E in D is Z  2 2 capα (E, D) = inf |∇u| + α|u| dx : u ∈ UE , D

where UE is the set of all functions u of the Sobolev space H01 (D) such that u ≥ 1 almost everywhere in a neighborhood of E. If a pointwise property holds for all x ∈ E except for the elements of a set Z ⊆ E with capα (Z) = 0, we say that the property holds quasi-everywhere on E and write q.e. The expression almost everywhere refers, as usual, to the Lebesgue measure. We notice the sets of zero capacity are the same, for every α > 0. For this reason, in the sequel we simply drop α since all concepts defined below are independent of α. The constant α plays a role only in the optimization process, as a parameter. A subset A of D is said to be quasi-open if for every  > 0 there exists an open subset A of D, such that A ⊆ A and capα (A \ A, D) < . A function f : D → R is said to be quasicontinuous (resp. quasi-lower semi-continuous) if for every  > 0 there exists a continuous (resp. lower semi-continuous) function f : D → R such that capα ({f 6= f }, D) < , where {f 6= f } = {x ∈ D : f (x) 6= f (x)}. It is well known (see, e.g., [34]) that every function u of the Sobolev space 3

H01 (D) has a quasi-continuous representative, which is uniquely defined up to a set of capacity zero. Throughout the paper, we identify the function u with its quasi-continuous representative, so that a pointwise condition can be imposed on u(x) for quasi-every x ∈ D. Equalities like u = 0 on a Borel set K are understood in the sense quasi-everywhere for a quasi-continuous representative. We denote by M0 (D) the set of all nonnegative Borel measures µ on D, such that i) µ(B) = 0 for every Borel set B ⊆ D with cap(B, D) = 0, ii) µ(B) = inf{µ(U ) : U quasi-open, B ⊆ U } for every Borel set B ⊆ D. We stress the fact that the measures µ ∈ M0 (D) do not need to be finite, and may take the value +∞. There is a natural way to identify a quasi-open set to a measure. More generally, given an arbitrary Borel subset E ⊆ Ω, we denote by ∞|E the measure defined by i) ∞|E (B) = 0 for every Borel set B ⊆ D with cap(B ∩ E, D) = 0, ii) ∞|E (B) = +∞ for every Borel set B ⊆ D with cap(B ∩ E, D) > 0. Definition 2.1 The α-capacity of a measure µ ∈ M0 (D) is defined by Z Z hZ i capα (µ) = inf |∇u|2 dx + α u2 dx + (u − 1)2 dµ . u∈H01 (D)

D

D

D

Definition 2.2 A sequence of functionals defined on a topological space V Fn : V → R Γ-converges to F in V if for every u ∈ V there exists a sequence un ∈ V such that un → u in V and F (u) ≥ lim sup Fn (un ), n→∞

and for every convergent sequence un → u in V F (u) ≤ lim inf Fn (un ). n→∞

The main property of the Γ-convergence is that every convergent sequence of minimizers of Fn has as limit a minimizer of F . For a measure µ ∈ M0 (D), we denote Fµ : H 1 (D) → R Z Z 2 Fµ (u) = |∇u| dx + |u|2 dµ. D

D

Definition 2.3 We say that a sequence (µn ) of measures in M0 (D) γ-converges to a measure µ ∈ M0 (D) if and only if Fµn Γ-converges in L2 (D) to Fµ . R Note that the γ-convergence is metrizable by the distance dγ (µ1 , µ2 ) = D |wµ1 − wµ2 |dx, where wµ is the variational solution of (formal) −∆wµ + µwµ = 1 in H01 (D) ∩ L2 (D, µ) (see [11, 21]). The precise sense of this equation is the following: wµ ∈ H01 (D) ∩ L2 (D, µ) and for every φ ∈ H01 (D) ∩ L2 (D, µ), Z Z Z ∇wµ ∇φdx + Ω

wµ φdµ = Ω

φdx. Ω

In view of the result of Hedberg [33], if A is an open subset of D, the solution of this equation associated to the measure ∞D\A is nothing else but the solution in the sense of distributions of −∆w = 1 in A, w ∈ H01 (A). We refer to [20] for the following result. 4

Proposition 2.4 The space M0 (D), endowed with the distance dγ , is a compact metric space. Moreover, the class of measures of the form ∞D\A , with A open (and smooth) subset of D, is dense in M0 (D).

3

The Continuous Model

Let D ⊆ R2 be a rectangle (symmetric with respect to the origin), the support of an image f : D → R. We assume for technical reasons that f ∈ H 1 (D) ∩ L∞ (D). One could formally work with functions having jumps (like SBV-functions, or local H 1 functions separated by jump sets), but the interpolation we perform which is based on elliptic PDE in H 1 cannot reconstruct jumps. Hence, a localization and an identification of contours should precede a local H 1 interpolation. For some Borel set K ⊆ D, which is assumed to be the known part of the image (i.e. f |K is known, while f |D\K is not), one “reconstructs” f by interpolating the missing data. Several interpolation processes can be employed, which roughly speaking consist in solving a partial differential equation with boundary data f |K . Our model is concerned with the harmonic interpolation based on equation (1). For every Borel set K ⊆ D, the weak solution uK ∈ H 1 (D) is the minimizer of  Z 2 1 |∇u| dx : u ∈ H (D), u = f q.e. on K . min D

This problem has a unique solution as soon as K has positive capacity. In order to define the model, one has to specify the norm of the best approximation of the image and the cost in terms of size of the set K, which should be expressed with a measure.

3.1

Analysis of the Model

Choice of the norm. Numerical evidence suggest to approach the gradient of f . This was observed in practice: On regions where the gradient is large (say around a contour of discrete discontinuities), it is more preferable to keep two parallel contours with significantly different values of f , and hence approach the gradient, than a single contour with low variation values. In this frame, the criterion reads Z min |∇uK − ∇f |2 dx. (2) K⊆D

D

Instead of the L2 norm of the ∇uK − ∇f above, one could also consider some Lp norm of uK − f . Since numerical evidence suggests to use (2), we concentrate our discussion on this norm, but most of the theoretical results remain valid without any modification. Choice of the measure. The constraint on K plays a crucial role in the shape analysis of the problem. There is a significant gap between the constraint in the continuous setting and the constraint in the discrete model, which is always the counting measure! In practice, in the discrete setting one intends to keep the lowest number of pixels. Consequently, an ideal model should impose a measure constraint on K which after discretization becomes the counting measure. A suitable way to deal with this problem is to consider fat pixels in association with the counting measure, and perform an asymptotic analysis. A first intuitive constraint would be expressed in terms of the Lebesgue measure and takes the form |K| ≤ c. (3) 5

Nevertheless, from a mathematical point of view, problem (2) associated to the constraint (3) is ill-posed (see the precise statement in Proposition 3.1 below). This is again a consequence of the transition from the discrete model to the continuous one. A pixel in the discrete setting corresponds in fact to a small square or a small ball (with positive area), while in the continuous one to a point! Since there are sets of zero Lebesgue measure but with positive capacity, constraint (3) may lead to almost optimal structures K of zero measure. Proposition 3.1 Problem (2)-(3), has in general no solution, the infimum in (2) being equal to zero. 2

Proof The proof is a direct consequence of the more general result below.

An alternative would be to replace the Lebesgue measures by the one dimensional Hausdorff measure H1 (K) ≤ c. (4) We prove in the sequel that (2)-(4) is in general ill-posed, unless a constraint on the number of connected components of K is added. This behavior is similar to the one observed for the Mumford-Shah functional (see [39]). Theorem 3.2 Problem (2)-(4) is in general ill-posed, in the sense that the infimum in (2) is zero, and there is no solution under constraint (4). 2

Proof Let Kn (c) = ∪i,j∈Z B ij (c) ∩ D, where Bij (c) is the closed ball of radius e−cn centered in (i/n, j/n). Following [19], for every g ∈ H −1 (D) the solutions of  −∆vn,c = g in D \ Kn (c), vn,c ∈ H01 (D \ Kn (c)) converge weakly in H01 (D) to vc , the solution of  −∆vc + cvc = g in D, vc ∈ H01 (D)

(5)

The same behavior can be observed if the boundary conditions of vn,c on ∂D are mixed, of the form vn,c = 0 on Γn (c) and ∂vn,c /∂n = g on ∂D \ Γn (c). Here g ∈ L2 (∂D) is fixed and Γn (c) ⊆ ∂D is on each edge, say [0, L], of ∂D of the form ∪i∈Z [i/n, (i + c−1 )/n] ∩ [0, L]. Using the capacity density condition (see for instance [11, Chapter 4]) and the locality of the γ-convergence, we have that the weak limit in H 1 (D) of the sequence vn,c is still the solution of (5). Taking a sequence ck → ∞, by a diagonal procedure we find Knk (ck ) ∪ Γnk (ck ) := Kk such that vKk → 0 weakly H 1 (D), and 1 2 n2k e−ck nk + H1 (∂D) → 0. ck Since ck → ∞, we obtain that the convergence vKk → 0 is strong in H 1 (D). Consequently, after solving (1) on D \ Kk , we have for the solution uk   −∆(uk − f ) = −∆f in D \ Kk , u − f = 0 on D \ Kk ,  ∂(uK −f ) ∂f = − ∂n on ∂D \ Kk . ∂n 6

Thus, uk − f converges strongly to zero in H01 (D). This infimum is, in general, not attained for a set K with finite Hausdorff measure. It is enough to consider a function f ∈ C 2 (D) for which the set {x ∈ D : ∆f 6= 0} has positive Lebesgue measure. In order to complete the proof, we replace in the construction of Kk the union of the discs Knk by the union of their boundaries. Since the Lebesgue measure of Knk asymptotically vanishes, the limit of uk remains unchanged. 2 Let us denote by ]K the number of the connected components of K. Then the following result can be established. Theorem 3.3 Given l ∈ N, problem (2)-(4) supplemented with the constraint ]K ≤ l has at least one solution. Proof Existence of a solution holds by using the continuity/compactness result in the Hausdorff complementary topology due to Sverak (see [11]) together with the Golab theorem. Indeed, let (Kn ) be a minimizing sequence for (2), such that H1 (Kn ) ≤ c and ]Kn ≤ l. The compactness of the Hausdorff metric provides a subsequence (still denoted using the same index) such that Kn converges to K. Then, ]K ≤ l and by the Golab theorem on the lower semicontinuity of the Hausdorff measure, we get H1 (K) ≤ c. In order to conclude one needs to prove that the energy is lower semicontinuous. In fact this is continuous, from the Sverak stability result applied to the equations −∆vn = −∆f in D \ Kn , vn ∈ H01 (D \ Kn ), 2

which has as solutions vn = uKn − f .

A proper mathematical analysis of this problem involves a constraint in terms of the α-capacity, which is the natural measure for the defect of continuity of Sobolev functions: capα (K) ≤ c.

(6)

Problem (2)-(6) is equivalent to Z max

min

K,capα (K)≤c

u∈H 1 (D),u=f on K

|∇u|2 ,

D

which penalizing the constraint becomes Z max

K⊆D

min

u∈H 1 (D),u=f on K

|∇u|2 dx − β capα (K).

(7)

D

Notice that (7) is a max-min problem associated to the Dirichlet energy into a Sobolev space with prescribed boundary values. This is to be compared to the cantilever problem in structure mechanics (see [11]) which, in the context of Neumann conditions on the free boundary, leads to a relaxation process. In order to discuss problem (7) we introduce the following notations. For every measure µ ∈ M0 (D), we set Fµ : H 1 (D) → R ∪ {+∞},  Z Z  2 |∇u| dx + (u − f )2 dµ if |u| ≤ |f |∞ , Fµ (u) = D D  +∞ else 7

and

Z E(µ) =

min Fµ (u) =

u∈H 1 (D)

min

u∈H 1 (D) D

|∇u|2 dx +

Z

(u − f )2 dµ.

D

We notice from the maximum principle that the minimizer above has to satisfy |u| ≤ |f |∞ , so it coincides with min Fµ (u). We also observe that the functionals Fµ are equi-coercive with respect to µ: Z |∇u|2 + u2 dx − |f |2∞ |D|. Fµ (u) ≥ D

For technical reasons, we do not allow the sets K to touch the boundary of D. For some δ > 0 we introduce the following notations: D−δ := {x ∈ D : d(x, ∂D) ≥ δ}, Kδ (D) := {K ⊆ D : K closed, K ⊆ D−δ }, and Mδ (D) := {µ ∈ M0 (D) : µbD\D−δ = 0}. The family Mδ (D) is compact with respect to the γ-convergence, as a consequence of the locality of the γ-convergence. We start with the following technical result (see also [21]). γ

Lemma 3.4 Let µn ∈ Mδ0 (D), µn −→ µ. Then capα (µn ) → capα (µ). Proof It is sufficient to prove that the Γ-convergence of the functionals, which can be done by a partition of unity. 2 Below are the mathematical main results of the paper. Theorem 3.5 If µn ∈ Mδ (D) γ-converges to µ, then µ ∈ Mδ (D) and Fµn Γ-converges to Fµ in L2 (D). Theorem 3.6 We have clγ Kδ (D) = Mδ (D), and sup (E(K) − β capα (K)) = k∈Kδ (D)

max (E(µ) − β capα (µ)).

µ∈Mδ (D)

As a consequence of Theorems 3.5-3.6, from every maximizing sequence in supk∈Kδ (D) (E(K) − β capα (K)) one can extract a γ-convergent subsequence such that the γ-limit measure is solution of the relaxed problem maxµ∈Mδ (D) (E(µ) − β capα (µ)), or Z Z max min |∇u|2 dx + (u − f )2 dµ − β capα (µ). (8) µ∈Mδ (D) u∈H 1 (D) D

D

8

Proof [of Theorem 3.5.] We shall prove independently both conditions of the Γ-convergence. Γ-liminf. Let un → u in L2 (D). Let ϕ ∈ Cc∞ (D), 0 ≤ ϕ ≤ 1, and ϕ = 1 on D−δ . Then un ϕ → uϕ in L2 (D), and from the γ-convergence µn → µ we have Z Z i Z hZ 2 2 2 |uϕ|2 dµ. |∇(uϕ)| dx + |un ϕ| dµn ≥ |∇(un ϕ)| dx + lim inf n→∞

D

D

D

D

Since µn is vanishing on D \ D−δ , by the locality property of the γ-convergence (see for instance [21]) we get that µ is also vanishing on D \ D−δ . Consequently, we have Z Z Z i hZ u2n dµn ≥ |∇ϕ|2 u2n dx + un ϕ∇un ∇ϕdx + lim inf |∇un |2 ϕ2 dx + 2 n→∞ Z Z Z D ZD D D 2 2 2 2 2 u dµ, |∇ϕ| u dx + uϕ∇u∇ϕdx + |∇u| ϕ dx + 2 D

D

D

D

or by eliminating the converging terms Z Z i Z hZ 2 2 2 2 2 u2 dµ. un dµn ≥ |∇u| ϕ dx + |∇un | ϕ dx + lim inf n→∞

D

D

D

D

Using 0 ≤ ϕ ≤ 1 one eliminates ϕ on the left hand side, and taking the supremum over all admissible ϕ on the right hand side Z  Z Z Z lim inf |∇un |2 dx + u2n dµn ≥ sup |∇u|2 ϕ2 dx + u2 dµ n→∞

D

ϕ

D

D

D

we get the Γ − lim inf inequality. Γ-limsup. Let u ∈ H 1 (D), |u| ≤ |f |∞ and u ˜ ∈ H01 (D) an extension of u on a dilation of D, say δ D . By the locality property of the γ-convergence, we still have that µn γ-converges to µ in Dδ for the operator H01 (Dδ ) 3 u 7→ −div (1D + ε1Dδ \D )∇u ∈ H −1 (Dδ ), for every ε > 0. Consequently, there exists a sequence uεn ∈ H01 (Dδ ) such that uεn → u ˜ in L2 (Dδ ) and Z Z Z 2 2 |∇˜ u| dx + ε |∇˜ u| dx + (˜ u − f )2 dµ ≥ δ D Z D \D Z D Z h i ε 2 ε 2 lim supn→∞ |∇˜ un | dx + ε |∇˜ u | dx + (˜ uεn − f )2 dµn , Dδ \D

D

and hence Z Z 2 |∇˜ u| dx + ε D

Dδ \D

2

Z

|∇˜ u| dx +

D

2

(˜ u − f ) dµ ≥ lim sup D

n→∞

hZ D

|∇˜ uεn |2 dx

Z +

i (˜ uεn − f )2 dµn .

D

The function u ˜ being fixed, we make ε → 0 and extract by a diagonal procedure a sequence uεnn converging in L2 (Dδ ) to u ˜, such that the Γ − lim sup inequality holds. 2 Proof [of Theorem 3.6.] On the one hand, Kδ (D) ⊆ Mδ (D) so the inclusion clγ Kδ (D) ⊆ Mδ (D) is obvious from the γ-compactness of Mδ (D). Conversely, let µ ∈ Mδ (D). By the density result of shapes [20] there exists a sequence of closed sets Kn ⊆ D such that D \ Kn γ-converges to µ. Moreover, the sequence Kn can be chosen such that Kn ⊆ (D−δ )1/n , from the locality property of the γ-convergence. Making a homothety εn Kn , with suitable εn > 0 we get εn Kn ⊆ D−δ ), so εn Kn ∈ Kδ (D). We can choose εn → 1, hence D \ εn Kn γ-converges to µ. 2 9

In the proof of this theorem, we cannot choose a priori the sequence Kn in D−δ since D−δ is a closed set. In the pathological case that D−δ would be a line, a measure can be supported on a line, while an open set can not be contained in a one dimensional set. The homothety can be performed because D−δ has a particular structure, being star shaped with respect to the origin.

3.2

Topological Asymptotic: Identifying the ”Influence” of Each Pixel

Many usual shape optimization algorithms are based on shape derivative steepest descent methods associated to a level set approach. The knowledge of the relaxed formulation (Theorem 3.6) to the interpolation problem leads to algorithms of new type, which compute the relaxed solutions (here the measure µ) that solve the relaxed problem (here (8)) and is followed by an appropriate projection which yields a shape approximation of the true solution (see [1], [8], [28], [31]). Due to the analogy of our formulation of the image interpolation problem with the cantilever problem, we will use a topological gradient based algorithm as in [31] (see also [7, 47]). It simply consists in starting with K = D and computing the asymptotic of the non-relaxed cost functional (7) with respect to performing small holes and eliminating those small balls which have the “least“ increasing effect on the functional. In the present case, it constitutes a powerful tool allowing for very fast convergence and very low cost. In this section we compute the topological gradient which turns to be related to the harmonicity defect of f . As observed in practice, the region of f which should be kept is the one where the |∆f | is large. Assume in all this section that f is smooth enough (roughly speaking, f ∈ C 2 (D)). Let us denote by K the compact set K \ B(x0 , ), where B(x0 , ) is the ball centered at x0 ∈ D with radius , and assume that x0 is an interior point and ε is small enough. We consider the functional Z |∇u|2 dx, J(K ) = min 1 u∈H (D),u=f on K D which is assumed to be minimized by uε . Then Z Z J(K ) − J(K) = |∇uε |2 dx − B(x0 ,)

|∇f |2 dx.

B(x0 ,)

Using equation (1) satisfied by uε onD \ Kε we get Z Z 2 J(K ) − J(K) = ∇uε ∇f − |∇f | dx = B(x0 ,ε)

∆f (f − uε )dx.

B(x0 ,ε)

We have ∆f (x) = ∆f (x0 ) + kx − x0 kO(1), and hence Z Z J(Kε ) − J(K) = ∆f (x0 ) (f − uε )dx + εO(1) B(x0 ,ε)

(f − uε )dx.

B(x0 ,ε)

It R is enough to compute the fundamental term in the asymptotic development of the expression B(x0 ,ε) (f − uε )dx. Using the harmonicity of uε we have Z Z Z Z Z ε ε (f − uε )dx = f dx − uε dσ = f dx − f dσ. 2 ∂B(x0 ,ε) 2 ∂B(x0 ,ε) B(x0 ,ε) B(x0 ,ε) B(x0 ,ε) We use the Taylor formula for f around x0 and get f (x) = f (x0 ) +

X ∂f (x0 )(xi − xi0 ) ∂xi

i=1,2

10

+

1 X ∂2f (x0 )(xi − xi0 )(xj − xj0 ) + kx − x0 k2 o(1). 2 ∂xi ∂xj i,j=1,2

Consequently, Z

ε f dx − 2 B(x0 ,ε)

ε ∂2f =− (x0 ) 4 ∂x2i Thus, we get

Z X 1 ∂2f f dσ = (x0 ) (xi − xi0 )2 dx 2 ∂x2i ∂B(x0 ,ε) B(x0 ,ε)

Z

i=1,2

Z

(xi − xi0 )2 dσ + ε4 o(1) = −

∂B(x0 ,ε)

∆f (x0 ) π + ε4 o(1). 4

π J(Kε ) − J(K) = −|∆f (x0 )|2 ε4 + ε4 o(1). 2

By trivial calculus, the asymptotic expansion of the capacity is at least of order ε4 and is independent on x0 . Finally, the algorithm we use is independent of the asymptotic expansion of capacity and takes into account only the defect of harmonicity of f , namely |∆f (x0 )|. This suggests to keep the points x0 where |∆f (x0 )| is maximal. From a practical point of view, this is the main result of our local shape analysis.

4

Optimal Distribution of Pixels: Asymptotics of the Finite Dimensional Model

∂f In this section we assume f ∈ H 2 (D), ∂n = 0 on ∂D. Moreover, we formally consider the problem in dimensions d = 2 and d = 3, since the 3D case has a more intuitive solution and leads to a better comprehension of the problem. Let m > 0 and n ∈ N, and let us define n m o Am,n := ∪ni=1 B(xi , r) : xi ∈ Rd , r = 1/d . n

In the sequel, a ball B(xi , r) will be called a fat pixel. We consider problem (2) for every K ∈ Am,n , i.e. Z min |∇uK − ∇f |2 dx. (9) K∈Am,n

D

Of course, it is sufficient to consider only centers xi in a r neighborhood of D. Let us rename vK = uK − f and g = ∆f ∈ L2 (D). Consequently, vK solves   −∆vK = g in D \ K, vK = 0 on K, (10)  ∂vK ∂n = 0 on ∂D \ K, and the optimization problem (9) can be reformulated as a compliance optimization problem Z min gvK dx. (11) K∈Am,n

D

A similar problem, with Dirichlet boundary conditions on ∂D, was studied in [12]. Although we deal here with Neumann boundary conditions on ∂D, if we choose to cover the boundary by balls, 11

d−1

we get rid of the Neumann boundary conditions by using only Cd n d balls. From an asymptotic point of view, this means that we can formally consider the Dirichlet boundary condition on ∂D. It is easy to observe that problem (11) has always an optimal solution, say Knopt , which γconverges to ∞D . Roughly speaking, the sequence (Knopt )n gives asymptotically a perfect approximation of f , but the number of fat pixels goes to infinity and no further information about their distribution is provided. This information (local density of Knopt ) can be obtained by using a different topology for the Γ-convergence of the (rescaled) energies. In this new frame, the minimizers are unchanged but their behavior is seen from a different point of view. For every K ∈ Am,n we define n

µK

1X := δxi ∈ P(Rd ), n i=1

where δx is the Dirac measure at the point x and P(Rd ) is the space of probability Borel measures on Rd . We introduce the functionals Fn : P(Rd ) → R ∪ {+∞},  2/d R n if µ = µK , K ∈ Am,n , D gvK dx Fn (µ) = +∞ else. We recall the following result from [12]. Theorem 4.1 Assume g ≥ 0. The sequence of functionals Fn Γ-converges with respect to the weak ? topology in P(Rd ) to Z g2 θ(mµ1/d F (µ) = a )dx, 2/d D µa where µ = µa dx + ν is the Radon decomposition of µ and θ(α) := inf{lim inf n2/d F (Kn ) : Kn ∈ Aα,n }. n

First, we notice that the hypothesis g ≥ 0 is not restrictive from a practical point of view, since we may formally split the discussion on the sets {∆f > 0} and {∆f < 0}. Second, as consequence of this result we have that µKnopt → µopt , weakly ? in P(Rd ), where µopt is a minimizer of F . The knowledge of the function θ would give information on the density of the absolute continuous part of µopt , with respect to the Lebesgue measure, thus on µKnopt for n large. Unfortunately, the function θ is not known explicitly, but following [12], a series of properties can be established. The function θ is positive, nonincreasing and vanishes from some point on. For a small α, the following inequalities hold: d=2:

C1 | log α| − C2 ≤ θ(α) ≤ C3 | log α|,

1 1 − C2 ≤ θ(α) ≤ C3 . α α Minimizing F leads to the following interpretation: On the regions where |g| = |∆f | is very large, µa has to be large enough in order to approach the value for which θ vanishes. In regions where |∆f | is small, µa may also be small. If we formally use the previous inequalities and write the Euler equation for the minimizer, we get 12 d=3:

C1

• for d = 2:

µ2a |1−log µa |

• for d = 3:

µa ≈ cm,f |∆f |,

≈ cm,f |∆f |2 ,

where cm,f are suitable constants. This result suggests to choose the interpolation data such that the pixel density is increasing with |∆f |. Such a strategy has a more relaxed character than the hard thresholding rule we derived in the previous section.

5

Mathematical Motivation for Dithering

Previous considerations in Section 3 suggest to select K as the level set of those points x where |∆f (x)| exceeds some threshold, but the reasoning in Section 4 indicates that such a hard rule is not optimal. We shall now present additional arguments based on potential theory why |∆f (x)| should rather serve as a fuzzy indicator for selecting a point x as a candidate for a good interpolation set K. While the decision whether some point x ∈ D belongs to K or D \ K is a binary decision, it is clear that |∆f (x)| may attain a continuum of nonnegative values. So how can we convert the information from |∆f | into a good interpolation set K without using a strict thresholding? In image analysis and computer graphics, a successful concept of turning a continuous grayscale image gc into a visually similar binary image gb is called dithering or digital halftoning [50]. It is widely used, e.g. when printing a grayscale image on a laser printer. So let us now argue how dithering can be used for our interpolation problem. In the following it is important to note that the differential equation (1) can be interpreted as a Poisson equation. Let uK be a solution to equation (1) then u is also a solution to the Poisson equation ∆u = 1K · ∆f (12) since uK is harmonic outside of K and uK coincides with f on K. Note that ∆f is to be understood in the distributional sense. We might think of ∆f as a Borel measure. Dithering is a technique used for representing primarily grayscale images as black and white images. Hereby the grayscale distribution is simulated for the human eye by a spacial distribution of black and white pixels. Several algorithms in numerous variants are available [50], but common to all is that when both the grayscale and its dithered version are blurred, a very similar visual impression should be created. SLet denote λ the Lebesgue measure on the image domain D composed of N pixels Ai : D = N k=1 Ak . In effect, dithering means an approximation of a grayscale image n n P P g : D → [0, 1] seen as a measure g · λ with density g w.r.t. λ by a measure µi , i.e. g · λ ≈ µi . i=1

i=1

The µi are probability measures concentrated on certain pixels Aki of the image, supp(µi ) ⊂ Aki . According to the notion of convergence of distributions [22] the approximation ‘≈‘ is understood in the sense that the difference Z ! Z n X ϕd µi ϕ · g dλ − D D i=1

is small for a ‘blurring kernel‘ ϕ on D. One may think of ϕ as a (truncated) Gaussian kernel with a not too small variance or a standard mollifier function. Possible choices of these measures are Dirac measures, µi = δzi with zi ∈ Aki , normalized volume measures, µi = λ(A1k ) 1Aki · λ, or correspondingly normalized surface measures on the boundary i 13

∂Aki of the pixels. A good dithering procedure preserves the average gray value of the image. Hence the ratio of the number of white pixels and the total number of pixels is fixed. This implies that the number of white pixels is given by the total number of pixels times the average gray value. Hence one can adjust a priori the number of white pixels, that is, the compression rate, and a dithering will produce by scaling the original image appropriately. Applying a dithering procedure to a scaled version of ∆f with a scaling factor s gives an approximation n X s∆f ≈ µi (13) i=1

with compactly supported µi on pixels Aki and where, most important, the number of pixels corresponds to a preassigned compression rate n/N . Then we can define K as a disjoint union R S K := ni=1 Aki . On the set K we use D ∆f dµi , the µi -averages of ∆f , to approximate ∆f with the measures µi from (13): n Z X 1K ∆f ≈ ∆f dµi · µi . (14) i=1

D

Note that the support of the measures on both sides is contained on K. Using the approximations above we are now able to reconstruct uK . It is known from potential theory that a solution to the Poisson equation on R2 , ∆u = ρ, with a compactly supported measure ρ is given by a convolution with the fundamental solution E2 of the Laplacian, the logarithmic potential [23] E2 (r) =

1 log(r), 2π

r > 0,

that is, u = E2 ∗ ρ . Hence, we can infer from equations (12) and (14) that n Z X u = E2 ∗ ∆u = E2 ∗ 1K ∆f ≈ ∆f dµi · E2 ∗ µi .

(15)

(16)

i=1

Remarks: 1. The considerations above show that dithering plays a vital role in finding a ‘good‘ set K. This is achieved by the approximation in (14) which also conveys the compression rate via the scaling factor s in (13). 2. A solution uK stemming from a ‘dither‘ set K is a reasonable approximation to f : f − uK = E2 ∗ (∆f − ∆uK ) = E2 ∗ (∆f − 1K ∆f ) . However, the selection by dithering of µi and hence of K ensures that we have on D ∆f ≈ 1K ∆f

(17)

which implies a small difference f − u in a suitable norm. This also brings to light that two extreme choices of K are not likely to produce a reasonably small difference f − u: homogeneous distributions such as completely stochastic or regular grid-like distributions which introduce errors where ∆f is large. Similarly, choosing the set K = |∆f | > t with a t adjusted to the desired compression rate also causes the quality of the approximation (17) to deteriorate: the concentration on super-level sets |∆f | > t neglects the valuable information in regions where f is flat, that is, where ∆f is small. 14

3. If the measures µi are point measures δzi , equation (16) collapses to u ≈

n P

∆f (zi )·E2 (·−zi ) .

i=1

4. The considerations above apply without restriction to any dimension d > 2. Instead of the logarithmic potential one has to use the appropriate Newtonian potential, and one has use a corresponding dithering procedure for multidimensional data. 5. The statements made about the Laplacian and its potentials can be applied essentially verbatim to any other linear differential operator whose fundamental solution is at ones disposal. Hence, the the Laplacian in the above can be replaced by, for example the Helmholtz operator (d = 2, 3), the Cauchy-Riemann operator (d = 2), or the polyharmonic operator, since their fundamental solutions are known [22] .

6

Approximation Theoretic Motivation

Since lossy data compression is essentially an approximation theoretic problem, it is interesting to complement the preceding considerations with an approximation theoretic motivation on how to choose the interpolation data in a reasonable way. In order to keep things as simple as possible, we restrict ourselves to the 1-D case with D = [a, b], and we assume that f ∈ C 2 [a, b] and the interpolation data are given by K = {x1 , x2 , ..., xn+1 } with a < x1 < x2 < ... < xn+1 < b. Solving u00 = 0 in some interval (xi , xi+1 ) with Dirichlet boundary conditions u(xi ) = f (xi ) and u(xi+1 ) = f (xi+1 ) yields linear interpolation: u(x) = f (xi ) +

x − xi (f (xi+1 ) − f (xi )). xi+1 − xi

Thus, the interpolation error in some point x ∈ [xi , xi+1 ] is given by x − xi (f (xi+1 ) − f (xi )) − f (x) e(x) := |(u(x) − f (x)| = f (xi ) + xi+1 − xi

(18)

(19)

Applying the mean value theorem three times, this becomes e(x) = f (xi ) + (x − xi )f 0 (ξ) − f (x) = (x − xi )f 0 (ξ) − (x − xi )f 0 (η) = (x − xi ) |ξ − η| |f 00 (ρ)|

(20)

with some suitable points ξ, η, ρ ∈ [xi , xi+1 ]. Using |ξ − η| ≤ xi+1 − xi =: hi and |f 00 (ρ)| ≤ max{|f 00 (x)| | x ∈ [xi , xi+1 ]} =: Mi , the worst case interpolation error in the interval [xi , xi+1 ] can be estimated by ei :=

max x∈[xi ,xi+1 ]

e(x) ≤ h2i Mi .

(21)

If one wants to minimize max{e1 , e2 , ..., en } one should select the interval widths hi such that e1 = e2 = ... = en . This means that p ∀i ∈ {1, ..., n} (22) 1/hi = c Mi with some constant c. Since 1/hi measures the local density of the interpolation points, this suggests that in some point x one should choose the density of the interpolation points proportional to 15

p |f 00 (x)|. Although one may argue that 1-D considerations are only of limited usefulness for the 2-D image interpolation problem, we observe that our simple approximation theoretic model gives suggestions that point in the same direction as the much more sophisticated reasonings from Sections 4 and 5: One should select the interpolation data such that their density is proportional to |∆f |p with some power p > 0.

7

Numerical Results

Let us now illustrate the mathematical discussions with numerical experiments. To this end, the Laplace equation has been discretized by finite differences, and the resulting linear system of equations is solved using the successive overrelaxation (SOR) method (see e.g. [44]). The CPU times for coding the images in our experiments are far below one second, and decoding is in the order of a second on a PC. If necessary, there still exist a number of options for speeding up this interpolation; see e.g. [3] and [38]. Figure 1(a) shows an original grayscale image f of size 257 × 257 pixels. In order to compute the modulus of the Laplacian in Figure 1(b), the image has been preprocessed by convolving it with a Gaussian of standard deviation σ = 1 pixel. This is a common procedure in image analysis in order to address the ill-posedness of differentiation (high sensitivity w.r.t. noise and quantization errors). If one selects the interpolation set K by thresholding the modulus of the Gaussian-smoothed Laplacian |∆fσ | such that 10 % of all pixels are kept, one obtains the set in Figure 1(c). The resulting interpolation in Figure 1(d) shows that this hard thresholding strategy is not optimal for reconstructing the image in high quality: Regions with a small spatial variation of the gray values are not represented at all in the interpolation set K, since their absolute value of the Laplacian is below the threshold. This leads to fairly poor results. Using the dithering strategy, however, gives a completely different interpolation set K. In our case we have applied one of the most popular dithering algorithms, namely the classical error diffusion method of Floyd and Steinberg [29]. It scans through all pixels, rounds the actual gray value either to black (0) or white (255), depending on which value is closer. Then it distributes the resulting error to the neighbors that have not been visited yet. Thus, the goal is to have a dithered image with the same average gray value as the original one. If one wants to obtain a dithered representation of |∆fσ | where e.g. 10 % of all pixels are white (255) and 90 % black (0), one multiplies |∆fσ | with a constant such that its mean amounts to 0.1 · 255 = 25.5, and applies Floyd-Steinberg dithering. In Figure 1(e) we observe that near edges where the modulus of the Laplacian is large, more points are chosen, but the dithering strategy also guarantees that some interpolation points are selected in relatively flat regions. The dithered interpolation set leads to very good results as is shown in Figure 1(f). This confirms our theoretical considerations from the Sections 4, 5, and 6.

8

Summary and Conclusions

We have analyzed the problem of finding optimal interpolation data for Laplacian-based interpolation. To this end, we have investigated a number of shape optimization approaches, a level set approach and an approximation theoretic reasoning. All theoretical findings emphasize the importance of the Laplacian for appropriate data selection, either by thresholding the modulus of the Laplacian or by interpreting it as a density for selecting the interpolation points. Numerical 16

Figure 1: (a) Top left: Original image f , 257 × 257 pixels. (b) Top center: |∆fσ | with σ = 1. (c) Top right: Thresholding of (b) such that 10 % of the pixels remain as interpolation data. (d) Bottom left: Interpolation using the “thresholded” set K from (c). (e) Bottom center: Floyd-Steinberg dithering of (b) such that 10 % of all pixels are selected. (f ) Bottom right: Interpolation using the “dithered” set K from (e). experiments clearly suggest to favor the density models. It is our hope that our paper helps a little bit to make shape optimization tools more popular in image processing, and to make researchers in shape optimization more aware of challenging image processing problems. Both fields have a lot to offer to each other.

References [1] G. Allaire, Shape Optimization by the Homogenization Method, vol. 146 of Applied Mathematical Sciences, Springer, New York, 2002. [2] H. A. Aly and E. Dubois, Image up-sampling using total-variation regularization with a new observation model, IEEE Transactions on Image Processing, 14 (2005), pp. 1647–1659. ¨hling, P. Hunziker, and M. Unser, Variational image reconstruction [3] M. Arigovindan, M. Su from arbitraily spaced samples: A fast multiresolution spline solution, IEEE Transactions on Image Processing, 14 (2005), pp. 450–460. [4] E. Bae, New PDE-based methods for surface and image reconstruction, Master’s thesis, Dept. of Mathematics, University of Bergen, Norway, 2007.

17

[5] S. Battiato, G. Gallo, and F. Stanco, Smart interpolation by anisotropic diffusion, in Proc. Twelvth International Conference on Image Analysis and Processing, Montova, Italy, Sept. 2003, IEEE Computer Society Press, pp. 572–577. [6] A. Belahmidi and F. Guichard, A partial differential equation approach to image zoom, in Proc. 2004 IEEE International Conference on Image Processing, vol. 1, Singapore, Oct. 2004, pp. 649–652. [7] L. J. Belaid, M. Jaoua, M. Masmoudi, and L. Siala, Image restoration and edge detection by topological asymptotic expansion, Comptes Rendus de l’Acad´emie des Sciences de Paris, S´erie I, 342 (2006), pp. 313–318. [8] M. Bendsoe and O. Sigmund, Topology Optimization: Theory, Methods and Applications, Springer, Berlin, 2003. [9] M. Bertalm´ıo, G. Sapiro, V. Caselles, and C. Ballester, Image inpainting, in Proc. SIGGRAPH 2000, New Orleans, LI, July 2000, pp. 417–424. ¨rz, Fast image inpainting based on coherence transport, Journal of Math[10] F. Bornemann and T. Ma ematical Imaging and Vision, 28 (2007), pp. 259–278. [11] D. Bucur and G. Buttazzo, Variational Methods in Shape Optimization Problems, vol. 65 of Progress in Nonlinear Differential Equations, Birkh¨auser, Basel, 2005. [12] G. Buttazzo, F. Santambrogio, and N. Varchon, Asymptotics of an optimal compliance-location problem, ESAIM: Control, Optimisation and Calculus of Variations, 12 (2006), pp. 752–769. ´s, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction [13] E. Cande from highly incomplete frequency information, IEEE Transactions on Information Theory, 52 (2006), pp. 489–509. [14] S. Carlsson, Sketch based coding of grey level images, Signal Processing, 15 (1988), pp. 57–83. [15] V. Caselles, J.-M. Morel, and C. Sbert, An axiomatic approach to image interpolation, IEEE Transactions on Image Processing, 7 (1998), pp. 376–386. [16] T. F. Chan and J. Shen, Non-texture inpainting by curvature-driven diffusions (CDD), Journal of Visual Communication and Image Representation, 12 (2001), pp. 436–449. [17]

, Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods, SIAM, Philadelphia, 2005.

[18] T. F. Chan and H. M. Zhou, Total variation improved wavelet thresholding in image compression, in Proc. Seventh International Conference on Image Processing, vol. II, Vancouver, Canada, Sept. 2000, pp. 391–394. [19] D. Cioranescu and F. Murat, Un terme ´etrange venu d’ailleurs, in Nonlinear Partial Differential Equations and Their Applications (Coll´ege de France Seminar, Vol. II), H. Brezis and J. L. Lions, eds., vol. 60 of Research Notes in Mathematics, Pitman, London, 1982, pp. 98–138. [20] G. Dal Maso, γ-convergence and µ-capacities, Annali della Scuola Normale Superiore di Pisa, 14 (1988), pp. 423–464. [21] G. Dal Maso and F. Murat, Asymptotic behavior and correctors for Dirichlet problems in perforated domains with homogeneous monotone operators, Annali della Scuola Normale Superiore di Pisa, 24 (1997), pp. 239–290. [22] R. Dautray and J. L. Lions, Mathematical Analysis and Numerical Methods for Science and Technology, vol. 2, Springer, Berlin, 1990. [23]

, Mathematical Analysis and Numerical Methods for Science and Technology, vol. 1, Springer, Berlin, 1990.

[24] H. Dell, Seed points in PDE-driven interpolation. Bachelor’s Thesis, Dept. of Computer Science, Saarland University, Saarbr¨ ucken, Germany, 2006.

18

[25] R. Distasi, M. Nappi, and S. Vitulano, Image compression by B-tree triangular coding, IEEE Transactions on Communications, 45 (1997), pp. 1095–1100. [26] J. H. Elder, Are edges incomplete?, International Journal of Computer Vision, 34 (1999), pp. 97–122. ´, Constrained [27] G. Facciolo, F. Lecumberry, A. Almansa, A. Pardo, V. Caselles, and B. Rouge anisotropic diffusion and some applications, in Proc. 2006 British Machine Vision Conference, vol. 3, Edinburgh, Scotland, Sept. 2006, pp. 1049–1058. [28] S. Finzi Vita, Numerical shape optimization for relaxed Dirichlet problems, Mathematical Methods in the Applied Sciences, 3 (1993), pp. 19–34. [29] R. W. Floyd and L. Steinberg, An adaptive algorithm for spatial grey scale, Proceedings of the Society of Information Display, 17 (1976), pp. 75–77. ´, J. Weickert, M. Welk, A. Bruhn, A. Belyaev, and H.-P. Seidel, Towards PDE-based [30] I. Galic image compression, in Variational, Geometric and Level-Set Methods in Computer Vision, N. Paragios, O. Faugeras, T. Chan, and C. Schn¨ orr, eds., vol. 3752 of Lecture Notes in Computer Science, Springer, Berlin, 2005, pp. 37–48. [31] S. Garreau, P. Guillaume, and M. Masmoudi, The topological asymptotic for PDE systems: the elasticity case, SIAM Journal on Control and Optimization, 39 (2001), pp. 1756–1778. [32] H. Grossauer and O. Scherzer, Using the complex Ginzburg–Landau equation for digital impainting in 2D and 3D, in Scale-Space Methods in Computer Vision, L. D. Griffin and M. Lillholm, eds., vol. 2695 of Lecture Notes in Computer Science, Springer, Berlin, 2003, pp. 225–236. [33] L. I. Hedberg, Spectral synthesis in Sobolev spaces and uniqueness of solutions of Dirichlet problems, Acta Mathematica, 147 (1981), pp. 237–263. [34] J. Heinonen, T. Kilpelainen, and O. Martio, Nonlinear Potential Theory of Degenerate Elliptic Equations, Clarendon Press, Oxford, 1993. [35] R. Hummel and R. Moniot, Reconstructions from zero-crossings in scale space, IEEE Transactions on Acoustics, Speech, and Signal Processing, 37 (1989), pp. 2111–2130. [36] P. Johansen, S. Skelboe, K. Grue, and J. D. Andersen, Representing signals by their toppoints in scale space, in Proc. Eighth International Conference on Pattern Recognition, Paris, France, Oct. 1986, pp. 215–217. [37] F. M. W. Kanters, M. Lillholm, R. Duits, B. J. P. Jansen, B. Platel, L. Florack, and B. M. ter Haar Romeny, On image reconstruction from multiscale top points, in Scale Space and PDE Methods in Computer Vision, R. Kimmel, N. Sochen, and J. Weickert, eds., vol. 3459 of Lecture Notes in Computer Science, Springer, Berlin, 2005, pp. 431–439. ¨ stler, M. Stu ¨rmer, C. Freundl, and U. Ru ¨de, PDE based video compression in real time, [38] H. Ko Tech. Rep. 07-11, Lehrstuhl f¨ ur Informatik 10, Univ. Erlangen–N¨ urnberg, Germany, 2007. [39] S. R. Kulkarni, S. Mitter, and T. J. Richardson, An existence theorem and lattice approximations for a variational problem arising in computer vision, in Signal Processing: Part I, L. Auslander, T. Kailath, and S. Mitter, eds., vol. 22 of The IMA Volumes in Mathematics and its Applications, Springer, London, 1990, pp. 189–210. [40] M. Lillholm, M. Nielsen, and L. D. Griffin, Feature-based image analysis, International Journal of Computer Vision, 52 (2003), pp. 73–95. [41] D. Liu, X. Sun, F. Wu, S. Li, and Y.-Q. Zhang, Image compression with edge-based inpainting, IEEE Transactions on Circuits, Systems and Video Technology, 17 (2007), pp. 1273–1286. [42] F. Malgouyres and F. Guichard, Edge direction preserving image zooming: A mathematical and numerical analysis, SIAM Journal on Numerical Analysis, 39 (2001), pp. 1–37.

19

[43] S. Masnou and J.-M. Morel, Level lines based disocclusion, in Proc. 1998 IEEE International Conference on Image Processing, vol. 3, Chicago, IL, Oct. 1998, pp. 259–263. [44] K. W. Morton and L. M. Mayers, Numerical Solution of Partial Differential Equations, Cambridge University Press, Cambridge, UK, 1994. [45] S. D. Rane, G. Sapiro, and M. Bertalmio, Structure and texture filling-in of missing image blocks in wireless transmission and compression applications, IEEE Transactions on Image Processing, 12 (2003), pp. 296–302. [46] A. Roussos and P. Maragos, Vector-valued image interpolation by an anisotropic diffusion-projection PDE, in Scale Space and Variational Methods in Computer Vision, F. Sgallari, F. Murli, and N. Paragios, eds., vol. 4485 of Lecture Notes in Computer Science, Springer, Berlin, 2007, pp. 104–115. [47] J. Sokolowski and A. Zochovski, On the topological derivative in shape optimization, SIAM Journal on Control and Optimization, 37 (1999), pp. 1251–1272. ´, V. Caselles, G. Sapiro, and F. Arandiga, Morse description and geometric encoding [48] A. Sole of digital elevation maps, IEEE Transactions on Image Processing, 13 (2004), pp. 1245–1262. ´ and R. Deriche, Vector-valued image regularization with PDEs: A common frame[49] D. Tschumperle work for different applications, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (2005), pp. 506–516. [50] R. Ulichney, Digital Halftoning, MIT Press, Cambridge, MA, 1987. [51] J. Weickert and M. Welk, Tensor field interpolation with PDEs, in Visualization and Processing of Tensor Fields, J. Weickert and H. Hagen, eds., Springer, Berlin, 2006, pp. 315–325. [52] Z. Xie, W. R. Franklin, B. Cutler, M. A. Andrade, M. Inanc, and D. M. Tracy, Surface compression using over-determined Laplacian approximation, in Advanced Signal Processing Algorithms, Architectures, and Implementations XVII, F. T. Luk, ed., vol. 5266 of Proceedings of SPIE, SPIE Press, Bellingham, 2007. [53] Z. W. Xiong, X. Y. Sun, F. Wu, and S. P. Li, Image coding with parameter-assistant inpainting, in Proc. 2007 IEEE International Conference on Image Processing, vol. 2, San Antonio, TX, Sept. 2007, pp. 369–372. [54] Y. Zeevi and D. Rotem, Image reconstruction from zero-crossings, IEEE Transactions on Acoustics, Speech, and Signal Processing, 34 (1986), pp. 1269–1277. [55] H. Zimmer, PDE-based image compression using corner information, Master’s thesis, Dept. of Computer Science, Saarland University, Saarbr¨ ucken, Germany, 2007. Zakaria Belhachmi: Laboratoire de Math´ematiques UMR 7122, Universit´e de Metz, Ile du Saulcy, F57045 Metz Cedex 1, France, [email protected] Dorin Bucur: Laboratoire de Math´ematiques CNRS-UMR 5127, Universit´e de Savoie, Campus Scientifique, 73 376 Le-Bourget-Du-Lac, France, [email protected] Bernhard Burgeth: Faculty of Mathematics and Computer Science, Saarland University, Building E1.1, 66041 Saarbr¨ ucken, Germany, [email protected] Joachim Weickert: Faculty of Mathematics and Computer Science, Saarland University, Building E1.1, 66041 Saarbr¨ ucken, Germany, [email protected]

20