Tianling Jin, Dennis Kriventsov and Jingang Xiong
1 Department of Mathematics, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong
2 Department of Mathematics,Rutgers University,110 Frelinghuysen Road,Piscataway, NJ 08854, USA
3 School of Mathematical Sciences, Laboratory of Mathematics and Complex Systems,MOE,Beijing Normal University,Beijing 100875,China
Abstract. We study a Rayleigh-Faber-Krahn inequality for regional fractional Laplacian operators. In particular, we show that there exists a compactly supported nonnegative Sobolev function u0 that attains the infimum(which will be a positive real number) of the setUnlike the corresponding problem for the usual fractional Laplacian, where the domain of the integration is Rn×Rn, symmetrization techniques may not apply here. Our approach is instead based on the direct method and new a priori diameter estimates. We also present several remaining open questions concerning the regularity and shape of the minimizers, and the form of the Euler-Lagrange equations.
Key words: Rayleigh-Faber-Krahn inequality, regional fractional Laplacian, first eigenvalue.

Depending on the choices ofn, σand ?, these two norms may or may not be equivalent. Even when they are equivalent (see Lemma 2.1), there are still subtle differences in how they depend on the domain ?.
One significant difference is the behavior of their corresponding best Sobolev constants:


As far as we are aware,it is unknown whether there are any such sets. A tempting conjecture would be that all such sets are balls, but the nature of the underlying equations does not support the arguments usually used to prove such symmetry results (symmetrization, moving planes; see the discussion below).

Inspired by the classical Rayleigh-Faber-Krahn inequality [9,13,21], we would like to study the variational problem

where|?| is the Lebesgue measure of ?.

In this paper, we prove the following existence result:

is achieved by u0.
The key difficulty in proving this is to obtain ana prioribound on the diameter of{u0>0}. In the shape optimization and free boundary literature, such bounds are common and are closely tied with lower bounds on the growth ofu0from the boundary of?{u0>0}, or perhaps of auxiliary functions related tou0: the idea is that ifu0grows at a prescribed rate from?{u0>0}, its support cannot have long,thin necks or many small connected components. Such growth estimates are usually obtained by using sets like{u0>0}BR(x)as competitors(with appropriately chosen functions). We proceed along these lines here as well,but carrying out the argument requires various non-standard modifications, some decidedly nonlocal multi-scale iteration procedures, and in the end does not result in “uniform” growth estimates onu0.
For related reasons, we are unable to show much more than stated in Theorem 1.1 concerning the nature of the minimizingu0(though see Lemma 5.3,which showsu0is bounded). Of particular interest is this question:
Open Question 1.2.Is (1.9) achieved by a continuous functionu0∈?Hσ(Rn)?
An affirmative answer would imply that problems (1.9) and (1.8) have identical minimizers (in the sense that the support of a minimizer of (1.9) is a minimizer of(1.8), and vice versa). The difficulty of proving this continuity is in estimating the energy difference betweenu0and its competitor whose support is slightly enlarged.See Section 7 for further discussion and additional open problems.
If one restricts the problem (1.3) to the class of convex sets, then we have

This paper is organized as follows. In Section 2, we recall some inequalities that are needed here. In Section 3, we prove the existence of minimizers for the convex case: this is much simpler, but illustrates the concepts in play. In Section 4, we reformulate the variational problem (1.8) as (1.9). In Section 5, we show a local pointwise upper bound on the minimizer. In Section 6, we show a lower bound of the minimizer, use it to estimate the diameter of the support ofu0, and conclude the proof of Theorem 1.1. In the last section, we discuss Open Question 1.2 and related topics.
Let us first recall a sharp Hardy inequality for fractional integrals on general domains. Let ??Rnbe an open set. For a directionω∈Sn?1, we define


where we used Theorem 2.1 in the last inequality.
We rewrite Lemma 2.1 into another form for convenience.

for every nonnegative function u∈?Hσ(Rn).
Proof.Without loss of generality, we assume that|Rn{u>0}|>0. We also assume that

giving the conclusion.
We first consider the special case of the minimizing problem among all convex sets.Proof of Theorem1.2.We will adapt the proof in Lin [17] for the Laplacian case.First, because of (2.1), we have

Let{?k}be a minimizing sequence. By John’s Lemma, we have that either (A)?kconverges in the Hausdorffdistance to a bounded convex set ?∞with|?∞|=1(we may assume that ?∞is open since the boundary of ?∞is of measure zero), or (B)there is a subsequence, still denoted ?k, such that ?kare contained in strips (after suitable rotations and translations, noting thatλ1,σ(?) is invariant under them) of form [?δk,δk]×[?Lk,Lk]n?1such thatδk →0+andLk →∞.

where ??is the Fourier transform of?, and we used the Plancherel theorem in the first equality. This proves (3.2).
Then we have

We reached a contradiction,and thus case(A)holds: ?kconverges in the Hausdorffdistance to a bounded convex open set ?∞with|?∞|=1.
Suppose that

This finishes the proof of this theorem.
In this section, we would like reformulate the variational problem (1.8) for domains to a variational problem for functions. Since||u(x)|?|u(y)||≤|u(x)?u(y)|,we know that the first eigenfunction, that are the solutions of (1.2), do not change signs in?. The next lemma states that they do not vanish in ?.

for every ?∈?Hσ(?). Then u is smooth and positive in?.
Proof.Foru,?∈?Hσ(?), we extenduand?to be identically zero in Rn?, and by Lemma 2.1, they are functions in ?Hσ(Rn). We can rewrite the integral as

Lemma 4.2.m(N)=inf{λ1,σ(?)+|?|:??BN is an open set}.
Proof.Let ??BNbe an open set,and letu∈?Hσ(?)be such thatλ1,σ(?)=In,σ,?[u],‖u‖L2=1. Then, by Lemma 4.1, we can choose thatu>0 in ?. Hence,

Taking the infimum over all open sets, we obtain


By Lemma 2.2,we know that{uk}is a bounded sequence in ?Hσ(Rn). Then passing to a subsequence, there existsu∈?Hσ(Rn) such thatuk ?uweakly in ?Hσ(Rn) anduk →ustrongly inL2(BN). Hence,u≥0 inBN,u≡0 in RnBNand‖u‖L2(BN)=1.Morevoer, by Fatou’s Lemma, we have

Thus,m(N) is attained byu.
Now we can show that the variational problem (1.8) is equivalent to (1.9).
Proposition 4.1.We have

Second, givenu∈?Hσ(Rn), it is elementary to check thatηNu∈?Hσ(Rn)andηNu→uin ?Hσ(Rn) asN →∞, whereηN(x)=η(x/N) andηis a standard radial cut-offfunction supported inB2and equal to 1 inB1. Hence,

which implies the conclusion.
Now we turn our attention to the regularity of minimizing functionsuform(N).At present,we only know that they exist and are nonnegative functions in the spaceHσ(Rn), supported onBN. In this section, we will show they are bounded and admit a kind of local maximum principle.

for every nonnegative function ?∈?Hσ(Rn).
Proof.Let?∈?Hσ(Rn) be a nonnegative function andut=(u?t?)+for any smallt>0. Then we have

Applying these two inequalities in (5.4), we obtain

withλas in (5.2).
Next is a Cacciopoli inequality, based onubeing a subsolution of this nonlocal equation. Set

Lemma 5.2.Let0 The integral over the second region can be estimated using the fact that the support ofηis a distanceR??from the complement ofBR: for all ε∈(0,1]. Proof.LetL>0, 0 Thus, we complete the proof. Remark 5.1.Theεin Lemma 5.3 interpolates between the local and nonlocal terms. It plays an essential role when we prove a lower bound foruin Section 6. Consequently, we have Theorem 5.1.There exists a positive constant C(n,σ)such that withλas in (5.2). The following corollary follows from Lemma 5.1 and Lemma 5.4, making precise the fact that a minimizerusolves the eigenvalue equation on its domain of positivity. Corollary 5.1.Let u∈?Hσ(Rn)be a nonnegative minimizer of m(N). Let ?∈?Hσ(Rn)such that where λ as in(5.2). We also can derive a lower bound for minimizers. The precise statement of it below takes on an unusual nonlocal form: it guarantees the existence of some (large) scaleR, possibly depending on the point considered, where supBR(x)uis at leastRσ. We note that this is not a “uniform” estimate (i.e., true for allR), nor do we expect a uniform estimate of this form to be valid: we believe that the exponentσhere is actually not the correct rate of growth forunear?{u>0}. Nonetheless,this estimate will be enough for concentration compactness arguments in proving existence below,while an optimal, uniform estimate appears to require new ideas. whereB·[·,·] is defined in (5.5). We may rewrite the right as wherev=u?w=(1?η)u. Now apply Lemma 5.4 withφ=vto give By choosingε=1/(2C(20)σ), andc?sufficiently small, this gives the promised estimate. This gives that the Lebesgue density of{u>0}at 0 is 0, which contradicts the assumptions.Corollary 6.1.Let u be as in Lemma6.1. There exists c=c(n,σ)such that for every Lebesgue point x of {u>0}. Proof.LetRandR?be the ones in Lemma 6.1. Assumex=0. Combining the results in Lemma 5.3 and Lemma 6.1, we have Proof.Select a representative ofuwhich vanishes except at Lebesgue points of{u>0}. By the Besicovitch covering theorem, from the collection of balls{B2(x):x∈{u>0}}, we can select a finite-overlapping subcover of{u>0},U={B2(xi)}i∈I.Together with Corollary 6.1, we have from above that LetM=#(I) be the number of balls inU: this gives thatM<∞and bounded in terms ofnandσonly. Assume thatuhas the following property:u=∑iui, whereuiare nonzero and|{ui>0}∩{uj>0}|=0 for alli/=j. Then we see that Let{Uj}j∈Jbe the connected components of the set∪i∈IB2(xi). If there is only one connected component, then the functionv(x)=u(x?x1) is supported onBM+1and we may conclude. If not, letuj=u|Ujand apply the above construction to find oneviwith energy less thanu. Note that as we have thatRi≥c, so a translate ofviwill be supported onB(M+1)/c. We use thisvito conclude. Now we are ready to prove Theorem 1.1. Proof of Theorem1.1.LetK(n,σ) be the one in Lemma 6.2, andN1,N2>K(n,σ).Letu1andu2be minimizers ofm(N1)andm(N2). Due to the translation invariance ofm(N),u1is a valid competitor ofu2form(N2), andu2is a valid competitor ofu1form(N1). Therefore,m(N1)=m(N2). Because of (4.2),u1is a desired minimzier. We know from Theorem 1.1 that the minimizing set is bounded with bounded support. A natural question is whether the minimizing eigenfunctionu0is continuous(and in particular,whether{u0>0}admits an open representative); this was stated as Open Question 1.2. We do not know how to prove this, and in this section, we would like to discuss the difficulties and present some related open questions. is a smooth function inB3/2and all its derivative are bounded inB3/2independent of ?. Then it follows from standard estimates for the fractional Laplacian in the whole space that for everyα∈(0,1), whereCdepends only onn, αandσ. In general, we letw(x)=v(x0+rx). Then where ~?=(??x)/rcontainingB2. Then we have Rescaling back, we have Thus, we complete the proof. Letu0be a minimizer in Theorem 1.1. Let us normalize it so that‖u0‖L2(Rn)=1,and hence,u0≤Cin Rn. Letx0∈Rn, R>0, and This is the difficulty of the above attempt of proving the minimizer’s global continuity. We do not know how to obtain such a sufficient decay estimate. If the solution to this open question is true, then it will lead to an affirmative answer to Open Question 1.2. The boundary behavior of solutions to homogeneous Dirichlet problems for the regional fractional Laplace equation (on sufficiently regular domains) was studied by Chen-Kim-Song [6], culminating in two-sided heat kernel estimates (and consequently, the two-sided Green’s function estimates for the regional fractional Laplacian in Proposition 4.2 of [10]). These results show that solutions on ? grow liked2s?1(·,??) from the boundary. It is natural, then, to ask whether such growth estimates can be established for our minimizersu0near?{u0>0}: Open Question 7.2.Does there existC>0 such that The point here is that we do not know that?{u0>0}is regular, so this is does not follow directly from[6]. A related question is whether there is a complementary uniformlowerbound on the growth of solutions: Open Question 7.3.Does there existc>0 such that for allR ∈(0,1) and allx∈?{u0>0}, This form of a lower bound is common in the free boundary literature. Note that the scaling here,R2s?1,is different from what was used in Lemma 6.1 to obtain a non-uniform lower bound: indeed, this difference in scaling explains why the estimate there could not be uniform inR. It also suggests new ideas, and possibly new competitor sets and functions, are needed to answer Open Questions 7.2 and 7.3, as most of the arguments in this paper do not detect thisR2s?1scaling. The Euler-Lagrange equation for the minimizer is another interesting open question. We have already seen thatu0satisfies but by analogy to the second order case [1] we expect another pointwise equation to be satisfied byu0along the boundary?{u0>0}, perhaps of the following form: Open Question 7.4.Does?{u0>0}have some approximate normal vectorsνxin an appropriate sense, and does Finally, let us suggest the following question about the shape of minimizers.Unlike the previous sequence of questions, a positive answer cannot be attained by local regularity or competitor arguments, and requires some new global approach.Open Question 7.5.Are minimizersu0radial and unique up to translations? Acknowledgements T. Jin is partially supported by Hong Kong RGC grants ECS 26300716 and GRF 16302519 and J. Xiong is partially supported by NSFC 11922104 and 11631002.Part of this work was completed while the second and third named authors were visiting The Hong Kong University of Science and Technology, to which they are grateful for providing very stimulating research environments and supports.






6 A bound from below and existence













7 Discussions and open questions











