1 Introduction

Let \(\Omega \) be a bounded open set in \(\mathbb {R}^n\), and \(k\) a nonnegative integer. We denote by \(W^k(\Omega )\) the Sobolev space

$$\begin{aligned} W^k(\Omega ) = \{ f\in L^2(\Omega ) \, : \, \partial ^\alpha f \in L^2(\Omega ) , \, |\alpha |\le k \}, \end{aligned}$$

where the derivatives are taken in the sense of distributions and endow the space with the norm

$$\begin{aligned} \Vert f\Vert _{k,\Omega } = \left( \sum _{|\alpha |\le k} \int _\Omega |\partial ^\alpha f |^2 \,d\lambda \right) ^{1/2}, \end{aligned}$$

where \(\alpha =(\alpha _1, \dots ,\alpha _n)\) is a multi-index , \(|\alpha |=\sum _{j=1}^n \alpha _j\) and

$$\begin{aligned} \partial ^\alpha f =\frac{\partial ^{|\alpha |}f}{\partial x_1^{\alpha _1}\dots \partial x_n^{\alpha _n}}. \end{aligned}$$

\(W^k(\Omega )\) is a Hilbert space. If \(\Omega \subset \mathbb {R}^n \, , \, n\ge 2\), is a bounded domain with a \(\mathcal {C}^1\) boundary, the Rellich–Kondrachov lemma says that for \(n>2\) one has

$$\begin{aligned} W^1(\Omega ) \subset L^r(\Omega ) \ , \ r\in [1, 2n/(n-2)) \end{aligned}$$

and that the imbedding is also compact; for \(n=2\) one can take \(r\in [1,\infty )\) (see, for instance, [4]); in particular, there exists a constant \(C_r\) such that

$$\begin{aligned} \Vert f\Vert _r \le C_r \Vert f\Vert _{1,\Omega }, \end{aligned}$$
(1.1)

for each \(f\in W^1(\Omega )\), where

$$\begin{aligned} \Vert f\Vert _r = \left( \int _\Omega |f|^r \, d\lambda \right) ^{1/r}. \end{aligned}$$

Now let \(\Omega \subseteq \mathbb {C}^n ( \cong \mathbb {R}^{2n} )\) be a smoothly bounded pseudoconvex domain. We consider the \(\overline{\partial }\)-complex

$$\begin{aligned} L^2(\Omega )\overset{\overline{\partial }}{\longrightarrow }L^2_{(0,1)}(\Omega ) \overset{\overline{\partial }}{\longrightarrow }\cdots \overset{\overline{\partial }}{\longrightarrow }L^2_{(0,n)}(\Omega )\overset{\overline{\partial }}{\longrightarrow }0\, , \end{aligned}$$
(1.2)

where \(L^2_{(0,q)}(\Omega )\) denotes the space of \((0,q)\)-forms on \(\Omega \) with coefficients in \(L^2(\Omega )\). The \(\overline{\partial }\)-operator on \((0,q)\)-forms is given by

$$\begin{aligned} \overline{\partial }\left( \sum _J\,^{'} a_J \, d\overline{z}_J \right) = \sum _{j=1}^n \sum _J\,^{'}\ \frac{\partial a_J}{\partial \overline{z}_j}d\overline{z}_j\wedge d\overline{z}_J, \end{aligned}$$
(1.3)

where \(\sum ^{'}\) means that the sum is only taken over strictly increasing multi-indices \(J\).

The derivatives are taken in the sense of distributions, and the domain of \(\overline{\partial }\) consists of those \((0,q)\)-forms for which the right-hand side belongs to \(L^2_{(0,q+1)}(\Omega )\). So \(\overline{\partial }\) is a densely defined closed operator, and therefore has an adjoint operator from \(L^2_{(0,q+1)}(\Omega )\) into \(L^2_{(0,q)}(\Omega )\) denoted by \(\overline{\partial }^*\).

We consider the \(\overline{\partial }\)-complex

$$\begin{aligned} L^2_{(0,q-1)}(\Omega )\underset{\underset{\overline{\partial }^* }{\longleftarrow }}{\overset{\overline{\partial }}{\longrightarrow }} L^2_{(0,q)}(\Omega ) \underset{\underset{\overline{\partial }^* }{\longleftarrow }}{\overset{\overline{\partial }}{\longrightarrow }} L^2_{(0,q+1)}(\Omega ), \end{aligned}$$
(1.4)

for \(1\le q \le n-1\).

We remark that a \((0,q+1)\)-form \(u=\sum _{J}^{'} u_J\,d\overline{z}_J\) belongs to \(\mathcal {C}^\infty _{(0,q+1)}(\overline{\Omega }) \cap {\text {dom}}(\overline{\partial }^*)\) if and only if

$$\begin{aligned} \sum _{k=1}^n u_{kK} \, \frac{\partial r}{\partial z_k} =0 \end{aligned}$$
(1.5)

on \(b\Omega \) for all \(K\) with \(|K|=q\), where \(r\) is a defining function of \(\Omega \) with \(|\nabla r(z)|=1\) on the boundary \(b\Omega \) (see, for instance, [14]).

The complex Laplacian \(\Box = \overline{\partial }\, \overline{\partial }^* + \overline{\partial }^*\, \overline{\partial }\), defined on the domain

$$\begin{aligned} {\text {dom}}(\Box ) = \{ u\in L^2_{(0,q)}(\Omega ) : u\in {\text {dom}}(\overline{\partial }) \cap {\text {dom}}(\overline{\partial }^*) , \overline{\partial }u\in {\text {dom}}(\overline{\partial }^*) , \overline{\partial }^* u \in {\text {dom}}(\overline{\partial }) \} \end{aligned}$$

acts as an unbounded, densely defined, closed and self-adjoint operator on \(L^2_{(0,q)}(\Omega )\), for \( 1\le q \le n\), which means that \(\Box = \Box ^*\) and \({\text {dom}}(\Box ) = {\text {dom}}(\Box ^*)\).

Note that

$$\begin{aligned} (\Box u,u)=( \overline{\partial }\, \overline{\partial }^* u+ \overline{\partial }^* \, \overline{\partial }u,u)=\Vert \overline{\partial }u \Vert ^2 + \Vert \overline{\partial }^* u \Vert ^2, \end{aligned}$$
(1.6)

for \(u\in {\text {dom}}(\Box )\).

If \(\Omega \) is a smoothly bounded pseudoconvex domain in \(\mathbb {C}^n\), the so-called basic estimate says that

$$\begin{aligned} \Vert \overline{\partial }u \Vert ^2 + \Vert \overline{\partial }^* u \Vert ^2 \ge c \, \Vert u\Vert ^2, \end{aligned}$$
(1.7)

for each \(u\in {\text {dom}}(\overline{\partial }) \cap {\text {dom}}(\overline{\partial }^* ) , \ c>0\).

This estimate implies that \(\Box : {\text {dom}}(\Box ) \longrightarrow L^2_{(0,q)}(\Omega )\) is bijective and has a bounded inverse

$$\begin{aligned} N: L^2_{(0,q)}(\Omega ) \longrightarrow {\text {dom}}(\Box ). \end{aligned}$$

\(N\) is called \(\overline{\partial }\)-Neumann operator. In addition

$$\begin{aligned} \Vert N u\Vert \le \frac{1}{c} \, \Vert u\Vert . \end{aligned}$$
(1.8)

A different approach to the \(\overline{\partial }\)-Neumann operator is related to the quadratic form

$$\begin{aligned} Q(u,v)= (\overline{\partial }u,\overline{\partial }v)+(\overline{\partial }^*u,\overline{\partial }^*v). \end{aligned}$$

For this purpose we consider the embedding

$$\begin{aligned} j : {\text {dom}}(\overline{\partial })\cap {\text {dom}}(\overline{\partial }^*) \longrightarrow L^2_{(0,q)}(\Omega ), \end{aligned}$$

where \({\text {dom}}(\overline{\partial })\cap {\text {dom}}(\overline{\partial }^*)\) is endowed with the graph-norm

$$\begin{aligned} u\mapsto \left( \Vert \overline{\partial }u\Vert ^2 + \Vert \overline{\partial }^*u \Vert ^2\right) ^{1/2}. \end{aligned}$$

The graph-norm stems from the inner product \(Q(u,v)\). The basic estimates (1.7) imply that \(j\) is a bounded operator with operator norm

$$\begin{aligned} \Vert j\Vert \le \frac{1}{\sqrt{c}}. \end{aligned}$$

By (1.7) it follows in addition that \({\text {dom}}(\overline{\partial })\cap {\text {dom}}(\overline{\partial }^*)\) endowed with the graph-norm \(u\mapsto (\Vert \overline{\partial }u\Vert ^2 + \Vert \overline{\partial }^*u \Vert ^2)^{1/2}\) is a Hilbert space.

The \(\overline{\partial }\)-Neumann operator \(N\) can be written in the form

$$\begin{aligned} N= j \circ j^*; \end{aligned}$$
(1.9)

details may be found in [14].

2 Compactness and Sobolev Inequalities

Here we apply a general characterization of compactness of the \(\overline{\partial }\)-Neumann operator \(N\) using a description of precompact subsets in \(L^2\)-spaces (see [10]).

Theorem 2.1

Let \(\Omega \subset \subset \mathbb {C}^n\) be a smoothly bounded pseudoconvex domain. The \(\overline{\partial }\)-Neumann operator \(N\) is compact if and only if for each \(\epsilon >0\) there exists \(\omega \subset \subset \Omega \) such that

$$\begin{aligned} \int _{\Omega \setminus \omega } |u(z)|^2\,d\lambda (z) \le \epsilon \left( \Vert \overline{\partial }u\Vert ^2 + \Vert \overline{\partial }^* u\Vert ^2\right) \end{aligned}$$

for each \(u\in dom\,(\overline{\partial }) \,\cap dom\,(\overline{\partial }^*)\).

Now let

$$\begin{aligned} \mathcal {W}^1_{(0,q)} (\Omega ):= \{ u\in L^2_{(0,q)}(\Omega ) : u\in dom\,(\overline{\partial }) \,\cap dom\,(\overline{\partial }^*) \} \end{aligned}$$

endowed with graph norm. As already mentioned above, this “complex” version of a Sobolev space \(\mathcal {W}^1_{(0,q)} (\Omega )\) is a Hilbert space.

It appears to be interesting to compare the standard Sobolev imbedding

$$\begin{aligned} W^1(\Omega ) \subset L^r(\Omega ) \ , \ r\in [1, 2n/(n-1)) \end{aligned}$$

where the derivatives are taken with respect of the real variables \(x_j =\mathfrak {R}z_j\) and \(y_j=\mathfrak {I}z_j\) for \(j=1,\dots ,n\), with the imbedding of the space \(\mathcal {W}^1_{(0,q)} (\Omega )\) endowed with graph norm, into \(L^r_{(0,q)}(\Omega )\). We have the following result.

Theorem 2.2

If \(\Omega \subset \subset \mathbb {C}^n\) is a smoothly bounded pseudoconvex domain and the inequality

$$\begin{aligned} \Vert u\Vert _r \le C ((\Vert \overline{\partial }u \Vert ^2 + \Vert \overline{\partial }^*u\Vert ^2)^{1/2} \end{aligned}$$
(2.1)

for some \(r>2\) and for all \(u\in dom\,(\overline{\partial }) \,\cap dom\,(\overline{\partial }^*)\) holds, then the \(\overline{\partial }\)-Neumann operator

$$\begin{aligned} N: L^2_{(0,q)}(\Omega ) \longrightarrow L^2_{(0,q)}(\Omega ) \end{aligned}$$

is compact.

Proof

To show this we have to check that the unit ball in \(\mathcal {W}^1_{(0,q)} (\Omega )\) is precompact in \(L^2_{(0,q)}(\Omega )\). By Proposition 2.1, we have to show that for each \(\epsilon >0\) there exists \(\omega \subset \subset \Omega \) such that

$$\begin{aligned} \int _{\Omega \setminus \omega } |u(z)|^2 \, d\lambda (z) < \epsilon ^2, \end{aligned}$$

for all \(u\) in the unit ball of \(\mathcal {W}^1_{(0,q)} (\Omega )\).

By (2.1) and Hölder’s inequality we have

$$\begin{aligned} \left( \int _{\Omega \setminus \omega } |u(z)|^2 \, d\lambda (z) \right) ^{\frac{1}{2}}&\le \left( \int _{\Omega \setminus \omega } |u(z)|^r \, d\lambda (z)\right) ^{\frac{1}{r}} \cdot |\Omega \setminus \omega |^{\frac{1}{2}-\frac{1}{r}} \\&\le C \, |\Omega \setminus \omega |^{\frac{1}{2}-\frac{1}{r}}. \end{aligned}$$

Now we can choose \(\omega \subset \subset \Omega \) such that the last term is \(< \epsilon \). \(\square \)

In the following theorem we suppose that a so-called subelliptic estimate holds. Subelliptic estimates are related to the geometric notion of finite type. We remark that the \(\overline{\partial }\)-Neumann problem for smoothly bounded strictly pseudoconvex domains is subelliptic with a gain of one derivative for \(N\) which is considerably stronger than compactness.

Theorem 2.3

Let \(\Omega \) be a bounded pseudoconvex domain in \(\mathbb {C}^n\) with boundary of class \(\mathcal {C}^\infty \). Suppose that \(0< \epsilon \le 1/2\) and that

$$\begin{aligned} dom\,(\overline{\partial }) \,\cap dom\,(\overline{\partial }^*) \subseteq W^{\epsilon }_{(0,q)} (\Omega ), \end{aligned}$$

and that there exists a constant \(C>0\) such that

$$\begin{aligned} \Vert u \Vert _{\epsilon , \Omega } \le C \left( \Vert \overline{\partial }u \Vert ^2 + \Vert \overline{\partial }^* u\Vert ^2\right) ^{1/2}, \end{aligned}$$
(2.2)

for all \(u \in dom\,(\overline{\partial }) \,\cap dom\,(\overline{\partial }^*)\), where \(W^{\epsilon }_{(0,q)} (\Omega )\) is the standard \(\epsilon \)-Sobolev space. Then the \(\overline{\partial }\)-Neumann operator

$$\begin{aligned} N: L^2_{(0,q)}(\Omega ) \longrightarrow L^2_{(0,q)}(\Omega ) \end{aligned}$$

is compact and \(N\) can be continuously extended as an operator

$$\begin{aligned} \tilde{N}: L^{\frac{2n}{n+\epsilon }}_{(0,q)}(\Omega ) \longrightarrow L^{\frac{2n}{n-\epsilon }}_{(0,q)}(\Omega ), \end{aligned}$$

which means that there is a constant \(C>0\) such that

$$\begin{aligned} \Vert \tilde{N} u \Vert _{\frac{2n}{n-\epsilon }} \le C \, \Vert u \Vert _{\frac{2n}{n+\epsilon }}, \end{aligned}$$
(2.3)

for each \(u\in L^{\frac{2n}{n+\epsilon }}_{(0,q)}(\Omega )\).

Proof

We use the continuous imbedding for the space \(W^{\epsilon }(\Omega ):\)

$$\begin{aligned} W^{\epsilon }(\Omega ) \longrightarrow L^r(\Omega ), \end{aligned}$$

for \(2\le r \le 2n/(n-\epsilon )\); see [1], Theorem 7.57. Hence we can choose \(r_0 >2\) to get

$$\begin{aligned} dom\,(\overline{\partial }) \,\cap dom\,(\overline{\partial }^*) \subseteq W^{\epsilon }_{(0,q)} (\Omega ) \subseteq L^{r_0}_{(0,q)} (\Omega ), \end{aligned}$$

and we can apply Theorem 2.2.

To show that \(N\) extends continuously recall that \(N=j \circ j^*\), where

$$\begin{aligned} j: dom\,(\overline{\partial }) \,\cap dom\,(\overline{\partial }^*) \longrightarrow L^2_{(0,q)}(\Omega ), \end{aligned}$$

see [14]. In our case \(j\) is a continuous operator into \(L^{\frac{2n}{n-\epsilon }}_{(0,q)}(\Omega )\), hence

$$\begin{aligned} j^* : L^{\frac{2n}{n+\epsilon }}_{(0,q)}(\Omega ) \longrightarrow dom\,(\overline{\partial }) \,\cap dom\,(\overline{\partial }^*), \end{aligned}$$

which proves the assertion. \(\square \)

Krantz [12], Beals et al. [2], Lieb and Range [13], and Bonami and Sibony [3] proved \(L^p\)-estimates and Lipschitz estimates for solution operators of the inhomogeneous \(\overline{\partial }\)-equation and the \(\overline{\partial }\)-Neumann operator using integral representations for the kernel of these operators, but without relationship to compactness and continuous extendability.

Remark 2.4

If \(\Omega \) is a bounded strictly pseudoconvex domain in \(\mathbb {C}^n\) with boundary of class \(\mathcal {C}^\infty \), then (2.2) is satisfied for \(\epsilon = 1/2\) (see [14], Proposition 3.1).

D’Angelo ([8, 9]) and Catlin [5–7]) give a characterization of when a subelliptic estimate holds in terms of the geometric notion of finite type; see also [14].

Corollary 2.5

Let \(\Omega \) be a smooth bounded pseudoconvex domain in \(\mathbb {C}^n, \ n \ge 2\). Let \(P \in b\Omega \) and assume that there is an m-dimensional complex manifold \(M\subset b\Omega \) through \(P \ (m\ge 1)\), and \(b\Omega \) is strictly pseudoconvex at \(P\) in the directions transverse to \(M\) (this condition is void when \(n=2\)). Then (2.1) is not satisfied for \((0,q)\)-forms with \(1\le q \le m\).

Proof

Theorem 4.21 of [14] gives that the \(\overline{\partial }\)-Neumann operator fails to be compact on \((0,q)\)-forms with \(1\le q \le m\). Hence we can again apply Proposition 2.2 to get the desired result. \(\square \)

Remark 2.6

If the Levi form of the defining function of \(\Omega \) is known to have at most one degenerate eigenvalue at each point (the eigenvalue zero has multiplicity at most 1), a disk in the boundary is an obstruction to compactness of \(N\) for \((0,1)\)-forms. A special case of this is implicit in [11] for domains fibered over a Reinhardt domain in \(\mathbb {C}^2\).