Compactness of Hankel Operators with Continuous Symbols

Abstract

Let $\Omega$ be a bounded convex Reinhardt domain in $\mathbb {C}^2$ and $\phi \in C({\overline{\Omega }})$. We show that the Hankel operator $H_{\phi }$ is compact if and only if $\phi$ is holomorphic along every non-trivial analytic disc in the boundary of $\Omega$.

Let $\Omega$ be a domain in $\mathbb {C}^n$ and let $L^2(\Omega )$ and $A^2(\Omega )$ denote square integrable functions on $\Omega$ and the Bergman space on $\Omega$ (the set of square integrable holomorphic functions on $\Omega$), respectively. Since $A^2(\Omega )$ is a closed subspace in $L^2(\Omega )$ the Bergman projection $P:L^2(\Omega )\rightarrow A^2(\Omega )$, the orthogonal projection, exists. Furthermore, let $H_{\phi }f=(I-P)(\phi f)$ for all $f\in A^2(\Omega )$ and $\phi \in L^{\infty }(\Omega )$. We note that $H_{\phi }$ is called the Hankel operator with symbol $\phi$. We refer the reader to [9, 11] and references there in for more information on these operators.

Hankel operators form an active research area in operator theory. Our interest lies in their compactness properties in relation to the behavior of the symbols on the boundary of the domain. On the unit disc $\mathbb {D}$ in $\mathbb {C}$ Axler [1] showed that, for f holomorphic on the unit disc $\mathbb {D}$, the Hankel operator $H_{{\overline{f}}}$ is compact on $A^2(\mathbb {D})$ if and only if f is in the little Bloch space (that is, $(1-|z|^2)|f'(z)|\rightarrow 0$ as $|z|\rightarrow 1$). This result has been extended into higher dimensions by Peloso [8] in case the domain is smooth bounded and strongly pseudoconvex. The same year, Li [7] characterized bounded and compact Hankel operators on strongly pseudoconvex domains for symbols that are square integrable only. Recently, Čučković and Şahutoğlu [2, Theorem 3] gave a characterization for compactness of Hankel operators on smooth bounded convex domains in $\mathbb {C}^2$ with symbols smooth up to the boundary. We note that even though they stated their result for smooth domains and smooth symbols on the closure, examination of the proof shows that $C^1$-smoothness of the domain and the symbol is sufficient. They proved the following theorem.

FormalPara Theorem

(Čučković–Şahutoğlu) Let $\Omega$ be a $C^1$-smooth bounded convex domain in $\mathbb {C}^2$ and $\phi \in C^1({\overline{\Omega }})$. Then the Hankel operator $H_{\phi }$ is compact on $A^2(\Omega )$ if and only if $\phi \circ f$ is holomorphic for any holomorphic function $f:\mathbb {D}\rightarrow b\Omega$.

In this paper we prove a similar result with symbols that are only continuous up to the boundary. The first result in this direction was proven by Le in [6]. He showed that for $\Omega =\mathbb {D}^n$, the polydisc in $\mathbb {C}^n$, and $\phi \in C({\overline{\Omega }})$, the Hankel operator $H_{\phi }$ is compact on $A^2(\Omega )$ if and only if $\phi =f+g$ where f and g are continuous on ${\overline{\Omega }},f=0$ on $b\Omega$, and g is holomorphic on $\Omega$. We prove the following theorem, generalizing Le’s result in $\mathbb {C}^2$.

FormalPara Theorem 1

Let $\Omega$ be a bounded convex Reinhardt domain in $\mathbb {C}^2$ and $\phi \in C({\overline{\Omega }})$. Then the Hankel operator $H_{\phi }$ is compact on $A^2(\Omega )$ if and only if $\phi \circ f$ is holomorphic for any holomorphic function $f:\mathbb {D}\rightarrow b\Omega$.

We note that in the theorem above there is no regularity restriction on the domain, but the class of domains is smaller than the one considered in [2]. It would be interesting to know if the same result is still true on convex domains in $\mathbb {C}^n$.

1 Proof of Theorem 1

Let us start by some notation. We denote

$$\begin{aligned} \mathbb {D}_r= & {} \{z\in \mathbb {C}:|z|<r\}, S_r=\{z\in \mathbb {C}: |z|=r\}, \\&A(0,\delta _1,\delta _2)=\{z\in \mathbb {C}: \delta _1<|z|<\delta _2\} \end{aligned}$$

for $r, \delta _1,\delta _2>0$.

In the next lemma we prove that any analytic disc $\Delta _0\subset b\Omega$ is contained in a disc that intersects the coordinate axis. This allows us to simplify the problem for convex Reinhardt domains, since any disc in $b\Omega$ must be horizontal or vertical.

Lemma 1

Let $\Omega$ be a bounded convex Reinhardt domain in $\mathbb {C}^2$ and $\Delta \subset b\Omega$ be a non-trivial analytic disc. Then there exists $r>0$ and $p\in \mathbb {C}$ such that either $\Delta \subset \mathbb {D}_r\times \{p\} \subset b\Omega$ or $\Delta \subset \{p\} \times \mathbb {D}_r\subset b\Omega$.

Proof

Suppose that $F(\mathbb {D})=\Delta$ is a non-trivial disc in $b\Omega$ where $F(\xi )=(f(\xi ),g(\xi ))$. Then either $f'g'\equiv 0$ or there exists $\xi _0\in \mathbb {D}$ such that $f'(\xi _0)g'(\xi _0)\ne 0$. In case $f'g'\equiv 0$, by identity principle, we conclude that either $f'\equiv 0$ or $g'\equiv 0$. That is, either f or g is constant.

On the other hand, if $f'(\xi _0)g'(\xi _0)\ne 0$ then the disc $\Delta$ is a smooth complex curve in a neighborhood $F(\xi _0)$. Furthermore, the fact that $\Omega$ is Reinhardt domain in $\mathbb {C}^2$ implies that $b\Omega$ is smooth locally in a neighborhood of $F(\xi _0)$. This can be seen as follows: Without loss of generality we assume that $f(\xi _0)\ne 0$. Let $\xi _0=x_0+iy_0$ and

$$\begin{aligned} G(x,y,\theta )=(e^{i\theta }f(x+iy),g(x+iy)). \end{aligned}$$

Then one can show that the image of G is a smooth surface in $\mathbb {C}^2$ near $G(\xi _0, 0)=F(\xi _0)$ as the Jacobian of G is of rank 3 at $(\xi _0,0)$. Since $b\Omega$ is a 3 dimensional surface we conclude that the boundary of $\Omega$ is smooth near near $F(\xi _0)$ as it can be seen as the image of $G(x,y,\theta )$. Then we can apply [3, Lemma 2] (since $b\Omega$ is smooth near $F(\xi _0)$) and use the identity principle to conclude that either f or g is constant. We reach a contradiction with the assumption that $f(\xi _0)\ne 0$. Therefore, either $\Delta$ is flat and horizontal (g is constant) or flat and vertical (f is constant).

For the rest of the proof, without loss of generality, we assume that $\Delta$ is horizontal. There exists $p\in \mathbb {C}$, $\delta _1> 0$, and $\delta _2>0$ such that

$$\begin{aligned} A(0,\delta _1,\delta _2)\times \{p\}\subset b\Omega . \end{aligned}$$

The assumption that $\Omega$ is convex and Reinhardt implies that $\Omega$ is complete. So,

$$\begin{aligned} \{(z,w)\in \mathbb {C}^2: |z|\le \delta _2, |w|\le |p|\}\subset {\overline{\Omega }}. \end{aligned}$$

(1)

Next, we will show that $\{(z,w)\in \mathbb {C}^2:|z|\le \delta _1, |w|>|p|\}\cap \Omega =\emptyset$. Suppose that there exists $(z_0,w_0)\in \{(z,w)\in \mathbb {C}:|z|\le \delta _1, |w|>|p|\}\cap \Omega$ and let $z\in \mathbb {C}$ such that $|z|=\delta _2$. We choose $\lambda >0$ small enough such that $(|z|-\lambda ,|p|-\lambda )\in \Omega$ and the line segment joining $(|z|-\lambda ,|p|-\lambda )$ with $(z_0,w_0)$, called $L_1$, is such that

$$\begin{aligned} L_1\cap (A(0,\delta _1,\delta _2)\times \{|p|e^{i\theta }:0\le \theta \le 2\pi \})\ne \emptyset . \end{aligned}$$

However, since $\overline{A(0,\delta _1,\delta _2)}\times \{|p|e^{i\theta }:0\le \theta \le 2\pi \}\subset b\Omega$, we conclude $L_1\cap b\Omega \ne \emptyset$. Since the initial and terminal points of $L_1$ lie in $\Omega$ and $\Omega$ is convex, we arrive at a contradiction. This shows that $\{(z,w)\in \mathbb {C}^2:|z|\le \delta _1, |w|>|p|\}\cap \Omega =\emptyset$. Combining this with (1) we conclude that $\{(z,w)\in \mathbb {C}^2: |z|\le \delta _2, |w|=|p|\}\subset b\Omega .$ $\square$

We take this opportunity to correct a typo in [3, Lemma 2]. In the statement of the lemma, the word “complete” should be “convex”. The lemma is proven for the correct domains: piecewise smooth bounded convex Reinhardt domains in $\mathbb {C}^2$.

Remark 1

Lemma 1 implies that if $\Omega \subset \mathbb {C}^2$ is a bounded convex Reinhardt domain, then any horizontal analytic disc in $b\Omega$ is contained in $\mathbb {D}_r\times S_{q}$ for some $r>0$ and $q>0$. Likewise, any vertical analytic disc in $b\Omega$ is contained in $S_{q'}\times \mathbb {D}_{r'}$ for some $r'>0$ and $q'>0$.

As in [3] we represent a complete Reinhardt domain $\Omega \subset \mathbb {C}^2$ as union of horizontal slices. In other words, let $H_{\Omega }$ be an open disc in $\mathbb {C}$ such that

$$\begin{aligned} \Omega =\bigcup _{w\in H_{\Omega }}\Delta _w\times \{w\} \end{aligned}$$

(2)

where $\Delta _w=\{z\in \mathbb {C}:|z|<r_w\}$ is the slice of $\Omega$ at w level. That is, $(z,w)\in \Omega$ if and only if $|z|<r_w$.

Lemma 2

([3]) Let $\phi \in C(\mathbb {C})$ and $f:\mathbb {C}\rightarrow \mathbb {C}$ be an entire function. Then

$$\begin{aligned} \Vert H_{\phi }^{\mathbb {D}_r} f\Vert _{L^2(\mathbb {D}_r)}\rightarrow \Vert H_{\phi }^{\mathbb {D}_{r_0}} f\Vert _{L^2(\mathbb {D}_{r_0})} \end{aligned}$$

as $r\rightarrow r_0$.

Lemma 1, Lemma 2, and [3, Lemma 3] imply the following corollary.

Corollary 1

Let $\Omega$ be a bounded convex Reinhardt domain in $\mathbb {C}^2, \phi \in C(\mathbb {C})$, and $\Delta _{w_0}\times \{w_0\}$ be a non-trivial analytic disc in $b\Omega$ where $w_0\in bH_{\Omega }$. Then

$$\begin{aligned} \lim _{H_{\Omega }\ni w\rightarrow w_0}\Vert H^{\Delta _w}_{\phi }(1)\Vert _{L^2(\Delta _w)} =\Vert H^{\Delta _{w_0}}_{\phi }(1)\Vert _{L^2(\Delta _{w_0})}. \end{aligned}$$

Lemma 3

Let $\Omega$ be a bounded convex Reinhardt domain in $\mathbb {C}^2$ and $\phi \in C({\overline{\Omega }})$. Furthermore, let $w_0\in bH_{\Omega }$ and $\phi _0(z,w)=\phi (z,w_0)$. Assume that $H_{\phi }$ is compact on $A^2(\Omega )$ and $\{g_j\}$ is a bounded sequence in $A^2(H_{\Omega })$ such that $g_j\rightarrow 0$ uniformly on $H_{\Omega }{\setminus } V$ as $j\rightarrow \infty$ for any open set V containing $w_0$. Then $H_{\phi _0}g_j\rightarrow 0$ as $j\rightarrow \infty$.

Proof

We note that $g_j\rightarrow 0$ weakly in $A^2(\Omega )$ as $j\rightarrow 0$. Hence, by compactness of $H_{\phi }$ we have $\Vert H_{\phi }g_j\Vert _{L^2(\Omega )}\rightarrow 0$ as $j\rightarrow \infty$. Now, we write

$$\begin{aligned} \Vert H_{\phi _0}g_j\Vert _{L^2(\Omega )}\le \Vert H_{\phi -\phi _0}g_j\Vert _{L^2(\Omega )}+\Vert H_{\phi }g_j\Vert _{L^2(\Omega )}. \end{aligned}$$

So, we just consider the first term on the right hand side of the above inequality. Since $\{g_j\}$ is a bounded sequence, there exists $M>0$ such that $\Vert g_j\Vert ^2_{L^2(\Omega )}\le M$ for all $j\in \mathbb {N}$. Furthermore, since $\phi -\phi _0$ is continuous on ${\overline{\Omega }}$ and $\phi -\phi _0=0$ on $\overline{\Delta _{w_0}}$ for all $\varepsilon >0$, there exists $\delta >0$ such that

$$\begin{aligned} \sup \{|\phi (z,w)-\phi _0(z,w)|^2:(z,w)\in {\overline{\Omega }}, |w-w_0|\le \delta \}<\frac{\varepsilon }{2M}. \end{aligned}$$

We note that, below, $V(\Omega )$ denotes the volume of $\Omega$ with respect to Lebesgue measure.

$$\begin{aligned} \Vert H_{\phi -\phi _0}g_j\Vert ^2_{L^2(\Omega )} \le&\,\Vert ({\phi -\phi _0})g_j\Vert ^2_{L^{2}(\{(z,w)\in \Omega : |w-w_0|\le \delta \})}\\&+V(\Omega )\Vert ({\phi -\phi _0})g_j\Vert ^2_{L^{\infty }(\{(z,w)\in \Omega : |w-w_0|>\delta \})}\\ <&\,\frac{\varepsilon }{2} +V(\Omega )\Vert ({\phi -\phi _0})g_j\Vert ^2_{L^{\infty }(\{(z,w)\in \Omega : |w-w_0|>\delta \})}. \end{aligned}$$

Since $(\phi -\phi _0)\in C({\overline{\Omega }})$ and $g_j\rightarrow 0$ uniformly on $\{(z,w)\in \Omega : |w-w_0|>\delta \}$ as $j\rightarrow \infty$, we conclude that for any $\delta ,\varepsilon >0$ there exists $j_0\in \mathbb {N}$ such that

$$\begin{aligned} V(\Omega )\Vert ({\phi -\phi _0})g_j\Vert ^2_{L^{\infty }(\{(z,w)\in \Omega : |w-w_0|>\delta \})}<\frac{\varepsilon }{2} \end{aligned}$$

for $j\ge j_0$. Therefore,

$$\begin{aligned} \Vert H_{\phi -\phi _0}g_j\Vert ^2_{L^2(\Omega )}<\varepsilon \end{aligned}$$

for $j\ge j_0$ and the proof of the lemma is complete. $\square$

Before we state the next lemma some explanation about the notation is in order. We think of the operators as defined on spaces on $\Omega$ unless the domain is indicated as a superscript. For instance, for an open subset V of $\Omega$ the operators $H_{\phi }^{V}$ and $P^{V}$ are defined on $A^2(V)$ and $L^2(V)$, respectively; whereas, $H_{\phi }$ and P are defined on $A^2(\Omega )$ and $L^2(\Omega )$, respectively. Furthermore, in the next two lemmas, we think of $\phi$ as a function of z (or as a function of (z, w) but independent of w). For instance, $\phi$ is a function of z in $H^{\Delta _w}_{\phi }$ and a function (z, w) (but independent of w) in $H_{\phi }$.

The following lemma is a special case of equation (3) in [3, pg. 637] for $\phi =\psi _0=\phi _0$ and $f_1= f_2\equiv 1$.

Lemma 4

([3]) Let $\Omega$ be a bounded convex Reinhardt domain in $\mathbb {C}^2$ and $\phi \in C({\overline{\Omega }})$ such that $\phi (z,w)=\phi (z,0)$ for $(z,w)\in \Omega$. Then

$$\begin{aligned} \Vert H_{\phi }g\Vert ^2_{L^2(\Omega )} =&\int _{H_{\Omega }}|g(w)|^2\int _{\Delta _w}|H^{\Delta _w}_{\phi }(1)(z)|^2dV(z)dV(w)\\&+\int _{\Omega }(H_{\phi }g)(z,w)\overline{P^{\Delta _w}(\phi )(z)g(w)}dV(z,w) \end{aligned}$$

for $g\in A^2(H_{\Omega })$

Similarly the following lemma is included in [3, pg 640] again for $\phi =\psi _0=\phi _0$ and $f_1= f_2\equiv 1$.

Lemma 5

([3]) Let $\Omega$ be a bounded convex Reinhardt domain in $\mathbb {C}^2$ and $\phi \in C({\overline{\Omega }})$ such that $\phi (z,w)=\phi (z,0)$ for $(z,w)\in \Omega$. Assume that $\{g_j\}$ is a bounded sequence in $A^2(H_{\Omega })$ such that $g_j\rightarrow 0$ uniformly on ${\overline{H}}_{\Omega }{\setminus } V$ for any open set V containing $w_0\in H_{\Omega }$. Then

$$\begin{aligned} \int _{\Omega }(H_{\phi }g_j)(z,w)\overline{P^{\Delta _w}(\phi )(z)g_j(w)}dV(w,z) \rightarrow 0 \quad \text { as }j\rightarrow \infty . \end{aligned}$$

The next lemma allows us to approximate the symbol with smooth appropriate symbols. We define $\Gamma _{\Omega }\subset b\Omega$ to be the closure of the union of all non-trivial analytic discs in $b\Omega$. That is,

$$\begin{aligned} \Gamma _{\Omega }=\overline{\bigcup \{f(\mathbb {D}):f:\mathbb {D}\rightarrow b\Omega \text { is non-constant holomorphic mapping}\}}. \end{aligned}$$

(3)

Lemma 6

Let $\Omega$ be a bounded convex Reinhardt domain in $\mathbb {C}^2$ that is not the product of two discs. Assume that $\Gamma _{\Omega }\ne \emptyset$ and $\phi \in C({\overline{\Omega }})$ such that $\phi \circ f$ is holomorphic for any holomorphic function $f:\mathbb {D}\rightarrow b\Omega$. Then there exists $\{\psi _n\}\subset C^{\infty }({\overline{\Omega }})$ such that

• i. $\psi _n\circ f$ is holomorphic for all n and for any holomorphic function $f:\mathbb {D}\rightarrow b\Omega$,

• ii. $\Vert \psi _n-\phi \Vert _{L^{\infty }(\Gamma _{\Omega })}\rightarrow 0$ as $n\rightarrow \infty$.

Proof

Let $\Delta _1=\mathbb {D}_{r_1}\times S_{s_1}$ be the family of horizontal analytic discs in $b\Omega$ as outlined in Lemma 1. Then for $0<r<1$ we define

$$\begin{aligned} \phi _{r}(z,w)=\phi (rz,w). \end{aligned}$$

Since $\phi \in C({\overline{\Omega }})$, one can show that

$$\begin{aligned} \phi _r\rightarrow \phi \text { uniformly on } {\overline{\Omega }}\text { as } r\rightarrow 1^-. \end{aligned}$$

We consider $\phi$, restricted to $\overline{\Delta _1}$, to be a function of $(z,\theta )$ for $z\in \overline{\mathbb {D}}_{r_1}$ and periodic in $\theta \in \mathbb {R}$ with period $2\pi$. By assumption, the function $\phi _r(.,\theta )$ is holomorphic on a neighborhood of $\overline{\mathbb {D}}_{r_1}$ for every $\theta \in \mathbb {R}$. Let $\gamma \in C^{\infty }_0( (-1,1))$ be such that $\gamma \ge 0$ and $\int _{-1}^{1}\gamma (\theta )d\theta =1$. Similarly, let $\chi \in C^{\infty }_0(\mathbb {D}_{r_1})$ be such that $\chi \ge 0$ and $\int _{\mathbb {D}_{r_1}}\chi (z)dV(z)=1$. Now, we define $\gamma _{\delta }(\theta )=\delta ^{-1}\gamma (\theta /\delta )$ and $\chi _{\varepsilon }(z)=\varepsilon ^{-2}\chi (z/\varepsilon )$. Notice that $\{\gamma _{\delta }\}_{\delta >0}$ and $\{\chi _{\varepsilon }\}_{\varepsilon >0}$ are approximate identities. We define the convolution

$$\begin{aligned} C_{r,\,\varepsilon }^{\phi }(z,\theta ) =\int _{-\pi }^{\pi }\int _{\mathbb {D}_{r_1}}\phi (r(z-\alpha ),(\theta -\theta ')) \chi _{\varepsilon }(\alpha )\gamma _{\varepsilon }(\theta ')dV(\alpha )d\theta '. \end{aligned}$$

One can show that for $\varepsilon >0$ sufficiently small (depending on r) the function $C_{r,\varepsilon }^{\phi }(.,\theta )$ is holomorphic on a neighborhood of $\overline{\mathbb {D}}_{r_1}$ for every $\theta \in \mathbb {R}$. Also the assumption that $\phi \in C({\overline{\Omega }})$ implies that

$$\begin{aligned} C_{r,\,\varepsilon }^{\phi }\rightarrow \phi _r \text { uniformly on }\overline{\Delta _1} \text { as } \varepsilon \rightarrow 0^+ \end{aligned}$$

for all $0<r<1$. Therefore, the functions $C_{r,\,\varepsilon }^{\phi }$ are holomorphic “along” horizontal analytic discs in $b\Omega$ for small $\varepsilon >0$. Now, we extend $C_{r,\,\varepsilon }^{\phi }$ as a $C^{\infty }$-smooth function onto ${\overline{\Omega }}$ and call this extension ${\widetilde{C}}_{r,\,\varepsilon }^{\phi }$.

If $b\Omega$ contains non-trivial vertical analytic discs $\Delta _2$ then we can use a similar construction on $\Delta _2$. That is, using the regularization procedure outlined above in this proof, we can construct a collection of functions ${\widetilde{B}}_{r,\varepsilon }^{\phi }\in C^{\infty }({\overline{\Omega }})$ such that ${\widetilde{B}}_{r,\,\varepsilon }^{\phi }$ are holomorphic “along” any vertical analytic disc in $\Delta _2$ for small $\varepsilon >0$ and

$$\begin{aligned} {\widetilde{B}}_{r,\varepsilon }^{\phi }\rightarrow \phi _r \text { uniformly on } {\overline{\Delta }}_2 \text { as } \varepsilon \rightarrow 0^+ \end{aligned}$$

for all $0<r<1$. Since $\Omega$ is not the product of discs, (hence $\overline{\Delta _1}\cap \overline{\Delta _2}=\emptyset$), there exists open sets F and G such that $\overline{\Delta _1}\subset F$, $\overline{\Delta _2}\subset G$, and ${\overline{F}}\cap {\overline{G}}=\emptyset$. Then we choose $\chi _F,\chi _G\in C^{\infty }_0(\mathbb {C}^2)$ such that $0\le \chi _G,\chi _F\le 1$, $\chi _G\equiv 1$ on $G, \chi _F\equiv 1$ on F, and $\chi _F+\chi _G\equiv 1$ on ${\overline{\Omega }}$.

We define

$$\begin{aligned} \phi ^{r,\,\varepsilon }=\chi _F {\widetilde{C}}_{r,\,\varepsilon }^{\phi }+\chi _G {\widetilde{B}}_{r,\,\varepsilon }^{\phi }. \end{aligned}$$

(4)

By construction, $\chi _F\equiv 0$ on G and $\chi _G\equiv 0$ on F. Furthermore, ${\widetilde{C}}_{r,\,\varepsilon }^{\phi }$ is holomorphic along $\Delta _1$, and ${\widetilde{B}}_{r}^{\phi }$ is holomorphic along $\Delta _2$ for small $\varepsilon >0$. For $n=1,2,\ldots$ we choose $r_n=(n-1)/n$ and $\varepsilon _n\rightarrow 0^+$ so that

• i. $\phi ^{r_n,\,\varepsilon _n}\circ h$ is holomorphic for all n and every holomorphic $h:\mathbb {D}\rightarrow b\Omega$,

• ii. $\phi ^{r_n,\,\varepsilon _n}\rightarrow \phi$ uniformly on $\Gamma _{\Omega }$ as $n\rightarrow \infty$.

Finally, we finish the proof by defining $\psi _n=\phi ^{r_n,\,\varepsilon _n}$.$\square$

Let X and Y be two normed linear spaces and $T:X\rightarrow Y$ be a bounded linear operator. We define the essential norm of T, denoted by $\Vert T\Vert _e$, as

$$\begin{aligned} \Vert T\Vert _e=\inf \{\Vert T-K\Vert :K:X\rightarrow Y \text { is a compact operator}\} \end{aligned}$$

where $\Vert \cdot \Vert$ denotes the operator norm.

Lemma 7

Let $\Omega$ be a bounded convex domain in $\mathbb {C}^n$ and $\Gamma _{\Omega }\ne \emptyset$ be defined as in (3). Assume that $\{\phi _n\}\subset C({\overline{\Omega }})$ is a sequence such that $\phi _n\rightarrow 0$ uniformly on $\Gamma _{\Omega }$ as $n\rightarrow \infty$. Then $\lim _{n\rightarrow \infty }\Vert H_{\phi _n}\Vert _e= 0$.

Proof

Let $\varepsilon >0$. Then there exists N such that $\sup \{|\phi _n(z,w)|:(z,w)\in \Gamma _{\Omega }\}<\varepsilon$ for $n\ge N$. For $n\ge N$ we choose an open neighborhood $U_{n,\,\varepsilon }$ of $\Gamma _{\Omega }$ such that $|\phi _n(z,w)|<\varepsilon$ for $(z,w)\in U_{n,\,\varepsilon }$. Furthermore, we choose a smooth cut-off function $\chi _{n,\,\varepsilon }\in C^{\infty }_0(U_{n,\,\varepsilon })$ such that $0\le \chi _{n,\,\varepsilon }\le 1$ and $\chi _{n,\,\varepsilon }=1$ on a neighborhood of $\Gamma _{\Omega }$.

Let us define

$$\begin{aligned} \phi _{1,\,n,\,\varepsilon }=\chi _{n,\,\varepsilon }\phi _n \quad \text { and }\quad \phi _{2,\,n,\,\varepsilon }=(1-\chi _{n,\,\varepsilon })\phi _n. \end{aligned}$$

Then $\phi _{n}=\phi _{1,\,n,\,\varepsilon }+\phi _{2,\,n,\,\varepsilon }$ and $|\phi _{1,\,n,\,\varepsilon }|< \varepsilon$ on ${\overline{\Omega }}$ while $\phi _{2,\,n,\,\varepsilon }=0$ on a neighborhood of $\Gamma _{\Omega }$ in ${\overline{\Omega }}$. Furthermore,

$$\begin{aligned} \Vert H_{\phi _{1,\,n,\,\varepsilon }}\Vert _e\le \Vert H_{\phi _{1,\,n,\,\varepsilon }}\Vert \le \sup \{ |\phi _{1,\,n,\,\varepsilon }(z,w)|:(z,w)\in {\overline{\Omega }}\}<\varepsilon . \end{aligned}$$

Next we will show that $H_{\phi _{2,\,n,\,\varepsilon }}$ is compact. Since $\phi _{2,\,n,\,\varepsilon }=0$ on a neighborhood of $\Gamma _{\Omega }$ in ${\overline{\Omega }}$, using convolution with approximate identity, one can choose $\{\psi _{k,\,n,\,\varepsilon }\}\subset C^{\infty }({\overline{\Omega }})$ such that $\psi _{k,\,n,\,\varepsilon }=0$ on a neighborhood of $\Gamma _{\Omega }$ in ${\overline{\Omega }}$ for all k and $\psi _{k,\,n,\,\varepsilon }\rightarrow \phi _{2,\,n,\,\varepsilon }$ uniformly on ${\overline{\Omega }}$ as $k\rightarrow \infty$. We choose finitely many open balls $U_j=B(p_j,r_j)$ for $j=1,\ldots , N$ such that $\Gamma _{\Omega }\subset \cup _{j=1}^NU_j, p_j\in \Gamma _{\Omega }$ , and $\psi _{k,\,n,\,\varepsilon }=0$ on $U_j$ for all j. Then we cover $b\Omega {\setminus } \cup _{j=1}^NU_j$ by finitely many open balls $U_j=B(p_j,r_j)$ for $j=N+1,\ldots ,M$ such that $p_j\in b\Omega$ and $U_j\cap \Gamma _{\Omega }=\emptyset$ for $j=N+1,\ldots , M$.

Below $R_V$ denotes the restriction operator onto $V\subset \Omega$. That is, $R_Vf=f|_V$ for $f\in A^2(\Omega )$. We note that $U_j\cap \Omega$ is a bounded convex domain with no analytic disc in the boundary for all $j=N+1,\ldots , M$. Then [4, Theorem 1.1] (see also [10, Theorem 4.26]) implies that the ${\overline{\partial }}$-Neumann operator on $U_j\cap \Omega$ is compact (for $j=N+1,\ldots ,M$) and [10, Proposition 4.1], in turn, implies that the Hankel operator $H^{U_j\cap \Omega }_{R_{U_j\cap \Omega }(\psi _{k,\,n,\,\varepsilon })}R_{U_j\cap \Omega }$ is compact for $j=N+1,\ldots , M$.

Therefore, we have chosen finitely many balls $U_j=B(p_j, r_j)$ for $j=1,\ldots ,M$ such that

• i. $p_j\in b\Omega$ and $b\Omega \subset \cup _{j=1}^M U_j$,

• ii. the operator $H^{U_j\cap \Omega }_{R_{U_j\cap \Omega }(\psi _{k,\,n,\,\varepsilon })}R_{U_j\cap \Omega }=0$ for $p_j\in \Gamma _{\Omega }$,

• iii. the operator $H^{U_j\cap \Omega }_{R_{U_j\cap \Omega }(\psi _{k,\,n,\,\varepsilon })}R_{U_j\cap \Omega }$ is compact for $p_j\not \in \Gamma _{\Omega }$.

So, the local Hankel operators $H^{U_j\cap \Omega }_{R_{U_j\cap \Omega }(\psi _{k,\,n,\,\varepsilon })}R_{U_j\cap \Omega }$ are compact for all $j=1,\ldots ,M$. Now we use [2, Proposition 1, (ii)] to conclude that $H_{\psi _{k,\,n,\,\varepsilon }}$ is compact. Hence $H_{\phi _{2,\,n,\,\varepsilon }}$ is compact and $\Vert H_{\phi _n}\Vert _e\le \varepsilon$ for $n\ge N$. Therefore, $\lim _{n\rightarrow \infty }\Vert H_{\phi _n}\Vert _e= 0$. $\square$

We will now show one implication of the main theorem on non-product domains if the symbol is smooth up to the boundary.

Lemma 8

Let $\Omega \subset \mathbb {C}^2$ be a bounded convex Reinhardt domain that is not the product of two discs and $\phi \in C^{\infty }({\overline{\Omega }})$. Assume that $\phi \circ f$ is holomorphic for any holomorphic function $f:\mathbb {D}\rightarrow b\Omega$. Then $H_{\phi }$ is compact on $A^2(\Omega )$.

Proof

If $b\Omega$ does not contain any non-trivial analytic disc the ${\overline{\partial }}$-Neumann operator is compact [10, Theorem 4.26] (see also [4, Theorem 1.1]). Furthermore, if the ${\overline{\partial }}$-Neumann operator is compact then $H_{\phi }$ is compact for all $\phi \in C({\overline{\Omega }})$ [10, Proposition 4.1]. So if $b\Omega$ does not contain any non-trivial analytic disc, there is nothing to prove as the operator $H_{\phi }$ is compact. Lemma 1 implies that the analytic discs in $b\Omega$ are flat and horizontal or flat and vertical. We assume that there are non-trivial vertical and horizontal analytic discs in $b\Omega$ as the proof is even simpler if there are no vertical or horizontal discs. Let $\Delta _1$ and $\Delta _2$ be the horizontal and the vertical discs in $b\Omega$. So there exists $0<r_1<s_2,0<r_2<s_1$ (since $\Omega$ is not product of two discs) such that

$$\begin{aligned} \Delta _1=\mathbb {D}_{r_1}\times S_{s_1} \quad \text { and }\quad \Delta _2=S_{s_2}\times \mathbb {D}_{r_2}. \end{aligned}$$

We note that $\Gamma _{\Omega }=\overline{\Delta _1}\cup \overline{\Delta _2}$ and $\overline{\Delta _1}\cap \overline{\Delta _2}=\emptyset$. Let us define

$$\begin{aligned} \phi _1(z,w)=\phi (z,w)-(|w|^2-s_1^2)\frac{1}{w} \frac{\partial \phi (z,w)}{\partial {\overline{w}}} \end{aligned}$$

for $w\ne 0$. We note that $\phi _1$ is a $C^{\infty }$-smooth function on ${\overline{\Omega }}$ for $w\ne 0$ and $\phi _1=\phi$ on $\Delta _1$. Furthermore, using the fact that $\phi (.,w)$ is holomorphic on $\mathbb {D}_{r_1}$ for $|w|=s_1$, one can verify that ${\overline{\partial }}\phi _1=0$ on $\Delta _1$. Similarly we define

$$\begin{aligned} \phi _2(z,w)=\phi (z,w)-(|z|^2-s_2^2)\frac{1}{z} \frac{\partial \phi (z,w)}{\partial {\overline{z}}} \end{aligned}$$

and one can verify that $\phi _2=\phi$ and ${\overline{\partial }}\phi _2=0$ on $\Delta _2$.

We choose $\chi _1, \chi _2\in C^{\infty }({\overline{\Omega }})$ such that

• i. $\chi _1\equiv 1$ on a neighborhood of $\overline{\Delta _1}$ and $\chi _1\equiv 0$ on a neighborhood of $\overline{\Delta _2}\cup \{(z,w)\in {\overline{\Omega }}:|w|=0\}$,

• ii. $\chi _2\equiv 1$ on a neighborhood of $\overline{\Delta _2}$ and $\chi _2\equiv 0$ on a neighborhood of $\overline{\Delta _1}\cup \{(z,w)\in {\overline{\Omega }}:|z|=0\}$.

Then we define

$$\begin{aligned} \psi =\chi _1 \phi _1+\chi _2 \phi _2\in C^{\infty }({\overline{\Omega }}). \end{aligned}$$

We note that $\psi =\phi$ and ${\overline{\partial }}\psi =0$ on $\Gamma _{\Omega }$. Lemma 7 implies that $H_{\phi -\psi }$ is compact on $A^2(\Omega )$. To finish the proof we only need to show that $H_{\psi }$ is compact. This can be done exactly in the same manner as the proof of $H^{\Omega }_{{\widetilde{\beta }}}$ is compact in [2, pp 3740]. $\square$

Proposition 1

Let $f\in C(\overline{\mathbb {D}^2})$ such that $f(e^{i\theta },.)$ and $f(.,e^{i\theta })$ are holomorphic on $\mathbb {D}$ for each fixed $\theta$. Then $H_f$ is compact on $A^2(\mathbb {D}^2)$.

Proof

Let $\mathbb {T}^2=\{(z,w)\in \mathbb {C}^2:|z|=|w|=1\}$ be the distinguished boundary and

$$\begin{aligned} F_N(z,w)=\sum _{|m|,|j|\le N}\left( 1-\frac{|m|}{N+1}\right) \left( 1-\frac{|j|}{N+1}\right) a_{mj}(f)z^{m}w^{j} \end{aligned}$$

where

$$\begin{aligned} a_{mj}(f)=\int _{\mathbb {T}^2}f(\zeta _1,\zeta _2)\zeta _1^{-m}\zeta _2^{-j}d\sigma (\zeta ) \end{aligned}$$

and $\sigma$ is the normalized Lebesgue measure on $\mathbb {T}^2$. We let $S_{N,2}$ be the N-th Fejér kernel on $\mathbb {T}^2$. As in [5, Chapter I, Section 9], it is just the product of the N-th Fejér kernels on the circle. Since $f\in C({\mathbb {T}^2})$, and the convolution $S_{N,2}*f=F_N$, Fejér’s Theorem on Cesàro summability (see, for example, [5, Section 9.2, pg 64] for homogeneous Banach spaces) implies that

$$\begin{aligned} \Vert F_N-f\Vert _{L^\infty (\mathbb {T}^2)}\rightarrow 0 \end{aligned}$$

as $N\rightarrow \infty$.

Now we claim that $a_{mj}(P)=0$ for any holomorphic polynomial P and $m\le -1$ or $j\le -1$. Let

$$\begin{aligned} P(z,w)=\sum _{l,k=0}^{n}b_{lk}z^lw^k \end{aligned}$$

and $m\le -1$ or $j\le -1$. Then

$$\begin{aligned} a_{mj}(P)&=\sum _{l,k=0}^n b_{lk}\langle \zeta _1^l\zeta _2^k,{\zeta _1}^m{\zeta _2}^j \rangle _{L^2(\mathbb {T}^2)}\nonumber \\&=\frac{1}{(2\pi )^2}\int _{0}^{2\pi }\int _{0}^{2\pi }\sum _{l,k=0}^{n}b_{lk} e^{i\theta _1l}e^{i\theta _2k}e^{-i\theta _1m}e^{-i\theta _2 j}d\theta _1 d\theta _2\nonumber \\&=0. \end{aligned}$$

(5)

Next we will show that $a_{mj}(f)=0$ for $m\le -1$ or $j\le -1$. Without loss of generality, we suppose that $j\le -1$. Since $f(e^{i \theta _1},.)$ is holomorphic on $\mathbb {D}$, using Mergelyan’s Theorem, there exists a sequence of holomorphic polynomials $\{P_{n,\,\theta _1}\}_{n\in \mathbb {N}}$ converging to f uniformly on $\overline{\mathbb {D}}$ as $n\rightarrow \infty$. Let us define $P_{n,\,\theta _1,r}(\xi )=P_{n,\theta _1}( r\xi )$ and $f_r(z,w)=f(z,rw)$ for $0<r<1$. Then $P_{n,\theta _1,r}\rightarrow f_r(e^{i\theta _1}, .)$ uniformly on $\overline{\mathbb {D}}$ as $n\rightarrow \infty$. As we have computed above in (5), one can show that $a_{mj}(P_{n,\,\theta _1,\,r})=0$ for all $m\in \mathbb {Z},n\in \mathbb {N}$, and $0<r<1$. So by taking limits as $n\rightarrow \infty$ we have $a_{mj}(f_r)=0$ for all $0<r<1$. Finally taking the limit as $r\rightarrow 1^-$ we conclude that $a_{mj}(f)=0$ for $j\le -1$. The proof for $m\le -1$ is similar. Hence we have shown that $a_{mj}(f)=0$ for $j\le -1$ or $m\le -1$.

We define

$$\begin{aligned} G_N(z,w)=\sum _{0\le m, j\le N}\left( 1-\frac{m}{N+1}\right) \left( 1-\frac{j}{N+1}\right) a_{mj}(f)z^{m}w^{j}. \end{aligned}$$

Since we have shown $G_N\equiv F_N$ on $\mathbb {T}^2$ , we have $\Vert G_N-f\Vert _{L^{\infty }(\mathbb {T}^2)}\rightarrow 0$ as $N\rightarrow \infty$. Since $(G_N-f)(e^{i\theta },w)$ is holomorphic in w and $(G_N-f)(z,e^{i\theta })$ is holomorphic in z, using the Maximum Modulus Principle for holomorphic functions, we have

$$\begin{aligned} \Vert G_N-f\Vert _{L^{\infty }(b\mathbb {D}^2)}\le \Vert G_N-f\Vert _{L^{\infty }(\mathbb {T}^2)}. \end{aligned}$$

So $\Vert G_N-f\Vert _{L^{\infty }(b\Omega )}\rightarrow 0$ as $N\rightarrow \infty$. Then Lemma  7 implies that $\Vert H_{G_N-f}\Vert _e\rightarrow 0$ as $N\rightarrow \infty$. Furthermore, $\Vert H_f\Vert _e=\Vert H_{G_n-f}\Vert _e$ as $H_{G_N}=0$. Therefore, we conclude that $\Vert H_f\Vert _e=0$. That is, $H_f$ is compact on $A^2(\mathbb {D}^2)$. $\square$

Remark 2

Even though we stated the previous proposition on $\mathbb {D}^2$ the same proof, with trivial modifications, works on products of two discs.

Now we are ready for the proof of the main result.

Proof of Theorem 1

First we will prove the sufficiency. Assume that $H_{\phi }$ is compact on $A^2(\Omega )$. If there is no non-trivial analytic disc in the boundary of $\Omega$ then there is nothing to prove. So assume that $\Delta =f(\mathbb {D})$ is a non-trivial disc in $b\Omega$ such that $\phi \circ f$ is not holomorphic. Without loss of generality we may assume that $\Delta$ is horizontal as the proof for vertical discs is similar. Let us fix $(z_0,w_0)\in \Delta$ and define $\alpha _j=(j-1)/j$. Then one can check that $\Vert (w-w_0)^{-\alpha _j}\Vert _{L^2(H)}\rightarrow \infty$ as $j\rightarrow \infty$. Let us define

$$\begin{aligned} g_j(w)=\frac{a_j}{(w-w_0)^{\alpha _j}} \end{aligned}$$

where $a_j=1/\Vert (w-w_0)^{-\alpha _j}\Vert _{L^2(H_{\Omega })}$. Then $\Vert g_j\Vert _{L^2(H_{\Omega })}=1$ for all j. Furthermore, $g_j\rightarrow 0$ uniformly on any compact subset in $\Omega$ as $j\rightarrow \infty$. Without loss of generality, we assume that $\Delta$ is the largest horizontal disc in $b\Omega$ passing through $(z_0,w_0)$ and $\phi _0$ be a continuous function on $\mathbb {C}^2$ such that $\phi _0(z,w)=\phi (z,w_0)$ for all $(z,w)\in \Omega$. That is, $\phi _0$ is the extension of $\phi |_{\Delta }$ to $\mathbb {C}$ in z. Since $\phi _0$ is not holomorphic (as a function of z) on $\Delta$ we have $H^{\Delta }_{\phi _0}(1)\ne 0$. That is, $\Vert H^{\Delta }_{\phi _0}(1)\Vert _{L^2(\Delta )}>0$. Then by Corollary 1, there exists $\beta >0$ and $\delta >0$ such that if $w\in H_{\Omega }$ and $|w-w_0|<\delta$, then

$$\begin{aligned} \Vert H^{\Delta _{w}}_{\phi _0}(1)\Vert _{L^2(\Delta _{w})}>\beta . \end{aligned}$$

Let us define $K=\{w\in H_{\Omega }:|w-w_0|\le \delta \}$. Then

$$\begin{aligned}&\int _{H_{\Omega }}|g_j(w)|^2\int _{\Delta _w}|H^{\Delta _w}_{\phi _0}(1)(z)|^2dV(z)dV(w)\\&\quad \ge \int _{K}|g_j(w)|^2\int _{\Delta _w}|H^{\Delta _w}_{\phi _0}(1)(z)|^2dV(z)dV(w)\\&\quad \ge \beta ^2\Vert g_j\Vert ^2_{L^2(K)}. \end{aligned}$$

However, since $\Vert g_j\Vert ^2_{L^2(H_{\Omega })}=1$ for all j and $g_j\rightarrow 0$ uniformly on any compact set away from $w_0$ we conclude that $\Vert g_j\Vert ^2_{L^2(K)}\ge 1/2$ for large j. Therefore, for large j we have

$$\begin{aligned} \int _{H_{\Omega }}|g_j(w)|^2\int _{\Delta _w}|H^{\Delta _w}_{\phi _0}(1)(z)|^2dV(z)dV(w) \ge \frac{\beta ^2}{2}>0. \end{aligned}$$

Then Lemma 4 and Lemma 5 imply that $\Vert H_{\phi _0}g_j\Vert ^2_{L^2(\Omega )}$ does not converge to 0 as $j\rightarrow \infty$. This contradicts Lemma 3 as we have assumed that $H_{\phi }$ is compact.

Finally we will prove the necessity. We assume $\phi \in C({\overline{\Omega }})$ is such that $\phi \circ f$ is holomorphic for any holomorphic function $f:\mathbb {D}\rightarrow b\Omega$. Furthermore, we assume that $\Omega$ is not the product of two discs as that case is covered in Proposition 1. Lemma 6 implies that there exists a family of functions $\{\psi _n\}\subset C^{\infty }({\overline{\Omega }})$ such that

• i. $\psi _n\circ f$ is holomorphic for any n and any holomorphic $f:\mathbb {D}\rightarrow b\Omega$,

• ii. $\psi _n\rightarrow \phi$ uniformly on $\Gamma _{\Omega }$ as $n\rightarrow \infty$.

Lemma 8 implies that $H_{\psi _n}$ is compact and Lemma 7 implies that $\Vert H_{\phi -\psi _n}\Vert _e\rightarrow 0$ as $n\rightarrow \infty$. Therefore,

$$\begin{aligned} \Vert H_{\phi }\Vert _e=\Vert H_{\phi }\Vert _e-\Vert H_{\psi _n}\Vert _e\le \Vert H_{\phi -\psi _n}\Vert _e. \end{aligned}$$

This implies $\Vert H_{\phi }\Vert _e=0$, proving that $H_{\phi }$ is compact on $A^2(\Omega )$. $\square$