Isolating blocks of isolated invariant continua and fixed point index

Abstract

In this paper we study some properties of isolated invariant continua for arbitrary homeomorphisms of ${{\mathbb {R}}}^n$. We study the existence of special isolating blocks for them which allow us to compute the fixed point indices of the iterates of arbitrary homeomorphisms at arbitrary isolated continua in dimension two. Among the consequences we would highlight the following:

• If $K \subset {{\mathbb {R}}}^2$ is an isolated invariant continuum that decomposes the plane in more than two connected components, then K contains a periodic orbit.

• A proper invariant continuum of the 2-sphere containing the set of periodic orbits of a homeomorphism is not isolated.

• If K is an isolated invariant continuum for $f: S^n \rightarrow S^n$, then $S^n {\setminus } K$ has a finite amount of connected components.

1 Introduction

The existence of a minimal homeomorphism of ${{\mathbb {R}}}^m$ is an open problem suggested by Ulam and contained in the Scottish Book [21]. In dimension 2, one can consider the more general problem of the existence or not of minimal homeomorphisms $f:{{\mathbb {R}}}^2 {{\setminus }} K \rightarrow {{\mathbb {R}}}^2 {{\setminus }} K$ with K a compact set. If $K= \emptyset$, the answer follows from the Brouwer’s translation arcs theorem. Handel, in [12], proved that, if K has at least two points, there are not minimal homeomorphisms of ${{\mathbb {R}}}^2 {{{\setminus }}} K$. Le Calvez and Yoccoz [18] and [19] solved completely the problem in the multi-punctured plane. Later Franks, in [7], gave an alternative proof using Conley index methods. A general result for a compact set K is given in [28]. More recently, it has been proved that there are no minimal orientation reversing homeomorphisms in ${{\mathbb {R}}}^3$ ([13]).

The key of the proof in [18] is the study of the fixed point indices of the iterations of an orientation preserving local homeomorphism $f:U \subset {{\mathbb {R}}}^2 \rightarrow {\mathbb R}^2$ in a neighborhood of a point p which is not an attractor nor a repeller isolated invariant set. This result was extended in [25] to the orientation reversing case using the following ideas:

If $f: U \subset {{\mathbb {R}}}^2 \rightarrow {{\mathbb {R}}}^2$ is a local homeomorphism and $p \in U$ is a non-repeller fixed point of f such that $\{p\}$ is an isolated invariant set, then there are an AR (absolute retract for metric spaces), D, containing a neighborhood $V \subset {{\mathbb {R}}}^2$ of p, a finite subset $\{q_1, \dots , q_m\} \subset D$ and a map $f': D \rightarrow D$ such that $f'|_V=f|_V$ and for every $k \in {{\mathbb {N}}}$, $Fix((f')^k) \subset \{p, q_1, \dots , q_m\}$. Moreover,

• a) (Le Calvez-Yoccoz) If f preserves the orientation, the sequence of fixed point indices of the iterates of f in ${{\mathbb {R}}}^2$ satisfy

  $$\begin{aligned} i_{{{\mathbb {R}}}^2}(f^{k}, p)=\left\{ \begin{array}{ll} 1-rq &{} \text { if } k \in r{{\mathbb {N}}} \\ 1 &{} \text { if } k \notin r{{\mathbb {N}}}, \end{array} \right. \end{aligned}$$

  where $k \in {{\mathbb {N}}}$, q is the number of periodic orbits of $f'$ (excluding p) and r is their period.

• b) Assume that f reverses the orientation (see [25]). Then,

  $$\begin{aligned} i_{{{\mathbb {R}}}^2}(f^{k}, p)= \left\{ \begin{array}{ll} 1 - \delta &{} \text { if } k \text { odd } \\ 1-(2q+\delta ) &{} \text { if } k \text { even }, \end{array} \right. \end{aligned}$$

where q and $\delta$ are the number of orbits of period 2 and 1 of $f'$ in $\{q_{1}, \dots , q_{m}\}$.

More results about the computation of this sequence for a fixed point p can be obtained in [1, 2, 6, 9,10,11, 13, 14, 17, 20, 23, 24] .

The aim of this paper was to extend the above result computing the fixed point index $i_{{{\mathbb {R}}}^n}(f^{k}, K)$ for K an isolated invariant continuum, with special interest in the planar case $n=2$. The techniques we employ are based on Conley index ideas. From the results of [20] it follows that the sequence is periodic if $n=2$.

We will prove, in a constructive way, the existence of certain special isolating blocks and index pairs, which we call strong filtration pairs, associated with an isolated invariant continuum K of a homeomorphism $f:S^n \rightarrow S^n$. The if isolating block, N, of K will be a connected manifold. Our principal interest is focused in the case $n=2$. In this situation, N and K will decompose the plane in the same number of components. This construction permits us to prove the Main Theorems which compute the fixed point indices of the iterations of a homeomorphism f at K.

Main Theorem 1

Let $f:U \subset {{\mathbb {R}}}^{2} \rightarrow f(U) \subset {\mathbb {R}}^{2}$ be a homeomorphism and let K be an isolated invariant continuum. Then,

$$\begin{aligned} i_{{{\mathbb {R}}}^{2}}(f^{k},K)= \left\{ \begin{array}{ll} 2-C(k)-P(k) &{} \quad \text { if } f^k \text { is orientation preserving.} \\ -C'(k)-P(k) &{} \quad \text { if } f^k \text { is orientation reversing.}, \end{array}\right. \end{aligned}$$

where $C(k) \ge 0$ is the number of components C of ${\mathbb R}^2 {{\setminus }} K$ such that $f^k(U \cap C) \subset C$. The integers $C'(k)$ (k odd) are also defined only in terms of the above components (as the number of them which are exit regions for $f^k$ in a neighborhood of K minus the cardinal of the rest). The integers $P(k) \ge 0$ depend on the behavior of $f^k$ in the exit set of N.

Let us observe that, if k is even or f is orientation preserving, the index is $i_{{{\mathbb {R}}}^{2}}(f^{k},K)=2-C(k)-P(k) \le 2$. On the other hand, if k is odd and f is orientation reversing, $i_{{{\mathbb {R}}}^{2}}(f^{k},K)=-C'(k)-P(k)$.

Corollary 1

Under the above conditions, if f is orientation preserving, $i_{{{\mathbb {R}}}^{2}}(f^{k},K) \le 2$. Moreover, there exists $k_0 \in {{\mathbb {N}}}$ such that $i_{{{\mathbb {R}}}^{2}}(f^{nk_0},K) \le 1$ for every $n \in {{\mathbb {N}}}$.

Corollary 2

Given any homeomorphism f, if K decomposes the plane into three or more components, there exists $k_0 \in {{\mathbb {N}}}$ such that $i_{{{\mathbb {R}}}^{2}}(f^{k_0},K) <0$. Consequently f has a periodic orbit in K.

A stronger version of the above result of existence of periodic orbits is well known to be true in the orientation reversing case (see [16] for example). However, here we find periodic points of non-zero index. On the other hand, note that using Corollary 4 below (see also [27]), Corollary 2 admits a simple proof if K is an attractor that decomposes the plane into three or more components. Indeed, one can construct a connected manifold N, isolating block of K, such that $f(N) \subset N$ and $\dim H_1(N; {{\mathbb {Q}}}) \ge 2$. Then, there exists $k \in {{\mathbb {N}}}$ such that the $i_{{{\mathbb {R}}}^2}(f^k,K)=\Lambda (f^k|_{N})<0$ where $\Lambda (f^k|_{N})$ stands for the Lefschetz number of $f^k|_N: N \rightarrow N$.

The knowledge of the structure of the set of periodic orbits of homeomorphisms of the sphere is an interesting problem that was stated explicitly by Le Calvez in his lecture in the ICM 2006. The question of whether cl(Per(f)) is isolated has been studied in [28] with restrictions on the homeomorphism (area-preserving). The next corollary is valid for invariant continua containing Per(f).

Corollary 3

Given a homeomorphism $f:S^2 \rightarrow S^2$, if $K \subsetneqq S^2$ is an invariant continuum which contains Per(f),  then K is not isolated. On the other hand, if K has a finite amount of connected components, $K= \bigcup _{i=1}^n K_i$, and for each $K_i$, $S^2 {{\setminus }} K_i$ has no invariant components $U_{i,j}$ for some $f^{n_{i,j}}$ which are locally attracted to K or repelled from K for $f^{n_{i,j}}$ ($f^{n_{i,j}}(U_{i,j} \cap U_K) \nsubseteq U_{i,j} \cap U_K$ and $f^{n_{i,j}}(U_{i,j} \cap U_K) \nsupseteq U_{i,j} \cap U_K$ for every neighborhood $U_K$ of K), then K is not isolated.

When K is not locally maximal but ${{\mathbb {R}}}^2 {{\setminus }} K$ has a finite number of components, one way to study the problem of the computation of the index could be made by using Carathéodory’s prime ends techniques, but we will not treat this situation here.

The structure of the article is the following; In Sect. 2 we introduce the Conley index techniques we will need for our particular setting. Sections 3 and 4 are devoted to prove the Main Theorem.

In Sect. 5 we see a theorem which relates the fixed point indices with the local dynamics of f in a neighborhood of K.

The reader is referred to the text of [4, 5, 15] and [22] for information about the fixed point index theory.

2 Preliminaries. Conley index and construction of adequate filtration pairs in ${{\mathbb {R}}}^n$

Let $U \subset X$ be an open set. By a local semidynamical system we mean a locally defined continuous map $f:U \rightarrow X$. We say that a function $\sigma :{{\mathbb {Z}}} \rightarrow X$ is a solution to f through x in $N \subset U$ if $f(\sigma (i))=\sigma (i+1)$ for all $i \in {{\mathbb {Z}}}$, $\sigma (0)=x$ and $\sigma (i) \in N$ for all $i \in {\mathbb Z}$. The invariant part of N, Inv(N, f), is defined as the set of all $x \in N$ that admit a solution to f through x in N, i. e. the set of all $x \in N$ such that there is a full orbit $\gamma$ such that $x \in \gamma \subset N$. $Inv^+(N,f)$ (respectively, $Inv^-(N,f)$) denotes the set of all $x \in N$ such that $f^n(x) \in N$ for every $n \in {{\mathbb {N}}}$ (respectively, $f^{-n}(x)$ is well defined and belongs to N for every $n \in {{\mathbb {N}}}$).

Given $A \subset B \subset N$, $cl(A), cl_{B}(A), int(A), int_{B}(A), \partial (A)$ and $\partial _{B}(A)$ will denote the closure of A, the closure of A in B, the interior of A, the interior of A in B, the boundary of A and the boundary of A in B.

A compact set $K \subset X$ is invariant if $f(K)=K$. An invariant compact set K is isolated with respect to f if there exists a compact neighborhood N of K such that $Inv(N,f)=K$. The neighborhood N is called an isolating neighborhood of K.

A compact set $N \subset f(U)$ is called isolating block if $f(N) \cap N \cap f^{-1}(N) \subset int(N)$. If $K =Inv(N,f)$ we will say that N is an isolating block of K.

We consider the exit set of N to be defined as

$$\begin{aligned} N^{-}=\{x \in N : f(x) \notin int(N)\}. \end{aligned}$$

Let $f: U \subset {{\mathbb {R}}}^{ n} \rightarrow f(U) \subset {\mathbb R}^{n}$ be a homeomorphism with U an open set and let $K \subset U$ be an isolated invariant continuum. Given $k \in {{\mathbb {N}}}$ we want to compute the fixed point index of $f^k$ in K, $i_{{{\mathbb {R}}}^{n}}(f^{k}, K)$, i.e. the fixed point index of each $f^k$ in a small enough neighborhood of K, with a special interest in the planar case.

We begin this study with the choice of an adequate isolating block N of K with topological properties which give us information about the dynamical behavior of f in a neighborhood of K.

Among all compact connected smooth manifolds (discs with a finite amount of holes if $n=2$) which are isolating blocks of K, we take N to be one of them which decomposes ${{\mathbb {R}}}^n$ into the minimum possible number of components. The existence of the manifold N is given in [8], Theorem 3.7.

If we consider $K \subset N \subset S^n$, the set $I(K)=S^n {{\setminus }} K$ is an open subset of $S^{n}$ and $S^n {{\setminus }} int(N)$ has p components $\{D_1, \dots , D_p\}$. Let us denote $\{I(K)_{j}\}$ the connected components of I(K) (Fig. 1).

Fig. 1

Picture of I(K)

Assume that $f_{|N}:N \rightarrow f(N)$ is the restriction of a global homeomorphism $f^*:S^n \rightarrow S^n$. If $n=2$, by the Schönflies theorem, every homeomorphism $f_{|N}:N \rightarrow f(N)$ satisfies that there is such a homeomorphism $f^*:S^2 \rightarrow S^2$.

Lemma 1

Any component $I(K)_{j}$ contains a component $D_{j} \in \{D_{1}, \dots , D_{p}\}$. As a consequence, I(K) decomposes into a finite number of components on which $f^*$ acts as a permutation.

Proof

Since N is an isolating block, if $I(K)_j \subset N$ then each image $f^n(I(K)_j) \subset I(K)$ is contained in N. In fact, if there exists $D_{i'} \subset I(K)_{j'}=f^*(I(K)_j) \subset f(N)$, it is obvious that $\partial (D_{i'}) \subset f(N)$. Since N is an isolating block, one gets $f(\partial (D_{i'})) \cap N = \emptyset$. On the other hand, $f(\partial (D_{i'}))$ is an hypersurface of $S^n$ without boundary, and it decomposes $S^n$ into two connected components with boundary $f(\partial (D_{i'}))$. Then, $f^*(D_{i'}) \cap N= \emptyset$ or $N \subset f^*(D_{i'})$. But, since $K \subset N$, it is only possible that $f^*(D_{i'}) \cap N= \emptyset$.

This shows that $I(K)_{j''}=f^*(I(K)_{j'})$ also contains at least another $D_{i''}$. Furthermore, $\partial (D_{i''}) \subset f(N)$. In fact, if there exists $x \in \partial (D_{i''})$ such that $x=f^*(x_0)$, with $x_0 \in I(K)_{j'} \cap int(D_{i_0'})$, then $D_{i_0'} \ne D_{i'}$ ($f^*(D_{i'}) \cap N=\emptyset$) and there exists $x' \in f(\partial (D_{i_0'})) \cap \partial (D_{i''})$. The set $\{x', f^{-1}(x'), f^{-2}(x')\} \subset N$ and $f^{-1}(x') \in \partial (D_{i_0'}) \subset \partial (N)$, but this is not possible because N is an isolating block.

By an induction argument, all the images by $f^*$ of $I(K)_{j}$ have elements of the finite family $\{D_{1}, \dots , D_{p}\}$. Then, there exists $n_{j}$ such that $(f^*)^{n_{j}}(I(K)_{j})=I(K)_{j}$ and it must contain a component of $S^n {{\setminus }} N$. This is a contradiction and we conclude that, if $I(K)_j \subset N$, each image $f^n(I(K)_j) \subset I(K)$ is contained in N. In this case, the set $K \cup cl(\bigcup _{n \in {{\mathbb {Z}}}}(f^*)^n(I(K)_j)) \ne K$ is an invariant continuum contained in N. This contradicts the fact that $Inv(N, f)=K$ and the proof is finished. $\square$

Lemma 2

Let $D_{j} \subset I(K)_{j}$. Then

$$\begin{aligned} f^*(D_{j}) \not \subset N \text { and } (f^*)^{-1}(D_{j}) \not \subset N. \end{aligned}$$

Proof

Let us prove that $f^*(D_{j}) \not \subset N$ (the other statement has an analogous proof). If $f^*(D_{j}) \subset N$, since N is an isolating block, $f^{-1}(\partial (D_j)) \cap N=\emptyset$. On the other hand, $f^{-1}(\partial (D_j))$ is an hypersurface of $S^n$ without boundary and it decomposes $S^n$ into two connected components with boundary $f^{-1}(\partial (D_j))$. Then, $(f^*)^{-1}(D_j) \cap N= \emptyset$ or $N \subset (f^*)^{-1}(D_j)$. Since $K \subset N$, it is only possible that $(f^*)^{-1}(D_j) \cap N=\emptyset$. We conclude that $(f^*)^{-1}(D_j) \subset int(D_{i}) \subset I(K)_i=(f^*)^{-1}(I(K)_j)$ with $D_i \ne D_j$.

Let us see that the set $N'=N \cup D_j$ is an isolating block of K. We only have to prove that

$$\begin{aligned} (f^*)^{-1}(N') \cap N' \cap (f^*)(N') \subset int(N') \end{aligned}$$

Since $(f^*)^{-1}(D_j) \cap N=\emptyset$ it is obvious that $(f^*)^{-1}(int(D_j)) \cap N=\emptyset$. On the other hand, $(f^*)^{-1}(N')=(f^*)^{-1}(N) \cup (f^*)^{-1}(int(D_j))$, $N'=N \cup int(D_j)$ and $(f^*)(N')=f^*(N) \cup f^*(int(D_j))$.

Let $x \in (f^*)^{-1}(N') \cap N' \cap (f^*)(N')$. Then $x \in N'=N \cup int(D_j)$. We have two situations:

• If $x \in int(D_j)$, then $x \in int(N')$ and we have finished.

• If $x \in N$, since $(f^*)^{-1}(int(D_j)) \cap N=\emptyset$ and $x \in (f^*)^{-1}(N')$, then $x \in (f^*)^{-1}(N)$. Finally, $x \in f^*(N')=f^*(N) \cup f^*(int(D_j))$ and we have two cases:

  • If $x \in f^*(N)$, since N is an isolating block of K, $x \in f^{-1}(N) \cap N \cap f(N) \subset int(N) \subset int(N')$.

  • If $x \in f^*(int(D_j))$, since $f^*(D_j) \subset N$, $f^*(int(D_j)) \subset int(N) \subset int(N')$.

The isolating block of K, $N'$, decomposes $S^n$ in one component less than N, which is a contradiction. $\square$

Lemma 3

$I(K)_{j}$ contains a unique component $D_{j}$.

Proof

Let us suppose that there are two components $D_{j,1} \ne D_{j,2}$ in $I(K)_{j}$ and let $D^0=\bigcup _{i=1}^p D_{i} \subset S^n$. Let $B_{j}$ be a compact and connected manifold (a disc if $n=2$) with $D_{j,1} \cup D_{j,2} \subset B_{j} \subset I(K)_{j}$. We consider a finite covering of $B_{j}$, in $I(K)_{j}$, formed by closed balls $\{B(x)\}_{x \in {{\mathcal {F}}}}$, with ${{\mathcal {F}}}$ a finite subset of $B_{j}$, and such that for all B(x) there exists some $n_x \in {{\mathbb {Z}}}$ with $(f^*)^{n_x}(B(x)) \subset int(D^{0})$. Let $n_0$ be a natural number such that $|n_x| \le n_0$ for all $x \in {{\mathcal {F}}}$.

Our goal is to construct an isolating block of K, $N_{n_{0}}$, with similar properties than N and such that $S^n {{\setminus }} N_{n_0}$ has less components than $S^n {{\setminus }} N$. We will arrive at a contradiction.

We define a manifold $E^{0}=D^{0} \cup V_{1} \cup V_{-1} \subset S^n$, with the following:

• $V_{\pm 1}$ are compact manifolds homeomorphic to $(f^*)^{\pm 1}(D^{0})$ such that $V_{\pm 1} \subset int((f^*)^{\pm 1}(D^{0}))$.

• $V_{\pm 1}$ are transversal to $D^{0}$ and $V_1$ is transversal to $V_{-1}$.

• $V_{\pm 1}$ intersect the same components of $S^n {{\setminus }} N$ than $(f^*)^{\pm 1}(D^0)$.

• If $x \in {{\mathcal {F}}}$ with $(f^*)^{\pm 1}(B(x)) \subset int(D^0)$, then $B(x) \subset int(V_{\mp 1})$.

Let $D^1$ be the union of $E^0$ and the connected components of $S^n {{\setminus }} E^0$ which do not contain K. It is obvious that $D^1$ is a compact manifold which does not intersect K and, by Lemma 2, $E^0$ and $D^1$ have at most p components (Fig. 2).

Fig. 2

Picture of $E^0$ and $D^1$

Let us define

$$\begin{aligned} N_{1}=N {{\setminus }} int(D^1) \end{aligned}$$

Note that $K \subset int(N_{1}) \subset int(N)$ and $D^0 \subset D^1$. The set $N_{1}$ is a compact and connected manifold (the component of $S^n {{\setminus }} int(E^0)$ which contains K), isolating block of K, such that $S^n {{\setminus }} N_1$ has at most p components. The set $N_{1}$ and the components of $D^1$ are in the conditions of Lemma 2.

Let us prove that $N_1$ is, indeed, an isolating block of K. We only have to see that, given $x \in \partial (N_1)$, $f(x) \notin N_1$ or $f^{-1}(x) \notin N_1$. Notice that $\partial (N_1) \subset \partial (D^0) \cup \partial (V_1) \cup \partial (V_{-1})$. If $x \in \partial (D^0)$, then $\{f(x),f^{-1}(x)\} \not \subset N$ and, since $N_1 \subset N$, $\{f(x),f^{-1}(x)\} \not \subset N_1$. On the other hand, if $x \in \partial (V_{\pm 1})$, then $f^{\mp 1}(x) \in int(D^0) \subset int(D^1)$ and $\{f(x),f^{-1}(x)\} \not \subset N_1$.

By an induction argument, define a manifold $E^{i-1}=D^{i-1} \cup V_{i} \cup V_{-i} \subset S^n$, with the following:

• $V_{\pm i}$ compact manifolds homeomorphic to $(f^*)^{\pm 1}(D^{i-1})$ such that $V_{\pm i} \subset int((f^*)^{\pm 1}(D^{i-1}))$.

• $V_{\pm i}$ transversal to $D^{i-1}$ and $V_{i}$ transversal to $V_{-i}$.

• $V_{\pm i}$ intersect the same components of $S^n {{\setminus }} N_{i-1}$ than $(f^*)^{\pm 1}(D^{i-1})$.

• If $x \in {{\mathcal {F}}}$ with $(f^*)^{\pm 1}(B(x)) \subset int(D^{i-1})$, then $B(x) \subset int(V_{\mp i})$.

If we repeat the last construction $n_{0}$ times, we get the pairs of manifolds and complementary components $\{(N, D^0), (N_1, D^1), \dots , (N_{n_0}, D^{n_0})\}$ with $D^0 \subset D^1 \subset \cdots \subset D^{n_0}$ and $N_{n_0} \subset \cdots \subset N_1 \subset N$. Each $D^m$ is a finite union of disjoint compact and connected manifolds (discs if $S^n=S^2$) and if $D^{m'} \subset D^m$, each component of $D^m$ contains, at least, a component of $D^{m'}$. Since $D^0$ has p components and $D_{j,1} \cup D_{j,2} \subset B_j \subset \bigcup _{x \in {{\mathcal {F}}}} B(x) \subset D^{n_0}$, then $D^{n_0}$ has less than p components.

The set $N_{n_{0}}$ is a compact connected manifold, isolating block of K and, since $S^n {{\setminus }} N_{n_0}$ has less than p components, we obtain a contradiction. $\square$

The last three lemmas give us the following corollaries:

Corollary 4

If K is an invariant and isolated continuum for a homeomorphism $f:U \subset {{\mathbb {R}}}^2 \rightarrow f(U) \subset {{\mathbb {R}}}^2$, there exists a manifold, N, isolating block of K, such that the inclusion $i:K \subset N$ is a shape equivalence. Then i induces isomorphisms in Čech (co)homology groups.

Remark 1

However, the set K does not have, in general, the shape of a manifold for $n \ge 3$. It is easy to find such sets K for $n \ge 3$, for example, if K is a hawaiian earring in ${\mathbb R}^3$ and f is a flow transversal to K with $Fix(f)=K$.

Corollary 5

If K is an invariant and isolated continuum for a homeomorphism $f:S^n \rightarrow S^n$, $S^n {{\setminus }} K$ has a finite amount of connected components.

Corollary 6

If $f: {{\mathbb {R}}}^n \rightarrow {{\mathbb {R}}}^n$ is a volume contracting homeomorphism and K is an isolated invariant continuum, then K does not decompose ${{\mathbb {R}}}^n$. Moreover, there exists an isolating block N of K, which is a manifold with boundary, such that ${{\mathbb {R}}}^n {{\setminus }} N$ is connected.

Definition 1

([8]). Let K be a compact isolated invariant set and suppose $L \subset N$ is a compact pair contained in the interior of the domain of f. The pair (N, L) is called a filtration pair for K provided N and L are each the closure of their interiors and

• (1) $cl(N {{\setminus }} L)$ is an isolating neighborhood of K.

• (2) $f(cl(N {{\setminus }} L)) \subset int(N)$ and

• (3) $f(L) \cap cl(N {{\setminus }} L) = \emptyset$.

In the next proposition we prove the existence of certain compact pairs that we call strong filtration pairs and that are of particular interest for us.

Proposition 1

Let $f:S^{n} \rightarrow S^{n}$ be a homeomorphism and let K be an isolated invariant continuum. Then, there exists a pair (N, L), which we call strong filtration pair, satisfying properties (1) and (2) of the filtration pairs and such that

• a) $N \subset S^n$ is a compact connected n-manifold, isolating block for K, in such a way that $S^{n} {{\setminus }} N$ has a minimum possible number of components $\{D_{1}, \dots , D_{p}\}$.

• b) $L=L_{1} \cup \cdots \cup L_{m} \subset N$ is a finite union of disjoint compact connected n-manifolds with boundary such that $L_i \cap N^- \ne \emptyset$ and $L_i \cap \partial (N) \ne \emptyset$ for all i. Each set $N {{\setminus }} L_i$ is connected and contains K when K is not a repeller.

• c) $\partial _{N}(L_{i})$ is a $(n-1)$-manifold, not necessarily connected, transversal to $\partial (N)$ if the intersection is not empty and $f(\partial _{N}(L_{i})) \cap cl(N {{\setminus }} L)=\emptyset$ for all i. Then, we obtain that $f(\partial _{N}(L_{i})) \subset int(L)$ and, if $\partial _N(L_i)$ is connected, $f(\partial _{N}(L_{i})) \subset int(L_j)$ for some j.

Proof

Let N be a compact connected manifold, isolating block for K, such that $S^n {{\setminus }} N$ has the minimum possible number of components $\{D_{1}, \dots , D_{p}\}$. The existence of N isolating block is given in the proof of Theorem 3.7 in [8]. Following the steps of the proof of Proposition 3 in [20] we can obtain a filtration pair (N, J). The set $J=J_{1} \cup \cdots \cup J_{m}$ can be selected as a finite union of disjoint n-dimensional compact connected manifolds such that the pair (N, J) is a filtration pair for K, with J a submanifold of N, $\partial _N(J_i)$ transversal to $\partial (N)$, $J_{i} \cap N^{-} \ne \emptyset$ for all $i \in \{1, \dots , m\}$. If K is not a repeller nor an attractor, $S^n {{\setminus }} J$ has no connected components included in N. If K is a repeller, there exists a unique connected component of $S^n {{\setminus }} J$ contained in N and this component contains K. If K is an attractor, J is empty (Fig. 3).

Fig. 3

Filtration pairs (N, J)

Fig. 4

Pairs (N, L)

If we add to each $J_i$ of the filtration pair (N, J) the connected components of $N {{\setminus }} J_i$ which do not intersect K, we obtain a compact and connected manifold denoted by $L_i$. Let us consider the pair (N, L) with $L= \bigcup _{i=1}^{m} L_i$. Each $L_i \subset I(K_j)$ for some j and the set $N {{\setminus }} L_{i}$ is connected and contains K (Fig. 4).

We can suppose that $L=L_{1} \cup \cdots \cup L_{m}$ are disjoints. The pair (N, L) follows properties (1) and (2) of filtration pairs:

• (1) $cl(N {{\setminus }} L)$ is an isolating neighborhood of K. In fact, K does not intersect the boundary of $cl(N {{\setminus }} L)$ and $K \subset cl(N {{\setminus }} L) \subset cl(N {{\setminus }} J)$ which is an isolating neighborhood of K.

• (2) $f(cl(N{{\setminus }} L)) \subset int(N)$. In fact, since $cl(N{{\setminus }} L) \subset cl(N {{\setminus }} J)$, then $f(cl(N{{\setminus }} L)) \subset f(cl(N {{\setminus }} J)) \subset int(N)$.

Property a) follows from the construction of the pair (N, L).

Property b) only needs the proof of $L_i \cap \partial (N) \ne \emptyset$. In fact, if this is not true, $L_i \subset int (N)$ and $f(\partial (L_i)) \subset int( N)$. Then, there exists $D_j \subset f(L_i)$ with $f(\partial (D_j)) \subset N^c$ and, therefore, $f(D_j) \subset N^c$. It is easy to see that the set $N'=N \cup D_j$ is a manifold, isolating block of K, with a hole less than N which is a contradiction.

Property c) is also easy to see from the construction of (N, L). $\square$

Let $f: U \subset {{\mathbb {R}}}^2 \rightarrow f(U) \subset {\mathbb R}^{2}$ be a homeomorphism, and let K be an isolated invariant continuum. Then, we have the following two remarks:

Remark 2

There exists a strong filtration pair (N, L), where N is a compact connected 2-manifold, isolating block of K, such that $S^2 {{\setminus }} int(N)$ has a minimum possible number of components $\{D_{1}, \dots , D_{p}\}$ which are discs, and $L=L_{1} \cup \cdots \cup L_{m} \subset N$ is a finite union of discs with, at most, one hole. Finally, $\partial _N(L_i)$ is an arc with $\partial _N(L_i) \cap \partial (N)=\{a_i, b_i\}$ two points, or $\partial _N(L_i)$ is a Jordan curve.

Remark 3

If $\partial _{N}(L_{j})$ is homeomorphic to $S^{1}$, then $L_{j}$ is a disc with a hole. Furthermore, $S^2 {{\setminus }} L_{j}$ has two connected components: one of them contains K and the other one is $int(D_i)$ for some i. On the other hand, if $\partial _{N}(L_{j})$ is an arc, then $L_{j}$ is a disc without holes. Therefore, $cl(N {{\setminus }} L)$ is a disc with the same amount of holes as N.

In the conditions of Proposition 1, let $cl(N{{\setminus }} L)/ \sim \quad = N_L$ be the quotient space obtained by identifying each component, $\partial _i$, of $\partial _{N}(L) \subset cl(N{{\setminus }} L)$ to a point $q_{i}$ ($i=1, \dots , m'$, with $m' \ge m$). Since $f(\partial _i) \subset int(L_{j})$ for some j, there exists a induced continuous map $f': N_L \rightarrow N_L$ with $f' \equiv f$ in a neighborhood of K, $f'(\{q_1, \dots , q_{m'}\}) \subset \{q_1, \dots , q_{m'}\}$ and $f'(U'(q_{i}))=q_{j}$ for a neighborhood $U'(q_{i})$ of $q_i$.

Obviously $Fix(f') \subset K \cup \{q_{1}, \dots , q_{m'}\}$ and, since $cl(N {{\setminus }} L)$ is an isolating neighborhood of K, $Fix((f')^{k}) \subset K \cup \{q_{1}, \dots , q_{m'}\}$.

By applying $f^*$ to the family $\{I(K)_{1}, \dots , I(K)_{p}\}$ we obtain a finite union of cycles. We have $p=t_{1}+ \cdots +t_{l}$, with $\{t_{1}, \dots , t_{l}\}$ the lengths of the cycles. Let us change the notation of the open sets $\{I(K)_{i}\}_{i=1}^p$ and the connected components $\{D_{i}\}_{i=1}^p$, by $\left\{ \{I(K)_{1,j}\}_{j=1}^{t_1}, \dots , \{I(K)_{l,j}\}_{j=1}^{t_l} \right\}$ and $\left\{ \{D_{1,j}\}_{j=1}^{t_1}, \dots , \{D_{l,j}\}_{j=1}^{t_l}, \right\}$ where $D_{i,j} \subset I(K)_{i,j}$ and each family $\{I(K)_{i,j}\}_{j=1}^{t_i}$, with $i \in \{1,\dots ,l\}$, is a cycle of length $t_{i}$. Let us observe that $(f^*)^{t_{i}}:I(K)_{i,j} \rightarrow I(K)_{i,j}$ with $i \in \{1, \dots , l\}$, $j \in \{1, \dots , t_{i}\}$.

Remark 4

a) Given a cycle $\{I(K)_{i,j}\}_j$, if $I(K)_{i,1}$ contains a component $L_{h_1}$ of L which decompose $S^n$ into two connected components, one of them $int(D_{i,1})$, then there exists $L_{h_j} \subset I(K)_{i,j}$ which decomposes $S^n$ in two connected components, one of them $int(D_{i,j})$. Moreover, there exists only one component of L, $L_{h_j}$, in each $I(K)_{i,j}$.

b) Given a cycle $\{I(K)_{i,j}\}_j$, if $I(K)_{i,1} \cap L= \emptyset$, then $I(K)_{i,j} \cap L= \emptyset$.

Definition 2

Let us decompose the set of lengths of cycles of $I(K)\subset S^n$, $\{t_{1}, \dots , t_{l}\}$ into three disjoint sets $t_{A}, t_{R}, t_{S} \subset \{t_{1}, \dots , t_{l}\}$ such that $t_{A} \cup t_{R} \cup t_{S}=\{t_{1}, \dots , t_{l}\}$, with $t_{A}$ the set of lengths of cycles corresponding to case b) of Remark 4 (attracting), $t_{R}$ the set of lengths of cycles corresponding to case a) of Remark 4 (repelling), and $t_{S}$ the rest of lengths of $\{t_{1},\dots ,t_{l}\}$ (hyperbolicity).

The next theorem is a very useful tool for detecting periodic orbits in K:

Theorem 1

Let $f: S^{n} \rightarrow S^n$ be a homeomorphism and let (N, L) be a strong filtration pair with $K=Inv(N,f)$. Then,

$$\begin{aligned} i_{{{\mathbb {R}}}^{n}}(f^{k}, K)=\sum _{i=0}^n (-1)^i tr((f')^k_*)-l_k, \end{aligned}$$

where $l_k \in {{\mathbb {N}}}$ is the number of $q_{i} \in \{q_{1}, \dots , q_{m'}\} \cap Fix((f')^{k})$.

Proof

By the additivity property of the fixed point index

$$\begin{aligned} i_{N_L}((f')^{k}, N_L)=i_{{{\mathbb {R}}}^{n}}(f^{k}, K)+\sum _{q_{i} \in \{q_{1}, \dots , q_{m'}\} \cap Fix((f')^{k})} i_{N_L}((f')^{k}, q_{i}) \end{aligned}$$

We have that, in the last equality, $i_{N_L}((f')^{k}, q_{i})=1$. Then, $i_{N_L}((f')^{k}, N_L)=i_{{{\mathbb {R}}}^{n}}(f^{k}, K)+l_k$.

On the other hand,

$$\begin{aligned} i_{N_L}((f')^{k}, N_L)=\Lambda ((f')^k)=\sum _{i=0}^n (-1)^i tr((f')^k_*) \end{aligned}$$

We conclude that

$$\begin{aligned} i_{{{\mathbb {R}}}^{n}}(f^{k}, K)=\sum _{i=0}^n (-1)^i tr((f')^k_*)-l_k \end{aligned}$$

$\square$

Let us see some examples obtained as corollaries of the above theorem.

Example

Let $f: S^{n} \rightarrow S^{n}$ be a homeomorphism and let (N, L) be a strong filtration pair with N an n-dimensional ball such that both $S^n {{\setminus }} int(N)$ and L are finite union of disjoint topological balls with $\partial _N(L_i)$ connected for all i. Denote by m the number of components of $S^n {{\setminus }} int(N)$. Then, there exists k such that

$$\begin{aligned} i_{{{\mathbb {R}}}^{n}}(f^{k}, K)=\sum _{i=0}^n (-1)^i tr((f')^k_*)-l_k=1+(-1)^{n-1}m-l_k \end{aligned}$$

The last equality follows from the fact that $H_i(N_L)=0$ for $i \notin \{0,n-1\}$, $H_{0}(N_L)={{\mathbb {Q}}}$ and $H_{n-1}(N_L)={{\mathbb {Q}}} \oplus \cdots \oplus {{\mathbb {Q}}}$ has m generators. The number k is selected in such a way that the generators of $H_{n-1}(N_L)$ are invariant under the action of $(f')^k_*$.

If n is even then $i_{{{\mathbb {R}}}^{n}}(f^{k}, K) =1-m -l_k$ for certain $k \in {{\mathbb {N}}}$. If N decomposes ${{\mathbb {R}}}^n$ into more than two connected components ($m \ge 2$), then $i_{{\mathbb R}^{n}}(f^{k}, K) =1-m -l_k \le -1$ for certain $k \in {\mathbb N}$, and there exist periodic orbits in K.

If n is odd, $i_{{{\mathbb {R}}}^{n}}(f^{k}, K) =1+m-l_k$ for certain $k \in {{\mathbb {N}}}$. In this case, if $m \ne l_k-1$ we detect periodic orbits in K.

Let us observe that in this example, if $L=\emptyset$, K is an attractor and we obtain:

If n is even, $i_{{{\mathbb {R}}}^{n}}(f^{k}, K) =1-m$.

If n is odd, $i_{{{\mathbb {R}}}^{n}}(f^{k}, K) =1+m \ge 1$ which give us the existence of periodic orbits in K.

3 Main theorem. Orientation preserving case

From now on, our attention will be focused on the two-dimensional case. By remark 2, we can suppose that L has m connected components and that $\partial _N(L)$ also has m connected components which give us, after the identification in $N_L$, the family of points $\{q_1, \dots , q_m\}$. Let us observe that $\partial (N {{\setminus }} L)/\sim$ is a finite and disjoint union of jordan curves and points. Take the projection $\pi :cl(N {{\setminus }} L) \rightarrow N_L$ and a retraction $r:N \rightarrow cl(N {{\setminus }} L)$ with $r(x)=x \text { if } x \in cl(N {{\setminus }} L)$ and r retracts each $L_i$ to $\partial _N(L_i)$.

Definition 3

Let $\theta =\{p_{1},\dots ,p_{s}\} \subset \{q_{1},\dots ,q_{m}\}$ be such that $f'(\theta )=\theta$. We say that $p_{i},p_{j} \in \theta$ are adjacent in $\theta$ if there is an arc $\gamma \subset \partial (N {{\setminus }} L) / \sim$ such that $\gamma \cap \theta =\{p_{i},p_{j}\}$.

Lemma 4

([25]). Let $\theta =\{p_{1}, \dots , p_{s}\} \subset \{q_{1}, \dots , q_{m}\}$ such that $f'(\theta )=\theta$. If $p_{i}, p_{j}$ are adjacent in $\theta$, then their images by $f'$, $p_{i+1}$ and $p_{j+1}$ are adjacent in $\theta$.

In this section we consider an orientation preserving homeomorphism $f:U \subset {{\mathbb {R}}}^2 \rightarrow f(U) \subset {{\mathbb {R}}}^2$.

Proposition 2

If f is orientation preserving and $\{I(K)_{i,j}\}_{j=1}^{t_i}$ is a cycle, the set of periodic orbits of $f'|_{(\pi \circ r)(\bigcup _{j=1}^{t_{i}} \partial (D_{i,j}))}$ is such that all its orbits have the same period $r_i=n_i t_i$ for some $n_i \in {{\mathbb {N}}}$.

Proof

Let us fix an orientation in $I(K)_{i,1} \cap \partial (N {{\setminus }} L) \simeq S^{1}$. The jordan curve $f(I(K)_{i,1} \cap \partial (N {{\setminus }} L)) \subset I(K)_{i,2}$ bounds $D_{i,2} \subset I(K)_{i,2}$ preserving orientation.

In case a) of Remark 4 we only have a periodic orbit of period $t_{i}$. This is also true in the n-dimensional case.

In case b) of Remark 4 the result is obvious because $(\pi \circ r)(\bigcup _{j=1}^{t_{i}} \partial (D_{i,1})) \cap \{q_{1}, \dots , q_{m}\}=\emptyset$. It is also true in the n-dimensional case.

In any other case, given two periodic orbits $\theta _{1}=\{p_{i1},\dots ,p_{i r_1}\}$ and $\theta _{2}=\{p'_{i1},\dots ,p'_{ir_2}\}$, by Lemma 4 it is easy to see that $r_1=r_2=n_i t_i$ for some $n_i \in {{\mathbb {N}}}$. In the n-dimensional case this is not true in general. $\square$

Corollary 7

In the conditions of the last proposition, and given $k \in {\mathbb N}$, $(f')^{k}$ has fixed points in $(\pi \circ r)(\bigcup _{j=1}^{t_{i}} \partial (D_{i,j})) \subset \partial (N {{\setminus }} L)/\sim$ if and only if $k \in r_{i}{\mathbb N}$. Then, the number of fixed points is $r_{i}q^{i}$, with $q^{i}$ the number of periodic orbits of $f'$ in $(\pi \circ r)(\bigcup _{j=1}^{t_{i}} \partial (D_{i,j}))$.

Remark 5

It is obvious that $r_{j}=q^{j}=0$ for all $t_{j} \in t_{A}$, $r_{j}=t_{j}$ with $q^{j}=1$ for all $t_{j} \in t_{R}$ and $r_{j}=n_j t_{j}$ for all $t_{j} \in t_{S}$.

The family of periods $\{r_{1}, \dots , r_{l}\}$ and the number of periodic orbits of each period $\{q^{1}, \dots , q^{l}\}$ permit us to compute the number of fixed points of $(f')^{k}$ in $\{q_{1}, \dots , q_{m}\}$ for all $k \in {{\mathbb {N}}}$.

Proposition 3

If f is orientation preserving, then

$$\begin{aligned} i_{N_L}((f')^{k}, N_L)= 2-\sum _{{\mathop {k \in t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \in t_{A} \cup t_{S}}}} t_{i} \end{aligned}$$

Proof

If K is not a repeller, $N_L$ is homeomorphic to a disc with a finite amount of holes. On the other hand, $H_{0}(N_L)={{\mathbb {Q}}}$, $H_{1}(N_L)={{\mathbb {Q}}} \oplus \cdots \oplus {{\mathbb {Q}}}$ with $\sum _{t_{i} \in t_{A} \cup t_S}t_{i}-1$ generators and $H_{2}(N_L)=0$. Since $i_{N_L}((f')^{k}, N_L)=\Lambda ((f')^{k}_*)$, from the study of the trace of $(f')^k_{*}:H_{1}(N_L) \rightarrow H_{1}(N_L)$ it is easy to obtain the value of the fixed point index (see Fig. 5).

Fig. 5

Picture of $N_L$ and behavior of $(f')^k_{*}$

If K is a repeller, $N_L \simeq S^2$ and $H_{0}(N_L)={\mathbb Q}, \quad H_{1}(N_L)=0, H_{2}(N_L)={{\mathbb {Q}}}$. We obtain $i_{N_L}((f')^{k}, N_L)=2$. $\square$

Main Theorem 2

(Orientation preserving case). Let $f:U \subset {{\mathbb {R}}}^{2} \rightarrow f(U) \subset {{\mathbb {R}}}^{2}$ be an orientation preserving homeomorphism and let K be an isolated invariant continuum. Then,

$$\begin{aligned} i_{{{\mathbb {R}}}^{2}}(f^{k},K)=2-\sum _{{\mathop {k \in t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \in t_{A} \cup t_{R} \cup t_S}}} t_{i}-\sum _{{\mathop {k \in n_i t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \in t_S}}} t_i n_i q^{i} \le 2 \end{aligned}$$

Proof

By the additivity property of the fixed point index,

$$\begin{aligned} i_{N_L}((f')^{k}, N_L)=i_{{{\mathbb {R}}}^{2}}(f^{k}, K)+\sum _{q_{i} \in \{q_{1}, \dots , q_{m}\} \cap Fix((f')^{k})} i_{N_L}((f')^{k}, q_{i}) \end{aligned}$$

Since the value of each index in the last summation is 1 ($f'$ is constant in a neighborhood of each $q_{i}$), we have

$$\begin{aligned} i_{{{\mathbb {R}}}^{2}}(f^{k},K)=2-\sum _{{\mathop {k \in t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \in t_{A} \cup t_{S}}}} t_{i}-\sum _{{\mathop {k \in r_{i}{{\mathbb {N}}}}\limits ^{r_{i} \in \{r_1, \dots r_l\}}}} r_{i}q^{i} \end{aligned}$$

obtaining the result we are looking for. $\square$

Remark 6

(Proofs of Corollaries 1, 2 and 3). Let us observe that the sequence $\{i_{{{\mathbb {R}}}^{2}}(f^{k},K)\}_k$ is periodic and, if K decomposes $S^2$ in p connected components,

$$\begin{aligned} 2-p-\sum _{t_{i} \in t_S} t_i n_i q^{i} \le i_{{\mathbb R}^{2}}(f^{k},K) \le 2 \end{aligned}$$

for all k. If $k \in \left( \prod _{t_i \in t_A \cup t_R} t_i\right) \left( \prod _{t_i \in t_S} n_it_i \right) {{\mathbb {N}}}$, then $i_{{{\mathbb {R}}}^{2}}(f^{k},K)=2-p-\sum _{t_{i} \in t_S} t_i n_i q^{i}$. This proves Corollary 1. Consequently, if $p \ge 3$, K has periodic orbits of period which divides $\left( \prod _{t_i \in t_A \cup t_R} t_i\right) \left( \prod _{t_i \in t_S} n_it_i \right)$ which proves Corollary 2. The proof of Corollary 3 is also easy. If $f:S^2 \rightarrow S^2$ is a homeomorphism with K an invariant continuum which contains Per(f), then K is not isolated because, if K were isolated, there should be $k_0 \in {{\mathbb {N}}}$ with $i_{{\mathbb R}^{2}}(f^{nk_0},K) \le 1$. Then,

$$\begin{aligned} 2=i_{S^2}(f^{2k_0}, S^2)=i_{{{\mathbb {R}}}^2}(f^{2k_0}, K) \le 1, \end{aligned}$$

and we obtain a contradiction. A similar argument permits us to prove the same result if K has a finite amount of connected components $K=\bigcup _{i=1}^n K_i$ with each $K_i$ in the conditions of the Corollary 3. If K were isolated, there should be $k_0 \in 2{{\mathbb {N}}}$ such that each $K_i$ is invariant and isolated for $f^{k_0}$ and $i_{{{\mathbb {R}}}^2}(f^{k_0}, K_{i}) \le 0$ ($t_S \ne \emptyset$ for each $K_i$). Then

$$\begin{aligned} 2=\sum _{i=1}^n i_{{{\mathbb {R}}}^2}(f^{k_0}, K_{i}) \le 0, \end{aligned}$$

which is a contradiction. $\square$

Let us suppose that the domain U of f is an open ball. In this case, there exists an invariant component, which we call $I(K)_0$, of I(K) ($t_0=1$) which contains $\infty$. We obtain the next corollary.

Corollary 8

If the domain U of f is an open ball, then $i_{{\mathbb R}^{2}}(f^{k}, K) \le 1$ for all $k \in {{\mathbb {N}}}$. Moreover, if K is an attractor or a repeller, $i_{{\mathbb R}^{2}}(f^{k},K)=1-\sum _{{\mathop {k \in t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \ne t_0}}} t_i$.

Example

Let us see an example of index 2 with U an open set which is not an open ball. Let $p=(p_1, p_2) \in U={{\mathbb {R}}}^2 {{\setminus }} \{0\}$. We define the homeomorphisms:

$S_1: U \rightarrow U$ as $S_1(p)=\frac{1}{|| p ||^2} \, p$, $S_2: {{\mathbb {R}}}^2 \rightarrow {{\mathbb {R}}}^2$ by $S_2(p_1,p_2)=(-p_1,p_2)$ and $f: {{\mathbb {R}}}^2 \rightarrow {\mathbb R}^2$ by $f(p)=||p||p$.

Both $S_1$ and $S_2$ are orientation reversing and f is orientation preserving. The map $g=f \circ S_2 \circ S_1 :U \rightarrow U$ is an orientation-preserving homeomorphism. The unit circle $K=\{p : ||p||=1\}$ is an isolated invariant continuum for g and it is easy to see if we consider the extension ${\bar{g}}:S^2 \rightarrow S^2$ that $i_{{{\mathbb {R}}}^2}(g,K)=2$.

4 Main theorem. Orientation-reversing case

Proposition 4

Let $f:U \subset {{\mathbb {R}}}^2 \rightarrow f(U) \subset {\mathbb R}^2$ be an orientation reversing homeomorphism and let K be an isolated invariant continuum. Given a cycle $\{I(K)_{i,j}\}_{j=1}^{t_i}$, there are two possibilities:

• a) $t_{i}$ is even. Then $f'|_{(\pi \circ r)(\bigcup _{j=1}^{t_{i}} \partial (D_{i,j}))}$ has $q^{i}$ periodic orbits of the same period $r_{i}=t_{i}n_{i}$.

• b) $t_{i}$ is odd. Then $f'|_{(\pi \circ r)(\bigcup _{j=1}^{t_{i}} \partial (D_{i,j}))}$ has $q^{i,1} \le 2$ periodic orbits of period $r_{i,1}=t_{i}$ and $q^{i,2}$ periodic orbits of period $r_{i,2}=2t_{i}$.

Proof

The case a) follows from Lemma 4. Let us see the case b). If $t_{i}$ is odd, the periodic orbits have period $r_{i,1}=t_{i}$ or $r_{i,2}=2t_{i}$. In fact, take a periodic orbit $\theta =\{p_{1},\dots ,p_{r}\}$ of period r. Let $p_{1} < p_{2}$ be adjacent in $\theta$ (with the ordering given by the orientation of $\partial (N {{\setminus }} L)/\sim$). By Lemma 4, $(f')^{t_{i}}(p_{1}) > (f')^{t_{i}}(p_{2})$ are adjacent in $\theta$. Let us observe that if $(f')^{t_{i}}(p_{1})=p_2$, then $(f')^{t_{i}}(p_{2})=p_1$ and $r=2t_i$. Let us suppose that $r > 2t_{i}$, and let $p_{3} \in \theta$, $p_3 \ne p_1$, such that $p_{2} < p_{3}$ are adjacent in $\theta$. Then, by an induction argument we find a point of $\theta$ with period $\le 2t_{i}$, which is a contradiction. Then $r \le 2t_{i}$.

Let us prove that the number of periodic orbits of period $t_{i}$ is $\le 2$. In fact, let $\theta _{1}, \theta _{2}$ be two periodic orbits of period $t_{i}$. If there is another periodic orbit $\theta _{3}$, we obtain, using Lemma 4 and the fact that f is orientation reversing, that the period of $\theta _{3}$ is $> t_{i}$. $\square$

Proposition 5

If f is orientation reversing, then

$$\begin{aligned} i_{N_L}((f')^{k}, N_L)= 1+(-1)^{k}-\sum _{{\mathop {k \in t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \in t_{A} \cup t_{S}}}} (-1)^{k} t_{i}. \end{aligned}$$

The proof of this result is analogous to the proof of Proposition 3 except that now f is orientation reversing. We leave it to the reader.

The family of lengths of cycles $\{t_{1},\dots ,t_{l}\}$ decomposes into two disjoint sets: the set of even lengths $t_{P}$ and the set of odd lengths $t_{I}$.

Main Theorem 3

(Orientation-reversing case). Let $f:U \subset {{\mathbb {R}}}^{2} \rightarrow f(U) \subset {{\mathbb {R}}}^{2}$ be an orientation-reversing homeomorphism and let K be an isolated invariant continuum. Then

$$\begin{aligned} i_{{{\mathbb {R}}}^{2}}(f^{k}, K)= & {} 1+(-1)^k-\sum \limits _{{\mathop {k \in t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \in t_{A} \cup t_{S}}}} (-1)^{k} t_{i} -\sum \limits _{{\mathop {k \in n_i t_{i}{\mathbb N}}\limits ^{t_{i} \in t_{P} \cap t_S}}} n_i t_{i}q^{i} \\&-\sum \limits _{{\mathop {k \in t_{i}{\mathbb N}}\limits ^{t_{i} \in t_{I} \cap t_S}}} t_{i}q^{i,1}-\sum \limits _{{\mathop {k \in 2t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \in t_{I} \cap t_S}}} 2t_{i}q^{i,2}-\sum \limits _{{\mathop {k \in t_{i}{{\mathbb {N}}}}\limits ^{t_{i} \in t_R}}}t_i. \end{aligned}$$

Proof

By the additivity property of the fixed point index

$$\begin{aligned} i_{N_L}((f')^{k}, N_L)=i_{{{\mathbb {R}}}^{2}}(f^{k}, K)+\sum _{q_{i} \in \{q_{1}, \dots , q_{m}\} \cap Fix((f')^{k})} i_{N_L}((f')^{k}, q_{i}), \end{aligned}$$

where each index of the last summation is 1. Then the result follows automatically. $\square$

Remark 7

If f is orientation reversing, the sequence $\{i_{{\mathbb R}^{2}}(f^{k},K)\}_k$ is periodic and, if K decompose $S^2$ in p connected components,

$$\begin{aligned} 2-p-\sum _{t_{i} \in t_{P} \cap t_S} n_i t_{i}q^{i} -\sum _{t_{i} \in t_{I} \cap t_S} t_{i}q^{i,1}-\sum _{t_{i} \in t_{I} \cap t_S} 2t_{i}q^{i,2} \le i_{{{\mathbb {R}}}^{2}}(f^{k}, K) \le 2 \end{aligned}$$

for all k even.

If $k \in 2\left( \prod _{t_i \in t_A \cup t_R} t_i\right) \left( \prod _{t_i \in t_S} n_it_i \right) {{\mathbb {N}}}$, with $n_i=1$ if $t_i \in t_I \cap t_S$, then $i_{{\mathbb R}^{2}}(f^{k},K)=2-p-\sum _{t_{i} \in t_{P} \cap t_S} n_i t_{i}q^{i} -\sum _{t_{i} \in t_{I} \cap t_S} t_{i}q^{i,1}-\sum _{t_{i} \in t_{I} \cap t_S} 2t_{i}q^{i,2}$.

Consequently, if $p \ge 3$, K has periodic orbits of periods which divide $2\left( \prod _{t_i \in t_A \cup t_R} t_i\right) \left( \prod _{t_i \in t_S} n_it_i \right)$ with $n_i=1$ if $t_i \in t_I \cap t_S$.

Remark 8

Let $f:U \subset {{\mathbb {R}}}^{2} \rightarrow f(U) \subset {\mathbb R}^{2}$ be a homeomorphism and let K be a compact isolated invariant set with a finite amount of connected components $K=\bigcup _{i=1}^{m_1}K_{1,i} \cup \cdots \cup \bigcup _{i=1}^{m_r}K_{r,i}$, such that $f(K_{i,j})=K_{i,j+1}$ and $f(K_{i,m_{i}})=K_{i,1}$. Then by the additivity property of the fixed point index applied to $f^k$, $i_{{{\mathbb {R}}}^{2}}(f^{k}, K)=\sum _{k \in m_{i}{{\mathbb {N}}}} m_{i} i_{{{\mathbb {R}}}^{2}}(f^{k}, K_{i,1})$.

Corollary 9

Let $f:S^{2} \rightarrow S^{2}$ be a homeomorphism and let K be a compact, invariant and proper set with a finite amount of connected components. Then $f:S^{2} {{\setminus }} K \rightarrow S^{2} {{\setminus }} K$ is not minimal.

Proof

Our proof is analogous to that given in [18] for K a finite amount of points. If $f|_{S^2 {{\setminus }} K}$ is minimal, K is an isolated invariant set which is not an attractor nor a repeller. For an adequate 2n we have that the components of K and $S^2 {{\setminus }} K$ are invariant for the orientation-preserving homeomorphism $f^{2n}$ and, by Remarks 6 and 8 applied to $f^{2n}$, $i_{S^2}(f^{2n}, K) \le 0$, but it is not possible because $2=\Lambda (f^{2n}_*)=i_{S^2}(f^{2n}, S^2)=i_{S^2}(f^{2n}, K)$. $\square$

5 Dynamical behavior and fixed point index

In this section we study the relationship between the fixed point indices $i_{{{\mathbb {R}}}^2}(f^n, K)$ and the local dynamics of f in a neighborhood of K. If (N, L) is a strong filtration pair, define $cl(N{{\setminus }} L)_K$ and ${{\mathbb {R}}}^2_K$ as the spaces obtained from $cl(N {{\setminus }} L)$ and ${{\mathbb {R}}}^2$ by the identification of K to a point [k] and define $f :cl(N {{\setminus }} L)_K \rightarrow {{\mathbb {R}}}^2_K$ as the map induced by f (with the same notation). Let us observe that $cl(N {{\setminus }} L)_K$ is the pointed union of a finite family of p discs, $E_i$, where p is de number of components of ${{\mathbb {R}}}^2 {{\setminus }} K$. Denote $cl(N {{\setminus }} L)_K=\bigvee _{i=1}^{p} E_i$ with $E_i \cap E_j=[k]$ for all $i \ne j$.

Theorem 2

Let K be an isolated continuum for a homeomorphism $f: U \subset {{\mathbb {R}}}^2 \rightarrow f(U) \subset {{\mathbb {R}}}^2$ which decomposes the plane in p components and let (N, L) be a strong filtration pair. Then there exist two families of closed discs, $\{A_1, \dots , A_a\}$, $\{R_1, \dots , R_r\}$, in $cl(N {{\setminus }} L)_K=\bigvee _{i=1}^{p} E_i$ and two families of continua without holes in $cl(N {{\setminus }} L)_K$, $\{S_1, \dots , S_s\}$, $\{U_1, \dots , U_s\}$, with

$$\begin{aligned} s= & {} -i_{{{\mathbb {R}}}^2}(f^d, K)+2-p. \\ a= & {} -i_{{{\mathbb {R}}}^2}(f^d, K)+2-\sum \limits _{t_i \in t_R \cup t_S}t_i - s.\\ r= & {} -i_{{{\mathbb {R}}}^2}(f^d, K)+2-\sum \limits _{t_i \in t_A \cup t_S}t_i - s. \end{aligned}$$

for $d=2\left( \prod _{t_i \in t_S} t_i n_i\right) \left( \prod _{t_i \in t_A \cup t_R} t_i\right)$, ($n_i=1$ if f is orientation reversing and $t_i \in t_I \cap t_S$). These sets satisfy the following properties (Fig. 6):

• 1. $\bigcup _{i=1}^s S_i \subset K^+$ and $\bigcup _{i=1}^s U_i \subset K^-$. The set $K^+$ is the connected component of $Inv^+(cl(N {{\setminus }} L)_K,f)$ which contains [k] and the set $K^-$ is the connected component of $Inv^-(cl(N {{\setminus }} L)_K,f)$ which contains [k].

• 2. $S_i \cap S_j = U_i \cap U_j = S_i \cap U_j = [k]$, every $S_i \subset E_k$ for some $k=1, \dots , p$ and every $U_j \subset E_{k'}$ for some $k'=1, \dots , p$.

• 3. $f^d(S_i) \subset S_i$, $f^{-d}(U_i) \subset U_i$, $\bigcap _{n \in {{\mathbb {N}}}} f^{nd} (S_i)=\bigcap _{n \in {\mathbb N}} f^{-nd} (U_i)=[k]$.

• 4. The sets $\{S_{i}\}_i$ and $\{U_{i}\}_i$ alternate in the circles of $\partial (cl(N {{\setminus }} L)) \subset cl(N {{\setminus }} L)_K$

• 5. $f^d(A_i) \subset int(A_i)$, $f^{-d}(R_j) \subset int(R_j)$ and $\bigcap _{n \in {{\mathbb {N}}}} f^{nd} (A_i) = \bigcap _{n \in {{\mathbb {N}}}} f^{-nd} (R_i) =[k]$. for all $i=1, \dots , a$ and $j=1, \dots , r$.

Fig. 6

Dynamics of $cl(N {\setminus } L)_K$

Proof

Let us consider the topological space $N_{L,K}$ obtained from the identification in $N_L$ of K to a point [k] and let $f':N_{L,K} \rightarrow N_{L,K}$ be the induced map obtained from $f':N_{L} \rightarrow N_{L}$. The space $N_{L,K}$ is the pointed union of $\sum _{t_i \in t_A \cup t_S} t_i$ discs, $\{C_j\}_j$, and $\sum _{t_i \in t_R} t_i=r$ spheres, $\{T_j\}_j$, with $C_i \cap C_j=T_i\cap T_j=C_i \cap T_j=[k]$. The map $(f')^d$ sends each disc to itself and each sphere to itself.

The rest of the proof is similar to that given in Proposition 7 of [26]. Let $\theta =\{p_1, \dots , p_s\} \subset \{q_1, \dots , q_m\} \subset N_{L,K}$ be the biggest subset on which $f'$ acts as a permutation and such that its points are not isolated points of $\pi _K(\partial (N{{\setminus }} L)) \subset N_{L,K}$ with $\pi _K :cl(N {{\setminus }} L)_K \rightarrow N_{L,K}$ defined as $\pi$ but in $cl(N {{\setminus }} L)_K$. It is clear that $\theta$ has $s=\sum _{t_i \in t_S} n_it_iq^i$ elements if f preserves orientation and $s=\sum _{t_i \in t_P \cap t_S} n_i t_i q^i+ \sum _{t_i \in t_I \cap t_S}t_iq^{i,1}+\sum _{t_i \in t_I \cap t_S}2t_iq^{i,2}$ elements if f reverses orientation. Moreover, $(f')^d(p_i)=p_i$ for all $i=1, \dots ,s$.

Define $A=\{x \in N_{L,K} \text { such that there exists } n_x \text { with } (f')^{n_x}(x) \in \theta \}$ and $A(p_i)$ the connected component of A which contains $p_i \in \theta$. It is clear that $A(p_i) \subset C_j$ for some disc $C_j$ of $N_{L,K}$.

Let $K_i=\bigcap _{n \in {{\mathbb {N}}}} (f')^{nd}(cl(A(p_i))$, $i=1, \dots , s$. Since $(f')^{d}(cl(A(p_i)) \subset cl(A(p_i))$, $K_i$ is a continuum which contains $p_i$ and [k] with $(f')^d(K_i)=K_i \subset C_j$. Define

$$\begin{aligned} U_i=(\pi _K^{-1}(K_i) \cap K^- ) \end{aligned}$$

Let us construct the sets $S_i$. If $p_{i-1}, p_i \in \theta$ are adjacent in $\pi _{K}(\partial (cl(N {{\setminus }} L)))$, let $\gamma \subset \pi _{K}(\partial (cl(N {{\setminus }} L)))$ be an arc joining $p_{i-1}$ and $p_i$ with $\gamma \cap \theta =\{p_{i-1}, p_i\}$ and $\gamma \subset C_j$ for some disc $C_j$ of the pointed union $N_{L,K}$. There is a component $K_{p_i}$ of $\partial (A(p_i))$ separating $p_{i}$ from $\theta {{\setminus }} p_i$, $[k] \in K_{p_i}$ with $\lim _{n \rightarrow \infty }(f')^{nd}(x)=[k]$ for all $x \in K_{p_i}$, and such that $K_{p_i} \cap \gamma \ne \emptyset$. Let $B_i$ be the connected component of $C_j {{\setminus }} (K_{i-1} \cup K_i)$ containing $K_{p_i} \cap \gamma$. Define

$$\begin{aligned} S_i=(\pi _K^{-1}(B_i) \cup [k]) \cap K^+ \end{aligned}$$

It is easy to prove that the sets $U_i$ and $S_i$ satisfy the properties 1 to 4 of the theorem. The details are left to the reader.

The equality relating the number s of sets $\{U_i\}$ and $\{S_i\}$ with the fixed point index, $s=-i_{{{\mathbb {R}}}^2}(f^d, K)+2-p$ follows, in the orientation preserving case, from the Main Theorem 1:

$$\begin{aligned} s = \sum _{t_i \in t_S} n_i t_i q^i=-i_{{{\mathbb {R}}}^2}(f^d, K)+2-p. \end{aligned}$$

For the orientation-reversing case we have, from the Main Theorem 2 the following:

$$\begin{aligned} s= & {} \sum _{t_i \in t_P \cap t_S} n_i t_i q^i+ \sum _{t_i \in t_I \cap t_S}t_iq^{i,1}+\sum _{t_i \in t_I \cap t_S}2t_iq^{i,2}\\= & {} -i_{{{\mathbb {R}}}^2}(f^d, K)+2-p. \end{aligned}$$

For every disc $E_i$ of the pointed union $\bigvee _{i=1}^{p+1} E_i$ such that $E_i \cap L=\emptyset$, define $A_i=E_i$ and for every $E_j$ such that $E_j \cap L$ is a circumference, define $R_j=E_j$. The proof of the property 5 is immediate.

The equalities relating a and r with the fixed point index follow from

$$\begin{aligned} a= & {} \sum _{t_i \in t_A} t_i=-i_{{{\mathbb {R}}}^2}(f^d, K)+2-\sum _{t_i \in t_R \cup t_S}t_i - s.\\ r= & {} \sum _{t_i \in t_R} t_i=-i_{{{\mathbb {R}}}^2}(f^d, K)+2-\sum _{t_i \in t_A \cup t_S}t_i - s. \end{aligned}$$

$\square$