Rectal cancer management has been a challenging area for surgeons. There are two main goals in rectal cancer management: oncologic outcome and quality of life. Cancer in the distal third of the rectum poses a significant challenge in maintaining these goals. In the early 1900's, Ernest Miles developed the abdominoperineal resection (APR), which has become the gold standard of treatment of rectal adenocarcinoma of the distal rectum [1]. In the 1980s, total mesorectal excision (TME) was described by Heald as an optimal oncological resection for rectal cancer [2]. With an improved understanding of rectal cancer, there has been a progressive change in management strategies and technical advances. This has allowed us to shift away from the APR with permanent colostomy to more sphincter-preserving methods, and more recently minimally invasive techniques such as laparoscopic surgery and transanal endoscopic microsurgery (TEM).

Sphincter preservation in the low pelvis has deterred surgeons because of its extreme difficulty and the challenge in obtaining adequate distal as well as circumferential margins. This is due to the confines of the bony pelvis as well as the tapering of the mesorectum. Historically, resections in this area have been prone to local recurrence (LR) [3] and have posed significant incontinence and poor quality-of-life issues for patients. In 1976, a rectal cancer management program designed to reduce LR and extend indications for sphincter preservation for rectal cancers located in the distal third of the rectum was developed. This originally included preoperative high-dose radiotherapy at 4,500 (cGy) and has since expanded to include preoperative high-dose radiotherapy of 5,580 cGy with concurrent 5-fluorouracil (5-FU) or capecitabine in conjunction with novel surgical techniques. The transanal abdominal transanal (TATA) proctosigmoidectomy with descending hand-sewn coloanal anastomosis was developed by Gerald Marks at Thomas Jefferson University in 1984 [4, 5].

TEM was introduced by Buess et al. in 1983 [6]. This operating microscope was initially used with resection of benign lesions in the low rectum. Early limitations were the higher LR and metastasis rates compared with more radical surgery [7]. Recurrence and metastasis rates were predictably high as a result of the risk of nodal involvement and the failure to address the nodal basin or pelvic sidewall with TEM alone [8]. The predicted risk of nodal involvement by T stage is 0–12 % for T1 lesions, 12–28 % for T2 lesions, and 36–79 % for T3 lesions [9]. As a result, traditionally, TEM was only indicated for patients who were not candidates for major abdominal surgery. Recently there has been an increasing body of literature supporting the use of TEM with T1 rectal cancers and favorable histology, and lesions with entry into the peritoneal cavity; more recently, in combination with high-dose chemoradiotherapy, it has become a minimally invasive alternative for the treatment of early stage rectal cancer, even in those with T2 and T3 lesions after neoadjuvant treatment [10–13].

Neoadjuvant chemoradiotherapy has been shown to reduce LR and to increase tumor downstaging, with 25–40 % complete response rates reported. With the excellent response rates to neoadjuvant therapy, the question is raised about whether surgical decisions made after treatment allowing for maximal tumor downstaging should lead to less invasive surgery with higher sphincter preservation rates. Despite this, APR rates in the literature remain surprisingly high, at 32–67 % [14–16]. At our institution, cancers of the distal 3 cm of the rectum that are not fixed 8–12 weeks after neoadjuvant chemoradiotherapy are managed either by laparoscopic TATA, ultralow anterior resection (LAR), or TEM.

In this retrospective study, we studied 161 patients with rectal cancer of the distal 3 cm who were treated with minimally invasive sphincter-preserving techniques—laparoscopic TATA (n = 106), TEM (n = 49), or LAR (n = 6)—after neoadjuvant high-dose radiotherapy and chemotherapy.

Methods

A prospectively compiled database from a single surgeon of a comprehensive rectal cancer program was retrospectively reviewed for all patients with rectal cancer in the distal 3 cm who had been treated with sphincter preservation by TATA, TEM, or LAR after neoadjuvant chemoradiotherapy during the years 1997–2011. The preoperative external beam radiotherapy was performed with a high energy three- or four-field technique, with an average of 5,400 cGy delivered. Patients who presented with metastatic disease, who did not undergo chemoradiotherapy before surgery, or who had tumors more than 3 cm from the anorectal ring were excluded. We identified 106 patients treated with TATA, 49 patients with TEM, and 6 patients with LAR. Parameters of patient demographics and clinical data, including clinical diagnosis, preoperative evaluation, intraoperative findings and events, postoperative course, and follow-up assessments, were documented on a standardized form and entered into a database prospectively.

Pretreatment evaluation included clinical evaluation, carcinoembryonic antigen levels, serum chemistries, and complete blood count. Synchronous lesions were ruled out by full colonoscopy. Tumors were evaluated for extent of invasion by computed tomography (CT), magnetic resonance imaging, or endorectal ultrasound. Metastatic disease was evaluated with a CT of the abdomen, pelvis, and chest.

Tumors were characterized by clinical assessment for level of the rectum, clinical stage, fixity, ulceration, size, and position. These were reassessed at 3-week intervals during and after neoadjuvant therapy. The tumor was also visualized by flexible endoscopy at presentation and again at 6–12 weeks after neoadjuvant therapy.

LR was defined as the first clinical, radiologic, or histopathologic evidence of tumor in the pelvis. Distant metastasis was defined as clinical, radiological, or pathologic evidence of disease outside the pelvis.

Baseline patient and tumor characteristics

For the 161 patients included in this study, the mean age was 62 years (range, 22–90 years). No patient was excluded because of age. The average distance from the anorectal ring by procedure was as follows: TATA, −1.3 cm (−1.0 to 3.0 cm), TEM −1.5 cm (−0.5 to 3.0 cm), and LAR −2.9 cm (2.5 to 3.0 cm). The distance from the anal verge was not recorded because of the extreme variability of the length of the anal canal. The true rectum ends and the anal canal begin at the levator-ani (anorectal ring) [17].

The clinical response to chemoradiotherapy was based on tumor size and mural involvement. The response was noted as either complete (no residual tumor, surface abnormality, or mural involvement), good (75 % or greater reduction in tumor size, and induration), moderate (25–75 % reduction in tumor size and induration), minimal (<25 % reduction in tumor size or induration), or no change.

It is standard practice at our institution to wait 8–12 weeks after treatment to allow for the maximal downstaging effects of chemoradiotherapy. The decision to perform sphincter preservation is based on many clinical factors, including proximity of the tumor to the sphincter complex, response to neoadjuvant therapy, preoperative sphincter function, and presence of prohibitive comorbidities. APR is recommended for patients presenting with fecal incontinence or for patients with cancers in the distal 3 cm that remain fixed after completion of chemoradiotherapy.

Procedures

All patients were treated by the following algorithm and then underwent a laparoscopic TATA, LAR, or TEM procedure (Fig. 1). TATA was performed in a standard laparoscopic or incisionless fashion. The TATA includes laparoscopic splenic flexure mobilization, colonic mobilization, pelvic dissection, TME, and delivery of the specimen through an abdominal incision with a diverting ostomy. Incisionless TATA differs in the delivery of the specimen being through the anus. The hallmark of the TATA is starting the operation transanally by performing a full-thickness circumferential incision at or just above the dentate line, followed by an intersphincteric dissection. The dissection is carried out in the intersphincteric plane between the puborectalis and the internal sphincter (Fig. 2). The upper portion of the internal sphincter is resected en bloc with the rectal specimen while preserving the external sphincter, puborectalis, and levator ani. The rectum was mobilized transanally to the level of the cervix in female subjects and the seminal vesicles in male subjects. This allows for a known distal margin while sparing the external sphincter and the distal half of the internal sphincter muscles. Laparoscopically, the rectum was dissected using a five-trocar technique (Fig. 3). A hand-sewn coloanal anastomosis with proximal diversion was performed.

Fig. 1
figure 1

Treatment algorithm for rectal cancer

Full size image
Fig. 2
figure 2

A Intersphincteric plane of dissection for TATA. B Developing the plane including the presacral fascia

Full size image
Fig. 3
figure 3

Trocar placement for TATA and LAR

Full size image

Patients were selected for TEM electively if they refused radical surgery or if there was disease regression to within the rectal wall of <3 cm. TEM was performed by first outlining the specimen with electrocautery for known margins (Fig. 4). A package of full mesorectum was resected out to the pelvic sidewall. These methods allow for known margins in an area of the rectum that approaches the anorectal ring. Other patients had a staged management, where there was an excellent response to neoadjuvant therapy, allowing us to perform a TEM excision of the residual lesion. If the pathological specimen was favorable (<ypT2, node negative), intense follow-up was offered. The TEM was performed by outlining the mass by electrocautery, providing for clear margins, and then performing a full-thickness total mesorectal disc excision of the involved rectum with known margins followed by closure of the defect. Patients with unfavorable lesions (i.e., ypT3 or N+) after local excision proceeded to TME. Finally, a third group of patients who were not considered medically fit to tolerate major abdominal surgery underwent TEM as a definitive therapy, regardless of final pathology.

Fig. 4
figure 4

Outlining of lesion for resection by TEM for known margins

Full size image

LAR was performed similarly to TATA in a standard laparoscopic fashion. The procedure started with mobilization of the splenic flexure, followed by placing the patient in a steep Trendelenburg position to remove the small bowel from the pelvis. Next the inferior mesenteric artery and vein were dissected, the left ureter identified, and the mesentery dissected via a medial-to-lateral approach. Next the rectum was dissected performing a TME, with stapled transection using an endoscopic stapler via the suprapubic port.

Postoperative treatment

Postoperatively, intravenous doxycycline was provided while the patient was hospitalized; this was continued orally for 10 days after discharge for all patients. A clear liquid diet was initiated on postoperative day 1 and advanced as tolerated. Patents were assessed at follow-up at 2 weeks after surgery, every 3 months for the first 2 years, every 4 months for the next 2 years, every 6 months for the fifth year, and yearly thereafter. Clinical and digital examinations were performed at each postoperative visit. Flexible sigmoidoscopy was performed at 6-month intervals for the first 2 years. Full colonoscopy was performed at 1 year followed by every 3 years. Carcinoembryonic antigen was measured at each visit, and CT of the abdomen and pelvis was performed at 6 months to 1 year. Positron emission tomography scan was performed when recurrence was suspected.

Results

Overall, the group consisted of 108 men and 53 women. The average (range) age by procedure was TATA 59.2 (22–85) years, TEM 67.8 (29–90) years, and LAR 62.8 (40–83) years. The mean (range) body mass index by procedure was TATA 26.7 (17.9–47.5) kg/m2, TEM 26.5 (17.2–45.1) kg/m2, and LAR 23.7 (20.1–26.5) kg/m2. Twelve patients (7.5 %) had previously received pelvic radiotherapy.

Five patients who had TATA required transfusion (4.7 %), and no patients who had TEM or LAR had a transfusion. Estimated blood loss by procedure and mean, median, and range were as follows: TATA 426, 300, 50–3,600 ml; LAR 333, 325, and 100–600 ml; TEM 117, 50, and 10–500 ml. The American Society of Anesthesiologists class by procedure is listed in Table 1. There was one conversion in the LAR group, for an overall conversion rate for laparoscopic cases (TATA and LAR) of 0.89 %.

Table 1 ASA class by procedure
Full size table

All patients included in this study had tumors located in the distal 3 cm on the rectum. Mean (range) level in the rectum from the anorectal ring by procedure was TATA 1.3 cm (−1.0 to 3.0 cm), TEM 1.5 cm (−0.5 to 3.0 cm), and LAR 2.9 cm (2.5 to 3.0 cm). Distance percentages by procedure are provided in Table 2. During this time period, 44 patients had cancers that remained fixed in the distal 3 cm, requiring either an APR or pelvic exenteration. Sphincter-preservation surgery (SPS) was performed in 78.5 % of patients with rectal cancer in the distal third of the rectum. Of patients treated with sphincter preservation, 96 % did not require a permanent colostomy.

Table 2 Distance from anorectal ring by procedure
Full size table

Neoadjuvant therapy

All patients received high-dose radiotherapy with an average of 5,300 cGy with concomitant 5-FU-based chemotherapy. Radiation-related morbidity was 16 % overall, with skin erythema, rash, and diarrhea being the most common. There were no significant morbidities. Response to preoperative chemoradiotherapy was good to complete in 61 patients (57.6 %) who received a TATA, 34 patients (69.4 %) who received TEM, and 5 patients (83 %) who received LAR.

Surgery

The mean (range) time from completion of treatment to surgery was 10.3 (6.1–27.8) weeks for TATA, 12.0 (7–46.8) weeks for TEM, and 8.9 (5.1–10.7) weeks for LAR. The TATA was performed in 106 patients (65.8 %), TEM in 49 patients (30.4 %), and LAR in 6 patients (3.7 %). For patients who received a TATA (n = 106), 51 were incisionless and 55 were laparoscopic with the specimen delivered through an abdominal incision.

Pathology

Pathologic T stage by procedure is listed in Table 3. Overall, disease stage was T4 (n = 1), T3 (n = 108), T2 (n = 48), T1 (n = 4). The pathologic complete response rate was 22 % overall; by procedure it was as follows: TATA 22.5 %, TEM 23.3 %, and LAR 0 %. Distal margins of the fixed pathologic specimen were less than 1 cm in 23.2 % of patients who had a TATA, 75 % TEM, and 0 % LAR (Table 4). We aimed for an in situ margin of ≥1 cm, if clear, for cancers in the distal 1 cm in order to avoid a permanent colostomy. Overall distal margin positivity was 1 %. The total positive circumferential margin rate was 3.8 % for TATA, 4.0 % for TEM, and 0 % for LAR. All resections were clinically R0.

Table 3 Pathologic T stage
Full size table
Table 4 Distal margin by procedure
Full size table

Lymph nodes were collected in standard fashion using fat-clearing solution. There was an average (range) lymph node collection by procedure of TATA 12 (0–93), TEM 1 (0–14), and LAR 7 (2–10). The numbers of lymph nodes retrieved by the procedure are listed in Table 5. We have shown in a previous publication that after neoadjuvant therapy, lymph node collection is variable [18].

Table 5 Number of nodes retrieved by procedure
Full size table

Complications

There were no perioperative mortalities. The morbidity rate for TATA was 13.2 % overall, with three anastomotic failures (2.8 %). Among the TATA cases, two patients had anastomotic dehiscence, one patient had presacral abscess, and one patient had a partial small bowel obstruction. Minor morbidity rate was observed in 8 patients with urinary retention and stomal hernia. Among patients undergoing TEM, 32 % had morbidity, with minor wound disruption being responsible for 16 % and major wound disruption 10 %.

Recurrence and survival

Overall LR was 3.7 %, with a Kaplan-Meier 5-year actuarial survival of 92 %. The mean (range) follow-up was TATA 38.6 (2–155.6) months, TEM 36.3 (0.4–112.8) months, and LAR 63.1 (7.4–211.7) months. LR was observed in 3 patients with TATA (2.8 %), with recurrence occurring at 10 months for 2 patients and at 12.2 months for 1 patient. Three patients with TEM had an LR (6.1 %) at 15, 27, and 80 months, respectively. There were no LRs in the LAR group. Distant metastasis was observed in 18.8 % of patients receiving TATA, with metastasis to liver in 8 patients, lung in 10 patients, and 1 patient with both liver and lung involvement. The Kaplan–Meier 5-year actuarial survival by procedure was TATA 95 %, TEM 88 %, and LAR 75 %.

This study demonstrates the ability to perform SPS for patients with adenocarcinoma in the distal third of the rectum after neoadjuvant therapy using minimally invasive techniques. We were able to accomplish this with an LR rate of 3.7 % and 5-year survival of 92 %. Using general algorithms for rectal cancer treatment, all of these patients, with cancers in the distal 3 cm of the rectum, would have undergone an APR. During the period of time of this study, we were able to accomplish sphincter preservation in 78.5 % of patients who presented with cancers in the distal third of the rectum. Only those patients who's cancer remained fixed after neoadjuvant therapy underwent a permanent colostomy. This was accomplished using a minimally invasive approach of TEM or laparoscopic TME utilizing the TATA technique or an LAR and stapled coloanal anastomosis. Only 1 patient required conversion. The rate of positive distal margins was <1 %.

Discussion

Although surgery remains the mainstay of the management of rectal cancer, it is well to consider the ideal characteristics of the treatment of rectal cancer. This includes controlling the cancer and avoiding a colostomy—and doing so with minimal morbidity and mortality, as well as minimal trauma to patients. As the location of a cancer progresses down into the distal 3 cm of the rectum, representing the bottom third of the rectum, the challenge of doing this without the need for a permanent colostomy becomes daunting. The key to extending sphincter preservation is a combination of extended high-dose chemoradiotherapy and a prolonged waiting period, as well as basing surgical decisions on postradiotherapy cancer characteristics. By combining these approaches with the TATA procedure, we have demonstrated excellent oncologic and functional results [10, 5]. Full-thickness local excision after neoadjuvant therapy has been described more extensively in the literature and is used in this patient population quite effectively [5, 10, 12].

The challenge for treating cancers in the distal third of the rectum remains how to achieve and handle minimal distal margins. Historically, a 5-cm and then a 2-cm margin was considered acceptable for an adequate oncologic resection for cancers in the nonirradiated rectum [19–21]. Recent studies showing the distal spread of tumor to less than 1 cm after neoadjuvant therapy has led authors to conclude that a 1-cm margin is adequate for an oncologic resection [22, 23]. This has extended sphincter preservation for low cancers. Although we strove for a 1-cm margin overall in vivo, in the TEM group, three quarters of the fixed specimens reported a margin of <1 cm; in addition, 22 % of TATA procedures ended in margins of <1 cm. Not surprisingly, none of the LAR patients had margins <1 cm, as a LAR would be impossible to perform in the distal 1 cm of the rectum. Overall, the positive distal margin rate in this patient population was only 0.6 %. Although we do not advocate accepting minimal margins for cancers higher in the rectum, the LR rate of 3.7 % argues for accepting microscopically clean margins while extending sphincter preservation in the distal third of the rectum. Additionally, the challenge of negative circumferential margins is as critical from an oncologic standpoint as the distal margin. In the distal 3 cm of the rectum, negative circumferential margins can be quite difficult to achieve while performing sphincter preservation. Because of the thinning of the rectum and the surgical challenge of operating in this area, positive circumferential margin rates have been reported between 4.4 and 11 % in the literature [3, 24, 25]. The circumferential positive rate in our experience of 3.8 % in the TATA group, 4 % for TEM, and 0 % for LAR certainly falls within this range of failure. Clearly neoadjuvant therapy allows for a closer circumferential and distal margin by sterilizing the pelvic side wall and lymphatics [26, 14]. The salient question that arises is whether the 4 % overall failures would have had a different outcome had the patients undergone an APR. The corollary question is, does it make sense to commit the other 96 % to a permanent colostomy in an attempt to accomplish this?

Historical data suggest that surgery alone will result in LR rates as high as 15–45 % [27–34]. With incorporation of neoadjuvant therapy, LR rates have improved dramatically, with studies reporting failure rates of 4–17 %. Preoperative chemoradiotherapy also offers added benefit, as previously noted, of tumor downstaging and sphincter preservation extension, with acceptable toxicity [35–40]. Rates of complete pathologic response range from 14 to 20 % [41–44]. Our findings also fit within the reported literature with a complete response rate of 22.3 %. It is our opinion, according to the results of our study, that by combining the effective downstaging of neoadjuvant therapy with decisions that are based on postradiotherapy tumor characteristics, we are able to extend sphincter preservation to 95 % of patients with mobile cancers of the distal 3 cm of the rectum after neoadjuvant therapy. In considering these data, one must recall that all of these patients would have otherwise undergone an APR. Overall, during this time period, 78.5 % of patients with cancers in the distal 3 cm of the rectum were able to receive sphincter preservation. By offering either TEM surgery or TATA, a known margin can be seen and felt before choosing a point of transection. This is thought to be a critical point for carcinomas that reside beneath the first valve of Houston.

Finally, all of these cases were performed in a minimally invasive fashion. To date, less than 10 % of rectal cancer is treated in the United States via minimally invasive techniques. We were able to accomplish this with a 1 % conversion rate and no need for hand-assisted surgery. By using a five-trocar technique, we are able to accomplish a TME with excellent LR and survival rates. Although we await large multi-institutional prospective trials to corroborate the point, this series certainly implies that a minimally invasive approach does not put at risk margins, LR, or failure. Postoperative complications were comparable to those in the literature [3, 25]. We observed no mortalities in this patient population. The major morbidity rate for LAR was 0 % and for TATA, 13.2 %, with the complications as described in the Results. This is well within the range of accepted morbidity for TME. The highest rate of morbidity was that of wound separation after TEM resection. As we have shown previously in a study of wound complications after TEM, the wound separation rate is 25 % in the irradiated rectum sewn to itself [45]. Although this is not insignificant, none of these patients required surgery, only oral antibiotics and pain medicine, and none required fecal diversion. This, in our opinion, represents a reasonable trade-off to avoid the inherent morbidity and mortality associated with a radical pelvic surgery.

There are significant limitations to this study. Foremost among them is that the results are obtained from surgery by a single surgeon. Although these data, as all single-surgeon reports, suffer from issues of reproducibility, it represents a significant experience over a long period of time showing good outcomes of minimally invasive surgery in the distal third of the rectum accomplishing good oncologic outcomes. The most significant limitation of this study is the absence of functional outcome. These data were not prospectively acquired and are currently in the process of being accumulated for future reports. Although this is an important issue in reporting outcomes in distal third of the rectum, this limitation does not negate the importance of the oncologic report, particularly performed in a minimally invasive fashion for cancers in the distal third of the rectum. We have established that sphincter preservation can be performed minimally invasively for distal rectal cancer, avoiding a permanent colostomy. It will take additional studies to see whether this approach can be applied widely.

Conclusions

Rectal cancer management has evolved greatly since the time of Ernest Miles. The paradigm shift incorporating not only oncologic cure but also improved quality of life, which in a patient with rectal cancer involves sphincter preservation, has been dramatically brought to the forefront. In reflecting on an ideal treatment for rectal cancer, the use of minimally invasive surgery in combination with neoadjuvant therapy has allowed sphincter preservation to be applied for cancers in distal third of the rectum without compromising oncologic outcomes. Our results show an excellent LR rate of <4 %, a 5-year survival rate of 92 %, and the ability to avoid permanent colostomy in 95 % of these patients. This offers hope and promise for patients with cancers in the distal 3 cm of the rectum. Current studies ongoing on minimally invasive treatment of rectal cancer will provide more definitive data regarding a minimally invasive approach for TME surgery. Additionally, future multi-institutional studies will be necessary to corroborate and establish these results as reproducible and widely performable. This experience, however, clearly establishes the basis of relying on the postradiotherapy characteristics of rectal cancer, accepting decreased margins in order to obtain sphincter preservation for cancers in the distal 3 cm of the rectum.