Introduction

Artificial intelligence (AI) has been successful in diagnosing Alzheimer’s disease (AD). Lu et al. reported that their deep neural networks could be used to diagnose cases of mild cognitive impairment (MCI) progressing toward AD with an accuracy of 82.5% using 1051 fluorodeoxyglucose (FDG)-positron emission tomography (PET) images in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) [1]. Thus, an AI-based algorithm can also be expected to be used to diagnose dementia; however, thousands of images will be required to train the algorithm [2]. Although we can use a large number of amyloid-negative and positive cases from ADNI, such datasets are not available in a public domain that include other dementia diseases such as those with Lewy bodies (DLB) or frontotemporal lobar degeneration (FTLD). We, therefore, need a new algorithm to synthesize PET images.

Deep convolutional generative adversarial network (DCGAN) is one of the approaches to be considered [3]. Wang succeeded in converting a low dose FDG–PET image to a high dose one [4]. Nevertheless, the algorithm requires exact paired images for low and high doses PET as training data. Although we can theoretically generate a low dose image using list mode data of a high dose PET, it is not feasible to have a pair of amyloid-negative and positive images from the same subject.

Therefore, this study presents the potential of a cycle-consistent generative adversarial network (CycleGAN) [5], an AI-based algorithm, to synthesize sufficient number of training images for multipurpose tasks for diagnosing dementia. We can train CycleGAN using a set of two image domains instead of an actual pair of images. It has been reported that CycleGAN can generate CT images using MRI images [6]. If CycleGAN is also applied to PET images for dementia, we can synthesize the images using a set of amyloid-negative images.

We have two investigative questions. One is whether CycleGAN can generate PET images for dementia from amyloid-negative ones. The other is whether a slice-by-slice operation can maintain the continuity of brain structure in the z-axis to train CycleGAN. A PET image is 3D and can be directly inputted to a CycleGAN; however, this is practically unacceptable. Usually, a PET image contains around 40 slices to cover the brain region. Only if we provide each slice separately to CycleGAN, an appropriate number of training PET images will be assured. Moreover, the 3D operation is burdensome for a computer due to memory and computation requirements, which would result in long computational times to train the CycleGAN. Consequently, we investigate the continuity of the generated PET image for dementia in a slice-by-slice operation.

Method

CycleGAN is an extended version of a GAN. GAN can generate various images artificially such as scenic photographs or paintings. GAN is equipped with an image generator and a discriminating module, which inspects whether a given image is a real image or a generated fake image; they are both trained using adversarial training [7]. CycleGAN consists of two GANs, i.e., two generators and two discriminators. One generator tries to synthesize an amyloid-positive image from a negative image, and another synthesizes an amyloid-negative image using a positive one, simultaneously. The discriminators decide whether a given image is original or synthesized. Once, a real amyloid-negative image is inputted, the CycleGAN converts it to an amyloid-positive image, and then it is converted to an estimated negative image. We train CycleGAN to minimize the difference between the inputted negative image and its estimates; this is called “a cycle-consistent fashion”. After completing the training phase, we can synthesize amyloid-positive images by inputting amyloid-negative images to the CycleGAN.

We inputted 43 amyloid-negative and 45 amyloid-positive standardized uptake value ratio (SUVR) images of 11C-PiB to CycleGAN for training, and another three amyloid-negative images were then applied to the trained CycleGAN to generate amyloid-positive-like images. Note that we applied them slice-by-slice and not as 3D images; therefore, 1699 and 1772 slices of negative and positive cases were used for training, respectively.

We selected images from the dataset acquired from June 2011 to December 2015 for other studies [8, 9]. We applied amyloid PET imaging with 11C-PiB. After 555 ± 185 MBq 11C-PiB injection, the data were acquired at 50–70 min. The images were acquired using ECAT Accel (Siemens AG, Erlangen, Germany), and they were reconstructed with an iterative algorithm (6 subsets, 16 iterations) that provided spatial and axial resolution in the range of 6–8 mm at full width and half maximum. The standard uptake value ratios (SUVR) were calculated relative to the SUV of the cerebellar cortices. The patients ranged from 43 to 89 years old. The study protocol was approved by the institutional ethics committee, and written informed consents were obtained from the subjects or their guardians.

We implemented the algorithm using TensorFlow (ver. 1.10.0) and Python 3.5 on Windows workstation with 3.20 GHz i7‒8700 CPU and GPU of Quadro P4000 with 64 GB memory. We got the code from https://github.com/XHUJOY/CycleGAN-tensorflow. The parameters of our CycleGAN were as below: the epoch was 90, the batch size was 4, λ was 10 that appeared in Eq. 3 of [5], and the filter was 3-by-3 with the stride of 2.

Results

We present the demonstrative synthesized amyloid-positive images in Fig. 1. After training CycleGAN, we inputted two negative images; both outputs indicated typical Aβ accumulation visually. The continuity of slices was preserved in the synthesized amyloid-positive slices as presented in Figs. 2 and 3. We required 10 h to train the CycleGAN.

Fig. 1
figure 1

Demonstrative results of synthesized amyloid-positive SUVR images. The upper row presents original amyloid-negative images, which are an input to CycleGAN; the lower row shows the synthesized positive images. Representative slices are shown. CycleGAN is inputted with two amyloid-negative of (a) and (b) and both synthesized images indicate a typical Aβ accumulation

Full size image
Fig. 2
figure 2

Typical synthesized amyloid-positive image to present the continuity in slice direction. To maintain adequate training images for CycleGAN and a reliable computational environment, amyloid-negative images are inputted slice-by-slice and not as 3D images in the training phase. After training, we input the slices of different amyloid-negative images to synthesize these slices. The slices are acceptable as a positive image visually, and the continuity in slice direction is maintained

Full size image
Fig. 3
figure 3

Demonstrative sagittal and coronal slices of Fig. 2. The original negative images are presented in (a), and the synthesized version with CycleGAN is in (b)

Full size image

Discussion

There is a significant potential in AI to realize a diagnostic algorithm for demential diseases such as AD, DLB, and FTLD. However, it is difficult to obtain adequate number of images that depict dementia; thousands of images are required to train such algorithms. We investigated synthesizing PET images depicting dementia using amyloid-negative images with CycleGAN, which is a recently developed deep-learning approach for image synthesis. As presented in Fig. 1, the algorithm can generate visually acceptable amyloid-positive images. Moreover, as presented in Figs. 2 and 3, the continuity in z-axis is preserved. The voxel size is 2.1 mm and 3.4 mm in a transversal slice and z-axis, respectively, and this difference causes the jaggy observed in Fig. 3 along z-axis.

CycleGAN can build a statistical model between two domains [5, 7]. In this study, CycleGAN tries to learn a difference between negative and positive images. We give CycleGAN the images in slice-by-slice manner, and therefore, CycleGAN can know variations in the difference. After a learning phase, we input negative images according to their slice order to synthesize a positive image volume.

We believe that AI approach is useful for various clinical situation; its usability and regulatory issue has been discussed in detail [10]. In this context, the goal of this study is to establish automated AI-based diagnostic algorithm for PET amyloid imaging. We will realize the algorithm using such amyloid images that are derived from a small number of PET centers where investigative activities are conducted and we can obtain high quality images, and we then apply the algorithm to other PET centers for diagnosis. For this purpose, the dataset augmented by our algorithm is required to train an AI algorithm.

From these results, we can recognize the potential of CycleGAN to realize a sufficient number of training images for AI-based demential diagnostic algorithms. It is our future research to perform a quantitative evaluation of the algorithm.