Keywords

1 Introduction

Changes in the intracellular calcium concentration are a very critical and universal signalling mechanism used by cells [1, 2]. It is important for a myriad of processes. In the case of humans, life starts with an increase in the calcium concentration upon fertilisation of the egg [3]. Cell death [1, 4] is frequently caused by a prolonged increase in the intracellular calcium concentration. But between fertilization and death, changes in the calcium concentration regulate a plethora of processes in an organism, like memory formation, heartbeat, blood pressure regulation, bones and teeth development, blood clotting, hormone functions, cell division, muscle contractions, and antigen recognition in the immune system [1, 2]. Because a prolonged calcium increase is a death signal, it is very important to keep the intracellular calcium concentration tightly controlled in a narrow range, typically around 100 nM [1].

Further, a dysregulation of the normal calcium signalling processes may lead to several diseases like cardiac arrhythmias [5], neuronal disorders like Alzheimer’s and Parkinson’s diseases [5, 6], or immune dysfunctions [7].

Fluorescence imaging is a powerful and commonly used technique to study calcium signalling in living cells. Cells or tissues are loaded with a calcium sensitive indicator and changes in the intracellular calcium concentration can be recorded in real time by capturing a video of the fluorescence images on a microscope [8]. One downside of the technique is a labour-intensive and often subjective image analysis process. The proposal described in this paper aims to automatically detect and quantify intracellular calcium changes in these videos. In addition to automating the analysis by biologists, our data analysis framework also provides a fast and reliable tool for fluorescent video data analysis in cell phenotyping. The cells being monitored are assumed static. Issues addressed here are processing time, replicability of the data analysis, photo bleaching artefacts, and human-biased variations that are observed during the manual analysis of intracellular calcium imaging experiments.

All in all, this paper presents a fully automatic system to objectively detect active regions of interest(ROIs), which we refer to as hotspots, and to quantify their change in calcium concentration over the time period when they are imaged and saved in a series of fluorescent frames. The automated framework defines as hotspots any areas in cells whose the calcium concentration fluctuates and goes through the four key stages: (i) A pre-processing stage to enhance the contrast and increase the signal to noise ratio of the input video frames using median filtering and adaptive histogram equalization; (ii) A first level segmentation stage to detect candidate hotspots in cells based on changes in the fluorescence intensity using adaptive thresholding and morphological operations; (iii) A second level segmentation stage to fuse and identify the best hotspots from the entire videos by introducing the concept of hit-ratio of calcium fluorescence; (iv) An extraction and quantification stage of the calcium concentration over time for individual hotspots to calculate maximum peak amplitude, area under the curve, peak frequency, and inter-spike interval of calcium changes.

The rest of our paper is organized as follows: background and previous work are presented in Sect. 2. Section 3 gives an overview of the proposed system, focusing on key points. Section 4, presents and discusses the results of our approach. Finally, in Sect. 5 possible future work are highlighted followed by concluding remarks.

2 Background and Relevant ExistingWork

Fluorescent calcium indicators were first developed in 1982 by Roger Tsien and colleagues [9]. Since then, indicators with different characteristicshave been developed: colours, calcium binding affinities, or targeted to different intracellular organelles to name a few [8, 10]. Our framework was developed using calcium imaging data from a cardiomyocyte-like cell type, called pulmonary vein sleeve cells (PVCs), loaded with the calcium indicator Oregon Green BAPTA-AM [11]. Aberrant signalling processes in PVCs are important for the development of atrial fibrillation, the most common cardiac arrhythmia (heart rhythm disorder). In contrast to other cardiomyocytes, PVCs show a high level of localized spontaneous increases in their intracellular calcium concentration. The high level of this spontaneous activity in PVCs is one important factor in the development of atrial fibrillation. A better understanding of the processes underlying the calcium signalling in PVCs might provide new mechanisms to treat or prevent atrial fibrillation [11,12,13,14]. However, the spontaneous activity makes calcium imaging data from these cells very time-consuming to analyse, which prompted the development of the automated image analysis software presented here.

In a typical manual analysis of fluorescent video frames of a calcium imaging experiment, the readout is a file/spreadsheet giving the change in fluorescence for each hotspot over the length of the experiment. In order to compare the results from different experiments, these values are normalized [8]. First, the background fluorescence is subtracted from the initial readout. This is commonly done by setting one small background region in video frames through an interactive toolbox in Image J. Then, the background subtracted data need to be normalized to allow a comparison of the magnitude of the changes between different experiments. For that, the minimum fluorescence (\( Fmin \)) for each hotspot needs to be measured, and the fluorescence at any given time (\( F \)) will be normalized to (\( Fmin) \). The normalized (\( F/Fmin) \) is then used to quantitate parameters like the maximum amplitude of the change in fluorescence, the frequency of (calcium) increases or the area under the curve for the (calcium) transients, depending on which characteristic of the calcium changes is most informative for the experiment. This manual procedure raises concerns about the optimal estimation of the frame background value, because it does not use all of the cell-free area of a frame as a background, which should be the more accurate value. There are also concerns about the minimum fluorescence because in cells with spontaneous activity, it is very hard to estimate the minimum fluorescence by avoiding outliners giving too low an absolute minimum, or photo bleaching artefacts that may suggest fluorescence values lower than they should be. For all those reasons, an automated solution was needed to provide better estimations of background values and minimum fluorescence, and to shorten the analysis time.

Existing work on automatic cellular phenotyping in fluorescence microscopy is rather limited [15]. This is mainly due to the complexity of quantification and characterization of calcium signals through sequences of 2D grayscale images that tend to be of a low quality where cells boundaries are not clearly recognizable, and true signals are highly corrupted by noise. Recent publications investigated various image processing techniques to eliminate the noise effect in fluorescence microscopy images [16,17,18]. Some publications presented different techniques for fluorescence image segmentation [19, 20], while others conducted several cellular parameters analysis in fluorescence microscopy [21, 22].

One of the first automatic method to analyse calcium signalling through fluorescent images was described in 1999 to detect and measure calcium sparks inside skeletal muscle cells and cardiac myocytes through confocal line scan images based on double thresholding [22]. Later on, the limitations of the pioneering method to deal with highly noisy images led to the development of the two-phase greedy pursuit algorithm (TPGP) for the detection of calcium sparks through confocal single images based on wavelet transforms for noise removal and background subtraction [21]. Thus, various versions of ‘home-built’ analysis software of calcium signalling were available, but none come with a comprehensive manual, allowing them to be used by the wider calcium signalling community. Further, knowing that fluorescence microscopic images are generally affected by noise of type Gaussian, Poisson, or a mixture of both [15], state-of-art de-noising techniques were proposed based on two main principles: variance stabilization [16, 23] and patch-based techniques [18, 24]. In the former category, the general approach consisted of converting Poisson noise into Gaussian noise, while in patch-based techniques, researchers relied on the measurement of the similarity between small subdivisions of the images i.e. image patches to detect their random degradations prior to any segmentation procedure.

Alongside those techniques, interesting segmentation methods were also made available for vesicle segmentation in fluorescent stack images [19], tubule boundary segmentation in rat kidneys videos [20], and mid-point based nuclei segmentation in fluorescent images [25]. It is also the case for object-independent segmentation techniques such as Fuzzy set logics [26] and hierarchical merge tree [27], presented as alternative solutions to process fluorescence microscopy images. The Jayaprakas’s ImageJ plugin [28], a less recent semi-automated method, allows both the manual setting of ROIs and the automatic measurement of their changes of fluorescence through an entire image stack. Data are output in graphics or exported into text or csv files for further analysis. But the plugin does not allow automated ROI segmentation.

To sum up, the literature in the area of intracellular calcium analysis based on fluorescence microscopic videos seems to be rich. However, this paper argues that the existing work on fully automated frameworks that take input videos and produce final measurements such as maximum peak amplitude, area under the curve, etc. of calcium changes over time, is very limited, and not available for use by the wider community.

3 The Proposed Solution for Calcium Signalling Analysis

As stated earlier, the proposed framework is based on four main tasks: pre-process individual video slides to increase their quality, identify regions with intracellular calcium changes (hotspots) by first-level segmentation of the input video frames, and detect calcium hotspots or best active ROIs by second-level segmentation. The final task of the proposed routine is the extraction and measurement of calcium signals in the detected hotspots. Figure 1 below highlights those stages.

Fig. 1.
figure 1

Flowchart diagram of the proposed framework

Full size image

3.1 Pre-processing by Median Filtering and Adaptive Histogram Equalization

Fluorescent video frames tend to be low-contrasted and corrupted by Gaussian noise. Therefore, we chose to perform median filtering over individual images to increase their signal to noise ratio. Then, we performed adaptive histogram equalization to enhance their contrast in such a way that changes of intracellular calcium concentration become different from the background pixels. Figures 2a, b, and c below demonstrate the effectiveness of the pre-processing in enhancing the quality of the fluorescent video frames.

Fig. 2.
figure 2

Image quality increased by pre-processing

Full size image

3.2 First Level Segmentation for Candidate Hotspots Detection

At this stage, we first aim to detect any changes in the intracellular calcium concentration expressed in the pre-processed video frames as transiently bright areas due to an increase in the fluorescence of the calcium indicator. By default any calcium change area is hotspot. Inside each frame, hotspots are detected based on a certain threshold pixel value T. To set the best T value, analysis of pixel intensity histograms of input frames suggested us that the fluorescence intensities of the hotspots correspond to the bright pixels that fall away from the mean of the frames. The ideal threshold T turned out to be equalling to \( mean + 2 standard deviation\,(std) \) of the pixel intensities of individual frames. As a result, from a sequence of grayscale images input, the system generates a list of binary frames, where white pixels belong to objects changing their fluorescence, and dark pixels refer to background elements, that can be either inside or outside the cell. After that, morphological filters were applied on the binary output to transform the obtained connected components into closed-boundary objects in the following order: closing, holes filling, erosion, small objects removal, and image dilation with small structuring element of disk-square shapes. Figure 3 below demonstrates the thresholding approach adopted to detect intracellular calcium changes in individual fluorescent images.

Fig. 3.
figure 3

Segmentation of calcium changes from fluorescent video frames

Full size image

The first-level segmentation ends with assigning a set of geometrical properties such as centroid coordinates, area or object size, bonding box parameters, pixel element locations, and default hit-ratio value, etc. to each default hotspots representing intracellular calcium changes. From the average size of all detected objects, the system estimates a small significant interval where to look for potential active regions or hotspots. The best interval of size was found out to be between \( \left[ {\frac{1}{4},\frac{3}{4}} \right] \) of the all objects average size, where to look for candidate calcium hotspots.

3.3 Second Level Segmentation for Frames Fusion Based on Hotspots’ Hit-Ratio

The second level segmentation stage of our proposal for calcium signal analysis aims to identify the intracellular regions that show the most frequent calcium changes. They correspond to the candidate hotspots with highest hit-ratio parameters. The hit-ratio in this context represents the number of times a given cellular region appears or fluoresces in the entire video. For example, a hit-ratio of 50% means that a change in fluorescence appears in the same place in 50% of the frames. To calculate the hit-ratio of a candidate hotspot localised in a frame, the system counts the number of overlaps of this hotspot with others identified in the remaining frames. Overlapping of two objects is first assumed as an overlap of their bounding boxes (rectangle fitting), then confirmed by the comparison of their pixel elements location, elements whose union make the objects.

Once the hit-ratio computation is done, the system merges potential active regions into a single frame in which hotspots do not overlap. The fusion of candidate hotspots from individual frames is achieved based on two thresholds specified by the end user: the minimum hit-ratio, and the minimum distance between hotspots (we chose >15% and 10 pixels, respectively in this paper). At the end of the second level segmentation, the list of all hotspots is finalised and the system is ready to move on to the final stage of the framework which is to extract the measurements representing calcium changes over time from each hotspot. Figure 4 below presents a screenshot of 41 hotspots detected by the system through the fusion of 7352 default hotspots from 300 frames.

Fig. 4.
figure 4

Best active regions labelled with hit-ratio after video fusion

Full size image

For visual purpose, hotspots are coloured according to their hit-ratio, the hottest ones corresponding to the most active regions, like highlighted in the figure.

3.4 Extraction and Measurement of Calcium Signal Inside Hotspots

This final procedure stands for the retrieval of meaningful information from original input video frames. It is the most essential part of the automatic analysis of cellular calcium signalling as it provides biologists with traces illustrating the intracellular calcium changes and numbers to quantitate various parameters to perform statistical analysis for the understanding of the physiological implications of the experiments.

Our proposed software automatically detects active regions inside cells and sets hotspots by direct quantification of the fluorescence within a given area at a given time. These direct fluorescence readouts of a given hotspot correspond to the average intensity of the pixel points whose union makes the hotspot region in every original grayscale frame (\( mi) \). From the initial readout of the fluorescence inside hotspotsanother measurement called the corrected fluorescence (\( F) \) is computed.\( F = mi - bg \), where \( bg \) is the background fluorescence in all areas of the frame that do not contain any cells such that \( bg = image - imageCells \). The background value represents the average value of anything pixel that is neither part of a calcium-change event, nor a cell tissue component. The optimum parameter to represent an image containing all cells was found out to be \( imageCells > mean - std/2 \) of a frame’s pixel intensities. In the same way, the system computes another calcium signal that is called normalized change of fluorescence \( (F/Fmin) \), where \( Fmin \) corresponds to the minimum fluorescence value inside a hotspot after the background subtraction. \( Fmin = imageCells + 1 \). The ratio \( (F/Fmin) \) is the ultimate normalized version of the initial fluorescence readout of hotspots, as it allows effective comparison between different experiments. From the ratio \( (F/Fmin) \) signal, the system retrieves meaningful parameters about the intracellular calcium changes. For the experiments performed, these are the maximum amplitudes of peak in intracellular calcium changes of active regions, the area under the curve, the frequency of calcium transients, the inter-spike interval between two consecutive peaks of calcium concentration change, and the time of slope from peak to a baseline that is customizable, as illustrated in Fig. 5.

Fig. 5.
figure 5

Some parameters extracted from calcium change of fluorescence

Full size image

4 Results and Discussions

Our proposal to automate the analysis of calcium imaging experiments based on fluorescence video imaging enables the automatic segmentation and measurement of intracellular calcium transients over time. Active hotspots can be filtered and monitored from portions or from entire input video files. The processes are fully automatic and rely on the video frames contents rather than users’ biological knowledge or experience.

In details, for example, Table 1 below shows that the overall processing time of our proposal is proportional to the size of the input video file. Big files tend to take more time but compare to the manual procedure, the system processing time is impressive. Indeed, it takes approximately 1 day for biologists to complete the manual analysis of 1000 video frames, whereas our proposed pipeline does the job in less than 5 min.

Table 1. Results from a PC running Matlab 2014 on windows 7 OS with 4 GB of RAM
Full size table

Although, further testing of our proposed system for hotspots detection in fluorescence microscopy videos is still required, we proved that the automatic segmentation of intracellular calcium concentration in individual frames is about 80% accurate.

Here, the percentage of accuracy refers to the ratio between the sum of True Positive and True Negative (TP + TN) pixels over the sum of all Positives and Negatives pixels [29]. Indeed, when using hand-outlined images of intracellular calcium changes as gold standard to isolate calcium changes, like shown in Fig. 6, we were able to compare the automated segmented frames against the visual detection. Table 2 below illustrates the pixel classification we implemented initially over a random sample of 20 frames per fluorescent video files of calcium signalling experiments. Category Positive corresponds to the successful identification of calcium changes by our system, while True illustrates the reference from the ground truth.

Fig. 6.
figure 6

Manual segmentation versus automated

Full size image
Table 2. Automated segmentation performance through pixel classification
Full size table

All in one, the comparison between biologist empirical methodology and our automatic approach establishes that our software can provide reliable results in shorter periods of time. The detection of intracellular calcium through individual frames is fast and accurate. The quantification of calcium concentration is easier and straightforward after ROIS detection.

5 Conclusion and Future Work

In this paper, we describe an automatic solution providing multi-level segmentation for the detection of changes in fluorescence in videos taken when using ion-sensitive fluorescent indicator dyes such as calcium indicators. Our proposal allows a fast and reliable detection of the hotspots to analyse and extract the changes of fluorescence over time. After the extraction of candidate hotspots from individual frames, we propose a new method based on the hit-ratio of fluorescence to identify the best active regions showing changes in the intracellular calcium concentration.

We also propose a normalized quantification of those changes over time for each hotspot. Further, the system enables the extraction of all major parameters researchers are typically interested in when analysing calcium imaging experiments, including some which are very time-consuming to perform manually. We have included the options of manual deleting hotspots which might not fit the criteria that researchers have identified. It is also possible to refine the hotspot detection by setting parameters such as minimum size and hit-ratio of hotspots as well as a minimum distance between them. The paper argues that obtaining the automatic measurements achieves a higher processing time compared with experts’ manual ones. Higher accuracy of our proposed method still needs to be proven by further testing. However the automation of the entire procedure makes data analysis reproducible and easy comparable.

Despite those advantages, we acknowledge that our proposed system has got some limitations. Using videos of tissues, we currently cannot identity individual cells. Further, we currently cannot follow hotspots in case their position shift during experiment course, which might happen. It is the case for cardiomyocytes calcium signalling experiments where high contractions may lead cells to move slightly. In addition to those drawbacks, the system is currently designed to analyse data only from single-wavelength fluorescent indicators. Ideally the system should be extended for the use of multi-wave length fluorescent dyes such as the ratio metric indicators like fura-2. We also think the future system should also enable advanced comparison of cells’ calcium activity by providing more useful parameters output. Finally, the handling of huge amount of data, and noisier raw images will have to be addressed.