Real-time Flare Detection in Ground-Based Hα Imaging at Kanzelhöhe Observatory

Abstract

Kanzelhöhe Observatory (KSO) regularly performs high-cadence full-disk imaging of the solar chromosphere in the Hα and Ca ii K spectral lines as well as in the solar photosphere in white light. In the frame of ESA’s (European Space Agency) Space Situational Awareness (SSA) program, a new system for real-time Hα data provision and automatic flare detection was developed at KSO. The data and events detected are published in near real-time at ESA’s SSA Space Weather portal ( http://swe.ssa.esa.int/web/guest/kso-federated ). In this article, we describe the Hα instrument, the image-recognition algorithms we developed, and the implementation into the KSO Hα observing system. We also present the evaluation results of the real-time data provision and flare detection for a period of five months. The Hα data provision worked in 99.96 % of the images, with a mean time lag of four seconds between image recording and online provision. Within the given criteria for the automatic image-recognition system (at least three Hα images are needed for a positive detection), all flares with an area ≥ 50 micro-hemispheres that were located within 60° of the solar center and occurred during the KSO observing times were detected, a number of 87 events in total. The automatically determined flare importance and brightness classes were correct in ∼ 85 %. The mean flare positions in heliographic longitude and latitude were correct to within ∼ 1°. The median of the absolute differences for the flare start and peak times from the automatic detections in comparison with the official NOAA (and KSO) visual flare reports were 3 min (1 min).

1 Introduction

Solar flares are sudden enhancements of radiation in localized regions on the Sun. The radiation enhancements are most prominent at short (EUV, X-rays) and long (radio) wavelengths, with only minor changes in the optical continuum emission. However, flares are well observed in strong absorption lines in the optical part of the spectrum, most prominently in the Hα Balmer line of neutral hydrogen at λ=656.3 nm. Flares typically occur within active regions of complex magnetic configuration (e.g. Sammis, Tang, and Zirin, 2000). They are the result of an impulsive release of magnetic energy previously stored in non-potential coronal magnetic fields injected via flux emergence and surface flows (Wiegelmann, Thalmann, and Solanki, 2014). The released energy is converted into the acceleration of high-energy particles, heating of the solar plasma, and mass motions (e.g., reviews by Priest and Forbes, 2002; Benz, 2008; Fletcher et al., 2011). Flares may or may not occur in association with coronal mass ejections (CMEs). However, the association rate is a strongly increasing function of the flare importance, and in the strongest and most geo-effective events both typically occur together (Yashiro et al., 2006).

CMEs, flares and solar energetic particles (SEPs), which are accelerated either promptly by the flare or by the interplanetary shock driven ahead of fast CMEs, are the main sources of severe space weather disturbances at Earth. CMEs can be observed only in a very limited way from ground because they appear faint, and the stray light in the Earth atmosphere impedes observations as well. They are best tracked in white-light images recorded from coronagraphs onboard space-based observatories. Flares are regularly observed at X-ray and (E)UV wavelengths from satellites, but they are also well observed from ground-based observatories in the Hα spectral line.

In addition to regular visual detection, reports and classifications of solar Hα flares by a network of observing stations that are distributed across the globe, and collection at NOAA’s National Geophysical Data Center (NGDC), there are also recent efforts to develop automatic flare-detection routines. The detection methods range from comparatively simple image-recognition methods that are based on intensity variation derived from running-difference images (Piazzesi et al., 2012), region-growing, and edge-based techniques (Veronig et al., 2000; Caballero and Aranda, 2014) to more complex algorithms using machine learning (Fernandez Borda et al., 2002; Qahwaji, Ahmed, and Colak, 2010; Ahmed et al., 2013) or support vector machine classifiers (Qu et al., 2003). These methods have been applied to space-borne image sequences in the EUV and soft X-ray range (e.g, Qahwaji, Ahmed, and Colak, 2010; Bonte et al., 2013; Caballero and Aranda, 2014) and also to ground-based Hα filtergrams (e.g., Veronig et al., 2000; Henney et al., 2011; Piazzesi et al., 2012; Kirk et al., 2013).

The flare classification system we used here is based on the Hα flare importance classification (Švestka, 1966). Figure 1 shows the relation between the optical Hα flare importance class and the X-ray flare class from the Geostationary Operational Environmental Satellites (GOES). The scatter plot contains all flares observed at KSO during the period January 1975 – October 2014 that were located within 60° from the central meridian. The associated GOES X-ray flares were automatically identified by the following criteria: the soft X-ray and Hα flare peak times occur within 10 min of each other and the heliographic positions are within 10°. Figure 1 reveals a high correlation between the Hα importance class (defined by the chromospheric flare area; cf. Table 2) and the GOES X-ray class (defined by the peak flux in the 1 – 8 Å channel). In total, the set comprises 2832 flares with the following distribution among the classes: the Hα importance is 81.2 % subflares, 15.4 % importance 1, 2.6 % importance 2, and 0.8 % importance 3 and 4; the GOES X-ray class is 86.0 % B and C, 12.5 % M, and 1.5 % X-class flares.

Figure 1

GOES X-ray flares plotted against Hα flares observed at KSO. The dark background represents the density of data points, the stars indicate the mean of the logarithmic X-ray flare class.

Space-based data have the advantage that there are no atmospheric disturbances (seeing, clouds) that degrade the image quality, but there is a delay in the data availability as a result of the data downlink. Ground-based data have the advantage that the data are immediately available for further processing and can thus be efficiently used for the real-time detection and alerting of transient events such as solar flares in the frame of a space weather alerting system – but with the drawback that the image sequences may suffer from data gaps and poor seeing conditions that cause a varying image quality. These circumstances have to be accounted for by the image-recognition algorithms applied.

In this article, we present an automatic image-recognition method that was developed for real-time detection and classification of solar flares and filament eruptions in ground-based Hα imagery. The algorithms have been implemented into the Hα observing system at Kanzelhöhe Observatory (KSO) to immediately process the recorded images and provide the outcome in almost real-time. This activity was performed in the frame of the space weather segment of ESA’s Space Situational Awareness (SSA) program, and the real-time Hα data and detection results are provided online at http://swe.ssa.esa.int/web/guest/kso-federated . We concentrate here on the automatic flare detection and classification system, which was implemented in the KSO observing system in June 2013, and present the evaluation of the system for a five-month period. The automatic detection of filaments and filament eruptions will be presented in a subsequent study, as the method is still being improved (first results are shown in Pötzi et al., 2014).

The article is structured as follows: In Section 2, we describe the KSO solar instruments and observations. In Section 3, we outline the image-recognition algorithms developed to automatically identify solar flares in Hα images and to follow their evolution (in terms of location, size, intensity enhancement, and classification). Section 4 outlines how the real-time detection and alert system was implemented in the KSO observing system. In Section 5, the outcome of the real-time flare detection system is evaluated for a test period of five months from the end of June to November 2013. In Section 6, we discuss the performance of the system.

2 KSO Instrumentation and Observations

Kanzelhöhe Observatory for Solar and Environmental Research (KSO; http://kso.ac.at ) is operated throughout the year at a mountain ridge in southern Austria near Villach (N 46°40.7′, 13°54.1′, altitude 1526 m). The site allows solar observations for about 300 days a year, typically 1400 hours of patrol time. KSO regularly performs high-cadence full-disk observations of the Sun in the Hα spectral line (Otruba and Pötzi, 2003), the Ca ii K spectral line (Hirtenfellner-Polanec et al., 2011), and in white-light (Otruba, Freislich, and Hanslmeier, 2008). Figure 2 shows an exemplary set of simultaneous KSO imagery in Hα, Ca ii K and white light recorded on January 6, 2014. All data are publicly available via the online KSO data archive at http://kanzelhohe.uni-graz.at/ (Pötzi, Hirtenfellner-Polanec, and Temmer, 2013).

Figure 2

Solar full-disk observations in white light (left), Ca ii K (middle), and Hα (right) recorded at Kanzelhöhe Observatory on January 6, 2014.

The observations are carried out by the KSO observing team during seven days a week, basically from sunrise to sunset, local weather conditions permitting. All instruments for solar observations are mounted on the KSO surveillance telescope, which comprises four refractors on a common parallactic mounting (Figure 3). The KSO Hα telescope is a refractor with an aperture ratio number of d/f=100/2000 and a Lyot band-pass filter centered on the Hα spectral line (λ=656.3 nm) with a full-width at half-maximum (FWHM) of 0.07 nm. For thermal protection an interference filter with an FWHM of 10 nm is placed in the light path. The Lyot filter can be tuned by turning the polarizers in narrow boundaries with little degradation of the filter characteristics. A beam splitter allows applying two detectors at the same time. Currently, the observations are solely carried out in the center of the Hα line.

Figure 3

Kanzelhöhe Observatory (left) and its solar patrol telescope consisting of four refractors observing the Sun in Hα, Ca ii K, and white light (right).

The CCD camera of the Hα image acquisition system is a Pulnix TM-4200GE with 2048×2048 pixels and a Gigabit Ethernet interface. A frame rate of seven images per second permits applying frame selection (Roggemann et al., 1994; Shine et al., 1995) to benefit from moments of good seeing. The image depth of the CCD camera is 12 bit, which allows simultaneously observing the quiet Sun and flares without overexposing the flare regions. To have good counts statistics under varying seeing conditions and to avoid saturation effects in strong flares, an automatic exposure control system is in place. The automatically controlled exposure time lies in the range 2.5 to 25 milliseconds. In the standard observing mode, the observing cadence of the Hα telescope system is six seconds. The plate scale of the full-disk observations is ∼ 1 arcsec, corresponding to about 720 km on the Sun. The telescope is guided by a microprocessor system, with (minor) corrections applied by automatically determining the solar disk center from the real-time Hα images.

3 Image Recognition Algorithm

The developed image-recognition algorithms make use of the main characteristics of the features in single Hα images as well as in image sequences. Solar flares are characterized by a distinct brightness increase of localized areas on the Sun. They typically reach their largest extent and highest intensity within some minutes up to some tens of minutes, followed by a gradual decay of the intensity due to the subsequent cooling of the solar plasma. Flares are categorized in importance classes based on their total area and their brightness enhancement with regard to the quiet-Sun level.

The image-recognition algorithm consists of four main building blocks. The preprocessing handles the different intensity distributions, large-scale inhomogeneities, and noise. The feature extraction step defines the characteristic attributes of the features to be detected and how to model them. In the multilabel segmentation step, the model is applied to “new”, i.e. previously unseen, images in almost real-time. In the postprocessing, every identified object is assigned and tracked via a unique ID, and the characteristic flare parameters are derived (location, area, start/peak time, etc.). In the following we give a basic description of these methods; more details can be found in Riegler et al. (2013) and Riegler (2013).

3.1 Preprocessing

The preprocessing has two goals: image normalization and feature enhancement. Across different Hα image sequences, the intensity distributions of the images are shifted and dilated. These differences in the distributions arise from different solar activity levels (e.g. many/few sunspots), seeing conditions, exposure time, etc. As the feature extraction strongly relies on the value of the image intensities, we normalize the image intensities by a zero-mean and whitening transformation:

$$\begin{aligned} \mu&= \frac{1}{|\varOmega|} \sum_{x \in\varOmega} f(x), \end{aligned}$$

(1)

$$\begin{aligned} \sigma&= \sqrt{ \frac{1}{|\varOmega|- 1} \sum_{x \in\varOmega} \bigl( f(x)- \mu \bigr)^2 }, \end{aligned}$$

(2)

$$\begin{aligned} f_n(x) &= \frac{f(x)- \mu}{\sigma}, \end{aligned}$$

(3)

where $\varOmega\subset\mathbb{R}^{2}$ is the image domain, μ the sample mean, and σ the standard deviation of the input image f. The normalized image f _n is given by a point-wise subtraction and division by the mean and standard deviation.

As a second step in the preprocessing, additive noise and large-scale intensity variations, caused by the center-to-limb variation and clouds, are removed by applying a structural bandpass filter. At the core of this particular filtering method is the total variation with the ℓ ₁ fitting term (TV-ℓ ₁) model (Chan and Esedoglu, 2005; Aujol et al., 2006), which is a signal- and image-denoising method based on minimizing a convex optimization problem given by

$$\begin{aligned} \min_u \| \nabla u \|_{2,1} + \lambda\| f- u \|_1, \end{aligned}$$

(4)

where f is the noisy observation of the image and u is the minimizer of the optimization problem. The first term, the total variation norm, regularizes the geometry of the solution, while the second term, the ℓ ₁ norm, ensures that the solution is close to the original image f. Finally, λ is a free parameter that can be used to control the amount of regularization. The main property of the TV-ℓ ₁ model is that it is contrast invariant. In other words, structures from the image are removed only in terms of their spatial extent and not in terms of their contrast to the background. To efficiently solve the optimization scheme we use the generic primal dual algorithm proposed in Chambolle and Pock (2011).

Chan and Esedoglu (2005) showed that by solving the TV-ℓ ₁ for a certain parameter λ, all structures having a minimal width of λ ⁻¹ are removed in the regularized image u. We use this fact in our structural bandpass filter by first removing small-scale noise from the image using a larger λ ₁, which results in image v ₁. In the next step, we remove larger structures by again regularizing the image v ₁ using a smaller λ ₂<λ ₁, such that the resulting image v _s contains only undesired large-scale structures such as brightness variations and clouds. The final result of the structural bandpass filter is then given by subtracting image v ₂ from image v ₁. Figure 4 illustrates the different steps of the structural bandpass applied to a sample KSO Hα filtergram with clouds.

Figure 4

Structural bandpass filter applied to an Hα image with clouds. a) Original image, f, b) denoised image, v ₁, c) large-scale variations, v ₂, d) resulting image, u, of the structural bandpass filter.

3.2 Feature Extraction

In the feature extraction step, two main problems have to be addressed. a) What are the characteristic attributes of flares and filaments, i.e. what distinguishes them from other solar regions? b) How can we efficiently model these attributes?

To solve the first problem, we assign a feature vector to each pixel. The most intuitive feature choice is the pixel intensity of the preprocessed images. We use the fact that filaments are darker than the background of the Hα images and that sunspots are even darker than filaments and that their typical geometries are different from filaments (round versus elongated objects). Flares are defined as objects with distinctly higher intensities than the background. It may also be advantageous to use the intensities of the pixels within a small local neighborhood. Furthermore, the contrast decreases from the center toward the limb. To incorporate this effect, the distance from the solar disk center to the pixel location has proven to be useful.

Based on the extracted feature vectors, we use a Gaussian mixture model to assign a class probability to each pixel of an Hα image. For the classes we use the features “flare”, “filament” and “sunspot”. The remaining part of the image is summarized in the class “background”.

Figure 5 illustrates the class probabilities in a histogram. The data we use for feature extraction and learning of the model are derived from labeled Hα images, where an expert annotated a set of KSO Hα images by assigning the pixels to the different classes. The figure shows that the probability distributions of the four classes are not distinctly separated. The overlaps between the classes sunspot–filament and background–flare are not a severe problem because most of the probabilities are well separable. In contrast, the probability distribution overlap between the classes filament–background does cause segmentation problems in the application. Additional methods that can be used to acquire a better distinction of filaments against the background are described in Riegler (2013).

Figure 5

Intensity distribution of the classes sunspot (red), filament (green), background (black), and flare (blue). The training examples are derived from preprocessed Hα images that were annotated by an expert.

3.3 Multilabel Segmentation

In principle, each pixel could be assigned to the class with the highest probability, but this would lead to a very noisy segmentation. To regularize the final segmentation, we adopt a total variation-based multilabel image segmentation model (Chambolle and Pock, 2011),

$$\begin{aligned} &\min_{ \{u_l\}_{l=1}^N } \sum_{l = 1}^N \int_\varOmega\mathrm {d}|\nabla u_l| + \sum _{l = 1}^N \int_\varOmega u_l q_l \,\mathrm{d}x \end{aligned}$$

(5)

$$\begin{aligned} &\mbox{such that} \quad u_l(x) \ge0, \quad\sum _{l=1}^N u_l(x) = 1, \end{aligned}$$

(6)

where the functions u _l and q _l , l=1,…,N are the binary class assignment functions and class-dependent weighting function, respectively. In the simplest case, the negative logarithm of the class probabilities can be used, but we apply an additional temporal smoothing by computing an exponential weighted moving average over the probabilities.

3.4 Postprocessing

The final step of the method is the postprocessing, which has two main goals. The first one is identifying each detected flare (and filament) with a unique ID, which should remain the same over the image sequence for the very same object. The second goal is deriving characteristic properties from the identified objects to categorize them.

3.4.1 Identification and Tracking

The identification task is solved by means of a connected-component labeling problem (Rosenfeld and Pfaltz, 1966). To track the objects in the Hα image sequence, we apply a simple propagation technique. From the segmentation we obtain the four binary images u _l for the four classes. The next step is to identify eight connected pixels that form a group and are separated by zeros from other groups, and to assign them an ID. The problem can be efficiently solved with a two-pass algorithm, as presented for example in Haralock and Shapiro (1991). In a first pass, temporary labels are assigned and the label equivalences are stored in a union-find data structure. Then a label equivalence is detected whenever two temporary labels are neighbors. In the second pass, the temporary labels are replaced by the actual labels that are given by the root of the equivalence class. The union-find data structure is a collection of disjoint sets and has two important functions. The union function combines two sets, while the find function returns the set that contains a given number. The data structure can be efficiently implemented with trees.

The connected component labeling ensures that every flare (and filament) has a unique ID per image. To guarantee that the ID remains the same throughout the image sequence, we propagate the ID of previous images. We assume that I _t (x) is the current component labeled segmentation and $\{I_{t-k}(x)\} _{k=1}^{n}$ the set of n previous component labeled segmentation results. Then we change the ID of a current component I _t to the ID j, where j is given by the components of the previous images that have the largest overlap with component I _t . This can be implemented in a pixel-wise fashion and a simple map data structure. For a given component of the current image and ID I _t , we iterate all overlapping pixels x of the set $\{I_{t-k}(x)\}_{k=1}^{n}$. If I _t−k (x)≠0, we increment the counter for the ID I _t−k (x) in the map. Finally, we assign the ID with the highest counter. Since flares often consist of two or more ribbons, flare detections that are located within a certain distance (set to 150 arcsec) are grouped into one ID. Figure 6 shows a sample Hα image with the flare detections and the assigned flare IDs. A sample Hα image with the filament detections is shown in Figure 7.

Figure 6

Sample KSO Hα image from May 10, 2014, in which three different flares are simultaneously present on the solar disk. The detected flare areas are indicated in different colors. Each flare is registered and tracked in time, i.e. from image to image, by a unique ID.

Figure 7

Sample KSO Hα image from March 31, 2014 together with the filament detections. Each filament is assigned and tracked by its unique ID (annotated close to each filament).

4 Implementation of the KSO Hα Observing System

To optimize the process and speed of the real-time data provision and flare detection, different computers are involved that run in parallel, each one performing a specific set of tasks:

• camera computer: image acquisition;

• workstation 1: quality check, data processing, and online data provision;

• workstation 2: image recognition, flare detection, and alerting.

Figure 8 shows a flow diagram of the tasks that are performed on each incoming Hα image, where the columns refer to different machines responsible for certain tasks. Each Hα image is grabbed by the camera computer and sent to workstation 1, where the image is checked for quality. If the image passes the quality criterion, it is processed and published on the web server. In parallel, the processed image is also transferred to workstation 2, which performs the image-recognition algorithm. If an event is detected, its characteristic parameters are calculated. If the event exceeds a certain threshold (i.e. flare area or importance class), a flare alert is published online at ESA’s SSA SWE portal and an alert e-mail is sent out. In the following we describe the different analysis steps in detail.

Figure 8

Flow diagram showing the main steps of the data-processing pipeline at the KSO Hα observing system. The overall process of data acquisition, quality check, processing, image recognition, event detection, and alerting involves four different machines (camera PC, raid system, workstations 1 and 2) and a web server where the results are published.

4.1 Image Acquisition, Processing, and Online Provision

Image Grabbing

The image acquisition is made in a fully automated mode, which includes automatic exposure control and the use of the frame selection technique (Shine et al., 1995). The CCD camera is controlled via a simple user interface; in standard patrol mode no user interaction is needed throughout the observation day.

Quality Check

All grabbed Hα images are checked for their quality. Clouds and poor seeing conditions result in low contrast and fuzzy images, which may cause difficulties for the image recognition. Since the quality test has to be performed for each image, a simple algorithm was implemented. The image quality is measured by three conditions that have to be fulfilled:

• The solar disk appears as a sphere with high accurateness: points on the solar limb are detected by a Sobel edge-enhancement filter. A circle is fitted through the detected limb points, and the relative error of the radius is computed.

• The large-scale intensity distribution is uniform: the solar image is rebinned to a 2×2 pixel image. The relative brightness differences of these four pixels define a measure of the intensity distribution.

• The image is sharp: the correlation between the raw image and a smoothed version of the image is computed. If the raw image is already fuzzy, it shows a high correlation with the smoothed image.

Based on these criteria, the images are classified into three quality groups: good, fair, and poor. Only images of “good” quality are sent to the image-recognition pipeline and the online data provision. Images classified “fair” or “poor” are moved to a temporary archive and are not considered in the subsequent analysis. However, we note that images of “fair” quality may still be acceptable and useful for visual inspection (e.g. for visual flare detection).

Image Processing

For all images that remain in the pipeline, decisive parameters such as the disk center, the solar radius, and maximum and mean brightness are derived. Together with additional information such as the acquisition time, instrument details, solar ephemeris for the recording time, the images are stored as FITS file (Pence et al., 2010). These images are on the one hand stored in the KSO data archive, on the other hand, they are fed into the subsequent pipeline of real-time data provision and image recognition.

Provision of Real-Time Images and Movies on the SSA SWE Portal

Each minute an image is selected for the real-time Hα display at ESA’s SSA SWE portal ( http://swe.ssa.esa.int/web/guest/kso-federated ; a snapshot is shown in Figure 9). The size of the image is reduced to 1024×1024 pixels and stored in jpeg format for fast and easy display. The image is overlaid with a solar coordinate grid and annotated with a header containing time information. For later validation, a log file that keeps track of the image acquisition time and the time when the image was provided online is updated for each image. Every five minutes an html-movie script that animates the latest hour of Hα images is generated and displayed at the SWE portal.

Figure 9

Screenshot of the ESA SSA SWE Hα subportal ( http://swe.esa.int/web/guest/kso-federated ). The subportal shows the real-time Hα image (middle), a list of the detected events of that day (right), and a 360° view around the observatory (bottom) to check the observing conditions, which is updated every minute.

4.2 Image Recognition, Flare Characterization, and Alerting

Image Recognition

Most of the iterative algorithms of the image recognition (cf. Section 3) are computationally intensive. However, they can be easily parallelized. Thus, it is possible to use the computational power of modern graphic processing units (GPU). The image-recognition algorithm has been implemented in the programming language C++ and installed on a dedicated machine with a high-performance GPU. The system benefits from the large number of processing units that are used for the highly parallelized computations. In its present form, the algorithm needs about 10 s to process the flare and filament recognition on one 2048×2048 pixel image, which allows event detection almost in real time. The results of the image-recognition algorithm are stored in feature log files containing tables of flares and filaments that have been detected. These feature log files are updated at each time step, i.e. with each new image that enters the pipeline, so that the evolution of the detected features can be computed.

Event Detection and Parameter Calculation

After the detection of a flare by the image recognition system, its characteristic properties and parameters are derived. For flares, these include the heliographic position, the flare area (which defines the importance class), the brightness class, and the flare start and peak times. These quantities need not only the information of a single Hα image, but also the information stored in the image recognition log files for the previous time steps. Handling of simultaneous flares is easily possible as each flare is identified via a unique ID that is propagated from image to image. In Figure 10 we show a sequence of Hα images that were recorded during a 2B class flare that occurred on May 10, 2014 (top panels) together with the segmented flare regions (bottom panels).

Figure 10

Top: Sequence of Hα images showing the evolution of a 2B flare that occurred on May 10, 2014. Bottom: segmented flare areas.

The flare area is calculated by the number of segmented pixels with the same ID. These are subsequently converted by the pixel-to-arcsec scale of that day to derive the area in millionths of the solar hemisphere, the so-called “micro-hemisphere”. The conversion procedure includes the information of the flare position to correct for the foreshortening effect toward the solar limb. The determined area is then directly converted into the flare importance class (subflares, 1, 2, 3, and 4) according to the official flare importance definitions (cf. Table 1). For the categorization into the flare brightness classes (Faint–Normal–Brilliant: F–N–B), the intensity values relative to the background are used. To this aim, we compute the mean, standard deviation, and the maximum and minimum of the pixel intensities within the segmented regions. For each detected feature, we apply a normalization by the difference between the maximum and mean brightness of the feature.

Table 1 Hα flare importance classes.

To characterize a flare, the evolution of the brightness and the area in each Hα image of the sequence has to be analyzed. For illustration, we show in Figure 11 the evolution of the area and brightness of a sample 1N flare that occurred on October 16, 2013. The flare classification is based on the following definitions:

• i) The flare start is defined as the time when the brightness enhancement is above the faint flare level for three consecutive images.

• ii) The peak time of the flare is defined as the time where the highest flare brightness is reached.

• iii) The flare position is defined by the location of the brightest flare pixel at the time of the flare peak.

• iv) The importance class of the flare is defined via the largest area of the flare and is updated when the area exceeds the level of a higher importance class.

• v) The flare end is defined as the time when the brightness has decreased below the faint level for ten consecutive images or when there is a data gap of more than 20 min.

• vi) Handling of data gaps: In case of data gaps < 20 min, the flare is considered to be in an evolving state if the flare brightness after the data gap is higher than before the gap. Data gaps of > 20 min define the end of the flare.

Figure 11

Illustration of the flare parameter calculation for the 1N flare of October 16, 2013. Top: Evolution of the flare area. Middle: Evolution of the flare brightness. The crosses show the data points, the black solid lines show the highest values for each minute. The thick solid line represents times where the intensity is above the faint flare level. The determined start (14:27 UT) and peak (14:31 UT) times are indicated by the green and blue bar. Note that the maximum brightness and the maximum area do not necessarily occur at the same time. Bottom: Snapshots of the flare at three different times (indicated by orange vertical lines on the top panels) together with the flare detections (orange areas).

Flare Alerts

If a flare is detected that exceeds a certain threshold, i.e. importance class, then a flare alert is published on the ESA SSA SWE portal and an alert email is sent out to registered users. Originally, it was intended to restrict the flare alert to events of Hα importance class 1 and higher. However, as a result of the weak activity during Cycle 24, we lowered the threshold to subflares exceeding a size of 50 micro-hemispheres to obtain sufficient statistics for the evaluation.¹

The event list on the ESA SSA SWE portal is updated every time a flare is detected, when more information on a flare becomes available during its evolution (e.g., the peak time), or when a flare that is already listed increases in its importance class. The flares are sorted according to decreasing start time, so that the most recent event appears in the first line of the table. As long as the flare brightness increases, no peak time is listed, but the event is annotated to be “ongoing” and marked in red in the event list in the Hα subportal (cf. Figure 9). When the flare brightness has decreased for a period of time > 2 min, the peak time is derived and provided in the event table. A list of all detected events is stored in the local raid system for later evaluation.

In addition, flare-alert e-mails are sent to a predefined list of users. They are issued when one of the following criteria is fulfilled: i) the flare detected is of importance class 1 or higher or a subflare with an area exceeding 50 micro-hemispheres, or ii) an ongoing flare reaches a higher importance class (e.g., a flare of importance 1 evolves further to a flare of importance 2).

5 Results

The system of near real-time Hα data provision, automatic flare detection, and alerting went online on June 26, 2013. In the following we present the results from the evaluation of the system for a period of five months from June 26 to November 30, 2013, in which it was run with the same set of parameters and definitions.

5.1 Real-Time Data Provision

To validate the online data provision, we evaluated the number of Hα images that were recorded by the KSO observing program and the number of Hα images that were provided online in almost real-time at the ESA SWE service portal. For this purpose, log files recorded for each image both the observation time and the time when the image was placed online to the SWE service portal.

During the evaluation period, we had a total of 563 hours of solar observations at KSO. In total, 395 129 Hα images were recorded. 281 806 images (71.3 %) were rated as “good”, 20 922 (5.3 %) as “fair”, and 92 401 (23.4 %) as “poor”. 33 765 Hα images (one per minute) out of the “good” sample were provided online at the SWE service portal, whereas 14 were erroneously skipped because of internal data stream errors. This means that in 99.96 % of the observation time, one image per minute was provided online at the SWE portal. The mean time lag between the recording of an image and its online provision was 3.6±0.9 seconds.

5.2 Real-Time Flare Detection and Classification

For the evaluation of the automated detection and alerting of Hα flares, we considered all flares that exceeded an area of 50 micro-hemispheres and that occurred within 60° from solar disk center during the KSO observing times. For flares closer to the solar limb than 60° from the center of the disk, projection effects become significant in determining the flare area. In addition, these flares are most likely not relevant for space weather disturbances at Earth. As discussed in Section 4.1, only images of “good” quality are fed to the automatic flare detection pipeline. For the automatic detection of a flare, we require that it is observed in at least three Hα images. Periods of > 20 min containing no images of “good” quality are defined as data gaps.

The data that are needed for the evaluation of the flare detection, classification, and alerting are derived from the log files that are created and updated during the observations (cf. Figure 8). The relevant parameters that are derived and evaluated are the following:

• Flares: heliographic position, start time, peak time, area, importance class, and brightness class;

• Alerts: time of issue.

For the evaluation, we compare the results obtained by the automated image recognition system developed, called Surya², against the official flare reports provided by the National Geophysical Data Center (NGDC) of the National Oceanic and Atmospheric Administration (NOAA) and by Kanzelhöhe Observatory. Both are obtained by visual inspection of the data by experienced observers.

The Space Weather Prediction Center (SWPC) of the US Department of Commerce, NOAA, is one of the national centers for environmental protection and provides official lists of solar events, available online at http://www.swpc.noaa.gov/ftpmenu/indices/events.html . The information on the flare events is collected from different observing stations from all over the world. Kanzelhöhe Observatory sends monthly flare reports to different institutions, including NGDC/NOAA and the World Data Center (WDC) for Solar Activity (Observatoire de Meudon). The visual KSO flare reports (KSOv) are online available at http://cesar.kso.ac.at/flare_data/kh_flares_query.php . We actually expect that the results of the automatic detections are on average closer to the visual KSO flare reports than the NOAA reports, as they are based on the data from the same observatory. However, it is also important to compare the outcome against the NOAA reports, as they provide an independent set of flare reports.

Table 2 in the Appendix lists all flare events (area ≥ 50 micro-hemispheres; located within 60° from disk center) that were detected in near real-time by the automated algorithm during the evaluation period. In total, 87 flares were detected by Surya; 69 were classified as subflares and 18 as flares of importance 1. This list includes three false detections (marked in boldface in the table), i.e. flares that were detected by Surya, but have no corresponding event reported by NOAA or KSOv. In addition, the list in Table 2 includes seven flares where Surya reported one flare, but NOAA and KSOv reported two separate events, as well as two NOAA (four KSOv) flares, where Surya split one flare into two or more events.

To evaluate the detection ability of Surya, we also checked all NOAA and KSOv flares reported during the KSO observation times (located within 60° from disk center) that were not detected by Surya. These are in total 60 flares (57 SF, 2 SN, and one 1N flare). There are basically two different reasons why these events were not detected by Surya:

• i) Data gaps: fewer than three images of “good” quality were available during the event, and thus the automatic detection algorithm was not run. 47 flares fall into this category. We note that single images as well as images of lower quality may still be sufficient to identify a flare by visual inspection (although with large uncertainties in the derived flare parameters). Indeed, for 19 of these events, visual flare reports from KSOv are available (18 subflares and one flare of importance 1).

• ii) NOAA reports a flare that is not listed by KSOv and for which – even after careful visual re-inspection of the KSO Hα image sequences – we cannot confirm the appearance of a flare. This applies to 13 events, all of them subflares.

For these reasons, these events are not expected to be detected by our automatic image-recognition system. This means that Surya basically detected all flares listed by the NOAA and KSOv flare reports when there were sufficient data available (i.e. at least three images during an event).

The next question we have to address is the accuracy of the flare parameters calculated by the automatic system. The accuracy of the peak flare area determined by Surya cannot be evaluated because the flare reports do not provide areas. However, the flare area is an intrinsic property that determines the importance classification of a flare (cf. Table 1). For the importance classifications we find that the importance classes reported by NOAA and KSOv do not coincide in seven cases. Thus we excluded those events from the evaluation of the importance classes because there is no unique reference value available. For the remaining set of flares, we find that in 86 % the automatically determined flare importance class coincides with the class given by NOAA and KSOv. The incorrectly classified events include five flares where Surya obtained a different importance class than reported by NOAA and KSOv and six cases where Surya split flares into two or more events. For the brightness classification, the NOAA and KSOv reports do not coincide for 15 events. For the remaining set, we find that in 85 % Surya determined the correct brightness class (cf. Table 2).

Figure 12 shows the absolute differences of the heliographic latitude and longitude of the flare center as obtained by the automatic algorithm against the values reported by NOAA and KSOv. The mean of the absolute difference for the latitude is 1.21° (0.25°) with respect to the NOAA (KSOv) flare reports and 1.36° (0.60°) for the longitude.

Figure 12

Distribution of the absolute differences of the flare heliographic latitude (left) and longitude (right) between the automatic detection by Surya and the NOAA (blue) and KSOv (red) reports.

Figures 13 and 14 show the distributions of the absolute differences of the flare start and peak times, respectively, derived by Surya in comparison with NOAA and KSOv. For the start and peak times, the median time difference is 3 min (1 min) with respect to NOAA (KSO). For 62 % (78 %) of the flares detected, the derived flare start times lie within ± 5 min with respect to the NOAA (KSOv) reports, and for 70 % (83 %) the flare peak times lie within ± 5 min with respect to NOAA (KSOv).

Figure 13

Distribution of the absolute differences between start times of the flares detected by Surya and reported by NOAA (blue) and KSOv (red).

Figure 14

Distribution of the absolute differences between peak times of the flares detected by Surya and reported by NOAA (blue) and KSOv (red).

5.3 Real-Time Flare Alerts

When a flare reaches a certain threshold, an alert e-mail is automatically generated and sent to a predefined list of users. The expected number of alerts is actually higher than the number of detected Surya flares listed in Table 2 because alerts are not only sent for the detection of an event, but also when the flare evolves to a higher importance class. This means that for a flare of importance 1, we have two alerts when the full flare evolution was covered: one when it reached the level of a subflare of area ≥ 50 micro-hemisphere, and another one when it reached an area of 100 micro-hemispheres, i.e. the threshold for an importance 1 flare. In total, we had 14 cases (14 %) where erroneously no flare-alert e-mails were issued. These include the four flares on July 21, 2013, the five flares on Oct 11, 2013 and the first two flares on October 15, 2013, where we had an error in the automatic e-mail script. In addition, no alert was sent for three flares that reached exactly the threshold area of 50 micro-hemispheres (August 15, 2013, 12:03 UT and 12:49; August 30, 2013, 06:14 UT). There were six false alerts in total. Three of the false alerts are related to the false flare detections (indicated in boldface in Table 2). The other three false alerts were double alerts, i.e. two identical e-mails had been sent for one flare.

We also evaluated the time between the occurrence of the flare and the issue of the alert e-mail. In this respect, occurrence means the time when the flare reaches the threshold area, i.e. an area of 50 micro-hemispheres for a subflare alert, or an area of 100 micro-hemispheres for an importance 1 alert, etc. Figure 15 shows the distribution of the time difference between the occurrence of the flare and the issue of the alert e-mail, giving a median of 1.5 min. In total, 89 % of the (true) alerts were issued within five minutes. However, we note that there are five cases where the delay is longer than ten minutes. These are mostly related to data gaps in the Hα image sequences. The detection of a flare demands that a flare is detected in at least three observations. However, if there is a longer data gap, for instance, between the second and third image of a sequence, then the alert (which is issued after the flare detection in the third image) may be substantially delayed with respect to the start of the flare (defined by the time of the first image in this sequence).

Figure 15

Distribution of the time differences between the start of the flare and the issue of the alert.

6 Discussion and Conclusions

The real-time Hα data provision worked perfectly fine, with a percentage of 99.96 % provisions online at the ESA SWE portal within less than four seconds of the observations. The automatic flare detections basically worked in all cases within the given criteria, i.e. within the demand that for a positive detection we need at least three Hα images of “good” quality during the event. In our five-month evaluation period, about 70 % of all the Hα images observed at KSO (with a regular cadence of 6 s) were classified as “good”. We note that for visual classifications by an experienced observer fewer images or images of lower quality may also suffice, but result in flare parameters with large uncertainties. This is reflected in the number of 87 flares that were detected by the automatic system, whereas the KSO visual reports included 19 additional events (18 SF, one 1N).

The automatically determined flare importance and brightness classifications were correct in about 85 % of the events. The misclassification of 15 % is similar to the ∼ 7 % (17 %) inconsistencies between the NOAA and KSOv flare reports for the importance (brightness) class. These differences in the official reports are related to different instruments, seeing conditions, and observers who derived the parameters. The mean of the calculated heliograpic longitude and latitude of the flare center was consistent with the official flare reports within ∼ 1°. The median of the absolute differences between the flare start and peak times of the automatic detections in comparison with the NOAA (KSOv) reports were 3 min (1 min). In ∼ 90 % of the flare-alert e-mails that were sent, the alert was issued within 5 min of the flare start. However, 15 % of the expected alerts had not been sent. The number of false flare detections and alerts was lower than 6 % of the total number of (true) alerts issued.

We note that our event set mostly consisted of subflares (69) and importance 1 flares (18). There was one event reported as 2N by NOAA and KSOv (October 10, 2013; cf. Table 2), which was misclassified as 1F by our automatic algorithm. Re-inspection of the processing of this event showed that the area had been correctly calculated (i.e. exceeded the threshold to an importance 2 flare), but as a result of a large data gap during the observing sequence, the algorithm erroneously applied an incorrect time (during the flare decay phase) for the calculation of the importance class.

We conclude that the automatic flare detection implemented at KSO that is provided online at ESA’s SSA SWE portal provides reliable and near real-time detection, classification, and alerting for solar Hα flares. The information on flare timing, strength, and heliographic position (which relates to the magnetic connectivity to Earth) that is derived in near real-time might be connected to SEP models, for instance. We note that the most important challenge of the algorithm is actually the handling of data gaps, which are the largest source of misclassification of the flare class and the split-up of one flare into more than one. Further systematic evaluation of the system at times of higher solar activity and more frequent occurrence of larger flares will be valuable for testing its ability for the automatic detection of the most severe space-weather effective events.

Appendix

Table 2 lists all flares detected by Surya with an area ≥ 50 micro-hemispheres and located within 60° of the solar disk during the period June 26 to November 30, 2013 together with the corresponding information from the NOAA and KSOv flare reports. Column 1 gives the observation date, columns 2 – 4 list the start time of the flare (from Surya, NOAA, KSOv), columns 5 – 7 the peak time, columns 8 – 10 the heliographic position, columns 11 – 13 the flare type, and column 14 the flare area as determined by Surya. False flare detections by Surya are marked in boldface. The letter B preceding the flare start time indicates that the event began earlier, a letter U after the time stamp marks uncertain values, and if a flare reached its peak after the observations ended, a D is added to the time stamp.

Table 2 Automatically detected flares in the period June 26 to November 30, 2013. Flares marked in boldface are false detections.