Keywords

1 Introduction

Remote sensing recognition of crops is the theoretical foundation of remote sensing of agricultural situations [1, 2], which is usually fulfilled by extracting unique spectral, textural and phenological characteristics of crops, and crop type identification using supervised classification, unsupervised classification and machine learning methods in combination with crop growth patterns and conditions [3]. Low spatial resolution remote sensing images such as MODIS data can cover long time series of crop growing period. It can also effectively identify crop phenological crossover phenomenon [4], but cannot identify crop types grown in small plots in complex terrain areas due to the interference of factors such as mixed cropping and plot size. Medium-resolution remote sensing images such as Landsat TM/ETM data can identify relatively more crop types due to the increased resolution. However, it is difficult to obtain long-term sequence images due to adverse weather conditions such as cloud, rain and fog and revisit cycles, which makes it difficult to use growth cycle characteristics (such as growth curve) for crop identification. High spatial resolution remote sensing images, such as GF-1 / 2 data, can extract abundant spectral information and spatial heterogeneity characteristics, which is helpful to improve crop recognition accuracy under complex planting patterns and terrain conditions. However, due to poor data continuity and limited coverage of spatial, the image processing of such images is more difficult [5]. In conclusion, any single remote sensing data source cannot fully reflect the spectral characteristics of different crops throughout the growing season due to the mutual restriction of temporal and spatial resolutions [6]. Therefore, it is of great theoretical research significance and practical application value to study the synergy and fusion of multi-source data in remote sensing recognition of crops.

2 Research Overview

Domestic and foreign scholars have used MODIS, Landsat TM / OLI, GF and HJ images to study crop classification. Early data type is single, often using single source image, which is divided into low spatial resolution and high spatial resolution. Low spatial resolution long-term sequence data can be used to detect large-scale crops, and the crop situation is analyzed by calculating the vegetation index. For example, Xiao et al. (2005) and Zheng et al. (2008) used MODIS data and SPOT-5 images to study the planting structure of specific crops. [7, 8]. Ridhika Aggarwal et al. (2014) [9] used remote sensing images of multi-temporal Landsat-8 OLI data to classify wheat of Radaur city, India. Qingyun Xu et al. (2014) [10] reconstructed NDVI time series curve using MOD09Q1 dataset and combined with crop phenology information to identify the types and cropping patterns of major crops in Shanxi Province. Supervised or unsupervised classification and machine learning methods are often used when using multi-temporal data from high spatial resolution images (Kim, 2014) [11]. For example, Huanxue Zhang et al. (2015) [12] used an object-oriented decision tree algorithm to classify crops from multi-temporal environmental satellite NDVI time series data. Wuyundeji et al. (2018) [13] used GF-1 image data to extract the area of spring wheat in the river-loop irrigation area and monitored the crop growth with NDVI, and found that the accuracy of the area extraction results reached 93.51%.

In recent years, in the research of crop classification and agricultural remote sensing, data source has changed from single-source data to multi-source data set [14], and crop identification method based on satellite remote sensing data collaboration has become a research hotspot. For example, Guangxiong Peng et al. (2009) [15] used multiple typical classification methods to identify and extract crops such as sugarcane and maize in Mile County, Yunnan Province, and the data he used were CBERS02B-CCD and Landsat-5 TM images of CMBR at two times. Songlin Wang (2015) [16] selected low and medium spatial resolution MODIS remote sensing images to extract crop cultivation area in Jiangsu Province, and used medium and high resolution HJ-1A/B images to verify their spatial distribution. Huinan Xin et al. (2016) [17] used a decision tree classification model to monitor crop cropping structure in the Aksu region of Xinjiang. The experimental procedure combined with the spectral information of the higher radiometric resolution multi-temporal Landsat8 OLI images. Aiming at the two key problems of common multi-source image data fusion methods and remote sensing crop recognition methods, this paper makes a systematic review.

Table 1. Cases of multi-source remote sensing data fusion applied to crop identification.
Full size table

3 The Fusion Method of Multi-source RS Data

With the rapid development of remote sensing technology, the acquisition of agricultural information gradually tends to the system of Satellite-UAV-Ground Internet of Things System, which can quickly acquire multi-source and multi-view farmland information data. Multi-source data need to be fused according to certain rules before using [18,19,20]. Based on the literature review of CNKI in the past decade, this paper introduces the fusion method and recognition method of multi-source remote sensing data. The application and recognition effect of each case are shown in Table 1.

3.1 Realization of Multi-source RS Data Fusion for High Temporal Resolution Targets

Multi-source remote sensing image collaboration can expand the frequency of repeated observation on the ground, effectively capture the optimal time window for crop recognition [5, 21], and achieve the goal of “time optimization”. Extracting the long time series spectral characteristics of crops by using the image of crop key growth period or whole growth period can solve the phenomenon of crop phenological period crossing and improve the recognition accuracy [22]. Multi-temporal remote sensing data can be divided into multi-phase homologous sensors and heterogeneous sensors according to different data sources [23].

Qinxue Xiong et al. [24] had used multi-period homogenous sensor data for their study. They selected 17 different time-phase MODIS data from May to December 2001 to analyze NDVI time series curves, and then applied hierarchical classification method and BP neural network method to supervise the classification of autumn crop in Jiangling County, Hubei Province. Crop recognition model is a combination of NDVI time series curve data with high temporal resolution extracted from MODIS data and Landsat ETM standard data. This model provides a reliable basis for high precision crop spatial distribution mapping. The study by PengYu Hao et al. [25] is a typical case of crop classification using heterogeneous sensors. They fused 15-view MODIS data and 7-view TM/HJ-1 data into vegetation index time-series data with both 30m spatial resolution, then transformed the TM/HJ-1 vegetation index into MODIS vegetation index by linear regression model. Finally, they used the minimum distance classification method to distinguish cotton, maize and other crops in Bole City, Xinjiang, and the recognition accuracy reached more than 90%. This study uses heterogeneous source data to establish vegetation reference curves. It eliminates the manual collection of training samples compared with the traditional supervised classification, achieves automatic extraction of crop planting area with high spatial resolution for long time series [26].

3.2 Realization of Multi-source RS Data Fusion for High Spatial Resolution Targets

The use of high spatial resolution remote sensing data can extract richer spectral information of features, clearer texture features and clearer spatial neighborhood geometric relation-ships, which provides new opportunities for high precision extraction of crop target classification and planting area [27, 28]. Small wave transform methods have been widely used in image fusion because of better spatial scale transformation matching [29], and easier understanding of the synthesized images [30].

For example, Xiaohe Gu et al. [31] used wavelet transform method to fuse MODIS temporal images with 250 m spatial resolution and TM images with 30 m resolution, and obtained time series fusion images with 30 m resolution. The minimum distance classifier combined with crop NDVI growth curve was used to distinguish the main crops in Yuanyang County, Henan Province, and effectively extract maize planting area and spatial distribution. Jie Li [32] and Tao Han et al. [33] found that Sentinel-2A could well extract crop distribution information due to higher spatial resolution in small-scale agricultural areas with complex agricultural structures. In Sentinel-2A, different features show significant differences in spectral characteristics and vegetation indices, which makes Sentinel-2A more suitable for the study of small-scale areas with complex feature structures and fragmented land masses. In addition, Bu and Osler et al. [34] showed that the “pixel-level scale extension” of different resolution data can effectively distinguish mixed pixels and identify feature boundaries, which can be applied in feature classification studies.

3.3 Multi-source RS Data Fusion with a Combination of Spatio-Temporal and Spectral Advantages

High spatial and temporal resolution data can improve the accuracy of ground interpretation, and hyperspectral images can obtain the continuous band of feature spectra, which will directly distinguish crop species [35,36,37,38,39]. Therefore, in complex terrain areas with small crop planting area, complex planting pattern and high frag-mentation of farmland landscape [43], it is still urgent to study crop classification by combining temporal and spatial advantages with spectral ad-vantages [40,41,42]. For example, Feifei Shi et al. (2018) [44] extracted crop NDVI time series data based on HJ CCD and Landsat 8 OLI data, while using HJ-1A HSI data to extract spectral feature variables to form a multi-source dataset. They used classification and regression tree (CART) and support vector machine (SVM) to classify major crops such as oilseed rape, wheat and potatoes in Xining City, a plateau region. Ling Ouyang et al. [45] selected GF-1 data and Landsat8 OLI data as remote sensing data sources, and conducted regression analysis on the spectral reflectance of the same ground object. The decision tree classification method was used to detect crop planting structure in Bei’an City of Heilongjiang Province based on crop phenology and spectral characteristics. Xiaohui Li et al. [46] accurately distinguished the cultivated land area of Datong City, Shanxi Province based on GF-1 image, and extracted the distribution of main crops by using landsat8 OLI image. In summary, it is feasible to use multi-source data fusion for crop identification in complex terrain areas.

4 Main Methods of RS Crop Recognition

Extracting important feature parameters of crops based on information such as reflectance spectra, colors, and textures of features and combining them with appropriate classification methods to distinguish crop types [47, 48] is the basis of crop identification from multi-source remote sensing data. In this paper, we introduce three methods for the application of multi-source remote sensing imagery in the field of crop identification, the results of each method and application cases are shown in Table 2.

Table 2. Results of remote sensing crop identification methods and application cases
Full size table

4.1 Recognition Methods Based on Temporal Phenological Features

Phenology knowledge show us that different crops are affected by climate, soil, hydrology and other factors in specific areas, and have different periodic growth and development laws [49]. Studies have shown that the time series of Normalized Difference Vegetation Index (NDVI), Enhanced Vegetation Index (EVI) and Normalized Difference Water Index (NDWI) can accurately reflect the dynamic change trend of crops in different periods [50, 51]. It contributes to solving the problem of ‘foreign matter congener spectrum’ in crop identification and is widely used in monitoring crop annual changes. The key to crop classification based on temporal phenological characteristics is the phenological period and characteristic parameters of crop growth [52]. However, the remote sensing data acquisition and processing are disturbed by many factors such as sensor noise [12] and solar altitude angle, which leads to abnormal fluctuations in the vegetation index curve of time series. Usually, smoothing denoising and eliminating abnormal points are used to reconstruct time series data.

At present, phenological characteristics combined with temporal remote sensing data is the mainstream of remote sensing crop classification research. For example, Ansai, Machao, Rongqun Zhang and Yuepeng Ping et al. [53,54,55,56] used MODIS time series data to establish vegetation time series curve, and classify the main crops in plain and hilly areas by extracting phenological indexes such as the beginning and end of crop growth season and the length of crop growth season. Xia Zhao et al. [57, 58] identified crops in Qinghai Province. The results showed that the recognition accuracy of spring wheat, potato and rape was more than 60%. Yanjun Yang [59] used five different classification methods to classify winter wheat, summer maize, rice and peanut through GF-1 WFV satellite images. The results showed that the NDVI time series curve after smoothing treatment could highlight the overall trend of crops.

4.2 Recognition Methods Based on Spectral Features

Remote sensing images record the electromagnetic wave information of ground objects. Because the spectral reflection characteristics are different, the images show different brightness, texture features and geometric structures [60]. And hyperspectral can record hundreds of narrow bands from visible light to infrared light, which are close to the actual spectrum of crops in the case of high spatial resolution. Therefore, the difference in spectral reflectance of crops can be used as a basis for judgment [61, 62]. The methods of crop recognition based on spectral features mainly include supervised classification and unsupervised classification. The main difference between them is whether there is prior knowledge.

Supervised classification is the process of using training samples to construct discriminant functions to identify classes of image elements [63], and the main methods are maximum likelihood, SVM and decision tree. Lin Zhu [64] used Sentinel-1 and Sentinel-2 multi-source remote sensing data for crop classification based on minimum distance, maximum likelihood, SVM and BP neural network. Crop classification experiments on a farm in Dali, Shaanxi Province show that the classification results of BP neural network without cloud cover are the best, and SVM with cloud cover are the best, the overall classification accuracy is more than 90%. Unsupervised classification only relies on statistical feature differences to achieve classification purposes, and mainly adopts cluster analysis methods such as iterative self-organizing data analysis algorithm (ISODATA) [65, 66]. Limin Wang et al. [65] used ISODATA to classify multi-temporal GF-1 WFV data in Langfang City, Hebei Province, and established semantic constraints. Winter wheat was identified according to Sigmoid spatial membership. The classification accuracy was 95.33% and the Kappa coefficient was 0.90. In short, supervised classification method has high classification accuracy, but requires prior knowledge, and the workload is large. Methods of crop classification depend on specific circumstances.

4.3 Comprehensive Feature Selection for Crop Recognition by Remote Sensing

Auxiliary data are non-image information used to assist image analysis, mainly including parameters such as elevation, slope, slope direction, and various thematic information [67, 68]. Using the spatial characteristics of natural elements and the texture characteristics [69] of measuring the spatial distribution of pixel neighborhood gray can improve the accuracy of crop recognition and effectively avoid the phenomenon of ‘same object, different spectrum’. In the hilly areas with high fragmentation, the spatial feature information can help to express the planting area boundary [43]. Crop classification research uses all the feature information to increase the data dimension, which will inevitably lead to Hughe phenomenon, reducing the recognition accuracy. Dimension reduction is the use of specific algorithms to select feature subsets that are important to the classification process, and has become a key step in processing high spatial resolution images. When feature selection is carried out, the classifier is limited by many factors such as the landscape structure of the study area, so the multi-classifier system has been widely used [70].

Na Wang [71] used GF-1 and HJ-1A images to extract the multi-temporal spectral characteristics, vegetation index characteristics (NDVI, perpendicular vegetation index PVI, difference vegetation index DVI, soil-adjusted vegetation index SAVI), texture characteristics (variance, information entropy, second-order distance, etc.) and band difference information of Sihong County, Jiangsu Province. Then, they design six classification schemes based on random forest classifier and SelectKBest method to select the optimized feature subset (A. spectral feature, B. spectral feature + band difference feature, C. spectral feature + vegetation index feature, D. spectral feature + texture feature, E. spectral feature + band difference feature + vegetation index feature + texture feature, and F. optimized feature subset). The classification results show that the recognition accuracy of multi-information comprehensive features of remote sensing crops is higher than that of single original spectral feature classification.

5 Accuracy Evaluation of Classification Results and Influencing Factors

The accuracy evaluation of crop classification refers to the comparison of the classification results with the actual data to determine the accuracy of various ground objects [1]. Commonly used methods for evaluation of classification results include confusion matrix, result superposition and ROC curve [72], and indicators for evaluation of accuracy include User's Accuracy, Producer's Accuracy, Overall Accuracy, Kappa coefficient, etc., as well as the calculation of absolute error and root mean square error based on departmental statistics [52]. Usually, the higher the resolution of data is, the stronger the recognition ability is. However, the distinction of crop categories is not entirely dependent on spatial resolution. It is necessary to combine the environmental characteristics of topography, geomorphology and soil in the study area and the relative difference between the brightness and structure of the surrounding objects [61]. We should comprehensively consider the above characteristics to obtain data with optimal resolution. In addition, the rationality of training samples and the heterogeneity within plots will also affect the classification accuracy, and the mixed pixel decomposition method is helpful to improve the classification accuracy of crops [73]. The Table 2 shows that the extraction accuracy of comprehensive features or combination of spectral and phenological features is higher.

6 Problems and Prospects

In recent years, with the rapid development of remote sensing technology, the research on crop recognition based on multi-source data has made great progress, but there are still some problems in the classification accuracy and feasibility. In the future, the theoretical system and technical methods of multi-source remote sensing crop identification should be further developed, and its practical application scope should be expanded to promote the development of agricultural remote sensing.

1) Establishing a technical method system for remote sensing crop identification in different ecological zones. The spatial distribution status of crops affects the recognition accuracy of crop types [74]. Because the growing environment of crops has differences, ecological zoning should be carried out according to the agricultural zoning system or farmland landscape, and we should establish a separate system of technical methods for crop identification. Meanwhile, when extracting crop information in areas with abundant crop species and complex terrain, the processing method of image partition can be used to improve the recognition accuracy. However, the size of spatial and temporal scales of different regions or ecological zones and the law of range boundary division need to be further studied, which will determine the selection of remote sensing image types, classification methods, etc.

2) Comprehensive classification features and multi-classifier system application research. There are many characteristic parameters extracted from crop recognition based on multi-source remote sensing data. In addition to the spectral features, temporal phenology differences and texture features, we can also try to classify the area, aspect ratio and shape index as the classification features. However, due to the diversity of information sources, there will be differences in classification, so the comprehensive application of information needs further research. At the same time, it is necessary to consider the contribution rate of different features to the recognition accuracy, and study the influence of feature combination on crop classification, so as to obtain the optimal feature collection in the study area. Studies have confirmed that the multi-classifier system is an effective solution to control the classification uncertainty of remote sensing images and improve the classification accuracy [75, 76]. Therefore, it’s application in crop recognition is a valuable research direction in the future.

3) In-depth exploration of remote sensing technology for crop identification in complex topographic areas. Now the domestic use of optical remote sensing for crop type identification mainly focuses on large area plain agricultural demonstration zone of staple crops such as rice, corn and wheat, cole, potato, soybean and cotton and other crops involved, but there is little research on regional specific crops such as barley and oats in alpine regions such as the Qinghai-Tibet Plateau. Therefore, the potential of remote sensing data to identify crops in complex terrain areas should be further explored, and remote sensing techniques applicable to identify these small crops in complex terrain areas should be studied to provide scientific basis for fine agricultural management of small agricultural areas.