1 Introduction

Cheques have an intriguing past, as evidently, it is still a mystery as to when exactly the concept of cheque has emerged. Speculations were made that the idea in itself was entertained during the Roman empire, however the very idea could not catch on. Thus, its usage came in the business hubs in the 1500s in regions near Holland and Amsterdam because of their richness in business opportunities during that time. Therefore, with its gradual acceptance in the commerce and enhanced usability it was slowly accepted as a means of monetary transaction. The word “cheque” was evidently originated in the 1700s as a common practice for monetary transactions, certain unique numbers were written on the piece of papers which were checked to keep track of and to proceed with the transaction. Financial transactions using cheques is still one of the widely accepted way across the globe. Almost in every continent, banking institutions have provision for their customers to use cheques for any financial transaction. In 2018, people in United Kingdom used 346 million cheques for payments and to acquire cash with totaling a value of £442 billion [13]. Few sample cheques of different countries (Malaysia, Maldives, USA, Kenya, UAE, and China etc.) are shown for reference purpose in Fig. 1:

Fig. 1
figure 1

Sample cheque images of different countries [15, 54, 55]

Full size image

Although cheque systems in banks for transactions is one of the most powerful tool, primarily due to its slow processing time, considerable complexities in cheque clearing process and gradually with an immense growth in the technology, the electronic-transactions have created a big shadow on the cheque system to cater the need of the banking sector. Therefore, we have proposed a mechanism to make the entire process of bank cheque verification not only fast but also reliable by using deep learning methods and image processing techniques. Due to various technical constraints, it was not feasible for the banking sector to implement the automation of the bank cheque clearing which required sophisticated measures for authentication and verification for the transfer of funds from accounts. Therefore, as a use case example, we have implemented and tested our proposed model on cheque samples of different Indian banks which follow the standard guidelines of the Reserve Bank of India (RBI). Before Jan 2019, the security features used in cheques at the country level by all Indian banks were lacking technical and security edge such as non-availability of void pantograph, Rupee symbol, micro-lettering, bank logo in Ultra Violet (UV) ink etc. However taking a major step in modernization of cheque based transactions, RBI made it mandatory to use the CTS-2010 cheque format as a standard format for all regulated banks [17]. There are certain mandatory security features in the CTS-2010 compliant cheques as shown in Fig. 2 and are mentioned below:

  • Bank Logo in UV ink

  • Printer name with CTS-2010

  • Void Pantograph

  • Micro-lettering

  • Date

  • Rupee Symbol

  • Signature

  • Account Number

Fig. 2
figure 2

CTS-2010 compliant sample bank cheque

Full size image

Cheque Truncation System (CTS) is a way to authenticate the cheque clearing process. The process flow of CTS is shown in Fig. 3. There are two steps authentication of any bank cheque leaflet/form before making any financial transaction using cheques. First step is to validate whether the leaflet used, is genuine or fake. Deepak at al. [10] proposed some methods to validate the bank cheque leaflet. Second step to authenticate the cheque is to verify about the details filled on that leaflet. In our research work, we focused on proposing some novel methods to automate the verification of the bank cheques. For this, we extracted the key information from the cheque leaflet which were required for verification of bank cheque and represented these features in the highlighted using bounding boxes. There are various techniques already available for the identification, authentication and verification of the bank cheques as shown in the research works presented by Dhanva et al. [23], Singh and Tigga [78], Gonge and Ghatol [28], Gonge and Ghatol [29], Menon et al. [56]. Unfortunately, the efficiency of the works are not up to mark. For the task of verification, we considered matching of Indian Financial System Code (IFSC), cheque number, account number, legal amount, courtesy amount and signature of cheque issuer as input components.

Fig. 3
figure 3

Process flow diagram of proposed work

Full size image

Optical Character Recognition (OCR) is an automated text reading process from images that includes machine printed (typographic) or handwritten character recognition and numeral recognition. Image processing is a process of analysing a digitised image for performing operations such as enhancement, compression, segmentation, or editing. The fundamental stages of image processing which almost all the image has to go through include the following components which are sequential in nature: acquiring the image, pre-processing of the image, segmenting the image, interpreting the image, and recognising the image. There are many ways to interpret any image. One effective way is using convolution neural networks (CNN). The deep learning approach is an advancement of machine learning with multiple layers allowing it to achieve higher level of accuracy for pattern recognition. However, it demands a large number of datasets for the desired training efficiency of the network. If we have a enough large database, its implementation will assist in precise estimation of the handwritten characters so as to give accurate algorithmic implementation ability for the bank cheques. This may benefit the banking sector by reducing the processing time and saving resources by automating the task of cheque clearance. We implemented our research work using MATLAB-2018a.

The major contribution of our research was to identify the various possible techniques to verify the bank cheque and then apply the best selected image processing and deep learning methods to improve the performance of automated verification in terms of accuracy. After that, we compared our results with existing work and showed the comparative analysis on how our innovation covers all primary aspects to verify a bank cheque in a more efficient and effective way in terms of accuracy. We describe the major contribution of our innovation in the following steps:

  • Captured the following key components of a scanned bank cheque image using image segmentation: Date, Account Number, Cheque Number, Signature of account holder, Courtesy amount detail, legal amount detail, associated bank information (IFS Code)

  • Validated bank branch information, genuineness of the cheque and customer account number by applying OCR method on IFSC segment, cheque number segment and account number segment respectively.

  • Obtained the pattern of signature and match it with the specimen signatures of account holder using SIFT and SVM methods.

  • Recognised the characters stated in the legal amount segment of the cheque using CNN method and convert it into the place value system.

  • Recognised the digits stated in the courtesy amount segment of the cheque using CNN and verify it with the extracted and converted legal amount information using the proposed algorithm.

This paper is organized as follows: Section 2 summarises the related work. Section 3 presents the complete methodology and proposed algorithm along with the models for verification of bank cheques. Section 4 presents the experimental results and analysis of our implemented model. Finally, section 5 concludes the research work and outlines future work.

2 Related work

In order to perform the task of getting a logical response for the input of an image, it requires a meaningful interpretation of the characters present in the image and, thus, we need to perform the character recognition for the various patterns present in an image. Singh and Sachan [77], Wankhede and Mohod [83], and Ramanathan et al. [70] presented their work to retrieve the characters from the image Optical Character Recognition (OCR). The segmentation of digits for which there are several techniques devised is the major challenge faced in the recognition of characters. Chen [12] proposed a general approach for handwritten digit segmentation using spectral clustering and achieved an accuracy of over 97%.

One of the earliest researches on font recognition was done by Dickinson [24], Liu and Shelton [51], Nagy and Shelton [62], Wang et al. [82], Zramdini and Ingold [88] in which they suggested a statistical approach for the Latin Language font recognition at the text line level depending on the global typographical features. They were able to achieve approximately 97% of recognition rate. For English language also, due to its widespread usage and commercial usability, Zhu and Wang [87], Dhande and Kharat [21], Pal et al. [64], Dhande and Kharat [22], and Saenthon and Sukkhadamrongrak [71] proposed few approaches for character recognition. Font recognition was done by using a wide array of techniques and they were able to achieve a satisfactory accuracy on text blocks.

Similarly, Ramanathan and Sharma [69] devised a method using Support Vector Machine (SVM) for classification and feature extraction by proposing a SVM-based multi-knowledge-based system, achieving an accuracy rate of 93.54%, which marked the continued focus on the OCR or segmentation for the character recognition in image processing methods. However, Stewart et al. [80] proposed a segmentation free recognition techniques in which they presented that for small training datasets, it is possible to manipulate either manually or automatically that segmentation and stitching improves handwriting recognition on dataset with few samples. In another attempt by Peres et al. [65] in 2018 proposed a novel handwritten methodology in which they used a hybrid CNN and SVM classifier with which they were able to achieve an overall accuracy of 96.7% using the MNIST dataset [47].

For intelligent character recognition to create and match dynamic templates as well as the conversion of characters into text, Kajale et al. [43] implemented the supervised machine learning methodology and achieved the accuracy close to 95% for the alpha numerals and special characters. The presence of the noise in the form of lines, watermarks, diagrams, patterns or all of these at the background is one of the important factors to create the performance variation in different character recognition techniques. Goto and Aso [30], Brisinello and Gibric [7], Brisinello et al. [8], and Feng and Gao [26] performed research involving extraction of the information from the colored or noisy background with satisfactory level of output. Contrary to the notion that output will drop significantly if there is a change in the background with varying levels of color change in the texts or introduction of noise in the form of patterns and figures. Jia and Wei [40] extracted the legal amount from the Chinese bank cheques and proposed a multiple channel binarization approach for extracting legal amount from color Chinese bank check images with a combination of global and local threshold. Sneha Singh et al. [79] proposed a method to recognise the legal amount of Indian bank cheques using correlation coefficient method. They achieved an average 76.4% word recognition accuracy on a self created dataset of 19215 word samples of 61 different words. Another researchers M Badrul Alam et al. [57] developed a tool to recognise the handwritten legal amount of Bangladesh bank cheques using ANN and achieved only 93.4% accuracy. Similarly, Jayadevan et al. [38] proposed a lexicon-driven segmentation recognition technique for the recognition of the legal amount words from the Indian bank checks written in English using Modified Quadratic Discriminant Function (MQDF) and claimed 97.20% accuracy.

Raghevendra and Danti [68] proposed a method to identify the bank details associated with the cheques using bank after removing the background noise. Researchers used the invariant geometrical features for accomplishing the recognition of the checks with 93.8% of accuracy only. To improve the security for verifying the cheque, another important feature of it is Magnetic Ink Character Recognition (MICR) code. Even though if we are able to perform the pattern recognition as well as the legal amount recognition using the above stated techniques one more component of a bank check that needs to be properly identified for the sake or security as well as determining the validity of the check is the Magnetic Ink Character Recognition (MICR), Shirai et al. [75] proposed a three pronged solution to the problem starting with the character color estimation and conversion into binary form to remove the background, to separate each character using the character pitch estimation and the removal of the unwanted noise & extraction of the required MICR code using the sign fragments removal by the stroke tracing.

Lu et al. [52] proposed techniques to deal with noisy images as the image that we have, may be a copy of the original image with some considerable noise such as shadow along with a spatial noise due to improper copying of this image. Researchers used local adaptive threshold algorithm in order to transform the gray-scale image into the binary image to capture the contours of the texts in the image and applied a double-filtering algorithm to remove the noise between the texts and the characters. Wiraatmaja et al. [84] proposed a deep convolutional de-noising auto-encoder and claimed the improvement in accuracy by 26.78% as compared to a simple OCR in average accuracy testing. Void Pantograph is another important feature which ensures whether an original image of the cheque was taken for verification or not. It refers to the process of creating a pantograph which is otherwise invisible to naked eyes on the original print. However, when copied with a printer, it is clearly visible making it a very important security features for security printing toolkit. Aronoff et al. [3] formulated a method to automate the optimization of void pantograph settings. Peymani and Soryani [66] proposed a model to identify machine generated as well as hand written characters for Farsi language images with an accuracy of 68% using an Arabic language dataset [25]. Bhadwal [5], Singh [11] and Agrawal [1] also developed different applications ranging from object recognition to edge detection by using image processing and computer vision methods. Ignat and Aciobăniei [35] proposed method of feature extraction for offline handwritten digit recognition by using rotation and filtering of edges for the extraction of the digit characteristics. KNN and SVM classifiers were used for classifying the features and claimed to achieve the accuracy about 99% on the MNIST dataset.

Hou et al. [34] implemented a model using convolutional neural Network for learning the handwritten numeric characters and perform scratchy or non-scratchy decisions with error rate of 0.40% on MNIST database. Li et al. [50] used grab cut segmentation based on image region by considering image texture and boundary information which helps them to perform segmentation with minimal user interaction. Mityakov et al. [58] in application of modern architectures of deep neural networks for solving practical problems explained the advent of sophisticated networks like GoogleNet [81] capable of automatic feature extraction and classification. Evidently watershed segmentation is one of the most widely used algorithm in terms of achieving feasible results however in cases of adhesion among components of the image, it becomes difficult to obtain remarkable results. Ji et al. [39] proposed an improved watershed algorithm based on distance transform for cell image segmentation.

Shenoy et al. [74] proposed a performance based string matching algorithm for Nvidia GPUs in which they are using Quick Search, Horspool and Brute Force string matching approaches allowing a performance boost. Whereas, Joshi et al. [41] proposed modified Rabin-Karp method by parallelizing conventional Rabin-Karp algorithm. Zhou and Gao [86] used double hash in order to perform multi-pattern matching and are able to reduce the number of comparison by two hashes thus giving a slight performance boost. Consequently all these approaches worked towards reducing the space-time complexity and improving the performance for string matching. Dhande & Kharat [21, 22] proposed a method to perform pattern recognition using vertical and horizontal projection methods along with convex hull algorithm for feature extraction for cursive English handwritten characters and implemented support vector machine (SVM) on CCC dataset [9]. Similarly, Dash et al. [20] proposed a perimetric complexity based on gestalt principle to develop a handwritten numeral recognition system for obtaining a point-wise correspondence from the matching of shape context based on affine transformation on a dataset of Odia numerals. Li et al. [49], Shen et al. [73], Shirazi et al. [76], Kalaiyarasi and Karthikeyan [44], Pornpanomchai et al. [67], Sahbi et al. [72], and Kaur [46] proposed various approaches to recognize of logos present on the cheque leaflets for identifying the issuing bank of the bank cheque.Few remarkable signature recognition methods are highlighted as shown in Table 1:

Table 1 Methods of signature recognition
Full size table

If we summarise the state of the art, many researchers worked on image segmentation, character recognition and signature recognition separately and also achieved remarkable performance. Few researchers considered bank cheque validation as their research application but none of the researcher proposed any complete solution to successfully implementing the automated cheque clearance process without any human intervention. While in our research, we proposed the solution starting from pre-processing of scanned cheque image to verification of cheque amount and signature. In further section, we described the entire methodology to implement the model and presented the results which are found satisfactory.

3 Methodology

The bank cheque verification tool required fulfillment of some key stages in sequential manner. At first, the tool ensured the genuineness of IFSC present on cheque then it verified the cheque number to ensure whether the cheque was from the set of cheque leaflets assigned to the account holder or not.

After that, the amount was verified by checking whether the customer was having sufficient amount in his account for which he released the cheque, and finally the signature of the cheque issuer(s) were verified. Only after the completion of these necessary verification steps, the cheque clearance was passed for transfer or withdrawal of funds.

The entire process of cheque clearance happened in a sequential manner and each step was mandatory for the next step. All the key features presented in the cheque verification are shown with their description in Table 2.

Table 2 Bank cheque verification key features and their description
Full size table

Different methods were used for extracting such information from the cheque leaflet that depended on the viability and level of accuracy. For example, to extract information from the machine typographic script, we use OCR method as it tends to produce results with higher level of accuracy whereas for hand-written words and numerical values, we can use deep learning based CNN. Similarly, for authenticating the signature(s), we extracted the features using SIFT algorithm and for classification, SVM produced optimum results. To implement these methods, we required noise free image therefore the better idea was to segment the image so as to acquire only the relevant information.

Figure 4 shows the broad steps using a flow-chart, involved in our research process of cheque verification. These steps encompassed certain sub-steps each of which were essential to meet the standards so as to extract the required information from the bank cheque. The entire methodology for bank cheque verification is summarized in Fig. 5.

Fig. 4
figure 4

Flow chart for bank cheque verification

Full size image
Fig. 5
figure 5

Block diagram of bank cheque verification process

Full size image

3.1 Image acquisition

The image acquisition of a bank cheque is crucial for the CTS. Generally, flatbed scanners are used to acquire such images. Due to orientation and irregularities presented in the scanned image(s), we were unable to use the acquired image(s) directly for the image processing operations therefore it required some pre-processing steps.

3.2 Image pre-processing

We used the cheque scanned images for our research. A sample scanned image is shown in Fig. 6. As a scanned image obtained from the scanner can not be used directly thus we needed to pre-process it, which involved two primary operations, i.e, rotation and removal of unnecessary background information. In this step, we rotated the scanned image with respect to the ‘Date Box’ (which is a common feature presented at the same part of every bank cheque) and then removed the background noise and extra information. Efficiency of the parameter identification considerably improves with removal of extra background information.

Fig. 6
figure 6

Raw scanned image of a sample cheque

Full size image

3.2.1 Rotation

As the scanned images may vary in terms of orientation therefore we used the date box present in all standard cheque leaflets and the relative invariant nature of the position was utilized. In order to perform the rotation of image, we determined the point of rotation and degree of rotation. The primary component essential for rotation to work was contour extraction, as we were able to determine the position values of date box by using it which used as our anchor for any set of operation related to length mapping. The process of contour extraction was explained in the forthcoming segmentation sub-section. Also, in order to perform the rotation, we used the midpoint of the image as the rotation point, and we used date box in order to determine the angle necessary for the rotation. Figure 7 shows the same process.

Fig. 7
figure 7

Possible scenarios for rotation of cheque leaflet

Full size image

As shown in the figure among the possibilities for rotation, we represented each quadrant with a, b, c, d and highlighted the quadrant consisting the contour of date box with red color. We performed rotation by 90°, 180° and 270° depending upon the quadrant in which the date box is present. In Fig. 7, the scenario the order of rotation for possibilities was 270°, 180° and 90°. Once the cheque orientation became correct, we proceed with the removal of unnecessary region available in the scanned cheque leaflet.

3.2.2 Removing background noise

There was lot of extra information in the scanned image which was to be removed. For the task of removing of background information, we used the date box present in the standard cheque template. The cheque dimensions of any bank in a country are standard and are followed by the norms decided by the government financial body. In India, RBI monitors and controls the functioning of all Indian banks. A sample cheque leaf with it’s standard dimensions regulated and decided by RBI, is shown in Fig. 8 [18]. All Indian banks are bound to follow these standards and geometric dimensions strictly. These geometric dimensions may vary for banks of different countries and can easily be calculated as per the respective country agency standards. Using the date box coordinates range and with the implementation of length mapping between the date box and the whole cheque image, we determined the range of the cheque as per the schematic representation of edge removal as mentioned in Fig. 9. In reference to data box, we extracted the regions of interest for courtesy amount, legal amount, signatures etc.

Fig. 8
figure 8

CTS-2010 cheque form specification as per RBI guidelines [18]

Full size image
Fig. 9
figure 9

Schematic representation for edge removal

Full size image

In the schematic representation, the black rectangle represents the scanned image of the bank cheque whereas, the red rectangle represents the edge of the cheque leaflet and the green rectangle represents the edge of the date box. The purpose of the edge removal task was to remove the region between the red and black rectangle and to retain the region in

the red rectangle only. We used following mathematical equation for length mapping of bank cheque leaflet as represented below in (1).

$$ \left\{ \begin{array}{l} \frac{AD}{(AD + a)} = \frac{(y_A-y_D)}{y_2-y_D}\\ \frac{AD}{(AD + c)} = \frac{(y_A-y_D)}{y_A-y_1}\\ \frac{AB}{(AB + d)} = \frac{(x_B-x_A)}{x_B-x_1}\\ \frac{AB}{(AB + b)} = \frac{(x_B-x_A)}{x_2-x_A} \end{array}\right. $$
(1)

Further the equation is simplified as (2).

$$ \left\{ \begin{array}{l} y_2 = (\frac{((y_A - y_D)\times(AD + a))}{AD}+y_D)\\ y_1 = (y_A - \frac{((y_A - y_D)\times(AD + c))}{AD})\\ x_1 = (x_B - \frac{((x_B - x_A)\times(AB + d))}{AB})\\ x_2 = (\frac{((x_B - x_A)\times(AB + b))}{AB}+x_A) \end{array}\right. $$
(2)

The parameters involved in the equations are explained in the Fig. 9. Here, dimension AB and AD are representing the length, breadth of the date box. Whereas, a, b, c, d are the distance from the edges of the cheque leaflet. Also, x and y are the horizontal and vertical coordinates of the date box. The key parameters of these equations are the boundary of cheque leaflet which is the coordinate range for unnecessary background information removal.

3.2.3 Grayscale operation

After removing the background noise of an RGB (a three channel) image, we converted it into grayscale (a single channel) image. The mathematical formula for this conversion is represented in (3) [42].

$$ Gray Image = 0.2126\times R + 0.7151\times G + 0.0721 \times B $$
(3)

Here RGB represents the Red, Green and Blue channels of the image and Gray Image represents the pixel value of the grayscale image.

3.2.4 Gaussian filtering

After converting the image into grayscale format, next pre-processing step was to remove the noise. For this, we applied Gaussian de-noising method which is used in a linear form of smoothing filter for a signal. This method of filtering is a form of convolution between the normal arrangement of values and the image. The formula for 2-D Gaussian function is explained in (4).

$$ G(x, y) = \left( \frac{1}{2\pi\sigma^2}\right)\times e^{\frac{-(x^2+y^2)}{2\sigma^2}} $$
(4)

Here G(x,y) is the normal arrangement of the values. Whereas, σ represents the standard deviation of the arrangement.

We used the binary image format of the gray-scale image for precised and accurate contour extraction, i.e., we converted the image into binary format and then determined the boundary points for the region using the formula as described in the (5).

$$ \left\{ \begin{array}{l} f(i, j-1) = 0\\ f(i, j) = 1\\ f(i, j) \geq 1\\ f(i, j+1) = 0 \end{array}\right. $$
(5)

In this equation, f(i,j) function for binary image represents the pixel value of the required point, and the functions f(i,j − 1) and f(i,j + 1) are representing the pixel values of the neighboring points for f(i,j).

3.2.5 Morphological operations

After the removal of noise from the image, last step of pre-processing was to perform morphological operations. There are much chances to have certain additional values present in the image even after applying filters which hamper the process of identification by using CNN. Therefore, we performed erosion and dilation for morphological operations which were able to determine the local minimum and local maximum. We applied dilation first followed by erosion. This closing operation was capable of closing small cracks in the image while not changing the shape of the it.

3.3 Segmentation

To use only the desired part of the image for various operations, the good idea is to do image segmentation. For this, we separated each pattern locally in the optimum way. Thus, we performed segmentation to ensure that the tool would use only the required information of image for its processing, while accessing the entire process. We separated each key parameter of the cheque leaflet using segmentation for correct identification and verification. Other than contour extraction in order to determine the position values for the date box as explained in previous section(s), we identified region of interest (ROI) as per the CTS-2010 cheque leaf standard dimensions as shown in Fig. 2. In order to remove the extra information from the desired region, we created a standard template using pixel values as per RBI guidelines on CTS-2010 for bank cheques as shown in Fig. 10. After the segmentation of bank cheque image, we separated each segment of key parameters for OCR to determine the patterns for verification. We also performed transfer learning approach in order to recognize the handwritten numeric digits present in the date and courtesy amount of cheque. For Indian banks, image segmentation work is considered automated as the cheque dimensions of different banks are standardised as per CTS-2010 standards while for different country cheques, it should be done either manually due to variation in cheque dimensions or separate dimensions will be defines as per the country or bank guidelines. Therefore we consider our approach for bank cheque leaf image segmentation as semi-automatic.

Fig. 10
figure 10

Sample bank cheque image segmentation

Full size image

Also, to identify the machine typographic characters of IFSC, cheque number and account number, we used OCR method while for the Similarly, signature verification, we used SIFT for feature extraction and SVM for classification.

3.4 CNN model for legal amount and courtesy amount identification

After extracting the image segments, we used CNN for handwritten numeric digits recognition of courtesy amount as well as to convert the legal amount into string. We used Deep Learning Toolbox (a MATLAB toolbox) for CNN implementation on two convolution layers with six and twelve filters as shown in Fig. 11. Whereas, in max pooling process, we used a window of 2 × 2 with one thousand epochs and one stride. After that, we received numeric output from the courtesy amount of bank cheques and converted it into string to compare with the legal amount string. We followed IPV system so as to be written in the words IPV format. For handwritten courtesy amount identification, we used transfer learning for digit identification. We used MNIST dataset [48] of handwritten digits which has sixty thousands examples in training set and ten thousands examples in test set. All the images in the dataset are normalized and centered in a fixed size. We used a dataset of two thousands samples (two hundred for each digit number ranging from 0 to 9) from training dataset and five hundred (fifty for each digit) samples from testing dataset. We distributed the entire dataset in 80% for the training and 20% for the testing using CNN. We trained with the training loss occurring with one thousand epochs, 0.01 base learning rate and seven iterations per epoch. Also for handwritten characters used in legal amount recognition, we used EMNIST dataset [16] of English letters identification. EMNIST dataset contains 88800 training examples and 14800 testing examples. We used a dataset of 4160 training samples (160 for each letter with mixed uppercase and lowercase alphabets in twenty six classes) and 1040 (forty for each letter) samples for testing dataset in our work. For CNN training, we distributed this training and testing dataset in a ratio of 80:20. Rest all the CNN training parameters are kept same as it was used for MNIST dataset training. Table 3 shows the training parameters for MNIST and EMNIST sample dataset training.

Fig. 11
figure 11

Proposed CNN architecture for handwritten digit recognition of courtesy amount

Full size image
Table 3 CNN model parameters for MNIST and EMNIST dataset
Full size table

3.4.1 Proposed transfer learning algorithm for amount verification

After identifying the digits from courtesy amount image segment and characters from legal amount image segment using CNN, we arranged the acquired numeric digits into the Indian Place Value (IPV) system for matching and comparing it with the legal amount identified text. For this, we developed an algorithm “transfer learning method” to identify different nuances of handwritten strings of numerals in accordance to the IPV chart. Then we converted courtesy amount into strings for comparing it with the legal amount. It is very essential to have a clear match between both the strings as only after that the process of funds transfer is supposed to be initiated. For this we convert the numeric values into strings and then they are compared with the legal amount strings. As the way of referring the amount IPV system be different due to local influence thus a dictionary function was used which kept all the possible ways, a word can be written.

figure i

We followed the algorithm as shown in Algorithm 1 to convert the numeric value of the courtesy amount as per the IPV numbering system and then to match with the strings of legal amount. Once we converted the numeric digits into string, next step was to match these strings. For matching the legal amount with courtesy amount, we used a dictionary function that verified the characters between both string values. If the strings match each other, the result was displayed as verified otherwise not-verified. Using this algorithm, we compared the legal amount with the courtesy amount for verification.

3.5 Signature verification

Next, we moved to signature verification step. In the dataset 114 images used for our experiments, four images do not contain signatures. Therefore, we used 110 handwritten offline signatures from scanned cheque images for signature verification task. These 110 signatures were created by 50 different users. Out of these signature samples, 80 were genuine while rest 30 were forged signatures. Each user’s signatures were found in one to three different cheque images. Figure 12 shows the overall signature verification sub-process of bank cheque. For feature extraction, we normalized each signature image segment in order to extract the features like resizing, thinning, rotating and cropping. The normalization experimental results are represented in the Fig. 13. From the figure, it is clearly shown that we identified the region of interest (of actual signature) by normalizing the signature image. We applied SIFT method on these normalized images for feature extraction and contour identification. SIFT works on a principle to reflect the structure of the image regions using key points and preventing the scale and orientation information of these structures. We applied SIFT method into four steps: i) Scale-space extrema counstruction (SSE), ii) Key point localization (KPL), iii) Orientation assignment (OA), iv) Key point descriptor extraction (KDE).

Fig. 12
figure 12

Account holder Signature classification process for bank cheque verification

Full size image
Fig. 13
figure 13

Normalization of Signature: a Original segment of Signature. b resized image. c morphological Operation: Thinning. d rotation of Image. e cropping

Full size image

SSE breaks down each image segment into Gaussian pyramid at different levels. It further breaks each level into small sub-levels with different variations using Laplacian of Gaussian (LoG) filters to get the prominent keypoints. Later in next part, secured locations and key points are detected and calculated. OA is responsible for calculating the scales and orientations of finalized keypoints. KDE is responsible to generate a SIFT descriptor against each selected key point.

After extracting SIFT descriptors and scales & orientations for each signature image, we applied SVM classifier on these two features dataset to match and classify the signature patterns. SVM is a machine learning method under supervised learning. It is used for both classification as well as regression. To verify the genuineness of the signatures, we applied SVM classifiers on SIFT extracted features. We distributed genuine and forged signatures data in uniform manner between training and testing dataset in ration of 80:20. For this, 88 signature data (with 66 genuine and 22 forged signature data) were taken in training dataset while 22 signature data (14 genuine and 8 forged signature data) were taken as testing dataset. In SVM, a separating hyperplane works as a discriminatory classifier which divides the data using a line or plane into two parts in two-dimensional space [53]. This method produces the classification results as data-points which are categorised ideally using optimal hyperplane.

4 Results and analysis

We experimented the problem stated in the research with the procedure elucidated above in terms of performance and accuracy using MATLAB software (deep learning toolbox and image processing toolbox). We used 114 bank cheque images in our research work. Out of which, 112 images were from IDRBT bank cheque dataset and 02 cheque images were scanned by us [14, 19]. We performed training and testing on the developed system with these cheque leaflets. We used different key parameter segments obtained for bank cheque leaflet for training and testing.

For handwritten digit recognition using CNN, we achieved the accuracy with a mean value of 99.14% as shown in Table 4. Similarly, the learning progress of training networks for character recognition on different epochs and iterations are shown Table 5. From the table, we can see that the mini batch loss is reduced up to 0.0077 with 99.94% accuracy at 850 iterations. The experimental results are represented in Fig. 17.

Table 4 Accurate recognition rate for individual digits
Full size table
Table 5 Experiment results of CNN for character recognition
Full size table

We achieved the collective loss less than 1% for the trained network. After extracting the numbers from the courtesy amount images, we converted these numbers into words using IPV system. Once we acquired the result in words for the courtesy amount, we compared it with the legal amount using the algorithm mentioned above and in case of matching the strings of courtesy amount and legal amount, we proceed with the verification of signature. We first normalized the segmented image, extracted the features using SIFT method and then we applied SVM classifier on extracted features to match and classify the patterns. Using SVM, we were able to achieve 98.1% accuracy.

In order to acquire the required segments for different cheque leaflets, we assigned different templates using pixel values. We took an example of axis bank cheques and its multiple segments to demonstrate the different key parameters which are represented in the Fig. 14. We used OCR technique and applied in-built OCR method for identifying all machine typographic text and CNN for hand written courtesy amount text. We used SIFT and SVM as referred in [85] for recognizing the signature patterns. Another sample cheque with leaflet segmentation is presented in the Fig. 15. Considering a sample cheque leaflet for experimenting the verification procedure, we got the experimental outcome as shown in the Fig. 16. We executed the segmented images from Fig. 15a, b, and c by OCR for text identification while for Fig. 15d and e, we implemented CNN respectively (Fig.17).

Fig. 14
figure 14

Segmentation of a bank cheque image: a Original scanned cheque, b cropped cheque leaflet, c gray-scale cheque image, d IFS code, e account number, f cheque number, g courtesy amount, and h signature

Full size image
Fig. 15
figure 15

Segmented regions of SBI bank cheque: a Original scanned cheque, b cropped cheque leaflet, c gray-scale cheque image, d IFS code, e account number, f cheque number, g legal amount, h courtesy amount, and i signature

Full size image
Fig. 16
figure 16

Image segments on left hand size and their OCR results (shown as value of s1) for machine printed and handwritten characters on right hand side for a IFS code, b cheque number, c account number, d legal amount, and e courtesy amount

Full size image
Fig. 17
figure 17

Experimental loss in training

Full size image

There are advances in the field of forensics still handwritten signatures are considered as one of the most reliable feature for certifying authenticity due to its unique style of writing. We can perform online as well as offline verification of signatures and both have their benefits and shortcomings. Some of the highly efficient methods for signature verification are summarized in the Table 1.

Module wise system performance comparison in terms of accuracy with the state of the art methods, is summarised in Table 6.

Table 6 Module wise performance analysis of proposed system with other existing works
Full size table

5 Conclusion and future work

We developed the model to verify the bank cheques using OCR, CNN, SIFT and SVM. We used OCR method to identify the machine typographic characters with desirable accuracy and efficiency, whereas, we performed CNN to give precise output for the handwritten digits written on the cheque leaflet. We proposed and implemented the algorithm to convert numbers into words to verify the cheque which is one of the major reason of its bounce-off and of halt the monetary transaction. In order to achieve this, we applied OCR technique to recognise the machine printing digits and achieved 97.7% accurate matching. Then, we used separate database to train the network and after achieving a desired level of accuracy after training, we used different data sets to test the trained model for matching the numbers into the words. Evidently, we achieved an accuracy of 99.14% for the digit recognition which is an improvement over the previous accuracy of 99.05% for the CNN used for digit recognition. Similarly, for character recognition using CNN, we were able to achieve the accuracy up to 99.94%. For signature recognition, we used SIFT and SVM classifier to determine the relevancy for verification of signature and achieved 98.1% accuracy. our innovation will help the banking organizations to automate the process of cheques clearance in faster and easier way in comparison to conventional and manual approach by saving time and manpower. In our research work, we performed the verification of bank cheques using the CNN, in an effort to automate the process of verification of the bank cheques based on the identification of the handwritten numeric digits. However, in order to successfully perform the clearance of bank cheque, we need to perform the validation of bank cheques as well as ability to function online using a database resolving the required queries in real time.

In the future, verification process can be clubbed with validation process of cheque leaflets by developing a system capable enough of sequentially performing all the required steps for clearance of bank cheques in real time using a customer-centric database with the required information of a bank customer and also allowing the bank to keep track of all the transactions via cheques in a centralized manner, making the process of cheques clearance completely automated with minimal human intervention and in minimal processing time. Currently, our innovation is applicable for the cheques written in English language only. In future, similar models may be proposed for other languages too.