1 Introduction

Visually impaired people use the Braille [1, 2] language for written communication. Braille was invented by Louis Braille in 1800 to empower the blind people to read and write books, letters and documents. In the traditional form of this language, 6 dots in 3 rows and 2 columns are used in different combinations to represent 26 alphabets uniquely shown in Fig. 1a. Later on various transcripts are also defined by various authors to map the Baillie to other languages like Spanish, German, and Italian etc. The character based Braille is referred as “grade 1” which occupy large space on pages and slow down the writing speed. To overcome these deficiencies, “grade 2” Braille [1,2,3] is defined to represent the “dotted characters” for words or group of characters shown in Fig. 1b. In more advanced form, “grade 3” Braille reflects the shorthand to provide symbolic form for larger words. But this type of Braille is not in common use. Braille can be written on embossed paper either manually or optically. The embossed paper, presents the dot pattern as raised symbols which can be recognized by touch sense. To write Braille manually or optically some specialized metal or plastic device and tools are required. These devices punch the holes on embossed paper to print the raised dots. The orientation and the punching pressure should be proper for faster, correct and optical reading of the Braille.

Fig. 1
figure 1

Samples of Braille Codes a Braille Character Codes b Braille Word Codes

Full size image

The quality of Braille recognition system [4,5,6,7] depends on various associated phenomenon. The dependency elements include the quality of capturing equipment, observed sample space, capturing environment, adapted pre-processing techniques etc. Various methods can be applied to resolve the irregularity issues and to generate the effective decisive features. The processing feature space can ensure the accuracy of textual prediction. The extracted features can be processed through some classifier to recognize the character. Various supervised and unsupervised [8] classifiers are available to recognize the class of dotted patterns. The testing data is once identified in specialized class, the natural language constructs can be mapped to recognize the actual character or word.

In this paper, an alignment and impurities robust model is presented to improve the capabilities and accuracy of Braille recognition system. In this model, various mathematical filters and operators are applied in composite form to perform a peered mapping of Braille character to text. In this section, the significance, scope and characterization of Braille recognition system is explained. In Sect. 2, various methods and framework provided by the earlier researchers to enhance the scope and accuracy of Braille recognition and character recognition is described. In Sect. 3, the proposed disruption and alignment robust model is provided with relative mathematical formulation. In Sect. 4, the implementation results applied on two different dataset is presented and discussed with comparative evaluation. In Sect. 5, the conclusion obtained from the work is presented.

2 Related work

Braille recognition system is valuable automated recognition method defined to read the documents or text written by visually challenged persons. To read these documents correctly, the visibility and alignment of the dotted pattern in scanned documents is the essential requirement. Various pre-processing methods are also applied by the researchers to highlight the pattern and extract the pattern accurately. Various classification methods are also proposed by the researchers to convert the Braille to English or other languages. In this section, various methods and formulations provided by the researchers is discussed. A study work on OBR (optical Braille recognition) for converting the Braille to Text script was provided by Hanumanthappa and Murthy [1]. The standard recognition model with each intermediate stage is also explained by the author. Key requirement of OBR is to identify the dot pattern accurately; a wider pre-processing and dot exploration stage is required. A study against skewness, noise, dot-cell exploration was provided by Isayed and Tahboub [2]. Author described the dot extraction based on horizontal and vertical projection profiling.

Various machine learning methods can be applied on extracted dots to identify Braille corresponding text correctly. Various supervised and unsupervised learning methods were invented by the researchers to handle the existing structural, quality and featured challenges. A work on Deep Learning [9] classifier against noise and corrupt input was introduced to minimize the recognition error. A fine regression tuner was applied against multi-class problem to generalize the classification process. A hybrid model using Haar Wavelet and SVM [4] was applied to tackle the geometric and color specific variations. The boundary separator and edge features from different orientations were processed to improve the accuracy of OBR. The contour [6] measure was used to generate the structural elements of Braille characters and map them over lookup table. Author applied the morphological operators and applied a serious of methods at pre-processing stage to extract the information accurately. Li and Yan [10] used the SVM classifier on segmented featured information extracted from Braille images. Author applied the contrast and geometric correction to reduce the recognition rate. A work on slanted image and deviation [11] preserved method was provided to improve the robustness of Braille recognition. The neighbour dot analysis and grid feature extraction and interpretation were implemented by the author to observe various aspects of image features. To extend the segmentation phenomenon for multi-class Braille recognition, the inter-class variation estimation and Gamma [12] distribution was applied to avoid the impact of asymmetric nature of captured pattern images. The statistical evaluation was also implied to increase the probability of segmentation process. The parameter specific estimation under rotation, indentation and orientation was provided by Tai et al. [13]. The extracted key elements were processed by belief propagation and hidden Markov model for probabilistic recognition. The orientation and angular adjustment was also implied to avoid the error in dot detection. Another work improves the Braille recognition against skewness was provided by Hassen and Assabie [14]. The scale and distance independent OBR was suggested by the author.

The ontology [15] theory was adopted to train the Braille characters for more effective and independent recognition. The expansion was provided to existing OBR system by learning the larger set of Braille characters and its mapping to Chinese text. A twin shadow [3] approach was considered in Braille character recognition under different illumination. An effective solution of Braille recognition was provided to resolve the problem of background texture and noise in real time capturing of characters. Yousefi et al. [16] has proposed a probabilistic statistical method to recognize the Braille document effectively. The method was adaptive to scaling, spacing and skewness parameters.

The capturing device plays an important role for effective extraction and recognition of Braille characters. A portable camera [17] based interactive Braille recognition was recommended for real time scenarios. The scanner is able to read the hand gesture and identify the required action based on action mapping. The interactive system is able to recognize the sign, dot pattern and the gesture for visually impaired people. A portable translation device was invented by Murray and Pasquale [18]. The device is able to recognize the dot area, compile it and translate it to the corresponding text. A pin-matrix device and graphical screen reader based HyperBraille [19] system was designed for Braille character recognition. The semi-graphic view was generated through widget to enable the interaction with blind users.

Braille is accepted by all languages, including German, Arabic, English, Italian etc., with specific dot pattern map to the language constructs. A work to generate the Chinese and Arabic [5] text from Braille by covering the neural and interlunation. Author applied the mathematical morphology to generate the structural elements and to extract the dot-cells. The structural information was processed one-to-one for recognition of text accurately. Another work on Braille to Urdu [7] transition using different image processing techniques was obtained. Author applied the grid pattern based mapping with threshold criteria specification to link the pattern vectors to corresponding Urdu alphabets. The conversion of Braille to Chinese [20] characters was provided segmented feature constraints. A chain of convolution operators, Gaussian method and boundary extractor were applied to enhance the pattern features and to extract the pattern accurately.

The basic formulation of each work stage of OBR is similar to the character recognition system. To understand the OBR, some study work on characterization of OCR and relative approaches is also done. A study work on character recognition system was provided by Govindan and Shivaprasad [21]. Author identifies the capabilities and gap of various character recognition methods. A recognition system on degraded characters using SIFT [22] feature processing was provided. The neighbour map was applied on segmented characters to recognize the relative class. Juneja and Gill [23] has presented an Art network based noise robust method for improving the digital character recognition. A work on Braille block [24] recognition was provided using PCA approach. Author has classified the blocks in four classes called vertical, right-slanted, left-slanted and dotted stop blocks. The pattern map was considered under camera height, angle and illumination issues. A transition method of scanned Braille book [25] to electronic Braille book was provided. The transition is performed for both the text and the graphics. Bebartta and Mohanty [26] has defined a method to recognize the Roman and Odia text based printed document recognition. The linear and circular content map was applied to identify the individual script from the bilingual documents.

Some other applications that increases the human–computer interactions include the hand sign, gesture, face expression or gaze recognition. A descriptive study on scope and directions of sign language [27] was provided to improve interaction capability of hearing and speech impaired people. Various methods for feature generation, template mapping and sign symbol recognition was provided. A color distribution and visual feature analysis based method for image saliency [28] detection was provided. The content and local feature based distance measures were applied for detection of objects. Kumar and Priyanka [29] has proposed the fingerprint recognition on corrupted and degraded images. The effect of environment, sink, sensor and other impression specific features were considered by the author. The image enhancement based mapping process was defined for effective fingerprint recognition.

3 Alignment and disruption robust mapper

Braille recognition system is required to read the documents, provided by visually challenged people, without the help of a Braille expert. The Braille script can be captured through specialized scanner or through portable camera devices. The alignment of the document placement or camera position results the misalignment of complete document. This misalignment can be identified as the skewed or rotated Braille characters. The dirt, dust or lighting conditions can also affects the capturing image quality in the form of noise. In this paper, a noise and alignment problem rectified peered solution is provided to recognize the Braille character correctly. In this paper, mathematical filters are applied in composite and sequential form to resolve the impact of image disturbance. Figure 2 shows the proposed Braille recognition model with specification of each integrated work stages. In this figure, the process stage is defined in gray box and the white box is representing the corresponding filter applied to implement that particular stage.

Fig. 2
figure 2

Alignment and disruption robust Braille recognition model

Full size image

The document or the character image captured through scanner device or through portable camera device is taken as the input to the system. To enhance the content quality of dotted pattern, the SD adaptive filter is applied to reduce the noise from rough captured image. To apply the grid mapper, the input patterns must be defined in structured and symmetric form. The extracted images cannot ensure the proper aligned capturing. The PCT (polar cosine transformation) [30] and geometric map through horizontal and vertical axes. To highlight the content, the emboss filter is applied using mathematical operators. The aligned, clear and highlighted image processed using binarization to separate the content and the background. The paper specific disturbance in the capturing are removed in this stage. Now the dot-pattern contents are present in black and the white background represents the paper. The grid mapper is applied on this extracted pattern image to perform the content verification and based on this verification; the binary code is generated for each character image. This binary coded array is compared with each Braille pattern code and corresponding English character. In this section, each of the intermediate stage is also explained.

The scanner or camera quality can disturb the images in the form of inclusive noise. A filtration window tracked and standard deviation adaptive evaluation method is defined to evaluate the degree of impurity. The smoothing vector is also defined to remove the noise and to improve the quality of particular segment. The difference value evaluation based on standard deviation and relative threshold specification corresponding to the center pixel is done to smooth the pixels in filtration window. The evaluation specific smoothening process is shown in Algorithm 1

figure a

Algorithm 1 processed the raw Braille image and applied the block adaptive evaluation method based on the center difference and standard deviation vector to set the rule for smoothing the block pixels. The results on noise adjustment are shown in Fig. 3. Figure 3a is showing the raw scanned image and Fig. 3b is showing the enhanced image.

Fig. 3
figure 3

Outcome of SD adaptive filtration method

Full size image

The scanned Braille document also suffers from the problem of misalignment. To locate the problem of misalignment, a geometric and content specific map is required respective to horizontal and vertical axis. To perform the geometric map, the peak points of the contents on the document are identified. At these peak points, the straight horizontal and vertical lines are placed for geometric evaluation. The angular evaluation is done based on mathematical mapping. Let (x1, y1) is the initial point and (x2, y2) is the peak point of horizontal line. The identify content peak points are represented by (cx1, cy1) and (cx2, cy2). From these coordinate points, m1 is the slope of line of geometric straight line and m2 is the slope of content touched line. The angular evaluation based on the slope vectors is shown in Eq. (1)

$$\tan \uptheta = \frac{{m_{1} - m_{2} }}{{1 + m_{1} *m_{2} }}.$$
(1)

If θ is zero, the Braille document is already aligned. If θ ≠ 0, then PCT (polar cosine transformation) is applied to align the document. Figure 4 is showing the misalignment problem on partial document image and the aligned document obtained after geometric map and PCT.

Fig. 4
figure 4

Geometric map based evaluation and PCT based alignment

Full size image

PCT is the harmonic transformation that rotate the image using cos function. The rotational vector is evaluated based on the radius and base coordinate specification. PCT evaluation is shown in Eq. (2)

$$R_{n}^{C} (r) = \cos (\pi nr^{2} )$$
(2)

where, r is radius, n is number of coordinate points.

Based on this, the aligned points of the image are shown in Eqs. (3) and (4)

$${\text{Real(i}},{\text{j)}} = {\text{BImg(i}},{\text{j)}}.*\cos (\pi *i*(r^{2} )).\cos (j.*\theta )$$
(3)
$${\text{Img(i}},{\text{j)}} = {\text{BImg(i}},{\text{j)}}.*\cos (i*\pi *(r^{2} )).*\sin (j.*\theta ).$$
(4)

Once these real and imaginary values are obtained for the image, the result value is generated. The intensity evaluation is obtained for the image relative to real and imaginary value shown in Eqs. (5) and (6)

$${\text{IBImg}}\_{\text{real}} = \left( {\frac{1}{\pi }} \right).*Real$$
(5)
$${\text{IBImg}}\_{\text{img}} = \left( {\frac{1}{\pi }} \right).*Img.$$
(6)

Based on this, the pixel rotated Braille document is obtained. This document is completely aligned. To extract the dot pattern from Braille document the emboss filter is executed. This filter highlights the core area using 3D shadow effect. In this method, the bump map is applied by subtracting the side area and to explore the key area. To apply this filter, the convolution matrix map is applied over the image. In this filter, the specific matrix of 3 × 3 size is tracked over the image as a window. The sum of products of window image pixels and the emboss matrix is substituted to each pixel of the window. The correlation adaptive matrix evaluates the symmetric measure for each pixel in the convolution window. Figure 5b is showing the embossed feature Braille image.

Fig. 5
figure 5

Binary coded transition of Braille document

Full size image

The emboss filter evaluate the gradient vector for each pixel respective to the light source. The positive projection is applied onto the vector pointing towards the light. The bump map is generated by applying the positional offset. The water level specific alpha channel is applied to highlight the emboss effect. After generating the embossed feature image, grid mapper is applied on each separated Braille character. The emboss filter separated the each of the dot pattern separately and quality it to recognize the Braille character accurately. For this recognition, the grid mapper is applied on each separated character. The separated character and relative grid mapper is shown in Fig. 5c, d. To perform the grid mapper, the positional and geometric estimation is applied over the character space. According to this evaluation, the grid structure is applied on the Braille. On each character, the grid cell is evaluated for the information containment. If dot exist, it is considered as binary 1 and if no data exist then it is marked as 0. In this way, each character with six dots is represented by six binary digits. This binary generated code is compared with available binary training set which represents each Braille character distinctively. Binary code of the test Braille character is mapped with Binary set code to recognize the equivalent textual character.

4 Analysis

In the previous section, the proposed robust model is suggested to enhance the Braille recognition under worst constraints. The proposed model is capable to handle the real time issues including the noise, file format, resolution and misalignment problem. The experimental evaluation is applied on three different sample sets. The Braille characters and document images are collected from random web sources. The features of these samplesets and corresponding comparative evaluation are also shown in Table 1. The impurities over these samplesets either exist initially or added explicitly to them.

Table 1 Comparative evaluation of proposed model
Full size table

Table 1, shows that these three sample sets able to defend the robust evaluation of the proposed framework. The samplesets are defined with different file formats, resolution, type and disruption inclusion. The comparative evaluation is taken against PCA and LDA methods. The comparative evaluation of accuracy for these sample sets is shown in Fig. 6.

Fig. 6
figure 6

Comparative accuracy evaluation on different samplesets

Full size image

To verify the significance of proposed model, the comparative evaluation is performed against PCA and LDA methods. The analysis is performed against available three sample sets of Braille documents and characters. Figure 6 depict that the proposed framework improved the accuracy over the existing evaluated methods.

5 Conclusion

Braille is the communication language adopted by visually impaired people for written conversation. To connect the Braille with normal communication, the transition between Braille and language text is required. In this paper, a disruption and alignment robust optical Braille recognition system is presented. The presented model, applied the mathematical, adaptive and geometric operators to extract and improve the Braille character features. PCT adaptive geometric filter is applied to correct the misalignment problem. Later, the emboss filter is applied using convolution filter to extract the dotted pattern clearly over the document. Finally the grid mapper is applied based on positional and structural measure to generate the binary code. This binary code is compared over the cookbook to recognize the character accurately. The experimentation is applied on three heterogeneous sample sets of Braille characters and documents. The comparative evaluation shows that the proposed model improved the accuracy extensively over PCA and LDA methods.

In this paper, the character mapping is done using grid adaptive content map. In future, more intelligent supervised learning such as neural network or probabilistic Bayesian networks can be applied to improve recognition rate. In future, the proposed recognition method can be applied on larger Braille character dataset. The work can be extended to perform character recognition or sign recognition.