Introduction

The automatic data analysis process, developed for a many years, is now moving away from tasks of simply interpreting and classifying the analysed data and is focusing mainly on reasoning about and understanding the meaning of the analyzed data sets [1, 2, 10, 12, 14].

In terms of activities aimed at understanding the analysed data, a new class of intelligent information systems called cognitive systems has been developed [3, 7]. Systems of this class execute not just a simple data/information analysis, but mainly aim at revealing the semantic information (meaning data) found in this data, understanding it and reasoning based on the semantic information it contains. This process is possible thanks to using grammar formalisms of the linguistic perception and data understanding for automatic reasoning processes [9, 11]. These processes are executed in a way modelled on the human brain and mechanism of description, interpretation, analysis, understanding and reasoning taking place in the human brain. Description, analysis, interpreting and reasoning processes were used by the author to design and describe new classes of cognitive information systems aimed at the semantic analysis of data and reasoning about the meaning of the analysed data sets. The main idea of cognitive information systems has been described in publications [4, 7]. In these papers, the author has been researching the use of cognitive analysis algorithms and methods for the interpretation process, understanding and reasoning processes performed by different kinds of information systems.

During the research conducted on cognitive systems that analyse data/information using cognitive data analysis and interpretation methods, a definition was proposed of a new type of cognitive analysis systems imitating the processes of purely human brain, cognitive perceptive analysis aimed at categorising data according to its meaning. The work on cognitive data analysis has led to proposing new classes of information systems, named cognitive information systems. Within the set of cognitive information systems, classes of systems designed for analysing various types of data were distinguished, described and characterised, and in addition, classes of systems in which in-depth semantic analysis was conducted were selected.

An innovative approach to the interpretation and semantic analysis of data was a class of cognitive systems dedicated to medical image analysis – the UBIAS (Understanding Based Image Analysis Systems). These systems designed for cognitively analysing image–type data, namely images of:

  • foot bones acquired in three different projections,

  • hand bones as well as images of long bone fractures of upper extremities.

The leading topic of this work is to characterize classes of information systems for interpreting and analysing image data as well as reasoning about it using the concept of cognitive data analysis. The concept of cognitive information systems allows us to demonstrate significant goals, properties and, more importantly, innovations of cognitive systems. The ability to conduct a complete analysis of image-type data arose as a result of transferring cognitive and thought processes occurring in natural biological structures like the human brain, to information systems, as a consequence of which information systems can analyse data using the foundations of cognitive analysis.

The description of cognitive processes comes from psychology and philosophy. Their advantage is their indisputable universality and their interdisciplinary nature in the applications and the correct implementations of cognitive processes. The essence and the novelty of this approach to data interpreting and analysis as well as to the process of reasoning based on the analysed data is the structure of cognitive information systems designed to support decision-making processes by using layers of semantic information contained in the data sets they describe. The cognitive analysis of image data and the semantic interpretation of the analysed data ultimately leading to the process of data understanding and cognitive analysis was made possible by the use of formalisms of linguistic perception and semantic reasoning based on linguistic formalisms [3, 9, 13].

Classification of cognitive systems

Cognitive information systems were developed on the foundation of intelligent information systems whose purpose was not just the simple analysis of data consisting in recording, processing and interpreting it, but primarily its deeper analysis by understanding and reasoning about the semantic information in the processed data [5, 6].

Cognitive information systems utilise methods which define structural reasoning techniques for matching defined patterns. A system carrying out a cognitive data analysis analyses data presented in various forms (numerical, signal, biometric, image, etc.).

In image data analysis, the structure of the image being analysed is compared during the analysis process to the structure of the pattern image. This comparison is performed using strings of derivation rules which enable the pattern to be generated unambiguously. These rules, referred to as productions, are defined in the introduced grammar, which in turn defines a certain formal language, called an image language. An image (information) thus recognised is assigned to the class to which the pattern that represents it belongs.

Cognitive information systems have been defined in the as the systems which [8]:

“describe intelligent information systems designed for conducting in-depth data analyses based on the semantic contents of the data. Semantic analyses are conducted with algorithms for describing this data based on processed expert information (for example in the form of knowledge bases) and the processes of machine (computer) perception and understanding of data performed, e.g., using mathematical linguistics.”

It has been proposed that cognitive information systems would be used for analysing medical images because there are many different disease entities occurring in the disease processes of different human organ, because these units are increasingly detectable and identifiable, but those systems could also be used to analyse strategic data (e.g. economic ratios) because there are varied situations of economic entitles that characterised their past, current and future states.

Medical images are among the most varied forms of data and have extremely deep and significant meaning interpretations. Similarity, the economic situation of a given entity described using sets of ratios can reveal various causes of a given situation, the different development of the organisation/company, and various situations of this entity in the future.

With regard to the purposes of the conducted semantic analysis, cognitive systems have been classified into main classes, which include [7]:

  • Understanding Based Decision Support Systems,

  • Understanding Based Image Analysis Systems,

  • Understanding Based Management Support Systems,

  • Understanding Based Person Authentication Systems,

  • Understanding Based Signal Analysis Systems,

  • Understanding Based Automatic Control Systems.

All proposed classes of cognitive information systems, allow to analyze different types of data. But the one point of this analysis is semantic description of data. It’s not important, what kinds of data system analyze, but it’s a very important, if the analyzed data have semantic meaning? Those meaning of data sets, is a clue of cognitive analysis. Presented classification of cognitive systems, are dedicated to selected disciplines, in which is possible to perform semantic analysis of data. Because in those disciplines, the data are very complex and multi-objects.

All the types of cognitive information systems presented before have wide-ranging applications, from economics, sociology and philosophy to technical and defence sciences, medicine or natural science. All cognitive system types use methods of cognitive analysis in their operation to extend the capabilities of classical data analysis technologies in order to reason based on the semantics of the analysed data or information.

Cognitive systems are used to understand and semantically analyse data. Their significant feature is that as a result of the analysis of data conducted by the system, this data can be semantically interpreted. This type of data description and analysis allows semantic information to be extracted, which information makes it possible to reason at the stage of the data analysis conducted.

Semantic analysis and data interpretation in UBIAS cognitive systems

The one class of cognitive information systems are the UBIAS systems. Those systems are dedicated to medical image analysis and semantic interpretation of medical images. The most important element of the presented analysis and reasoning process is that cognitive processes which lead to semantic reasoning occur in both the human cognitive process and the system’s information process that conducts the semantic interpretation and analysis of data. This process is based on the cognitive resonance [4] which occurs during the thought process, and which becomes the starting point for the process of data understanding. This phenomenon consists in extracting semantic information and determining the meaning contained in the analyzed type of data, which enables the reasoning [7].

Semantic analysis is a process consisting in assessing the semantic layers of the analyzed data sets [9], which include the features and parameters characteristic for the analyzed data (Fig. 1).

Fig. 1
figure 1

The semantic analysis process. Figure 1 presents the semantic analysis processes, in which the first step of medical image analysis is the preprocessing of images. The preprocessing consist the acquisition and select medical images for analysis processes. Also, the preprocessing of images include the processes of image transformation like binarisation, segmentation, filtration and detection of the shape and components. These processes allow to extracted the characteristic features and parameters of the analysed medical images. Characteristics of the analysed images make a group and representation of medical images. After this, it’s possible to define a formal grammar by linguistic formalisms. These formalisms are used to semantic analysis, extraction of meaning of the analysed data/images

Full size image

UBIAS systems proposed by the author for the purpose of analysing medical images are very intensively developed, with research conducted on the semantic analysis of varied images of lesions of various human organs and body parts [3, 4–8].

The semantic analysis of medical images has been applied to analysing the following images:

  • X-rays of metatarsal and foot bones in three different projections – dorsoplanar, external lateral or internal lateral,

  • X-rays of the wrist and hand bones,

  • X-rays of long bone fractures of the upper extremities.

UBIAS systems were used to semantically analyse all the above classes of medical images.

UBIAS systems for foot bone analysis

UBIAS systems designed for the semantic analysis of foot bone images have been defined for the tasks of analysing foot bone deformations and pathologies, in particular for analysing:

  • various types of foot bone fractures,

  • degenerations leading to skeleton deformations,

  • bone displacements,

  • the appearance of an additional bone among foot bones,

  • the appearance of haematomas, calcifications and various irregularities in the structure of foot bones.

The analysis of foot bone images was proposed for three projections in which medical images of metatarsal bones are acquired:

  • the dorsoplanar projection of metatarsal bones,

  • the external lateral projection of metatarsal bones,

  • the internal lateral projection of metatarsal bones.

The semantic analysis of metatarsal bone images has been extended to include the analysis of foot bone images by including toe bones in the semantic description of foot bone images. An analysis of foot bone images was proposed with reference to possible lesions and deformations, which include:

  • toe and foot bone fracture,

  • degenerative lesions,

  • deformations of foot bones,

  • arthritis,

  • tuberculosis of bones.

The types of deformations, are very typical for foot bone pathologies. For performed analysis, it was especially selected foot bone fractures, foot deformations, and pathologies as arthritis or tuberculosis, as a typical foot bone lesions.

A detailed description of the solutions proposed for the semantic analysis of foot bone images, including the introduction of definitions of grammars and formal descriptions that are to support the semantic analysis, interpretation and description of analysed data in the image form, has been dedicated to cognitive analysis of many kinds of food deformations [5].

Cognitive reasoning methods were used to detect all the above groups of pathologies of foot bones. The results achieved confirm the suitability of the cognitive approach, although the unanimous identification of all disease entities turned out to be extremely difficult due to slight changes in the input medical images which were used to take the decision to classify the case under consideration to a specific disease entity.

The research conducted by using analyses of images depicting lesions of foot bones has demonstrated that the cognitive analysis of image data can significantly enhance the capabilities of modern information systems and systems supporting medical diagnostics. In particular, this research has proven that a correctly built formal grammar makes it possible to precisely establish and then run an analysis and describe selected diagnostic cases, isolating significant semantic information from them about the nature of the processes running and lesions occurring in foot bone structures.

UBIAS systems for hand bone analysis

UBIAS systems designed for the semantic analysis of hand bone images have been defined for the tasks of analysing hand bone deformations and pathologies, in particular for analysing:

  • finger fractures,

  • hand bone fractures,

  • degenerative changes (bone fusions or bone atrophy),

  • hand deformations,

  • bone displacements,

  • arthritis,

  • tuberculosis of bones.

A detailed description of the solutions proposed for the semantic analysis of wrist and hand bone images, including the introduction of definitions of grammars and formal descriptions that are to support the semantic analysis, interpretation and description of analysed image data, has been dedicated to cognitive analysis of many kinds of hand deformations [5, 7].

Cognitive reasoning methods were used to detect all the above groups of pathological conditions of hand and wrist bones. The results produced by the semantic analysis of images of hand and wrist bones confirm the suitability of the cognitive approach in a situation in which the unanimous identification of all disease entities was difficult due to subtle changes in the input data which was used to take decisions to classify the considered case of a disease entity.

UBIAS systems for long bone fracture analysis

Cognitive information systems designed for analysing images showing fractures of long bones of upper extremities have been proposed for various images of long bone fractures, which include:

  • oblique fractures,

  • transverse fractures,

  • spiral fractures,

  • longitudinal fractures,

  • comminuted fractures,

  • fractures after which a part of the bone has not returned to the correct position relative to the other part – a displaced fracture.

A detailed description of the proposed solutions for the semantic analysis of images depicting long bone fractures, including the introduction of definitions of grammars and formal descriptions that are to support the cognitive data analysis, the interpretation and the semantic description of the analysed image data.

During this research it was shown that a correctly built formal grammar makes it possible to run an analysis and describe selected medical images precisely, isolating significant semantic information from them about the nature of the processes progressing and lesions occurring.

Linguistic formalisms have been used in UBIAS systems for the semantic analysis of various image data. An assessment of the utility of semantic analysis methods for image data has shown that the solutions introduced for the purpose of cognitively analysing and interpreting data are correct.

Semantic analysis in E-UBIAS cognitive systems

The systems for the semantic analysis of different medical images have been extended to include the processes of the personal description and biometric analysis of data. Enhancing systems for the semantic analysis of data by adding reasoning and interpretation processes based on biometric (personal) features made it possible to semantically analyse extended data sets.

This innovative approach to the personal and semantic analysis of data was implemented by combining the cognitive analysis of hand bones with the biometric analysis of hand features. This analysis was carried out by defining individual sets of hand geometry features and biometric traits.

The solutions proposed for the semantic analysis of wrist and hand bone images and the biometric analysis of the hand, including the introduction of definitions of sets which can be used to carry out any extended analysis of personal features, has been dedicated to semantic analysis of image data [2, 5, 6, 7].

In addition, a class of systems was proposed for a personal analysis carried out by defining individual sets of minutiae in combination with the semantic analysis of the structure of hand bones. Defining sets of characteristic minutiae in the personal analysis process provided the opportunity to extend systems for the semantic analysis of medical images by adding personal features. The detailed description of the proposed solutions enhancing processes of image data analysis has been presented in [8].

In addition, it was proposed to enhance the process of image data analysis with a stage at which the system learns new solutions. The novel approach to personal and semantic data analysis was implemented by adding stages of learning new solutions not yet defined in the system’s knowledge base to the semantic analysis of foot bone images. This analysis was carried out after adding new, formerly unknown elements to knowledge bases [10].

This stage of the biometric analysis carried out by cognitive systems was used in the process of the semantic analysis of image data, after the process of personal verification and identification as well as personalisation was added to this stage. The introduction of the biometric analysis offers great opportunities for extending cognitive systems because biometric, personal information which increases the accuracy of assigning a person to the set of analysed data can be added to all processes of medical data analysis.

The main features – important points – of this solution, is a universality. This analysis is possible to perform not only simple processing, like description, statistical or quality analysis, etc., but it’s also possible to evaluate the semantic description of analysed data sets. This analysis allows to extract the most important features from analysed data sets. Those features allow to describe data sets, all external situation, and all determinants created our data. So, it’s a very specific analysis, which allows to answer the question: what is the meaning of our data? This meaning is also included in biometric features. Then, also in E-UBIAS cognitive systems, is possible to used proposed in this publication solutions.

Conclusions

The presented approach to the subject of broadly-understood semantic data analysis originated from a cognitive informatics. In this paper it was describe main aspects of cognitive informatics and methods of semantic analysis. Cognitive informatics as a newly-defined scientific field on purpose, because even though this branch of informatics has been developed by many scientists and researchers in the past years, its formal definition appeared only relatively recently as an attempt to formalise the tasks that this field of science deals with. This is because informatics has always been treated as a branch of sciences about learning (particularly in the philosophical approach), and therefore of cognitive science. It was only recently that the role and significance that cognitive informatics plays in cognitive processes was recognised.

The main role in cognitive data analysis processes play the linguistics formalisms. Advanced graph formalisms were used in UBIAS systems analysing foot bone and hand bone images, while sequential formalisms were applied to analyse long bone fractures. The used of different type of formal grammar it’s depends on the type of data, characteristics features of data sets, etc.

The cognitive informatics is dedicated to semantic analysis of different kinds of data.

Future directions of developments of cognitive information systems will be focus on the semantic analysis of the following areas:

  • economic data,

  • data management,

  • strategic data analysis,

  • personal analysis and identification,

  • biometric marking and personal cryptography,

  • vision and decision systems.