Introduction

Cognitive computing promulgates the development of computer systems which can model human behaviours like perception, attention and thoughts. Cognitive computing systems can collect and process individual, social and healthcare data for better disease diagnosis and patient engagement. It offers a combination of multidisciplinary technologies like artificial intelligence, deep learning, machine learning, big data analytics and natural language processing systems to identify types and symptoms of a disease. A mental disorder is a medical condition that influences the normal behaviour of a person. A mental disorder may impair the physical, emotional and social well-being of a person. Impairment to memory, attention and cognitive control is fundamental to any mental disorder. Furthermore, it may result in aberrations in mood, thinking, work, body functions, emotions, social engagements and general behaviour either once, recurrently or even permanently. Researchers have reported more than four hundred and fifty types of mental disorders with discrete symptoms. Few of the prominent mental disorders are depressive disorder, mood disorder, personality disorder, addictive disorder, sleep disorder, post-traumatic disorders, dementia, bipolar disorder and schizophrenia along with stress and anxiety disorders [1].

More than 10.7% of the world’s population is suffering from aforementioned mental disorders, most prominent being depression and anxiety which add more than 548 million to the global burden of the diseases [2, 3]. The repercussions of mental disorders are diverse, including but not limited to low employee productivity, high suicide rates, early mortality, dangerous driving, early dropping out of education, poverty, disability, physical pain and being prone to other diseases like infections and cardiovascular diseases [4,5,6,7,8,9]. The prevalence of mental disorders also leads to loss of economic growth and places a burden on the economic well-being of the patient as well as the caregivers [10]. This makes early detection of mental disorders imperative not just for health but also for development. This has led to the inclusion of mental health as one of the targets in Goal 3 of United Nations Sustainable Development Goals [11]. Subsequently, research community has laid emphasis on early detection of mental disorders in order to deploy suitable therapeutic measures.

Clinical assessment methods for detecting mental disorders rely on patients’ self-reporting and the expertise of the examiner. These methods lack precision due to social stigma, lack of knowledge, subjective bias of the examiner, time-intensive nature of the examination and lack of consistency [12]. To eliminate subjective bias and to reduce examination time, different rating scales for measuring mental disorders have been developed. These scales are used to measure the severity of the mental disorder as a function of perceived symptoms as inferred from the assessee [13,14,15]. However, the rating scales have some inherent limitations; they are inconsistent in handling the heterogeneous nature of mental disorders and results obtained from different scales are different. The rating scale cannot encompass all the symptoms associated with a particular disorder and they did not take into consideration the effects of examination environment, emotions and demographics in the rating scales [16].

This has led to exploration of novel detection techniques which could handle the heterogeneous nature of the mental disorders, eliminate subjective bias and include the effects of emotions along with demographics like age and gender on detection mechanism. As a result, mental disorder detection techniques based on machine learning and biomarkers have emerged. It is because human cognition is closely associated with the biological processes of an individual. The electrical activity of the nervous system controls almost every aspect of the human body including heart signals, sweat glands and even human cognition. Thus, any changes in human mental states like mood changes, emotional changes or mental disorders may greatly affect the activity of the brain, heart, skin, facial expressions, speech and even other biological processes like respiration and temperature. Various research studies have shown that the activity and power of different EEG bands, inter-hemispheric symmetry and statistical features like peak, variance, entropy and energy are contrasting between healthy individuals and people suffering from mental disorders. For example, depressed persons have increased beta values in their brain waves as compared to healthy persons [17,18,19,20,21,22]. Similarly, there is a difference in statistical features of heart rate variability and other ECG characteristics like peak and median frequency of healthy and non-healthy individuals [23,24,25,26,27,28]. The skin conductance response and statistical characteristics of electrodermal activity are also different for healthy and non-healthy individuals [29,30,31]. Electromyography (EMG) is a procedure to measure muscle movements and motor neuron activity. It has been observed that mental stress may result in a different EMG activity in healthy and non-healthy individuals [32,33,34]. Recent studies also investigated correlation between mental disorders and eye movements [35,36,37,38]. This has led to exploration of objective methodologies which aim to automatically decipher human cognition by machine learning with the help of biological signals of an individual. Table 1 highlights different cognitive tasks that are related to different biological processes.

Table 1 Relation of human cognitive tasks with biological processes
Full size table

Modalities like EEG, GSR, EMG, ECG, facial expressions, eye movements, online handwritten signals and speech can facilitate extraction of prominent features to be used as biomarkers. Such biomarkers could further be used for automatic detection of mental disorders using machine learning algorithms. In reference to the previously stated arguments, a systematic literature review is required to promulgate state of art research contributions in this domain. Furthermore, it may help contemporary researchers to investigate numerous multidisciplinary open research problems from the amalgamation of psychology, sociology, machine learning, computer science and behavioural sciences. The following research questions were formulated to attain the goal of this study:

  1. 1.

    Which physiological and behavioural modalities can facilitate the detection of mental disorders using machine learning?

  2. 2.

    Which prominent features of these modalities can be used in detection of mental disorders using machine learning?

  3. 3.

    Which are the most suited machine learning algorithms to exploit these modalities for detecting mental disorders?

  4. 4.

    What are the prominent challenges in detection of mental disorders using biomarker modalities?

The rest of this manuscript is organised as follows: the “Survey Methodology” section elaborates the adopted methodology and the novelty of this survey in comparison to previously published literature, the “Different Approaches of Mental Disorder Detection” section provides a discussion on different modalities used to detect mental disorders, the “Discussion” section offers a summarisation of most prominent features and most suited machine learning algorithms to detect mental disorders using these modalities and the “Research Challenges” section concludes contemporary research gaps, key findings and research challenges.

Survey Methodology

This section elaborates the adopted methodology for conducting the systematic literature survey. For this survey, the relevant literature from 2010 to 2021 has been studied based on the previously stated research questions. Recommendations of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) model were adopted for identification, screening, validating and inclusion of research literature as shown in Fig. 1 [55]. The articles were searched using keywords like detection + mental disorders, analysis + mental disorders, prediction + mental disorders, identification + mental disorders, modalities + mental disorders, machine learning + mental disorders, physiological signals + mental disorder and verbal and non-verbal behaviour, from databases like ScienceDirect, IEEEXplore, SpringerLink, ACM Digital Library, Taylor & Francis Online and Wiley Online Library. Pre-screening exclusion was done based on duplication and article language. Articles published in English language were only considered for screening. Screening and exclusion were based on the relevance to the survey as inferred from the title, abstract and conclusion of the article. Post-screening exclusion of articles was based on user-defined inclusion criteria like articles should have a minimum of five citations per year and should be published in 2010 or later.

Fig. 1
figure 1

Survey methodology

Full size image

Mendeley was used for aggregation, filtering and removal of articles to build a consolidated library of relevant research articles.

Numerous surveys on detection of mental disorders using physiological and behavioural patterns have been published in recent past. During literature evaluation, it was observed that existing literature is primarily investigating one mental disorder based on a single modality. To the best of our knowledge, multiple physiological signals along with behavioural patterns for detection of mental disorders using machine learning techniques have not been explored to the desired extent. Table 2 offers a comparison of existing literature with this survey:

Table 2 Comparison of current survey with previous surveys
Full size table

Different Approaches of Mental Disorder Detection

Physiological signals and human behaviour provide various biomarkers with respect to different mental states and disorders. These signals or behavioural aspects are called modalities. Data from these modalities can be used in the detection of mental disorders. Features from multiple modalities in collaboration with machine learning algorithms can be used to determine the prevalence, type and severity of mental disorders. A modality can be used either as unimodal or multimodal approach. Types and usage of unimodal and multimodal approaches are detailed as under:

Unimodal Approach

In unimodal approach, a single modality is used as a standalone differentiator for classification. Various modalities have been used in unimodal approach for detection and classification of mental disorders, mood changes and emotional states of an individual. The following subsections provide an overview of contemporary works which have utilised a single modality for detection or prediction of mental disorder type or severity.

Electroencephalogram (EEG)

EEG captures the brain waves corresponding to different activities happening within the central nervous system. It is acquired through electrodes placed on the scalp. Different features of EEG in time domain and frequency domain can be extracted to analyse the type of signal. Most prominent features include statistical features in time and frequency domain such as mean, median, variance, standard deviation, skewness and kurtosis along with time-domain features like peak amplitude and energy. Frequency-domain features include different frequency bands in EEG signals like delta, theta, alpha, beta and gamma. The features of EEG signals have been classified using machine learning methods to detect different mental disorders. Table 3 provides an overview of research studies which have explored the use of only EEG signals in detection of mental disorders.

Table 3 Contemporary studies which have employed EEG
Full size table

Electrodermal Activity (EDA) or Galvanic Skin Response (GSR)

EDA or GSR refers to the electrical response of human skin with respect to variation in sweat secretion. It is expressed in terms of Skin Conductance Response (SCR) and Skin Conductance Level (SCL). The changes in electrodermal activity due to any mental disorder or during a mental stress task are a potential biomarker of mental disorders. Table 4 provides an overview of research studies which have explored the use of EDA or GSR signals for detection of mental disorders.

Table 4 Contemporary studies which have employed EDA/GSR
Full size table

Electrocardiogram (ECG)

Electrocardiogram captures the electrical activity in the heart. It is composed of a P-Wave followed by a QRS Complex which is followed by a T-Wave. The peak of the signal is the R of the QRS complex. The time between two successive R peaks is referred to as RR interval. Since the brain modulates the electrical activity of the human body, hence the ECG of an individual also reflects the changes in mental state. Different features of ECG have been used with machine learning algorithms to discriminate between normal and aberrated ECGs and therefore can be utilised for automatic detection of mental disorders. Prominent features include statistical measures like mean, median and standard deviation as well as other features like heart rate variability (HRV), root mean sum of squares of differences between normal to normal beat intervals (RMSSD), proportion of successive normal to normal beat intervals that differ more than k ms (pNNk), standard deviation of successive differences (SDSD), low-frequency band (LF), high-frequency band (HF) and ratio of LF to HF. Table 5 provides an overview of research studies which have explored the use of ECG signals only in detection of mental disorders.

Table 5 Contemporary studies which have employed ECG
Full size table

Verbal and Non-verbal Behaviour

Verbal and non-verbal behaviour is closely correlated with physical, mental and emotional state of an individual. Any mental impairment or emotional change reflects in the behaviour of an individual. The analysis of changes in facial expressions, eye movements and speech in the presence of unfavourable mental states can provide biomarkers for mental disorder detection. Table 6 provides an overview of research studies which have explored the use of human behaviour in detection of mental disorders.

Table 6 Contemporary studies which have employed human behaviour
Full size table

Online Handwritten Signals

The exploration of novel bias-free methods for detection of mental disorders has led to research in various newer paradigms; with online handwritten signals being a prominent one. The benefits of this approach are that handwriting is a common daily task and does not need any specialised training for participants. As can be seen in Table 7, various studies have explored the use of online handwritten signals related to time, space and pressure in the field of sentiment analysis, emotion recognition and by extension mental disorder detection.

Table 7 Contemporary studies which have employed online handwritten signals
Full size table

Multimodal Approach

In multimodal approaches, two or more modalities are used for detecting mental disorders. There are two prominent approaches for fusion of multiple modalities—feature level fusion and decision level fusion. In feature level fusion, the features from multiple modalities are collected to form a single feature-set. A single classifier is then run on this feature-set to make decisions. On the other hand, in decision level fusion, multiple classifiers are run independently on feature-sets of individual modalities. The results of these multiple classifiers are then studied to form a single decision. Table 8 provides an overview of research studies which have adopted a multimodal approach and used more than one biological signal in detection of mental disorders.

Table 8 Contemporary studies which have adopted a multimodal approach
Full size table

Discussion

This section discusses the various findings obtained after extensive analysis of selected literature. Various findings obtained from the survey are discussed in the following subsections. As stated in the “Different Approaches of Mental Disorder Detection” section, physiological signals like EEG, GSR, ECG and EMG and behavioural patterns like handwriting, speech, eye movements and facial expressions have been extensively used for detection of mental disorders like depression, schizophrenia, bipolar disorder and mental stress. Table 9 provides a summary of different studies that have been discussed. The table shows the correlation between mental disorder studied, modalities used for detection and associated machine learning algorithms.

Table 9 Summary of articles surveyed
Full size table

Different Biomarkers Used in Mental Disorder Detection

The different modalities used in mental disorder detection are EEG, ECG, EDA/GSR, EMG, eye movements, speech, online handwritten signals and facial expressions. Figure 2 illustrates the use of different modalities in detecting mental disorders using machine learning. It can be inferred that in the selected corpus of related research studies, physiological signals like EEG, ECG and GSR have been most popular amongst researchers while behaviour-based modalities such as speech, facial expressions and eye movements have been explored less often. It has been observed that almost 31% of the publications opted for EEG as a modality while ECG and GSR were preferred by 25% and 19% of researchers respectively. On the contrary, facial expressions and speech have been recommended by just 8% of the total research publications. One reason for this can be that behaviour of a person is voluntary and can be controlled by the person, thereby not revealing the true results [106,107,108]. Second reason can be that the experimental environment can change the normal behaviour of the person [109]. Another reason can be that behavioural changes vary to a great extent between individuals and can be subjective [110]. Still, research studies indicate that behavioural modalities provide important insights into mental disorder detection.

Fig. 2
figure 2

Percentage of modalities used in research publications from corpus collected

Full size image

Features Used in Mental Disorder Detection

Table 10 provides an overview of the most prominent features of physiological and behavioural signals used along with detailing the type of mental disorder they have been used for.

Table 10 Different features used in mental disorder detection
Full size table

Prominent Machine Learning Algorithms Employed in Mental Disorder Detection

The research studies included in the selected corpus have used different algorithms with various biomarkers for automatic classification and detection of mental disorders. However, the prominent ones are Support Vector Machines, k-nearest neighbour, Logistic Regression, Decision Trees, Naïve Bayes, Linear Discriminant Analysis, Random Forest, Artificial Neural Networks and CNN. As seen in Fig. 3, amongst the selected corpus of articles, SVM has been used about 24% of the times with k-nearest neighbour, Logistic Regression, Decision Trees and Naïve Bayes algorithms with around 8–10% usage. This can be due to the fact that SVM provides a fine balance between complexity and performance and therefore has been used as a baseline in most of the research studies. Also, Artificial Neural Network–based algorithms have been collectively used by about only 17% of the studies despite their high efficiency. Also, some algorithms like AdaBoost, Bayes Net, Radial Basis Function Network, Quadratic Discriminant Analysis, Partial Least Square, Gradient Boost and Gaussian Mixture Model have been used very sparsely as seen in Fig. 3 because of their high complexity when dealing with multi-feature data.

Fig. 3
figure 3

Percentage of algorithms from corpus which was used for detection of mental disorders

Full size image

Research Challenges

The traditional methods of mental disorder detection like questionnaires rely on information provided by the individual. Besides being static in nature, these methods also cannot effectively verify the authenticity of the collected data. Hence, robust methods are required to identify or correct fabricated or tampered data. The research community has recommended the use of biomarkers related to various physiological and behavioural modalities like EEG, GSR, EMG, ECG, facial expressions, eye movements and speech for detection of mental disorders. While these approaches are objective in nature, behaviour can be faked or controlled by an individual. An individual who is conscious can easily manipulate his or her facial expressions, speech and eye movements. This mandates the requirement of methods, which can segregate genuine and controlled behaviour. Furthermore, emotional state of an individual plays a major role in the collection of data that has originated from human response, be it physiological or behavioural in nature [111]. Therefore, it becomes important to incorporate the effect of emotions on signals and further onto the analysis. Investigating this entire domain from the purview of decision intelligence may open numerous different research tracks related to mental health and diagnosis. Quantification of emotive feedback of respondents is also a major challenge for limited experimentation to diagnose mental disorders through facial expressions. Also, physiological responses vary between individuals due to human morphology and demographic factors like gender, race and age [112]. This variability needs to be managed to produce efficient and usable results. Furthermore, the experimental setups mentioned in the literature surveyed are inconsistent. For instance, different studies have mentioned different upper or lower bandwidths for frequency bands of EEG and ECG signals. Furthermore, the results obtained depend hugely on what features have been selected and which algorithm has been implemented. The selection of features-algorithms pair should be optimum to attain best results. Furthermore, collecting data from individuals suffering from mental disorders presents an ethical challenge [113]. That could be the reason for limited availability of standardised and annotated publicly available datasets pertaining to mental disorders.

Conclusion

Mental disorders are one of the leading contributors to global burden of diseases. They are one of the emerging challenges of the society. They can lead to lack of attention at work, dropping out of education, bad social behaviour and even suicide. So, there is a need for self-diagnostic methods which can automate the detection of mental disorders. The self-diagnosis tools for detection of mental disorders like questionnaires and rating scales are inconsistent and static in nature. They cannot encompass the diversity of mental disorders, inter-individual variability and impact of emotional state of an individual. Furthermore, there are no standard baselines for mental disorder datasets available. Stress of one person can be anxiety for another person and even depression for a third person. Hence, a multi-faceted approach is recommended where one can utilise data from physiological signals, behavioural patterns and even data obtained from various online portals like social media to efficiently and effectively detect the prevalence, type and severity of mental disorders.