Project Description

The project aims to solve the fundamental problem of how emotions form in ontogenesis and how they are recognized and manifested in human behavior. The goal of this cross-cultural study is to develop an interdisciplinary approach to the study, diagnosis, and correction of emotional sphere disorders in children using physiological, psychophysiological, psychological, and artificial intelligence methods. The novelty lies in the development of a complex standardized approach to the analysis and diagnosis of emotional sphere disorders in children, adapted to the language and cultural characteristics of the child’s country of residence, the child’s age, the type of developmental disorders, and psycho-neurological states. The study will focus on the voice, speech features, facial expression, and behavior of children, as well as the peculiarities of children’s perception of emotions. The study will involve 500 children, including 250 Russian and 250 Indian children with typical development and children with developmental disabilities. The standardized approach to the assessment and diagnosis of emotional sphere disorders in children, based on objective qualitative and quantitative data, can be applied in clinic, defectology, and pedagogical practice.

Methodology

The project will use automatic recognition systems to classify emotional states based on children’s voice and speech. The systems will be based on the extraction of acoustic features, statistical models, and deep neural networks. The study will also use a new approach based on Generative Adversarial Networks (GAN) to expand the training data sets and increase the accuracy of face recognition. Cross-cultural and cross-corpus recognition of emotions of typically developing children and children with atypical development is a new scientific direction of research with a wide range of applications from clinical practice to multimedia teaching systems.

Expected Outcomes

The project will obtain new data on the recognition of emotional speech and facial expressions of children. The data on the similarity and difference of the emotional sphere of Russian and Indian children, the peculiarities of perception and expression of different emotional states will be obtained.

The similarities and differences of features of the emotional speech in two languages, common and specific features allowing correctly recognize child’s emotions by facial expression will be revealed. The study will compare the data of perceptual experiments and machine recognition of emotions in speech, facial expressions, and nonverbal behavior of children, taking into account diagnosis, age, gender, and language. The proposed project will be the first to bring together specialists from different fields, providing new data based on complex interdisciplinary knowledge. This will allow us to develop new fundamental approaches to the rehabilitation and education of children with developmental disabilities.