Credit: MixedEmotions project A European team of researchers, including Universidad Politécnica de Madrid, has developed an open-source toolbox to assess emotions in texts, audios and videos.
MixedEmotions Toolkit is a set of open-source tools that analyzes emotions. This toolbox is the result of the research project MixedEmotions. The aim of this project is the automatic recognition of emotions through text, audio and video processing.
Our actions are affected by both our mood and the way we perceive others. There is a growing demand for automatic analysis of emotions in different fields. The possible applications are wide, including call centers, smart environments, brand reputation analysis and assistive technology.
First, analysis tools can be highly complex. Emotion detection may require age, gender or facial recognition. Second, the analysis also requires both prior knowledge and linguistic resources that are not always public, and third, these tools are usually designed in one language, generally in English.
Adapting these tool to other languages is an arduous task that requires specific resources on each language. The combination of these issues reduces the range of freely available tools.
The aim of MixedEmotions was to increase the number of these tools. The developed tools are adapted to various European languages in order to recognize the multicultural and multilingual outlook of the current technology.
In order to show the utility of these tools, the researchers tested them in the context of three concrete use cases: a smart TV application that provides emotion-driven recommendations, a call center monitoring system that assesses the mood and the reaction of clients in each call, and an online brand reputation system for companies that study opinions and responses of customers.
The Intelligent System Group (GSI) from UPM contributed to the project. First, they led the linked data modeling for services and semantic vocabulary. As a result, all the project tools use this type of vocabulary based on the linked data principles.
This eases the interoperability, since analyses in various modalities are conducted using fusion techniques. Fernando Sánchez, a GSI researcher says, "Given the relevance of this issue, we developed a community group on the World Wide Web consortium (W3C), an international community focused on the development of standards to ensure the long term Web growth, to discuss this modeling semantic and transfer the results."
Second, the GSI group developed Senpy, a software package for developing and publishing emotion analysis services and tools, mainly focused on text processing. Finally, the group improved the emotion analysis through social context, that is, additional information about the user, the content and different relationships in the social networks.
Explore further: New search tools open up access to medical information
More information: Paul Buitelaar et al. MixedEmotions: An Open-Source Toolbox for Multimodal Emotion Analysis, IEEE Transactions on Multimedia (2018). DOI: 10.1109/TMM.2018.2798287
