Statistics from the National Institute of Mental Health has shown that public speaking is the number one fear people have. ROC Speak is a platform that allows people to practice their public speaking online and get both objective and subjective feedback by leveraging audiovisual signals, machine learning, and the crowd.
Here's the system overview:
When a user records a practice video, the video is sent to the server for audio and video analysis to extract nonverbal behaviors such as smile intensity, movement, pitch & volume modulation, and word intonation. The system then displays feedback on those aspects augmented with the most helpful subjective feedback crowdsourced from Mechanical Turkers. Users can also choose to get subjective feedback by giving a link to their peers. The intuition behind our design is that although machine is good at identifying subtle nonverbal behavior that humans may not be able to easily detect, it lags behind in contextual understanding. We combined both subjective and objective feedback in order to provide users with better understanding of their speaking performance.
The system is in active development.
Michelle Fung, Yina Jin, Ru Zhao, M. Ehsan Hoque, ROC Speak: Semi-Automated Personalized Feedback on Nonverbal Behavior from Recorded Videos, Proceedings of the 17th International Conference on Ubiquitous Computing (UbiComp), September 2015.