A Sequence of Tools for Multimodal Analysis of Verbal and Nonverbal Behaviors Related to Interactional Rapport

We describe a method for annotation and analysis of multimodal video data using a sequence of specialized tools that we have made interoperable. We present initial, test-sample results derived with this method. This report details the building blocks of cross-language, multi-modal analyses planned for a large corpus of audio-videotaped, dyadic, conversation data comprising elicitations from Gulf-region Arabic speakers, Mexican Spanish speakers, and American English speakers. We discuss how our approach meets the challenge of readying audio, video, and transcription text data from these three diverse languages for annotation and comparative analysis of multimodal language behaviors related to maintenance of interactional rapport.