Co-speech gesture annotation from video
Goes from a plain video of someone talking to gesture annotations that open in ELAN. A free body-tracking model finds the speaker’s hands in every frame, and from how the hands move the notebooks find where each gesture starts and ends and the key moments inside it.
It runs on an ordinary laptop, the videos never leave the computer, and every step is explained in plain language, so it can be used with no programming experience. I built it so labs without motion capture can still study gesture.
This line of work began as the MultiGest tool, presented at ICPhS 2023, CorpusPhon 2024 and the International Multimodal Communication Symposium 2024. The current public version is packaged as a hands-on workshop.